java – Deflater. Deflate and small output buffers

I saw a strange situation, using java 8u45's small output buffer and Java util. Deflater. When the deflate (byte [] B, int off, int len, int flush) method is used with a small output buffer

(I'm developing some low-level network code related to the upcoming extension of websocket - extension, so small buffers are realistic for me)

Example code:

package deflate;

import java.nio.charset.StandardCharsets;
import java.util.zip.Deflater;

public class DeflaterSmallBufferBug
{
    public static void main(String[] args)
    {
        boolean Nowrap = true;
        Deflater deflater = new Deflater(Deflater.DEFAULT_COMPRESSION,Nowrap);

        byte[] input = "Hello".getBytes(StandardCharsets.UTF_8);

        System.out.printf("input is %,d bytes - %s%n",input.length,getHex(input,input.length));

        deflater.setInput(input);

        byte[] output = new byte[input.length];

        // break out of infinite loop seen with bug
        int maxloops = 10;

        // Compress the data
        while (maxloops-- > 0)
        {
            int compressed = deflater.deflate(output,output.length,Deflater.SYNC_FLUSH);
            System.out.printf("compressed %,compressed,getHex(output,compressed));

            if (compressed < output.length)
            {
                System.out.printf("Compress success");
                return;
            }
        }

        System.out.printf("Exited compress (maxloops left %d)%n",maxloops);
    }

    private static String getHex(byte[] buf,int offset,int len)
    {
        StringBuilder hex = new StringBuilder();
        hex.append('[');
        for (int i = offset; i < (offset + len); i++)
        {
            if (i > offset)
            {
                hex.append(' ');
            }
            hex.append(String.format("%02X",buf[i]));
        }
        hex.append(']');
        return hex.toString();
    }
}

In the above example, I tried to generate compressed bytes for the input "hello" using a 5-byte output buffer

I assume the following result bytes:

buffer 1 [ F2 48 CD C9 C9 ]
buffer 2 [ 07 00 00 00 FF ]
buffer 3 [ FF ]

Translate as

[ F2 48 CD C9 C9 07 00 ] <-- the compressed data
[ 00 00 FF FF ]          <-- the deflate tail bytes

But when deflator When deflate() is used with a small buffer, this normal loop will continue 5 bytes of compressed data indefinitely (it seems to be displayed only in buffers of 5 bytes or less)

Produce the result output of the above demonstration

input is 5 bytes - [48 65 6C 6C 6F]
compressed 5 bytes - [F2 48 CD C9 C9]
compressed 5 bytes - [07 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
compressed 5 bytes - [FF 00 00 00 FF]
Exited compress (maxloops left -1)

If the input / output is greater than 5 bytes, the problem seems to disappear (just enter the string "hellox" to test by yourself)

The result of setting the buffer to 6 bytes (input as "hellox")

input is 6 bytes - [48 65 6C 6C 6F 78]
compressed 6 bytes - [F2 48 CD C9 C9 AF]
compressed 6 bytes - [00 00 00 00 FF FF]
compressed 5 bytes - [00 00 00 FF FF]
Compress success

Even these results are a bit odd to me because it seems to have two reduced tail byte sequences

So, I think my final question is, do I miss something about the use of deflator, which is strange to me, or does it point to possible errors in the JVM deflator implementation itself?

Updated: August 7, 2015

The discovery has been accepted as bugs java. com/JDK-8133170

Solution

This is a zlib "function" in zlib There are records in H:

What happens is that every time you use Z_ SYNC_ Every flush call deflate () inserts a five - byte flush tag Since you did not provide enough output space to get the tag, you called again to get more output, but asked it to insert another refresh tag at the same time

What you should do is use Z_ SYNC_ Flush calls deflate () once, then uses Z_ NO_ Flush (or no_flush in Java), use additional deflate() calls to get all available output if necessary