I am creating GZIPs on demand by streaming data, but I need to split it because the receving end has a hard code limit. When I Flush() and Close(), I see that the underyling byte buffer grows by 13 bytes. I looked at the source code of Gzip Close:
func (z *Writer) Close() error {
if z.err != nil {
return z.err
}
if z.closed {
return nil
}
z.closed = true
if !z.wroteHeader {
z.Write(nil)
if z.err != nil {
return z.err
}
}
z.err = z.compressor.Close()
if z.err != nil {
return z.err
}
le.PutUint32(z.buf[:4], z.digest)
le.PutUint32(z.buf[4:8], z.size)
_, z.err = z.w.Write(z.buf[:8])
return z.err
}
It indeed writes something but is there someway to determine it more pragmatic than just saying 13 bytes? There can be headers etc. I just want to have a safe margin, is there any possibilities that it can grow way larger than 13 bytes? I can happily set 1kb margin and live with it.
The 13 bytes are the maximum value to my knowledge. 8 bytes come from the gzip footer, the two PutUint32
calls. The other 5 bytes are added by the huffmann compressor which ads an empty final block when the compressor is closed. It will add 3 bits (= 1 byte) for the final block header and 2 bytes for the length 0
and another 2 bytes which are inverted length 0xffff
. So i assume you can calculate with those 13 bytes.
A conservative upper bound for the gzip-compressed output is:
n + ((n + 7) >> 3) + ((n + 63) >> 6) + 23
where n
is the size of the input in bytes.