Gzip Middleware =============== This Go package which wraps HTTP *server* handlers to transparently gzip the response body, for clients which support it. For HTTP *clients* we provide a transport wrapper that will do gzip decompression faster than what the standard library offers. Both the client and server wrappers are fully compatible with other servers and clients. This package is forked from the dead [nytimes/gziphandler](https://github.com/nytimes/gziphandler) and extends functionality for it. ## Install ```bash go get -u github.com/klauspost/compress ``` ## Documentation [![Go Reference](https://pkg.go.dev/badge/github.com/klauspost/compress/gzhttp.svg)](https://pkg.go.dev/github.com/klauspost/compress/gzhttp) ## Usage There are 2 main parts, one for http servers and one for http clients. ### Client The standard library automatically adds gzip compression to most requests and handles decompression of the responses. However, by wrapping the transport we are able to override this and provide our own (faster) decompressor. Wrapping is done on the Transport of the http client: ```Go func ExampleTransport() { // Get an HTTP client. client := http.Client{ // Wrap the transport: Transport: gzhttp.Transport(http.DefaultTransport), } resp, err := client.Get("https://google.com") if err != nil { return } defer resp.Body.Close() body, _ := ioutil.ReadAll(resp.Body) fmt.Println("body:", string(body)) } ``` Speed compared to standard library `DefaultTransport` for an approximate 127KB JSON payload: ``` BenchmarkTransport Single core: BenchmarkTransport/gzhttp-32 1995 609791 ns/op 214.14 MB/s 10129 B/op 73 allocs/op BenchmarkTransport/stdlib-32 1567 772161 ns/op 169.11 MB/s 53950 B/op 99 allocs/op BenchmarkTransport/zstd-32 4579 238503 ns/op 547.51 MB/s 5775 B/op 69 allocs/op Multi Core: BenchmarkTransport/gzhttp-par-32 29113 36802 ns/op 3548.27 MB/s 11061 B/op 73 allocs/op BenchmarkTransport/stdlib-par-32 16114 66442 ns/op 1965.38 MB/s 54971 B/op 99 allocs/op BenchmarkTransport/zstd-par-32 90177 13110 ns/op 9960.83 MB/s 5361 B/op 67 allocs/op ``` This includes both serving the http request, parsing requests and decompressing. ### Server For the simplest usage call `GzipHandler` with any handler (an object which implements the `http.Handler` interface), and it'll return a new handler which gzips the response. For example: ```go package main import ( "io" "net/http" "github.com/klauspost/compress/gzhttp" ) func main() { handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "text/plain") io.WriteString(w, "Hello, World") }) http.Handle("/", gzhttp.GzipHandler(handler)) http.ListenAndServe("0.0.0.0:8000", nil) } ``` This will wrap a handler using the default options. To specify custom options a reusable wrapper can be created that can be used to wrap any number of handlers. ```Go package main import ( "io" "log" "net/http" "github.com/klauspost/compress/gzhttp" "github.com/klauspost/compress/gzip" ) func main() { handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "text/plain") io.WriteString(w, "Hello, World") }) // Create a reusable wrapper with custom options. wrapper, err := gzhttp.NewWrapper(gzhttp.MinSize(2000), gzhttp.CompressionLevel(gzip.BestSpeed)) if err != nil { log.Fatalln(err) } http.Handle("/", wrapper(handler)) http.ListenAndServe("0.0.0.0:8000", nil) } ``` ### Performance Speed compared to [nytimes/gziphandler](https://github.com/nytimes/gziphandler) with default settings, 2KB, 20KB and 100KB: ``` λ benchcmp before.txt after.txt benchmark old ns/op new ns/op delta BenchmarkGzipHandler_S2k-32 51302 23679 -53.84% BenchmarkGzipHandler_S20k-32 301426 156331 -48.14% BenchmarkGzipHandler_S100k-32 1546203 818981 -47.03% BenchmarkGzipHandler_P2k-32 3973 1522 -61.69% BenchmarkGzipHandler_P20k-32 20319 9397 -53.75% BenchmarkGzipHandler_P100k-32 96079 46361 -51.75% benchmark old MB/s new MB/s speedup BenchmarkGzipHandler_S2k-32 39.92 86.49 2.17x BenchmarkGzipHandler_S20k-32 67.94 131.00 1.93x BenchmarkGzipHandler_S100k-32 66.23 125.03 1.89x BenchmarkGzipHandler_P2k-32 515.44 1345.31 2.61x BenchmarkGzipHandler_P20k-32 1007.92 2179.47 2.16x BenchmarkGzipHandler_P100k-32 1065.79 2208.75 2.07x benchmark old allocs new allocs delta BenchmarkGzipHandler_S2k-32 22 16 -27.27% BenchmarkGzipHandler_S20k-32 25 19 -24.00% BenchmarkGzipHandler_S100k-32 28 21 -25.00% BenchmarkGzipHandler_P2k-32 22 16 -27.27% BenchmarkGzipHandler_P20k-32 25 19 -24.00% BenchmarkGzipHandler_P100k-32 27 21 -22.22% benchmark old bytes new bytes delta BenchmarkGzipHandler_S2k-32 8836 2980 -66.27% BenchmarkGzipHandler_S20k-32 69034 20562 -70.21% BenchmarkGzipHandler_S100k-32 356582 86682 -75.69% BenchmarkGzipHandler_P2k-32 9062 2971 -67.21% BenchmarkGzipHandler_P20k-32 67799 20051 -70.43% BenchmarkGzipHandler_P100k-32 300972 83077 -72.40% ``` ### Stateless compression In cases where you expect to run many thousands of compressors concurrently, but with very little activity you can use stateless compression. This is not intended for regular web servers serving individual requests. Use `CompressionLevel(-3)` or `CompressionLevel(gzip.StatelessCompression)` to enable. Consider adding a [`bufio.Writer`](https://golang.org/pkg/bufio/#NewWriterSize) with a small buffer. See [more details on stateless compression](https://github.com/klauspost/compress#stateless-compression). ### Migrating from gziphandler This package removes some of the extra constructors. When replacing, this can be used to find a replacement. * `GzipHandler(h)` -> `GzipHandler(h)` (keep as-is) * `GzipHandlerWithOpts(opts...)` -> `NewWrapper(opts...)` * `MustNewGzipLevelHandler(n)` -> `NewWrapper(CompressionLevel(n))` * `NewGzipLevelAndMinSize(n, s)` -> `NewWrapper(CompressionLevel(n), MinSize(s))` By default, some mime types will now be excluded. To re-enable compression of all types, use the `ContentTypeFilter(gzhttp.CompressAllContentTypeFilter)` option. ### Range Requests Ranged requests are not well supported with compression. Therefore any request with a "Content-Range" header is not compressed. To signify that range requests are not supported any "Accept-Ranges" header set is removed when data is compressed. If you do not want this behavior use the `KeepAcceptRanges()` option. ### Flushing data The wrapper supports the [http.Flusher](https://golang.org/pkg/net/http/#Flusher) interface. The only caveat is that the writer may not yet have received enough bytes to determine if `MinSize` has been reached. In this case it will assume that the minimum size has been reached. If nothing has been written to the response writer, nothing will be flushed. ## License [Apache 2.0](LICENSE)