VictoriaMetrics/lib
Aliaksandr Valialkin 67bc407747 lib/promscrape: reduce memory and CPU usage when Prometheus staleness tracking is enabled for metrics from deleted / disappeared scrape targets
Store the scraped response body instead of storing the parsed and relabeld metrics.
This should reduce memory usage, since the response body takes less memory than the parsed and relabeled metrics.
This is especially true for Kubernetes service discovery, which adds many long labels for all the scraped metrics.

This should also reduce CPU usage, since the marshaling of the parsed
and relabeld metrics has been substituted by response body copying.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
2021-08-21 21:24:07 +03:00
..
auth all: use %w instead of %s for wrapping errors in fmt.Errorf 2020-06-30 23:33:46 +03:00
backup adds check for region with custom s3 endpoint (#1465) 2021-07-27 12:39:10 +03:00
bloomfilter lib/bloomfilter: fix TestLimiterConcurrent 2021-05-24 05:18:29 +03:00
buildinfo
bytesutil Perform conversion from string to []byte according to rule #6 at https://golang.org/pkg/unsafe/#Pointer 2020-08-05 11:55:12 +03:00
cgroup lib/cgroup: set GOGC to 50 by default if it isn't set 2021-07-05 12:34:01 +03:00
consts all: open-sourcing cluster version 2019-05-23 00:25:38 +03:00
decimal all: add support for Prometheus staleness markers 2021-08-13 12:13:15 +03:00
encoding all: add go:build lines for Go1.17 2021-07-26 15:50:46 +03:00
envflag lib/envflag: add a link to docs for -envflag.enable 2021-08-11 10:32:40 +03:00
envtemplate all: support %{ENV_VAR} placeholders in yaml configs in all the vm* components 2020-08-13 17:17:06 +03:00
fastnum Makefile: add build and test rules with enabled race detector. These rules have -race suffix 2020-03-05 12:05:16 +02:00
fasttime lib: extract common code for returning fast unix timestamp into lib/fasttime 2020-05-14 23:06:50 +03:00
filestream fixes solaris build (#1345) 2021-06-04 11:56:06 +03:00
flagutil improve docs for cli flags (#1202) 2021-04-12 12:28:36 +03:00
fs all: add go:build lines for Go1.17 2021-07-26 15:50:46 +03:00
handshake lib/handshake: pass io.EOF unmodified to the caller for BufferedConn.Read, so it could properly detect the end of stream 2021-05-27 12:09:17 +03:00
httpserver lib/httpserver: add is_set label to flag metrics 2021-07-13 15:10:18 +03:00
influxutils lib/influxutils: return response compatible with InfluxDB 1.8.4 2021-03-15 22:20:38 +02:00
ingestserver lib/httpserver: print full requestURI in httpserver.Errorf 2021-07-07 13:11:29 +03:00
leveledbytebufferpool lib/promscrape: an attempt to reduce memory usage when vmagent scrapes targets with varying number of metrics 2021-03-15 11:49:29 +02:00
logger all: add go:build lines for Go1.17 2021-07-26 15:50:46 +03:00
memory all: add go:build lines for Go1.17 2021-07-26 15:50:46 +03:00
mergeset lib/storage: reset perKeyMisses stats less frequently 2021-07-12 14:34:54 +03:00
metricsql lib/metricsql: move it to a separate repository - github.com/VictoriaMetrics/metrics 2020-04-28 15:30:06 +03:00
netutil lib/netutil: enable IPv6 UDP listening if -enableTCP6 command-line flag is passed to VictoriaMetrics 2021-03-17 00:19:30 +02:00
pacelimiter lib/pacelimiter: increase scalability for multi-CPU system 2020-08-06 18:33:07 +03:00
persistentqueue all: add support for Prometheus staleness markers 2021-08-13 12:13:15 +03:00
procutil all: add go:build lines for Go1.17 2021-07-26 15:50:46 +03:00
promauth lib/promauth: cache the client TLS certificate for up to a second 2021-07-02 13:20:18 +03:00
prompb app/vminsert: moved -maxInsertRequestSize command-line flag out of lib/prompb in order to prevent its inclusion in vmselect and vmstorage apps 2020-01-28 22:53:50 +02:00
prompbmarshal all: use %w instead of %s for wrapping errors in fmt.Errorf 2020-06-30 23:33:46 +03:00
promrelabel lib/promrelabel: add tests for verifying that regex works as expected in single quotes and double quotes 2021-07-27 10:53:03 +03:00
promscrape lib/promscrape: reduce memory and CPU usage when Prometheus staleness tracking is enabled for metrics from deleted / disappeared scrape targets 2021-08-21 21:24:07 +03:00
protoparser lib/protoparser/clusternative: remove unused field - unmarshalWork.lastResetTime 2021-07-02 13:32:59 +03:00
proxy lib/promscrape: reload auth tokens from files every second 2021-05-14 20:03:35 +03:00
storage all: add support for Prometheus staleness markers 2021-08-13 12:13:15 +03:00
storagepacelimiter lib/storage: remove prioritizing of merging small parts over merging big parts, since it doesn't work as expected 2020-07-30 20:02:22 +03:00
syncwg
tenantmetrics app/vmselect: export per-tenant stats on the number of requests and the cumulative request duration 2021-02-16 23:30:29 +02:00
timerpool lib/timerpool: use timer pool in concurrency limiters 2019-05-28 17:30:10 +03:00
uint64set Revert "lib/uint64set: allow reusing bucket16 structs inside uint64set.Set via uint64set.Release method" 2021-07-06 18:26:56 +03:00
workingsetcache lib/workingsetcache: switch from split cache to full cache after the cache size exceeds 95% of split capacity 2021-07-15 16:53:35 +03:00
writeconcurrencylimiter all: properly handle CPU limits set on the host system/container 2020-12-08 21:07:03 +02:00