VictoriaMetrics/lib
Aliaksandr Valialkin f3e5c9c246
lib/bytesutil: cache results for all the input strings, which were passed during the last 5 minutes from FastStringMatcher.Match(), FastStringTransformer.Transform() and InternString()
Previously only up to 100K results were cached.
This could result in sub-optimal performance when more than 100K unique strings were actually used.
For example, when the relabeling rule was applied to a million of unique Graphite metric names
like in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3466

This commit should reduce the long-term CPU usage for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3466
after all the unique Graphite metrics are registered in the FastStringMatcher.Transform() cache.

It is expected that the number of unique strings, which are passed to FastStringMatcher.Match(),
FastStringTransformer.Transform() and to InternString() during the last 5 minutes,
is limited, so the function results fit memory. Otherwise OOM crash can occur.
This should be the case for typical production workloads.
2022-12-12 14:47:00 -08:00
..
appmetrics all: add ability to push internal metrics to remote storage system specified via -pushmetrics.url 2022-07-21 20:15:29 +03:00
auth app/vminsert: allows parsing tenant id from labels (#3009) 2022-09-30 17:28:35 +03:00
awsapi lib/promscrape/discovery/ec2: expose __meta_ec2_region label in the same way as Prometheus 2.39 does 2022-09-30 20:49:08 +03:00
backup lib/backup: remove logging duplicate path values in a single error message 2022-12-03 21:55:12 -08:00
blockcache all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package 2022-06-21 20:24:28 +03:00
bloomfilter
buildinfo
bytesutil lib/bytesutil: cache results for all the input strings, which were passed during the last 5 minutes from FastStringMatcher.Match(), FastStringTransformer.Transform() and InternString() 2022-12-12 14:47:00 -08:00
cgroup all: use os.{Read|Write}File instead of ioutil.{Read|Write}File 2022-08-21 23:55:20 +03:00
consts app/vminsert: reduce the max packet size, which vminsert can send to vmstorage 2022-04-05 15:39:58 +03:00
decimal
encoding lib/storage: validate timestamps in the block only if they use encoding, which needs validation 2022-10-21 00:54:37 +03:00
envflag lib/envflag: small refactoring after 518c340ae3 and 02096e06d0 2022-10-29 02:29:19 +03:00
envtemplate lib/envtemplate: allow non-env var names inside "%{ ... }" 2022-11-07 13:16:00 +02:00
fastnum
fasttime
filestream lib/filestream: remove logging redundant path values in a single error message 2022-12-03 22:02:04 -08:00
flagutil app/vmagent/remotewrite: allow specifying per--remoteWrite.url disk limits for persistent queue with pending data 2022-10-01 18:41:21 +03:00
fs lib/fs: remove logging redundant path values in a single error message 2022-12-03 22:00:43 -08:00
handshake all: substitute zeroTime with time.Time{}, since this generates more optimal binary code 2022-02-07 14:36:41 +02:00
httpserver lib/httpserver: link to url format docs in error message emitted on parse error for the provided url path 2022-11-28 18:49:03 -08:00
influxutils lib/flagutil: rename Array to ArrayString 2022-10-01 18:28:19 +03:00
ingestserver lib/httpserver: extract the code responsible for initializing server-side TLS config into netutil.GetServerTLSConfig 2022-03-17 19:46:20 +02:00
leveledbytebufferpool all: make fmt via the upcoming Go1.19 2022-07-11 19:23:25 +03:00
logger lib/logger: prettify logging the defined command-line flags 2022-08-07 22:58:41 +03:00
lrucache all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package 2022-06-21 20:24:28 +03:00
memory lib/memory: export process_memory_limit_bytes metric, which shows the amounts of memory the current process has access to 2022-04-07 15:24:08 +03:00
mergeset lib/{storage,mergeset}: log the duration for flushing in-memory parts on graceful shutdown 2022-12-05 21:55:21 -08:00
metricsql all: make fmt via the upcoming Go1.19 2022-07-11 19:23:25 +03:00
netutil app/{vminsert,vmselect}/netstorage: allow calling Init()+MustStop() in a loop 2022-10-25 14:43:05 +03:00
pacelimiter
persistentqueue lib/fs: add canOverwrite arg to WriteFileAtomically when it is allowed to overwrite the file atomically if it already exists 2022-10-26 01:08:35 +03:00
procutil lib/procutil: stop immediately after receiving the second SIGINT or SIGTERM signal 2022-10-20 21:58:49 +03:00
promauth lib/{httpserver,netutil}: allow to define min and max TLS version of the http server (#3109) 2022-09-26 17:38:43 +03:00
prompb
prompbmarshal
promrelabel lib/promscrape: implement target-level and metric-level relabel debugging 2022-12-10 02:25:56 -08:00
promscrape lib/promscrape: allow editing relabeling configs and labels at /target-relabel-debug page 2022-12-10 12:47:47 -08:00
promutils lib/promscrape: implement target-level and metric-level relabel debugging 2022-12-10 02:25:56 -08:00
protoparser lib/protoparser/datadog: do not re-use previously parsed field values if they are missing in the currently parsed message 2022-12-11 13:09:41 -08:00
proxy lib/promauth: add ability to send additional http headers in requests to scrape targets 2022-06-22 20:40:50 +03:00
pushmetrics lib/flagutil: rename Array to ArrayString 2022-10-01 18:28:19 +03:00
querytracer lib/querytracer: fix remaining tests after 49ebc48809 2022-12-08 18:18:50 -08:00
regexutil lib/promrelabel: there is no need in calling regex.HasPrefix() after the optimization at 17289ff481 2022-09-30 12:28:25 +03:00
snapshot all: subsitute ioutil.ReadAll with io.ReadAll 2022-08-22 00:16:04 +03:00
storage lib/storage: follow-up after 7c0ae3a86a 2022-12-08 18:18:49 -08:00
storagepacelimiter
syncwg
tenantmetrics app/vminsert: allows parsing tenant id from labels (#3009) 2022-09-30 17:28:35 +03:00
timerpool
uint64set
vmselectapi {app/vmstorage,app/vmselect}: add API to get list of existing tenants (#3348) 2022-11-25 10:32:45 -08:00
workingsetcache lib/workingsetcache: expose -cacheExpireDuration command-line flag for fine-tuning of the cache expiration 2022-11-17 21:55:11 +02:00
writeconcurrencylimiter fix: change thrashing typo (#2317) 2022-03-16 13:05:55 +02:00