Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-12 20:55:22 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
c41a9b8d17
VictoriaMetrics/lib
History
Roman Khavronenko c41a9b8d17
lib/bytesutil: smooth buffer growth rate (#6761) 
Before, buffer growth was always x2 of its size, which could lead to
excessive memory usage when processing big amount of data.
For example, scraping a target with hundreds of MBs in response could
result into hih memory spikes in vmagent because buffer has to double
its size to fit the response. See
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6759

The change smoothes out the growth rate, trading higher allocation rate
for lower mem usage at certain conditions.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f28f496a9d)
2024-08-07 16:59:23 +02:00
..
appmetrics all: add -metrics.exposeMetadata command-line flag, which can be used for adding TYPE and HELP metadata for metrics exposed at /metrics page 2023-12-19 03:26:02 +02:00
auth lib/auth: add NewTokenPossibleMultitenant() for parsing auth token, which can be multitenant 2023-08-30 14:13:51 +02:00
awsapi lib/awsapi: properly assume role with webIdentity token (#5495) 2023-12-20 19:07:04 +02:00
backup lib/backup/s3remote: fix typos (#6694) 2024-07-29 14:30:21 +02:00
blockcache all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
bloomfilter lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:10:04 +02:00
bufferedwriter app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils 2023-07-06 17:22:23 -07:00
buildinfo
bytesutil lib/bytesutil: smooth buffer growth rate (#6761) 2024-08-07 16:59:23 +02:00
cgroup lib/cgroup: remove SetGOGC() function 2024-02-05 12:13:08 +02:00
consts app/vminsert: reduce the max packet size, which vminsert can send to vmstorage 2022-04-05 15:39:58 +03:00
decimal lib/slicesutil: add helper functions for setting slice length and extending its capacity 2024-05-12 11:33:49 +02:00
encoding lib/encoding: optimize UnmarshalVarUint64, UnmarshalVarInt64 and UnmarshalBytes a bit 2024-05-14 01:30:25 +02:00
envflag lib/envflag: do not allow unsupported form for boolean command-line flags in the form -boolFlag value 2023-08-17 13:37:05 +02:00
envtemplate allowed using dashes and dots in environment variables names (#4009) 2023-03-24 17:57:19 -07:00
fastnum lib/fastnum: use unsafe.Slice() instead of deprecated reflect.SliceHeader 2024-02-29 17:17:24 +02:00
fasttime lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:10:04 +02:00
filestream lib/filestream: do not measure read / write duration from / to in-memory buffers 2024-01-23 14:53:35 +02:00
flagutil lib/httpserver: skip basic auth check for additional request paths, which should call httpserver.CheckAuthFlag() 2024-07-16 01:08:41 +02:00
formatutil app/vmbackupmanager: add metrics for better observability (#488) 2022-12-20 14:18:43 -08:00
fs fix typos in comments 2024-08-06 16:30:10 +02:00
handshake lib/handshake: substitute time.Now() with fastttime.UnixTimestamp(), since profiling shows time.Now() is slow 2024-01-23 18:39:28 +02:00
htmlcomponents lib/htmlcomponents: use relative links for the top page and for favicon.ico 2023-11-13 20:28:17 +01:00
httpserver all: consistently use stringsutil.JSONString() for formatting JSON strings with fmt.* functions instead of using "%q" formatter 2024-07-17 14:01:37 +02:00
httputils lib/{httputils,netutil}: move httputils.GetStatDialFunc to netutil.NewStatDialFunc 2024-07-15 23:05:46 +02:00
influxutils app/{vminsert,vmagent}: add healthcheck for influx ingestion endpoints (#6749) 2024-08-05 09:45:32 +02:00
ingestserver Revert c6c5a5a186 and b2765c45d0 2024-07-03 23:57:49 +02:00
leveledbytebufferpool lib/leveledbytebufferpool: do not pool byte slices bigger than 2^18 bytes 2024-06-13 17:02:05 +02:00
logger all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
logstorage app/vlinsert/loki: use easyproto instead for parsing Loki protobuf messages 2024-07-10 03:05:55 +02:00
lrucache all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
memory all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:15:42 +01:00
mergeset lib/mergeset: fix typos in comments 2024-08-07 16:59:22 +02:00
metricsql all: make fmt via the upcoming Go1.19 2022-07-11 19:23:25 +03:00
netutil lib/netutil: validate TLS cert and key files immediately (#6621) 2024-07-29 14:30:20 +02:00
persistentqueue app/vmagent/remotewrite: follow-up for 87fd400dfc 2024-07-13 02:30:10 +02:00
procutil all: cleanup: remove // +build ... lines, since they are no longer needed after Go1.17, and the minimum supported Go version for VictoriaMetrics source code is Go1.20 2023-11-13 19:15:42 +01:00
promauth all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
prompb Revert "Exemplar support (#5982)" 2024-07-03 16:09:18 +02:00
prompbmarshal Revert "Exemplar support (#5982)" 2024-07-03 16:09:18 +02:00
promrelabel lib: consistently use regexp.Regexp.ReplaceAllLiteralString instead of regexp.Regexp.ReplaceAllString in places where the replacement cannot contain matching group placeholders 2024-07-17 12:57:43 +02:00
promscrape app/vmagent: read __sample_limit__ from labels (#6665) (#6666) 2024-08-07 09:57:48 +02:00
promutils all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
protoparser all: consistently use stringsutil.JSONString() for formatting JSON strings with fmt.* functions instead of using "%q" formatter 2024-07-17 14:01:37 +02:00
proxy all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
pushmetrics lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop() 2024-01-16 21:18:22 +02:00
querytracer lib: consistently use regexp.Regexp.ReplaceAllLiteralString instead of regexp.Regexp.ReplaceAllString in places where the replacement cannot contain matching group placeholders 2024-07-17 12:57:43 +02:00
ratelimiter app/vmagent: properly shutdown when -maxIngestionRate limit is reached 2024-04-03 02:41:11 +03:00
regexutil lib/logstorage: work-in-progress 2024-05-25 22:59:21 +02:00
slicesutil lib/slicesutil: add helper functions for setting slice length and extending its capacity 2024-05-12 11:33:49 +02:00
snapshot lib: consistently use f-tests instead of table-driven tests 2024-07-09 22:39:13 +02:00
storage Revert "refactor(vmstorage): Refactor the code to reduce the time complexity of MustAddRows and improve readability (#6629)" 2024-07-25 14:43:00 +02:00
streamaggr lib/streamaggr: remove resetState arg from aggrState.flushState() 2024-08-07 11:46:49 +02:00
stringsutil all: consistently use stringsutil.JSONString() for formatting JSON strings with fmt.* functions instead of using "%q" formatter 2024-07-17 14:01:37 +02:00
syncwg
tenantmetrics lib/encoding/zstd: switch back from atomic.Pointer to atomic.Value for map[...]... 2023-07-20 21:54:51 -07:00
timerpool
timeutil all: add up to 10% random jitter to the interval between periodic tasks performed by various components 2024-01-22 18:39:16 +02:00
uint64set lib/uint64set: optimize Set.Has() for nil Set - it should be inlined now 2024-07-16 00:00:46 +02:00
vmselectapi lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:10:04 +02:00
workingsetcache lib: consistently use atomic.* types instead of atomic.* functions 2024-02-24 02:10:04 +02:00
writeconcurrencylimiter app/vmagent/remotewrite: clarify the reason behind the default value for -remoteWrite.queues in the same way as the reason for -maxConcurrentInserts is defined at 73f5fb0f0c 2024-03-06 13:57:53 +02:00