VictoriaMetrics/lib
Aliaksandr Valialkin 53c2135d2a
lib/storage: tune the logic for pre-populating of the per-day inverted index for the next day
- Postpone the pre-poulation to the last hour of the current day. This should reduce the number
  of useless entries in the next per-day index, which shouldn't be created there,
  when the corresponding time series are stopped to be pushed during the current day.

- Make the pre-population more smooth in time by using the hash of MetricID instead of MetricID itself
  when calculating the need for for the given MetricID pre-population.

- Sync the logic for pre-population of the next day inverted index with the logic of pre-populating tsid cache
  after indexdb rotation. This should improve code maintainability.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/430
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401
2022-02-12 16:39:33 +02:00
..
auth all: use %w instead of %s for wrapping errors in fmt.Errorf 2020-06-30 23:33:46 +03:00
backup adds restore.lock (#1988) 2021-12-22 13:10:56 +02:00
blockcache lib/blockcache: use higher number of shards for higher number of CPU cores 2022-02-10 19:11:11 +02:00
bloomfilter lib/promscrape: add the ability to limit the number of unique series per each scrape target 2021-09-01 16:08:12 +03:00
buildinfo
bytesutil lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations 2022-02-01 11:20:20 +02:00
cgroup lib/cgroup: fall back to runtime.NumCPU() when determining process_cpu_cores_available metric if it is impossible to determine cpu quota via cgroups 2022-01-31 20:31:12 +02:00
consts
decimal all: fix build issues and tests for Apple M1 2021-10-27 15:07:19 +03:00
encoding lib/encoding: substitute 64-bits.LeadingZeros64() with bits.Len64() 2022-02-01 11:20:20 +02:00
envflag lib/envflag: use flag.Set for setting the flags from env vars 2021-10-20 00:46:53 +03:00
envtemplate all: support %{ENV_VAR} placeholders in yaml configs in all the vm* components 2020-08-13 17:17:06 +03:00
fastnum
fasttime lib: extract common code for returning fast unix timestamp into lib/fasttime 2020-05-14 23:06:50 +03:00
filestream lib/fs: add vm_filestream_read_duration_seconds_total and vm_filestream_write_duration_seconds_total metrics 2021-12-02 09:13:20 +02:00
flagutil lib/flagutil: do not expose sensitive info (passwords, keys and urls) at /flags page 2021-10-20 00:51:15 +03:00
fs app: allow specifying http and https urls in the following command-line flags 2021-12-03 00:11:47 +02:00
handshake all: substitute zeroTime with time.Time{}, since this generates more optimal binary code 2022-02-07 14:36:41 +02:00
httpserver lib/httpserver: add missing 127.0.0.1 hostname to the logged address for http and pprof server if the address starts with ':' 2021-12-08 16:15:12 +02:00
influxutils all: consistently return application/json content-type without charset=utf-8 2021-11-09 18:07:22 +02:00
ingestserver lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations 2022-02-01 11:20:20 +02:00
leveledbytebufferpool lib/promscrape: an attempt to reduce memory usage when vmagent scrapes targets with varying number of metrics 2021-03-15 11:49:29 +02:00
logger lib/logger: removed broken test after 746ee191e8 2022-01-24 12:15:11 +02:00
memory all: add go:build lines for Go1.17 2021-07-26 15:50:46 +03:00
mergeset lib/mergeset: tune indexdb/{indexBlocks,dataBlocks} cache sizes further according to production stats 2022-02-10 19:11:13 +02:00
metricsql lib/metricsql: move it to a separate repository - github.com/VictoriaMetrics/metrics 2020-04-28 15:30:06 +03:00
netutil lib/netutil: increase dial timeout from 1 second to 5 seconds 2022-02-07 12:33:40 +02:00
pacelimiter lib/pacelimiter: increase scalability for multi-CPU system 2020-08-06 18:33:07 +03:00
persistentqueue lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations 2022-02-01 11:20:20 +02:00
procutil all: add go:build lines for Go1.17 2021-07-26 15:50:46 +03:00
promauth app: allow specifying http and https urls in the following command-line flags 2021-12-03 00:11:47 +02:00
prompb
prompbmarshal all: use %w instead of %s for wrapping errors in fmt.Errorf 2020-06-30 23:33:46 +03:00
promrelabel all: typo fix: unexected -> unexpected 2021-12-20 17:40:13 +02:00
promscrape Show gce sdconfig zone on vmagent:8429/config (#2178) 2022-02-12 00:48:38 +02:00
promutils lib/promscrape: support prometheus-like duration in scrape configs (#2169) 2022-02-11 16:17:51 +02:00
protoparser all: follow-up after 4bdd10ab90 2022-02-01 17:49:28 +02:00
proxy lib/promscrape: properly show proxy_url option value at /config page 2021-10-26 21:24:22 +03:00
storage lib/storage: tune the logic for pre-populating of the per-day inverted index for the next day 2022-02-12 16:39:33 +02:00
storagepacelimiter lib/storage: remove prioritizing of merging small parts over merging big parts, since it doesn't work as expected 2020-07-30 20:02:22 +03:00
syncwg
tenantmetrics app/vmselect: export per-tenant stats on the number of requests and the cumulative request duration 2021-02-16 23:30:29 +02:00
timerpool
uint64set Revert "lib/uint64set: allow reusing bucket16 structs inside uint64set.Set via uint64set.Release method" 2021-07-06 18:26:56 +03:00
workingsetcache lib/workingsetcache: use the original cache size limits when rotating caches 2022-02-08 01:18:27 +02:00
writeconcurrencylimiter all: properly handle CPU limits set on the host system/container 2020-12-08 21:07:03 +02:00