Aliaksandr Valialkin
6cee5338b2
lib/storage: prevent from infinite loop if {__graphite__="..."}
filter matches a metric name with *
, [
or {
chars
...
The idea has been borrowed from https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1137
2021-03-18 14:53:47 +02:00
Aliaksandr Valialkin
e061a4fa19
docs/Single-server-VictoriaMetrics.md: remove outdated message about experimental mode for -pure
builds
2021-03-18 13:49:17 +02:00
Aliaksandr Valialkin
4fb049bcba
lib/fs: reduce the frequency of failed to remove directory ... due to NFS lock
log warnings
...
Log `failed to remove directory ... due to NFS lock` warning only if the directory cannot be removed in one second.
2021-03-18 13:24:46 +02:00
Aliaksandr Valialkin
17d4a6e900
vendor: update github.com/VictoriaMetrics/metrics from v1.16.0 to v1.17.0
2021-03-17 23:23:04 +02:00
Aliaksandr Valialkin
904dababcc
vendor: update github.com/VictoriaMetrics/metrics from v1.15.3 to v1.16.0
...
This adds the following new metrics for each VictoriaMetrics app:
* process_resident_memory_anonymous_bytes - the RSS share for memory allocated by the process itself.
This share cannot be freed by the OS, so it must be taken into account by OOM killer.
* process_resident_memory_pagecache_bytes - the RSS share for page cache memory (aka memory-mapped files).
This share can be freed by the OS at any time, so it must be ignored by OOM killer.
2021-03-17 17:59:40 +02:00
Aliaksandr Valialkin
45dabfac1b
lib/storage: faster move heavy filters to the end of list
2021-03-17 15:12:13 +02:00
Aliaksandr Valialkin
b1713e3fcd
app/vmselect/promql: typo fix after 9666834045
2021-03-17 15:12:11 +02:00
Aliaksandr Valialkin
9666834045
app/vmselect/promql: merge adjancent buckets with the smallest summary number of hits in buckets_limit()
function
...
This should improve accuracy for the returned buckets
2021-03-17 14:31:41 +02:00
Aliaksandr Valialkin
20ac89c4e0
docs/CHANGELOG.md: cut v1.56.0
2021-03-17 02:04:55 +02:00
Aliaksandr Valialkin
bd0c6a095e
docs/CHANGELOG.md: do not mention reduction in query duration
...
There was a signficant refactoring in the code responsible for time series search,
so it can result in both speed ups and slow downs depending on used queries.
2021-03-17 01:56:58 +02:00
Aliaksandr Valialkin
7bc728bf53
app/vmselect: add vm_index_search_duration_seconds
histogram for monitoring the performance of index search
2021-03-17 01:17:41 +02:00
Aliaksandr Valialkin
828669e4e1
all: make golint
happy
2021-03-17 00:49:28 +02:00
Aliaksandr Valialkin
ccfb0ae2d3
lib/storage: limit loops count in order to reduce max CPU usage during filter search
2021-03-17 00:49:26 +02:00
Aliaksandr Valialkin
576a80b3d9
lib/storage: do not modify filterLoopsCount stats with loopsCount stats
...
Such a modification can result in incorrect filter sorting later
2021-03-17 00:49:26 +02:00
Aliaksandr Valialkin
f104f3eb2a
all: make golangci-lint happy after the commit 6378205415
2021-03-17 00:24:40 +02:00
Aliaksandr Valialkin
6378205415
lib/netutil: enable IPv6 UDP listening if -enableTCP6
command-line flag is passed to VictoriaMetrics
...
This is a follow-up for 18cfc4be7b
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1131
2021-03-17 00:16:17 +02:00
Nikolay
18cfc4be7b
Adds udp6 support for ingest servers ( #1134 )
...
with flag -enableUDP6 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1131
2021-03-17 00:03:06 +02:00
Aliaksandr Valialkin
0ce557951f
app/vmselect/netstorage: reduce mutex contention when unpacking data on a system with high number of CPU cores
2021-03-16 21:51:31 +02:00
Aliaksandr Valialkin
35bb44d317
lib/uint64set: optimize bucket16.addMulti a bit
2021-03-16 21:09:07 +02:00
Aliaksandr Valialkin
aba955fa16
Makefile: prepare vmutils-windows-*.zip archive on make release-vmutils
command
...
The archive contains the following executables for Windows:
* vmagent
* vmalert
* vmauth
* vmctl
Other components - vmbackup, vmrestore, victoria-metrics - aren't supported for Windows yet
2021-03-16 20:52:41 +02:00
Aliaksandr Valialkin
fd86a7dc1d
lib/storage: time series search optimization according to production workload profiling
...
Do not pass filter metric ids to getMetricIDsForTagFilter, since it has been appeared that this slows down
the function by multiple times when it finds big number of metricIDs (tens of millions).
2021-03-16 20:01:43 +02:00
Aliaksandr Valialkin
264d3432ac
vendor: make vendor-update
2021-03-16 19:08:36 +02:00
Aliaksandr Valialkin
e36fbfae5b
lib/storage: further tuning for time series search
2021-03-16 18:46:22 +02:00
Aliaksandr Valialkin
f0a4157f89
app/vmselect/promql: do not crash if histogram_over_time()
function name contains uppercase letters such as Histogram_over_time()
2021-03-16 12:24:21 +02:00
Aliaksandr Valialkin
645cf6746c
vendor: update github.com/VictoriaMetrics/metrics from v1.15.2 to v1.15.3
2021-03-16 12:15:50 +02:00
Aliaksandr Valialkin
031d256810
docs/Articles.md: added a link to https://www.youtube.com/watch?v=QgLMztnj7-8
2021-03-15 23:01:49 +02:00
Aliaksandr Valialkin
dd7e82c34f
app/vmstorage: add -logNewSeries
command-line flag for determining the source of series churn rate
2021-03-15 22:38:50 +02:00
Aliaksandr Valialkin
37323c57c9
lib/influxutils: return response compatible with InfluxDB 1.8.4
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1124
2021-03-15 22:19:59 +02:00
f41gh7
acf2a82d3c
typo fix
2021-03-15 23:03:52 +03:00
Aliaksandr Valialkin
85a95bf60c
all: various fixes in command-line flag descriptions
2021-03-15 21:59:25 +02:00
Aliaksandr Valialkin
7c37e9aea9
app/{vminsert,vmagent}: a follow-up for b1aa8c3d8f
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1124
2021-03-15 21:37:55 +02:00
Nikolay
b1aa8c3d8f
adds fake response for telegraph queries ( #1130 )
...
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1124
2021-03-15 21:10:47 +02:00
Aliaksandr Valialkin
9c77e34ef9
lib/storage: further tuning for time series selector code
2021-03-15 20:31:34 +02:00
Aliaksandr Valialkin
923cdb0552
app/vmselect/promql: reduce overhead on scrape interval estimation
...
It should be enough to use the first 20 datapoints instead of 100 datapoints for scrape interval estimation.
2021-03-15 20:31:33 +02:00
Aliaksandr Valialkin
82aab87446
app/vmselect/promql: fix tests after 2dae0a2c47
2021-03-15 20:18:59 +02:00
Aliaksandr Valialkin
fb82c4b9fa
lib/uint64set: reduce the size of bucket16 by storing smallPool by pointer.
...
This reduces CPU time spent on bucket16 copying inside bucket13.addBucketAtPos.
2021-03-15 17:23:31 +02:00
Aliaksandr Valialkin
eb103e1527
lib/uint64set: optimize Set.AddMulti for large sorted sets
2021-03-15 17:10:40 +02:00
Aliaksandr Valialkin
43504ebd14
lib/uint64set: optimize bucket16.add and bucket16.addMulti a bit
2021-03-15 16:58:12 +02:00
Aliaksandr Valialkin
fb935e6e2c
docs/CHANGELOG.md: typo fix: FATURE -> FEATURE
2021-03-15 14:58:19 +02:00
Aliaksandr Valialkin
3ccf7ea20c
lib/storage: tune per-day index search
2021-03-15 13:31:55 +02:00
Aliaksandr Valialkin
2dae0a2c47
app/vmselect: add round_digits
query arg to /api/v1/query
and /api/v1/query_range
handlers for limiting the number of decimal digits after the point
2021-03-15 12:36:33 +02:00
Roman Khavronenko
b457739f87
Single dashboard ( #1126 )
...
* dashboard: update single node dashboard
* add panel `Open FDs` for file descriptors metrics;
* add panel `Disk writes/reads` to show the real read/write
load on storage layer;
* add `process_resident_memory_bytes` metric to memory usage panel;
* add stats panel to show available CPUs, memory and disk space;
* rm flags panel since it didn't prove its usefulness.
* alerts: add alert for reaching FDs limit
2021-03-15 12:04:24 +02:00
Aliaksandr Valialkin
6d91842c83
docs/Cluster-VictoriaMetrics.md: sync with cluster README.md
2021-03-15 11:51:40 +02:00
Aliaksandr Valialkin
c14dafce43
lib/promscrape: an attempt to reduce memory usage when vmagent scrapes targets with varying number of metrics
...
Do not cache too big byte buffers and too big writeRequestCtx objects,
since it is cheaper to re-create them instead of wasting RAM for their caching.
This reverts 7f6f350ee1
2021-03-15 11:45:39 +02:00
Aliaksandr Valialkin
7f6f350ee1
lib/promscrape: return back the logic for flushing big buffers to storage from the commit 3fd8653b40
...
This should reduce memory usage when vmagent scrapes targets with big number of metrics and `-promscrape.streamParse` isn't enabled
2021-03-14 22:26:00 +02:00
Aliaksandr Valialkin
b88806ecbf
lib/promscrape/discovery/kubernetes: do not start object watcher until initial objects are loaded
2021-03-14 21:55:00 +02:00
Aliaksandr Valialkin
83edbb7cab
lib/promscrape: retry service discovery in a few seconds if it starts returning 0 targets
...
This should reduce recovery time from temporary issues during service discovery
2021-03-14 21:53:23 +02:00
Aliaksandr Valialkin
bf15d6a6a2
lib/promscrape: remove duplicate target
word in error message
2021-03-14 21:52:02 +02:00
Aliaksandr Valialkin
d409898515
lib/promscrape/discovery/kubernetes: further optimize kubernetes service discovery for the case with many scrape jobs
...
Do not re-calculate labels per each scrape job - reuse them instead for scrape jobs with identical Kubernetes role
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1113
2021-03-14 21:14:53 +02:00
Aliaksandr Valialkin
7a16e8e3a2
lib/promscrape/discovery: fixes after 133b288681
...
- Removed a deadlock in addAPIWatcher
- Do not create unused ScrapeWork objects
- Do not spend CPU resources on creating objectByKey map in addAPIWatcher
This work is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1125
2021-03-13 15:18:51 +02:00