Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-15 16:30:55 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
4,486 Commits 114 Branches 748 Tags 504 MiB
0b76d07c21
Commit Graph

126 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
daefb64f38
app/vmselect: expose additional histograms at /metrics page, which may help get more insights for the query workload 
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792
 2022-06-28 20:18:31 +03:00
Aliaksandr Valialkin
7d5d33fd71
lib/storage: return marshaled metric names from SearchMetricNames 
Previously SearchMetricNames was returning unmarshaled metric names.
This wasn't great for vmstorage, which should spend additional CPU time
for marshaling the metric names before sending them to vmselect.

While at it, remove possible duplicate metric names, which could occur when
multiple samples for new time series are ingested via concurrent requests.

Also sort the metric names before returning them to the client.
This simplifies debugging of the returned metric names across repeated requests to /api/v1/series
 2022-06-28 18:16:32 +03:00
Aliaksandr Valialkin
399d4c36ae
app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day 2022-06-28 12:55:20 +03:00
Aliaksandr Valialkin
a667d339be
app/vmselect/netstorage/netstorage.go: group metrics in order to improve readability a bit 2022-06-27 14:00:24 +03:00
Aliaksandr Valialkin
08de733924
app/vmselect/netstorage: assume the response is full if up to -replicationFactor-1 vmstorage nodes are unavailable 
This is a follow-up for ee5c502446

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1767
 2022-06-27 12:21:26 +03:00
Aliaksandr Valialkin
bc9d704ef4
app/vmselect/netstorage: remove Get prefix from netstorage functions 
This makes these function names more consistent with the server side
 2022-06-27 00:37:49 +03:00
hagen1778
e40d015e9a vmselect: make vm_partial_results_total consistent 
Metrics `vm_partial_results_total` and `vm_requests_total` serving
the similar purpose, but contain inconsistent set of labels.
This change updates `vm_partial_results_total` labels to be consistent
with `vm_requests_total`.

The change breaks backward compatibility with assumption that
`vm_partial_results_total` wasn't widely used, since it is
not documented and absent in the alerts and dashboards.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-06-24 13:50:26 +02:00
Nikolay
ee5c502446
app/vmselect: fixes partial response with replicationFactor (#2777) 
* app/vmselect: fixes partial response with replicationFactor
Allow partial response if it meets replicationFactor configured at vmselect
https://t.me/VictoriaMetrics_ru1/38490

* docs/CHANGELOG.md: document this change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-23 20:17:24 +03:00
Aliaksandr Valialkin
dceca7e864
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package 
This is a follow-up for fe2269b999
 2022-06-21 20:27:30 +03:00
Aliaksandr Valialkin
b28c6febf9
app/{vminsert,vmselect}: add -vmstorageDialTimeout command-line flag for tuning the maximum time needed for establishing connections to vmstorage 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
 2022-06-20 15:17:34 +03:00
Aliaksandr Valialkin
da1d1e83df
app/{vmselect,vmstorage}: properly pass seriesCountByLabelName and seriesCountByFocusLabelValue entries from vmstorage to vmselect 2022-06-16 10:44:29 +03:00
Aliaksandr Valialkin
ee9954082f
app/vmselect/netstorage: properly aggregate seriesCountByLabelName and seriesCountByFocusLabelValue obtained from multiple vmselect nodes at /api/v1/status/tsdb 2022-06-15 16:48:40 +03:00
Aliaksandr Valialkin
45fa9d798d
app/vmselect: accept focusLabel query arg at /api/v1/status/tsdb 2022-06-14 18:39:00 +03:00
Aliaksandr Valialkin
61e03f172b
app/vmselect: optimize /api/v1/labels and /api/v1/label/.../values handlers when match[] query arg is passed to them 2022-06-12 14:06:24 +03:00
Aliaksandr Valialkin
4a94cd81ce
app/vmselect: add optional limit query arg to /api/v1/labels and /api/v1/label_values endpoints 
This arg allows limiting the number of sample values returned from these APIs
 2022-06-10 10:24:07 +03:00
Aliaksandr Valialkin
a9ea3fee38
lib/querytracer: make it easier to use by passing trace context message to New and NewChild 
The context message can be extended by calling Donef.
If there is no need to extend the message, then just call Done.
 2022-06-08 21:16:12 +03:00
Aliaksandr Valialkin
2b343d8bd0
app: properly collect and merge /api/v1/status/tsdb info from vmstorage nodes 
The collection has been broken in f2754c3e90

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233
 2022-06-08 19:26:09 +03:00
Dmytro Kozlov
f2754c3e90
Cardinality explorer (#2625) 
* Cardinality explorer

* vmui, vmselect: updated field name, added description to spinner

* make vmui-update

* updated const name, make vmui-update

* lib/storage: changes calculation for totalSeries values

* added static files

* wip

* wip

* wip

* wip

* docs/CHANGELOG.md: document cardinality explorer feature

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233

Co-authored-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-06-08 18:54:27 +03:00
Aliaksandr Valialkin
c92bc5394f
app/vmselect/netstorage: properly read trace from vmstorage when it returns error message to vmselect 2022-06-01 14:35:00 +03:00
Aliaksandr Valialkin
afced37c0b
all: add initial support for query tracing 
See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
 2022-06-01 02:31:44 +03:00
Aliaksandr Valialkin
a4a15a462b
app/vmselect/netstorage: bump RPC API versions for vmselect->vmstorage communications 
This is a follow-up after b843f0e229
 2022-04-08 12:36:04 +03:00
Aliaksandr Valialkin
b843f0e229
app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs 2022-03-26 11:28:14 +02:00
Aliaksandr Valialkin
89ead3daca
app/vmselect/netstorage: report vmstorage errors to vmselect clients even if partial responses are allowed 
If a vmstorage is reachable and returns an application-level error to vmselect,
then such error must be returned to the caller even if partial responses are allowed,
since it usually means cluster mis-configuration.

Partial responses may be returned only if some vmstorage nodes are temporarily unavailable.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1941
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/678
 2022-02-21 21:17:05 +02:00
Aliaksandr Valialkin
5f266370c5
all: follow-up after 4bdd10ab90 
Properly use new bytesutil.Resize* functions
 2022-02-01 17:49:28 +02:00
Aliaksandr Valialkin
02b2bfcff3
lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations 
Follow-up for f4989edd96

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
 2022-02-01 11:20:20 +02:00
Aliaksandr Valialkin
6232eaa938
lib/bytesutil: split Resize() into ResizeNoCopy() and ResizeWithCopy() functions 
Previously bytesutil.Resize() was copying the original byte slice contents to a newly allocated slice.
This wasted CPU cycles and memory bandwidth in some places, where the original slice contents wasn't needed
after slize resizing. Switch such places to bytesutil.ResizeNoCopy().

Rename the original bytesutil.Resize() function to bytesutil.ResizeWithCopy() for the sake of improved readability.

Additionally, allocate new slice with `make()` instead of `append()`. This guarantees that the capacity of the allocated slice
exactly matches the requested size. The `append()` could return a slice with bigger capacity as an optimization for further `append()` calls.
This could result in excess memory usage when the returned byte slice was cached (for instance, in lib/blockcache).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
 2022-01-25 15:28:42 +02:00
Aliaksandr Valialkin
bc3923111b
lib/storage: return dedup interval in milliseconds from GetDedupInterval() 
This removes duplicate .Milliseconds() calls after GetDedupInterval() calls.
 2021-12-15 13:27:27 +02:00
Aliaksandr Valialkin
cdfe854c9b
lib/storage: explicitly pass dedupInterval to DeduplicateSamples() and deduplicateSamplesDuringMerge() 
This improves the code readability and debuggability, since the output of these functions
stops depending on global state.
 2021-12-14 20:52:29 +02:00
Aliaksandr Valialkin
a746fe8778
app/vmselect/promql: follow-up after 57b3320478 2021-09-24 01:25:04 +03:00
Aliaksandr Valialkin
7ad54041fe app/{vminsert,vmselect}: automatically add missing port in -storageNode lists passed to vminsert and vmselect 
This should simplify manual setup of the cluster according to https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#cluster-setup
 2021-09-15 18:04:30 +03:00
Aliaksandr Valialkin
99004a6a40 app/vmselect/netstorage: unpack time series data in mostly local big chunks 
This should improve performance on multi-CPU systems for queries selecting time series with big number of raw samples
 2021-07-30 12:26:33 +03:00
Aliaksandr Valialkin
cbb81c2ce9 app/vmselect/netstorage: do not query Go maps with unsafe string keys, since this breaks in Go 1.17 2021-07-30 10:28:19 +03:00
Aliaksandr Valialkin
49bf3abf67 app/vmselect: follow-up for 626073bca8 
* Rename -search.maxMetricsPointSearch to -search.maxSamplesPerQuery, so it is more consistent with the existing -search.maxSamplesPerSeries
* Move the -search.maxSamplesPerQuery from vmstorage to vmselect, so it could effectively limit the number of raw samples obtained from all the vmstorage nodes
* Document the -search.maxSamplesPerQuery in docs/CHANGELOG.md
 2021-07-28 18:00:04 +03:00
Aliaksandr Valialkin
3921d8afae app/vmselect: prevent from possible deadlock when f callback blocks inside RunParallel 2021-07-26 15:50:45 +03:00
Aliaksandr Valialkin
b92702f6d5 app/vmselect/netstorage: use more scalable algorithm for ditributing the work among among multiple channels on systems with big number of CPU cores 2021-07-16 00:35:36 +03:00
Aliaksandr Valialkin
5830ce2706 app/vmselect/netstorage: add -search.maxSamplesPerSeries command-line option for limiting the number of samples a query can process per each series 
This should prevent from out of memory crashes like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1067
 2021-07-15 16:53:35 +03:00
Aliaksandr Valialkin
6c42db87a8 app/vmselect/netstorage: improve scalability of series unpacking on multi-CPU systems 2021-07-15 15:40:53 +03:00
Aliaksandr Valialkin
a0b001bfec app/vmselect/netstorage: remove duplicate limiter on concurrent queries 
It duplicates the `-search.maxConcurrentRequests` limiter.
 2021-05-24 19:13:04 +03:00
Nikolay
be87be34a4 Adds tsdb match filters (#1282) 
* init work on filters

* init propose for status filters

* fixes tsdb status
adds test

* fix bug

* removes checks from test
 2021-05-12 17:16:58 +03:00
Aliaksandr Valialkin
ecfd6fe78d app/vmselect: remove -search.storageTimeout command-line flag, since it has the same meaning as -search.maxQueryDuration 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
 2021-03-30 14:54:36 +03:00
Aliaksandr Valialkin
7bafaad46d app/vmselect: prevent from possible incomplete query results after timed out query 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
 2021-03-30 13:34:57 +03:00
Aliaksandr Valialkin
05e301cfa0 app/vmselect/netstorage: reduce mutex contention when unpacking data on a system with high number of CPU cores 2021-03-16 21:52:05 +02:00
Aliaksandr Valialkin
1b90a091cf app/vmselect/netstorage: cleanup after 4805b80977 2021-02-24 11:43:09 +02:00
weng zhao
4805b80977
vmselect: fix vmselect cache partial result when timeout (#1085) 2021-02-24 11:39:56 +02:00
Aliaksandr Valialkin
73c9da16b8 app/vmselect/netstorage: reuse timeseriesWork objects in order to reduce memory allocations 2021-02-16 16:09:00 +02:00
Aliaksandr Valialkin
a3a09a3c6e app/vmselect/netstorage: use unsafe string as a key for a map when the map already contains the given key 
This should prevent from a memory allocation and a string copy.
 2021-02-16 15:46:31 +02:00
Aliaksandr Valialkin
1a237c6903 all: properly handle CPU limits set on the host system/container 
This can reduce memory usage on systems with enabled CPU limits.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946
 2020-12-08 21:07:03 +02:00
Aliaksandr Valialkin
9660774fd1 app/vmselect/graphite: remove duplicate name tag from /tags/autoComplete/tags handler 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/942
 2020-12-07 01:10:02 +02:00
Aliaksandr Valialkin
3d2ce31cad app/vmselect/netstorage: code readability improvement: rename *RequestErrors to *Errors 2020-11-23 15:00:15 +02:00
Aliaksandr Valialkin
433ae806ac app/vmselect: implement /tags/tagSeries and /tags/tagMultiSeries` in order to be consistent with single-node VictoriaMetrics 2020-11-23 14:57:08 +02:00