Aliaksandr Valialkin
3d4c312ba2
app/vmselect/netstorage: improve the speed of queries over big number of time series on multi-CPU system
...
Reduce inter-CPU communications when processing the query over big number of time series.
This should improve performance for queries over big number of time series
on systems with many CPU cores.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2896
Based on b596ac3745
Thanks to @zqyzyq for the idea.
2022-07-25 09:22:28 +03:00
Aliaksandr Valialkin
c0c9f30870
lib/pushmetrics: properly handle errors when initializing pushmetrics
2022-07-22 13:38:25 +03:00
Aliaksandr Valialkin
59bb42be06
app/vmselect/vmui: make vmui-update
after edecd2493c
2022-07-21 21:00:36 +03:00
Roman Khavronenko
04c4f8bafd
vmselect: return correct error for second part of expression ( #2893 )
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-21 20:40:47 +03:00
Aliaksandr Valialkin
fe68bb3ba7
all: follow-up after 46f803fa7a
...
Add -pushmetrics.* command-line flags to all the VictoriaMetrics apps
2022-07-21 20:18:25 +03:00
Aliaksandr Valialkin
f992f96a88
app/vmselect/promql: execute q1
and q2
from q1 op q2
in parallel if labels pushdown cannot be applied
...
This should improve query performance if VictoriaMetrics has enough resources for processing `q1` and `q2` in parallel.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2886
2022-07-19 14:29:41 +03:00
Aliaksandr Valialkin
d920b0afec
app/vmselect/promql: properly return q1
series from q1 ifnot q2
when q2
returns nothing
2022-07-18 14:24:23 +03:00
Aliaksandr Valialkin
f933d5f142
app/vmselect/promql/transform.go: reuse evalNumber() function for constructing timezone_offset() results
2022-07-18 14:24:22 +03:00
Aliaksandr Valialkin
70b9925bf7
app: fix make publish-*
after ed93330e66
...
Add missing `-linux` substring to built binary names for copying into Docker images
2022-07-14 11:01:34 +03:00
Yury Molodov
1ca20caa4b
vmui: optimize table view ( #2867 )
...
* feat: optimize table view
* fix: add column display setting
* app/vmselect: `make vmui-update`
Also document the change at docs/CHANGELOG.md
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-14 00:16:57 +03:00
Aliaksandr Valialkin
da6c85a2f6
all: follow-up for d99ba3481b
2022-07-13 17:17:08 +03:00
Dmytro Kozlov
4e4def9df8
Rename release packages ( #2810 )
...
* makefile: add os to each release file
* makefile: update vmutils arm64
* makefile: update victoria-metrics release process
* makefile: update publish with os
* makefile: update publish with os
* makefile: change tar library
* update release logic
* copy all releases
* sort command by GOOS
* rollback commands
* rollback OSARCH
* fix commands
* cleanup
* fix windows build
* sort build by GOOS, update README.md
2022-07-13 17:11:01 +03:00
Aliaksandr Valialkin
919939ee9f
app/vmselect/promql: consistency update after 93fbd0c54b
2022-07-13 12:34:28 +03:00
Roman Khavronenko
214e01499c
promql: return step
as scrapeInterval when it can't be calculated ( #2865 )
...
The change allows to specify default value for `getScrapeInterval`
function when actual interval can't be calculated.
Before the change, function were returning `maxSilenceInterval` (5m)
in such cases, which may be not correct for instant queries processing.
The specific scenario where using `maxSilenceInterval` caused issues
is the following:
1. Series becomes stale;
2. Client (in this case vmalert) continues to request series every 15s;
3. Database returns empty results as expected;
4. But at some specific moment of time database returns datapoints from `now()-5m`,
because lookback window was extended to `maxSilenceInterval`.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-13 12:34:28 +03:00
Aliaksandr Valialkin
5867708b3d
app/vmselect/promql: validate function name before evaluating its arguments
...
This avoids unneeded evaluation of args for unknown functions
2022-07-12 19:49:54 +03:00
Aliaksandr Valialkin
fbb403b5c0
app/vmselect/netstorage: optimize mergeSortBlocks() for the worst case when blocks contain interleaved samples
2022-07-12 12:30:24 +03:00
Aliaksandr Valialkin
aee08117e9
app/vmselect/netstorage: add mergeSortBlocks benchmark for the worstcase
2022-07-12 12:26:27 +03:00
Dmytro Kozlov
6734d77d97
vmselect/prometeus: Add limit param to api/v1/series api endpoint ( #2851 )
...
* issue-2841: Add limit param to api/v1/series api endpoint
* issue-2841: add change log
* issue-2841: update logic
* issue-2841: simplify logic
* issue-2841: simplify logic, add information to documentation
2022-07-11 20:37:20 +03:00
Aliaksandr Valialkin
ed2126aec3
app/vmselect: follow-up after 6dda254e01
2022-07-11 19:48:18 +03:00
Roman Khavronenko
6dda254e01
vmselect: cover special cases for vmalert's routing in cluster version ( #2844 )
...
* vmselect: cover special cases for vmalert's routing
* remove trailing `/` from requests
* redirect to vmalert's home page when `/vmalert` is requested.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* vmselect: fix review comments
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* Update app/vmselect/main.go
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-11 19:33:33 +03:00
Aliaksandr Valialkin
eab8ebbe11
all: make fmt
via the upcoming Go1.19
2022-07-11 19:23:25 +03:00
Aliaksandr Valialkin
c0af52228a
app/vmselect/netstorage: add benchmarks for mergeSortBlocks
...
This is a follow-up for 743ff84863
2022-07-11 12:53:46 +03:00
Aliaksandr Valialkin
d442ee4610
app/vmselect/netstorage: optimize mergeSortBlocks function
...
- Use binary search instead of linear scan when locating the run of smallest timestamps
in blocks with intersected time ranges. This should improve performance
when merging blocks with big number of samples
- Skip samples with duplicate timestamps. This should increase query performance
in cluster version of VictoriaMetrics with the enabled replication.
2022-07-09 00:35:38 +03:00
Roman Khavronenko
78da074ad3
metricsql: properly evaluate timezone_offset
over time interval ( #2842 )
...
* metricsql: properly evaluate `timezone_offset` over time interval
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* Update docs/CHANGELOG.md
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-08 14:04:40 +03:00
Aliaksandr Valialkin
decbf84e55
app/vmselect/vmui: follow-up after 0bf6841140
...
* Document the bugfix in docs/CHANGELOG.md
* Run `make vmui-update` for updating static js files for vmui, which are included into vmselect
2022-07-08 13:22:57 +03:00
Aliaksandr Valialkin
6bb4bcd30c
app/vmui: tune visual presentation of trace view
2022-07-06 14:16:24 +03:00
Aliaksandr Valialkin
eee384c9c9
app/vmselect: add -clusternative.tls*
options for mTLS setup in multi-level clusters
2022-07-06 13:48:21 +03:00
Aliaksandr Valialkin
195dccf678
app/vmselect: add ability to query vmselect
from another vmselect
2022-07-06 13:19:45 +03:00
Aliaksandr Valialkin
714958deab
all: follow-up after ed89106274
2022-07-06 12:48:52 +03:00
Aliaksandr Valialkin
cdd89d9cc2
app/vmselect: properly generate response for /api/v1/series
...
The response has been broken in 7d5d33fd71
2022-07-06 12:46:23 +03:00
Aliaksandr Valialkin
270e555f47
lib/vmselectapi: pass maxSuffixes arg to tagValueSuffixes RPC call
2022-07-06 12:46:22 +03:00
Aliaksandr Valialkin
f4df43f7cc
app/vmselect/netstorage: remove unused auth.Token arg
2022-07-06 12:46:21 +03:00
Aliaksandr Valialkin
78eeca6f0d
lib/vmselectapi: rename deleteMetrics to more correct deleteSeries
2022-07-06 12:46:21 +03:00
Roman Khavronenko
f51bc07d97
vmselect: allow proxying requests to vmalert ( #2833 )
...
The change allows to proxy requests with prefix `/vmalert`
to the vmalert component if `-vmalert.proxyURL` is set.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825
and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2831
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-06 10:47:51 +02:00
Aliaksandr Valialkin
10c63f5853
app/vmselect: make fmt
after f3ece83e67
2022-07-05 14:56:53 +03:00
Aliaksandr Valialkin
8516670582
app/vmselect/promql: properly calculate histogram_quantile over unexpected le
buckets
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819
2022-07-05 13:20:41 +03:00
Aliaksandr Valialkin
fa08220d27
app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2787
2022-06-30 22:40:26 +03:00
Aliaksandr Valialkin
4fb0f15322
all: readability improvements for query traces
...
- show dates in human-readable format, e.g. 2022-05-07, instead of a numeric value
- limit the maximum length of queries and filters shown in trace messages
2022-06-30 18:19:43 +03:00
Dmytro Kozlov
67753cc6f0
vmui: update render logic for nested component ( #2795 )
...
* vmui: update render logic for nested component, avoid rerender, remove local storage usage for tracing flag
* docs/url-examples.md: fix various documentation issues there
* docs: add Troubleshooting doc
This doc contains troubleshooting guides for typical problems with VictoriaMetrics.
* docs/Troubleshooting.md: add troubleshooting guide for cluster instability
* wip
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-30 15:48:22 +03:00
Aliaksandr Valialkin
daefb64f38
app/vmselect: expose additional histograms at /metrics
page, which may help get more insights for the query workload
...
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792
2022-06-28 20:18:31 +03:00
Aliaksandr Valialkin
9adff0b686
app/vmselect/promql: show the number of scanned samples in the query trace
2022-06-28 19:32:03 +03:00
Aliaksandr Valialkin
c21f5a508f
app/vmselect/prometheus: reduce the default value for -search.maxSeries from 100k to 30k
...
Production experience shows that 100k is too big for /api/v1/series .
It leads to increased CPU usage when Grafana queries /api/v1/series over VictoriaMetrics
with big number of time series during auto-completion and when modifying template variables.
2022-06-28 18:22:24 +03:00
Aliaksandr Valialkin
7d5d33fd71
lib/storage: return marshaled metric names from SearchMetricNames
...
Previously SearchMetricNames was returning unmarshaled metric names.
This wasn't great for vmstorage, which should spend additional CPU time
for marshaling the metric names before sending them to vmselect.
While at it, remove possible duplicate metric names, which could occur when
multiple samples for new time series are ingested via concurrent requests.
Also sort the metric names before returning them to the client.
This simplifies debugging of the returned metric names across repeated requests to /api/v1/series
2022-06-28 18:16:32 +03:00
Aliaksandr Valialkin
399d4c36ae
app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day
2022-06-28 12:55:20 +03:00
Aliaksandr Valialkin
a667d339be
app/vmselect/netstorage/netstorage.go: group metrics in order to improve readability a bit
2022-06-27 14:00:24 +03:00
Aliaksandr Valialkin
6386f117c8
all: show timeRange in traces in human-readable format instead of timestamps in milliseconds
2022-06-27 13:42:57 +03:00
Aliaksandr Valialkin
08de733924
app/vmselect/netstorage: assume the response is full if up to -replicationFactor-1 vmstorage nodes are unavailable
...
This is a follow-up for ee5c502446
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1767
2022-06-27 12:21:26 +03:00
Aliaksandr Valialkin
bc9d704ef4
app/vmselect/netstorage: remove Get prefix from netstorage functions
...
This makes these function names more consistent with the server side
2022-06-27 00:37:49 +03:00
hagen1778
e40d015e9a
vmselect: make vm_partial_results_total
consistent
...
Metrics `vm_partial_results_total` and `vm_requests_total` serving
the similar purpose, but contain inconsistent set of labels.
This change updates `vm_partial_results_total` labels to be consistent
with `vm_requests_total`.
The change breaks backward compatibility with assumption that
`vm_partial_results_total` wasn't widely used, since it is
not documented and absent in the alerts and dashboards.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-24 13:50:26 +02:00
Dmytro Kozlov
f28cbcc7b5
vmui: added query tracing ( #2748 )
...
* vmui: added query tracing
* vmui: updated ui
* vmui: update tracing logic, fix bugs, disable tracing by default
* vmui: use empty message as props
* vmui: fixed ui, added delete for each tacing data, show query in header
* vmui: added timelines
* vmui: speedup render
* vmui: use memo for sorting
* vmui: use Trace model, remove unused functions, simplify part of code
* vmui: update recursive logic
* vmui: fix set query to header
* vmui: code cleanup, remove unused code
* vmui: remove unused type, rename component
* wip
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-23 23:00:45 +03:00
Nikolay
ee5c502446
app/vmselect: fixes partial response with replicationFactor ( #2777 )
...
* app/vmselect: fixes partial response with replicationFactor
Allow partial response if it meets replicationFactor configured at vmselect
https://t.me/VictoriaMetrics_ru1/38490
* docs/CHANGELOG.md: document this change
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-23 20:17:24 +03:00
Aliaksandr Valialkin
7429dfbe9e
app/vmselect: add -search.setLookbackToStep
command-line flag for making the gap filling algorithm similar to InfluxDB data model
...
This option should override `-search.maxStalenessInterval` for most cases when users migrate from InfluxDB to VictoriaMetrics
2022-06-22 14:20:02 +03:00
Aliaksandr Valialkin
4f64da874d
app/vmselect: typo fix in the exported metric name: vm_http_request_total -> vm_http_requests_total
2022-06-22 13:18:03 +03:00
Aliaksandr Valialkin
dceca7e864
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package
...
This is a follow-up for fe2269b999
2022-06-21 20:27:30 +03:00
Aliaksandr Valialkin
fe2269b999
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package
...
This package already has the same name, so there is no need in explicit name
2022-06-21 20:24:28 +03:00
Aliaksandr Valialkin
b28c6febf9
app/{vminsert,vmselect}: add -vmstorageDialTimeout
command-line flag for tuning the maximum time needed for establishing connections to vmstorage
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2022-06-20 15:17:34 +03:00
Dmytro Kozlov
5bc13e2fe8
vmui: added focusLabel, enable cardinality app configuratior ( #2736 )
...
* vmui: added focusLabel, enable app configuratior
* vmui: set focusLabel if {labelName!=""}
* wip
* docs/CHANGELOG.md: mention about focusLabel feature in cardinality explorer
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2730
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-17 13:04:09 +03:00
Roman Khavronenko
cbe39bbb8d
vmselect: limit end
param max value by 2d in future ( #2729 )
...
* vmselect: limit `end` param max value by 2d in future
The change is applied only to service handlers like `/labels` or `/series`
and limits the `end` param by max value <= now() + 2 days. The same limit
is applied for the ingested data, so no reason to allow to request data
in future far than that.
The change is also needed for corner cases like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2669
where too high `end` value triggers inefficient global index search.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* docs/CHANGELOG.md: document the bugfix
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-16 20:51:40 +03:00
Aliaksandr Valialkin
da1d1e83df
app/{vmselect,vmstorage}: properly pass seriesCountByLabelName and seriesCountByFocusLabelValue entries from vmstorage to vmselect
2022-06-16 10:44:29 +03:00
Aliaksandr Valialkin
ee9954082f
app/vmselect/netstorage: properly aggregate seriesCountByLabelName and seriesCountByFocusLabelValue obtained from multiple vmselect nodes at /api/v1/status/tsdb
2022-06-15 16:48:40 +03:00
Aliaksandr Valialkin
45fa9d798d
app/vmselect: accept focusLabel
query arg at /api/v1/status/tsdb
2022-06-14 18:39:00 +03:00
Aliaksandr Valialkin
fb77843639
lib/storage: show top labels with the highest number of series in cardinality explorer
2022-06-14 16:34:13 +03:00
Dmytro Kozlov
ed9b4c6f6d
vmui: refactor Cardinality panel ( #2726 )
...
* vmui: refactor Cardinality panel
* vmui: change width of the search panel
* vmui: code cleanup
* vmui: code cleanup
* vmui: fixed vulnerability (npm audit fix)
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-14 14:40:44 +03:00
Yury Molodov
65107a4585
vmui: enhancements ( #2638 ) ( #2717 )
...
* feat: make datepicker to be set to last 30 min by default
* fix: correct spinner while loading data
* feat: change legend style
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-13 09:49:58 +03:00
Aliaksandr Valialkin
61e03f172b
app/vmselect: optimize /api/v1/labels
and /api/v1/label/.../values
handlers when match[]
query arg is passed to them
2022-06-12 14:06:24 +03:00
Aliaksandr Valialkin
4a94cd81ce
app/vmselect: add optional limit
query arg to /api/v1/labels
and /api/v1/label_values
endpoints
...
This arg allows limiting the number of sample values returned from these APIs
2022-06-10 10:24:07 +03:00
Aliaksandr Valialkin
1335698ba7
app/vmselect/prometheus: extract common code for obtaining common query args into getCommonParams() function
2022-06-09 20:33:36 +03:00
Aliaksandr Valialkin
cb39eada77
all: improve query tracing coverage for indexdb search
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
2022-06-09 20:04:02 +03:00
Aliaksandr Valialkin
ad44eadd94
app/vmselect/graphite: remove additional redundant Request.ParseForm() calls after 38c785b851
2022-06-09 13:22:23 +03:00
Aliaksandr Valialkin
7fe60dad10
app/vmselect: remove redundant calls to Request.ParseForm()
...
Request.ParseForm() is implicitly called by the first call to Request.FormValue()
2022-06-09 13:09:52 +03:00
Aliaksandr Valialkin
a9ea3fee38
lib/querytracer: make it easier to use by passing trace context message to New and NewChild
...
The context message can be extended by calling Donef.
If there is no need to extend the message, then just call Done.
2022-06-08 21:16:12 +03:00
Aliaksandr Valialkin
2b343d8bd0
app: properly collect and merge /api/v1/status/tsdb info from vmstorage nodes
...
The collection has been broken in f2754c3e90
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233
2022-06-08 19:26:09 +03:00
Dmytro Kozlov
f2754c3e90
Cardinality explorer ( #2625 )
...
* Cardinality explorer
* vmui, vmselect: updated field name, added description to spinner
* make vmui-update
* updated const name, make vmui-update
* lib/storage: changes calculation for totalSeries values
* added static files
* wip
* wip
* wip
* wip
* docs/CHANGELOG.md: document cardinality explorer feature
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233
Co-authored-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-08 18:54:27 +03:00
Aliaksandr Valialkin
68b6ddfb14
all: follow-up after 8edb390e21
...
- Remove unused js bloatware from /targets page. This strips down binary size by more than 100Kb
- Add /service-discovery page for API compatibility with Prometheus
- Properly load bootstrap.min.css from /prometheus/targets
- Serve static contents for /targets page from app/vminsert instead of app/vmselect, because /targets page is served from there
2022-06-07 01:05:53 +03:00
Dmytro Kozlov
ce8aade80e
lib/promscrape: adds service discovery visualization for /targets page( #2675 )
...
* lib/promscrape: updated template
* lib/promscrape: fixed click on unhealthy and all btns
* app/vmselect: jquery scripts into static folder
Co-authored-by: f41gh7 <nik@victoriametrics.com>
2022-06-04 01:11:23 +03:00
Aliaksandr Valialkin
c92bc5394f
app/vmselect/netstorage: properly read trace from vmstorage when it returns error message to vmselect
2022-06-01 14:35:00 +03:00
Aliaksandr Valialkin
afced37c0b
all: add initial support for query tracing
...
See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
2022-06-01 02:31:44 +03:00
Aliaksandr Valialkin
f0ab5ca61f
app/vmselect/vmui: make vmui-update
after 492a615a88
2022-05-26 09:43:30 +03:00
Yury Molodov
b9ef9ddbfd
vmui: import dashboards ( #2642 )
...
* fix: switch dashboards import to fetch
* make vmui-update
2022-05-26 09:43:29 +03:00
Aliaksandr Valialkin
4409b9f376
app/vmselect/vmui: make vmui-update
2022-05-20 14:56:25 +03:00
Yury Molodov
d13437afd0
vmui: setup predefined dashboards without build ( #2541 )
...
vmui: support predefined dashboards in json format
See https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmselect/vmui/dashboards
2022-05-20 13:46:19 +03:00
Aliaksandr Valialkin
68bfa05437
app/vmselect/promql: do not return values from label_value()
if the original time series has no values at the selected timestamps
2022-05-09 17:58:39 +03:00
Aliaksandr Valialkin
8babb4aebc
app/vmselect/vmui: make vmui-update
after 450d879eaa
2022-05-05 22:11:41 +03:00
Aliaksandr Valialkin
3fee369997
app/vmselect: follow-up after 8639e79d38
2022-05-04 23:41:00 +03:00
Nikolay
48041305a2
app/vmselect: adds proxy for rules API ( #2516 )
...
* app/vmselect: adds proxy for rules and alerts API
It allows to visualization for rules at grafana
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1739
* Update app/vmselect/main.go
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-05-04 10:12:15 +03:00
Dima Lazerka
b5fce4190c
Code cleanup ( #343 )
...
* Small code cleanup: remove Request from params
* Extract common params to all export handlers
* Renamed ExportParams -> exportParams
* wip
Co-authored-by: Dzmitry Lazerka <dlazerka@gmail.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-03 15:53:58 +03:00
Aliaksandr Valialkin
4fa3cd701c
docs/Cluster-VictoriaMetrics.md: typo fix: serparated -> separated
2022-05-03 15:20:11 +03:00
Aliaksandr Valialkin
75bd7f1539
app/vmui: fix up/down arrow keys on multi-line query after a186434b50
2022-05-03 10:51:08 +03:00
Aliaksandr Valialkin
361b08c30e
lib/storage: leave the last sample per each discrete interval during the deduplicaton
...
This aligns better with staleness logic in Prometheus - https://prometheus.io/docs/prometheus/latest/querying/basics/#staleness
2022-05-02 21:59:31 +03:00
Aliaksandr Valialkin
7ca32c21c8
app/vmui: execute query by pressing enter
in the same way as Prometheus does
...
Multi-line query can be entered via `shift-enter` in the query input field
2022-05-02 20:24:43 +03:00
Yury Molodov
6292185abf
vmui: support node v.18 ( #2529 )
...
* fix: add support vmui node18
* fix: remove @mui/styles (legacy styling solution)
* Update app/vmui/Dockerfile-build
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-02 20:24:42 +03:00
Nikolay
022b2875ad
app/vmselect: adds API /api/v1/status/buildinfo ( #2515 )
...
* app/vmselect: adds API /api/v1/status/buildinfo
it should fix an compability error with grafana 8.5 prometheus datasource
https://github.com/grafana/grafana/pull/46771
* Update main.go
2022-04-29 13:02:43 +03:00
Aliaksandr Valialkin
90f9ed260e
app/vmselect/vmui: make vmui-update
after da04e9d1de
2022-04-29 13:02:43 +03:00
Dima Lazerka
e6ee235707
Export "null" in jsonl instead of NaN ( #2518 )
...
* Export "null" in jsonl instead of NaN
The NaN appeared because of staleness markers that were added for compatibility. I think it's better to use json `null`, implemented here.
Also maybe it also makes sense to add a flag like `?skip-staleness-markers=true` to `/export`, to skip nulls at all?
* Update app/vmselect/prometheus/export.qtpl
* app/vmselect/prometheus/export.qtpl.go: `make quicktemplate-gen`
* docs/CHANGELOG.md: document the change
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-29 12:51:03 +03:00
Yury Molodov
dac203320a
vmui: expression alias ( #2495 )
...
* feat: add alias for queries
* docs: update docs for predefined dashboards
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-27 10:59:51 +03:00
Aliaksandr Valialkin
6eb1580158
app/vmselect/promql: add tlast_change_over_time(m[d])
function, which returns the timestamp for the last change of m
on the given lookbehind window d
2022-04-27 10:58:40 +03:00
Yury Molodov
eae6f68be2
vmui: add support relative time ( #2504 )
...
* feat: add support relative time
* app/vmselect: `make vmui-update`
* docs/CHANGELOG.md: document the change
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-26 15:46:47 +03:00
Aliaksandr Valialkin
dac24aa342
app/vmselect/promql: properly handle scalar default vector
, scalar if vector
and scalar ifnot vector
queries
...
Previously `vector` time series could be unexpectedly returned from such queries
2022-04-21 15:34:14 +03:00
Aliaksandr Valialkin
bc54ae9608
app/vmselect/promql: fix comparison to nan
...
The comparison to nan has been broken in d335cc886c
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150
2022-04-21 14:56:41 +03:00
Aliaksandr Valialkin
a5cfe5d13e
app/vmselect/promql: add drop_common_labels()
function
2022-04-21 14:20:36 +03:00
Aliaksandr Valialkin
7ca08945fd
app/vmselect/promql: fix q default b
where b
may have empty time series
2022-04-21 11:43:09 +03:00
Aliaksandr Valialkin
25b841c6ed
app/vmselect/promql: fix duplicate time series
error on joins against time series filtered by values
...
This should prevent from `duplicate time series` errors when executing the following query:
kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1)
where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series
2022-04-20 22:21:20 +03:00
Aliaksandr Valialkin
0ef7a05fc0
app/vmselect/promql: rename removeNaNs() to more clear removeEmptySeries()
2022-04-20 19:53:24 +03:00
Yury Molodov
60c4e0022a
fix: prevent graph hiding without data ( #2456 )
...
* fix: prevent graph hiding without data
* fix: add yaxis labels default
* app/vmselect: `make vmui-update`
* docs/CHANGELOG.md: document the change
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-16 17:16:17 +03:00
Aliaksandr Valialkin
3e2a4c07cd
app/vmui: further improvements for number display on graphs
...
This is a follow-up for c4d2cd8336
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2409
2022-04-12 16:02:37 +03:00
Aliaksandr Valialkin
4197c61ef8
app/vmselect: make vmui-update
2022-04-12 14:35:37 +03:00
Aliaksandr Valialkin
3605e1743b
app/vmselect/promql: allow calling InitRollupResultCache+StopRollupResultCache multiple times during tests
2022-04-11 12:32:23 +03:00
Aliaksandr Valialkin
a4a15a462b
app/vmselect/netstorage: bump RPC API versions for vmselect->vmstorage communications
...
This is a follow-up after b843f0e229
2022-04-08 12:36:04 +03:00
Aliaksandr Valialkin
feff0ef286
app/vmselect: make vmui-update
2022-04-01 12:55:53 +03:00
Yury Molodov
b2f0acaca4
vmui: grid support for predefined panels ( #2386 )
...
* update packages
* feat: add setting width for predefined panels
* docs: update doc by predefined dashboards
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-01 12:49:24 +03:00
Yury Molodov
7cbf19812d
vmui: predefined panels ( #2243 )
...
* feat: add basic components for predefined dashboards
* fix: change display alert
* feat: add autosize and unit for axes
* feat: add component for CircularProgress
* feat: change layout for predefined dashboards
* feat: add override step for predefined panels
* feat: add override step for predefined panels
* feat: change yaxis limits for predefined panels
* fix: rename flag for hide legend
* feat: add formatted panel description
* feat: add README.md for dashboard setup
* feat: validate dashboard settings
* feat: add unit for y-ticks
* fix: correct display error for dashboards
* fix: disable auto refresh after route change
* update package-lock.json
* fix: add basename for BrowserRouter
* fix: add dynamic basename for routing
* update packages
* feat: add a pre-defined dashboard "per-job resource usage"
* feat: display unit in the hover-tooltip
* fix: change routing and home layout
* fix: change axis width calc
* updated packages
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-03-26 13:05:21 +02:00
Aliaksandr Valialkin
b843f0e229
app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs
2022-03-26 11:28:14 +02:00
Aliaksandr Valialkin
84b3234f3d
docs/CHANGELOG.md: document a1e17e91f8
2022-03-21 15:37:03 +02:00
Aliaksandr Valialkin
85695faa94
app/vmselect/bufferedwriter: suppress trivial network errors, which can be generated by remote side
...
These errors include `broken pipe` and `reset by peer`.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2334
2022-03-18 19:27:33 +02:00
Aliaksandr Valialkin
95da5e52c4
app/vmselect/vmui: make vmui-update
2022-03-18 13:18:57 +02:00
Yury Molodov
235ef947ed
vmui: update packages ( #2264 )
...
* update package.json
* update package-lock.json
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-03-03 18:50:44 +02:00
Aliaksandr Valialkin
99ee1c13c0
app/vmselect/promql: reduce the maximum number of label values, which can be propagated from one side of the binary operation to another side of the binary operation from 10K to 1K
...
There are user reports that 10K unique values in a single label filter may lead to performance and memory usage issues
2022-02-24 04:03:28 +02:00
Aliaksandr Valialkin
244c23ea2c
lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes
...
This should reduce memory usage under high time series churn rate
2022-02-23 13:42:27 +02:00
Aliaksandr Valialkin
701eb0495b
app/vmselect: make vmui-update
2022-02-22 21:02:20 +02:00
Aliaksandr Valialkin
89ead3daca
app/vmselect/netstorage: report vmstorage errors to vmselect clients even if partial responses are allowed
...
If a vmstorage is reachable and returns an application-level error to vmselect,
then such error must be returned to the caller even if partial responses are allowed,
since it usually means cluster mis-configuration.
Partial responses may be returned only if some vmstorage nodes are temporarily unavailable.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1941
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/678
2022-02-21 21:17:05 +02:00
Yury Molodov
cdd632985f
vmui: change logic autocomplete ( #2196 )
...
* fix: change autocomplete display logic
* fix: change z-index for labels of input fields
* fix: change autocomplete display logic
* wip
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-21 15:44:54 +02:00
Aliaksandr Valialkin
c723ac6a8f
app/vmui: small fixes
...
* Remove unneeded dependency on `numeral` package
* Properly parse numbers obtained from /api/v1/query_range according to
https://prometheus.io/docs/prometheus/latest/querying/api/#expression-query-result-formats
* Optimize updating processing the received data from /api/v1/query_range
* Make smoother zoom on `ctrl+scroll`
* Reduce the number of points received from /api/v1/query_range by 2x in order to reduce load on backend
2022-02-14 16:31:31 +02:00
Aliaksandr Valialkin
989668beba
app/vmselect/promql: return at most one time series from absent_over_time()
in the same way as Prometheus does
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130
2022-02-12 15:46:43 +02:00
Roman Khavronenko
791cad8c2e
lib/promscrape: support prometheus-like duration in scrape configs ( #2169 )
...
* lib/promscrape: support prometheus-like duration in scrape configs
The change allows to specify duration values like `1d`, `1w`
for fields `scrape_interval`, `scrape_timeout`, etc.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/817#issuecomment-1033384766
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* lib/blockcache: make linter happy
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* lib/promscrape: support prometheus-like duration in scrape configs
* add support for extra fields `scrape_align_interval` and `scrape_offset`;
* support Prometheus duration parsing for `__scrape_interval__`
and `__scrape_duration__` labels;
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* wip
* wip
* docs/CHANGELOG.md: document the feature
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-11 16:17:51 +02:00
Aliaksandr Valialkin
d1c17fe385
app/vmselect/vmui: make vmui-update
2022-02-08 17:29:35 +02:00
Aliaksandr Valialkin
1173964d8d
app/vmselect/promql: clarify comments on why the right side of if
and and
operators are executed at first
2022-02-03 00:26:42 +02:00
Aliaksandr Valialkin
10367d7e4c
app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values
...
Such filters may slow down time series search, so just skip them.
This is a follow-up for e7f1ceeb84
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827
2022-02-02 23:42:25 +02:00
Aliaksandr Valialkin
5f266370c5
all: follow-up after 4bdd10ab90
...
Properly use new bytesutil.Resize* functions
2022-02-01 17:49:28 +02:00
Aliaksandr Valialkin
13c2692ca1
app/vmselect/promql: properly handle foo or bar
queries
...
Such queries may miss `bar` results after the commit e7f1ceeb84
because common label filters from `foo` could be mistakenly applied to `bar`.
2022-02-01 17:43:10 +02:00
Aliaksandr Valialkin
02b2bfcff3
lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations
...
Follow-up for f4989edd96
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-02-01 11:20:20 +02:00
Aliaksandr Valialkin
784cdcb080
app/vmselect/promql: check for binary operation in case-insensitive manner when deciding which side of the operation to perform the first
...
PromQL and MetricsQL operators are case-insensitive
2022-01-31 22:07:35 +02:00
Aliaksandr Valialkin
04d6596298
app/vmselect/promql: optimize queries, which join on _info
metrics.
...
Automatically add common filters from one side of binary operation
to the other side before sending the query to storage subsystem.
See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/
and https://www.robustperception.io/exposing-the-software-version-to-prometheus
2022-01-31 20:25:15 +02:00
Aliaksandr Valialkin
d95d945046
app/vmselect/vmui: make vmui-update
2022-01-31 12:42:21 +02:00
Aliaksandr Valialkin
a3eafd2e7f
app/vmselect/vmui: make vmui-update
after e3995572bb
2022-01-25 17:52:37 +02:00
Aliaksandr Valialkin
6232eaa938
lib/bytesutil: split Resize() into ResizeNoCopy() and ResizeWithCopy() functions
...
Previously bytesutil.Resize() was copying the original byte slice contents to a newly allocated slice.
This wasted CPU cycles and memory bandwidth in some places, where the original slice contents wasn't needed
after slize resizing. Switch such places to bytesutil.ResizeNoCopy().
Rename the original bytesutil.Resize() function to bytesutil.ResizeWithCopy() for the sake of improved readability.
Additionally, allocate new slice with `make()` instead of `append()`. This guarantees that the capacity of the allocated slice
exactly matches the requested size. The `append()` could return a slice with bigger capacity as an optimization for further `append()` calls.
This could result in excess memory usage when the returned byte slice was cached (for instance, in lib/blockcache).
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-01-25 15:28:42 +02:00
Aliaksandr Valialkin
b5abea0e36
app/vmselect/vmui: make vmui-update
2022-01-24 12:25:00 +02:00
Yury Molodov
196bef8348
vmui: fixed display type switching ( #2088 )
...
* fix: correct switch display type
* docs/CHANGELOG.md: document the bugfix
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-21 16:57:14 +02:00
Yury Molodov
034012c80f
vmui: fix time range selector ( #2085 )
...
* fix: add date validate for time range
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document the bugfix
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-21 12:03:41 +02:00
Aliaksandr Valialkin
e8900c90a8
app/vmui: fix last 6 months
time range picker
...
Previously it was incorrectly selecting 6 minutes instead of 6 months.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1960
2022-01-18 23:28:49 +02:00
Aliaksandr Valialkin
f4a9ae28f3
app/vmui: properly calculate graph range for y axis
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2037
2022-01-18 22:23:40 +02:00
Aliaksandr Valialkin
f31776d0e4
app/vmui: reduce the refresh interval during graph scrolling/zooming from 1 second to 300 milliseconds
...
One second feels too laggy, so let's reduce the refresh interval to 300 milliseconds.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064
2022-01-18 22:23:40 +02:00
Yury Molodov
1d19303f35
fix: remove buffer period ( #2078 )
...
* fix: remove buffer period
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document the implemented feature
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 22:23:40 +02:00
Yury Molodov
5b286a4728
vmui: correct url encoding ( #2067 )
...
* fix: correct encode multi-line queries
* fix: change autocomplete for correct arrows work
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document the bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2039
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 21:32:30 +02:00
Yury Molodov
d4ccd48faa
vmui: correct calc axes limits ( #2058 )
...
* fix: correct calc axes limits
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 21:27:26 +02:00
Yury Molodov
e548fc9021
vmui: change layout ( #2054 )
...
* fix: change query reset
* feat: replace @codemirror to text field
* feat: switch to Preact from React
* fix: optimize mui imports
* feat: move time selector to Header
* checkout
* fix: remove unused vars
* update package-lock.json
* fix: correct styles
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 12:47:01 +02:00
Aliaksandr Valialkin
cc76174387
app/vmselect/promql: properly keep metric names when optimized path is used for aggregate function calculations
...
For example, `sum(rate(...) keep_metric_names) by (__name__)` didn't leave the original metric name because of this issue.
This is a follup-up for 1bdc71d917
Udates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
2022-01-17 15:33:39 +02:00
Aliaksandr Valialkin
5a1e4a140f
app/vmselect/promql: implement keep_metric_names
modifier for transform and rollup functions
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
2022-01-14 04:15:35 +02:00
Aliaksandr Valialkin
53c9a81f2b
app/vmselect/promql: add stale_samples_over_time()
function
2022-01-14 01:49:24 +02:00
Aliaksandr Valialkin
0580a58feb
app/vmselect/promql: add support for @
modifier
...
Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier
Extend the support with the following features:
* Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()`
* Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)`
returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]`
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348
2022-01-13 22:15:19 +02:00
Aliaksandr Valialkin
7d635a2311
app/vmselect/promql: fix limit_offset() test
...
The test has been broken in addae7fc6a
2022-01-13 16:58:47 +02:00
Yury Molodov
7ee09286ef
vmui: switching to Preact ( #2053 )
...
* feat: replace @codemirror to text field
* feat: switch to Preact from React
* fix: optimize mui imports
* fix: remove unused vars
* update package-lock.json
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-11 10:34:01 +02:00
Aliaksandr Valialkin
cf5553d18e
app/vmselect/vmui: run make vmui-update
after 89ff7b2465
2022-01-03 21:06:34 +02:00
Yury Molodov
85dc41007b
vmui: legend fixes ( #1995 )
...
* feat: add a reset query by clicking the logo
* feat: add sequence number for query fields
* feat: invert behavior on the graph's legend
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-23 12:15:21 +02:00
Roman Khavronenko
d1c474c5b3
vmselect: update /query_exemplars placeholder ( #2000 )
...
Grafana expects `data` in response to be a slice and logs an err
if it is not:
```
err="[]v1.ExemplarQueryResult: decode slice: expect [ or n, but found , error found in #0 byte of ...||..., bigger context ...||..."
```
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1999
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-12-23 12:08:40 +02:00
Yury Molodov
0952caca24
vmui: add custom start range ( #1989 )
...
* feat: add custom start range
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-21 20:22:48 +02:00
Yury Molodov
4c9e563c89
vmui: graph fixes ( #1982 )
...
* fix: remove disabling custom step when zooming
* feat: add a dynamic calc of the width of the graph
* fix: add validate y-axis limits
* fix: correct axis limits for value 0
* fix: change logic create time series
* fix: change types for tooltip
* fix: correct points on the line
* fix: change the logic for set graph width
* fix: stop checking the period when auto-refresh is enabled
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-20 17:37:45 +02:00
Aliaksandr Valialkin
b393a16a25
app/vmselect/vmui: make vmui-update
2021-12-20 13:50:55 +02:00
Aliaksandr Valialkin
3396f08874
app/vmselect/promql: follow-up after 177e345d8a
...
* Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions.
* Simplify their implementation
* Mention these functions in docs/CHANGELOG.md
2021-12-20 13:15:06 +02:00
匠心零度
cc4725843b
add Prometheus semantics function :changes_prometheus、delta_prometheus、increase_prometheus ( #1972 )
...
Co-authored-by: lirenzuo <lirenzuo@shein.com>
2021-12-20 13:15:05 +02:00
Aliaksandr Valialkin
c2b0a6109b
app/vmselect/prometheus: typo fix after bf0e3ad2171219b7883b149999c4dc0ac15b9797
2021-12-17 11:52:42 +02:00
Aliaksandr Valialkin
7c3239966e
app/vmselect/vmui: make vmui-update
2021-12-17 11:52:42 +02:00
Aliaksandr Valialkin
d9f075622b
app/vmselect: de-duplicate data exported via /api/v1/export/csv
by default
...
Previously the exported data wasn't de-duplicated.
Now it is possible to export the raw data without deduplication
by passing reduce_mem_usage=1 query arg to /api/v1/export/csv
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1837
2021-12-17 11:52:42 +02:00
Aliaksandr Valialkin
c170841951
app/vmselect/promql: add timestamp_with_name(m[d])
function
...
This function works the same as `timestamp()`, but doesn't remove source time series names.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388
2021-12-15 23:38:09 +02:00
Yury Molodov
603be150c4
vmui: introduce application mode ( #1949 )
...
* feat: add a label for the Query field
* fix: change zoom position
* fix: add description and error code to alerts
* fix: correct logic query history
* fix: correct update query history
* feat: add custom step
* update package-lock.json
* feat: introduce application mode
* build vmui
* Revert "build vmui"
This reverts commit c0e2415550
.
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-15 21:34:04 +02:00
Aliaksandr Valialkin
bc3923111b
lib/storage: return dedup interval in milliseconds from GetDedupInterval()
...
This removes duplicate .Milliseconds() calls after GetDedupInterval() calls.
2021-12-15 13:27:27 +02:00
Aliaksandr Valialkin
cdfe854c9b
lib/storage: explicitly pass dedupInterval to DeduplicateSamples() and deduplicateSamplesDuringMerge()
...
This improves the code readability and debuggability, since the output of these functions
stops depending on global state.
2021-12-14 20:52:29 +02:00
Yury Molodov
b3389ce26b
vmui: custom step ( #1942 )
...
* feat: add a label for the Query field
* fix: change zoom position
* fix: add description and error code to alerts
* fix: correct logic query history
* fix: correct update query history
* feat: add custom step
* update package-lock.json
* docs: document that VMUI now supports overriding of `step` query arg, which is passed to `/api/v1/query_range`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-14 14:52:44 +02:00
Yury Molodov
28655e8eb8
vmui: minor fixes ( #1936 )
...
* feat: add a label for the Query field
* fix: change zoom position
* fix: add description and error code to alerts
* fix: correct logic query history
* fix: correct update query history
* app/vmselect/vmui: `make vmui-update`
* docs/CHANGELOG.md: document bugfixes
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-13 13:44:32 +02:00
Aliaksandr Valialkin
3d86e6353b
app/vmselect/vmui: make vmui-update
2021-12-13 13:33:35 +02:00
Yury Molodov
6367edae1d
vmui: add a label for the Query field ( #1923 )
...
* feat: add a label for the Query field
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-12 19:10:56 +02:00
Aliaksandr Valialkin
addae7fc6a
app/vmselect/promql: preserve the order of time series passed to limit_offset()
function
...
Previously time series passed to `limit_offset()` were shuffled according to hash for their labels.
This was unexpected behaviour for most users.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1920 and https://github.com/VictoriaMetrics/VictoriaMetrics/issues/951
2021-12-12 18:07:15 +02:00
Yury Molodov
39e29a50f1
vmui: multiple queries ( #1916 )
...
* feat: change duration by "enter"
* fix: optimize data processing for chart
* feat: set minimum step to 1ms
* update dependencies
* feat: remove save the last query to local storage
* fix: handle an error in a table with subqueries
* feat: store display type in URL
* Revert "feat: store display type in URL"
This reverts commit ccc242c69a
.
* feat: store display type in URL
* refactor: move the time setting to a folder
* refactor: move the query configurator to a folder
* refactor: move the auth settings to a folder
* feat: improve styles
* feat: add multi query
* update package-lock
* feat: add display multiple queries
* feat: add limits for multiple queries
* update dependencies
* feat: add history for multiple queries
* feat: add line type to legend
* feat: change style for switch
* feat: change the logic for axes limits for multiple queries
* update package-lock.json
* update dependencies
* feat: add the filter to legend
* wip
* lib/httpserver: add missing 127.0.0.1 hostname to the logged address for http and pprof server if the address starts with ':'
This allows copy-pasting the url to http server from logs.
* lib/httpserver: add missing 127.0.0.1 hostname to the logged address for http and pprof server if the address starts with ':'
This allows copy-pasting the url to http server from logs.
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-08 16:43:18 +02:00
Aliaksandr Valialkin
54d4a1c959
app/vmselect: accept optional extra_filters[]
query args for all the supported Prometheus querying APIs
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863
2021-12-06 17:33:49 +02:00
Aliaksandr Valialkin
df810cfc9a
app/vmselect/vmui: make vmui-update
2021-12-06 10:18:03 +02:00
Aliaksandr Valialkin
120fedf1ac
app/vmselect/vmui: make vmui-update
2021-11-30 01:38:59 +02:00
Yury Molodov
500945499f
vmui: optimize render ( #1852 )
...
* feat: change duration by "enter"
* fix: optimize data processing for chart
* feat: set minimum step to 1ms
* update dependencies
* update package-lock
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-11-30 00:57:35 +02:00
Aliaksandr Valialkin
648d788a45
app/vmselect/vmui: make vmui-update
2021-11-29 21:58:11 +02:00
Aliaksandr Valialkin
d5ae9e9619
app/vmselect/vmui: make vmui-update
2021-11-29 12:23:12 +02:00
Michael Fuller
8a94dd36d1
vmselect: in promql evaluation, return bytes requested when rollup memory limiter is unable to satisfy the request ( #1838 )
...
Co-authored-by: Michael Fuller <mfuller@digitalocean.com>
2021-11-29 12:14:55 +02:00
Aliaksandr Valialkin
84decc4254
app/vmselect/promql: consistently return zero from deriv(const)
2021-11-17 18:01:58 +02:00
Aliaksandr Valialkin
aa6d11a33f
app/vmselect/promql: add now()
function, which returns the current timestamp as a floating-point value in seconds
2021-11-17 16:42:11 +02:00
Aliaksandr Valialkin
1eb756fd11
app/vmselect/promql: arrange function names in the code in alphabetical order
...
This should simplify code maintenance in the future
2021-11-14 13:54:43 +02:00
Aliaksandr Valialkin
82cf9f26db
app/vmselect/promql: prevent from incorrect calculations for deriv()
over multiple samples with identical timestamps
2021-11-12 13:53:30 +02:00
Aliaksandr Valialkin
4fb19fe34b
all: consistently return application/json
content-type without charset=utf-8
...
The `application/json` content-type has utf-8 encoding by default.
See https://stackoverflow.com/questions/9254891/what-does-content-type-application-json-charset-utf-8-really-mean
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897
2021-11-09 18:07:22 +02:00
Aliaksandr Valialkin
31486f2244
app/vmselect/promql: properly return durations smaller than one second from duration_over_time() function
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780
2021-11-09 11:42:43 +02:00
Yury Molodov
89e0f66bfc
vmui: migration MUI Core v4 to v5 ( #1795 )
...
* migration MUI Core v4 to v5
* app/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-11-08 15:46:57 +02:00
Aliaksandr Valialkin
3a1ac218ba
app/vmselect/promql: add duration_over_time(m[d], max_interval)
function
...
This function calculates the actual lifetime of the time series on the given lookbehdind window `d`
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780
2021-11-08 13:14:57 +02:00
Yury Molodov
654686fd40
vmui: fix graph reset ( #1788 )
...
* feat: add query history
* fix: change detect keyUp for nav query history
* feat: set default query history
* feat: change graph legend
* update dependencies
* update codemirror version
* fix: correct update period time after zoom/pan
* fix: optimize data processing for the graph
* fix: eliminate memory leaks related to mouse events
* fix: correct display of straight line
* Merge branch 'master' into vmui-fix-reset-graph
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-11-05 20:58:49 +02:00
Aliaksandr Valialkin
be9c7713d7
app/vmselect: make vmui-update
2021-11-04 12:16:55 +02:00
Aliaksandr Valialkin
a102fca4ac
app/vmselect/promql: add limit_offset(limit, offset, q)
function, which can be used for paging over big number of time series
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1778
2021-11-03 16:03:10 +02:00
Aliaksandr Valialkin
43bf0333eb
app/vmselect/promql: add label_graphite_group()
function for extracting groups from Graphite metric names
2021-11-03 13:20:34 +02:00
Yury Molodov
a16c88095c
vmui: correct migration material-ui ( #1758 )
...
* migration material-ui
* fix: rollback popover
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-27 20:43:24 +03:00
Aliaksandr Valialkin
99784b21c1
all: fix build issues and tests for Apple M1
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1653
2021-10-27 15:07:19 +03:00
Yury Molodov
e7591577f0
vmui: update dependencies ( #1754 )
...
* update dependencies
* update codemirror version
* app/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-26 19:32:31 +03:00
Aliaksandr Valialkin
d8ce86d685
app/vmselect/promql: reduce the precision from 15 significant digits to 13 significant digits when comparing float64 results in tests
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1738
2021-10-24 13:32:25 +03:00
Yury Molodov
3ccc37a98b
vmui: query history ( #1732 )
...
* feat: add query history
* fix: change detect keyUp for nav query history
* feat: set default query history
* app/vmselect/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-22 12:29:27 +03:00
Yury Molodov
744f6a98eb
vmui: features ( #1711 )
...
* feat: initial uPlot graph
* feat: add zoom/pan for graph
* fix: add zoom by ctrl/mac
* fix: remove unused code
* feat: add toggle cache for fetch
* feat: add fix y-axis limits
* fix: stop point events while panning
* fix: change getting cursor position when scaling
* feat: add cursor tooltip to graph
* fix: uninstall chart.js
* fix: change link for create an issue
* fix: set default cache value to true
* app/vmalert: follow-up after 0e2486df56
* docs/CHANGELOG.md: document 5416e18007
* app/vmui: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-18 15:19:01 +03:00
Aliaksandr Valialkin
ed994873fd
app/vmselect/promql: randomize the static selection of time series returned from limitk()
...
Sort series by a hash calculated from the series labels. This should guarantee "random" selection of the returned time series.
Previously the selection could be biased, since time series were sorted alphabetically by label names and label values.
2021-10-16 21:16:34 +03:00
Aliaksandr Valialkin
da0adf446d
app/vmselect/promql: typo fix in comment: didsn't -> didn't
2021-10-16 13:00:10 +03:00
Aliaksandr Valialkin
48c3668b3d
app/vmselect/promql: add atan2
binary operator, which is going to be added in Prometheus 2.31
...
See https://github.com/prometheus/prometheus/pull/9248
2021-10-11 21:17:23 +03:00
Aliaksandr Valialkin
b3d28f3872
app/vmselect/promql: add missing trigonometric functions, which are going to be added in Prometheus 2.31
...
See https://github.com/prometheus/prometheus/issues/9233
2021-10-11 21:17:21 +03:00
Aliaksandr Valialkin
f1e1d20ac4
app/vmselect/promql: consistently return the same set of time series from limitk()
function
...
This is the expected behaviour by most users.
2021-10-08 19:55:29 +03:00
Aliaksandr Valialkin
cf87050d83
app/vmui: follow-up after 7bfb44113e
...
* Run `vmui-update`
* Document the changes in README.md and CHANGELOG.md
2021-10-08 15:11:09 +03:00
Aliaksandr Valialkin
cbf01f7384
app/vmselect/promql: substitute rollupFuncsCannotAdjustWindow with rollupFuncsCanAdjustWindow
...
The list of functions, which can adjust lookbehind window is more limited than the rest of functions,
so it is better from maintainability and readability PoV using the allowlist instead of blocklist.
2021-10-07 13:19:22 +03:00
Aliaksandr Valialkin
7e6ec0bbe9
app/vmselect/promql: return back the behaviour for deriv()
function when the lookbehind window doesnt contain enough points
...
It is expected that the `deriv(m[d])` returns non-empty value if the lookbehind window `d`
contains less than 2 samples in the same way as `rate()` does.
This is a follow-up after 3e084be06b
.
2021-10-07 12:55:58 +03:00
Roman Khavronenko
0a8804d6aa
app/vmselect: make predict_linear
and deriv
compatible with Prometheus ( #1681 )
...
Previously, `predict_linear` returned slightly different results comparing
to Prometheus. The change makes linear regression algorithm compatible
with Prometheus.
`deriv` was excluded from the list of functions which can adjust the time
window for the same reasons.
2021-10-07 12:55:56 +03:00
Aliaksandr Valialkin
ec6eb03d65
app/vmselect/promql: add topk_last
and bottomk_last
functions
2021-09-30 13:23:27 +03:00
Roman Khavronenko
95ddfda894
app/vmselect: fix binary comparison func ( #1667 )
...
The fix makes the binary comparison func to check for NaNs
before executing the actual comparison. This prevents VM
to return values for non-existing samples for expressions
which contain bool comparisons. Please see added test
for example.
2021-09-30 13:04:10 +03:00
Roman Khavronenko
57eb5ef194
app/vmselect: fix testRowsEqual
func NaN checks ( #1666 )
...
It appeared, that `testRowsEqual` NaN comparison was incorrect.
The fix caused some tests to fail. Please see the change and
tests updated.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-30 13:04:08 +03:00
Yury Molodov
a6037a8ce8
vmui: fixed bug with time range (time zone) ( #1661 )
...
* fix: set date in query string in utc format
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-29 01:01:27 +03:00
Roman Khavronenko
444a8882f4
app/vmselect: disable time-window adjustment for min/max_over_time
funcs ( #1658 )
...
Adjustment results into discrepancy between Prometheus and VM on time windows
smaller than scrape interval.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-29 00:43:54 +03:00
Roman Khavronenko
f46f847531
app/vmselect: always return zero for stddev
func if there is only one value ( #1659 )
...
The fix will always return zero if received set of items consists of one
element only, which also means no deviation.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-29 00:42:19 +03:00
Roman Khavronenko
72468c9d75
app/vmselect: return NaN instead of 0 for empty value sets ( #1660 )
...
The change affects `count/stddev/stdvar_over_time` funcs and makes
them to return NaN instead of zero when there is no datapoints
in a time window.
This is needed for improving compatibility with Prometheus.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-29 00:38:24 +03:00
Roman Khavronenko
c8ac8f44b2
app/vmselect: rm quantile_over_time
fast-path optimisations ( #1662 )
...
The removed fast path optimisations weren't consistent with
`quantile` function behavior and results into discrepancy.
Specifically, results didn't match in cases when:
* 0 < phi > 1;
* values contain only one element.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-29 00:35:59 +03:00
Yury Molodov
80df31b2ee
vmui: use Chart.js as default engine for graph ( #1634 )
...
* feat: add Plotly as default engine for graph
* fix: remove unused components
* feat: use Chart.js as default engine graph
* fix: correct styles for loader
* feat: add zoom/pan for chart
* feat: add height for chart
* fix: remove unused code
* fix: remove empty units from duration
* fix: change debounce for pan to 500ms
* fix: add utility for plugins register globally
* fix: optimize render graph
* feat: add buffer data for zoom
* fix: add limits for zoom in/out
* fix: change update data while zooming
* app/vmselect: `make vmui-update`
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-27 22:27:49 +03:00
Aliaksandr Valialkin
4b3951fd86
app/vmselect/promql: add rollup_scrape_interval(m[d])
function
...
It calculates the min, max and avg scrape intervals for m over the given lookbehind window d
2021-09-27 19:21:55 +03:00
Aliaksandr Valialkin
eff31c10ec
app/vmselect/promql: follow-up after 526dd93b32
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612
2021-09-27 18:57:03 +03:00
Roman Khavronenko
6061464d80
app/vmselect: quantile
func compatiblity with Prometheus ( #1646 )
...
* app/vmselect: `quantile` func compatiblity with Prometheus
The `quantile` func was previously calculated by https://github.com/valyala/histogram
package. The result of such calculation was always the closest real value to
requested quantile. While in Prometheus implementation interpolation is used.
Such difference may result into discrepancy in output between Prometheus and
VictoriaMetrics.
This commit adds a Prometheus-like `quantile` function. It also used by other
functions which depend on it, such as `quantiles`, `quantile_over_time`, `median` etc.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* app/vmselect: `quantile` review fixes
* quantile functions were split into multiple to provide
different API for already sorted data;
* float64sPool is used for reducing allocations. Items in pool may have
different sizes, but defining a new pool was complicates due to name collisions;
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-27 18:57:02 +03:00
Aliaksandr Valialkin
a746fe8778
app/vmselect/promql: follow-up after 57b3320478
2021-09-24 01:25:04 +03:00
Roman Khavronenko
7ac0559d4f
app/vmselect: make sorting for query result similar to Prometheus ( #1647 )
...
* app/vmselect: make sorting for query result similar to Prometheus
Updated sorting allows to get the order of series in result similar or equal
to what Prometheus returns.
The change is needed for compatibility reasons.
* Update app/vmselect/promql/exec_test.go
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-24 01:25:03 +03:00
Aliaksandr Valialkin
1115c2f235
app/vmselect/promql: align the behavior of or
, and
and unless
operators with on (labels)
modifier to Prometheus
...
Previously VictoriaMetrics could return unexpected result of the right-hand side operand
had multiple time series with the given set of labels mentioned in `on(labels)`.
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1643
2021-09-24 00:56:35 +03:00
Aliaksandr Valialkin
3f1104ea74
app/vmselect: fix accessing /graphite/*
endpoints
2021-09-21 14:06:18 +03:00
n4mine
a5fcc9e356
fix: typo, dddresses -> addresses ( #1630 )
2021-09-20 14:27:34 +03:00
Aliaksandr Valialkin
76c650bbda
app/vmselect/promql: add quantiles_over_time("phiLabel", phi1, ..., phiN, m[d])
function for calculating multiple quantiles at once
2021-09-17 23:35:04 +03:00
Aliaksandr Valialkin
81d24045c2
app/vmselect/promql: add histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) function
...
This function calculates multiple quantiles over the given buckets at once
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573
2021-09-17 13:39:06 +03:00
Aliaksandr Valialkin
4ed7548ced
app/vmselect/promql: optimize quantiles() calculation
...
Calculate quantiles in one go instead of calculating each quantile individually
2021-09-17 12:38:21 +03:00
f41gh7
348f705d95
follow-up after e2f99214f3e518f79c1c96ff918820a69d350ba1
2021-09-16 13:56:28 +03:00
Roman Khavronenko
05bf6cca82
vmselect: add stub for graphite/functions
requests
...
The request `graphite/functions` is sent by Grafana datasource
and results into error logs in vmselect.
2021-09-16 13:51:37 +03:00
Aliaksandr Valialkin
3f8de4f82f
app/vmselect/promql: add mad(q)
and outliers_mad(tolerance, q)
functions to MetricsQL
2021-09-16 13:35:38 +03:00
Aliaksandr Valialkin
a4eaf1a0a1
app/vmselect/prometheus: make more clear log messages for errors during sending data to remote clients
2021-09-16 13:35:37 +03:00
Aliaksandr Valialkin
7ad54041fe
app/{vminsert,vmselect}: automatically add missing port in -storageNode
lists passed to vminsert
and vmselect
...
This should simplify manual setup of the cluster according to https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#cluster-setup
2021-09-15 18:04:30 +03:00
Aliaksandr Valialkin
794d941125
app/vmui: follow-up after 777ff75874
...
The commit contains the following changes:
- Show vmui when requesting /graph page in order to be compatible with Prometheus datasource in Grafana.
- Properly encode query args at vmui url.
- Set the number of points on the graph to the number of horizontal pixels divided by 2. Previously it was hardcoded to 30.
- Do not save server url to persistent storage at browser, since it should be always obtained from the url.
- Run `make vmui-update` for updating vmui embedded into VictoriaMetrics.
2021-09-15 17:41:00 +03:00
Aliaksandr Valialkin
19516aedb5
app/vmselect/promql: use Prometheus-compatible label value formatting for count_values
function
2021-09-13 13:48:18 +03:00
Aliaksandr Valialkin
9b07cb2988
app/vmselect/promql: keep metric name in rollup_candlestick
results, since they don't change the original series meaning
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1600
2021-09-09 19:24:10 +03:00
Aliaksandr Valialkin
3788d4f4eb
app/vmselect: show useful endpoints when requested /select/<accountID>/
page
2021-08-29 12:05:14 +03:00
Aliaksandr Valialkin
39bb6bdd79
app/vmselect/promql: add quantile("phiLabel", phi1, ..., phiN, q)
aggregate function to MetricsQL
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573
2021-08-27 18:40:25 +03:00
Aliaksandr Valialkin
335de30083
app/vmselect/promql: make fmt
after 0078486ea7
2021-08-23 23:05:34 +03:00
Aliaksandr Valialkin
40b06e84f8
app/vmselect/promql: rename sign()
function to sgn()
in order to be consistent with Prometheus
...
See https://github.com/prometheus/prometheus/pull/8457 for details.
2021-08-23 11:46:29 +03:00
Aliaksandr Valialkin
91534057a3
app/vmselect/prometheus: do not extend [d]
to the detected interval between samples for first_over_time(m[d])
...
This is for the sake of consistency with similar change for the last_over_time(m[d]) at a724229b5d
2021-08-21 19:56:56 +03:00
Roman Khavronenko
1ccb77904b
vmselect: update vm_request_duration_seconds
value when request fails ( #1537 )
...
Before, metric `vm_request_duration_seconds` was update only on successful
attempts which could be misleading. For example, timeout errors on netstorage
request may be not accounted in the metric and won't be visible on dashboards.
Using `defer` statement to update the metric after query arguments validation
may improve the situation.
2021-08-19 14:07:00 +03:00
Aliaksandr Valialkin
ee1f3414d1
app/vmselect/promql: do not override [d]
at last_over_time(m[d])
if [d]
is smaller than scrape_interval
...
Since most users do not expect the overriding of explicitly set `[d]`.
2021-08-19 10:33:10 +03:00
Aliaksandr Valialkin
5d92fafc40
app/vmselect: add -search.noStaleMarkers
command-line flag for disabling stale markers handling in queries
...
This option allows reducing CPU usage a bit when VictoriaMetrics is used
for collecting and processing non-Prometheus data. For example, InfluxDB line protocol, Graphite, OpenTSDB, CSV, etc.
2021-08-18 13:58:06 +03:00
Aliaksandr Valialkin
49886ecbc8
app/vmselect/promql: add bitmap_and(), bitmap_or() and bitmap_xor() functions to MetricsQL
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1541
2021-08-17 13:22:15 +03:00
Aliaksandr Valialkin
38065bec7b
app/vmselect/promql: move common condition to dropStaleNaNs in order to improve code maintainability
2021-08-17 11:00:58 +03:00
Aliaksandr Valialkin
5420c3d967
app/vmselect/promql: drop staleness marks before calling rollupConfig.Do
...
This allows dropping staleness marks only once and then calculate multiple rollup functions on the result.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
2021-08-15 13:22:26 +03:00
Aliaksandr Valialkin
6c4c54eaad
Revert "app/vmselect/promql: properly handle Prometheus staleness marks in removeCounterResets functions"
...
This reverts commit 94dfcb6747a3b29a11d14e71bea21a2312bb6346.
It is better to remove staleness marks (decimal.StaleNaN) before calling rollupConfig.Do, e.g. in preFunc
2021-08-15 13:22:24 +03:00
Aliaksandr Valialkin
af4a306d7b
app/vmselect/promql: properly handle Prometheus staleness marks in removeCounterResets functions
...
Prometheus stalenss marks shouldn't be changed in removeCounterResets. Otherwise they will be converted to an ordinary NaN values,
which couldn't be removed in dropStaleNaNs() function later. This may result in incorrect calculations for rollup functions.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
2021-08-14 12:45:31 +03:00
Aliaksandr Valialkin
c1f81f08d4
all: add support for Prometheus staleness markers
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845
2021-08-13 12:13:15 +03:00
Aliaksandr Valialkin
b35ae791f1
app/vmselect: make vmui-update
after the commit 4ae14df864a7e327955f44941295a286175423b3
2021-08-11 13:42:53 +03:00
Aliaksandr Valialkin
60cfa5f100
app/vmselect/promql: add present_over_time(m[d])
function, which will be available starting from Prometheus 2.29.0
...
See https://github.com/prometheus/prometheus/releases/tag/v2.29.0-rc.0 and https://github.com/prometheus/prometheus/pull/9097
2021-08-03 12:21:53 +03:00
Nikolay
3f3ad13753
adds /rules and /alerts api for grafana ( #1504 )
...
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-08-02 17:29:49 +03:00
Aliaksandr Valialkin
99004a6a40
app/vmselect/netstorage: unpack time series data in mostly local big chunks
...
This should improve performance on multi-CPU systems for queries selecting time series with big number of raw samples
2021-07-30 12:26:33 +03:00
Aliaksandr Valialkin
cbb81c2ce9
app/vmselect/netstorage: do not query Go maps with unsafe string keys, since this breaks in Go 1.17
2021-07-30 10:28:19 +03:00
Aliaksandr Valialkin
b709fa387a
app/vmselect: follow-up for ed95bc9531
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1493
2021-07-29 09:48:47 +03:00
arnoldyahad
ed95bc9531
Add case prometheus/rules for grafana 8 ( #1502 )
2021-07-29 06:15:35 +03:00
Aliaksandr Valialkin
49bf3abf67
app/vmselect: follow-up for 626073bca8
...
* Rename -search.maxMetricsPointSearch to -search.maxSamplesPerQuery, so it is more consistent with the existing -search.maxSamplesPerSeries
* Move the -search.maxSamplesPerQuery from vmstorage to vmselect, so it could effectively limit the number of raw samples obtained from all the vmstorage nodes
* Document the -search.maxSamplesPerQuery in docs/CHANGELOG.md
2021-07-28 18:00:04 +03:00
Aliaksandr Valialkin
5d255846ac
all: add go:build
lines for Go1.17
...
See https://tip.golang.org/doc/go1.17#gofmt for more details
2021-07-26 15:50:46 +03:00
Aliaksandr Valialkin
3921d8afae
app/vmselect: prevent from possible deadlock when f callback blocks inside RunParallel
2021-07-26 15:50:45 +03:00
Aliaksandr Valialkin
c3e6ce1db9
app/vmselect: make vmui-update
after a91d41f12a
2021-07-26 10:32:01 +03:00
Aliaksandr Valialkin
b047feeb8b
app/vmselect/promql: properly handle (a op b) default N
if (a op b)
returns NaN series
...
The result should be a series with `N` values and `a op b` labels. Previously such series has been removed from the result.
2021-07-16 01:44:24 +03:00
Aliaksandr Valialkin
b92702f6d5
app/vmselect/netstorage: use more scalable algorithm for ditributing the work among among multiple channels on systems with big number of CPU cores
2021-07-16 00:35:36 +03:00
Aliaksandr Valialkin
df117f85bd
app/vmselect: do not track queries with less than 1ms execution time at /api/v1/status/top_queries
...
This should improve the readability and usefullness of the /api/v1/status/top_queries when debugging slow queries
or queries that take too much cpu time.
2021-07-15 16:53:35 +03:00
Aliaksandr Valialkin
5830ce2706
app/vmselect/netstorage: add -search.maxSamplesPerSeries
command-line option for limiting the number of samples a query can process per each series
...
This should prevent from out of memory crashes like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1067
2021-07-15 16:53:35 +03:00
Aliaksandr Valialkin
6c42db87a8
app/vmselect/netstorage: improve scalability of series unpacking on multi-CPU systems
2021-07-15 15:40:53 +03:00
Aliaksandr Valialkin
9add9d86a6
app/vmselect/promql: duration handling improvements in MetricsQL queries
...
- Support durations anywhere in MetricsQL queries. E.g. sum_over_time(m[1h])/1h is equivalent to sum_over_time(m[1h])/3600
- Support durations without suffix. E.g. rate(m[300]) is equivalent to rate(m[5m])
2021-07-12 17:19:32 +03:00
Aliaksandr Valialkin
f5fa177141
Revert "app/vmselect: expose vmui at /select/<accountID>/prometheus/vmui additionally to /select/<accountID>/vmui"
...
This reverts commit 885a79def6799f288e14df05b35a12569659ab85.
Reason for revert: Grafana doesn't allows accessing /select/<accountID>/prometheus/vmui :(
2021-07-12 09:08:43 +03:00
Aliaksandr Valialkin
ddaa12050d
app/vmselect: expose vmui at /select/<accountID>/prometheus/vmui additionally to /select/<accountID>/vmui
...
The /select/<accountID>/prometheus/vmui is needed for accessing via server-side Prometheus datasource for Grafana.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-10 12:52:25 +03:00
Aliaksandr Valialkin
0b98f6c7ff
app/vmselect: expose vmui at /vmselect/<accountID>/vmui/
instead of /vmselect/<accountID>/prometheus/vmui/
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-10 12:32:21 +03:00
Aliaksandr Valialkin
98e049ba6d
app/vmui: move source code from https://github.com/VictoriaMetrics/vmui to app/vmui
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-09 17:13:51 +03:00
Aliaksandr Valialkin
2c5e1cd893
app/vmselect: move web ui from /ui to /select/<accountID>/prometheus/ui
...
This way the UI is available for every tenant (aka accountID) and the UI can automatically determine the needed per-tenant datasource path from page referer.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
2021-07-08 13:14:50 +03:00
Aliaksandr Valialkin
acb7a95c64
app/vmselect: follow-up after aa11ef6d3b
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4
2021-07-07 17:45:09 +03:00
tony
aa11ef6d3b
add vmui for vmselect component ( #1431 )
...
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-07-07 17:04:23 +03:00
Aliaksandr Valialkin
9c19719ad6
app/{vminsert,vmselect}: export vminsert_request_duration_seconds and vmselect_request_duration_seconds histograms
2021-07-07 13:27:23 +03:00
Aliaksandr Valialkin
ceda2b1df4
lib/httpserver: print full requestURI in httpserver.Errorf
...
This should simplify debugging.
2021-07-07 13:11:29 +03:00
Aliaksandr Valialkin
44855f0c9b
app/{vmselect,vmstorage}: clarify the description for -dedup.minScrapeInterval
command-line flag
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1426
2021-07-02 15:06:41 +03:00
Aliaksandr Valialkin
856aecae05
app/vmselect/promql: return the last timestamp for the max / min value from tmax_over_time()
and tmin_over_time()
function as most users expect
2021-06-23 14:18:37 +03:00
Aliaksandr Valialkin
b92d110cad
app/vmselect: log slow requests to all the /api/v1/*
handlers if their execution time exceeds -search.logSlowQueryDuration
2021-06-18 19:07:03 +03:00
Roman Khavronenko
a15c947045
promql: fix increase_pure
calculation for cases with stale series ( #1381 )
...
Due to staleness handling, increase_pure were using incorrect previous value
during calculation in cases where series disappears for period longer
than staleness period and then returns back. The fix suppose to account
for a real datapoint value before staleness takes place. The fix should
remove unexpected spikes while using `increase_pure` for staled series.
2021-06-15 17:37:51 +03:00
John Belmonte
3e79f3994e
spelling fix: synonym ( #1363 )
2021-06-11 10:58:48 +03:00
Aliaksandr Valialkin
e8e7f03394
app/vmselect/promql: typo fix in the comment
2021-06-09 18:34:57 +03:00
Aliaksandr Valialkin
a0b001bfec
app/vmselect/netstorage: remove duplicate limiter on concurrent queries
...
It duplicates the `-search.maxConcurrentRequests` limiter.
2021-05-24 19:13:04 +03:00
Aliaksandr Valialkin
180829b8c2
app/vmselect/promql: add timezone_offset(tz)
function
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1306
2021-05-20 11:54:06 +03:00
Nikolay
be87be34a4
Adds tsdb match filters ( #1282 )
...
* init work on filters
* init propose for status filters
* fixes tsdb status
adds test
* fix bug
* removes checks from test
2021-05-12 17:16:58 +03:00
Aliaksandr Valialkin
56b08390f6
app/vmselect/promql: allow to use 2x more memory for query processing in cluster mode compared to single-node mode
...
`vmselect` has no `vmstorage`-related caches. So it can use more memory for query processing compared to single-node VictoriaMetrics.
2021-05-12 14:43:49 +03:00
Aliaksandr Valialkin
6dc5d3b357
all: rename https://victoriametrics.github.io to https://docs.victoriametrics.com
2021-04-20 20:20:01 +03:00
Aliaksandr Valialkin
e48fd5776b
app/vmselect/promql: fix tests after d3fa0ccabd
2021-04-08 00:17:40 +03:00
Aliaksandr Valialkin
f90bf265f4
app/vmselect/promql: properly detect aggregate topk*
and bottomk*
aggregate functions in order to disable duplicate sorting
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1189
2021-04-08 00:10:33 +03:00
Aliaksandr Valialkin
9ce3b7e1dd
app/vmselect: return data:null
instead of data:[]
from /api/v1/query_exemplars
, since Grafana throws an error otherwise
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1186
2021-04-07 23:34:01 +03:00
Aliaksandr Valialkin
3d5f1f779f
app/vmselect: do not sort series returned from topk*
and bottomk*
functions, since these series are already sorted in user-expected order
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1189
2021-04-07 14:16:02 +03:00
Aliaksandr Valialkin
7fce4e9fb4
app/vmselect: return dumb response on /api/v1/query_exemplars
request
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1186
2021-04-05 23:28:21 +03:00
Aliaksandr Valialkin
2585058a5f
Makefile: prepare arm64
and amd64
release archives for cluster version on make release
command
2021-04-05 23:01:45 +03:00
Aliaksandr Valialkin
4aa59cae7c
app/vmselect/promql: do not delete dst_label
if src_label
is empty in label_copy(q, src_label, dst_label)
and label_move(q, src_label, dst_label)
2021-04-03 22:06:50 +03:00
Aliaksandr Valialkin
b9469de410
app/vmselect/promql: add ability to set label value additionally to label name for the remaining sum of time series returned from topk_*
and bottomk_*
functions in the form: topk_min(N, m, "label=value")
2021-04-02 23:56:30 +03:00
Aliaksandr Valialkin
4028d692f5
app: do not process non-GET requests on at /
handler
2021-04-02 22:56:38 +03:00
Aliaksandr Valialkin
31d3d02d25
app/vmselect/promql: remove the limit on the number of time series that can be sorted, since it may confuse users
...
Always sort time series returned from `/api/v1/query` and `/api/v1/query_range` unless `sort_*` function is used at top level of the query.
2021-04-02 16:50:02 +03:00
Aliaksandr Valialkin
d8c769e6af
app/vmselect: add -search.maxStatusRequestDuration
command-line flag for limiting the duration of requests to /api/v1/status/*
and /api/v1/series/count
2021-03-30 21:39:32 +03:00
Aliaksandr Valialkin
0e29d48628
app/vmselect: remove mentioning for -search.storageTimeout
from command-line flag descriptions
...
This is a follow-up for ecfd6fe78d
2021-03-30 15:07:40 +03:00
Aliaksandr Valialkin
ecfd6fe78d
app/vmselect: remove -search.storageTimeout
command-line flag, since it has the same meaning as -search.maxQueryDuration
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2021-03-30 14:54:36 +03:00
Aliaksandr Valialkin
7bafaad46d
app/vmselect: prevent from possible incomplete query results after timed out query
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2021-03-30 13:34:57 +03:00
Nikolay
022fe4efd0
makes tenant metrics names consistent with enterprise version ( #1152 )
2021-03-29 11:59:04 +03:00
Aliaksandr Valialkin
698ccca6ad
app/vmselect/prometheus: explain why partial responses are disallowed during data export
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1148
2021-03-25 18:39:07 +02:00
Aliaksandr Valialkin
b473c21915
app/vmselect/promql: do not merge time series during requests to /api/v1/query
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1141
2021-03-25 13:56:23 +02:00
Aliaksandr Valialkin
d1e773266f
app/vmselect/graphite: accept and enforce extra_label
in all the Graphite APIs
2021-03-23 15:30:15 +02:00
Aliaksandr Valialkin
27bb614016
app/vmselect: move getEnforcedTagFiltersFromRequest to searchtuils, since it will be used in Graphite functions soon
2021-03-23 14:57:57 +02:00
Aliaksandr Valialkin
7f9863254d
app/vmselect: improve description for -search.maxPointsPerTimeseries
command-line flag
2021-03-22 16:44:36 +02:00
Aliaksandr Valialkin
92531a38c4
app/vmselect/promql: increment key prefix for faster reset for rollup result cache
2021-03-22 11:59:39 +02:00
Aliaksandr Valialkin
799461d8bf
app/vmselect/graphite: follow-up after 529d7be26b
2021-03-18 16:33:00 +02:00
Nikolay
8807410a00
changes metricsFind api ( #1137 )
...
it should be able mitigate crash if label value contains *,[ or { symbols
2021-03-18 16:32:59 +02:00
Aliaksandr Valialkin
667d129e1e
app/vmselect/promql: typo fix after 9666834045
2021-03-17 15:11:54 +02:00
Aliaksandr Valialkin
69201806f8
app/vmselect/promql: merge adjancent buckets with the smallest summary number of hits in buckets_limit()
function
...
This should improve accuracy for the returned buckets
2021-03-17 14:31:29 +02:00
Aliaksandr Valialkin
05e301cfa0
app/vmselect/netstorage: reduce mutex contention when unpacking data on a system with high number of CPU cores
2021-03-16 21:52:05 +02:00
Aliaksandr Valialkin
f54ece438d
app/vmselect/promql: do not crash if histogram_over_time()
function name contains uppercase letters such as Histogram_over_time()
2021-03-16 12:23:55 +02:00
Aliaksandr Valialkin
e2717d84c0
all: various fixes in command-line flag descriptions
2021-03-15 22:03:49 +02:00
Aliaksandr Valialkin
30ab0eec27
app/vmselect/promql: reduce overhead on scrape interval estimation
...
It should be enough to use the first 20 datapoints instead of 100 datapoints for scrape interval estimation.
2021-03-15 20:32:35 +02:00
Aliaksandr Valialkin
8844b38745
app/vmselect/promql: fix tests after 2dae0a2c47
2021-03-15 20:20:27 +02:00
Aliaksandr Valialkin
6b9bba7448
app/vmselect: add round_digits
query arg to /api/v1/query
and /api/v1/query_range
handlers for limiting the number of decimal digits after the point
2021-03-15 12:38:59 +02:00
Aliaksandr Valialkin
b6dcb37fca
app/vmselect/prometheus: do not include datapoints with timestamps matching t-d
when returning results from /api/v1/query?query=m[d]&time=t
as Prometheus does
2021-03-12 12:18:37 +02:00
Aliaksandr Valialkin
942890b1bb
app/vmselect/promql: follow up for 433fff0006
2021-03-09 12:54:23 +02:00
Nikolay
28e450cd7c
duplicate timeseries fix for prometheus_buckets function ( #1119 )
...
* try fix for prometheus_buckets
* merge possible end of the bucket collision
2021-03-09 12:54:21 +02:00
Aliaksandr Valialkin
5277507932
app/vmselect/promql: add histogram_avg()
, histogram_stddev()
and histogram_stdvar()
functions to MetricsQL
2021-03-04 14:12:52 +02:00
Aliaksandr Valialkin
3b1c2f03c3
app/vmselect/querystats: show the number of matching queries in the top by average duration and in the top by summary duration
...
This should help debugging slow queries.
2021-02-28 19:42:29 +02:00
Nikolay
673b10dd7f
adds enforced tag filters into cache key ( #1095 )
2021-02-27 00:23:38 +02:00
Aliaksandr Valialkin
e38ca28d99
app/vmselect/promql: increase accuracy for buckets_limit()
function for small limits by skipping the first and the last buckets during merge
...
The first and the last buckets are usually `[0 ... leMin]` and `(leMax ... +Inf)`. If they are merged with adjancent buckets,
then the resulting accuracy can suffer.
2021-02-26 22:56:04 +02:00
Aliaksandr Valialkin
0144b164c7
app/vmselect: add sign(q)
and clamp(q, min, max)
functions, which will be added in the upcoming Prometheus release
...
See https://twitter.com/roidelapluie/status/1363428376162295811
The `last_over_time(m[d])` function already exists in MetricsQL.
2021-02-24 17:26:27 +02:00
Aliaksandr Valialkin
1b90a091cf
app/vmselect/netstorage: cleanup after 4805b80977
2021-02-24 11:43:09 +02:00
weng zhao
4805b80977
vmselect: fix vmselect cache partial result when timeout ( #1085 )
2021-02-24 11:39:56 +02:00
Aliaksandr Valialkin
d16effc29e
app/vmselect/promql: properly calculate histogram_quantile() over zero buckets and only a single non-zero
le="+Inf"` bucket like Prometheus does
2021-02-24 00:41:30 +02:00
Aliaksandr Valialkin
5dbe88a1c6
app/vmselect/promql: add increase_pure()
function to MetricsQL
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962
2021-02-22 19:15:31 +02:00
Aliaksandr Valialkin
bd1d906eee
app/vmselect/promql: reduce the probability of duplicate time series
errors when querying Kubernetes metrics
2021-02-18 22:08:13 +02:00
Aliaksandr Valialkin
3062ff0fdb
app/vmselect: export per-tenant stats on the number of requests and the cumulative request duration
...
The metrics are:
- vm_vmselect_http_requests_total{accountID="...",projectID="..."} - the total number of select requests per each tenant
- vm_vmselect_http_duration_ms_total{accountID="...",projectID="..."} - the total duration in milliseconds for per-tenant select requests
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/932
2021-02-16 23:30:29 +02:00
Aliaksandr Valialkin
73c9da16b8
app/vmselect/netstorage: reuse timeseriesWork objects in order to reduce memory allocations
2021-02-16 16:09:00 +02:00
Aliaksandr Valialkin
a3a09a3c6e
app/vmselect/netstorage: use unsafe string as a key for a map when the map already contains the given key
...
This should prevent from a memory allocation and a string copy.
2021-02-16 15:46:31 +02:00
Aliaksandr Valialkin
46e98ed490
vendor: update github.com/VictoriaMetrics/metrics from v1.13.1 to v1.14.0
...
The new version switches from log-linear histograms to log-based histograms,
which provide up to 3.6 times better accuracy.
2021-02-15 15:11:15 +02:00
Aliaksandr Valialkin
c769f8321d
deployment/docker: embed tzdata into prod Go app instead of installing it into base docker image
...
While this increases app size by 700Kb, this allows using -loggerTimezone in a scratch base image
See https://github.com/golang/go/issues/38017
2021-02-12 04:56:27 +02:00
Aliaksandr Valialkin
ff7850aec0
deployment/docker: use docker buildx
for creating multiarch builds
...
See https://github.com/docker/buildx/
2021-02-12 04:35:35 +02:00
Aliaksandr Valialkin
ee4288987b
app/vmselect/prometheus: treat match
query arg in the same way as match[]
query arg
2021-02-11 15:01:55 +02:00
Aliaksandr Valialkin
54d2f67924
app/vmselect/promql: properly make copies of EvalConfig
2021-02-11 12:41:37 +02:00
Aliaksandr Valialkin
5973fd4067
app/vmselect/promql: make a copy of EvalConfig when executing q1
and q2
in parallel for q1 binary_op q2
...
This should prevent from data races if the underlying functions modify EvalConfig contents.
2021-02-10 23:06:58 +02:00
Aliaksandr Valialkin
985c3e301d
app/vmselect: parallelize q1 <binary_op> q2
queries by running q1
and q2
in parallel
...
This should reduce query execution times.
2021-02-10 23:00:56 +02:00
Aliaksandr Valialkin
8de3a329ff
app/vmselect: deprecate -search.treatDotsAsIsInRegexps
in favor to {__graphite__="foo.*.bar"}
syntax
2021-02-03 20:36:42 +02:00
Aliaksandr Valialkin
8249f13104
app/vmselect,lib/storage: properly parse Graphite selectors with inner wildcards
...
Example: foo{bar{x,yz},a[b-c],*de}
2021-02-03 20:16:28 +02:00
Aliaksandr Valialkin
4b930b9ffe
app/vmselect: add ability to set Graphite-compatible filter via {__graphite__="foo.*.bar"}
syntax
2021-02-03 01:17:19 +02:00
Aliaksandr Valialkin
7d23f3ff3a
app/vmselect: add ability to set an additional label filters via extra_label
query arg
2021-02-01 17:42:35 +02:00
Nikolay
b8bc1c2e0f
Graphite vmalert wip ( #112 )
...
* init implementation for graphite alerts
* adds graphite support for vmalert
* small fix
* changes vmalert graphite api with type
* updates tests
* small fix
* fixes graphite parse
* Fixes graphite from time
2021-02-01 15:28:30 +02:00
Aliaksandr Valialkin
5328a102e0
app/vmselect: unconditionally deny partial responses from /api/v1/export*
...
It is expected that `/api/v1/export*` returns full data, so there is no sense in partial responses there.
2021-01-27 14:39:53 +02:00
Aliaksandr Valialkin
44c74f1e79
app/vmselect/promql: improve documentation for -search.maxPointsPertimeseries
command-line flag
...
This should reduce incorrect usage and assumptions for this flag.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1020
2021-01-22 13:00:35 +02:00
Aliaksandr Valialkin
e55205220b
app/vmselect: add -search.maxStepForPointsAdjustment
command-line flag, which can be used for disabling adjustment for points returned from /api/v1/query_range
handler if they have timestamps closer than -search.latencyOffset
to the current time
2021-01-19 22:57:50 +02:00
Aliaksandr Valialkin
5856611291
app/vmselect/graphite: extract getCanonicalPath() function from loop body inside getCanonicalPaths()
2021-01-18 17:31:27 +02:00
Aliaksandr Valialkin
c5bdab5a4c
app/vmselect/promql: add ability to pass multiple labels to sort_by_label
and sort_by_label_desc
functions
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/992
2021-01-13 12:43:47 +02:00
Aliaksandr Valialkin
8cae98aa78
app/vmselect/promql: properly parse escaped multibyte utf8 code sequences in metric names and labels names
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/990
2021-01-13 10:59:32 +02:00
Aliaksandr Valialkin
df6e399f73
app/vmselect/promql: add tfirst_over_time(m[d])
and tlast_over_time(m[d])
MetricsQL functions for returning timestamps for the first and the last samples in m
over d
2021-01-12 16:12:47 +02:00
Aliaksandr Valialkin
47872ada7e
app/vmselect/promql: do not ajdust offset
value provided in the query
...
Previously it could be modified in order to improve response cache hit ratio.
This is unneeded, since cache hit ratio should remain good because the query time range
should be already aligned to multiple of `step` values.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/976
2020-12-27 14:10:15 +02:00
Aliaksandr Valialkin
5bbf200de2
app/vmselect: add per-tenant /api/v1/status/top_queries
handler
2020-12-27 12:53:50 +02:00
Aliaksandr Valialkin
0e739efc88
app/vmselect/promql: simplify defer call for querystats.RegisterQuery
2020-12-27 12:07:56 +02:00
Aliaksandr Valialkin
44932098b5
app/vmselect/querystats: reduce the default number of last queries to track from 100K to 20K
...
This should reduce memory usage in constrained environments
2020-12-25 17:40:32 +02:00
Aliaksandr Valialkin
e6deb39064
app/vmselect: refactor /api/v1/stats/top_queries
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907
2020-12-25 17:24:25 +02:00
Nikolay
76d092c091
Adds query stats handler ( #945 )
...
* Adds query stat handler,
for query and query_range api, victoriametrics tracks query execution time,
stats are expored at /api/v1/status/queries endpoint with topN param
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907
* fixed query stats bugs
* improves queryStats tracker
* improves query stat
* small fix
* fix tests
* added more tests
* fixes 386 tests
* naming fixes
* adds drop for outdated records
2020-12-25 17:24:24 +02:00
Nikolay
0b87f02602
fixes panic ( #979 )
...
* fixes panic
https://github.com/VictoriaMetrics/helm-charts/issues/89
* add fast-path
* Apply suggestions from code review
Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2020-12-24 09:01:40 +02:00
Aliaksandr Valialkin
262cf81757
app/vmselect: properly parse negative combined offsets such as -1h2m3s
...
Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.
2020-12-19 01:25:03 +02:00
Aliaksandr Valialkin
8d1031c29a
app/vmselect/promql: return expected increase()
result for the first point on the graph with value not exceeding 100
2020-12-15 14:10:50 +02:00
Aliaksandr Valialkin
a2eb451de4
app/{vmagent,vminsert}: follow-up for ce8c2dd1f1
: return /targets
page in HTML when requested via web browser
2020-12-14 14:13:01 +02:00
Aliaksandr Valialkin
d6f9bf2d19
app/vmselect/graphite: properly handle wildcards and charsets inside curly braces
...
For example, `foo{bar*,[a-f]a*b}` should match `foobar`, `foobar123`, `foofab`, etc.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/952
2020-12-11 17:26:32 +02:00
Aliaksandr Valialkin
1a237c6903
all: properly handle CPU limits set on the host system/container
...
This can reduce memory usage on systems with enabled CPU limits.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946
2020-12-08 21:07:03 +02:00
Aliaksandr Valialkin
bd8eef2528
app/vmselect/promql: do not reduce lookbehind window for any_rollup_func(m)
to -search.maxStalenessInterval
. It should equal to step
value passed to /api/v1/query_range
as most users expect
2020-12-08 15:17:05 +02:00
Aliaksandr Valialkin
9660774fd1
app/vmselect/graphite: remove duplicate name
tag from /tags/autoComplete/tags
handler
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/942
2020-12-07 01:10:02 +02:00
Aliaksandr Valialkin
d242c2f2bd
app/vmselect/promql: add count_eq_over_time(m[d], N)
and count_ne_over_time(m[d], N)
for calculating the number of samples in m
over d
that are equal / not equal to N
2020-12-05 12:31:01 +02:00
Aliaksandr Valialkin
bdac2171f1
all: do not print usage info for all the flags when incorrect command-line flag is passed
...
This should improve usability for VictoriaMetrics apps that have big number of command-line flags,
i.e. all the apps.
2020-12-03 21:46:19 +02:00
Aliaksandr Valialkin
8cf76d8747
app/vmselect/promql: add label_uppercase(q, label1, ... labelN)
and label_lowercase(q, label1, ... labelN)
functions
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/936
2020-12-03 21:46:18 +02:00
Aliaksandr Valialkin
11bbb3552d
app/vmselect/promql: make fmt
2020-12-02 21:34:15 +02:00
Aliaksandr Valialkin
9e98a8f3d3
app/vmselect/promql: return nan
from minute(m)
when m
equals to nan
...
This aligns VictoriaMetrics behaviour with Prometheus behaviour.
The issue has been spotted in https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/
2020-12-02 20:16:40 +02:00
Aliaksandr Valialkin
def513355e
app/vmselect/promql: do not return 0
value from sum_over_time(m[d])
when there are no samples on the given d
window.
...
This aligns the behaviour of `sum_over_time()` with other `_over_time()` functions and with Prometheus behavior.
2020-12-02 13:12:33 +02:00
Aliaksandr Valialkin
490c70a958
app/vmselect: return metric
values from time() cmp_op metric
query when cmp_op
comparison is true
...
This aligns MetricsQL behavior to Prometheus' one.
The issue has been identified at https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/
2020-12-02 12:09:40 +02:00
Aliaksandr Valialkin
4ef7158e89
app/vmselect/promql: return nan
from a >bool b
if a
is nan
in the same way as Prometheus does
2020-12-02 00:28:56 +02:00
Aliaksandr Valialkin
adf45b730c
app/vmselect/searchutils: return elapsed time in Deadline.String() output
...
This should improve debuggability for error messages containing Deadline.String() output
2020-12-01 00:14:36 +02:00
Aliaksandr Valialkin
e1297c0b78
app/vmselect: add /tags/delSeries
handler from Graphite Tags API
...
See https://graphite.readthedocs.io/en/stable/tags.html#removing-series-from-the-tagdb
2020-11-23 15:32:14 +02:00
Aliaksandr Valialkin
3d2ce31cad
app/vmselect/netstorage: code readability improvement: rename *RequestErrors to *Errors
2020-11-23 15:00:15 +02:00
Aliaksandr Valialkin
433ae806ac
app/vmselect: implement /tags/tagSeries
and /tags/tagMultiSeries` in order to be consistent with single-node VictoriaMetrics
2020-11-23 14:57:08 +02:00
Aliaksandr Valialkin
7987129baa
app/vmselect/netstorage: move common code for requests execution on all the storage nodes to startStorageNodesRequest func
2020-11-23 10:51:48 +02:00
Aliaksandr Valialkin
25a57ced6c
app/vmselect/netstorage: prevent from data races in ProcessSearchQuery and in Export funcs when -replicationFactor > 1
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2020-11-23 10:25:51 +02:00
Aliaksandr Valialkin
f4fd917e4f
lib/fs: replace fs.OpenReaderAt with fs.MustOpenReaderAt
...
All the callers for fs.OpenReaderAt expect that the file will be opened.
So it is better to log fatal error inside fs.MustOpenReaderAt instead of leaving this to the caller.
2020-11-23 09:57:30 +02:00
Aliaksandr Valialkin
1dcb438c3b
app/vmselect/netstorage: typo fix after 990eb29a9b
2020-11-23 01:09:43 +02:00
Aliaksandr Valialkin
85eecf5801
app/vmselect/netstorage: add -replicationFactor
command-line flag for reducing query duration when a part of vmstorage nodes are temporarily slow and/or temporarily unavailable
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2020-11-23 00:39:53 +02:00
Aliaksandr Valialkin
990eb29a9b
app/vmselect/netstorage: move common code for collecting query results from vmstorage nodes to collectResults function
2020-11-23 00:16:02 +02:00
Aliaksandr Valialkin
e72ccc9239
app/vmselect: add remoteAddr to slow query log in order to improve debuggability
...
This will simplify identifying the client that sends slow queries to VictoriaMetrics.
2020-11-18 20:40:02 +02:00
Aliaksandr Valialkin
ea4afb201b
app/vmselect/netstorage: typo fix in a comment inside SearchMetricNames func
2020-11-18 01:35:37 +02:00
Aliaksandr Valialkin
7d76fdedcc
app/vmselect: use storage.NewSearchQuery() instead of constructing storage.SearchQuery in-place
...
This should prevent from bugs when AccountID and ProjectID aren't set in storage.SearchQuery.
2020-11-16 18:04:33 +02:00
Aliaksandr Valialkin
911c6d3bcd
app/vmselect: add missing graphite prefix to /tags/autoComplete/{tags,values}
2020-11-16 18:04:24 +02:00
Aliaksandr Valialkin
f7f866d83b
app/vmselect/netstorage: typo fix
2020-11-16 15:54:45 +02:00
Aliaksandr Valialkin
59fb75717e
app/vmselect/netstorage: apply Graphite filter after substituting __name__
with name
2020-11-16 15:50:53 +02:00
Aliaksandr Valialkin
eb763bcb9d
app/vmselect/graphite: add /tags/autoComplete/values
handler from Graphite Tags API
2020-11-16 15:29:29 +02:00
Aliaksandr Valialkin
f2f16d8e79
app/vmselect/graphite: add /tags/autoComplete/tags
handler from Graphite Tags API
...
See https://graphite.readthedocs.io/en/stable/tags.html#auto-complete-support
2020-11-16 14:58:10 +02:00
Aliaksandr Valialkin
2f4421b86c
app/vmselect/prometheus: return __name__
label if match[]
query to /api/v1/labels
matches at least a single time series
2020-11-16 13:54:50 +02:00
Aliaksandr Valialkin
852aed62f7
app/vmselect/prometheus: improve performance for /api/v1/labels
and /api/v1/label/<labelName>/values
on time ranges exceeding one day when match[]
query arg is set
2020-11-16 13:46:51 +02:00
Aliaksandr Valialkin
e969346e3e
app/vmselect/prometheus: fix deadlock in /api/v1/series
on a time range exceeding one day
2020-11-16 13:30:57 +02:00
Aliaksandr Valialkin
eea1be0d5c
app/vmselect/graphite: add /tags/findSeries handler from Graphite Tags API
...
See https://graphite.readthedocs.io/en/stable/tags.html#exploring-tags
2020-11-16 12:52:23 +02:00
Aliaksandr Valialkin
97100b1d42
app/vmselect/graphite: apply filter then limit
2020-11-16 12:52:18 +02:00
Aliaksandr Valialkin
5889273920
app/vmselect/graphite: add /tags/<tag_name>
handler for Graphite Tags API
2020-11-16 03:41:41 +02:00
Aliaksandr Valialkin
99cb1a70cf
app/vmselect/graphite: add /tags
handler from Graphite Tags API
...
See https://graphite.readthedocs.io/en/stable/tags.html#exploring-tags
2020-11-16 02:57:20 +02:00
Aliaksandr Valialkin
2ac5f00d98
app/vmselect: propagate errors from vmstorage
to response to the client if -search.denyPartialResponse
command-line flag is set
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/891
This commit also adds `"isPartial":{true|false}` field to `/api/v1/*` responses. `"isPartial":true` is set when the response
is based on a partial data because some of vmstorage nodes weren't available during query processing.
2020-11-14 13:20:10 +02:00
Aliaksandr Valialkin
8f42e59e05
app/vmselect/promql: remove spikes from increase()
and delta()
results on time series with spare irregular data points
...
Do not take into account spare data point value if the next point will is located too far from the current point.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894
2020-11-13 15:23:37 +02:00
Aliaksandr Valialkin
da6d82a8dd
app/vmselect/promql: assume that time series value doesnt change during gaps when calculating increase() and delta()
...
This should remove unexpected spikes at the end of gaps.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894
2020-11-13 14:59:32 +02:00
Aliaksandr Valialkin
7ceaf4ba8f
all: consistently return text-based HTTP responses with charset=utf-8
...
This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897
2020-11-13 10:30:21 +02:00
Aliaksandr Valialkin
4f2c5877db
app/vmselect: add -search.treatDotsAsIsInRegexps
command-line flag for automatic escaping of dots in regexp label filters
2020-11-11 12:40:28 +02:00
Aliaksandr Valialkin
a78bf34ff3
app/vmselect: do not return isPartialResponse=true
when all the storageNodes return errors
2020-11-10 18:48:57 +02:00
Aliaksandr Valialkin
8f3339fa81
app/vmselect/promql: do not return data points in the end of the selected time range for time series ending in the middle of the selected time range
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/887
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845
2020-11-10 14:51:55 +02:00
Aliaksandr Valialkin
6385432611
app/vmselect: typo fix in a description for -search.minStalenessInterval
: mimimum->minimum
2020-11-10 01:18:59 +02:00
Aliaksandr Valialkin
efebc3b6fb
app/vmselect/promql: code cleanup after 43823addea
2020-11-06 01:31:33 +02:00
n4mine
3127aa92b5
app/vmselect/promql: fix when the parameter of maxValue()
, minValue()
leading by NaN
. it will cause {top,bottom}k_{max,min}
return inappropriate result ( #883 )
2020-11-06 01:31:31 +02:00
Aliaksandr Valialkin
72011bcc45
app/vmselect: properly handle errors in GetLabelsOnTimeRange and GetLabelValuesOnTimeRange
2020-11-05 01:36:34 +02:00
Aliaksandr Valialkin
c5e6c5f5a6
app/vmselect: optimize querying for /api/v1/labels
and /api/v1/label/<name>/values
when start
and end
args are set
2020-11-05 01:19:29 +02:00
Aliaksandr Valialkin
66de02fbb4
app/vmselect/promql: allow dropping trailing sample only for default_rollup
function
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/850
2020-11-02 02:11:06 +02:00
Aliaksandr Valialkin
6b623eba02
app/vmselect/promql: go fmt
2020-11-02 00:18:24 +02:00
Aliaksandr Valialkin
7c0b658865
app/vmselect/promql: do not drop trailing datapoints for instant queries
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
2020-11-02 00:12:53 +02:00
Aliaksandr Valialkin
c4464594b7
app/vmselect/promql: allow passing optional third argument to topk_*
and bottomk_*
functions in order to obtain sum of time series outside top/bottom K
2020-10-20 20:09:55 +03:00
Aliaksandr Valialkin
ee2902ddaf
app/vmselect/promql: an attempt to improve heuristics for dropping trailing data points in time series
...
Now trailing data points are additionally dropped for time series with a single raw sample
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
2020-10-17 10:44:26 +03:00
Aliaksandr Valialkin
b9a4601c97
app/vmselect/promql: return a single time series at max from absent()
function like Prometheus does
2020-10-13 15:56:10 +03:00
Aliaksandr Valialkin
217c192c88
app/vmselect/promql: improve time series staleness detection
...
This should prevent from double counting for time series at the time when it changes label.
The most common case is in K8S, which changes pod uid label with each new deployment.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
2020-10-13 12:20:08 +03:00
Aliaksandr Valialkin
f877e703c8
app/vmselect/promql: fix mode_over_time
calculations
...
Previously `mode_over_time` could return garbage due to improper shuffling of input data points.
2020-10-13 11:58:30 +03:00
Aliaksandr Valialkin
d884ab13dc
app/vmselect/prometheus: fix golangci-lint warning
2020-10-13 09:36:18 +03:00
Aliaksandr Valialkin
0867dea5fc
app/vmselect: add ability to export data in CSV format via /api/v1/export/csv
2020-10-12 20:08:08 +03:00
Aliaksandr Valialkin
2d03d0e2dd
app/vmselect/promql: keep metric name after applying more functions, which dont change time series meaning
...
Functions are:
* keep_last_value
* keep_next_value
* interpolate
* running_min
* running_max
* running_avg
* range_min
* range_max
* range_avg
* range_first
* range_last
* range_quantile
* smooth_exponential
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
2020-10-12 11:48:38 +03:00
Aliaksandr Valialkin
3881c84afe
Revert "app/vmselect/promql: remove metric name after applying ceil
, floor
and round
functions in order to be more consistent with Prometheus"
...
This reverts commit ac45082216
.
Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users -
functions, which don't change the meaning of the time series shouldn't drop metric name.
Now the following functions do not drop metric names:
* ceil
* floor
* round
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
2020-10-12 11:48:38 +03:00
Aliaksandr Valialkin
79d70480b7
Revert "app/vmselect/promql: remove metric name after applying clamp_min
and clamp_max
functions in order to be consistent with Prometheus"
...
This reverts commit bb61a4769b
.
Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users -
functions, which don't change the meaning of the time series shouldn't drop metric name.
Now the following functions do not drop metric name:
* clamp_min
* clamp_max
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
2020-10-12 11:48:38 +03:00
Aliaksandr Valialkin
8c37b63ea9
Revert "app/vmselect/promql: remove metric name from results of certain rollup functions in order to be consistent with Prometheus"
...
This reverts commit e5202a4eae
.
Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users -
functions, which don't change the meaning of the time series shouldn't drop metric name.
Now the following functions do not drop metric name:
* max_over_time
* min_over_time
* avg_over_time
* quantile_over_time
* geomean_over_time
* mode_over_time
* holt_winters
* predict_linear
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674
2020-10-12 11:48:38 +03:00
Aliaksandr Valialkin
f6ee6efc34
app/vmselect/promql: add missing label filters to binary operands before query execution
...
This implements the optimization described at https://utcc.utoronto.ca/~cks/space/blog/sysadmin/PrometheusLabelNonOptimization
See also https://github.com/cortexproject/cortex/issues/3253
2020-10-07 21:17:11 +03:00
Aliaksandr Valialkin
abfd3a8fab
app/{vminsert,vmselect,vmstorage}: add a link to https://victoriametrics.github.io/Cluster-VictoriaMetrics.html from main page of every cluster component
2020-10-06 15:30:07 +03:00
Aliaksandr Valialkin
3e0c473cc9
app/vmselect/promql: fill gaps on graphs for range_*
and running_*
functions
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/806
2020-10-02 14:02:47 +03:00
Aliaksandr Valialkin
61f4801b93
app/vmselect/graphite: do not substitute path
and path.
with path..
in /metrics/find/?format=completer
output
2020-09-29 18:03:45 +03:00
Aliaksandr Valialkin
0d082cdf53
app/vmselect/netstorage: remove unused code
2020-09-29 11:55:08 +03:00
Aliaksandr Valialkin
c47972d843
app/vmselect/prometheus: check for errors returned from bufferedwriter.Write
...
This makes `make errcheck` happy
2020-09-29 11:36:54 +03:00
Aliaksandr Valialkin
e66f7edfc9
app/vmselect/graphite: properly handle case when /metrics/find
finds both leaf and node for the given query=prefix.*
...
In this case only node must be returned with stripped dot in the end of id as carbonapi does
2020-09-29 11:29:38 +03:00
Aliaksandr Valialkin
6d8c23fdbd
app/{vminsert,vmselect}: skip accountID and projectID when marshaling/unmarshaling MetricName in /api/v1/export/native and /api/v1/import/native
...
This is needed in order to be able to migrate native data from/to single-node VictoriaMetrics
2020-09-28 00:58:58 +03:00
Aliaksandr Valialkin
db14f22fc0
app/vmselect: stop /api/v1/export/*
execution if client disconnects
2020-09-27 23:52:42 +03:00
Aliaksandr Valialkin
aadbd014ff
all: add native format for data export and import
...
The data can be exported via [/api/v1/export/native](https://victoriametrics.github.io/#how-to-export-data-in-native-format ) handler
and imported via [/api/v1/import/native](https://victoriametrics.github.io/#how-to-import-data-in-native-format ) handler.
2020-09-27 17:36:38 +03:00
Aliaksandr Valialkin
973df09686
app/vmselect/netstorage: do not spend CPU time on unpacking empty blocks during /api/v1/series
calls
2020-09-24 20:44:15 +03:00
Aliaksandr Valialkin
b8bce348c5
app/vmselect/promql: properly limit implicitly set rollup window to -search.maxStalenessInterval
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/784
2020-09-23 23:24:09 +03:00
Aliaksandr Valialkin
8627365b48
app/vmselect/prometheus: code cleanup after 3ba507000c
2020-09-23 13:04:51 +03:00
Aliaksandr Valialkin
1fce79518a
app/vmselect/prometheus: return timestamps from /api/v1/query
, which match the time
query arg
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720
2020-09-23 12:59:23 +03:00
Aliaksandr Valialkin
0468cdf33e
app/vmselect/netstorage: properly pre-allocate space for sbs
2020-09-22 23:51:01 +03:00
Aliaksandr Valialkin
69eb9783e6
app/vmselect: make sure the request doesnt wait in pending queue more than the configured timeout
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2020-09-22 01:21:42 +03:00
Aliaksandr Valialkin
e564725641
app/vmselect/searchutils: fixed tests after 2eb72e09ab
2020-09-21 21:31:28 +03:00
Aliaksandr Valialkin
07c6226334
app/vmselect: use time
value rounded to seconds if it isnt passed to /api/v1/query
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720
2020-09-21 21:24:46 +03:00
Aliaksandr Valialkin
a1bebb660c
app/vmselect/graphite: return proper results /metrics/find?query=foo.*.bar
according to Graphite Metrics API
2020-09-18 11:53:28 +03:00
Aliaksandr Valialkin
9793008734
app/vmselect: add -search.storageTimeout command-line flag for limiting the maximum duration of query execution per each -storageNode
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2020-09-16 21:33:47 +03:00
Aliaksandr Valialkin
a9205fe308
app/vmselect: prevent from closing connection to vmstorage on query timeout by setting +2 secs deadline on connection comparing to query deadline
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2020-09-16 21:14:00 +03:00
Aliaksandr Valialkin
1587f83fa0
app/vmselect/netstorage: typo fix after 03dfccfbed
2020-09-16 00:10:33 +03:00
Aliaksandr Valialkin
03dfccfbed
app/vmselect/netstorage: reduce memory usage when the time range from query touches big number of samples per each time series
2020-09-15 21:08:09 +03:00
Aliaksandr Valialkin
27cd5555e6
app/vmselect/netstorage: mention that RunParallel or Cancel must be called on the returned results from ProcessSearchQuery
2020-09-15 20:39:43 +03:00
Aliaksandr Valialkin
1f0c0b0f6b
app/vmselect: typo fix in -search.maxStalenessInterval description
2020-09-15 14:25:34 +03:00
Aliaksandr Valialkin
b776a93608
app/vmselect/promql: support composite durations like Prometheus 2.21 does
...
The following durations are supported now: `1h5m35s` or `1s543ms`
See https://github.com/prometheus/prometheus/releases/tag/v2.21.0
and https://github.com/prometheus/prometheus/pull/7713
2020-09-11 22:17:24 +03:00
Aliaksandr Valialkin
d3ad0d365e
app/vmselect: move Deadline from netstorage to searchutils
...
This removes dependency on netstorage from searchutils.
2020-09-11 13:39:13 +03:00
Aliaksandr Valialkin
579c20756a
app/vmselect: substitute inf values at smooth_exponential with the previous values
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/757
2020-09-11 12:23:56 +03:00
Aliaksandr Valialkin
d67e6d3d2e
app/vmselect: skip infinite values when calculating smooth_exponential
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/757
2020-09-11 11:57:53 +03:00
Aliaksandr Valialkin
06427a184f
app/vmselect/graphite: typo fix in label name for vm_request_duration_seconds metric
2020-09-11 01:59:52 +03:00
Aliaksandr Valialkin
f307e6f432
app/vmselect: initial implementation of Graphite Metrics API
...
See https://graphite-api.readthedocs.io/en/latest/api.html#the-metrics-api
2020-09-11 00:30:20 +03:00
Aliaksandr Valialkin
f3a79abfb4
app/vmselect/promql: go fmt
2020-09-08 15:18:57 +03:00
Aliaksandr Valialkin
4f06eed1c1
app/vmselect/promql: adjust integrate()
calculations to be more similar to calculations from InfluxDB: attempt #2
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/701
2020-09-08 14:36:23 +03:00
Aliaksandr Valialkin
0d0b606455
app/vmselect/promql: adjust integrate()
calculations to be more similar to calculations from InfluxDB
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/701
2020-09-08 14:24:02 +03:00
Aliaksandr Valialkin
db91045348
app/vmselect/promql: increase floating point calculations accuracy by dividing by 1e3
instead of multiplying by 1e-3
2020-09-08 14:01:02 +03:00
Aliaksandr Valialkin
804304c365
app/vmselect: add missing deletion for temporary files on partial responses when -search.denyPartialResponse=true
2020-09-04 02:23:12 +03:00
Aliaksandr Valialkin
478d8f8393
app/vmselect/promql: add count_le_over_time(m[d], le)
and count_gt_over_time(m[d], gt)
functions
...
These functions returns the number of raw samples that don't exceed `le` or are bigger than `gt`.
These functions are complement to already existing `share_le_over_time(m[d], le)` and `share_gt_over_time(m[d], gt)`.
2020-09-03 15:28:58 +03:00
Aliaksandr Valialkin
3490160fd0
app/vmselect: unconditionally align time range boundaries to step for subqueries as Prometheus does
2020-09-03 13:22:06 +03:00
Aliaksandr Valialkin
6aab2f4989
all: allow using KB
, MB
, GB
, KiB
, MiB
and GiB
suffixes in command-line flag values related to byte sizes or byte rates
2020-08-16 17:08:28 +03:00
Aliaksandr Valialkin
285665e93b
app/vmselect/promql: allow passing multiple args to aggregate functions such as avg(q1, q2, q3)
2020-08-15 01:15:16 +03:00
Aliaksandr Valialkin
6721e47ae9
app: respect CPU limits set via cgroups
...
Update GOMAXPROCS to limits set via cgroups. This should reduce CPU trashing and reduce memory usage
for cases when VictoriaMetrics components run in containers with CPU limits.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/685
2020-08-11 23:01:03 +03:00
Aliaksandr Valialkin
62b6e54622
app/vmselect: reduce memory usage when exporting time series with big number of samples via /api/v1/export
if max_rows_per_line
is set to non-zero value
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/685
2020-08-10 20:57:43 +03:00
Aliaksandr Valialkin
c9f5c5623f
app/vmselect/netstorage: vary batch size for data unpacking depending on the available CPU cores
...
This should reduce contention on the channel with unpack work for systems with high number of CPU cores
2020-08-10 15:16:48 +03:00
Aliaksandr Valialkin
95a8c492ef
app/vmselect/promql: properly handle -n^m
like Prometheus does
...
`-n^m` must be handled as `-(n^m)` instead of `(-n)^m`.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/675
2020-08-07 07:42:42 +03:00
Aliaksandr Valialkin
7f93d61a56
app/vmselect/promql: remove metric name after applying clamp_min
and clamp_max
functions in order to be consistent with Prometheus
...
This improves VictoriaMetrics score at https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 23:42:55 +03:00
Aliaksandr Valialkin
01000505a0
app/vmselect/promql: remove metric name after applying ceil
, floor
and round
functions in order to be more consistent with Prometheus
...
This improves VictoriaMetrics score at https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 23:34:03 +03:00
Aliaksandr Valialkin
75bff1a567
app/vmselect/promql: remove metric name from results of certain rollup functions in order to be consistent with Prometheus
...
Rollup functions:
- avg_over_time
- min_over_time
- max_over_time
- quantile_over_time
This improves VictoriaMetrics results at https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 23:29:18 +03:00
Aliaksandr Valialkin
8835004a4c
app/vmselect: properly handle PromQL queries like scalar1 < metric < scalar2
like Prometheus does
...
This fixes some cases from https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 23:21:14 +03:00
Aliaksandr Valialkin
14ddb8a34e
app/vmselect/netstorage: reduce CPU contention when upacking time series blocks by unpacking batches of such blocks instead of a single block
...
This should improve query performance on systems with big number of CPU cores (16 and more)
2020-08-06 17:50:13 +03:00
Aliaksandr Valialkin
46c98cd97a
app/vmselect/netstorage: reduce contention on unpackworkCh and timeseriesWorkCh for multi-CPU system by providing more capacity for these chans
2020-08-06 17:22:39 +03:00
Aliaksandr Valialkin
a04f4a3d9a
app/vmselect: use warning level instead of info level for logging slow queries that take longer than -search.logSlowQueryDuration
2020-08-04 20:24:38 +03:00
Aliaksandr Valialkin
bdb881c43b
app/vmselect/promql: add zscore-related functions: zscore_over_time(m[d])
and zscore(q) by (...)
2020-08-03 21:52:15 +03:00
Aliaksandr Valialkin
94471a1273
app: remove duplicate *-pure makefile rules
2020-07-31 20:01:30 +03:00
Aliaksandr Valialkin
a2aa3a60eb
app/vmselect: show X-Forwarded-For
contents on /api/v1/status/active_queries
page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/659
2020-07-31 20:01:09 +03:00
Aliaksandr Valialkin
106e302d7a
all: add mssing APP_NAME to vm*-GOARCH builds
2020-07-31 13:45:32 +03:00
Aliaksandr Valialkin
0c00fe70cf
app/vmselect: do not adjust start
and end
query args passed to /api/v1/query_range
when -search.disableCache
command-line flag is set
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/563
2020-07-30 23:14:56 +03:00
Aliaksandr Valialkin
338ee47d60
app/vmselect/promql: return non-empty value from rate_over_sum(m[d])
even if a single data point is located in the given [d]
window
...
Just divide the data point value by the window duration in this case.
2020-07-29 12:37:34 +03:00
Aliaksandr Valialkin
717c554fb0
app/vmselect/promql: remove rollupFuncArg.realPrevValue handling, since the corner case in increase()
is handled in another way now
...
See e00cfc854d
for the approach used now.
2020-07-29 12:37:34 +03:00
Aliaksandr Valialkin
d9037b3970
app/vmselect/promql: fill gaps with 0 in rate_over_sum
response when the last value before the selected time window isnt empty
2020-07-29 12:37:34 +03:00
Aliaksandr Valialkin
0f6f0d30d3
app/vmselect: show query origin (aka remote_addr or client address) on the /api/v1/status/active_queries
page for every query
2020-07-28 15:14:40 +03:00
Aliaksandr Valialkin
9dccedc599
app/vmselect/promql: return empty values from group()
if all the time series have no values at the given timestamp
...
This aligns `group()` behaviour to Prometheus
2020-07-28 13:41:04 +03:00
Aliaksandr Valialkin
b191e425b3
app/vmselect/promql: improve further the accuracy of buckets_limit() function
...
The accuracy is increased by mergin the smallest bucket with the smallest adjacent bucket.
2020-07-26 12:10:56 +03:00
Aliaksandr Valialkin
43871e79c6
app/vmselect/promql: avoid dropping inf
bucket in buckets_limit
...
The `le="inf"` bucket must be preserved in order to maintain the maximum level of accuracy.
2020-07-25 17:00:25 +03:00
Aliaksandr Valialkin
978c1e930e
app/vmselect/promql: optimize buckets_limit(k, buckets)
for big number of buckets
2020-07-25 13:24:33 +03:00
Aliaksandr Valialkin
51cbf27077
app/vmselect/promql: improve the accuracy of buckets_limit(k, buckets)
function
...
Now it properly merges the bucket with the previous bucket after deletion.
2020-07-24 17:07:30 +03:00
Aliaksandr Valialkin
cf69b1ea6f
app/vmselect/promql: add buckets_limit(k, buckets)
function, which limits the number of buckets per time series to k
...
This function works with both Prometheus-style and VictoriaMetrics-style buckets.
The function removes buckets with the lowest values in order to reserve the highest precision.
The function is useful for building heatmaps in Grafana from too big number of buckets.
2020-07-24 16:14:12 +03:00
Aliaksandr Valialkin
45334f61de
app/vmselect: fix tests for rate_over_sum
2020-07-24 02:35:09 +03:00
Aliaksandr Valialkin
3526e8768a
app/vmselect/promql: typo fix after 3e557c9861
2020-07-24 02:15:23 +03:00
Aliaksandr Valialkin
8d1721d128
app/vmselect/promql: add rate_over_sum(m[d])
function to MetricsQL, which returns rate over sum of m
values over d
duration
...
Something like `sum_over_time(m[d]) / d`, but more accurate.
2020-07-24 01:17:15 +03:00
Aliaksandr Valialkin
88e8bed0c9
app/vmselect/promql: allow setting [d]
window smaller than the interval between raw points for avg_over_time
...
This makes `avg_over_time` behavior consistent with `sum_over_time` and `count_over_time` behaviors.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/636
2020-07-23 22:25:33 +03:00
Aliaksandr Valialkin
fb3d1380ac
lib/storage: respect -search.maxQueryDuration
when searching for time series in inverted index
...
Previously the time spent on inverted index search could exceed the configured `-search.maxQueryDuration`.
This commit stops searching in inverted index on query timeout.
2020-07-23 21:22:05 +03:00
Aliaksandr Valialkin
dbf3038637
lib/storage: add more fine-grained pace limiting for search
2020-07-23 19:21:49 +03:00
Aliaksandr Valialkin
16a4b1b20c
app/vmselect/netstorage: protect from too smart compiler, which may break memory usage optimization in tmpBlocksFileWrapper.WriteBlocks
2020-07-23 17:57:24 +03:00
Aliaksandr Valialkin
55ed07add1
app/vmselect: typo fix after 0168e21fe32776e2f7f003f88e0e6e490eb2dcb0g
2020-07-23 14:11:15 +03:00
Aliaksandr Valialkin
7aa5b48508
app/vmselect: reduce memory usage when querying big number of time series with long labels
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/646
2020-07-23 13:48:58 +03:00
Aliaksandr Valialkin
20d0c41ac5
app/vmselect/prometheus: support d
, w
and y
suffixes for durations passed to step
in /api/v1/query_range
like Prometheus does
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/641
2020-07-22 16:27:27 +03:00