Commit Graph

1211 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
78114e85d6
vendor: update github.com/VictoriaMetrics/metricsql from v0.50.0 to v0.51.0
Updates https://github.com/VictoriaMetrics/metricsql/pull/7
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3589
2023-01-05 01:50:26 -08:00
Aliaksandr Valialkin
ac890b3081
docs: update -help outputs for vm* tools 2023-01-03 23:27:31 -08:00
Aliaksandr Valialkin
d794e971fc
app/vmui: small usability improvements
- Show in the line tooltip the number of the query which generates the given line.
  This simplifies comparison of lines generated by multiple queries.

- Show metric name as __name__ label in the line tooltip in the same way as other labels are shown there.
  This makes the label information in the tooltip more consistent.

- Properly quote label values with JSON.stringify(). This prevents from improper formatting
  when label values contain doublequote chars.

- Remove double curly braces artifact at graph legend for lines without names and labels.

- Properly use modifier for regular expressions across the code.
2022-12-29 15:00:10 -08:00
Aliaksandr Valialkin
71f2979669
app/vmselect/vmui: make vmui-update after 1720bddb4f 2022-12-29 12:20:20 -08:00
ChenyuanHu
8dfe95761e
app/vmselect/prometheus: no need manually call queryDuration.UpdateDuration (#3564)
There is no need to manually call `queryDuration.UpdateDuration(startTime)`, because `defer queryDuration.UpdateDuration(startTime)` is executed at the beginning of the function(L660).
2022-12-29 10:41:51 -08:00
Aliaksandr Valialkin
f27bb19213
app/vmselect/searchutils: accept partial RFC3339 values at time, start and end query args
This simplifies manual usage of the APIs. For example, the following query
would return the results over the 2022 year.

  /api/v1/query_range?start=2022&end=2023&step=1d&query=...

This is equivalent to:

  /api/v1/query_range?start=2022-01-01T00:00:00Z&end=2023-01-01T00:00:00Z&step=1d&query=...
2022-12-28 19:46:21 -08:00
Yury Molodov
6f21435d2d
vmui: fix step field (#3561)
* feat: use a unit next to the step value

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-12-28 16:01:33 -08:00
Artem Navoiev
393f4ab86f
update links to grafana dashboards (#3534)
docs: update links to grafana dashboards

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2022-12-28 11:22:02 -08:00
Aliaksandr Valialkin
197e58f1f6
app/vmui: show min, max and avg lines at Explore metrics graphs when instance is selected in the same way as when only the job is selected
This improves consistency of the graphs.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3386
2022-12-23 23:21:23 -08:00
Aliaksandr Valialkin
eaaae2fab8
app/vmselect/vmui: make vmui-update after 0dca224ec3 2022-12-23 22:26:23 -08:00
Aliaksandr Valialkin
8a1775f3a6
app/vmui: follow-up after f6d31f5216
- Document the feature at docs/CHANGELOG.md.
- Document the metrics explorer at https://docs.victoriametrics.com/#metrics-explorer .
- Properly set `start` and `end` args for the selected time range
  when performing the request, which returns metric names.
- Improve queries, so they return lower number of lines and labels.
  This should improve metrics' exploration.
- Properly encode label filters and query args before passing them to VictoriaMetrics.
- Various cosmetic fixes.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3386
2022-12-22 17:17:58 -08:00
Aliaksandr Valialkin
507e719c3f
app/vmselect/promql: add tests for d3de110070 2022-12-21 20:25:37 -08:00
Aliaksandr Valialkin
a77bb09e69
app/vmselect/promql: make sure that label_replace() doesn't create an empty dst_label if the src_label doesn't match regex 2022-12-21 20:25:37 -08:00
Aliaksandr Valialkin
45a34dca4e
app/vmselect/vmui: make vmui-update after 731d189fa9 2022-12-20 14:51:41 -08:00
Aliaksandr Valialkin
7fb314bb59
app/vmselect/promql: do not extend too short lookbehind window for rate() function if it is set explicitly
Previously too short lookbehind window d for rate(m[d]) could be automatically extended
if it didn't cover at least two raw samples. This was needed in order to guarantee
non-empty results from rate(m[d]) on short time ranges.

Now the lookbehind window isn't extended if it is set explicitly,
since it is expected that the user knows what he is doing.

The lookbehind window continues to be extended when needed if it isn't set explicitly.
For example, in the case of rate(m).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3483
2022-12-20 00:18:32 -08:00
Aliaksandr Valialkin
8d7d8f0125
app/vmselect/prometheus: follow-up after 86dae56bd0
Return error if the provided latency_offset query arg cannot be parsed.
This should simplify debugging in production.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3481
2022-12-16 17:16:26 -08:00
Roman Khavronenko
f8a3514e99
vmselect: support overriding of -search.latencyOffset (#3489)
support overriding of `-search.latencyOffset` value via
URL param `latency_offset`.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3481

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-16 17:14:23 -08:00
Aliaksandr Valialkin
1a88fe5b1f
lib/flagutil/bytes.go: properly handle values bigger than 2GiB on 32-bit architectures
This fixes handling of values bigger than 2GiB for the following command-line flags:

- -storage.minFreeDiskSpaceBytes
- -remoteWrite.maxDiskUsagePerURL
2022-12-14 19:29:57 -08:00
Aliaksandr Valialkin
f8e008f5ad
app/vmselect: show the link to /expand-with-exprs at /select/0/ page 2022-12-14 16:23:19 -08:00
Aliaksandr Valialkin
6d13ca2b88
app/vmselect: add /select/0/prometheus/expand-with-exprs page 2022-12-14 16:23:17 -08:00
Aliaksandr Valialkin
7f022e03f9
app/vmselect/vmui: make vmui-update 2022-12-14 12:02:05 -08:00
Aliaksandr Valialkin
d66d28f917
app/vmselect/vmui: make vmui-update 2022-12-12 17:49:07 -08:00
Aliaksandr Valialkin
808ce815e4
app/vmselect/promql: allow passing inf arg into functions, which accept numeric limit on the number of output time series
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3461
2022-12-10 22:48:01 -08:00
Aliaksandr Valialkin
a521135b7b
lib/promscrape: allow editing relabeling configs and labels at /target-relabel-debug page
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407
2022-12-10 12:47:47 -08:00
Aliaksandr Valialkin
72ea387c86
app/vmselect: properly serve static css for /select/.../metric-relabel-debug page 2022-12-10 03:04:43 -08:00
Aliaksandr Valialkin
97b41e727c
lib/promscrape: implement target-level and metric-level relabel debugging
Target-level debugging is performed by clicking the 'debug' link at the corresponding target
on either http://vmagent:8429/targets page or on http://vmagent:8428/service-discovery page.

Metric-level debugging is perfromed at http://vmagent:8429/metric-relabel-debug page.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407

See https://docs.victoriametrics.com/vmagent.html#relabel-debug
2022-12-10 02:25:56 -08:00
Roman Khavronenko
909cd04c55
lib/storage: keep sample with the biggest value on timestamp conflict (#3421)
The change leaves raw sample with the biggest value for identical
timestamps per each `-dedup.minScrapeInterval` discrete interval
when the deduplication is enabled.

```
benchstat old.txt new.txt
name                                         old time/op    new time/op    delta
DeduplicateSamples/minScrapeInterval=1s-10      817ns ± 2%     832ns ± 3%      ~     (p=0.052 n=10+10)
DeduplicateSamples/minScrapeInterval=2s-10     1.56µs ± 1%    2.12µs ± 0%   +35.19%  (p=0.000 n=9+7)
DeduplicateSamples/minScrapeInterval=5s-10     1.32µs ± 3%    1.65µs ± 2%   +25.57%  (p=0.000 n=10+10)
DeduplicateSamples/minScrapeInterval=10s-10    1.13µs ± 2%    1.50µs ± 1%   +32.85%  (p=0.000 n=10+10)

name                                         old speed      new speed      delta
DeduplicateSamples/minScrapeInterval=1s-10   10.0GB/s ± 2%   9.9GB/s ± 3%      ~     (p=0.052 n=10+10)
DeduplicateSamples/minScrapeInterval=2s-10   5.24GB/s ± 1%  3.87GB/s ± 0%   -26.03%  (p=0.000 n=9+7)
DeduplicateSamples/minScrapeInterval=5s-10   6.22GB/s ± 3%  4.96GB/s ± 2%   -20.37%  (p=0.000 n=10+10)
DeduplicateSamples/minScrapeInterval=10s-10  7.28GB/s ± 2%  5.48GB/s ± 1%   -24.74%  (p=0.000 n=10+10)
```

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3333
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-08 18:18:36 -08:00
Aliaksandr Valialkin
353aad9da9
app/vmselect/searchutils: do not print flag name responsible for query timeout if the timeout isn't reached
This should make the log message more clear
2022-12-08 13:08:02 -08:00
Aliaksandr Valialkin
a257c5fe53
app/vmselect: typo fixes in code comments 2022-12-06 20:59:17 -08:00
Aliaksandr Valialkin
6423c0b901
app/vmselect/vmui: make vmui-update after 7645d9ae00 2022-12-05 23:10:42 -08:00
Aliaksandr Valialkin
c902500efc
app/vmselect/promql: add range_trim_spikes(phi, q) function for trimming phi percent of largest spikes per each time series returned by q 2022-12-05 21:55:24 -08:00
Aliaksandr Valialkin
3a25a4b1de
app/{vminsert,vmselect}: speed up TestInitStopNodes() 2022-12-03 23:53:14 -08:00
Zakhar Bessarab
e407e7243a
{app/vmstorage,app/vmselect}: add API to get list of existing tenants (#3348)
* {app/vmstorage,app/vmselect}: add API to get list of existing tenants

* {app/vmstorage,app/vmselect}: add API to get list of existing tenants

* app/vmselect: fix error message

* {app/vmstorage,app/vmselect}: fix error messages

* app/vmselect: change log level for error handling

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-11-25 10:32:45 -08:00
Aliaksandr Valialkin
693d68f307
app/vmselect/vmui: make vmui-update after 37cda9abd0 2022-11-25 07:34:03 -08:00
Aliaksandr Valialkin
2f7152deb3
app/vmselect/vmui: make vmui-update after eb772aa50e 2022-11-24 17:37:33 -08:00
Aliaksandr Valialkin
587f90b4cb
app/vmselect/vmui: make vmui-update after 7dc2349913 2022-11-22 15:36:19 +02:00
Aliaksandr Valialkin
2f9df62795
app/vmselect/vmui: make vmui-update after 7d1b3e7e14 2022-11-22 00:35:05 +02:00
Aliaksandr Valialkin
ad548abd6a
app/vmselect/promql: add range_normalize(q1, ..., qN) function for normalizing query results into [0..1] value range
This may be useful for analyzing correlation between time series with different value ranges
2022-11-21 23:25:35 +02:00
Aliaksandr Valialkin
2d361a834d
app/vmselect/promql: properly return an empty result from limit_offset() if offset exceeds the number of inner time series
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3312
2022-11-21 16:48:10 +02:00
Aliaksandr Valialkin
b0fefe562a
app/vmselect/promql: optimize e1 op e2 when e1 returns an empty result
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3349
2022-11-21 16:09:49 +02:00
Aliaksandr Valialkin
353b137ff0
app/vmselect/netstorage: typo fix after 61736e4a1d 2022-11-19 00:53:54 +02:00
Aliaksandr Valialkin
97eafbe4a7
app/vmselect: clarify that it isnt recommended setting -replicationFactor at vmselect nodes even if the replication is enabled at vminsert nodes 2022-11-18 14:04:12 +02:00
Aliaksandr Valialkin
61736e4a1d
app/vmselect/netstorage: remove superflouos map lookup at ProcessSearchQuery
This should reduce CPU usage a bit during querying
2022-11-18 13:49:59 +02:00
Aliaksandr Valialkin
eb784ff399
app/vmselect/netstorage: emit more useful information in query traces when some of vmstorage nodes return errors or if there is no need to wait for their responses 2022-11-18 13:01:42 +02:00
Yury Molodov
60aaf4b2c2
vmui: add trace analyzer (#3310)
* refactor: change structure project

* refactor: change structure project

* fix: add hooks for set query params

* refactor: add index for pages

* docs: add TESTCASES.md

* refactor: restructure components

* feat: add page with trace analyzer

* fix: change detect trace data

* Update app/vmui/packages/vmui/src/pages/TracePage/index.tsx

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Update app/vmui/packages/vmui/src/pages/TracePage/index.tsx

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* fix: change descriptions on trace page

* Update app/vmui/packages/vmui/src/pages/TracePage/index.tsx

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* feat: add base components

* feat: add reset styles

* docs: add description about trace analyzer

* feat: add styles for custom panel page

* feat: add styles for predefined panels

* feat: add style for TracingsView.tsx

* feat: add Alerts

* feat: add Tooltip.tsx

* fix: correct styles

* feat: add DatePicker.tsx

* feat: add tables

* feat: add theme provider

* fix: replace using callbacks as props to handlers

* fix: correct update time

* fix: change TimePicker.tsx

* fix: correct styles

* fix: update packages

* vmui: refactor code, remove material-ui

* feat: add paste json for trace analyzer

* vmui: update trace analyzer docs

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-11-17 22:23:05 +02:00
Aliaksandr Valialkin
7969ca6c80
app/vmselect/vmui: make vmui-update after 51bfd1ab80 2022-11-17 18:57:44 +02:00
Aliaksandr Valialkin
f67c2a2f8f
app/vmselect/vmui: make vmui-update after bc8a782f74 2022-11-17 01:21:56 +02:00
Aliaksandr Valialkin
0cc59b9f95
app/vmselect/promql: add range_stdvar() and range_stddev() functions for calculating variance and deviation over time series on the selected time range 2022-11-17 01:03:02 +02:00
Aliaksandr Valialkin
facf7efa3a
app/vmselect/promql: add range_linear_regression(q) function for calculating simple linear regression for the selected time series on the selected time range 2022-11-17 00:39:17 +02:00
Aliaksandr Valialkin
58e7963828
app/vmselect/vmui: make vmui-update after 7130af7fd2 2022-11-09 16:43:09 +02:00
Aliaksandr Valialkin
fe8d40f12c
app/{vminsert,vmselect}: test initialization with different number of storage nodes
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3329
2022-11-09 11:48:39 +02:00
Zakhar Bessarab
fd36c9b3b1
docs/{vminsert,vmselect}: clarify flag description for clusternativeListenAddr (#3314)
docs/{vminsert,vmselect}: clarify flag description for `clusternativeListenAddr`
2022-11-04 09:49:53 +01:00
Zakhar Bessarab
7afa67dba0
{app/vmselect,app/vminsert}: ensure -storageNodes flag has no empty values (#3291) 2022-11-01 14:54:55 +02:00
Aliaksandr Valialkin
a02a5f646f
app/vmselect/vmui: make vmui-update after 54e1865d17 2022-10-28 14:52:16 +03:00
Aliaksandr Valialkin
09cf136154
app/{vminsert,vmselect}: add support for automatic discovery and update of vmstorage nodes
Thanks to @dmitryk-dk for the initial implemenation at https://github.com/VictoriaMetrics/VictoriaMetrics-enterprise/pull/446
2022-10-28 13:13:03 +03:00
Aliaksandr Valialkin
976bbe3677
app/{vminsert,vmselect}: limit the access to storageNodes to getStorageNodesBucket and setStorageNodesBucket functions
This makes the code more maintainable and earier to test.
2022-10-28 11:41:55 +03:00
Aliaksandr Valialkin
46450995ae
app/vmselect/vmui: make vmui-update after eae6063450 2022-10-26 02:51:09 +03:00
Aliaksandr Valialkin
ecb71a7221
lib/fs: add canOverwrite arg to WriteFileAtomically when it is allowed to overwrite the file atomically if it already exists 2022-10-26 01:08:35 +03:00
Aliaksandr Valialkin
4f53147ed4
app/{vminsert,vmselect}/netstorage: allow calling Init()+MustStop() in a loop
Previously netstorage.MustStop() call didn't free up all the resources,
so the subsequent call to nestorage.Init() would panic.

This allows writing tests, which call nestorage.Init() + nestorage.MustStop() in a loop.
2022-10-25 14:43:05 +03:00
Aliaksandr Valialkin
596251eb87
app/vmselect/vmui: make vmui-update after 274e235bf7 2022-10-24 21:29:57 +03:00
Aliaksandr Valialkin
c92aef39b5
app/vmselect/promql: expose missing metric vm_cache_size_max_bytes{type="promql/rollupResult"} 2022-10-23 12:14:07 +03:00
Aliaksandr Valialkin
574b9f4f78
app/vmselect/vmui: make vmui-update after 080562030d 2022-10-18 10:51:05 +03:00
Yury Molodov
a50f2e141c
vmui: limit number of plotted series (#3229)
* feat: add maximum display series by tabs

* feat: add warning on PredefinedPanels.tsx

* docs/CHANGELOG.md: vmui limit number of plotted series

* docs/CHANGELOG.md: vmui limit number of plotted series

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-10-13 12:14:53 +03:00
Aliaksandr Valialkin
9bf8d64605
app/vmselect/promql: follow-up for 930f1ee153
Document the change at docs/CHANGELOG.md
Apply it to histogram_quantile() in the same way as to histogram_share()

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3225
2022-10-13 12:06:24 +03:00
Siqi Liu
c0fc716426
BUGFIX: properly calculate histogram_quantile with the same value and different string le (#3225)
Co-authored-by: 647(siki.liu) <siki.liu@huolala.cn>
2022-10-13 12:06:24 +03:00
Aliaksandr Valialkin
a0f3247b14
app/vmselect/promql: properly handle zero and negative values for -search.maxMemoryPerQuery
This is a follow-up for 04a05f161c

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-12 09:33:16 +03:00
Aliaksandr Valialkin
098c9bda27
app/vmselect: return back the logic for limits the amounts of memory occupied by concurrently executed queries if -search.maxMemoryPerQuery isn't set
This is needed for preserving backwards compatibility with the previous releases of VictoriaMetrics.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-10 21:54:18 +03:00
Aliaksandr Valialkin
d8057fca0b
docs: mention -search.maxMemoryPerQuery in the description to -search.maxConcurrentQueries command-line flag
This is a follow-up for 5138eaeea0

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-09 13:58:18 +03:00
Aliaksandr Valialkin
938ff7bba6
app/vmselect: allow limiting per-query memory usage via -search.maxMemoryPerQuery command-line flag
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2022-10-08 01:14:01 +03:00
Yury Molodov
9e4e5abac6
vmui: auto-update chart after query field removed (#3210)
* feat: run query after query field removed

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-10-07 23:08:22 +03:00
Aliaksandr Valialkin
98bd843f3c
app/{vminsert,vmselect,vmstorage}: add package-{vminsert,vmselect,vmstorage}-{goarch} Makefile rules for building arch-specific docker images 2022-10-07 20:17:01 +03:00
Aliaksandr Valialkin
58ee1b9edb
app/vmselect/vmui: make vmui-update after a54987f671 2022-10-07 03:31:00 +03:00
Aliaksandr Valialkin
b05ff477c8
app/vmselect/promql: properly calculate vm_rows_scanned_per_query histogram for rollup functions, which take into account only a few samples on the provided lookbehind window 2022-10-06 23:23:17 +03:00
Aliaksandr Valialkin
baa11a778d
app/vmselect/promql: properly calculate quantiles_over_time() over a single raw sample 2022-10-06 22:37:38 +03:00
Aliaksandr Valialkin
958c1f291c
app: follow-up after ec04fcac93
* Optimize fast path for /api/v1/import when importing numeric values
* Move the docs about the change from features to bugfixes at docs/CHANGELOG.md
* Update tests at lib/protoparser/vmimport

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3161
2022-10-06 14:54:15 +03:00
Dmytro Kozlov
4064db27a8
Properly parse json when export import metric (#3180)
* app/vmselect: properly work when export import json from `api/v1/{export, import}` API

* app/vmselect: update convert function

* app/vmselect: export null if `math.IsNaN(v)`

* app/vmselect: get float from json

* lib/protoparser: add test

* docs: add change log

* lib/protoparser: make export import api compatible
2022-10-06 14:54:14 +03:00
Aliaksandr Valialkin
43bdd96a6e
app/vmselect: improve performance scalability on multi-CPU systems for /api/v1/export/... endpoints 2022-10-01 22:16:07 +03:00
Aliaksandr Valialkin
5497997b72
app/vmselect/promql: increase scalability of incremental aggregate calculations on systems with many CPU cores
Use sync.Map instead of a global mutex there. This should lift scalability limits
on systems with many CPU cores.
2022-10-01 20:14:14 +03:00
Aliaksandr Valialkin
972fcfcd17
app/vmselect/prometheus: improve scalability of /federate endpoint on systems with many CPU cores
Minimize usage of global lock inside bufferedwriter.Write() when processing `/federate` data
on systems with many CPU cores
2022-10-01 20:14:13 +03:00
Aliaksandr Valialkin
ae6c7edf04
app/vmselect: do not export NaN values for stale metrics at /federate endpoint
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3185
2022-10-01 19:48:21 +03:00
Aliaksandr Valialkin
6f9ce3f6d6
lib/flagutil: rename Array to ArrayString
This makes the ArrayString more consistent with other Array* types.

While at it, add ArrayBytes type, which will be used for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3071
2022-10-01 18:28:19 +03:00
Aliaksandr Valialkin
93e84a1c57
lib/httpserver: use 302 redirects instead of 301 redirects
Incorrect 301 redirects can be cached by user agents such as web browsers.
This can complicate recovery procedure after the incorrect redirect is fixed,
e.g. web browser cache must be reset.

The related issue - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1752
2022-10-01 16:56:43 +03:00
Aliaksandr Valialkin
98175493d7
app/vmselect/promql: remove empty series before applying aggregate function
Previously empty series (e.g. series with all NaN samples) were passed to aggregate functions.
Such series must be ingored by all the aggregate functions.
So it is better from consistency PoV filtering out empty series before applying aggregate functions.
2022-09-30 08:44:06 +03:00
Roman Khavronenko
273472dc20
app/vmselect: ignore empty series for limit_offset (#3178)
* app/vmselect: ignore empty series for `limit_offset`

VictoriaMetrics doesn't return empty series (with all NaN values) to
the user. But such series are filtered after transform functions.
It means `limit_offset` will account for empty series as well.

For example, let's consider following data set:
```
time series:
foo{label="1"} NaN, NaN, NaN, NaN // empty series
foo{label="2"} 1, 2, 3, 4
foo{label="3"} 4, 3, 2, 1
```

When user requests all series for metric `foo` the empty series
will be filtered out:
```
/query=foo:
foo{label="v2"} 1, 2, 3, 4
foo{label="v3"} 4, 3, 2, 1
```

But `limit_offset(1, 1, foo)` is applied to original series, not filtered yet.
So it will return `foo{label="v2"}` (skips the first in list)
```
/query=limit_offset(1, 1, foo):
foo{label="v2"} 1, 2, 3, 4
```

Expected result would be to apply `limit_offset` to already filtered list,
so in result we receive `foo{label="v3"}`:
```
/query=limit_offset(1, 1, foo):
foo{label="v3"} 4, 3, 2, 1
```

The change does exactly that - filters empty series before applying `limit_offset`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* app/vmselect: ignore empty series for `limit_offset`

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-30 08:44:05 +03:00
Roman Khavronenko
4acbf6e7d4
vmselect/rollup: rm workaround for slow-changing counters (#3163)
The workaround was introduced to fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962.
However, it didn't prove itself useful. Instead, it is recommended using `increase_pure` function.

Removing the workaround makes VM to produce accurate results when calculating
`delta` or `increase` functions over slow-changing counters with vary intervals
between data points.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-26 15:39:49 +03:00
Dmytro Kozlov
36ea8537d9
vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric) (#2982)
* vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric)

* vmselect/promql: fix tests, add documentation

* vmselect/promql: update test

* vmselect/promql: update for alphanumeric sorting, fix tests

* vmselect/promql: remove comments

* vmselect/promql: cleanup

* vmselect/promql: avoid memory allocations, update functions descriptions

* vmselect/promql: make linter happy (remove ineffectual assigment)

* vmselect/promql: add test case, fix behavior when strings are equal

* vendor: update github.com/VictoriaMetrics/metricsql from v0.44.1 to v0.45.0

this adds support for sort_by_label_numeric and sort_by_label_numeric_desc functions

* wip

* lib/promscrape: read response body into memory in stream parsing mode before parsing it

This reduces scrape duration for targets returning big responses.

The response body was already read into memory in stream parsing mode before this change,
so this commit shouldn't increase memory usage.

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-14 17:42:07 +03:00
Aliaksandr Valialkin
b6bdbb4787
app/vmselect/vmui: make vmui-update after 1304824201 2022-09-14 13:32:43 +03:00
Yury Molodov
465894d720
vmui: fix data processing (#3092)
* fix: change data processing

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-12 10:56:39 +03:00
Yury Molodov
c8de98e03f
vmui: add lists of top queries (#3065)
* feat: add lists of top queries

* fix: change the field label

* refactor: add handlers for readability

* app/vmselect: `make vmui-update`

* docs: document `top queries` tab

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-08 21:44:43 +03:00
Dmytro Kozlov
ec273eafef
app/{vmselect, vminsert}: fails with error when user defines equal addresses in the -storageNodes flag (#3082) 2022-09-08 21:17:58 +03:00
Aliaksandr Valialkin
f0eea5b02d
app/vmselect/netstorage: fix a typo, which leads to incorrect query results in VictoriaMetrics cluster
The typo has been introduced in the commit 1a254ea20c

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3067
2022-09-08 13:46:40 +03:00
Dmytro Kozlov
96ecec877d
vmselect/{promql, prometheus}: show flag names which user can update in error message (#3049)
* vmselect/{promql, prometheus}: show flag names which user can update in error message

* vmselect/{promql, prometheus}: fix typo
2022-09-06 14:48:20 +03:00
Aliaksandr Valialkin
b7f3569522
app/vmselect/prometheus: follow-up after 50e2524bc2
- Add getCommonParamsWithDefaultDuration function and use it at /api/v1/series, /api/v1/labels and /api/v1/label/.../values
- Document the default behaviour for setting 5 minutes time range if start arg isn't passed to /api/v1/series, /api/v1/labels and /api/v1/label/.../values
- Document the change at docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3052
2022-09-05 11:57:08 +03:00
匠心零度
65d1124dc4
api prometheus/api/v1/label/../values time not specified, (#3052)
modify default start values
2022-09-05 11:51:32 +03:00
Aliaksandr Valialkin
c6b74148cf
app/vmselect/promql: consistently calculate rate_over_sum(m[d]) as sum_over_time(m[d])/d
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3045
2022-09-02 23:19:05 +03:00
Aliaksandr Valialkin
9cca3a0a1b
app/vmselect/netstorage: fix potential panic under high load
The panic may trigger during data blocks' processing received
from vmstorage nodes when some of vmstorage nodes return an error
or when `-replicationFactor` is set to values higher than 2 at `vmselect`.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3058
2022-09-02 21:36:15 +03:00
Aliaksandr Valialkin
024e2f18da
app/vmselect/promql: evaluate union() args in parallel in order to increase query performance
Note that the parallel execution of `union()` args may take more memory and CPU time
than the sequential execution if args contain heavy queries, which may load all the available CPU,
disk and memory resources and vmselect and vmstorage levels.
2022-09-02 21:01:04 +03:00
匠心零度
80067fd03a
reduce unnecessary vmstorage query (#3031)
* reduce unnecessary vmstorage query

* reduce unnecessary vmstorage query

* rollback limit logic /api/v1/label/*
2022-08-30 12:34:23 +03:00
Aliaksandr Valialkin
8aaaf221cc
app/vmselect/promql: follow-up after 2d71b4859c
- Use getScalar() function for obtaining the expected scalar from phi arg
- Reduce the error message returned to the user when incorrect phi is passed to histogram_quantiles
- Improve the description of this bugfix in the docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3026
2022-08-27 01:38:17 +03:00
Dmytro Kozlov
2cf29cc4d1
vmselect/promql: fix panic in histogram_quantiles function (#3029)
* vmselect/promql: fix panic in histogram_quantiles function

* Update docs/MetricsQL.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-08-27 01:34:54 +03:00
Dmytro Kozlov
d32a6359b0
vmselect/promql: enable search.maxPointsSubqueryPerTimeseries for sub-queries (#2963)
* vmselect/promql: enable search.maxPointsPerTimeSeriesSubquery for sub-queries

* vmselect/promql: cleanup

* vmselect/promql: rename config flag

* vmselect/promql: add tests

* vmselect/promql: use test object instead of log

* vmselect/promql: fix posible panic is subquery has more points. add description

* vmselect/promql: update tests descriptions

* vmselect/promql: update doInternal validation

* vmselect/promql: fix linter

* vmselect/promql: fix linter

* vmselect/promql: update documentation and release notes

* wip

- Properly apply -search.maxPointsSubqueryPerTimeseries limit to subqueries.
  Previously the -search.maxPointsPerTimeseries limit was unexpectedly applied to subqueries
  if it was smaller than the -search.maxPointsSubqueryPerTimeseries .
- Clarify docs for -search.maxPointsSubqueryPerTimeseries command-line flag .
- Document -search.maxPointsPerTimeseries and -search.maxPointsSubqueryPerTimeseries flags at https://docs.victoriametrics.com/#resource-usage-limits .
- Update docs/CHANGELOG.md .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2922

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-24 15:27:41 +03:00
Aliaksandr Valialkin
08b8467e97
app/vmselect/netstorage: make golangci-lint happy by naming the unused padding field as _ 2022-08-22 00:32:37 +03:00
Aliaksandr Valialkin
9ddd2699fd
all: remove the remaining bits of io/ioutil
The io/ioutil package is deprecated since Go1.16 - see https://tip.golang.org/doc/go1.16#ioutil

VictoriaMetrics requires at least Go1.18, so it is time to remove the io/ioutil from source code

This is a follow-up for 02ca2342ab
2022-08-22 00:22:41 +03:00
Aliaksandr Valialkin
06f6de6d47
all: use os.{Read|Write}File instead of ioutil.{Read|Write}File
The ioutil.{Read|Write}File is deprecated since Go1.16 -
see https://tip.golang.org/doc/go1.16#ioutil

VictoriaMetrics needs at least Go1.18, so it is safe to remove ioutil usage
from source code.

This is a follow-up for 02ca2342ab
2022-08-21 23:55:20 +03:00
Aliaksandr Valialkin
87e0d69bf4
app/vmselect/netstorage: fix a bug introduced in 1a254ea20c
The bug results in `duplicate output time series` error
because the same time series is added two times into the orderedMetricNames list
inside the tmpBlocksFileWrapper.Finalize().

While at it, properly release all the tmpBlocksFile structs on tbf.Finalize() error.
Previously only the remaining tbf entries were released. This could result in resource leak.
2022-08-17 14:07:51 +03:00
Aliaksandr Valialkin
7d7cf2b6fd
app/vmselect: follow-up after 63e0f16062
* Explicitly store a pointer to UserReadableError in the error interface.
  Previously Go automatically converted the value to a pointer before storing in the error interface.

* Add Unwrap() method to UserReadableError, so it can be used transparently with the other code,
  which calls errors.Is() and errors.As().

* Document the change in docs/CHANGELOG.md
2022-08-15 13:53:19 +03:00
Roman Khavronenko
8a26ec435d
vmselect: introduce UserReadableError type of error (#2894)
When read query fails, VM returns rich error message with
all the details. While these details might be useful
for debugging specific cases, they're usually too verbose
for users.
Introducing a new error type `UserReadableError` is supposed
to allow to return to user only the most important parts
of the error trace. This supposed to improve error readability
in web interfaces such as VMUI or Grafana.

The full error trace is still logged with the full context
and can be found in vmselect logs.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-15 13:53:18 +03:00
Yury Molodov
dc52b283a3
vmui: shortcut keys legend (#2971)
* feat: add shortcut modal

* feat: add shortcut descriptions

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-15 12:04:30 +03:00
Aliaksandr Valialkin
1a254ea20c
app/vmselect/netstorage: remove common contention points related to inter-CPU communcations
This should improve vmselect performance scalability on systems with many CPU cores.

The following tasks were done:

- Use separate temporary files for storing the data read from each vmstorage node.
  This may result in the following potential issues:
  - Up to N times higher memory usage for performing each query where N is the number
    of vmstorage nodes known to vmselect.
    This issue shouldn't increase chances of out of memory errors in most cases,
    since per-query memory overhead is quite low comparing to the overall vmselect memory usage.
  - Up to N times higher number of open temporary files where N is the number
    of vmstorage nodes known to vmselect.
    This issue should be fixed by increasing the limit on the number of open files.

- Use separate counters per each vmstorage node for various stats calculation
  when reading the data from vmstorage nodes.
2022-08-11 23:22:56 +03:00
Aliaksandr Valialkin
ec3df0b913
app/vmselect/netstorage: improve scalability of blocks processing on systems with multiple CPU cores
Previously a single syncwg.WaitGroup was used for tracking the lifetime of processBlock callbacks
across all the per-vmstorage goroutines. This could be slow on systems with many CPU cores
because of inter-CPU synchronization overhead.

Use a separate per-vmstorage sync.WaitGroup instead in order to reduce inter-CPU synchronization overhead.
This should imrpove performance for heavy queries over big number of blocks on multi-CPU systems.
2022-08-11 21:37:24 +03:00
Aliaksandr Valialkin
1996e36cf0
app/vmselect/netstorage: prevent from calling processBlocks callback after the exit from ProcessBlocks function
This should prevent from panic at multi-level vmselect
when the top-level vmselect is configured with -replicationFactor > 1
2022-08-08 13:32:44 +03:00
Aliaksandr Valialkin
2635211bf4
app/vmselect/netstorage: properly detect and log timeout errors when querying vmstorage from vmselect
This change is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2937

Thanks to @isodude for the initial pull request.
2022-08-08 00:21:05 +03:00
Yury Molodov
a2ecf311b8
vmui: graph action on moush hold and move (#2915)
* fix: change event for graph panning

* fix: change detect key

* feat: add zoom in with mouse selection

* - document the change
- run `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-07 23:56:08 +03:00
Aliaksandr Valialkin
21f588e9cf
app/vmselect/promql: fix TestVmrangeBucketsToLE test after 8516670582
Remove 'AccountID=0, ProjectID=0, ' prefix for cluster version
2022-08-07 00:15:10 +03:00
Aliaksandr Valialkin
43185353bc
app/vmselect/netstorage: cleanup after 92630c1ab4
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2896
2022-08-04 18:34:38 +03:00
Aliaksandr Valialkin
a1e49606ed
app/{vmselect,vmalert}: properly generate http redirects if -http.pathPrefix command-line flag is set
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2918
2022-08-02 13:01:13 +03:00
Aliaksandr Valialkin
c81d2b4c18
app/vmselect/netstorage: initializes tsw.rowsProcessed before calling tsw.f, since tsw.f can modify r.Timestamps and r.Values lengths 2022-07-30 00:39:14 +03:00
Aliaksandr Valialkin
5ddae2e293
app/vmselect/netstorage: re-use random generator used for series shuffle in Result.RunParallel
This should reduce CPU usage needed for rand.Rand initialization
2022-07-30 00:31:00 +03:00
Dmytro Kozlov
ab645b5fab
vmselect/promql: add tests for vmrangeBucketsToLE (#2907)
* vmselect/promql: add tests for vmrangeBucketsToLE

* vmselect/promql: cleanup

* vmselect/promql: cleanup

* vmselect/promql: fix panic tests want result

* vmselect/promql: cleanup

* vmselect/promql: update test name

* vmselect/promql: fix linter error

* vmselect/promql: refactor testcases

* vmselect/promql: cleanup

* vmselect/promql: remove unused reassign to workers, fix typo

* wip

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-26 20:43:43 +03:00
Aliaksandr Valialkin
8a83c59956
app/vmselect/promql: reduce the diff for f148cffc8a
This is a follow-up for c826f06366
2022-07-26 19:21:16 +03:00
Alan Liang
f97de473f4
vmselect: fix vmrangeBucketsToLE func may panic when ts value equal zero (#2902)
Co-authored-by: alanwzliang <alanwzliang@tencent.com>
2022-07-25 10:49:48 +03:00
Aliaksandr Valialkin
764fc04756
app/vmalert: fix after da10962d4c 2022-07-25 09:48:41 +03:00
Roman Khavronenko
da10962d4c
vmselect: cover special cases for vmalert's routing in single-node version (#2845)
* vmselect: cover special cases for vmalert's routing in single-node version

* remove trailing `/` from requests
* redirect to vmalert's home page when `/vmalert` is requested.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: fix review comments

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update app/vmselect/main.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-25 09:43:44 +03:00
Aliaksandr Valialkin
3d4c312ba2
app/vmselect/netstorage: improve the speed of queries over big number of time series on multi-CPU system
Reduce inter-CPU communications when processing the query over big number of time series.
This should improve performance for queries over big number of time series
on systems with many CPU cores.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2896

Based on b596ac3745
Thanks to @zqyzyq for the idea.
2022-07-25 09:22:28 +03:00
Aliaksandr Valialkin
c0c9f30870
lib/pushmetrics: properly handle errors when initializing pushmetrics 2022-07-22 13:38:25 +03:00
Aliaksandr Valialkin
59bb42be06
app/vmselect/vmui: make vmui-update after edecd2493c 2022-07-21 21:00:36 +03:00
Roman Khavronenko
04c4f8bafd
vmselect: return correct error for second part of expression (#2893)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-21 20:40:47 +03:00
Aliaksandr Valialkin
fe68bb3ba7
all: follow-up after 46f803fa7a
Add -pushmetrics.* command-line flags to all the VictoriaMetrics apps
2022-07-21 20:18:25 +03:00
Aliaksandr Valialkin
f992f96a88
app/vmselect/promql: execute q1 and q2 from q1 op q2 in parallel if labels pushdown cannot be applied
This should improve query performance if VictoriaMetrics has enough resources for processing `q1` and `q2` in parallel.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2886
2022-07-19 14:29:41 +03:00
Aliaksandr Valialkin
d920b0afec
app/vmselect/promql: properly return q1 series from q1 ifnot q2 when q2 returns nothing 2022-07-18 14:24:23 +03:00
Aliaksandr Valialkin
f933d5f142
app/vmselect/promql/transform.go: reuse evalNumber() function for constructing timezone_offset() results 2022-07-18 14:24:22 +03:00
Aliaksandr Valialkin
70b9925bf7
app: fix make publish-* after ed93330e66
Add missing `-linux` substring to built binary names for copying into Docker images
2022-07-14 11:01:34 +03:00
Yury Molodov
1ca20caa4b
vmui: optimize table view (#2867)
* feat: optimize table view

* fix: add column display setting

* app/vmselect: `make vmui-update`

Also document the change at docs/CHANGELOG.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-14 00:16:57 +03:00
Aliaksandr Valialkin
da6c85a2f6
all: follow-up for d99ba3481b 2022-07-13 17:17:08 +03:00
Dmytro Kozlov
4e4def9df8
Rename release packages (#2810)
* makefile: add os to each release file

* makefile: update vmutils arm64

* makefile: update victoria-metrics release process

* makefile: update publish with os

* makefile: update publish with os

* makefile: change tar library

* update release logic

* copy all releases

* sort command by GOOS

* rollback commands

* rollback OSARCH

* fix commands

* cleanup

* fix windows build

* sort build by GOOS, update README.md
2022-07-13 17:11:01 +03:00
Aliaksandr Valialkin
919939ee9f
app/vmselect/promql: consistency update after 93fbd0c54b 2022-07-13 12:34:28 +03:00
Roman Khavronenko
214e01499c
promql: return step as scrapeInterval when it can't be calculated (#2865)
The change allows to specify default value for `getScrapeInterval`
function when actual interval can't be calculated.

Before the change, function were returning `maxSilenceInterval` (5m)
in such cases, which may be not correct for instant queries processing.
The specific scenario where using `maxSilenceInterval` caused issues
is the following:
1. Series becomes stale;
2. Client (in this case vmalert) continues to request series every 15s;
3. Database returns empty results as expected;
4. But at some specific moment of time database returns datapoints from `now()-5m`,
because lookback window was extended to `maxSilenceInterval`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-13 12:34:28 +03:00
Aliaksandr Valialkin
5867708b3d
app/vmselect/promql: validate function name before evaluating its arguments
This avoids unneeded evaluation of args for unknown functions
2022-07-12 19:49:54 +03:00
Aliaksandr Valialkin
fbb403b5c0
app/vmselect/netstorage: optimize mergeSortBlocks() for the worst case when blocks contain interleaved samples 2022-07-12 12:30:24 +03:00
Aliaksandr Valialkin
aee08117e9
app/vmselect/netstorage: add mergeSortBlocks benchmark for the worstcase 2022-07-12 12:26:27 +03:00
Dmytro Kozlov
6734d77d97
vmselect/prometeus: Add limit param to api/v1/series api endpoint (#2851)
* issue-2841: Add limit param to api/v1/series api endpoint

* issue-2841: add change log

* issue-2841: update logic

* issue-2841: simplify logic

* issue-2841: simplify logic, add information to documentation
2022-07-11 20:37:20 +03:00
Aliaksandr Valialkin
ed2126aec3
app/vmselect: follow-up after 6dda254e01 2022-07-11 19:48:18 +03:00
Roman Khavronenko
6dda254e01
vmselect: cover special cases for vmalert's routing in cluster version (#2844)
* vmselect: cover special cases for vmalert's routing

* remove trailing `/` from requests
* redirect to vmalert's home page when `/vmalert` is requested.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmselect: fix review comments

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update app/vmselect/main.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-11 19:33:33 +03:00
Aliaksandr Valialkin
eab8ebbe11
all: make fmt via the upcoming Go1.19 2022-07-11 19:23:25 +03:00
Aliaksandr Valialkin
c0af52228a
app/vmselect/netstorage: add benchmarks for mergeSortBlocks
This is a follow-up for 743ff84863
2022-07-11 12:53:46 +03:00
Aliaksandr Valialkin
d442ee4610
app/vmselect/netstorage: optimize mergeSortBlocks function
- Use binary search instead of linear scan when locating the run of smallest timestamps
  in blocks with intersected time ranges. This should improve performance
  when merging blocks with big number of samples

- Skip samples with duplicate timestamps. This should increase query performance
  in cluster version of VictoriaMetrics with the enabled replication.
2022-07-09 00:35:38 +03:00
Roman Khavronenko
78da074ad3
metricsql: properly evaluate timezone_offset over time interval (#2842)
* metricsql: properly evaluate `timezone_offset` over time interval

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update docs/CHANGELOG.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-08 14:04:40 +03:00
Aliaksandr Valialkin
decbf84e55
app/vmselect/vmui: follow-up after 0bf6841140
* Document the bugfix in docs/CHANGELOG.md
* Run `make vmui-update` for updating static js files for vmui, which are included into vmselect
2022-07-08 13:22:57 +03:00
Aliaksandr Valialkin
6bb4bcd30c
app/vmui: tune visual presentation of trace view 2022-07-06 14:16:24 +03:00
Aliaksandr Valialkin
eee384c9c9
app/vmselect: add -clusternative.tls* options for mTLS setup in multi-level clusters 2022-07-06 13:48:21 +03:00
Aliaksandr Valialkin
195dccf678
app/vmselect: add ability to query vmselect from another vmselect 2022-07-06 13:19:45 +03:00
Aliaksandr Valialkin
714958deab
all: follow-up after ed89106274 2022-07-06 12:48:52 +03:00
Aliaksandr Valialkin
cdd89d9cc2
app/vmselect: properly generate response for /api/v1/series
The response has been broken in 7d5d33fd71
2022-07-06 12:46:23 +03:00
Aliaksandr Valialkin
270e555f47
lib/vmselectapi: pass maxSuffixes arg to tagValueSuffixes RPC call 2022-07-06 12:46:22 +03:00
Aliaksandr Valialkin
f4df43f7cc
app/vmselect/netstorage: remove unused auth.Token arg 2022-07-06 12:46:21 +03:00
Aliaksandr Valialkin
78eeca6f0d
lib/vmselectapi: rename deleteMetrics to more correct deleteSeries 2022-07-06 12:46:21 +03:00
Roman Khavronenko
f51bc07d97
vmselect: allow proxying requests to vmalert (#2833)
The change allows to proxy requests with prefix `/vmalert`
to the vmalert component if `-vmalert.proxyURL` is set.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825
and https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2831

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-06 10:47:51 +02:00
Aliaksandr Valialkin
10c63f5853
app/vmselect: make fmt after f3ece83e67 2022-07-05 14:56:53 +03:00
Aliaksandr Valialkin
8516670582
app/vmselect/promql: properly calculate histogram_quantile over unexpected le buckets
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819
2022-07-05 13:20:41 +03:00
Aliaksandr Valialkin
fa08220d27
app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2787
2022-06-30 22:40:26 +03:00
Aliaksandr Valialkin
4fb0f15322
all: readability improvements for query traces
- show dates in human-readable format, e.g. 2022-05-07, instead of a numeric value
- limit the maximum length of queries and filters shown in trace messages
2022-06-30 18:19:43 +03:00
Dmytro Kozlov
67753cc6f0
vmui: update render logic for nested component (#2795)
* vmui: update render logic for nested component, avoid rerender, remove local storage usage for tracing flag

* docs/url-examples.md: fix various documentation issues there

* docs: add Troubleshooting doc

This doc contains troubleshooting guides for typical problems with VictoriaMetrics.

* docs/Troubleshooting.md: add troubleshooting guide for cluster instability

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-30 15:48:22 +03:00
Aliaksandr Valialkin
daefb64f38
app/vmselect: expose additional histograms at /metrics page, which may help get more insights for the query workload
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792
2022-06-28 20:18:31 +03:00
Aliaksandr Valialkin
9adff0b686
app/vmselect/promql: show the number of scanned samples in the query trace 2022-06-28 19:32:03 +03:00
Aliaksandr Valialkin
c21f5a508f
app/vmselect/prometheus: reduce the default value for -search.maxSeries from 100k to 30k
Production experience shows that 100k is too big for /api/v1/series .
It leads to increased CPU usage when Grafana queries /api/v1/series over VictoriaMetrics
with big number of time series during auto-completion and when modifying template variables.
2022-06-28 18:22:24 +03:00
Aliaksandr Valialkin
7d5d33fd71
lib/storage: return marshaled metric names from SearchMetricNames
Previously SearchMetricNames was returning unmarshaled metric names.
This wasn't great for vmstorage, which should spend additional CPU time
for marshaling the metric names before sending them to vmselect.

While at it, remove possible duplicate metric names, which could occur when
multiple samples for new time series are ingested via concurrent requests.

Also sort the metric names before returning them to the client.
This simplifies debugging of the returned metric names across repeated requests to /api/v1/series
2022-06-28 18:16:32 +03:00
Aliaksandr Valialkin
399d4c36ae
app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day 2022-06-28 12:55:20 +03:00
Aliaksandr Valialkin
a667d339be
app/vmselect/netstorage/netstorage.go: group metrics in order to improve readability a bit 2022-06-27 14:00:24 +03:00
Aliaksandr Valialkin
6386f117c8
all: show timeRange in traces in human-readable format instead of timestamps in milliseconds 2022-06-27 13:42:57 +03:00
Aliaksandr Valialkin
08de733924
app/vmselect/netstorage: assume the response is full if up to -replicationFactor-1 vmstorage nodes are unavailable
This is a follow-up for ee5c502446

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1767
2022-06-27 12:21:26 +03:00
Aliaksandr Valialkin
bc9d704ef4
app/vmselect/netstorage: remove Get prefix from netstorage functions
This makes these function names more consistent with the server side
2022-06-27 00:37:49 +03:00
hagen1778
e40d015e9a vmselect: make vm_partial_results_total consistent
Metrics `vm_partial_results_total` and `vm_requests_total` serving
the similar purpose, but contain inconsistent set of labels.
This change updates `vm_partial_results_total` labels to be consistent
with `vm_requests_total`.

The change breaks backward compatibility with assumption that
`vm_partial_results_total` wasn't widely used, since it is
not documented and absent in the alerts and dashboards.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-24 13:50:26 +02:00
Dmytro Kozlov
f28cbcc7b5
vmui: added query tracing (#2748)
* vmui: added query tracing

* vmui: updated ui

* vmui: update tracing logic, fix bugs, disable tracing by default

* vmui: use empty message as props

* vmui: fixed ui, added delete for each tacing data, show query in header

* vmui: added timelines

* vmui: speedup render

* vmui: use memo for sorting

* vmui: use Trace model, remove unused functions, simplify part of code

* vmui: update recursive logic

* vmui: fix set query to header

* vmui: code cleanup, remove unused code

* vmui: remove unused type, rename component

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-23 23:00:45 +03:00
Nikolay
ee5c502446
app/vmselect: fixes partial response with replicationFactor (#2777)
* app/vmselect: fixes partial response with replicationFactor
Allow partial response if it meets replicationFactor configured at vmselect
https://t.me/VictoriaMetrics_ru1/38490

* docs/CHANGELOG.md: document this change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-23 20:17:24 +03:00
Aliaksandr Valialkin
7429dfbe9e
app/vmselect: add -search.setLookbackToStep command-line flag for making the gap filling algorithm similar to InfluxDB data model
This option should override `-search.maxStalenessInterval` for most cases when users migrate from InfluxDB to VictoriaMetrics
2022-06-22 14:20:02 +03:00
Aliaksandr Valialkin
4f64da874d
app/vmselect: typo fix in the exported metric name: vm_http_request_total -> vm_http_requests_total 2022-06-22 13:18:03 +03:00
Aliaksandr Valialkin
dceca7e864
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package
This is a follow-up for fe2269b999
2022-06-21 20:27:30 +03:00
Aliaksandr Valialkin
fe2269b999
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package
This package already has the same name, so there is no need in explicit name
2022-06-21 20:24:28 +03:00
Aliaksandr Valialkin
b28c6febf9
app/{vminsert,vmselect}: add -vmstorageDialTimeout command-line flag for tuning the maximum time needed for establishing connections to vmstorage
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2022-06-20 15:17:34 +03:00
Dmytro Kozlov
5bc13e2fe8
vmui: added focusLabel, enable cardinality app configuratior (#2736)
* vmui: added focusLabel, enable app configuratior

* vmui: set focusLabel if {labelName!=""}

* wip

* docs/CHANGELOG.md: mention about focusLabel feature in cardinality explorer

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2730

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-17 13:04:09 +03:00
Roman Khavronenko
cbe39bbb8d
vmselect: limit end param max value by 2d in future (#2729)
* vmselect: limit `end` param max value by 2d in future

The change is applied only to service handlers like `/labels` or `/series`
and limits the `end` param by max value <= now() + 2 days. The same limit
is applied for the ingested data, so no reason to allow to request data
in future far than that.

The change is also needed for corner cases like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2669
where too high `end` value triggers inefficient global index search.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-16 20:51:40 +03:00
Aliaksandr Valialkin
da1d1e83df
app/{vmselect,vmstorage}: properly pass seriesCountByLabelName and seriesCountByFocusLabelValue entries from vmstorage to vmselect 2022-06-16 10:44:29 +03:00
Aliaksandr Valialkin
ee9954082f
app/vmselect/netstorage: properly aggregate seriesCountByLabelName and seriesCountByFocusLabelValue obtained from multiple vmselect nodes at /api/v1/status/tsdb 2022-06-15 16:48:40 +03:00
Aliaksandr Valialkin
45fa9d798d
app/vmselect: accept focusLabel query arg at /api/v1/status/tsdb 2022-06-14 18:39:00 +03:00
Aliaksandr Valialkin
fb77843639
lib/storage: show top labels with the highest number of series in cardinality explorer 2022-06-14 16:34:13 +03:00
Dmytro Kozlov
ed9b4c6f6d
vmui: refactor Cardinality panel (#2726)
* vmui: refactor Cardinality panel

* vmui: change width of the search panel

* vmui: code cleanup

* vmui: code cleanup

* vmui: fixed vulnerability (npm audit fix)

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-14 14:40:44 +03:00
Yury Molodov
65107a4585
vmui: enhancements (#2638) (#2717)
* feat: make datepicker to be set to last 30 min by default

* fix: correct spinner while loading data

* feat: change legend style

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-13 09:49:58 +03:00
Aliaksandr Valialkin
61e03f172b
app/vmselect: optimize /api/v1/labels and /api/v1/label/.../values handlers when match[] query arg is passed to them 2022-06-12 14:06:24 +03:00
Aliaksandr Valialkin
4a94cd81ce
app/vmselect: add optional limit query arg to /api/v1/labels and /api/v1/label_values endpoints
This arg allows limiting the number of sample values returned from these APIs
2022-06-10 10:24:07 +03:00
Aliaksandr Valialkin
1335698ba7
app/vmselect/prometheus: extract common code for obtaining common query args into getCommonParams() function 2022-06-09 20:33:36 +03:00
Aliaksandr Valialkin
cb39eada77
all: improve query tracing coverage for indexdb search
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
2022-06-09 20:04:02 +03:00
Aliaksandr Valialkin
ad44eadd94
app/vmselect/graphite: remove additional redundant Request.ParseForm() calls after 38c785b851 2022-06-09 13:22:23 +03:00
Aliaksandr Valialkin
7fe60dad10
app/vmselect: remove redundant calls to Request.ParseForm()
Request.ParseForm() is implicitly called by the first call to Request.FormValue()
2022-06-09 13:09:52 +03:00
Aliaksandr Valialkin
a9ea3fee38
lib/querytracer: make it easier to use by passing trace context message to New and NewChild
The context message can be extended by calling Donef.
If there is no need to extend the message, then just call Done.
2022-06-08 21:16:12 +03:00
Aliaksandr Valialkin
2b343d8bd0
app: properly collect and merge /api/v1/status/tsdb info from vmstorage nodes
The collection has been broken in f2754c3e90

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233
2022-06-08 19:26:09 +03:00
Dmytro Kozlov
f2754c3e90
Cardinality explorer (#2625)
* Cardinality explorer

* vmui, vmselect: updated field name, added description to spinner

* make vmui-update

* updated const name, make vmui-update

* lib/storage: changes calculation for totalSeries values

* added static files

* wip

* wip

* wip

* wip

* docs/CHANGELOG.md: document cardinality explorer feature

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2233

Co-authored-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-08 18:54:27 +03:00
Aliaksandr Valialkin
68b6ddfb14
all: follow-up after 8edb390e21
- Remove unused js bloatware from /targets page. This strips down binary size by more than 100Kb
- Add /service-discovery page for API compatibility with Prometheus
- Properly load bootstrap.min.css from /prometheus/targets
- Serve static contents for /targets page from app/vminsert instead of app/vmselect, because /targets page is served from there
2022-06-07 01:05:53 +03:00
Dmytro Kozlov
ce8aade80e
lib/promscrape: adds service discovery visualization for /targets page(#2675)
* lib/promscrape: updated template

* lib/promscrape: fixed click on unhealthy and all btns

* app/vmselect: jquery scripts into static folder

Co-authored-by: f41gh7 <nik@victoriametrics.com>
2022-06-04 01:11:23 +03:00
Aliaksandr Valialkin
c92bc5394f
app/vmselect/netstorage: properly read trace from vmstorage when it returns error message to vmselect 2022-06-01 14:35:00 +03:00
Aliaksandr Valialkin
afced37c0b
all: add initial support for query tracing
See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
2022-06-01 02:31:44 +03:00
Aliaksandr Valialkin
f0ab5ca61f
app/vmselect/vmui: make vmui-update after 492a615a88 2022-05-26 09:43:30 +03:00
Yury Molodov
b9ef9ddbfd
vmui: import dashboards (#2642)
* fix: switch dashboards import to fetch

* make vmui-update
2022-05-26 09:43:29 +03:00
Aliaksandr Valialkin
4409b9f376
app/vmselect/vmui: make vmui-update 2022-05-20 14:56:25 +03:00
Yury Molodov
d13437afd0
vmui: setup predefined dashboards without build (#2541)
vmui: support predefined dashboards in json format

See https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmselect/vmui/dashboards
2022-05-20 13:46:19 +03:00
Aliaksandr Valialkin
68bfa05437
app/vmselect/promql: do not return values from label_value() if the original time series has no values at the selected timestamps 2022-05-09 17:58:39 +03:00
Aliaksandr Valialkin
8babb4aebc
app/vmselect/vmui: make vmui-update after 450d879eaa 2022-05-05 22:11:41 +03:00
Aliaksandr Valialkin
3fee369997
app/vmselect: follow-up after 8639e79d38 2022-05-04 23:41:00 +03:00
Nikolay
48041305a2
app/vmselect: adds proxy for rules API (#2516)
* app/vmselect: adds proxy for rules and alerts API
It allows to visualization for rules at grafana
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1739

* Update app/vmselect/main.go

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-05-04 10:12:15 +03:00
Dima Lazerka
b5fce4190c
Code cleanup (#343)
* Small code cleanup: remove Request from params

* Extract common params to all export handlers

* Renamed ExportParams -> exportParams

* wip

Co-authored-by: Dzmitry Lazerka <dlazerka@gmail.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-03 15:53:58 +03:00
Aliaksandr Valialkin
4fa3cd701c
docs/Cluster-VictoriaMetrics.md: typo fix: serparated -> separated 2022-05-03 15:20:11 +03:00
Aliaksandr Valialkin
75bd7f1539
app/vmui: fix up/down arrow keys on multi-line query after a186434b50 2022-05-03 10:51:08 +03:00
Aliaksandr Valialkin
361b08c30e
lib/storage: leave the last sample per each discrete interval during the deduplicaton
This aligns better with staleness logic in Prometheus - https://prometheus.io/docs/prometheus/latest/querying/basics/#staleness
2022-05-02 21:59:31 +03:00
Aliaksandr Valialkin
7ca32c21c8
app/vmui: execute query by pressing enter in the same way as Prometheus does
Multi-line query can be entered via `shift-enter` in the query input field
2022-05-02 20:24:43 +03:00
Yury Molodov
6292185abf
vmui: support node v.18 (#2529)
* fix: add support vmui node18

* fix: remove @mui/styles (legacy styling solution)

* Update app/vmui/Dockerfile-build

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-02 20:24:42 +03:00
Nikolay
022b2875ad
app/vmselect: adds API /api/v1/status/buildinfo (#2515)
* app/vmselect: adds API /api/v1/status/buildinfo
it should fix an compability error with grafana 8.5 prometheus datasource
https://github.com/grafana/grafana/pull/46771

* Update main.go
2022-04-29 13:02:43 +03:00
Aliaksandr Valialkin
90f9ed260e
app/vmselect/vmui: make vmui-update after da04e9d1de 2022-04-29 13:02:43 +03:00
Dima Lazerka
e6ee235707
Export "null" in jsonl instead of NaN (#2518)
* Export "null" in jsonl instead of NaN

The NaN appeared because of staleness markers that were added for compatibility. I think it's better to use json `null`, implemented here.

Also maybe it also makes sense to add a flag like `?skip-staleness-markers=true` to `/export`, to skip nulls at all?

* Update app/vmselect/prometheus/export.qtpl

* app/vmselect/prometheus/export.qtpl.go: `make quicktemplate-gen`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-29 12:51:03 +03:00
Yury Molodov
dac203320a
vmui: expression alias (#2495)
* feat: add alias for queries

* docs: update docs for predefined dashboards

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-27 10:59:51 +03:00
Aliaksandr Valialkin
6eb1580158
app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d 2022-04-27 10:58:40 +03:00
Yury Molodov
eae6f68be2
vmui: add support relative time (#2504)
* feat: add support relative time

* app/vmselect: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-26 15:46:47 +03:00
Aliaksandr Valialkin
dac24aa342
app/vmselect/promql: properly handle scalar default vector, scalar if vector and scalar ifnot vector queries
Previously `vector` time series could be unexpectedly returned from such queries
2022-04-21 15:34:14 +03:00
Aliaksandr Valialkin
bc54ae9608
app/vmselect/promql: fix comparison to nan
The comparison to nan has been broken in d335cc886c

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150
2022-04-21 14:56:41 +03:00
Aliaksandr Valialkin
a5cfe5d13e
app/vmselect/promql: add drop_common_labels() function 2022-04-21 14:20:36 +03:00
Aliaksandr Valialkin
7ca08945fd
app/vmselect/promql: fix q default b where b may have empty time series 2022-04-21 11:43:09 +03:00
Aliaksandr Valialkin
25b841c6ed
app/vmselect/promql: fix duplicate time series error on joins against time series filtered by values
This should prevent from `duplicate time series` errors when executing the following query:

kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1)

where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series
2022-04-20 22:21:20 +03:00
Aliaksandr Valialkin
0ef7a05fc0
app/vmselect/promql: rename removeNaNs() to more clear removeEmptySeries() 2022-04-20 19:53:24 +03:00
Yury Molodov
60c4e0022a
fix: prevent graph hiding without data (#2456)
* fix: prevent graph hiding without data

* fix: add yaxis labels default

* app/vmselect: `make vmui-update`

* docs/CHANGELOG.md: document the change

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-16 17:16:17 +03:00
Aliaksandr Valialkin
3e2a4c07cd
app/vmui: further improvements for number display on graphs
This is a follow-up for c4d2cd8336

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2409
2022-04-12 16:02:37 +03:00
Aliaksandr Valialkin
4197c61ef8
app/vmselect: make vmui-update 2022-04-12 14:35:37 +03:00
Aliaksandr Valialkin
3605e1743b
app/vmselect/promql: allow calling InitRollupResultCache+StopRollupResultCache multiple times during tests 2022-04-11 12:32:23 +03:00
Aliaksandr Valialkin
a4a15a462b
app/vmselect/netstorage: bump RPC API versions for vmselect->vmstorage communications
This is a follow-up after b843f0e229
2022-04-08 12:36:04 +03:00
Aliaksandr Valialkin
feff0ef286
app/vmselect: make vmui-update 2022-04-01 12:55:53 +03:00
Yury Molodov
b2f0acaca4
vmui: grid support for predefined panels (#2386)
* update packages

* feat: add setting width for predefined panels

* docs: update doc by predefined dashboards

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-04-01 12:49:24 +03:00
Yury Molodov
7cbf19812d
vmui: predefined panels (#2243)
* feat: add basic components for predefined dashboards

* fix: change display alert

* feat: add autosize and unit for axes

* feat: add component for CircularProgress

* feat: change layout for predefined dashboards

* feat: add override step for predefined panels

* feat: add override step for predefined panels

* feat: change yaxis limits for predefined panels

* fix: rename flag for hide legend

* feat: add formatted panel description

* feat: add README.md for dashboard setup

* feat: validate dashboard settings

* feat: add unit for y-ticks

* fix: correct display error for dashboards

* fix: disable auto refresh after route change

* update package-lock.json

* fix: add basename for BrowserRouter

* fix: add dynamic basename for routing

* update packages

* feat: add a pre-defined dashboard "per-job resource usage"

* feat: display unit in the hover-tooltip

* fix: change routing and home layout

* fix: change axis width calc

* updated packages

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-03-26 13:05:21 +02:00
Aliaksandr Valialkin
b843f0e229
app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs 2022-03-26 11:28:14 +02:00
Aliaksandr Valialkin
84b3234f3d
docs/CHANGELOG.md: document a1e17e91f8 2022-03-21 15:37:03 +02:00
Aliaksandr Valialkin
85695faa94
app/vmselect/bufferedwriter: suppress trivial network errors, which can be generated by remote side
These errors include `broken pipe` and `reset by peer`.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2334
2022-03-18 19:27:33 +02:00
Aliaksandr Valialkin
95da5e52c4
app/vmselect/vmui: make vmui-update 2022-03-18 13:18:57 +02:00
Yury Molodov
235ef947ed
vmui: update packages (#2264)
* update package.json

* update package-lock.json

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-03-03 18:50:44 +02:00
Aliaksandr Valialkin
99ee1c13c0
app/vmselect/promql: reduce the maximum number of label values, which can be propagated from one side of the binary operation to another side of the binary operation from 10K to 1K
There are user reports that 10K unique values in a single label filter may lead to performance and memory usage issues
2022-02-24 04:03:28 +02:00
Aliaksandr Valialkin
244c23ea2c
lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes
This should reduce memory usage under high time series churn rate
2022-02-23 13:42:27 +02:00
Aliaksandr Valialkin
701eb0495b
app/vmselect: make vmui-update 2022-02-22 21:02:20 +02:00
Aliaksandr Valialkin
89ead3daca
app/vmselect/netstorage: report vmstorage errors to vmselect clients even if partial responses are allowed
If a vmstorage is reachable and returns an application-level error to vmselect,
then such error must be returned to the caller even if partial responses are allowed,
since it usually means cluster mis-configuration.

Partial responses may be returned only if some vmstorage nodes are temporarily unavailable.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1941
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/678
2022-02-21 21:17:05 +02:00
Yury Molodov
cdd632985f
vmui: change logic autocomplete (#2196)
* fix: change autocomplete display logic

* fix: change z-index for labels of input fields

* fix: change autocomplete display logic

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-21 15:44:54 +02:00
Aliaksandr Valialkin
c723ac6a8f
app/vmui: small fixes
* Remove unneeded dependency on `numeral` package
* Properly parse numbers obtained from /api/v1/query_range according to
  https://prometheus.io/docs/prometheus/latest/querying/api/#expression-query-result-formats
* Optimize updating processing the received data from /api/v1/query_range
* Make smoother zoom on `ctrl+scroll`
* Reduce the number of points received from /api/v1/query_range by 2x in order to reduce load on backend
2022-02-14 16:31:31 +02:00
Aliaksandr Valialkin
989668beba
app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130
2022-02-12 15:46:43 +02:00
Roman Khavronenko
791cad8c2e
lib/promscrape: support prometheus-like duration in scrape configs (#2169)
* lib/promscrape: support prometheus-like duration in scrape configs

The change allows to specify duration values like `1d`, `1w`
for fields `scrape_interval`, `scrape_timeout`, etc.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/817#issuecomment-1033384766
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/blockcache: make linter happy

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/promscrape: support prometheus-like duration in scrape configs

* add support for extra fields `scrape_align_interval` and `scrape_offset`;
* support Prometheus duration parsing for `__scrape_interval__`
and `__scrape_duration__` labels;

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

* wip

* docs/CHANGELOG.md: document the feature

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-11 16:17:51 +02:00
Aliaksandr Valialkin
d1c17fe385
app/vmselect/vmui: make vmui-update 2022-02-08 17:29:35 +02:00
Aliaksandr Valialkin
1173964d8d
app/vmselect/promql: clarify comments on why the right side of if and and operators are executed at first 2022-02-03 00:26:42 +02:00
Aliaksandr Valialkin
10367d7e4c
app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values
Such filters may slow down time series search, so just skip them.

This is a follow-up for e7f1ceeb84

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827
2022-02-02 23:42:25 +02:00
Aliaksandr Valialkin
5f266370c5
all: follow-up after 4bdd10ab90
Properly use new bytesutil.Resize* functions
2022-02-01 17:49:28 +02:00
Aliaksandr Valialkin
13c2692ca1
app/vmselect/promql: properly handle foo or bar queries
Such queries may miss `bar` results after the commit e7f1ceeb84
because common label filters from `foo` could be mistakenly applied to `bar`.
2022-02-01 17:43:10 +02:00
Aliaksandr Valialkin
02b2bfcff3
lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations
Follow-up for f4989edd96

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-02-01 11:20:20 +02:00
Aliaksandr Valialkin
784cdcb080
app/vmselect/promql: check for binary operation in case-insensitive manner when deciding which side of the operation to perform the first
PromQL and MetricsQL operators are case-insensitive
2022-01-31 22:07:35 +02:00
Aliaksandr Valialkin
04d6596298
app/vmselect/promql: optimize queries, which join on _info metrics.
Automatically add common filters from one side of binary operation
to the other side before sending the query to storage subsystem.

See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/
and https://www.robustperception.io/exposing-the-software-version-to-prometheus
2022-01-31 20:25:15 +02:00
Aliaksandr Valialkin
d95d945046
app/vmselect/vmui: make vmui-update 2022-01-31 12:42:21 +02:00
Aliaksandr Valialkin
a3eafd2e7f
app/vmselect/vmui: make vmui-update after e3995572bb 2022-01-25 17:52:37 +02:00
Aliaksandr Valialkin
6232eaa938
lib/bytesutil: split Resize() into ResizeNoCopy() and ResizeWithCopy() functions
Previously bytesutil.Resize() was copying the original byte slice contents to a newly allocated slice.
This wasted CPU cycles and memory bandwidth in some places, where the original slice contents wasn't needed
after slize resizing. Switch such places to bytesutil.ResizeNoCopy().

Rename the original bytesutil.Resize() function to bytesutil.ResizeWithCopy() for the sake of improved readability.

Additionally, allocate new slice with `make()` instead of `append()`. This guarantees that the capacity of the allocated slice
exactly matches the requested size. The `append()` could return a slice with bigger capacity as an optimization for further `append()` calls.
This could result in excess memory usage when the returned byte slice was cached (for instance, in lib/blockcache).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-01-25 15:28:42 +02:00
Aliaksandr Valialkin
b5abea0e36
app/vmselect/vmui: make vmui-update 2022-01-24 12:25:00 +02:00
Yury Molodov
196bef8348
vmui: fixed display type switching (#2088)
* fix: correct switch display type

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-21 16:57:14 +02:00
Yury Molodov
034012c80f
vmui: fix time range selector (#2085)
* fix: add date validate for time range

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-21 12:03:41 +02:00
Aliaksandr Valialkin
e8900c90a8
app/vmui: fix last 6 months time range picker
Previously it was incorrectly selecting 6 minutes instead of 6 months.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1960
2022-01-18 23:28:49 +02:00
Aliaksandr Valialkin
f4a9ae28f3
app/vmui: properly calculate graph range for y axis
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2037
2022-01-18 22:23:40 +02:00
Aliaksandr Valialkin
f31776d0e4
app/vmui: reduce the refresh interval during graph scrolling/zooming from 1 second to 300 milliseconds
One second feels too laggy, so let's reduce the refresh interval to 300 milliseconds.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064
2022-01-18 22:23:40 +02:00
Yury Molodov
1d19303f35
fix: remove buffer period (#2078)
* fix: remove buffer period

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the implemented feature

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 22:23:40 +02:00
Yury Molodov
5b286a4728
vmui: correct url encoding (#2067)
* fix: correct encode multi-line queries

* fix: change autocomplete for correct arrows work

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document the bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2039

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 21:32:30 +02:00
Yury Molodov
d4ccd48faa
vmui: correct calc axes limits (#2058)
* fix: correct calc axes limits

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 21:27:26 +02:00
Yury Molodov
e548fc9021
vmui: change layout (#2054)
* fix: change query reset

* feat: replace @codemirror to text field

* feat: switch to Preact from React

* fix: optimize mui imports

* feat: move time selector to Header

* checkout

* fix: remove unused vars

* update package-lock.json

* fix: correct styles

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-18 12:47:01 +02:00
Aliaksandr Valialkin
cc76174387
app/vmselect/promql: properly keep metric names when optimized path is used for aggregate function calculations
For example, `sum(rate(...) keep_metric_names) by (__name__)` didn't leave the original metric name because of this issue.

This is a follup-up for 1bdc71d917

Udates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
2022-01-17 15:33:39 +02:00
Aliaksandr Valialkin
5a1e4a140f
app/vmselect/promql: implement keep_metric_names modifier for transform and rollup functions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949
2022-01-14 04:15:35 +02:00
Aliaksandr Valialkin
53c9a81f2b
app/vmselect/promql: add stale_samples_over_time() function 2022-01-14 01:49:24 +02:00
Aliaksandr Valialkin
0580a58feb
app/vmselect/promql: add support for @ modifier
Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier

Extend the support with the following features:
* Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()`
* Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)`
  returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348
2022-01-13 22:15:19 +02:00
Aliaksandr Valialkin
7d635a2311
app/vmselect/promql: fix limit_offset() test
The test has been broken in addae7fc6a
2022-01-13 16:58:47 +02:00
Yury Molodov
7ee09286ef
vmui: switching to Preact (#2053)
* feat: replace @codemirror to text field

* feat: switch to Preact from React

* fix: optimize mui imports

* fix: remove unused vars

* update package-lock.json

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-01-11 10:34:01 +02:00
Aliaksandr Valialkin
cf5553d18e
app/vmselect/vmui: run make vmui-update after 89ff7b2465 2022-01-03 21:06:34 +02:00
Yury Molodov
85dc41007b
vmui: legend fixes (#1995)
* feat: add a reset query by clicking the logo

* feat: add sequence number for query fields

* feat: invert behavior on the graph's legend

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-23 12:15:21 +02:00
Roman Khavronenko
d1c474c5b3
vmselect: update /query_exemplars placeholder (#2000)
Grafana expects `data` in response to be a slice and logs an err
if it is not:
```
err="[]v1.ExemplarQueryResult: decode slice: expect [ or n, but found , error found in #0 byte of ...||..., bigger context ...||..."
```

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1999
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-12-23 12:08:40 +02:00
Yury Molodov
0952caca24
vmui: add custom start range (#1989)
* feat: add custom start range

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-21 20:22:48 +02:00
Yury Molodov
4c9e563c89
vmui: graph fixes (#1982)
* fix: remove disabling custom step when zooming

* feat: add a dynamic calc of the width of the graph

* fix: add validate y-axis limits

* fix: correct axis limits for value 0

* fix: change logic create time series

* fix: change types for tooltip

* fix: correct points on the line

* fix: change the logic for set graph width

* fix: stop checking the period when auto-refresh is enabled

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-20 17:37:45 +02:00
Aliaksandr Valialkin
b393a16a25
app/vmselect/vmui: make vmui-update 2021-12-20 13:50:55 +02:00
Aliaksandr Valialkin
3396f08874
app/vmselect/promql: follow-up after 177e345d8a
* Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions.
* Simplify their implementation
* Mention these functions in docs/CHANGELOG.md
2021-12-20 13:15:06 +02:00
匠心零度
cc4725843b
add Prometheus semantics function :changes_prometheus、delta_prometheus、increase_prometheus (#1972)
Co-authored-by: lirenzuo <lirenzuo@shein.com>
2021-12-20 13:15:05 +02:00
Aliaksandr Valialkin
c2b0a6109b
app/vmselect/prometheus: typo fix after bf0e3ad2171219b7883b149999c4dc0ac15b9797 2021-12-17 11:52:42 +02:00
Aliaksandr Valialkin
7c3239966e
app/vmselect/vmui: make vmui-update 2021-12-17 11:52:42 +02:00
Aliaksandr Valialkin
d9f075622b
app/vmselect: de-duplicate data exported via /api/v1/export/csv by default
Previously the exported data wasn't de-duplicated.
Now it is possible to export the raw data without deduplication
by passing reduce_mem_usage=1 query arg to /api/v1/export/csv

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1837
2021-12-17 11:52:42 +02:00
Aliaksandr Valialkin
c170841951
app/vmselect/promql: add timestamp_with_name(m[d]) function
This function works the same as `timestamp()`, but doesn't remove source time series names.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388
2021-12-15 23:38:09 +02:00
Yury Molodov
603be150c4
vmui: introduce application mode (#1949)
* feat: add a label for the Query field

* fix: change zoom position

* fix: add description and error code to alerts

* fix: correct logic query history

* fix: correct update query history

* feat: add custom step

* update package-lock.json

* feat: introduce application mode

* build vmui

* Revert "build vmui"

This reverts commit c0e2415550.

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-15 21:34:04 +02:00
Aliaksandr Valialkin
bc3923111b
lib/storage: return dedup interval in milliseconds from GetDedupInterval()
This removes duplicate .Milliseconds() calls after GetDedupInterval() calls.
2021-12-15 13:27:27 +02:00
Aliaksandr Valialkin
cdfe854c9b
lib/storage: explicitly pass dedupInterval to DeduplicateSamples() and deduplicateSamplesDuringMerge()
This improves the code readability and debuggability, since the output of these functions
stops depending on global state.
2021-12-14 20:52:29 +02:00
Yury Molodov
b3389ce26b
vmui: custom step (#1942)
* feat: add a label for the Query field

* fix: change zoom position

* fix: add description and error code to alerts

* fix: correct logic query history

* fix: correct update query history

* feat: add custom step

* update package-lock.json

* docs: document that VMUI now supports overriding of `step` query arg, which is passed to `/api/v1/query_range`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-14 14:52:44 +02:00
Yury Molodov
28655e8eb8
vmui: minor fixes (#1936)
* feat: add a label for the Query field

* fix: change zoom position

* fix: add description and error code to alerts

* fix: correct logic query history

* fix: correct update query history

* app/vmselect/vmui: `make vmui-update`

* docs/CHANGELOG.md: document bugfixes

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-13 13:44:32 +02:00
Aliaksandr Valialkin
3d86e6353b
app/vmselect/vmui: make vmui-update 2021-12-13 13:33:35 +02:00
Yury Molodov
6367edae1d
vmui: add a label for the Query field (#1923)
* feat: add a label for the Query field

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-12 19:10:56 +02:00
Aliaksandr Valialkin
addae7fc6a
app/vmselect/promql: preserve the order of time series passed to limit_offset() function
Previously time series passed to `limit_offset()` were shuffled according to hash for their labels.
This was unexpected behaviour for most users.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1920 and https://github.com/VictoriaMetrics/VictoriaMetrics/issues/951
2021-12-12 18:07:15 +02:00
Yury Molodov
39e29a50f1
vmui: multiple queries (#1916)
* feat: change duration by "enter"

* fix: optimize data processing for chart

* feat: set minimum step to 1ms

* update dependencies

* feat: remove save the last query to local storage

* fix: handle an error in a table with subqueries

* feat: store display type in URL

* Revert "feat: store display type in URL"

This reverts commit ccc242c69a.

* feat: store display type in URL

* refactor: move the time setting to a folder

* refactor: move the query configurator to a folder

* refactor: move the auth settings to a folder

* feat: improve styles

* feat: add multi query

* update package-lock

* feat: add display multiple queries

* feat: add limits for multiple queries

* update dependencies

* feat: add history for multiple queries

* feat: add line type to legend

* feat: change style for switch

* feat: change the logic for axes limits for multiple queries

* update package-lock.json

* update dependencies

* feat: add the filter to legend

* wip

* lib/httpserver: add missing 127.0.0.1 hostname to the logged address for http and pprof server if the address starts with ':'

This allows copy-pasting the url to http server from logs.

* lib/httpserver: add missing 127.0.0.1 hostname to the logged address for http and pprof server if the address starts with ':'

This allows copy-pasting the url to http server from logs.

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-08 16:43:18 +02:00
Aliaksandr Valialkin
54d4a1c959
app/vmselect: accept optional extra_filters[] query args for all the supported Prometheus querying APIs
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863
2021-12-06 17:33:49 +02:00
Aliaksandr Valialkin
df810cfc9a
app/vmselect/vmui: make vmui-update 2021-12-06 10:18:03 +02:00
Aliaksandr Valialkin
120fedf1ac
app/vmselect/vmui: make vmui-update 2021-11-30 01:38:59 +02:00
Yury Molodov
500945499f
vmui: optimize render (#1852)
* feat: change duration by "enter"

* fix: optimize data processing for chart

* feat: set minimum step to 1ms

* update dependencies

* update package-lock

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-11-30 00:57:35 +02:00
Aliaksandr Valialkin
648d788a45
app/vmselect/vmui: make vmui-update 2021-11-29 21:58:11 +02:00
Aliaksandr Valialkin
d5ae9e9619
app/vmselect/vmui: make vmui-update 2021-11-29 12:23:12 +02:00
Michael Fuller
8a94dd36d1
vmselect: in promql evaluation, return bytes requested when rollup memory limiter is unable to satisfy the request (#1838)
Co-authored-by: Michael Fuller <mfuller@digitalocean.com>
2021-11-29 12:14:55 +02:00
Aliaksandr Valialkin
84decc4254
app/vmselect/promql: consistently return zero from deriv(const) 2021-11-17 18:01:58 +02:00
Aliaksandr Valialkin
aa6d11a33f
app/vmselect/promql: add now() function, which returns the current timestamp as a floating-point value in seconds 2021-11-17 16:42:11 +02:00
Aliaksandr Valialkin
1eb756fd11
app/vmselect/promql: arrange function names in the code in alphabetical order
This should simplify code maintenance in the future
2021-11-14 13:54:43 +02:00
Aliaksandr Valialkin
82cf9f26db
app/vmselect/promql: prevent from incorrect calculations for deriv() over multiple samples with identical timestamps 2021-11-12 13:53:30 +02:00
Aliaksandr Valialkin
4fb19fe34b
all: consistently return application/json content-type without charset=utf-8
The `application/json` content-type has utf-8 encoding by default.
See https://stackoverflow.com/questions/9254891/what-does-content-type-application-json-charset-utf-8-really-mean

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897
2021-11-09 18:07:22 +02:00
Aliaksandr Valialkin
31486f2244
app/vmselect/promql: properly return durations smaller than one second from duration_over_time() function
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780
2021-11-09 11:42:43 +02:00
Yury Molodov
89e0f66bfc
vmui: migration MUI Core v4 to v5 (#1795)
* migration MUI Core v4 to v5

* app/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-11-08 15:46:57 +02:00
Aliaksandr Valialkin
3a1ac218ba
app/vmselect/promql: add duration_over_time(m[d], max_interval) function
This function calculates the actual lifetime of the time series on the given lookbehdind window `d`

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780
2021-11-08 13:14:57 +02:00
Yury Molodov
654686fd40
vmui: fix graph reset (#1788)
* feat: add query history

* fix: change detect keyUp for nav query history

* feat: set default query history

* feat: change graph legend

* update dependencies

* update codemirror version

* fix: correct update period time after zoom/pan

* fix: optimize data processing for the graph

* fix: eliminate memory leaks related to mouse events

* fix: correct display of straight line

* Merge branch 'master' into vmui-fix-reset-graph

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-11-05 20:58:49 +02:00
Aliaksandr Valialkin
be9c7713d7
app/vmselect: make vmui-update 2021-11-04 12:16:55 +02:00
Aliaksandr Valialkin
a102fca4ac
app/vmselect/promql: add limit_offset(limit, offset, q) function, which can be used for paging over big number of time series
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1778
2021-11-03 16:03:10 +02:00
Aliaksandr Valialkin
43bf0333eb
app/vmselect/promql: add label_graphite_group() function for extracting groups from Graphite metric names 2021-11-03 13:20:34 +02:00
Yury Molodov
a16c88095c
vmui: correct migration material-ui (#1758)
* migration material-ui

* fix: rollback popover

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-27 20:43:24 +03:00
Aliaksandr Valialkin
99784b21c1
all: fix build issues and tests for Apple M1
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1653
2021-10-27 15:07:19 +03:00
Yury Molodov
e7591577f0
vmui: update dependencies (#1754)
* update dependencies

* update codemirror version

* app/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-26 19:32:31 +03:00
Aliaksandr Valialkin
d8ce86d685
app/vmselect/promql: reduce the precision from 15 significant digits to 13 significant digits when comparing float64 results in tests
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1738
2021-10-24 13:32:25 +03:00
Yury Molodov
3ccc37a98b
vmui: query history (#1732)
* feat: add query history

* fix: change detect keyUp for nav query history

* feat: set default query history

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-22 12:29:27 +03:00
Yury Molodov
744f6a98eb
vmui: features (#1711)
* feat: initial uPlot graph

* feat: add zoom/pan for graph

* fix: add zoom by ctrl/mac

* fix: remove unused code

* feat: add toggle cache for fetch

* feat: add fix y-axis limits

* fix: stop point events while panning

* fix: change getting cursor position when scaling

* feat: add cursor tooltip to graph

* fix: uninstall chart.js

* fix: change link for create an issue

* fix: set default cache value to true

* app/vmalert: follow-up after 0e2486df56

* docs/CHANGELOG.md: document 5416e18007

* app/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-18 15:19:01 +03:00
Aliaksandr Valialkin
ed994873fd
app/vmselect/promql: randomize the static selection of time series returned from limitk()
Sort series by a hash calculated from the series labels. This should guarantee "random" selection of the returned time series.
Previously the selection could be biased, since time series were sorted alphabetically by label names and label values.
2021-10-16 21:16:34 +03:00
Aliaksandr Valialkin
da0adf446d
app/vmselect/promql: typo fix in comment: didsn't -> didn't 2021-10-16 13:00:10 +03:00
Aliaksandr Valialkin
48c3668b3d
app/vmselect/promql: add atan2 binary operator, which is going to be added in Prometheus 2.31
See https://github.com/prometheus/prometheus/pull/9248
2021-10-11 21:17:23 +03:00
Aliaksandr Valialkin
b3d28f3872
app/vmselect/promql: add missing trigonometric functions, which are going to be added in Prometheus 2.31
See https://github.com/prometheus/prometheus/issues/9233
2021-10-11 21:17:21 +03:00
Aliaksandr Valialkin
f1e1d20ac4
app/vmselect/promql: consistently return the same set of time series from limitk() function
This is the expected behaviour by most users.
2021-10-08 19:55:29 +03:00
Aliaksandr Valialkin
cf87050d83
app/vmui: follow-up after 7bfb44113e
* Run `vmui-update`
* Document the changes in README.md and CHANGELOG.md
2021-10-08 15:11:09 +03:00
Aliaksandr Valialkin
cbf01f7384
app/vmselect/promql: substitute rollupFuncsCannotAdjustWindow with rollupFuncsCanAdjustWindow
The list of functions, which can adjust lookbehind window is more limited than the rest of functions,
so it is better from maintainability and readability PoV using the allowlist instead of blocklist.
2021-10-07 13:19:22 +03:00
Aliaksandr Valialkin
7e6ec0bbe9
app/vmselect/promql: return back the behaviour for deriv() function when the lookbehind window doesnt contain enough points
It is expected that the `deriv(m[d])` returns non-empty value if the lookbehind window `d`
contains less than 2 samples in the same way as `rate()` does.

This is a follow-up after 3e084be06b .
2021-10-07 12:55:58 +03:00
Roman Khavronenko
0a8804d6aa
app/vmselect: make predict_linear and deriv compatible with Prometheus (#1681)
Previously, `predict_linear` returned slightly different results comparing
to Prometheus. The change makes linear regression algorithm compatible
with Prometheus.

`deriv` was excluded from the list of functions which can adjust the time
window for the same reasons.
2021-10-07 12:55:56 +03:00
Aliaksandr Valialkin
ec6eb03d65
app/vmselect/promql: add topk_last and bottomk_last functions 2021-09-30 13:23:27 +03:00
Roman Khavronenko
95ddfda894
app/vmselect: fix binary comparison func (#1667)
The fix makes the binary comparison func to check for NaNs
before executing the actual comparison. This prevents VM
to return values for non-existing samples for expressions
which contain bool comparisons. Please see added test
for example.
2021-09-30 13:04:10 +03:00
Roman Khavronenko
57eb5ef194
app/vmselect: fix testRowsEqual func NaN checks (#1666)
It appeared, that `testRowsEqual` NaN comparison was incorrect.
The fix caused some tests to fail. Please see the change and
tests updated.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-30 13:04:08 +03:00
Yury Molodov
a6037a8ce8
vmui: fixed bug with time range (time zone) (#1661)
* fix: set date in query string in utc format

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-29 01:01:27 +03:00
Roman Khavronenko
444a8882f4
app/vmselect: disable time-window adjustment for min/max_over_time funcs (#1658)
Adjustment results into discrepancy between Prometheus and VM on time windows
smaller than scrape interval.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-29 00:43:54 +03:00
Roman Khavronenko
f46f847531
app/vmselect: always return zero for stddev func if there is only one value (#1659)
The fix will always return zero if received set of items consists of one
element only, which also means no deviation.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-29 00:42:19 +03:00
Roman Khavronenko
72468c9d75
app/vmselect: return NaN instead of 0 for empty value sets (#1660)
The change affects `count/stddev/stdvar_over_time` funcs and makes
them to return NaN instead of zero when there is no datapoints
in a time window.
This is needed for improving compatibility with Prometheus.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-29 00:38:24 +03:00
Roman Khavronenko
c8ac8f44b2
app/vmselect: rm quantile_over_time fast-path optimisations (#1662)
The removed fast path optimisations weren't consistent with
`quantile` function behavior and results into discrepancy.
Specifically, results didn't match in cases when:
* 0 < phi > 1;
* values contain only one element.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-29 00:35:59 +03:00
Yury Molodov
80df31b2ee
vmui: use Chart.js as default engine for graph (#1634)
* feat: add Plotly as default engine for graph

* fix: remove unused components

* feat: use Chart.js as default engine graph

* fix: correct styles for loader

* feat: add zoom/pan for chart

* feat: add height for chart

* fix: remove unused code

* fix: remove empty units from duration

* fix: change debounce for pan to 500ms

* fix: add utility for plugins register globally

* fix: optimize render graph

* feat: add buffer data for zoom

* fix: add limits for zoom in/out

* fix: change update data while zooming

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-27 22:27:49 +03:00
Aliaksandr Valialkin
4b3951fd86
app/vmselect/promql: add rollup_scrape_interval(m[d]) function
It calculates the min, max and avg scrape intervals for m over the given lookbehind window d
2021-09-27 19:21:55 +03:00
Aliaksandr Valialkin
eff31c10ec
app/vmselect/promql: follow-up after 526dd93b32
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612
2021-09-27 18:57:03 +03:00
Roman Khavronenko
6061464d80
app/vmselect: quantile func compatiblity with Prometheus (#1646)
* app/vmselect: `quantile` func compatiblity with Prometheus

The `quantile` func was previously calculated by https://github.com/valyala/histogram
package. The result of such calculation was always the closest real value to
requested quantile. While in Prometheus implementation interpolation is used.
Such difference may result into discrepancy in output between Prometheus and
VictoriaMetrics.

This commit adds a Prometheus-like `quantile` function. It also used by other
functions which depend on it, such as `quantiles`, `quantile_over_time`, `median` etc.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* app/vmselect: `quantile` review fixes

* quantile functions were split into multiple to provide
different API for already sorted data;
* float64sPool is used for reducing allocations. Items in pool may have
different sizes, but defining a new pool was complicates due to name collisions;

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-09-27 18:57:02 +03:00
Aliaksandr Valialkin
a746fe8778
app/vmselect/promql: follow-up after 57b3320478 2021-09-24 01:25:04 +03:00
Roman Khavronenko
7ac0559d4f
app/vmselect: make sorting for query result similar to Prometheus (#1647)
* app/vmselect: make sorting for query result similar to Prometheus

Updated sorting allows to get the order of series in result similar or equal
to what Prometheus returns.
The change is needed for compatibility reasons.

* Update app/vmselect/promql/exec_test.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-24 01:25:03 +03:00
Aliaksandr Valialkin
1115c2f235
app/vmselect/promql: align the behavior of or, and and unless operators with on (labels) modifier to Prometheus
Previously VictoriaMetrics could return unexpected result of the right-hand side operand
had multiple time series with the given set of labels mentioned in `on(labels)`.

See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1643
2021-09-24 00:56:35 +03:00
Aliaksandr Valialkin
3f1104ea74
app/vmselect: fix accessing /graphite/* endpoints 2021-09-21 14:06:18 +03:00
n4mine
a5fcc9e356
fix: typo, dddresses -> addresses (#1630) 2021-09-20 14:27:34 +03:00
Aliaksandr Valialkin
76c650bbda app/vmselect/promql: add quantiles_over_time("phiLabel", phi1, ..., phiN, m[d]) function for calculating multiple quantiles at once 2021-09-17 23:35:04 +03:00
Aliaksandr Valialkin
81d24045c2 app/vmselect/promql: add histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) function
This function calculates multiple quantiles over the given buckets at once

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573
2021-09-17 13:39:06 +03:00
Aliaksandr Valialkin
4ed7548ced app/vmselect/promql: optimize quantiles() calculation
Calculate quantiles in one go instead of calculating each quantile individually
2021-09-17 12:38:21 +03:00
f41gh7
348f705d95 follow-up after e2f99214f3e518f79c1c96ff918820a69d350ba1 2021-09-16 13:56:28 +03:00