Roman Khavronenko
fe4934f0ec
app/vmui: send step param for instant queries ( #5639 )
...
The change reverts https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3896
due to reasons explained in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3896#issuecomment-1896704401
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-22 00:25:31 +02:00
Roman Khavronenko
148e14b3f2
app/vmselect: properly calculate start param for queries with too big look-behind window ( #5630 )
...
Properly determine time range search for instant queries with too big look-behind window like `foo[100y]`.
Previously, such queries could return empty responses even if `foo` is present in database.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5553
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-21 23:47:09 +02:00
Aliaksandr Valialkin
2b67944eb4
app/vmselect/graphite: properly handle -N index for the array of N items
...
This is a follow-up for 70cd09e736https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5581
2024-01-17 00:16:37 +02:00
Aliaksandr Valialkin
b3823bc091
app/vmctl: disallow insecure https connections to vm-native-dst-addr and vm-native-src-addr by default
...
It is better from security PoV to disallow insecure https connections
to vm-native-dst-addr and vm-native-src-addr . This also maintains backwards compatibility
with vmctl before the commit 828aca82e9https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5595
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5606
2024-01-17 00:15:50 +02:00
Yury Molodov
9588b9bd19
vmui/vmanomaly: add support models that produce only anomaly_score ( #5594 )
...
* vmui/vmanomaly: add support models that produce only `anomaly_score`
* vmui/vmanomaly: fix display legend
* vmui/vmanomaly: update comment on anomaly threshold
2024-01-17 00:12:43 +02:00
Aliaksandr Valialkin
b7fcdb1985
deployment/docker: update Go builder from Go1.21.5 to Go1.21.6
2024-01-17 00:05:24 +02:00
Aliaksandr Valialkin
84f11a9e6d
app/vmselect/promql: simplify the code after 388d020b7c
...
Add a test, which verifies the correct sorting of float64 slices with NaNs.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5506
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5509
2024-01-16 22:35:51 +02:00
Aliaksandr Valialkin
6ba2fd3312
app/vmselect/promql: follow-up for ce4f26db02
...
- Document the bugfix at docs/CHANGELOG.md
- Filter out NaN values before sorting as suggested at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5509#discussion_r1447369218
- Revert unrelated changes in lib/filestream and lib/fs
- Use simpler test at app/vmselect/promql/exec_test.go
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5509
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5506
2024-01-16 22:13:13 +02:00
Zongyang
cb37df5723
FIX bottomk doesn't return any data when there are no time range overlap between timeseries ( #5509 )
...
* FIX sort order in bottomk
* Add lessWithNaNsReversed for bottomk
* Add ut for TopK
* Move lt from loop
* FIX lint
* FIX lint
* FIX lint
* Mod log format
---------
Co-authored-by: xiaozongyang <xiaozngyang@kanyun.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-01-16 22:12:49 +02:00
Aliaksandr Valialkin
015c0a4d1a
app/vmselect/promql: consistently sort results of a or b query
...
Previously the order of results returned from `a or b` query could change with each request
because the sorting for such query has been disabled in order to satisfy
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763 .
This commit executes `a or b` query as `sortByMetricName(a) or sortByMetricName(b)`.
This makes the order of returned time series consistent across requests,
while maintaining the requirement from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763 ,
e.g. `b` results are consistently put after `a` results.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5393
2024-01-16 22:12:15 +02:00
Aliaksandr Valialkin
825cfdb5ef
app/vmstorage: expose proper types for storage metrics when -metrics.exposeMetadata command-line flag is set
...
This is a follow-up for 326a77c697
2024-01-16 22:03:31 +02:00
Aliaksandr Valialkin
9b449dfadf
app/vmstorage: deregister storage metrics before stopping the storage
...
This prevents from possible nil pointer dereference issues when the storage metrics
are read after the storage is stopped.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
2024-01-16 21:26:03 +02:00
Aliaksandr Valialkin
9e5e514faf
lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop()
...
Previously the was a race condition when the background goroutine still could try collecting metrics
from already stopped resources after returning from pushmetrics.Stop().
Now the pushmetrics.Stop() waits until the background goroutine is stopped before returning.
This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5549
and the commit fe2d9f6646https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
2024-01-16 21:18:22 +02:00
rbizos
62db64e71b
Handling negative index in Graphite groupByNode/aliasByNode ( #5581 )
...
Handeling the error case with -1
Signed-off-by: Raphael Bizos <r.bizos@criteo.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-01-16 20:55:27 +02:00
Aliaksandr Valialkin
d566aa7d78
lib/prompbmarshal: switch to github.com/VictoriaMetrics/easyproto
2024-01-16 20:48:30 +02:00
Aliaksandr Valialkin
063ea8c773
app/vmalert/remotewrite: properly calculate vmalert_remotewrite_dropped_rows_total
...
It was calculating the number of dropped time series instead of the number of dropped samples.
While at it, drop vmalert_remotewrite_dropped_bytes_total metric, since it was inconsistently calculated -
at one place it was calculating raw protobuf-encoded sample sizes, while at another place it was calculating
the size of snappy-compressed prompbmarshal.WriteRequest protobuf message.
Additionally, this metric has zero practical sense, so just drop it in order to reduce the level of confusion.
2024-01-16 20:47:13 +02:00
Aliaksandr Valialkin
f7b589e38a
lib/prompb: switch to github.com/VictoriaMetrics/easyproto
2024-01-16 20:43:09 +02:00
Aliaksandr Valialkin
7d40506744
lib/prompb: change type of Label.Name and Label.Value from []byte to string
...
This makes it more consistent with lib/prompbmarshal.Label
2024-01-16 20:41:37 +02:00
Dmytro Kozlov
b95d6f5f5e
app/vmctl: add insecure skip verify flags for source and destination addresses for native protocol ( #5606 )
...
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5595
2024-01-16 20:29:41 +02:00
hagen1778
2a7207f38a
app/all: follow-up after 84d710beab
...
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-09 13:17:09 +01:00
zhdd99
84d710beab
lib/pushmetrics: fix a panic caused by pushing metrics during the graceful shutdown process of vmstorage nodes. ( #5549 )
...
Co-authored-by: zhangdongdong <zhangdongdong@kuaishou.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-01-09 13:01:03 +01:00
Dmytro Kozlov
37a76de800
app/vmui: fix broken link for the statistic inaccuracy explanation ( #5568 )
...
(cherry picked from commit 105c6b2eb7
2024-01-08 20:15:04 +01:00
Hui Wang
c14e229b20
vmalert: automatically add exported_ prefix for original evaluation… ( #5398 )
...
automatically add `exported_` prefix for original evaluation result label if it's conflicted with external or reserved one,
previously it was overridden.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5161
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 1f477aba41
2023-12-22 16:10:33 +01:00
Aliaksandr Valialkin
12de0d39eb
lib/protoparser/datadogv2: take into account source_type_name field, since it contains useful value such as kubernetes, docker, system, etc.
2023-12-21 23:05:52 +02:00
Aliaksandr Valialkin
62a105d9e9
app/{vminsert,vmagent}: preliminary support for /api/v2/series ingestion from new versions of DataDog Agent
...
This commit adds only JSON support - https://docs.datadoghq.com/api/latest/metrics/#submit-metrics ,
while recent versions of DataDog Agent send data to /api/v2/series in undocumented Protobuf format.
The support for this format will be added later.
Thanks to @AndrewChubatiuk for the initial implementation at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5094
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4451
2023-12-21 20:50:27 +02:00
Roman Khavronenko
4837616df6
app/vmselect: drop rollupDefault function as duplicate ( #5502 )
...
* app/vmselect: drop `rollupDefault` function as duplicate
It is unclear why there are two identical fns `rollupDefault`
and `rollupDistinct`. Dropping one of them.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* Update app/vmselect/promql/rollup.go
* Update app/vmselect/promql/rollup.go
---------
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-12-21 11:23:20 +02:00
Aliaksandr Valialkin
3a9cf13aaa
app/{vmagent,vmalert}: add the ability to set OAuth2 endpoint params via the corresponding *.oauth2.endpointParams command-line flags
...
This is a follow-up for 5ebd5a0d7bhttps://github.com/VictoriaMetrics/VictoriaMetrics/pull/5427
2023-12-20 21:38:16 +02:00
Morgan
64e96fccd9
Expose OAuth2 Endpoint Parameters to cli ( #5427 )
...
The user may which to control the endpoint parameters for instance to
set the audience when requesting an access token. Exposing the
parameters as a map allows for additional use cases without requiring
modification.
2023-12-20 21:38:13 +02:00
Aliaksandr Valialkin
c888d76c4b
app/vmselect/netstorage: make sure that at least a single result is collected from every storage group before deciding whether it is OK to skip results from the remaining storage nodes
2023-12-20 19:53:49 +02:00
Aliaksandr Valialkin
261c173f4b
all: use Gauge instead of Counter for *_config_last_reload_successful metrics
...
This allows exposing the correct TYPE metadata for these labels when the app runs with -metrics.exposeMetadata command-line flag.
See https://github.com/VictoriaMetrics/metrics/pull/61#issuecomment-1860085508 for more details.
This is follow-up for 326a77c697
2023-12-20 14:25:44 +02:00
Yury Molodov
d0b047a2bf
vmui: add vmanomaly explorer ( #5401 )
2023-12-20 14:15:25 +02:00
Roman Khavronenko
bcb2b8247c
vmctl: rename vm-native-disable-retries to vm-native-disable-per-metric-migration ( #5476 )
...
The change supposed to better reflect the meaning of this flag.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-12-17 19:19:22 +02:00
Roman Khavronenko
5f14fa94dd
vmctl: retry requests that failed in the very end for vm-native ( #5475 )
...
Before, retries happened only on writes into a network connection
between source and destination. But errors returned by server after
all the data was transmitted were logged, but not retried.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 664fa5cb78
2023-12-15 11:54:08 +01:00
Hui Wang
ed4f77575f
vmalert: validate schema for -external.url ( #5450 )
...
Requests with wrong or no schema in `-external.url` could be rejected by alertmanager.
So we validate schema on start up.
(cherry picked from commit 9253c24dd6
2023-12-15 11:54:07 +01:00
Dima Lazerka
44c113f829
VMUI: Handle unknown query error response type ( #5451 )
...
* VMUI: Handle unknown query error response type
* vmui: add error text for unknown error type
* Simplify nested `if`s for unknown error
Accepting @Loori-R's suggestion
Co-authored-by: Yury Molodov <yurymolodov@gmail.com>
---------
Co-authored-by: Yury Moladau <yurymolodov@gmail.com>
(cherry picked from commit cd277e3f84
2023-12-15 11:54:07 +01:00
Aliaksandr Valialkin
42629dead1
app/vmstorage: addd missing -inmemoryDataFlushInterval command-line flag
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
2023-12-14 20:47:49 +02:00
Anton Tykhyy
51af1dfff7
Fix sum(aggr_over_time) 'got 1 args' error ( #3028 ) ( #5414 )
...
app/vmselect/promql/eval.go:evalAggrFunc shunts evaluation
of AggrFuncExpr over rollupFunc over MetricsExpr to an optimized
path. tryGetArgRollupFuncWithMetricExpr() checks whether expression
can be shunted, but it mangles the AggrFuncExpr when the aggregation
function has more than one argument. This results in queries like
`sum(aggr_over_time("avg_over_time",m))` failing with error message
'expecting at least 2 args to "aggr_over_time"; got 1 args' while
the analogous query `sum(avg_over_time(m))` executes successfully.
This fix removes the unnecessary mangling.
Signed-off-by: Anton Tykhyy <atykhyy@gmail.com>
2023-12-14 12:49:01 +02:00
Aliaksandr Valialkin
4ee42e9e73
app/vmauth: allow specifying an empty retry_status_codes and and zero drop_src_path_prefix_parts in order to override user-level setting
...
Previously `retry_status_codes: []` and `drop_src_path_prefix_parts: 0` at `url_map` were equivalent to missing values.
This was resulting in using the user-level values instead.
2023-12-14 01:06:50 +02:00
Aliaksandr Valialkin
51acf0179c
app/vmauth: add ability to route requests to different backends depending on the request host
2023-12-14 00:47:00 +02:00
Yury Molodov
e76c44c5b4
vmui: autocomplete usability improvements ( #5422 )
...
* vmui: add show quick tip for autocomplete
* vmui: auto-completion usability improvements #5348
* vmui: add const for min symbols in autocomplete
* Use proper queries to VictoriaMetrics
* vmui: fix comments for autocomplete
* app/vmselect: run `make vmui-update`
---------
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-12-13 00:33:27 +02:00
Aliaksandr Valialkin
e4bb2808f1
app/vmselect: add support for vmstorage groups with independent -replicationFactor per group
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5197
See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#vmstorage-groups-at-vmselect
Thanks to @zekker6 for the initial pull request at https://github.com/VictoriaMetrics/VictoriaMetrics-enterprise/pull/718
2023-12-13 00:14:34 +02:00
hagen1778
3b841fe9ce
app/vmctl: follow-up after 6af732b6f7
...
Make docs more clear about new feature.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 242472086b
2023-12-12 13:45:35 +01:00
Dmytro Kozlov
63fc200f16
app/vmctl: enable range steps in reverse order ( #5444 )
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5376
(cherry picked from commit 6af732b6f7
2023-12-12 13:45:35 +01:00
hagen1778
307bcb8d4d
app/vmctl: follow-up after 27668c9d01
...
* remove duplications in error messages
* mention the change in CHANGELOG.md
27668c9d0139c405ed4d
2023-12-11 15:38:22 +01:00
wozz
61f940591f
vmctl: check error in response from influxdb ( #5446 )
...
(cherry picked from commit 27668c9d01
2023-12-11 15:38:20 +01:00
Aliaksandr Valialkin
842aba3f46
deployment/docker: update base Docker image from alpine:3.18.5 to alpine:3.19.0
...
See https://www.alpinelinux.org/posts/Alpine-3.19.0-released.html
2023-12-10 02:28:31 +02:00
Aliaksandr Valialkin
3d6517b05e
app/vmselect: add -search.maxResponseSeries command-line flag for limiting the number of time series a single response can return
...
This limit can be used for preventing from high memory usage at Grafana when the response returns too many series.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5372
2023-12-10 00:54:32 +02:00
Aliaksandr Valialkin
55eb48f5ee
app: make more clear that -tls enables https at -httpListenAddr
2023-12-10 00:25:23 +02:00
Aliaksandr Valialkin
49552eaa15
app/vmauth: add support for hot standby mode via first_available load balancing policy
...
vmauth in `hot standby` mode sends requests to the first url_prefix while it is available.
If the first url_prefix becomes unavailable, then vmauth falls back to the next url_prefix.
This allows building highly available setup as described at https://docs.victoriametrics.com/vmauth.html#high-availability
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4792
2023-12-08 23:32:10 +02:00
Roman Khavronenko
276e9301f4
app/vmalert: sanitize label names before sending to Alertmanager ( #5442 )
...
Before, vmalert would send notifications with labels containing characters
not supported by Alertmanager validator, resulting into validation errors
like `msg="Failed to validate alerts" err="invalid label set: invalid name "foo.bar"`
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-12-08 18:09:07 +02:00
