Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-15 08:23:34 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
7,468 Commits 114 Branches 748 Tags 504 MiB
84f11a9e6d
Commit Graph

2986 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
84f11a9e6d
app/vmselect/promql: simplify the code after 388d020b7c 
Add a test, which verifies the correct sorting of float64 slices with NaNs.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5506
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5509
 2024-01-16 22:35:51 +02:00
Aliaksandr Valialkin
6ba2fd3312
app/vmselect/promql: follow-up for ce4f26db02 
- Document the bugfix at docs/CHANGELOG.md
- Filter out NaN values before sorting as suggested at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5509#discussion_r1447369218
- Revert unrelated changes in lib/filestream and lib/fs
- Use simpler test at app/vmselect/promql/exec_test.go

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5509
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5506
 2024-01-16 22:13:13 +02:00
Zongyang
cb37df5723
FIX bottomk doesn't return any data when there are no time range overlap between timeseries (#5509) 
* FIX sort order in bottomk

* Add lessWithNaNsReversed for bottomk

* Add ut for TopK

* Move lt from loop

* FIX lint

* FIX lint

* FIX lint

* Mod log format

---------

Co-authored-by: xiaozongyang <xiaozngyang@kanyun.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-01-16 22:12:49 +02:00
Aliaksandr Valialkin
015c0a4d1a
app/vmselect/promql: consistently sort results of a or b query 
Previously the order of results returned from `a or b` query could change with each request
because the sorting for such query has been disabled in order to satisfy
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763 .

This commit executes `a or b` query as `sortByMetricName(a) or sortByMetricName(b)`.
This makes the order of returned time series consistent across requests,
while maintaining the requirement from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763 ,
e.g. `b` results are consistently put after `a` results.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5393
 2024-01-16 22:12:15 +02:00
Aliaksandr Valialkin
825cfdb5ef
app/vmstorage: expose proper types for storage metrics when -metrics.exposeMetadata command-line flag is set 
This is a follow-up for 326a77c697
 2024-01-16 22:03:31 +02:00
Aliaksandr Valialkin
9b449dfadf
app/vmstorage: deregister storage metrics before stopping the storage 
This prevents from possible nil pointer dereference issues when the storage metrics
are read after the storage is stopped.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
 2024-01-16 21:26:03 +02:00
Aliaksandr Valialkin
9e5e514faf
lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop() 
Previously the was a race condition when the background goroutine still could try collecting metrics
from already stopped resources after returning from pushmetrics.Stop().
Now the pushmetrics.Stop() waits until the background goroutine is stopped before returning.

This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5549
and the commit fe2d9f6646 .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
 2024-01-16 21:18:22 +02:00
rbizos
62db64e71b
Handling negative index in Graphite groupByNode/aliasByNode (#5581) 
Handeling the error case with -1

Signed-off-by: Raphael Bizos <r.bizos@criteo.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
 2024-01-16 20:55:27 +02:00
Aliaksandr Valialkin
d566aa7d78
lib/prompbmarshal: switch to github.com/VictoriaMetrics/easyproto 2024-01-16 20:48:30 +02:00
Aliaksandr Valialkin
063ea8c773
app/vmalert/remotewrite: properly calculate vmalert_remotewrite_dropped_rows_total 
It was calculating the number of dropped time series instead of the number of dropped samples.

While at it, drop vmalert_remotewrite_dropped_bytes_total metric, since it was inconsistently calculated -
at one place it was calculating raw protobuf-encoded sample sizes, while at another place it was calculating
the size of snappy-compressed prompbmarshal.WriteRequest protobuf message.
Additionally, this metric has zero practical sense, so just drop it in order to reduce the level of confusion.
 2024-01-16 20:47:13 +02:00
Aliaksandr Valialkin
f7b589e38a
lib/prompb: switch to github.com/VictoriaMetrics/easyproto 2024-01-16 20:43:09 +02:00
Aliaksandr Valialkin
7d40506744
lib/prompb: change type of Label.Name and Label.Value from []byte to string 
This makes it more consistent with lib/prompbmarshal.Label
 2024-01-16 20:41:37 +02:00
Dmytro Kozlov
b95d6f5f5e
app/vmctl: add insecure skip verify flags for source and destination addresses for native protocol (#5606) 
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5595
 2024-01-16 20:29:41 +02:00
hagen1778
2a7207f38a
app/all: follow-up after 84d710beab 
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5548
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-09 13:17:09 +01:00
zhdd99
84d710beab
lib/pushmetrics: fix a panic caused by pushing metrics during the graceful shutdown process of vmstorage nodes. (#5549) 
Co-authored-by: zhangdongdong <zhangdongdong@kuaishou.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
 2024-01-09 13:01:03 +01:00
Dmytro Kozlov
37a76de800
app/vmui: fix broken link for the statistic inaccuracy explanation (#5568) 
(cherry picked from commit 105c6b2eb7)
 2024-01-08 20:15:04 +01:00
Hui Wang
c14e229b20
vmalert: automatically add exported_ prefix for original evaluation… (#5398) 
automatically add `exported_` prefix for original evaluation result label if it's conflicted with external or reserved one,
previously it was overridden.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5161

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 1f477aba41)
 2023-12-22 16:10:33 +01:00
Aliaksandr Valialkin
12de0d39eb
lib/protoparser/datadogv2: take into account source_type_name field, since it contains useful value such as kubernetes, docker, system, etc. 2023-12-21 23:05:52 +02:00
Aliaksandr Valialkin
62a105d9e9
app/{vminsert,vmagent}: preliminary support for /api/v2/series ingestion from new versions of DataDog Agent 
This commit adds only JSON support - https://docs.datadoghq.com/api/latest/metrics/#submit-metrics ,
while recent versions of DataDog Agent send data to /api/v2/series in undocumented Protobuf format.
The support for this format will be added later.

Thanks to @AndrewChubatiuk for the initial implementation at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5094

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4451
 2023-12-21 20:50:27 +02:00
Roman Khavronenko
4837616df6
app/vmselect: drop rollupDefault function as duplicate (#5502) 
* app/vmselect: drop `rollupDefault` function as duplicate

It is unclear why there are two identical fns `rollupDefault`
and `rollupDistinct`. Dropping one of them.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update app/vmselect/promql/rollup.go

* Update app/vmselect/promql/rollup.go

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2023-12-21 11:23:20 +02:00
Aliaksandr Valialkin
3a9cf13aaa
app/{vmagent,vmalert}: add the ability to set OAuth2 endpoint params via the corresponding *.oauth2.endpointParams command-line flags 
This is a follow-up for 5ebd5a0d7b

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5427
 2023-12-20 21:38:16 +02:00
Morgan
64e96fccd9
Expose OAuth2 Endpoint Parameters to cli (#5427) 
The user may which to control the endpoint parameters for instance to
set the audience when requesting an access token. Exposing the
parameters as a map allows for additional use cases without requiring
modification.
 2023-12-20 21:38:13 +02:00
Aliaksandr Valialkin
c888d76c4b
app/vmselect/netstorage: make sure that at least a single result is collected from every storage group before deciding whether it is OK to skip results from the remaining storage nodes 2023-12-20 19:53:49 +02:00
Aliaksandr Valialkin
261c173f4b
all: use Gauge instead of Counter for *_config_last_reload_successful metrics 
This allows exposing the correct TYPE metadata for these labels when the app runs with -metrics.exposeMetadata command-line flag.
See https://github.com/VictoriaMetrics/metrics/pull/61#issuecomment-1860085508 for more details.

This is follow-up for 326a77c697
 2023-12-20 14:25:44 +02:00
Yury Molodov
d0b047a2bf
vmui: add vmanomaly explorer (#5401) 2023-12-20 14:15:25 +02:00
Roman Khavronenko
bcb2b8247c
vmctl: rename vm-native-disable-retries to vm-native-disable-per-metric-migration (#5476) 
The change supposed to better reflect the meaning of this flag.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-17 19:19:22 +02:00
Roman Khavronenko
5f14fa94dd
vmctl: retry requests that failed in the very end for vm-native (#5475) 
Before, retries happened only on writes into a network connection
between source and destination. But errors returned by server after
all the data was transmitted were logged, but not retried.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 664fa5cb78)
 2023-12-15 11:54:08 +01:00
Hui Wang
ed4f77575f
vmalert: validate schema for -external.url (#5450) 
Requests with wrong or no schema in  `-external.url` could be rejected by alertmanager.
So we validate schema on start up.

(cherry picked from commit 9253c24dd6)
 2023-12-15 11:54:07 +01:00
Dima Lazerka
44c113f829
VMUI: Handle unknown query error response type (#5451) 
* VMUI: Handle unknown query error response type

* vmui: add error text for unknown error type

* Simplify nested `if`s for unknown error

Accepting @Loori-R's suggestion

Co-authored-by: Yury Molodov <yurymolodov@gmail.com>

---------

Co-authored-by: Yury Moladau <yurymolodov@gmail.com>
(cherry picked from commit cd277e3f84)
 2023-12-15 11:54:07 +01:00
Aliaksandr Valialkin
42629dead1
app/vmstorage: addd missing -inmemoryDataFlushInterval command-line flag 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
 2023-12-14 20:47:49 +02:00
Anton Tykhyy
51af1dfff7
Fix sum(aggr_over_time) 'got 1 args' error (#3028) (#5414) 
app/vmselect/promql/eval.go:evalAggrFunc shunts evaluation
of AggrFuncExpr over rollupFunc over MetricsExpr to an optimized
path. tryGetArgRollupFuncWithMetricExpr() checks whether expression
can be shunted, but it mangles the AggrFuncExpr when the aggregation
function has more than one argument. This results in queries like
`sum(aggr_over_time("avg_over_time",m))` failing with error message
'expecting at least 2 args to "aggr_over_time"; got 1 args' while
the analogous query `sum(avg_over_time(m))` executes successfully.
This fix removes the unnecessary mangling.

Signed-off-by: Anton Tykhyy <atykhyy@gmail.com>
 2023-12-14 12:49:01 +02:00
Aliaksandr Valialkin
4ee42e9e73
app/vmauth: allow specifying an empty retry_status_codes and and zero drop_src_path_prefix_parts in order to override user-level setting 
Previously `retry_status_codes: []` and `drop_src_path_prefix_parts: 0` at `url_map` were equivalent to missing values.
This was resulting in using the user-level values instead.
 2023-12-14 01:06:50 +02:00
Aliaksandr Valialkin
51acf0179c
app/vmauth: add ability to route requests to different backends depending on the request host 2023-12-14 00:47:00 +02:00
Yury Molodov
e76c44c5b4
vmui: autocomplete usability improvements (#5422) 
* vmui: add show quick tip for autocomplete

* vmui: auto-completion usability improvements #5348

* vmui: add const for min symbols in autocomplete

* Use proper queries to VictoriaMetrics

* vmui: fix comments for autocomplete

* app/vmselect: run `make vmui-update`

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2023-12-13 00:33:27 +02:00
Aliaksandr Valialkin
e4bb2808f1
app/vmselect: add support for vmstorage groups with independent -replicationFactor per group 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5197

See https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#vmstorage-groups-at-vmselect

Thanks to @zekker6 for the initial pull request at https://github.com/VictoriaMetrics/VictoriaMetrics-enterprise/pull/718
 2023-12-13 00:14:34 +02:00
hagen1778
3b841fe9ce
app/vmctl: follow-up after 6af732b6f7 
Make docs more clear about new feature.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 242472086b)
 2023-12-12 13:45:35 +01:00
Dmytro Kozlov
63fc200f16
app/vmctl: enable range steps in reverse order (#5444) 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5376

(cherry picked from commit 6af732b6f7)
 2023-12-12 13:45:35 +01:00
hagen1778
307bcb8d4d
app/vmctl: follow-up after 27668c9d01 
* remove duplications in error messages
* mention the change in CHANGELOG.md

27668c9d01
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 39c405ed4d)
 2023-12-11 15:38:22 +01:00
wozz
61f940591f
vmctl: check error in response from influxdb (#5446) 
(cherry picked from commit 27668c9d01)
 2023-12-11 15:38:20 +01:00
Aliaksandr Valialkin
842aba3f46
deployment/docker: update base Docker image from alpine:3.18.5 to alpine:3.19.0 
See https://www.alpinelinux.org/posts/Alpine-3.19.0-released.html
 2023-12-10 02:28:31 +02:00
Aliaksandr Valialkin
3d6517b05e
app/vmselect: add -search.maxResponseSeries command-line flag for limiting the number of time series a single response can return 
This limit can be used for preventing from high memory usage at Grafana when the response returns too many series.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5372
 2023-12-10 00:54:32 +02:00
Aliaksandr Valialkin
55eb48f5ee
app: make more clear that -tls enables https at -httpListenAddr 2023-12-10 00:25:23 +02:00
Aliaksandr Valialkin
49552eaa15
app/vmauth: add support for hot standby mode via first_available load balancing policy 
vmauth in `hot standby` mode sends requests to the first url_prefix while it is available.
If the first url_prefix becomes unavailable, then vmauth falls back to the next url_prefix.
This allows building highly available setup as described at https://docs.victoriametrics.com/vmauth.html#high-availability

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4792
 2023-12-08 23:32:10 +02:00
Roman Khavronenko
276e9301f4
app/vmalert: sanitize label names before sending to Alertmanager (#5442) 
Before, vmalert would send notifications with labels containing characters
  not supported by Alertmanager validator, resulting into validation errors
  like `msg="Failed to validate alerts" err="invalid label set: invalid name "foo.bar"`

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2023-12-08 18:09:07 +02:00
Alexander Marshalov
e9cf39f519
added field version to the response for /api/v1/status/buildinfo API for using more efficient API in Grafana for receiving label values, added additional info about setup Grafana datasource (#5370) (#5437) 2023-12-07 16:41:56 +02:00
Aliaksandr Valialkin
32aea90847
app/vmselect/prometheus: go fmt after b39e9257eb 2023-12-07 16:05:01 +02:00
Aliaksandr Valialkin
9f79342e6a
app/vmselect/prometheus: properly encode Prometheus label values at /federate endpoint 
Prometheus spec says that only \, \n and " must be escaped inside label values.
See 995743836e/content/docs/instrumenting/exposition_formats.md (L90)

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5431
 2023-12-07 15:36:50 +02:00
Aliaksandr Valialkin
12e94f10cc
deployment/docker: update Go builder from Go1.21.4 to Go1.21.5 
See https://github.com/golang/go/issues?q=milestone%3AGo1.21.5+label%3ACherryPickApproved
 2023-12-06 22:33:27 +02:00
Dmytro Kozlov
6a41e1ec0c
app/vmalert: replace error metrics for gauges with counter metrics (#5217) 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5160

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 935bec447b)
 2023-12-06 19:41:34 +01:00
Aliaksandr Valialkin
509339bf63
app/vmselect: properly adjust the lower bound for the time range where raw samples must be selected for default_rollup() function 
Previously the lower bound could be too small, which could result in missing values at the beginning of the graph
for default_rollup() function. This function is automatically applied to all the series selectors if they aren't
explicitly wrapped into a rollup function - see https://docs.victoriametrics.com/MetricsQL.html#implicit-query-conversions

While at it, properly take into account `-search.minStalenessInterval` command-line flag when adjusting
the lower bound for the selected time range.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5388
 2023-12-06 14:46:18 +02:00