Commit Graph

6263 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
9fa3f1dc57
app/vmselect/promql: do not extend too short lookbehind window for rate() function if it is set explicitly
Previously too short lookbehind window d for rate(m[d]) could be automatically extended
if it didn't cover at least two raw samples. This was needed in order to guarantee
non-empty results from rate(m[d]) on short time ranges.

Now the lookbehind window isn't extended if it is set explicitly,
since it is expected that the user knows what he is doing.

The lookbehind window continues to be extended when needed if it isn't set explicitly.
For example, in the case of rate(m).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3483
2022-12-20 00:18:20 -08:00
Aliaksandr Valialkin
eabb8762ee
docs/Articles.md: add a link to https://www.youtube.com/watch?v=Mesc6JBFNhQ 2022-12-19 21:40:51 -08:00
Michal Kralik
fd53f86c84
build: fix issue with missing docker scan (#3501) 2022-12-19 15:22:45 -08:00
Aliaksandr Valialkin
680925f872
docs/CHANGELOG.md: add a warning for releases between v1.83.0 and v1.85.1 that it is recommended upgrading to v1.85.2 because of the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3502
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3502
2022-12-19 13:36:49 -08:00
Aliaksandr Valialkin
d2ad184377
docs/CHANGELOG.md: consistently use YYYY-MM-DD format for release dates
The previously used DD-MM-YYYY format could be confused with the MM-DD-YYYY format.
The YYYY-MM-DD format reduces this confusion.
2022-12-19 13:33:05 -08:00
Aliaksandr Valialkin
944effca54
lib/storage: do not check for the result returned by db.doExtDB() where this isn't necessary
This simplifies the code a bit
2022-12-19 13:23:13 -08:00
Aliaksandr Valialkin
6530344a8f
docs/CHANGELOG.md: cut v1.85.2 2022-12-19 13:09:29 -08:00
Aliaksandr Valialkin
9a0308ab32
vendor: make vendor-update 2022-12-19 13:08:13 -08:00
Aliaksandr Valialkin
0bf3ae9559
lib/promscrape/discovery/consul: expose service tags in individual labels __meta_consul_tag_<tagname>
This simplifies copying service tags to target labels with the following relabeling rule:

- action: labelmap
  regex: __meta_consul_tag_(.+)

See https://stackoverflow.com/questions/44339461/relabeling-in-prometheus
2022-12-19 13:08:11 -08:00
Aliaksandr Valialkin
6c98b56935
lib/storage: search for TSIDs for the given metricIDs in the previous indexdb if they aren't found in the current indexdb
The issue triggers after the indexdb rotation for time series, which stop receiving new samples.
This results in missing data for such time series in query responses.

This commit should address the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3502

The issue has been introduced in 2dd93449d8
2022-12-19 12:03:09 -08:00
Aliaksandr Valialkin
dc0b08efb0
lib/storage: optimize partSearch.searchBHS() for common case when the TSID for the current block header is bigger or equal to the current tsid
This should help improving performance at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3425
2022-12-19 10:28:03 -08:00
Aliaksandr Valialkin
057fb2120b
lib/storage: properly set buf capacity inside marshalMetricID
Previously it was always set to 0. In theory this could result into incorrect marshaling
of metricIDs.

The issue has been introduced in 5e4dfe50c6
2022-12-19 10:14:38 -08:00
Aliaksandr Valialkin
4cb83f0f4a
lib/logger: follow-up for 72f8fce107
- Document the change at docs/CHANELOG.md
- Log fatal errors if the -loggerJSONFields contains unexpected values
- Rename -loggerJsonFields to -loggerJSONFields for the sake of consistency naming commonly used in Go

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2348
2022-12-16 17:42:07 -08:00
Michal Kralik
72f8fce107
lib/logger: support for renaming json fields (#3488) 2022-12-16 17:26:32 -08:00
Aliaksandr Valialkin
56a9ea3753
docs/vmalert.md: mention latency_offset query arg, which has been added in 86dae56bd0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3481
2022-12-16 17:20:37 -08:00
Aliaksandr Valialkin
285841bcce
app/vmselect/prometheus: follow-up after 86dae56bd0
Return error if the provided latency_offset query arg cannot be parsed.
This should simplify debugging in production.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3481
2022-12-16 17:13:20 -08:00
Aliaksandr Valialkin
65f8fc527f
lib/promscrape: stop dropping metric name if relabeling rules do not instruct to do this on the /metric-relabel-debug page 2022-12-16 17:02:41 -08:00
Roman Khavronenko
86dae56bd0
vmselect: support overriding of -search.latencyOffset (#3489)
support overriding of `-search.latencyOffset` value via
URL param `latency_offset`.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3481

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-16 16:54:57 -08:00
Michal Kralik
07e9322157
build: nightly builds at 2:48am (#3490) 2022-12-16 16:46:24 -08:00
Roman Khavronenko
e40c7d6efa
dashboards: respect $job var in sub-vars for cluster dash (#3487)
Previously, $job_select, $job_storage and $job_insert
didn't respect the $job filter. This change updates
the variable queries to account for set $job variable.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-16 09:53:32 +01:00
Aliaksandr Valialkin
17a244571b
docs/keyConcepts.md: update the list of supported data ingestion protocols
- Add DataDog protocol
- Remove native protocol, since it isn't intended for general-purpose usage by external clients
2022-12-15 12:01:59 -08:00
Aliaksandr Valialkin
ad8852759d
lib/storage: skip missing tsids in the current block header by using binary search
This improves performance by up to 10x when big number of the requested TSIDs
are missing in the searched parts.

This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3425
2022-12-14 22:06:51 -08:00
Aliaksandr Valialkin
4de9d35458
lib/flagutil/bytes.go: properly handle values bigger than 2GiB on 32-bit architectures
This fixes handling of values bigger than 2GiB for the following command-line flags:

- -storage.minFreeDiskSpaceBytes
- -remoteWrite.maxDiskUsagePerURL
2022-12-14 19:26:31 -08:00
Aliaksandr Valialkin
5d30080555
lib/flagutil: support for TB and TiB suffixes for command-line flags, which accept byte sizes 2022-12-14 17:52:32 -08:00
Aliaksandr Valialkin
38341802c2
app/vmselect: add /expand-with-exprs page 2022-12-14 16:18:55 -08:00
Aliaksandr Valialkin
1896c47fee
.wwhrd.yml: add ISC license, which is used by github.com/davecgh/go-spew
This license is compatible with Apache2

See https://github.com/davecgh/go-spew/blob/master/LICENSE
2022-12-14 14:55:46 -08:00
Aliaksandr Valialkin
231569f89b
docs/vmagent.md: small formatting fix 2022-12-14 14:25:35 -08:00
Aliaksandr Valialkin
0c54ff20eb
docs/CHANGELOG.md: fix the link to the issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3466 2022-12-14 14:17:17 -08:00
Aliaksandr Valialkin
76445bfd98
docs/CHANGELOG.md: add release date for v1.85.1 2022-12-14 14:12:07 -08:00
Aliaksandr Valialkin
09a70d3e90
vendor: make vendor-update 2022-12-14 12:13:54 -08:00
Aliaksandr Valialkin
ac5948b3f3
app/vmselect/vmui: make vmui-update 2022-12-14 12:01:48 -08:00
Aliaksandr Valialkin
ea44c39377
docs/CHANGELOG.md: cut v1.85.1 2022-12-14 11:57:49 -08:00
Aliaksandr Valialkin
e0009ec466
docs/Cluster-VictoriaMetrics.md: mention the vm_storage_is_read_only metric, which can help debugging readonly mode at vmstorage 2022-12-14 09:31:28 -08:00
Aliaksandr Valialkin
7a61bafe59
docs/vmagent.md: clarify that relabeling is actually a debugging at relabel debug section 2022-12-13 15:46:21 -08:00
Aliaksandr Valialkin
b89d862aa5
docs/CHANGELOG.md: document the bugfix at a50120a212 2022-12-13 09:36:24 -08:00
Zakhar Bessarab
a50120a212
lib/backup/azremote: fix copying for parts larger than 256M by using async copy (#3479)
* lib/backup/azremote: fix copying for parts larger than 256M by using async copy

* lib/backup/azremote: add description of an error for log message
2022-12-13 09:32:57 -08:00
Yury Molodov
d3418bafc0
fix: prevent run query when selecting autocomplete option (#3480) 2022-12-13 09:30:05 -08:00
Aliaksandr Valialkin
0d41d933e9
lib/mergeset: reduce the parts threshold before starting assisted merges
This should improve query speed in general case.

This is a follow-up for d1af6046c7
2022-12-13 09:13:49 -08:00
Aliaksandr Valialkin
33bff00890
app/vmselect/vmui: make vmui-update 2022-12-12 17:46:30 -08:00
Yury Molodov
cca3bc756b
vmui: minor enhancements (#3471)
* update package-lock.json

* fix: correct handle click by "action" on cardinality page

* fix: correct styles for icons width

* feat: add layout with copyright

* feat: add website and issue to footer
2022-12-12 17:44:13 -08:00
Dima Lazerka
bde93844de
Add Anomaly Detection to enterprise features list (#3476)
* Add Anomaly Detection to enterprise features list

* Update docs/enterprise.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-12-12 17:42:44 -08:00
Aliaksandr Valialkin
d1af6046c7
lib/{mergeset,storage}: do not block small merges by pending big merges - assist with small merges instead
Blocked small merges may result into big number of small parts, which, in turn,
may result in increased CPU and memory usage during queries, since queries need to inspect
all the existing small parts.

The issue has been introduced in 8189770c50

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
2022-12-12 17:00:50 -08:00
Aliaksandr Valialkin
3b18931050
lib/bytesutil: cache results for all the input strings, which were passed during the last 5 minutes from FastStringMatcher.Match(), FastStringTransformer.Transform() and InternString()
Previously only up to 100K results were cached.
This could result in sub-optimal performance when more than 100K unique strings were actually used.
For example, when the relabeling rule was applied to a million of unique Graphite metric names
like in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3466

This commit should reduce the long-term CPU usage for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3466
after all the unique Graphite metrics are registered in the FastStringMatcher.Transform() cache.

It is expected that the number of unique strings, which are passed to FastStringMatcher.Match(),
FastStringTransformer.Transform() and to InternString() during the last 5 minutes,
is limited, so the function results fit memory. Otherwise OOM crash can occur.
This should be the case for typical production workloads.
2022-12-12 14:41:13 -08:00
Zakhar Bessarab
ebcb4ab617
{app/{vmbackup/vmrestore}: update path example to use Azure terminology for consistency (#3475) 2022-12-12 22:17:22 +03:00
Roman Khavronenko
8286f9608f
vmalert: support $for or .For template variables (#3474)
support `$for` or `.For` template variables  in alert's annotations.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3246

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-12 22:16:10 +03:00
Roman Khavronenko
eb275be99d
dashboards: add VersionChange annotation (#3473)
The new annotation is hidden by default and suppose to show
component `short_version` label change on the panels.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-12 16:32:26 +01:00
Aliaksandr Valialkin
7ae744fce6
lib/protoparser/datadog: do not re-use previously parsed field values if they are missing in the currently parsed message
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3432
2022-12-11 13:09:25 -08:00
Aliaksandr Valialkin
4a4b3c2462
vendor: update github.com/klauspost/compress from v1.15.12 to v1.15.13 2022-12-11 02:10:51 -08:00
Aliaksandr Valialkin
9f642d10ff
docs/CHANGELOG.md: cut v1.85.0 2022-12-11 02:01:11 -08:00
Aliaksandr Valialkin
38f8e8adc3
docs/CHANGELOG.md: document changes at v1.79.6 2022-12-11 01:51:36 -08:00