Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-15 16:30:55 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
5,651 Commits 114 Branches 748 Tags 504 MiB
955e1c1729
Commit Graph

2226 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
3834602c9d
docs/CHANGELOG.md: document 1df87807fd 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3513
 2022-12-22 15:22:21 -08:00
Aliaksandr Valialkin
cc482b89a3
lib/promrelabel: add support for keepequal and dropequal relabeling actions 
These actions are supported by Prometheus starting from v2.41.0

See https://github.com/prometheus/prometheus/pull/11564 ,
https://github.com/prometheus/prometheus/issues/11556
and https://github.com/prometheus/prometheus/issues/3756

Side note:

It's a pity that Prometheus developers decided inventing `keepequal` and `dropequal`
relabeling actions instead of adding support for `keep_if_equal` and `drop_if_equal` relabeling
actions supported by VictoriaMetrics since June 2020 - see 2a39ba639d .
 2022-12-21 20:06:09 -08:00
Yury Molodov
6f17c1e866
vmui: fix change timezone (#3519) 
vmui: fix time picker with changed time zone
 2022-12-21 13:08:42 -08:00
Aliaksandr Valialkin
30b15111aa
docs/CHANGELOG.md: cut v1.85.3 2022-12-20 14:51:42 -08:00
Roman Khavronenko
998d89685f
vmagent: respect -usePromCompatibleNaming if no relabeling is set (#3511) 
* vmagent: respect `-usePromCompatibleNaming` if no relabeling is set

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3493

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmagent: upd test

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-12-20 14:34:29 -08:00
Yury Molodov
88f408941c
fix: change the logic for hide query (#3514) 2022-12-20 14:30:21 -08:00
Zakhar Bessarab
decf46d72b
app/vmbackupmanager: add metrics for better observability (#488) 
* app/vmbackupmanager: add metrics for better observability, include more information to `/api/v1/backups` API call response

* app/vmbackupmanager: drop old metrics before creating new ones

* app/vmbackupmanager: use `_total` postfix for counter metrics

* app/vmbackupmanager: remove `_total` postfix for gauge-like metrics

* app/vmbackupmanager: add `_last_run_failed` metrics for backups and retention

* app/vmbackupmanager: address review feedback

* app/vmbackupmanager: fix metric name

* app/vmbackupmanager: address review feedback, remove background updates of metrics, add restoring state of `_last_run_failed` metric from remote storage

* app/vmbackupmanager: improve performance for backup size calculation

* app/vmbackupmanager: refactor backup and retention runs to deduplicate each run logic

* {app/vmbackupmanager,lib/formatutil}: move HumanizeBytes into lib package

* app/vmbackupmanager: fix creating new metrics instead of reusing existing ones

* lit/formatutil: add comment to make linter happy

* app/vmbackupmanager: address review feedback
 2022-12-20 14:18:43 -08:00
Aliaksandr Valialkin
1ff62629f4
lib/storage: clear the err if it is set to io.EOF when searching for the TSID by metricID 
This is expected error after when recently added indexdb data isn't available for search yet
or wasn't flushed to disk after unclean shutdown of VictoriaMetrics.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3515
 2022-12-20 14:05:53 -08:00
Roman Khavronenko
f2e7e11382
docs: fix link typo in operator docs (#3508) 2022-12-20 10:22:40 -08:00
Aliaksandr Valialkin
874aea98b3
docs/CHANGELOG.md: document the change at 547f07463b29c09c62c9af35eac9cee6764b3286 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2612
 2022-12-20 10:21:22 -08:00
Aliaksandr Valialkin
276e867119
docs/CHANGELOG.md: formatting fix 2022-12-20 01:06:42 -08:00
Aliaksandr Valialkin
7fb314bb59
app/vmselect/promql: do not extend too short lookbehind window for rate() function if it is set explicitly 
Previously too short lookbehind window d for rate(m[d]) could be automatically extended
if it didn't cover at least two raw samples. This was needed in order to guarantee
non-empty results from rate(m[d]) on short time ranges.

Now the lookbehind window isn't extended if it is set explicitly,
since it is expected that the user knows what he is doing.

The lookbehind window continues to be extended when needed if it isn't set explicitly.
For example, in the case of rate(m).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3483
 2022-12-20 00:18:32 -08:00
Aliaksandr Valialkin
4c6c8c50b5
docs/Articles.md: add a link to https://www.youtube.com/watch?v=Mesc6JBFNhQ 2022-12-19 21:41:02 -08:00
Aliaksandr Valialkin
1ed0d07aa7
docs/CHANGELOG.md: add a warning for releases between v1.83.0 and v1.85.1 that it is recommended upgrading to v1.85.2 because of the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3502 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3502
 2022-12-19 13:37:20 -08:00
Aliaksandr Valialkin
cac56859eb
docs/CHANGELOG.md: consistently use YYYY-MM-DD format for release dates 
The previously used DD-MM-YYYY format could be confused with the MM-DD-YYYY format.
The YYYY-MM-DD format reduces this confusion.
 2022-12-19 13:33:09 -08:00
Aliaksandr Valialkin
371f9085c9
docs/CHANGELOG.md: cut v1.85.2 2022-12-19 13:09:48 -08:00
Aliaksandr Valialkin
9330da3195
lib/promscrape/discovery/consul: expose service tags in individual labels __meta_consul_tag_<tagname> 
This simplifies copying service tags to target labels with the following relabeling rule:

- action: labelmap
  regex: __meta_consul_tag_(.+)

See https://stackoverflow.com/questions/44339461/relabeling-in-prometheus
 2022-12-19 13:02:56 -08:00
Aliaksandr Valialkin
11bd290201
lib/storage: search for TSIDs for the given metricIDs in the previous indexdb if they aren't found in the current indexdb 
The issue triggers after the indexdb rotation for time series, which stop receiving new samples.
This results in missing data for such time series in query responses.

This commit should address the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3502

The issue has been introduced in 2dd93449d8
 2022-12-19 11:56:49 -08:00
Aliaksandr Valialkin
70c720d640
lib/logger: follow-up for 72f8fce107 
- Document the change at docs/CHANELOG.md
- Log fatal errors if the -loggerJSONFields contains unexpected values
- Rename -loggerJsonFields to -loggerJSONFields for the sake of consistency naming commonly used in Go

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2348
 2022-12-16 17:42:31 -08:00
Aliaksandr Valialkin
2a229a319e
docs/vmalert.md: mention latency_offset query arg, which has been added in 86dae56bd0 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3481
 2022-12-16 17:20:51 -08:00
Aliaksandr Valialkin
8d7d8f0125
app/vmselect/prometheus: follow-up after 86dae56bd0 
Return error if the provided latency_offset query arg cannot be parsed.
This should simplify debugging in production.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3481
 2022-12-16 17:16:26 -08:00
Roman Khavronenko
f8a3514e99
vmselect: support overriding of -search.latencyOffset (#3489) 
support overriding of `-search.latencyOffset` value via
URL param `latency_offset`.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3481

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-12-16 17:14:23 -08:00
Aliaksandr Valialkin
2e597555ec
lib/promscrape: stop dropping metric name if relabeling rules do not instruct to do this on the /metric-relabel-debug page 2022-12-16 16:44:12 -08:00
Aliaksandr Valialkin
c8942f819f
docs/keyConcepts.md: update the list of supported data ingestion protocols 
- Add DataDog protocol
- Remove native protocol, since it isn't intended for general-purpose usage by external clients
 2022-12-15 12:02:51 -08:00
Aliaksandr Valialkin
1a88fe5b1f
lib/flagutil/bytes.go: properly handle values bigger than 2GiB on 32-bit architectures 
This fixes handling of values bigger than 2GiB for the following command-line flags:

- -storage.minFreeDiskSpaceBytes
- -remoteWrite.maxDiskUsagePerURL
 2022-12-14 19:29:57 -08:00
Aliaksandr Valialkin
3a28a52667
lib/flagutil: support for TB and TiB suffixes for command-line flags, which accept byte sizes 2022-12-14 17:53:18 -08:00
Aliaksandr Valialkin
6d13ca2b88
app/vmselect: add /select/0/prometheus/expand-with-exprs page 2022-12-14 16:23:17 -08:00
Aliaksandr Valialkin
ed7d204274
docs/vmagent.md: small formatting fix 2022-12-14 14:25:46 -08:00
Aliaksandr Valialkin
97a9d2eb04
docs/CHANGELOG.md: fix the link to the issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3466 2022-12-14 14:17:27 -08:00
Aliaksandr Valialkin
5a288e98bf
docs/CHANGELOG.md: add release date for v1.85.1 2022-12-14 14:12:35 -08:00
Aliaksandr Valialkin
e3675d3679
docs/CHANGELOG.md: cut v1.85.1 2022-12-14 11:57:57 -08:00
Aliaksandr Valialkin
7d1d82d094
docs/Cluster-VictoriaMetrics.md: mention the vm_storage_is_read_only metric, which can help debugging readonly mode at vmstorage 2022-12-14 09:32:21 -08:00
Aliaksandr Valialkin
cca16cb498
docs/vmagent.md: clarify that relabeling is actually a debugging at relabel debug section 2022-12-13 15:46:30 -08:00
Aliaksandr Valialkin
03ff3be993
docs/CHANGELOG.md: document the bugfix at a50120a212 2022-12-13 09:36:37 -08:00
Dima Lazerka
b410e53897
Add Anomaly Detection to enterprise features list (#3476) 
* Add Anomaly Detection to enterprise features list

* Update docs/enterprise.md

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-12-12 17:43:16 -08:00
Aliaksandr Valialkin
2a190f6451
lib/{mergeset,storage}: do not block small merges by pending big merges - assist with small merges instead 
Blocked small merges may result into big number of small parts, which, in turn,
may result in increased CPU and memory usage during queries, since queries need to inspect
all the existing small parts.

The issue has been introduced in 8189770c50

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
 2022-12-12 17:01:33 -08:00
Aliaksandr Valialkin
f3e5c9c246
lib/bytesutil: cache results for all the input strings, which were passed during the last 5 minutes from FastStringMatcher.Match(), FastStringTransformer.Transform() and InternString() 
Previously only up to 100K results were cached.
This could result in sub-optimal performance when more than 100K unique strings were actually used.
For example, when the relabeling rule was applied to a million of unique Graphite metric names
like in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3466

This commit should reduce the long-term CPU usage for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3466
after all the unique Graphite metrics are registered in the FastStringMatcher.Transform() cache.

It is expected that the number of unique strings, which are passed to FastStringMatcher.Match(),
FastStringTransformer.Transform() and to InternString() during the last 5 minutes,
is limited, so the function results fit memory. Otherwise OOM crash can occur.
This should be the case for typical production workloads.
 2022-12-12 14:47:00 -08:00
Zakhar Bessarab
600023a8f3
{app/{vmbackup/vmrestore}: update path example to use Azure terminology for consistency (#3475) 2022-12-12 14:43:25 -08:00
Roman Khavronenko
a44af871d3
vmalert: support $for or .For template variables (#3474) 
support `$for` or `.For` template variables  in alert's annotations.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3246

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-12-12 14:42:16 -08:00
Aliaksandr Valialkin
87390443d1
lib/protoparser/datadog: do not re-use previously parsed field values if they are missing in the currently parsed message 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3432
 2022-12-11 13:09:41 -08:00
Aliaksandr Valialkin
77aedb6008
docs/CHANGELOG.md: cut v1.85.0 2022-12-11 02:01:20 -08:00
Aliaksandr Valialkin
c135ea36c7
docs/CHANGELOG.md: document changes at v1.79.6 2022-12-11 01:51:48 -08:00
Aliaksandr Valialkin
2e1a44cfd0
docs/CHANGELOG.md: document 461158a437 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3427
 2022-12-10 23:42:29 -08:00
Aliaksandr Valialkin
7d996ced51
app/{vmagent,vminsert}/datadog: make the host label optional in DataDog data ingestion protocol 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3432
 2022-12-10 23:33:17 -08:00
Aliaksandr Valialkin
808ce815e4
app/vmselect/promql: allow passing inf arg into functions, which accept numeric limit on the number of output time series 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3461
 2022-12-10 22:48:01 -08:00
Aliaksandr Valialkin
afa232eebb
docs/CHANGELOG.md: add a link to the issue related to reduced CPU and memory usage at vmalert 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3464

The related commit - b97bd01605
 2022-12-10 22:21:58 -08:00
Aliaksandr Valialkin
00d953ab66
docs: clarify that single-node VictoriaMetrics also provides functionality for relabel debugging 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407
 2022-12-10 20:50:51 -08:00
Aliaksandr Valialkin
97b41e727c
lib/promscrape: implement target-level and metric-level relabel debugging 
Target-level debugging is performed by clicking the 'debug' link at the corresponding target
on either http://vmagent:8429/targets page or on http://vmagent:8428/service-discovery page.

Metric-level debugging is perfromed at http://vmagent:8429/metric-relabel-debug page.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407

See https://docs.victoriametrics.com/vmagent.html#relabel-debug
 2022-12-10 02:25:56 -08:00
Aliaksandr Valialkin
7c311952aa
docs/url-examples.md: add missing whitespace for proper heading for the example on how to send data via OpenTSDB protocol 2022-12-09 17:33:55 -08:00
Aliaksandr Valialkin
ceed09877e
docs/CHANGELOG.md: document b97bd01605 2022-12-09 11:49:53 -08:00
Aliaksandr Valialkin
04abd5e113
docs/CHANGELOG.md: document the bugfix at 05b42601c3 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3247
 2022-12-08 18:35:47 -08:00
Aliaksandr Valialkin
e56d5e1918
lib/storage: follow-up after 7c0ae3a86a 
- Update docs at https://docs.victoriametrics.com/#deduplication
- Optimize the deduplication loop a bit

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3333
 2022-12-08 18:18:49 -08:00
Roman Khavronenko
909cd04c55
lib/storage: keep sample with the biggest value on timestamp conflict (#3421) 
The change leaves raw sample with the biggest value for identical
timestamps per each `-dedup.minScrapeInterval` discrete interval
when the deduplication is enabled.

```
benchstat old.txt new.txt
name                                         old time/op    new time/op    delta
DeduplicateSamples/minScrapeInterval=1s-10      817ns ± 2%     832ns ± 3%      ~     (p=0.052 n=10+10)
DeduplicateSamples/minScrapeInterval=2s-10     1.56µs ± 1%    2.12µs ± 0%   +35.19%  (p=0.000 n=9+7)
DeduplicateSamples/minScrapeInterval=5s-10     1.32µs ± 3%    1.65µs ± 2%   +25.57%  (p=0.000 n=10+10)
DeduplicateSamples/minScrapeInterval=10s-10    1.13µs ± 2%    1.50µs ± 1%   +32.85%  (p=0.000 n=10+10)

name                                         old speed      new speed      delta
DeduplicateSamples/minScrapeInterval=1s-10   10.0GB/s ± 2%   9.9GB/s ± 3%      ~     (p=0.052 n=10+10)
DeduplicateSamples/minScrapeInterval=2s-10   5.24GB/s ± 1%  3.87GB/s ± 0%   -26.03%  (p=0.000 n=9+7)
DeduplicateSamples/minScrapeInterval=5s-10   6.22GB/s ± 3%  4.96GB/s ± 2%   -20.37%  (p=0.000 n=10+10)
DeduplicateSamples/minScrapeInterval=10s-10  7.28GB/s ± 2%  5.48GB/s ± 1%   -24.74%  (p=0.000 n=10+10)
```

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3333
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-12-08 18:18:36 -08:00
Aliaksandr Valialkin
676de127aa
all: update Go builder from v1.19.3 to v1.19.4 
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.4+label%3ACherryPickApproved
 2022-12-08 17:04:41 -08:00
Aliaksandr Valialkin
a13f16e48a
lib/promscrape: allow using sample_limit and series_limit options in stream parsing mode 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3458
 2022-12-08 17:04:12 -08:00
Aliaksandr Valialkin
81da751e27
docs/Cluster-VictoriaMetrics.md: typo fix 2022-12-07 12:26:45 -08:00
Aliaksandr Valialkin
78582c3e82
docs/CHANGELOG.md: document the addition of file-based discovery of vmstorage nodes 2022-12-07 12:05:08 -08:00
Aliaksandr Valialkin
d009ba1def
docs/Cluster-VictoriaMetrics.md: make docs-sync after 5de8330ce00adfc5ac794070d30a2617ddc14bf2 2022-12-07 12:02:28 -08:00
Aliaksandr Valialkin
255c04bc20
lib/querytracer: put the version of VictoriaMetrics in the first message of query trace 
This should simplify further debugging, since the first thing to start the debugging by query trace
is to know the version of VictoriaMetrics, which produced this trace.
 2022-12-07 09:49:51 -08:00
Aliaksandr Valialkin
8162e4ee3f
docs: follow-up after e1bf2a85d0559d112908ce81597f3261d3a085c0 
- Document the change at docs/CHANGELOG.md
- Run `make docs-sync` for copying app/vmgateway/README.md to docs/vmgateway.md
  in order to propagate docs' changes to https://docs.victoriametrics.com/vmgateway.html
 2022-12-06 21:05:38 -08:00
Aliaksandr Valialkin
6680a17fec
docs: document the addition of -storageNode.discoveryInterval command-line flag 2022-12-06 19:57:46 -08:00
Aliaksandr Valialkin
12307bc888
docs/Articles.md: change the link to How we tried using VictoriaMetrics and Thanos at the same time from Russian to English article 2022-12-06 16:31:12 -08:00
Roman Khavronenko
5bbb88902e
vmalert: correctly return error for RW failures (#3452) 
* vmalert: correctly return error for RW failures

By mistake, in 0989649ad0 the error
for remote write failures weren't return to user.
This change fixes it.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-12-06 16:31:12 -08:00
Aliaksandr Valialkin
72ea7b90d5
docs/Articles.md: add a link to the talk "How do We Keep Metrics for a Long Time in VictoriaMetrics" 2022-12-06 00:55:47 -08:00
Aliaksandr Valialkin
9449b93281
docs/Articles.md: link to a video for the talk 'VictoriaMetrics: scaling to 100 million metrics per second' 2022-12-06 00:53:17 -08:00
Aliaksandr Valialkin
f9b199e9e9
docs/CHANGELOG.md: document 1e0666abb4 2022-12-05 23:10:42 -08:00
Aliaksandr Valialkin
412508c81d
docs: follow-up for 7645d9ae00 
- Document the change at docs/CHANGELOG.md
- Document the feature at https://docs.victoriametrics.com/#vmui

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3446
 2022-12-05 22:57:45 -08:00
Yury Molodov
daaea87931
vmui: timezone select (#3414) 
* feat: add timezone selection

* vmui: provide feature timezone select

* fix: correct timezone with relative time

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-12-05 22:57:16 -08:00
Yury Molodov
08a68a2829
vmui: improve table view (#3377) 
* vmui: add compact table view (#3365)

* feat: add compact table view

* fix: add overflow table

* fix: change table styles

* vmui: compact table view

* Update docs/CHANGELOG.md

Co-authored-by: Michal Kralik <michal.kralik@percona.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-12-05 22:56:53 -08:00
Aliaksandr Valialkin
98110a06d1
docs/CHANGELOG.md: document fd43b5bad0 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3444
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3445
 2022-12-05 22:02:16 -08:00
Aliaksandr Valialkin
c902500efc
app/vmselect/promql: add range_trim_spikes(phi, q) function for trimming phi percent of largest spikes per each time series returned by q 2022-12-05 21:55:24 -08:00
Aliaksandr Valialkin
e465cf5d57
docs/vmctl.md: make docs-sync after 86c31f2955 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2930
 2022-12-05 17:24:37 -08:00
Zakhar Bessarab
c3276e57b4
app/vmctl: add option to migrate between clusters with automatic tenants discovery (#3450) 2022-12-05 17:20:12 -08:00
Aliaksandr Valialkin
d2e34b8052
{dashboards,alerts}: subtitute {type="indexdb"} with {type=~"indexdb.*"} inside queries after 8189770c50 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
 2022-12-05 16:00:42 -08:00
Aliaksandr Valialkin
7d5c64eb7a
all: add -inmemoryDataFlushInterval command-line flag for controlling the frequency of saving in-memory data to disk 
The main purpose of this command-line flag is to increase the lifetime of low-end flash storage
with the limited number of write operations it can perform. Such flash storage is usually
installed on Raspberry PI or similar appliances.

For example, `-inmemoryDataFlushInterval=1h` reduces the frequency of disk write operations
to up to once per hour if the ingested one-hour worth of data fits the limit for in-memory data.

The in-memory data is searchable in the same way as the data stored on disk.
VictoriaMetrics automatically flushes the in-memory data to disk on graceful shutdown via SIGINT signal.
The in-memory data is lost on unclean shutdown (hardware power loss, OOM crash, SIGKILL).

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
 2022-12-05 15:28:09 -08:00
Aliaksandr Valialkin
0b8e7deabd
lib/mergeset: drop the crufty code responsible for direct upgrade from releases prior v1.28.0 
Upgrade to v1.84.0, wait until the "finished round 2 of background conversion" message
appears in the log and then upgrade to newer release.
 2022-12-03 21:18:41 -08:00
Aliaksandr Valialkin
df4eca2807
docs/CHANGELOG.md: consistently add - prefix in front of command-line flags 
This is a follow-up for bcba5d2a78
 2022-12-02 19:14:02 -08:00
Aliaksandr Valialkin
86632658f0
docs: follow-up after 30fea30685 
- Run `make docs-sync`, so app/vmalert/README.md is copied to docs/vmalert.md
- Clarify the feature description in the docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3408
 2022-12-02 19:14:02 -08:00
Denys Holius
5cbe8101c8
docs/Articles.md: fir broken link (#3433) 2022-12-02 19:12:46 -08:00
Roman Khavronenko
2d8ac5355b
docs: fix typo in cluster's README (#3430) 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-12-02 19:11:45 -08:00
Roman Khavronenko
31ca22109e
vmalert: fix replay step param (#3428) 
The recent change in modifying default value
of `datasource.queryStep` flag resulted in situation
where replay mode was always running queries with
step=`datasource.queryStep`. When it should always
use rule's evaluation interval.

The fix is related not to replay mode only, but
for all Range requests. Now step param is set
individually for each mode.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-12-02 19:09:30 -08:00
Zakhar Bessarab
59f889cd3f
app/vmalert: add remoteWrite.sendTimeout command-line flag to configure timeout for sending data to remoteWrite.url (#3423) 
* app/vmalert: add `remoteWrite.sendTimeout` command-line flag to configure timeout for sending data to `remoteWrite.url`

* vmalert: remove WriteTimeout from clients Cfg
No need to have it as a part of configuration struct:
* the client isn't used by other packages;
* there are no internal tests to check the WriteTimeout.

* vmalert: remove DisablePathAppend from clients Cfg
No need to have it as a part of configuration struct:
* the client isn't used by other packages;
* there are no internal tests to check the DisablePathAppend.

Co-authored-by: hagen1778 <roman@victoriametrics.com>
 2022-12-02 19:03:34 -08:00
Roman Khavronenko
435f6f3add
vmalert: properly pass headers during the restore procedure (#3420) 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3418

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-12-02 18:53:44 -08:00
Aliaksandr Valialkin
50e6a137bb
docs/vmagent.md: update after 959f06d175 2022-11-29 21:34:00 -08:00
Aliaksandr Valialkin
be6da5053f
lib/promscrape: optimize service discovery speed 
- Return meta-labels for the discovered targets via promutils.Labels
  instead of map[string]string. This improves the speed of generating
  meta-labels for discovered targets by up to 5x.

- Remove memory allocations in hot paths during ScrapeWork generation.
  The ScrapeWork contains scrape settings for a single discovered target.
  This improves the service discovery speed by up to 2x.
 2022-11-29 21:26:23 -08:00
Aliaksandr Valialkin
8440c822fa
all: follow-up after 05cf8a6ecc 2022-11-29 21:11:42 -08:00
Aliaksandr Valialkin
2a107cc8a7
app/vmalert: substitute -datasource.disablePathAppend with -remoteRead.disablePathAppend in the description for -datasource.url command-line flag 
This is a follow-up for 959f06d175
 2022-11-29 21:11:18 -08:00
Dmytro Kozlov
002c028f22
vmctl: support of the remote read protocol (#3232) 
vmctl: support of the remote read protocol

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
 2022-11-29 21:08:47 -08:00
Max Golionko
d272a8270b
vmalert: flag reference update (#3415) 
* flag reference update

there is no flag `-datasource.disablePathAppend` and datasource actually checking for `-remoteRead.disablePathAppend`

* update source for doc as well
 2022-11-29 20:38:02 -08:00
Zakhar Bessarab
5edaff5165
doc/operator: update formatting for backup section, add FAQ section (#3416) 
* doc/operator: update formatting for backup section, add FAQ section

* doc/operator: address review feedback

* doc/operator: add note about difference between `VMRestore` and `VMBackupmanager` as init containers

* Update docs/operator/backups.MD

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
 2022-11-29 20:28:50 -08:00
Aliaksandr Valialkin
a547226b53
docs/CHANGELOG.md: document 027ab74efb 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3402
 2022-11-28 18:55:17 -08:00
Aliaksandr Valialkin
20ceb3a2ca
docs/url-examples.md: allow linking to how to send OpenTSDB/Graphite data chapters 2022-11-28 18:42:56 -08:00
Aliaksandr Valialkin
8ce5b095b7
lib/promscrape: add exported_ prefix to metric names exported by scrape targets if they clash with automatically generated metrics 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3406
 2022-11-28 18:37:34 -08:00
Aliaksandr Valialkin
ab96fed372
docs/Articles.md: add a link to slides about scaling to 100 million metrics per second 2022-11-28 16:43:58 -08:00
Aliaksandr Valialkin
d0461bec25
docs/CHANGELOG.md: cut v1.84.0 2022-11-25 19:59:30 -08:00
Aliaksandr Valialkin
94ab66e0a4
docs/README.md: make docs-sync after 58d459e8a8 2022-11-25 16:56:04 -08:00
Aliaksandr Valialkin
c5eebaffd8
app/{vminsert,vmagent}: follow-up after 53a63c6c4c 
Extend /api/v1/import/prometheus with the support for Pushgateway way of specifying additional labels.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1415
 2022-11-25 16:42:38 -08:00
Zakhar Bessarab
e407e7243a
{app/vmstorage,app/vmselect}: add API to get list of existing tenants (#3348) 
* {app/vmstorage,app/vmselect}: add API to get list of existing tenants

* {app/vmstorage,app/vmselect}: add API to get list of existing tenants

* app/vmselect: fix error message

* {app/vmstorage,app/vmselect}: fix error messages

* app/vmselect: change log level for error handling

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-11-25 10:32:45 -08:00
Aliaksandr Valialkin
2f7152deb3
app/vmselect/vmui: make vmui-update after eb772aa50e 2022-11-24 17:37:33 -08:00
Aliaksandr Valialkin
6b2c85d6bf
docs/MetricsQL.md: document that histogram_share() accepts optional boundsLabel arg 2022-11-24 17:27:43 -08:00
Aliaksandr Valialkin
c7bd0f5082
docs/vmagent.md: typo fix 2022-11-24 17:19:56 -08:00
Aliaksandr Valialkin
a5e31c9c23
docs/Articles.md: add a link to "The cost of scale in Prometheus ecosystem" talk 2022-11-23 00:11:42 +02:00
Yury Molodov
a59c7a0dd7
vmui: add set up series custom limits (#3368) 
* feat: add set up series custom limits

* feat: add button for show series without limits

* fix: resolve merge conflicts
 2022-11-22 15:36:17 +02:00
Aliaksandr Valialkin
8f53c41aa1
docs/Articles.md: add a link to https://www.youtube.com/watch?v=_zORxrgLtec (OSA Con 2022: Specifics of data analysis in Time Series Databases) 2022-11-22 01:08:12 +02:00
Aliaksandr Valialkin
afc35485c1
app/vminsert: add missing vm_relabel_config_* metrics after 03d88bc066 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3345
 2022-11-22 00:48:13 +02:00
Roman Khavronenko
d1169c1559
vmagent: expose metrics for tracking config state (#3375) 
Expose `vm_relabel_config_*` and `vm_promscrape_config_*` metrics
for tracking relabel and scrape configuration hot-reloads.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3345
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-11-22 00:48:12 +02:00
Yury Molodov
5924aaf7e5
vmui: add copy button to row on Table view (#3363) 
* feat: add copy button to row on Table view

* vmui: add copy button to row on Table view

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-11-22 00:35:05 +02:00
Yury Molodov
05712cfc8d
vmui: sticky tooltip (#3376) 
* feat: add ability to make tooltip "sticky"

* vmui: add ability to make tooltip "sticky"
 2022-11-22 00:35:05 +02:00
Aliaksandr Valialkin
5c65b3c7dc
docs/CHANGELOG.md: link to the related issue for range_normalize() function 2022-11-21 23:27:11 +02:00
Aliaksandr Valialkin
ad548abd6a
app/vmselect/promql: add range_normalize(q1, ..., qN) function for normalizing query results into [0..1] value range 
This may be useful for analyzing correlation between time series with different value ranges
 2022-11-21 23:25:35 +02:00
Aliaksandr Valialkin
c33bcae457
lib/promscrape/discovery/gce: do not pass filter arg when discovering zones 
The filter arg isn't supported by zones API in GCE.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3202
 2022-11-21 22:32:45 +02:00
Aliaksandr Valialkin
9cb76c9cd3
app/vmselect/promql: allow using SI and IEC suffixes in numeric values inside queries 
For example, 10Ki is equivalent to 10*1024, while 5.3M is equivalent to 5.3*1000*1000
 2022-11-21 21:28:34 +02:00
Aliaksandr Valialkin
2d361a834d
app/vmselect/promql: properly return an empty result from limit_offset() if offset exceeds the number of inner time series 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3312
 2022-11-21 16:48:10 +02:00
Aliaksandr Valialkin
b0fefe562a
app/vmselect/promql: optimize e1 op e2 when e1 returns an empty result 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3349
 2022-11-21 16:09:49 +02:00
Roman Khavronenko
0475f8a38e
vmalert: add default list of alerting rules (#3373) 
The default list of alerting rules contains the basic
rules for checking vmalert's health state and is recommended
to use for monitoring vmalert deployments.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-11-21 16:09:47 +02:00
Aliaksandr Valialkin
6fe8eec745
all: add a link to https://docs.victoriametrics.com/enterprise.html into description for enterprise flags 2022-11-21 15:44:54 +02:00
Aliaksandr Valialkin
0e51967970
app/{vminsert,vmselect}: add -storageNode.filter command-line flag for filtering the discovered storage nodes 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3353
 2022-11-21 15:20:20 +02:00
Aliaksandr Valialkin
97eafbe4a7
app/vmselect: clarify that it isnt recommended setting -replicationFactor at vmselect nodes even if the replication is enabled at vminsert nodes 2022-11-18 14:04:12 +02:00
Yury Molodov
60aaf4b2c2
vmui: add trace analyzer (#3310) 
* refactor: change structure project

* refactor: change structure project

* fix: add hooks for set query params

* refactor: add index for pages

* docs: add TESTCASES.md

* refactor: restructure components

* feat: add page with trace analyzer

* fix: change detect trace data

* Update app/vmui/packages/vmui/src/pages/TracePage/index.tsx

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Update app/vmui/packages/vmui/src/pages/TracePage/index.tsx

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* fix: change descriptions on trace page

* Update app/vmui/packages/vmui/src/pages/TracePage/index.tsx

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* feat: add base components

* feat: add reset styles

* docs: add description about trace analyzer

* feat: add styles for custom panel page

* feat: add styles for predefined panels

* feat: add style for TracingsView.tsx

* feat: add Alerts

* feat: add Tooltip.tsx

* fix: correct styles

* feat: add DatePicker.tsx

* feat: add tables

* feat: add theme provider

* fix: replace using callbacks as props to handlers

* fix: correct update time

* fix: change TimePicker.tsx

* fix: correct styles

* fix: update packages

* vmui: refactor code, remove material-ui

* feat: add paste json for trace analyzer

* vmui: update trace analyzer docs

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-11-17 22:23:05 +02:00
Aliaksandr Valialkin
f96bf852ea
docs/CHANGELOG.md: document the fix for CPU usage spikes 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3343
 2022-11-17 22:03:03 +02:00
Aliaksandr Valialkin
7969ca6c80
app/vmselect/vmui: make vmui-update after 51bfd1ab80 2022-11-17 18:57:44 +02:00
Yury Molodov
734172d2d0
vmui: add ability hide query (#3359) 
* feat: add ability hide query

* fix: change logic hide query

* fix: remove console.log
 2022-11-17 18:57:42 +02:00
Yury Molodov
423eb73051
vmui/refactor (#3298) 
* refactor: change structure project

* refactor: change structure project

* fix: add hooks for set query params

* refactor: add index for pages

* docs: add TESTCASES.md

* refactor: restructure components

* feat: add base components

* feat: add reset styles

* feat: add styles for custom panel page

* feat: add styles for predefined panels

* feat: add style for TracingsView.tsx

* feat: add Alerts

* feat: add Tooltip.tsx

* fix: correct styles

* feat: add DatePicker.tsx

* feat: add tables

* feat: add theme provider

* fix: replace using callbacks as props to handlers

* fix: correct update time

* fix: change TimePicker.tsx

* fix: correct styles

* fix: update packages

* vmui: refactor code, remove material-ui

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-11-17 01:21:54 +02:00
Aliaksandr Valialkin
0cc59b9f95
app/vmselect/promql: add range_stdvar() and range_stddev() functions for calculating variance and deviation over time series on the selected time range 2022-11-17 01:03:02 +02:00
Aliaksandr Valialkin
facf7efa3a
app/vmselect/promql: add range_linear_regression(q) function for calculating simple linear regression for the selected time series on the selected time range 2022-11-17 00:39:17 +02:00
Aliaksandr Valialkin
c8d7e1312c
lib/workingsetcache: add -prevCacheRemovalPercent command-line flag for tuning memory usage vs CPU usage ratio 
Reduce the default value of this flag from 1% to 0.2% after 71335e6024

This flag should help determining the best ratio for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3343
 2022-11-16 12:41:37 +02:00
Aliaksandr Valialkin
6e4c9d5f6b
docs/CHANGELOG.md: document changes in v1.79.5 release 2022-11-11 01:28:12 +02:00
Aliaksandr Valialkin
6d984cdeff
docs/CHANGELOG.md: cut v1.83.1 2022-11-10 14:07:20 +02:00
Aliaksandr Valialkin
3bdd1727b7
docs/CHANGELOG.md: document the fix at 71335e6024 2022-11-10 13:47:32 +02:00
Aliaksandr Valialkin
2091693f16
lib/protoparser/opentsdb: follow-up after 04b0e4e7bf 
- Simplify the parser code to be less error prone
- Document the change
- Add a test for OpenTSDB put line with trailing whitespace without tags

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3290
 2022-11-09 15:36:15 +02:00
Aliaksandr Valialkin
28e4364fd2
docs/CHANGELOG.md: document 7130af7fd2 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2814
 2022-11-09 12:16:23 +02:00
Denys Holius
c09cee96ca
guides/README.md: fix link to guide-delete-or-replace-metrics.html (#3331) 2022-11-09 12:01:23 +02:00
Roman Khavronenko
8ee464b22b
bump go version to 1.19.3 (#3327) 
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2022-11-09 11:56:38 +02:00
Zakhar Bessarab
e188b59382
docs/operator: fix description for SA at VMClusterSpec (#3313) 2022-11-09 11:51:47 +02:00
Aliaksandr Valialkin
8540dd669b
app/vminsert/netstorage: move nodesHash from global state to storageNodesBucket 
This should prevent from panics when the list of discovered vmstorage nodes changes.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3329
 2022-11-09 11:45:24 +02:00
Aliaksandr Valialkin
abf7e4e72f
lib/promscrape/discovery/consul: add __meta_consul_partition label in the same way as Prometheus does 
See https://github.com/prometheus/prometheus/pull/11482
 2022-11-07 15:26:45 +02:00
Aliaksandr Valialkin
d3035b1ca1
lib/storage: follow-up for 790768f20b 
- Document the bugfix at docs/CHANGELOG.md
- Simplify the bugfix a bit
 2022-11-07 14:18:06 +02:00
Aliaksandr Valialkin
68382fe580
docs/Single-server-VictoriaMetrics.md: mention about security certifications at Security chapter 
This is a follow-up for 94bd49402e
 2022-11-05 10:09:38 +02:00
Artem Navoiev
6cd36a7fc3
docs: Add link to security page from Readme (#3286) 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2022-11-05 10:09:38 +02:00
Dmytro Kozlov
ae05b475d0
docs/operator: change VMAgentRemoteWriteSettings.MaxDiskUsagePerURL type from int32 to int64 (#3307) 
* docs/operator: change VMAgentRemoteWriteSettings.MaxDiskUsagePerURL type from int32 to int64

* docs: operator updates api description

Co-authored-by: f41gh7 <nik@victoriametrics.com>
 2022-11-05 10:09:38 +02:00
Aliaksandr Valialkin
56ef9f2d88
docs/guides/guide-vmcluster-multiple-retention-setup.md: clarify docs after a75d85b11e 2022-11-05 10:09:11 +02:00
Artem Navoiev
b030662e3e
update multi-tenancy guide. Add infromation about Enterprise and Rete… (#3285) 
docs: update multi-tenancy guide

Add information about Enterprise and Retention Filters

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2022-11-05 10:09:11 +02:00
Zakhar Bessarab
fd36c9b3b1
docs/{vminsert,vmselect}: clarify flag description for clusternativeListenAddr (#3314) 
docs/{vminsert,vmselect}: clarify flag description for `clusternativeListenAddr`
 2022-11-04 09:49:53 +01:00
Aliaksandr Valialkin
45fa0adeb7
docs/Single-server-VictoriaMetrics.md: follow-up after a1a97b9321 
- Remove trailing whitespace at the end of lines
- Remove redundant sentence stating that time series matching the given selector will be deleted.
  It should be clear from the surrounding context.
 2022-11-01 10:53:19 +02:00
Dmytro Kozlov
03c55fa646
docs: clarify information about usage of the api/v1/admin/tsdb/delete_series API (#3287) 
docs: clarify information about usage of the `api/v1/admin/tsdb/delete_series` API
 2022-11-01 10:53:19 +02:00
Aliaksandr Valialkin
616c9f1fb0
docs/Release-Guide.md: instruct to update VictoriaMetrics version in deployment/docker/docker-compose*.yml files after creating new release 
This is a follow-up for d1509f4559
 2022-11-01 10:32:02 +02:00
Aliaksandr Valialkin
426a9b3bb1
docs/enterprise.md: mention that feature requests from enterprise customers are prioritized 2022-11-01 10:27:35 +02:00
Aliaksandr Valialkin
ac7c43c246
docs: typo fixes 2022-10-29 04:53:03 +03:00
Aliaksandr Valialkin
1357107a11
docs/CHANGELOG.md: cut v1.83.0 2022-10-29 02:55:08 +03:00
Aliaksandr Valialkin
cf5efb93d9
app/vmbackupmanager: add functionality for automated restore from backup 2022-10-29 02:31:06 +03:00