Commit Graph

5470 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
2b55d167d7
app/vmagent/remotewrite: add benchmarks for comparing the performance of standard Snappy encoder with github.com/klauspost/compress/s2 encoder
The standard Snappy encoder from github.com/golang/snappy shows quite good performance number
for compressing the Prometheus remote_write proto messages according to the added benchmarks,
so there is no need in switching to github.com/klauspost/compress/s2 yet.
2022-09-19 14:28:09 +03:00
Roman Khavronenko
b4410b1c63
Dashboards (#3120)
* dashboards/cluster: few updates

* apply consistent formatting across panels;
* make resource usage panels per component more detailed;
* add extra panels to vmselect for displaying
`vm_rows_read_per_query`, `vm_rows_scanned_per_query`,
`vm_rows_read_per_series` and `vm_series_read_per_query` metrics.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/single: few updates

* apply consistent formatting across panels;
* add extra panels to Performance for displaying
`vm_rows_read_per_query`, `vm_rows_scanned_per_query`,
`vm_rows_read_per_series` and `vm_series_read_per_query` metrics.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmagent: few updates

* apply consistent formatting across panels;
* add panels for showing number of samples ingested
or scraped;
* adapt resource usage panels for multiple selected jobs/instances;
* add adhoc variable;
* display vmagent's version in Stats.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards/vmalert: few updates

* apply consistent formatting across panels;
* adapt resource usage panels for multiple selected jobs/instances;
* show vmalert version in Stats section.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-16 21:24:32 +02:00
Roman Khavronenko
622bbedbe1
vmalert: always re-evaluate Annotations (#3119)
* vmalert: always re-evaluate Annotations

Previously, Annotations were evaluated only:
1. On alert creating.
2. On alert's value change.

This is premature optimization. It was assumed that since annotations
could contain only text with alert's labels or value - there is no need
in spending resources to re-compile Annotations.

Later, template function `query` was added, which can execute
arbitrary queries and return different results on every evaluation.
So if it was used in annotations, it would be executed only on init
or value change.

Another case when optimization caused an issue - annotations hot reload.
In this case, annotations of the active alert won't change even if Rule's
annotations were changed.

This fix enables Annotations re-evaluation on each iteration to resolve
issues above. It would have some impact on performance, but it is unlikely
it will be noticeable.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: add tp Changelog

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-16 16:19:10 +02:00
Roman Khavronenko
3484673566
vmalert: add Troubleshooting section to docs (#3115)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-15 16:15:39 +02:00
Dmytro Kozlov
606166ef68
docs/managed_victoriametrics: add how to restore password section (#3116)
docs/managed_victoriametrics: add how to restore password section
2022-09-15 16:02:31 +02:00
Roman Khavronenko
9c95c81534
vmalert: print example of curl command for rule's state (#3112)
The change adds an example of `curl` command to the Rule's page.
The command is generated for each recorded state. It is supposed
user can just copy&execute the command to see what was returned
to vmalert.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-15 12:40:22 +02:00
Aliaksandr Valialkin
455002922e
docs/Cluster-VictoriaMetrics.md: update -help output after explicit marking of enterprise flags 2022-09-15 13:22:58 +03:00
Aliaksandr Valialkin
e7995375b5
docs: update -help output after explicit mentioning of enterprise flags 2022-09-15 13:22:57 +03:00
Aliaksandr Valialkin
4193af4571
docs/vmauth.md: update -help output after explicit marking of enterprise flags 2022-09-15 13:22:57 +03:00
Aliaksandr Valialkin
3b2599c659
docs/vmalert.md: update -help output after explicit marking of enterprise flags 2022-09-15 13:22:57 +03:00
Aliaksandr Valialkin
bbecd27557
docs: update docs with explicitly marked enterprise command-line flags for VictoriaMetrics and vmagent 2022-09-15 13:22:56 +03:00
Dmytro Kozlov
a9629cc32d
app/vmctl: add description about influx-skip-database-label flag (#3111) 2022-09-15 10:46:54 +02:00
Aliaksandr Valialkin
b869c757a9
docs/FAQ.md: add an answer for What is the difference between single-node and cluster versions of VictoriaMetrics? 2022-09-15 09:56:41 +03:00
Dmytro Kozlov
b75f1854c5
vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric) (#2982)
* vmselect/promql: add alphanumeric sort by label (sort_by_label_numeric)

* vmselect/promql: fix tests, add documentation

* vmselect/promql: update test

* vmselect/promql: update for alphanumeric sorting, fix tests

* vmselect/promql: remove comments

* vmselect/promql: cleanup

* vmselect/promql: avoid memory allocations, update functions descriptions

* vmselect/promql: make linter happy (remove ineffectual assigment)

* vmselect/promql: add test case, fix behavior when strings are equal

* vendor: update github.com/VictoriaMetrics/metricsql from v0.44.1 to v0.45.0

this adds support for sort_by_label_numeric and sort_by_label_numeric_desc functions

* wip

* lib/promscrape: read response body into memory in stream parsing mode before parsing it

This reduces scrape duration for targets returning big responses.

The response body was already read into memory in stream parsing mode before this change,
so this commit shouldn't increase memory usage.

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-14 17:41:09 +03:00
Aliaksandr Valialkin
5306f79fd1
docs/Articles.md: add a link to https://www.groundcover.com/blog/prometheus-grafana-kubernetes 2022-09-14 15:11:19 +03:00
Aliaksandr Valialkin
56ce7ce85b
lib/promscrape: typo fix after 74c00a8762 2022-09-14 15:06:50 +03:00
Roman Khavronenko
877940a131
vmalert: add experimental feature of storing Rule's evaluation state (#3106)
vmalert: add experimental feature of storing Rule's evaluation state

The new feature keeps last 20 state changes of each Rule
in memory. The state are available for view on the Rule's
view page. The page can be opened by clicking on `Details`
link next to Rule's name on the `/groups` page.

States change suppose to help in investigating cases when Rule
doesn't generate alerts or records.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-14 14:04:24 +02:00
Aliaksandr Valialkin
99bc18774c
app/vmselect/vmui: make vmui-update after 1304824201 2022-09-14 13:33:06 +03:00
Roman Khavronenko
efea51a9ee
bump Go version to 1.19.1 (#3108)
The reason is to cover vulnerability GO-2022-0969
Found in: net/http@go1.18.5
Fixed in: net/http@go1.19.1
More info: https://pkg.go.dev/vuln/GO-2022-0969

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-14 12:29:19 +02:00
Aliaksandr Valialkin
74c00a8762
lib/promscrape: read response body into memory in stream parsing mode before parsing it
This reduces scrape duration for targets returning big responses.

The response body was already read into memory in stream parsing mode before this change,
so this commit shouldn't increase memory usage.
2022-09-14 13:15:29 +03:00
Aliaksandr Valialkin
ccad651a61
lib/promscrape/discovery/kubernetes: add more context on WatchEvent parse error
This should improve debugging issues with Kubernetes API server
2022-09-13 19:36:55 +03:00
Yury Molodov
1304824201
vmui: fix query params saving in URL (#3104) 2022-09-13 17:05:26 +02:00
Aliaksandr Valialkin
3af24a5b7c
docs: consistently use docs.victoriametrics.com instead of victoriametrics.github.io in all the links 2022-09-13 16:50:13 +03:00
Aliaksandr Valialkin
523ff25077
vendor: make vendor-update 2022-09-13 16:44:44 +03:00
Aliaksandr Valialkin
0ead64b6cf
app/vmalert: follow-up after 8441375da2
- Rename logDebug() to logDebugf() and pass format string together
  with format args directly to logDebugf(). This eliminates fmt.Sprintf()
  overhead at logDebug() call site when debugging is disabled.

- Format labels in debug message in Prometheus format, e.g. {label1="value1",...labelN="valueN"}

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3025
2022-09-13 16:35:14 +03:00
Roman Khavronenko
8441375da2
vmalert: add debug mode for alerting rules (#3055)
* vmalert: add `debug` mode for alerting rules

Debug information includes alerts state changes and requests
sent to the datasource. Debug can be enabled only on rule's
level. It might be useful for debugging unexpected
behaviour of alerting rule.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3025

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: review fixes

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update app/vmalert/alerting.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>

* vmalert: go fmt

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-13 16:25:43 +03:00
Aliaksandr Valialkin
25c9a1604a
docs/CHANGELOG.md: document atomic directory deletion
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3038
2022-09-13 16:17:39 +03:00
Aliaksandr Valialkin
ce2c07c5a7
lib/mergeset: atomically remove part dirs
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3038
2022-09-13 16:17:38 +03:00
Aliaksandr Valialkin
042a532f70
lib/storage: substitute remaining calls to fs.MustRemoveAll with fs.MustRemoveDirAtomic
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3038
2022-09-13 16:17:38 +03:00
Aliaksandr Valialkin
68e32b0764
lib/storage: atomically remove parts inside partitions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3038
2022-09-13 16:17:38 +03:00
Aliaksandr Valialkin
db4f0fe6fc
docs/ExtendedPromQL.md: move the page to the bottom of the contents list at http://docs.victoriametrics.com 2022-09-13 16:17:37 +03:00
Aliaksandr Valialkin
e041c913bd
Revert "docs/ExtendedPromQL.md: remove outdated doc"
This reverts commit 971e3d83f7.

Reason for revert: the ExtendedPromQL doc is still referred from third-party sites.
2022-09-13 16:17:37 +03:00
Aliaksandr Valialkin
340ada871d
lib/storage: atomically remove partitions, which went outside the configured retention
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3038
2022-09-13 16:17:37 +03:00
Aliaksandr Valialkin
978dcb4574
lib/storage: properly remove cache directory contents if reset_cache_on_startup file is located there
Previously the cache directory was removed. This could result in error when the cache directory
is mounted to a separate filesystem.
2022-09-13 16:17:36 +03:00
Aliaksandr Valialkin
5f28ca1f42
lib/storage: atomically remove snapshot directories
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3038
2022-09-13 16:17:36 +03:00
Craig Rodrigues
57ea8dbb36
docs: Use headings in DataDog section to make things easier to read (#3089)
Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>

Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>
2022-09-12 10:58:25 +03:00
Yury Molodov
1b41169415
vmui: fix data processing (#3092)
* fix: change data processing

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-12 10:55:11 +03:00
Yury Molodov
b5f4060520
fix: change columns for Top Queries (#3093) 2022-09-12 10:44:35 +03:00
Aliaksandr Valialkin
c48ff746c6
docs/FAQ.md: add a link to https://victoriametrics.com/blog/mimir-benchmark/ to VictoriaMetrics vs Mimir chapter 2022-09-12 10:37:28 +03:00
Aliaksandr Valialkin
c4af0e833a
docs/Articles.md: add a link to https://victoriametrics.com/blog/mimir-benchmark/ 2022-09-11 15:23:35 +03:00
Yury Molodov
9541ef2e9e
vmui: add lists of top queries (#3065)
* feat: add lists of top queries

* fix: change the field label

* refactor: add handlers for readability

* app/vmselect: `make vmui-update`

* docs: document `top queries` tab

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-09-08 21:43:37 +03:00
John Belmonte
defced2599
MetricsQL doc spellcheck (#3080) 2022-09-08 21:28:03 +03:00
Aliaksandr Valialkin
53b0c2eee4
docs/CHANGELOG.md: document ec273eafef
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3076
2022-09-08 21:25:30 +03:00
Aliaksandr Valialkin
e7635e1c83
Makefile: remove github-create-release and github-upload-assets commands from publish-release
This is a follow-up for b9231c715a
2022-09-08 21:02:41 +03:00
Aliaksandr Valialkin
7c2fa1bc48
vendor: make vendor-update 2022-09-08 18:51:49 +03:00
Aliaksandr Valialkin
aa0c6ed27f
Makefile: consistently use go install instead of go get for installing various binaries needed during build/test/check of the code
`go install` is the preferred way for installing go binaries starting
from the minimum supported Go version for VictoriaMetrics - Go1.18 -
see https://tip.golang.org/doc/go1.18#go-command
2022-09-08 18:43:05 +03:00
Aliaksandr Valialkin
28b6dec1f4
.github/workflows/main.yml: stop setting GO111MODULE=on env var, since it is unnecessary in Go1.18 and newer versions 2022-09-08 18:41:56 +03:00
Aliaksandr Valialkin
5dad557868
Makefile: check for vulnerabilities in used Go packages with govulncheck when running make check-all
See https://go.dev/blog/vuln
2022-09-08 18:35:34 +03:00
Aliaksandr Valialkin
f81dfaf20d
deployment/docker: update Go builder for prod binaries from Go1.19.0 to Go1.19.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.1+label%3ACherryPickApproved
2022-09-08 18:27:10 +03:00
Aliaksandr Valialkin
b9231c715a
docs/Release-Guide.md: require manual push of the created release tags to public Github
The automated push of release tags to Github require specifying the remote repository name
when doing `git push <remote-repo-name> v1.xx.y`.
The remote repository name can differ in different environments,
so it cannot be put into Makefile rule.

TODO: create a Makefile rule, which generates standard remote names for public
and private repositories in Git, so `git push` for release tags could be automated then.
2022-09-08 15:00:18 +03:00