Commit Graph

254 Commits

Author SHA1 Message Date
Roman Khavronenko
70c3664b40
alerts: remove show_at label for RequestErrorsToAPI alert (#3455)
Alert `RequestErrorsToAPI` could be permanently triggered due to
mistakes in clients configuration. However, such requests are unlikely
to cause VM health state change. So there is no need in displaying
this alert because there will be no correlation caused by it.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-07 09:48:59 -08:00
Denys Holius
5232775709
deployment/docker: bump Grafana version to v9.2.7 (#3454)
see https://grafana.com/blog/2022/11/29/grafana-security-release-new-versions-with-high-severity-security-fix-for-cve-2022-31097/
2022-12-07 09:48:15 -08:00
Roman Khavronenko
ff1c654006
deployment: update the README (#3447)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-05 22:06:16 -08:00
Aliaksandr Valialkin
d2e34b8052
{dashboards,alerts}: subtitute {type="indexdb"} with {type=~"indexdb.*"} inside queries after 8189770c50
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3337
2022-12-05 16:00:42 -08:00
Roman Khavronenko
73340dcb01
dashboards: add Disk space usage % and Disk space usage % by type panels (#3436)
The new panels have been added to the vmstorage and drilldown rows.

`Disk space usage %` is supposed to show disk space usage percentage.
This panel is now also referred by `DiskRunsOutOfSpace` alerting rule.
This panel has Drilldown option to show absolute values.

`Disk space usage % by type` shows the relation between datapoints
and indexdb size. It supposed to help identify cases when indexdb
starts to take too much disk space.
This panel has Drilldown option to show absolute values.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-12-05 00:19:36 -08:00
Roman Khavronenko
b2f45b4856
dashboards: update VM single dash (#3400)
The change list is the following:
* bump Grafana version to 9.2.6;
* replace old "Graph" panel with "TimeSeries" panel;
* show % usage of Mem and CPU additionally to of absolute values;
* `Caches` row was removed. All needed info for caches is now part of `Troubleshooting`;
* add Annotations for Alert triggers. Not all alerts are supposed to be displayed
on the dashboard, but only those with label `show_at: dashboard`.
See `alerts.yml` change.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-11-29 20:39:05 -08:00
Denys Holius
a90fbafa23
deployment/docker: bump grafana version to latest v9.2.6 (#3398) 2022-11-28 16:43:58 -08:00
Aliaksandr Valialkin
92a7f71ee1
deployment/docker: update VictoriaMetrics tag from v1.83.1 to v1.84.0 2022-11-25 22:29:20 -08:00
Aliaksandr Valialkin
d0461bec25
docs/CHANGELOG.md: cut v1.84.0 2022-11-25 19:59:30 -08:00
Roman Khavronenko
cae148d5c6
dashboards: cluster dashboard update (#3380)
The purpose of the update is to make the dash more usable
for large installations with many instances. Panels which showed
metrics per-instance (Mem, CPU) now are showing metrics per-job or min/max/avg
aggregations in % instead. This supposed to help immediately to identify
resource shortage and remain usable for small and big installations.

For cases when detailed info is needed, to the bottom of the dashboard
a new row `Drilldown` was added. Panels like Mem or CPU now contain
a `data-link` named `Drilldown` (cis shown on line click) which takes
user to more detailed panel.

The change list is the following:
* bump Grafana version to 9.1.0;
* replace old "Graph" panel with "TimeSeries" panel;
* improve Uptime panel to show number of instances per job;
* show % usage of Mem and CPU instead of absolute values;
* `Caches` row was removed. All needed info for caches is now part of `Troubleshooting`;
* add `Drilldown` section for detailed resource usage;
* add Annotations for Alert triggers. Not all alerts are supposed to be displayed
on the dashboard, but only those with label `show_at: dashboard`.
See `alerts-cluster.yml` change.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-11-24 13:20:10 -08:00
Aliaksandr Valialkin
afc35485c1
app/vminsert: add missing vm_relabel_config_* metrics after 03d88bc066
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3345
2022-11-22 00:48:13 +02:00
Roman Khavronenko
d1169c1559
vmagent: expose metrics for tracking config state (#3375)
Expose `vm_relabel_config_*` and `vm_promscrape_config_*` metrics
for tracking relabel and scrape configuration hot-reloads.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3345
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-11-22 00:48:12 +02:00
Roman Khavronenko
67c60dbf86
vmalert: bump alerting rules evaluation interval to reasonable 30s (#3374)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-11-21 16:48:09 +02:00
Roman Khavronenko
0475f8a38e
vmalert: add default list of alerting rules (#3373)
The default list of alerting rules contains the basic
rules for checking vmalert's health state and is recommended
to use for monitoring vmalert deployments.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-11-21 16:09:47 +02:00
Aliaksandr Valialkin
513257d536
deployment/docker: update VictoriaMetrics version from v1.83.0 to v1.83.1 2022-11-11 01:25:00 +02:00
Aliaksandr Valialkin
61bcbfd697
deployment/docker: update Go builder from v1.19.2 to v1.19.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.3+label%3ACherryPickApproved
2022-11-05 10:19:24 +02:00
Denys Holius
63c7a84ab9
docker-compose: bump version of container tags for VictoriaMetrics components (#3294)
* deployment/docker/docker-compose-cluster.yml: bump VictoriaMetrics Cluster components to the latest v1.83.0 version

* deployment/docker/docker-compose.yml: bump VictoriaMetrics Single node and vmutils to the latest v1.83.0 version
2022-11-01 10:32:02 +02:00
Aliaksandr Valialkin
7ae038766c
app/vmalert/templates: properly escape all the special chars in quotesEscape function
Previously the `quotesEscape` function was escaping only double quotes.
This wasn't enough, since the input string could contain other special chars,
which must be escaped when put inside JSON string. For example, carriage return and line feed chars (\n\r),
backslash char, etc. This led to the following issues, which were improperly fixed:

- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/890 - this issue
  was "fixed" by introducing the `crlfEscape` function, which led to unnecessary
  complications in user templates, while not fixing various corner cases
  such as backslash chars in the input string.
  See 1de15ad490

- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3139 - this issue
  was "fixed" by urlencoding the whole string passed to -external.alert.source
  command-line flag. This led to invalid urls, which couldn't be parsed by Grafana.
  See 00c838353d
  and 4bd0244599

This commit properly encodes the input string passed to `quotesEscape`, so it can be safely embedded inside JSON strings.

This commit deprecates crlfEscape template function and adds the following new template functions:

- strvalue and stripDomain - these functions are supported by Prometheus, so they were added
  for compatibility purposes.
- jsonEscape and htmlEscape for converting the input string to valid quoted JSON string
  and for html-escaping the input string, so it could be safely embedded as a plaintext
  into html.

This commit also documents all supported template functions at https://docs.victoriametrics.com/vmalert.html#template-functions
The deprecated crlfEscape function isn't documented on purpose, since its usefulness is negative in general case.
2022-10-28 00:08:50 +03:00
Aliaksandr Valialkin
19ea0eead2
deployment/docker: update Go builder from v1.19.1 to v1.19.2
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.2+label%3ACherryPickApproved
2022-10-06 02:02:30 +03:00
Zakhar Bessarab
434b00cee8
docker-compose: move TooManyLogs into vm-health alerts set (#3199) 2022-10-05 22:42:31 +03:00
Denys Holius
ebfccada32
Adds packer build for server with VM Single node in vultr.com marketplace (#3142)
* adds packer build for server with VM Single node in vultr.com marketplace

* fix missed varibale
2022-09-26 16:45:29 +03:00
Roman Khavronenko
7ba51c57ae
deployment/docker: fix image versions for cluster components (#3150)
Cluster components always have `-cluster` suffix. The change fixes
incorrect image tag in docker-compose manifest.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-26 08:29:58 +03:00
Denys Holius
99777b3d2a
deployment/docker: add version tag for docker containers (#3141)
* deployment/docker/docker-compose.yml: adds version tags for VictoriaMetrics containers

* deployment/docker/docker-compose-cluster.yml: adds version tags for VictoriaMetrics containers
2022-09-22 19:18:50 +03:00
Aliaksandr Valialkin
101581e053
deployment/docker/provisioning/dashboards: sync with master branch after 5714a68ac6 2022-09-21 12:06:01 +03:00
Roman Khavronenko
f772ee8326
deployment/docker: move cluster compose env to master branch (#3130)
* deployment/docker: move cluster compose env to master branch

The change supposed to simplify the process of maintaining for
single/cluster docker-compose envs, alerts, dashboards. It also
supposes to reduce confusion for users when looking for cluster
related alerts/configs.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* deployment/docker: move cluster compose env to master branch

Review updates.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-21 12:03:10 +03:00
Denys Holius
a186f21f4c
deployment/docker/docker-compose: expose port for vminsert (#3096)
deployment/docker: open port for vminsert because it can't be reached from nginx/curl/etc as it run at random port in OS
2022-09-14 07:13:56 +02:00
Aliaksandr Valialkin
e619fa8b38
deployment/docker: update Go builder for prod binaries from Go1.19.0 to Go1.19.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.19.1+label%3ACherryPickApproved
2022-09-08 18:27:55 +03:00
Max Golionko
e07f23a1b9
moved cluster dashboard to master (#3074)
dashboards: move cluster dashboard to master branch

This change should simplify dashboards management.
2022-09-08 11:47:25 +03:00
Denys Holius
6f92fb8783
04-install-victoriametrics.sh: update download link for tar.gz
see Update note 5 at https://docs.victoriametrics.com/CHANGELOG.html#v1790
2022-08-30 12:22:42 +03:00
Denys Holius
683039cb63
RELEASE_GUIDE.md: updated the document considering the fixes 2022-08-30 12:21:28 +03:00
Denys Holius
72da5f127a
remove not needed template.json 2022-08-30 12:21:28 +03:00
Denys Holius
ec35ec2125
template.pkr.hcl: fix wrong variable name 2022-08-30 12:19:31 +03:00
Denys Holius
b71b58b2cf
deployment/docker/Makefile: bump version of Alpine linux to latest 3.16.2 to fix CVE-2022-37434 (#3035)
see https://alpinelinux.org/posts/Alpine-3.13.12-3.14.8-3.15.6-3.16.2-released.html
2022-08-30 09:42:45 +03:00
Denys Holius
fb80735a9a
deployment: bump Grafana version to latest 9.1.0 (#2995)
see more at https://grafana.com/blog/2022/08/16/grafana-9.1-release/
2022-08-18 12:20:56 +02:00
Aliaksandr Valialkin
690b505975
deployment/docker: specify docker image tags for all the docker images for reproducible docker-compose up runs 2022-08-09 12:30:22 +03:00
Aliaksandr Valialkin
f28f4ad551
deployment/docker: update Go builder from Go1.18.5 to Go1.19.0
See https://tip.golang.org/doc/go1.19

Notable changes:

* GOMEMLIMIT environment variable - see https://tip.golang.org/doc/gc-guide
* Faster CPU profiler
* Faster sort algorithm
2022-08-07 21:31:20 +03:00
Aliaksandr Valialkin
573f6c8d6c
Makefile: remove redundant -mod=vendor option when running Go tools
The `-mod=vendor` is automatically set when there is a `vendor` directory
starting from Go1.14 - see https://go.dev/doc/go1.14#go-command

Since the minimum supported Go version for VictoriaMetrics is Go1.17,
then the `-mod=vendor` option is no longer needed.
2022-08-07 20:40:41 +03:00
Denys Holius
9b4024cd62
deployment/docker/Makefile: added docker-scan (#2916)
* deployment/docker/Makefile: added docker-scan

docker-scan based on native 'docker scan' function that use snyk.io, see https://docs.docker.com/engine/scan/

* set to call 'docker-scan after release binaries but before publishing
2022-08-02 13:26:43 +03:00
Aliaksandr Valialkin
f2e56f0dfd
deployment/docker: update Go builder from v1.18.4 to v1.18.5
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.5+label%3ACherryPickApproved
2022-08-02 13:13:11 +03:00
Aliaksandr Valialkin
536ee9fc18
deployment/docker: update alpine base image from 3.16.0 to 3.16.1
See https://alpinelinux.org/posts/Alpine-3.16.1-released.html
2022-07-19 19:51:36 +03:00
Roman Khavronenko
23e85e0fc5
vmagent: expose metric vmagent_remotewrite_queues (#2871)
The new metric `vmagent_remotewrite_queues` exports a static value of
number of configured remote write queus. This metric is useful to
calculate total saturation per each configured URL with given number
of queues. See corresponding changes to vmagent alerts and dashboard.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-18 14:41:04 +03:00
Aliaksandr Valialkin
8b5b64856b
deployment/docker: update Grafana from v9.0.2 to v9.0.3
See https://grafana.com/blog/2022/07/14/grafana-v9-0-3-8-5-9-8-4-10-and-8-3-10-released-with-high-severity-security-fix/
2022-07-18 14:27:42 +03:00
Aliaksandr Valialkin
83221bc480
deployment/docker: update Go builder from go1.18.3 to go1.18.4
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.4+label%3ACherryPickApproved
2022-07-13 20:31:54 +03:00
Aliaksandr Valialkin
da6c85a2f6
all: follow-up for d99ba3481b 2022-07-13 17:17:08 +03:00
Dmytro Kozlov
4e4def9df8
Rename release packages (#2810)
* makefile: add os to each release file

* makefile: update vmutils arm64

* makefile: update victoria-metrics release process

* makefile: update publish with os

* makefile: update publish with os

* makefile: change tar library

* update release logic

* copy all releases

* sort command by GOOS

* rollback commands

* rollback OSARCH

* fix commands

* cleanup

* fix windows build

* sort build by GOOS, update README.md
2022-07-13 17:11:01 +03:00
Denys Holius
083af28cf8
deployment/docker/docker-compose.yml: update Grafana from v8.5.1 to v9.0.2 (#2859)
See https://grafana.com/blog/2022/06/14/grafana-9.0-release-oss-and-cloud-features
2022-07-12 20:03:28 +03:00
Roman Khavronenko
a42063909f
alerts: correct expression for DiskRunsOutOfSpaceIn3Days (#2856)
The negative value for ETA can happen when deduplication is enabled
and `rate` over `vm_deduplicated_samples_total` becomes bigger
than actual ingestion rate.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-12 14:14:47 +02:00
Yurii Kravets
14397ba23e
Changed the level type in alerts.yml for TooManyLogs alert (#2759)
alerts: filter out non error log messages for `TooManyLogs`

Info and Warn error levels aren't always a result of malfunctioning
or faulty state. So we filter them out.
2022-06-20 16:45:52 +02:00
Aliaksandr Valialkin
079fdd3158
all: update Go builder for production builds from 1.18.2 to 1.18.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.3+label%3ACherryPickApproved
2022-06-20 14:30:48 +03:00
Aliaksandr Valialkin
fe9f59fcd6
all: replace bash with console blocks in all the *.md files
This is a follow-up for 954a7a6fc6
2022-06-19 23:02:02 +03:00