vmalert: update docs to highlight the state restore requirements; (#833)

Address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/830
This commit is contained in:
Roman Khavronenko 2020-10-13 18:32:43 +03:00 committed by GitHub
parent 94978af9bc
commit bc42b5598f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -11,6 +11,7 @@ rules against configured address.
* Prometheus [alerting rules definition format](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/#defining-alerting-rules) * Prometheus [alerting rules definition format](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/#defining-alerting-rules)
support; support;
* Integration with [Alertmanager](https://github.com/prometheus/alertmanager); * Integration with [Alertmanager](https://github.com/prometheus/alertmanager);
* Keeps the alerts [state on restarts](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmalert#alerts-state-on-restarts);
* Lightweight without extra dependencies. * Lightweight without extra dependencies.
### Limitations: ### Limitations:
@ -121,14 +122,6 @@ annotations:
[ <labelname>: <tmpl_string> ] [ <labelname>: <tmpl_string> ]
``` ```
`vmalert` has no local storage and alerts state is stored in process memory. Hence, after reloading of `vmalert` process
alerts state will be lost. To avoid this situation, `vmalert` may be configured via following flags:
* `-remoteWrite.url` - URL to Victoria Metrics or VMInsert. `vmalert` will persist alerts state into the configured
address in form of timeseries with name `ALERTS` via remote-write protocol.
* `-remoteRead.url` - URL to Victoria Metrics or VMSelect. `vmalert` will try to restore alerts state from configured
address by querying `ALERTS` timeseries.
##### Recording rules ##### Recording rules
The syntax for recording rules is following: The syntax for recording rules is following:
@ -147,6 +140,22 @@ labels:
For recording rules to work `-remoteWrite.url` must specified. For recording rules to work `-remoteWrite.url` must specified.
#### Alerts state on restarts
`vmalert` has no local storage, so alerts state is stored in the process memory. Hence, after reloading of `vmalert`
the process alerts state will be lost. To avoid this situation, `vmalert` should be configured via the following flags:
* `-remoteWrite.url` - URL to VictoriaMetrics (Single) or VMInsert (Cluster). `vmalert` will persist alerts state
into the configured address in the form of time series named `ALERTS` and `ALERTS_FOR_STATE` via remote-write protocol.
These are regular time series and may be queried from VM just as any other time series.
The state stored to the configured address on every rule evaluation.
* `-remoteRead.url` - URL to VictoriaMetrics (Single) or VMSelect (Cluster). `vmalert` will try to restore alerts state
from configured address by querying time series with name `ALERTS_FOR_STATE`.
Both flags are required for the proper state restoring. Restore process may fail if time series are missing
in configured `-remoteRead.url`, weren't updated in the last `1h` or received state doesn't match current `vmalert`
rules configuration.
#### WEB #### WEB
`vmalert` runs a web-server (`-httpListenAddr`) for serving metrics and alerts endpoints: `vmalert` runs a web-server (`-httpListenAddr`) for serving metrics and alerts endpoints: