* vmalert: remove `vmalert_execution_duration_seconds` metric The summary for `vmalert_execution_duration_seconds` metric gives no additional value comparing to `vmalert_iteration_duration_seconds` metric. * vmalert: update config reload success metric properly Previously, if there was unsuccessfull attempt to reload config and then rollback to previous version - the metric remained set to 0. * vmalert: add Grafana dashboard to overview application metrics * docker: include vmalert target into list for scraping * vmalert: extend notifier metrics with addr label The change adds an `addr` label to metrics for alerts_sent and alerts_send_errors to identify which exact address is having issues. The according change was made to vmalert dashboard. * vmalert: update documentation and docker environment for vmalert's dashboard Mention Grafana's dashboard in vmalert's README in a new section #Monitoring. Update docker-compose env to automatically add vmalert's dashboard. Update docker-compose README with additional info about services.
2.0 KiB
Docker compose environment for VictoriaMetrics
To spin-up VictoriaMetrics cluster, vmagent, vmalert, Alertmanager and Grafana run the following command:
docker-compose up
For single version check docker compose in master branch.
VictoriaMetrics
VictoriaMetrics cluster in this environment consists of
vminsert, vmstorage and vmselect components. Only vmselect
has exposed port :8481
and the rest of components are available
only inside of environment.
The communication scheme between components is the following:
- vmagent sends scraped metrics to vminsert;
- vminsert forwards data to vmstorage;
- vmselect is connected to vmstorage for querying data;
- grafana is configured with datasource pointing to vmselect;
- vmalert is configured to query vmselect and send alerts state and recording rules to vminsert;
- alertmanager is configured to receive notifications from vmalert.
vmagent
vmagent is used for scraping and pushing timeseries to
VictoriaMetrics instance. It accepts Prometheus-compatible
configuration prometheus.yml
with listed targets for scraping.
vmalert
vmalert evaluates alerting rules (alerts.yml
) to track VictoriaMetrics
health state. It is connected with AlertManager for firing alerts,
and with VictoriaMetrics for executing queries and storing alert's state.
alertmanager
AlertManager accepts notifications from vmalert
and fires alerts.
All notifications are blackholed according to alertmanager.yml
config.
Grafana
To access service open following link.
Default creds:
- login -
admin
- password -
admin
Grafana is provisioned by default with following entities:
- VictoriaMetrics datasource
- Prometheus datasource
- VictoriaMetrics overview dashboard