Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-15 00:13:30 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

docs/Cluster-VictoriaMetrics.md: move the Improve re-routing performance during restart to more appropriate place

Browse Source 
Previoulsy it was mistakenly inserted between `No downtime strategy` and 'Minimum downtime strategy' chapters.

This is a follow-up for 37997abd14
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5293
This commit is contained in:
Aliaksandr Valialkin 2024-02-05 14:41:25 +02:00
parent f222cf9200
commit 67d166f181
No known key found for this signature in database
GPG Key ID: 52C003EE2BCDB9EB
1 changed files with 17 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
docs/Cluster-VictoriaMetrics.md
View File

@ -486,20 +486,6 @@ This strategy allows upgrading the cluster without downtime if the following con
If at least a single condition isn't met, then the rolling restart may result in cluster unavailability
during the config update / version upgrade. In this case the following strategy is recommended.
#### Improving re-routing performance during restart
`vmstorage` nodes may experience increased usage for CPU, RAM and disk IO during
[rolling restarts](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#no-downtime-strategy),
since they need to process higher load when some of `vmstorage` nodes are temporarily unavailable in the cluster.
It is possible to reduce resource usage spikes by running more `vminsert` nodes and by passing bigger values
to `-storage.vminsertConnsShutdownDuration` (available from [v1.95.0](https://docs.victoriametrics.com/CHANGELOG.html#v1950))
command-line flag at `vmstorage` nodes.
In this case `vmstorage` increases the interval between gradual closing of `vminsert` connections during graceful shutdown.
This reduces data ingestion slowdown during rollout restarts.
Make sure that the `-storage.vminsertConnsShutdownDuration` is smaller than the graceful shutdown timeout configured at the system which manages `vmstorage`
(e.g. Docker, Kubernetes, systemd, etc.). Otherwise the system may kill `vmstorage` node before it finishes gradual closing of `vminsert` connections.
### Minimum downtime strategy
1. Gracefully stop all the `vminsert` and `vmselect` nodes in parallel.
@ -518,6 +504,23 @@ The `minimum downtime` strategy has the following benefits comparing to `no down
- It allows minimizing the duration of config update / version upgrade for clusters with big number of nodes
of for clusters with big `vmstorage` nodes, which may take long time for graceful restart.
## Improving re-routing performance during restart
`vmstorage` nodes may experience increased usage for CPU, RAM and disk IO during
[rolling restarts](https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#no-downtime-strategy),
since they need to process higher load when some of `vmstorage` nodes are temporarily unavailable in the cluster.
It is possible to reduce resource usage spikes by running more `vminsert` nodes and by passing bigger values
to `-storage.vminsertConnsShutdownDuration` (available from [v1.95.0](https://docs.victoriametrics.com/CHANGELOG.html#v1950))
command-line flag at `vmstorage` nodes.
In this case `vmstorage` increases the interval between gradual closing of `vminsert` connections during graceful shutdown.
This reduces data ingestion slowdown during rollout restarts.
Make sure that the `-storage.vminsertConnsShutdownDuration` is smaller than the graceful shutdown timeout configured at the system which manages `vmstorage`
(e.g. Docker, Kubernetes, systemd, etc.). Otherwise the system may kill `vmstorage` node before it finishes gradual closing of `vminsert` connections.
See also [minimum downtime strategy](#minimum-downtime-strategy).
## Cluster availability
VictoriaMetrics cluster architecture prioritizes availability over data consistency.