Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-23 20:37:12 +01:00
Code Issues Actions 8 Packages Projects Releases Wiki Activity
58ae3772fe
VictoriaMetrics/docs/guides/guide-vmcluster-multiple-retention-setup/README.md
Andrii Chubatiuk 35d77a3bed
docs: updated guides structure, removed deprecated sort option (#6767) 
### Describe Your Changes

* `sort` param is unused by the current website engine, and was present only for compatibility
with previous website engine. It is time to remove it as it makes no effect
* re-structure guides content into folders to simplify assets management

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-08-07 16:48:08 +02:00

2.9 KiB
Raw Blame History

Objective

Setup Victoria Metrics Cluster with support of multiple retention periods within one installation.

Enterprise Solution

VictoriaMetrics enterprise supports specifying multiple retentions for distinct sets of time series and tenants via retention filters.

Open Source Solution

Community version of VictoriaMetrics supports only one retention period per vmstorage node via -retentionPeriod command-line flag.

A multi-retention setup can be implemented by dividing a victoriametrics cluster into logical groups with different retentions.

Example: Setup should handle 3 different retention groups 3months, 1year and 3 years. Solution contains 3 groups of vmstorages + vminserts and one group of vmselects. Routing is done by vmagent by splitting data streams. The -retentionPeriod sets how long to keep the metrics.

The diagram below shows a proposed solution

Setup

Implementation Details

  1. Groups of vminserts A know about only vmstorages A and this is explicitly specified via -storageNode configuration.
  2. Groups of vminserts B know about only vmstorages B and this is explicitly specified via -storageNode configuration.
  3. Groups of vminserts C know about only vmstorages A and this is explicitly specified via -storageNode configuration.
  4. vmselect reads data from all vmstorage nodes via -storageNode configuration with deduplication setting equal to vmagent's scrape interval or minimum interval between collected samples.
  5. vmagent routes incoming metrics to the given set of vminsert nodes using relabeling rules specified at -remoteWrite.urlRelabelConfig configuration.

Multi-Tenant Setup

Every group of vmstorages can handle one tenant or multiple one. Different groups can have overlapping tenants. As vmselect reads from all vmstorage nodes, the data is aggregated on its level.

Additional Enhancements

You can set up vmauth for routing data to the given vminsert group depending on the needed retention.