From 4029c539b9c6a8dfcd36e8c1d757bc19ede742c2 Mon Sep 17 00:00:00 2001 From: Dmytro Kozlov Date: Fri, 10 Mar 2023 12:29:08 +0200 Subject: [PATCH] app/vmctl: update importing tips when migrating data with overlapping time range (#3941) app/vmctl: update importing tips when migrating data with overlapping time range --- app/vmctl/README.md | 3 +++ docs/vmctl.md | 3 +++ 2 files changed, 6 insertions(+) diff --git a/app/vmctl/README.md b/app/vmctl/README.md index fd23c5ba56..000e463a0d 100644 --- a/app/vmctl/README.md +++ b/app/vmctl/README.md @@ -790,6 +790,9 @@ to allocated CPU resources of your VictoriaMetrics installation. to the corresponding source address. 9. `vmctl` supports `--vm-native-disable-http-keep-alive` to allow `vmctl` to use non-persistent HTTP connections to avoid error `use of closed network connection` when run a longer export. +10. Migrating data with overlapping time range for destination data can produce duplicates series at destination. +To avoid duplicates on the destination set `-dedup.minScrapeInterval=1ms` for `vmselect` and `vmstorage`. +This will instruct `vmselect` and `vmstorage` to ignore duplicates with match timestamps. In this mode `vmctl` acts as a proxy between two VM instances, where time series filtering is done by "source" (`src`) and processing is done by "destination" (`dst`). So no extra memory or CPU resources required on `vmctl` side. Only diff --git a/docs/vmctl.md b/docs/vmctl.md index bfa2b91574..d0bef4292d 100644 --- a/docs/vmctl.md +++ b/docs/vmctl.md @@ -794,6 +794,9 @@ to allocated CPU resources of your VictoriaMetrics installation. to the corresponding source address. 9. `vmctl` supports `--vm-native-disable-http-keep-alive` to allow `vmctl` to use non-persistent HTTP connections to avoid error `use of closed network connection` when run a longer export. +10. Migrating data with overlapping time range for destination data can produce duplicates series at destination. +To avoid duplicates on the destination set `-dedup.minScrapeInterval=1ms` for `vmselect` and `vmstorage`. +This will instruct `vmselect` and `vmstorage` to ignore duplicates with match timestamps. In this mode `vmctl` acts as a proxy between two VM instances, where time series filtering is done by "source" (`src`) and processing is done by "destination" (`dst`). So no extra memory or CPU resources required on `vmctl` side. Only