diff --git a/README.md b/README.md index a9d272a46..bc50fbd59 100644 --- a/README.md +++ b/README.md @@ -102,7 +102,7 @@ Just download [VictoriaMetrics executable](https://github.com/VictoriaMetrics/Vi The following command-line flags are used the most: * `-storageDataPath` - VictoriaMetrics stores all the data in this directory. Default path is `victoria-metrics-data` in the current working directory. -* `-retentionPeriod` - retention for stored data. Older data is automatically deleted. Default retention is 1 month. See [these docs](#retention) for more details. +* `-retentionPeriod` - retention for stored data. Older data is automatically deleted. Default retention is 1 month. See [the Retention section](#retention) for more details. Other flags have good enough default values, so set them only if you really need this. Pass `-help` to see [all the available flags with description and default values](#list-of-command-line-flags). @@ -744,7 +744,7 @@ The delete API is intended mainly for the following cases: * One-off deleting of accidentally written invalid (or undesired) time series. * One-off deleting of user data due to [GDPR](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation). -It isn't recommended using delete API for the following cases, since it brings non-zero overhead: +Using the delete API is not recommended in the following cases, since it brings a non-zero overhead: * Regular cleanups for unneeded data. Just prevent writing unneeded data into VictoriaMetrics. This can be done with [relabeling](#relabeling). @@ -753,7 +753,7 @@ It isn't recommended using delete API for the following cases, since it brings n time series occupy disk space until the next merge operation, which can never occur when deleting too old data. [Forced merge](#forced-merge) may be used for freeing up disk space occupied by old data. -It is better using `-retentionPeriod` command-line flag for efficient pruning of old data. +It's better to use the `-retentionPeriod` command-line flag for efficient pruning of old data. ## Forced merge @@ -1147,7 +1147,7 @@ write data to the same VictoriaMetrics instance. These vmagent or Prometheus ins VictoriaMetrics stores time series data in [MergeTree](https://en.wikipedia.org/wiki/Log-structured_merge-tree)-like data structures. On insert, VictoriaMetrics accumulates up to 1s of data and dumps it on disk to `<-storageDataPath>/data/small/YYYY_MM/` subdirectory forming a `part` with the following -name pattern `rowsCount_blocksCount_minTimestamp_maxTimestamp`. Each part consists of two "columns": +name pattern: `rowsCount_blocksCount_minTimestamp_maxTimestamp`. Each part consists of two "columns": values and timestamps. These are sorted and compressed raw time series values. Additionally, part contains index files for searching for specific series in the values and timestamps files. @@ -1177,24 +1177,24 @@ See also [how to work with snapshots](#how-to-work-with-snapshots). ## Retention -Retention is configured with `-retentionPeriod` command-line flag. For instance, `-retentionPeriod=3` means -that the data will be stored for 3 months and then deleted. -Data is split in per-month partitions inside `<-storageDataPath>/data/{small,big}` folders. -Data partitions outside the configured retention are deleted on the first day of new month. +Retention is configured with the `-retentionPeriod` command-line flag, which takes a number followed by a time unit character - `h(ours)`, `d(ays)`, `w(eeks)`, `y(ears)`. If the time unit is not specified, a month is assumed. For instance, `-retentionPeriod=3` means that the data will be stored for 3 months and then deleted. The default retention period is one month. +Data is split in per-month partitions inside `<-storageDataPath>/data/{small,big}` folders. +Data partitions outside the configured retention are deleted on the first day of the new month. Each partition consists of one or more data parts with the following name pattern `rowsCount_blocksCount_minTimestamp_maxTimestamp`. Data parts outside of the configured retention are eventually deleted during [background merge](https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282). -In order to keep data according to `-retentionPeriod` max disk space usage is going to be `-retentionPeriod` + 1 month. -For example if `-retentionPeriod` is set to 1, data for January is deleted on March 1st. +The maximum disk space usage for a given `-retentionPeriod` is going to be (`-retentionPeriod` + 1) months. +For example, if `-retentionPeriod` is set to 1, data for January is deleted on March 1st. -VictoriaMetrics supports retention smaller than 1 month. For example, `-retentionPeriod=5d` would set data retention for 5 days. -Please note, time range covered by data part is not limited by retention period unit. Hence, data part may contain data +Please note, the time range covered by data part is not limited by retention period unit. Hence, data part may contain data for multiple days and will be deleted only when fully outside of the configured retention. -It is safe to extend `-retentionPeriod` on existing data. If `-retentionPeriod` is set to lower -value than before then data outside the configured period will be eventually deleted. +It is safe to extend `-retentionPeriod` on existing data. If `-retentionPeriod` is set to a lower +value than before, then data outside the configured period will be eventually deleted. + +VictoriaMetrics does not support indefinite retention, but you can specify an arbitrarily high duration, e.g. `-retentionPeriod=100y`. ## Multiple retentions diff --git a/docs/Single-server-VictoriaMetrics.md b/docs/Single-server-VictoriaMetrics.md index 0b6f5727e..4188db54b 100644 --- a/docs/Single-server-VictoriaMetrics.md +++ b/docs/Single-server-VictoriaMetrics.md @@ -106,7 +106,7 @@ Just download [VictoriaMetrics executable](https://github.com/VictoriaMetrics/Vi The following command-line flags are used the most: * `-storageDataPath` - VictoriaMetrics stores all the data in this directory. Default path is `victoria-metrics-data` in the current working directory. -* `-retentionPeriod` - retention for stored data. Older data is automatically deleted. Default retention is 1 month. See [these docs](#retention) for more details. +* `-retentionPeriod` - retention for stored data. Older data is automatically deleted. Default retention is 1 month. See [the Retention section](#retention) for more details. Other flags have good enough default values, so set them only if you really need this. Pass `-help` to see [all the available flags with description and default values](#list-of-command-line-flags). @@ -748,7 +748,7 @@ The delete API is intended mainly for the following cases: * One-off deleting of accidentally written invalid (or undesired) time series. * One-off deleting of user data due to [GDPR](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation). -It isn't recommended using delete API for the following cases, since it brings non-zero overhead: +Using the delete API is not recommended in the following cases, since it brings a non-zero overhead: * Regular cleanups for unneeded data. Just prevent writing unneeded data into VictoriaMetrics. This can be done with [relabeling](#relabeling). @@ -757,7 +757,7 @@ It isn't recommended using delete API for the following cases, since it brings n time series occupy disk space until the next merge operation, which can never occur when deleting too old data. [Forced merge](#forced-merge) may be used for freeing up disk space occupied by old data. -It is better using `-retentionPeriod` command-line flag for efficient pruning of old data. +It's better to use the `-retentionPeriod` command-line flag for efficient pruning of old data. ## Forced merge @@ -1151,7 +1151,7 @@ write data to the same VictoriaMetrics instance. These vmagent or Prometheus ins VictoriaMetrics stores time series data in [MergeTree](https://en.wikipedia.org/wiki/Log-structured_merge-tree)-like data structures. On insert, VictoriaMetrics accumulates up to 1s of data and dumps it on disk to `<-storageDataPath>/data/small/YYYY_MM/` subdirectory forming a `part` with the following -name pattern `rowsCount_blocksCount_minTimestamp_maxTimestamp`. Each part consists of two "columns": +name pattern: `rowsCount_blocksCount_minTimestamp_maxTimestamp`. Each part consists of two "columns": values and timestamps. These are sorted and compressed raw time series values. Additionally, part contains index files for searching for specific series in the values and timestamps files. @@ -1181,24 +1181,24 @@ See also [how to work with snapshots](#how-to-work-with-snapshots). ## Retention -Retention is configured with `-retentionPeriod` command-line flag. For instance, `-retentionPeriod=3` means -that the data will be stored for 3 months and then deleted. -Data is split in per-month partitions inside `<-storageDataPath>/data/{small,big}` folders. -Data partitions outside the configured retention are deleted on the first day of new month. +Retention is configured with the `-retentionPeriod` command-line flag, which takes a number followed by a time unit character - `h(ours)`, `d(ays)`, `w(eeks)`, `y(ears)`. If the time unit is not specified, a month is assumed. For instance, `-retentionPeriod=3` means that the data will be stored for 3 months and then deleted. The default retention period is one month. +Data is split in per-month partitions inside `<-storageDataPath>/data/{small,big}` folders. +Data partitions outside the configured retention are deleted on the first day of the new month. Each partition consists of one or more data parts with the following name pattern `rowsCount_blocksCount_minTimestamp_maxTimestamp`. Data parts outside of the configured retention are eventually deleted during [background merge](https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282). -In order to keep data according to `-retentionPeriod` max disk space usage is going to be `-retentionPeriod` + 1 month. -For example if `-retentionPeriod` is set to 1, data for January is deleted on March 1st. +The maximum disk space usage for a given `-retentionPeriod` is going to be (`-retentionPeriod` + 1) months. +For example, if `-retentionPeriod` is set to 1, data for January is deleted on March 1st. -VictoriaMetrics supports retention smaller than 1 month. For example, `-retentionPeriod=5d` would set data retention for 5 days. -Please note, time range covered by data part is not limited by retention period unit. Hence, data part may contain data +Please note, the time range covered by data part is not limited by retention period unit. Hence, data part may contain data for multiple days and will be deleted only when fully outside of the configured retention. -It is safe to extend `-retentionPeriod` on existing data. If `-retentionPeriod` is set to lower -value than before then data outside the configured period will be eventually deleted. +It is safe to extend `-retentionPeriod` on existing data. If `-retentionPeriod` is set to a lower +value than before, then data outside the configured period will be eventually deleted. + +VictoriaMetrics does not support indefinite retention, but you can specify an arbitrarily high duration, e.g. `-retentionPeriod=100y`. ## Multiple retentions