mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-11-23 12:31:07 +01:00
docs/Single-server-VictoriaMetrics.md: document that the deduplication is applied only when exporting data in JSON line format
The exported data isn't de-duplicated by default due to performance reasons. It is expected that the de-duplication is applied during importing the exported data. The deduplication is applied only when exporting data via /api/v1/export if `reduce_mem_usage=1` query arg isn't passed to the request. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1837
This commit is contained in:
parent
d666755159
commit
f06495c50a
@ -789,7 +789,7 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
|
|||||||
|
|
||||||
Optional `max_rows_per_line` arg may be added to the request for limiting the maximum number of rows exported per each JSON line.
|
Optional `max_rows_per_line` arg may be added to the request for limiting the maximum number of rows exported per each JSON line.
|
||||||
Optional `reduce_mem_usage=1` arg may be added to the request for reducing memory usage when exporting big number of time series.
|
Optional `reduce_mem_usage=1` arg may be added to the request for reducing memory usage when exporting big number of time series.
|
||||||
In this case the output may contain multiple lines with distinct samples for the same time series.
|
In this case the output may contain multiple lines with samples for the same time series.
|
||||||
|
|
||||||
Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts
|
Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts
|
||||||
of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data:
|
of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data:
|
||||||
@ -802,6 +802,9 @@ The maximum duration for each request to `/api/v1/export` is limited by `-search
|
|||||||
|
|
||||||
Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-data-in-json-line-format).
|
Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-data-in-json-line-format).
|
||||||
|
|
||||||
|
The [deduplication](#deduplication) is applied to the data exported via `/api/v1/export` by default. The deduplication
|
||||||
|
isn't applied if `reduce_mem_usage=1` query arg is passed to the request.
|
||||||
|
|
||||||
|
|
||||||
### How to export CSV data
|
### How to export CSV data
|
||||||
|
|
||||||
@ -826,6 +829,8 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
|
|||||||
|
|
||||||
The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data).
|
The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data).
|
||||||
|
|
||||||
|
The [deduplication](#deduplication) isn't applied for the data exported in CSV. It is expected that the de-duplication is performed during data import.
|
||||||
|
|
||||||
|
|
||||||
### How to export data in native format
|
### How to export data in native format
|
||||||
|
|
||||||
@ -849,6 +854,8 @@ The exported data can be imported to VictoriaMetrics via [/api/v1/import/native]
|
|||||||
The native export format may change in incompatible way between VictoriaMetrics releases, so the data exported from the release X
|
The native export format may change in incompatible way between VictoriaMetrics releases, so the data exported from the release X
|
||||||
can fail to be imported into VictoriaMetrics release Y.
|
can fail to be imported into VictoriaMetrics release Y.
|
||||||
|
|
||||||
|
The [deduplication](#deduplication) isn't applied for the data exported in native format. It is expected that the de-duplication is performed during data import.
|
||||||
|
|
||||||
|
|
||||||
## How to import time series data
|
## How to import time series data
|
||||||
|
|
||||||
|
@ -789,7 +789,7 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
|
|||||||
|
|
||||||
Optional `max_rows_per_line` arg may be added to the request for limiting the maximum number of rows exported per each JSON line.
|
Optional `max_rows_per_line` arg may be added to the request for limiting the maximum number of rows exported per each JSON line.
|
||||||
Optional `reduce_mem_usage=1` arg may be added to the request for reducing memory usage when exporting big number of time series.
|
Optional `reduce_mem_usage=1` arg may be added to the request for reducing memory usage when exporting big number of time series.
|
||||||
In this case the output may contain multiple lines with distinct samples for the same time series.
|
In this case the output may contain multiple lines with samples for the same time series.
|
||||||
|
|
||||||
Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts
|
Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts
|
||||||
of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data:
|
of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data:
|
||||||
@ -802,6 +802,9 @@ The maximum duration for each request to `/api/v1/export` is limited by `-search
|
|||||||
|
|
||||||
Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-data-in-json-line-format).
|
Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-data-in-json-line-format).
|
||||||
|
|
||||||
|
The [deduplication](#deduplication) is applied to the data exported via `/api/v1/export` by default. The deduplication
|
||||||
|
isn't applied if `reduce_mem_usage=1` query arg is passed to the request.
|
||||||
|
|
||||||
|
|
||||||
### How to export CSV data
|
### How to export CSV data
|
||||||
|
|
||||||
@ -826,6 +829,8 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
|
|||||||
|
|
||||||
The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data).
|
The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data).
|
||||||
|
|
||||||
|
The [deduplication](#deduplication) isn't applied for the data exported in CSV. It is expected that the de-duplication is performed during data import.
|
||||||
|
|
||||||
|
|
||||||
### How to export data in native format
|
### How to export data in native format
|
||||||
|
|
||||||
@ -849,6 +854,8 @@ The exported data can be imported to VictoriaMetrics via [/api/v1/import/native]
|
|||||||
The native export format may change in incompatible way between VictoriaMetrics releases, so the data exported from the release X
|
The native export format may change in incompatible way between VictoriaMetrics releases, so the data exported from the release X
|
||||||
can fail to be imported into VictoriaMetrics release Y.
|
can fail to be imported into VictoriaMetrics release Y.
|
||||||
|
|
||||||
|
The [deduplication](#deduplication) isn't applied for the data exported in native format. It is expected that the de-duplication is performed during data import.
|
||||||
|
|
||||||
|
|
||||||
## How to import time series data
|
## How to import time series data
|
||||||
|
|
||||||
|
@ -793,7 +793,7 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
|
|||||||
|
|
||||||
Optional `max_rows_per_line` arg may be added to the request for limiting the maximum number of rows exported per each JSON line.
|
Optional `max_rows_per_line` arg may be added to the request for limiting the maximum number of rows exported per each JSON line.
|
||||||
Optional `reduce_mem_usage=1` arg may be added to the request for reducing memory usage when exporting big number of time series.
|
Optional `reduce_mem_usage=1` arg may be added to the request for reducing memory usage when exporting big number of time series.
|
||||||
In this case the output may contain multiple lines with distinct samples for the same time series.
|
In this case the output may contain multiple lines with samples for the same time series.
|
||||||
|
|
||||||
Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts
|
Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts
|
||||||
of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data:
|
of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data:
|
||||||
@ -806,6 +806,9 @@ The maximum duration for each request to `/api/v1/export` is limited by `-search
|
|||||||
|
|
||||||
Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-data-in-json-line-format).
|
Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-data-in-json-line-format).
|
||||||
|
|
||||||
|
The [deduplication](#deduplication) is applied to the data exported via `/api/v1/export` by default. The deduplication
|
||||||
|
isn't applied if `reduce_mem_usage=1` query arg is passed to the request.
|
||||||
|
|
||||||
|
|
||||||
### How to export CSV data
|
### How to export CSV data
|
||||||
|
|
||||||
@ -830,6 +833,8 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
|
|||||||
|
|
||||||
The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data).
|
The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data).
|
||||||
|
|
||||||
|
The [deduplication](#deduplication) isn't applied for the data exported in CSV. It is expected that the de-duplication is performed during data import.
|
||||||
|
|
||||||
|
|
||||||
### How to export data in native format
|
### How to export data in native format
|
||||||
|
|
||||||
@ -853,6 +858,8 @@ The exported data can be imported to VictoriaMetrics via [/api/v1/import/native]
|
|||||||
The native export format may change in incompatible way between VictoriaMetrics releases, so the data exported from the release X
|
The native export format may change in incompatible way between VictoriaMetrics releases, so the data exported from the release X
|
||||||
can fail to be imported into VictoriaMetrics release Y.
|
can fail to be imported into VictoriaMetrics release Y.
|
||||||
|
|
||||||
|
The [deduplication](#deduplication) isn't applied for the data exported in native format. It is expected that the de-duplication is performed during data import.
|
||||||
|
|
||||||
|
|
||||||
## How to import time series data
|
## How to import time series data
|
||||||
|
|
||||||
|
Loading…
Reference in New Issue
Block a user