Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-23 20:37:12 +01:00
Code Issues Actions 8 Packages Projects Releases Wiki Activity

docs/Single-server-VictoriaMetrics.md: document that the deduplication is applied only when exporting data in JSON line format

Browse Source 
The exported data isn't de-duplicated by default due to performance reasons.
It is expected that the de-duplication is applied during importing the exported data.

The deduplication is applied only when exporting data via /api/v1/export if `reduce_mem_usage=1` query arg isn't passed to the request.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1837
This commit is contained in:
Aliaksandr Valialkin 2021-11-30 13:02:50 +02:00
parent d666755159
commit f06495c50a
No known key found for this signature in database
GPG Key ID: A72BEC6CD3D0DED1
3 changed files with 24 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
README.md
View File

@ -789,7 +789,7 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
Optional `max_rows_per_line` arg may be added to the request for limiting the maximum number of rows exported per each JSON line. Optional `max_rows_per_line` arg may be added to the request for limiting the maximum number of rows exported per each JSON line.
Optional `reduce_mem_usage=1` arg may be added to the request for reducing memory usage when exporting big number of time series. Optional `reduce_mem_usage=1` arg may be added to the request for reducing memory usage when exporting big number of time series.
In this case the output may contain multiple lines with distinct samples for the same time series. In this case the output may contain multiple lines with samples for the same time series.
Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts
of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data: of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data:
@ -802,6 +802,9 @@ The maximum duration for each request to `/api/v1/export` is limited by `-search
Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-data-in-json-line-format). Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-data-in-json-line-format).
The [deduplication](#deduplication) is applied to the data exported via `/api/v1/export` by default. The deduplication
isn't applied if `reduce_mem_usage=1` query arg is passed to the request.
### How to export CSV data ### How to export CSV data
@ -826,6 +829,8 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data). The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data).
The [deduplication](#deduplication) isn't applied for the data exported in CSV. It is expected that the de-duplication is performed during data import.
### How to export data in native format ### How to export data in native format
@ -849,6 +854,8 @@ The exported data can be imported to VictoriaMetrics via [/api/v1/import/native]
The native export format may change in incompatible way between VictoriaMetrics releases, so the data exported from the release X The native export format may change in incompatible way between VictoriaMetrics releases, so the data exported from the release X
can fail to be imported into VictoriaMetrics release Y. can fail to be imported into VictoriaMetrics release Y.
The [deduplication](#deduplication) isn't applied for the data exported in native format. It is expected that the de-duplication is performed during data import.
## How to import time series data ## How to import time series data

9
docs/README.md
View File

@ -789,7 +789,7 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
Optional `max_rows_per_line` arg may be added to the request for limiting the maximum number of rows exported per each JSON line. Optional `max_rows_per_line` arg may be added to the request for limiting the maximum number of rows exported per each JSON line.
Optional `reduce_mem_usage=1` arg may be added to the request for reducing memory usage when exporting big number of time series. Optional `reduce_mem_usage=1` arg may be added to the request for reducing memory usage when exporting big number of time series.
In this case the output may contain multiple lines with distinct samples for the same time series. In this case the output may contain multiple lines with samples for the same time series.
Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts
of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data: of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data:
@ -802,6 +802,9 @@ The maximum duration for each request to `/api/v1/export` is limited by `-search
Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-data-in-json-line-format). Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-data-in-json-line-format).
The [deduplication](#deduplication) is applied to the data exported via `/api/v1/export` by default. The deduplication
isn't applied if `reduce_mem_usage=1` query arg is passed to the request.
### How to export CSV data ### How to export CSV data
@ -826,6 +829,8 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data). The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data).
The [deduplication](#deduplication) isn't applied for the data exported in CSV. It is expected that the de-duplication is performed during data import.
### How to export data in native format ### How to export data in native format
@ -849,6 +854,8 @@ The exported data can be imported to VictoriaMetrics via [/api/v1/import/native]
The native export format may change in incompatible way between VictoriaMetrics releases, so the data exported from the release X The native export format may change in incompatible way between VictoriaMetrics releases, so the data exported from the release X
can fail to be imported into VictoriaMetrics release Y. can fail to be imported into VictoriaMetrics release Y.
The [deduplication](#deduplication) isn't applied for the data exported in native format. It is expected that the de-duplication is performed during data import.
## How to import time series data ## How to import time series data

9
docs/Single-server-VictoriaMetrics.md
View File

@ -793,7 +793,7 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
Optional `max_rows_per_line` arg may be added to the request for limiting the maximum number of rows exported per each JSON line. Optional `max_rows_per_line` arg may be added to the request for limiting the maximum number of rows exported per each JSON line.
Optional `reduce_mem_usage=1` arg may be added to the request for reducing memory usage when exporting big number of time series. Optional `reduce_mem_usage=1` arg may be added to the request for reducing memory usage when exporting big number of time series.
In this case the output may contain multiple lines with distinct samples for the same time series. In this case the output may contain multiple lines with samples for the same time series.
Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts Pass `Accept-Encoding: gzip` HTTP header in the request to `/api/v1/export` in order to reduce network bandwidth during exporing big amounts
of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data: of time series data. This enables gzip compression for the exported data. Example for exporting gzipped data:
@ -806,6 +806,9 @@ The maximum duration for each request to `/api/v1/export` is limited by `-search
Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-data-in-json-line-format). Exported data can be imported via POST'ing it to [/api/v1/import](#how-to-import-data-in-json-line-format).
The [deduplication](#deduplication) is applied to the data exported via `/api/v1/export` by default. The deduplication
isn't applied if `reduce_mem_usage=1` query arg is passed to the request.
### How to export CSV data ### How to export CSV data
@ -830,6 +833,8 @@ unix timestamp in seconds or [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) val
The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data). The exported CSV data can be imported to VictoriaMetrics via [/api/v1/import/csv](#how-to-import-csv-data).
The [deduplication](#deduplication) isn't applied for the data exported in CSV. It is expected that the de-duplication is performed during data import.
### How to export data in native format ### How to export data in native format
@ -853,6 +858,8 @@ The exported data can be imported to VictoriaMetrics via [/api/v1/import/native]
The native export format may change in incompatible way between VictoriaMetrics releases, so the data exported from the release X The native export format may change in incompatible way between VictoriaMetrics releases, so the data exported from the release X
can fail to be imported into VictoriaMetrics release Y. can fail to be imported into VictoriaMetrics release Y.
The [deduplication](#deduplication) isn't applied for the data exported in native format. It is expected that the de-duplication is performed during data import.
## How to import time series data ## How to import time series data