From b21860b481f4463a5730486b828fc5fbf73b087b Mon Sep 17 00:00:00 2001 From: Vika Date: Fri, 24 Nov 2023 10:53:22 +0000 Subject: [PATCH] update wiki pages --- CHANGELOG.md | 1 + Cluster-VictoriaMetrics.md | 2 +- README.md | 4 ++-- Single-server-VictoriaMetrics.md | 4 ++-- vmagent.md | 2 +- 5 files changed, 7 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 644b53c..8e2fd6c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,7 @@ The sandbox cluster installation is running under the constant load generated by Released at 2023-11-16 * FEATURE: dashboards: use `version` instead of `short_version` in version change annotation for single/cluster dashboards. The update should reflect version changes even if different flavours of the same release were applied (custom builds). +* FEATURE: lower limit for `import.maxLineLen` cmd-line flag from 100MB to 10MB in order to prevent excessive memory usage during data import. Please note, the line length of exported data can be limited with `max_rows_per_line` query arg passed to `/api/v1/export`. The change affects vminsert/vmagent/VictoriaMetrics single-node. * BUGFIX: fix a bug, which could result in improper results and/or to `cannot merge series: duplicate series found` error during [range query](https://docs.victoriametrics.com/keyConcepts.html#range-query) execution. The issue has been introduced in [v1.95.0](https://docs.victoriametrics.com/CHANGELOG.html#v1950). See [this bugreport](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5332) for details. * BUGFIX: improve deadline detection when using buffered connection for communication between cluster components. Before, due to nature of a buffered connection the deadline could have been exceeded while reading or writing buffered data to connection. See [this pull request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5327). diff --git a/Cluster-VictoriaMetrics.md b/Cluster-VictoriaMetrics.md index 3b68ba0..cae5314 100644 --- a/Cluster-VictoriaMetrics.md +++ b/Cluster-VictoriaMetrics.md @@ -949,7 +949,7 @@ Below is the output for `/path/to/vminsert -help`: Whether to use proxy protocol for connections accepted at -httpListenAddr . See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt . With enabled proxy protocol http server cannot serve regular /metrics endpoint. Use -pushmetrics.url for metrics pushing -import.maxLineLen size The maximum length in bytes of a single line accepted by /api/v1/import; the line length can be limited with 'max_rows_per_line' query arg passed to /api/v1/export - Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 104857600) + Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 10485760) -influx.databaseNames array Comma-separated list of database names to return from /query and /influx/query API. This can be needed for accepting data from Telegraf plugins such as https://github.com/fangli/fluent-plugin-influxdb Supports an array of values separated by comma or specified via multiple flags. diff --git a/README.md b/README.md index 2e56fcb..443ba34 100644 --- a/README.md +++ b/README.md @@ -1594,7 +1594,7 @@ The format follows [JSON streaming concept](http://ndjson.org/), e.g. each line ``` Note that every JSON object must be written in a single line, e.g. all the newline chars must be removed from it. -Every line length is limited by the value passed to `-import.maxLineLen` command-line flag (by default this is 100MB). +Every line length is limited by the value passed to `-import.maxLineLen` command-line flag (by default this is 10MB). It is recommended passing 1K-10K samples per line for achieving the maximum data ingestion performance at [/api/v1/import](#how-to-import-data-in-json-line-format). Too long JSON lines may increase RAM usage at VictoriaMetrics side. @@ -2628,7 +2628,7 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li Whether to use proxy protocol for connections accepted at -httpListenAddr . See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt . With enabled proxy protocol http server cannot serve regular /metrics endpoint. Use -pushmetrics.url for metrics pushing -import.maxLineLen size The maximum length in bytes of a single line accepted by /api/v1/import; the line length can be limited with 'max_rows_per_line' query arg passed to /api/v1/export - Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 104857600) + Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 10485760) -influx.databaseNames array Comma-separated list of database names to return from /query and /influx/query API. This can be needed for accepting data from Telegraf plugins such as https://github.com/fangli/fluent-plugin-influxdb Supports an array of values separated by comma or specified via multiple flags. diff --git a/Single-server-VictoriaMetrics.md b/Single-server-VictoriaMetrics.md index 9c8f6a6..1a234a9 100644 --- a/Single-server-VictoriaMetrics.md +++ b/Single-server-VictoriaMetrics.md @@ -1602,7 +1602,7 @@ The format follows [JSON streaming concept](http://ndjson.org/), e.g. each line ``` Note that every JSON object must be written in a single line, e.g. all the newline chars must be removed from it. -Every line length is limited by the value passed to `-import.maxLineLen` command-line flag (by default this is 100MB). +Every line length is limited by the value passed to `-import.maxLineLen` command-line flag (by default this is 10MB). It is recommended passing 1K-10K samples per line for achieving the maximum data ingestion performance at [/api/v1/import](#how-to-import-data-in-json-line-format). Too long JSON lines may increase RAM usage at VictoriaMetrics side. @@ -2636,7 +2636,7 @@ Pass `-help` to VictoriaMetrics in order to see the list of supported command-li Whether to use proxy protocol for connections accepted at -httpListenAddr . See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt . With enabled proxy protocol http server cannot serve regular /metrics endpoint. Use -pushmetrics.url for metrics pushing -import.maxLineLen size The maximum length in bytes of a single line accepted by /api/v1/import; the line length can be limited with 'max_rows_per_line' query arg passed to /api/v1/export - Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 104857600) + Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 10485760) -influx.databaseNames array Comma-separated list of database names to return from /query and /influx/query API. This can be needed for accepting data from Telegraf plugins such as https://github.com/fangli/fluent-plugin-influxdb Supports an array of values separated by comma or specified via multiple flags. diff --git a/vmagent.md b/vmagent.md index 754e105..e577abc 100644 --- a/vmagent.md +++ b/vmagent.md @@ -1494,7 +1494,7 @@ See the docs at https://docs.victoriametrics.com/vmagent.html . Whether to use proxy protocol for connections accepted at -httpListenAddr . See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt . With enabled proxy protocol http server cannot serve regular /metrics endpoint. Use -pushmetrics.url for metrics pushing -import.maxLineLen size The maximum length in bytes of a single line accepted by /api/v1/import; the line length can be limited with 'max_rows_per_line' query arg passed to /api/v1/export - Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 104857600) + Supports the following optional suffixes for size values: KB, MB, GB, TB, KiB, MiB, GiB, TiB (default 10485760) -influx.databaseNames array Comma-separated list of database names to return from /query and /influx/query API. This can be needed for accepting data from Telegraf plugins such as https://github.com/fangli/fluent-plugin-influxdb Supports an array of values separated by comma or specified via multiple flags.