Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-15 08:23:34 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

docs/vmagent.md: mention that it may be useful to disable on-disk data persistence when reading data from Kafka or Google PubSub

Browse Source 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2110
This commit is contained in:
Aliaksandr Valialkin 2023-12-04 23:12:02 +02:00
parent 85fcefaa34
commit 5388d7ba12
No known key found for this signature in database
GPG Key ID: 52C003EE2BCDB9EB
1 changed files with 9 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
docs/vmagent.md
View File

@ -1186,6 +1186,10 @@ with the following config:
data_format = "influx"
```
`vmagent` buffers messages read from Google PubSub topic on local disk if the remote storage at `-remoteWrite.url` cannot keep up with the data ingestion rate.
In this case it may be useful to disable on-disk data persistence in order to prevent from unbounded growth of the on-disk queue.
See [these docs](https://docs.victoriametrics.com/vmagent.html#disabling-on-disk-persistence).
#### Consume metrics from multiple topics
`vmagent` can read messages from different topics in different formats. For example, the following command starts `vmagent`, which reads plaintext
@ -1292,7 +1296,7 @@ by passing multiple `-kafka.consumer.topic` command-line flags to `vmagent`.
`vmagent` consumes messages from Kafka brokers specified by `-kafka.consumer.topic.brokers` command-line flag.
Multiple brokers can be specified per each `-kafka.consumer.topic` by passing a list of brokers delimited by `;`.
For example, `-kafka.consumer.topic.brokers=host1:9092;host2:9092`.
For example, `-kafka.consumer.topic.brokers='host1:9092;host2:9092'`.
The following command starts `vmagent`, which reads metrics in InfluxDB line protocol format from Kafka broker at `localhost:9092`
from the topic `metrics-by-telegraf` and sends them to remote storage at `http://localhost:8428/api/v1/write`:
@ -1314,6 +1318,10 @@ topic = "influx"
data_format = "influx"
```
`vmagent` buffers messages read from Kafka topic on local disk if the remote storage at `-remoteWrite.url` cannot keep up with the data ingestion rate.
In this case it may be useful to disable on-disk data persistence in order to prevent from unbounded growth of the on-disk queue.
See [these docs](https://docs.victoriametrics.com/vmagent.html#disabling-on-disk-persistence).
#### Command-line flags for Kafka consumer
These command-line flags are available only in [enterprise](https://docs.victoriametrics.com/enterprise.html) version of `vmagent`,