VictoriaMetrics/app/vmagent
Aliaksandr Valialkin 172ae1adf7
Revert c6c5a5a186 and b2765c45d0
Reason for revert:

There are many statsd servers exist:

- https://github.com/statsd/statsd - classical statsd server
- https://docs.datadoghq.com/developers/dogstatsd/ - statsd server from DataDog built into DatDog Agent ( https://docs.datadoghq.com/agent/ )
- https://github.com/avito-tech/bioyino - high-performance statsd server
- https://github.com/atlassian/gostatsd - statsd server in Go
- https://github.com/prometheus/statsd_exporter - statsd server, which exposes the aggregated data as Prometheus metrics

These servers can be used for efficient aggregating of statsd data and sending it to VictoriaMetrics
according to https://docs.victoriametrics.com/#how-to-send-data-from-graphite-compatible-agents-such-as-statsd (
the https://github.com/prometheus/statsd_exporter can be scraped as usual Prometheus target
according to https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter ).

Adding support for statsd data ingestion protocol into VictoriaMetrics makes sense only if it provides
significant advantages over the existing statsd servers, while has no significant drawbacks comparing
to existing statsd servers.

The main advantage of statsd server built into VictoriaMetrics and vmagent - getting rid of additional statsd server.
The main drawback is non-trivial and inconvenient streaming aggregation configs, which must be used for the ingested statsd metrics (
see https://docs.victoriametrics.com/stream-aggregation/ ). These configs are incompatible with the configs for standalone statsd servers.
So you need to manually translate configs of the used statsd server to stream aggregation configs when migrating
from standalone statsd server to statsd server built into VictoriaMetrics (or vmagent).

Another important drawback is that it is very easy to shoot yourself in the foot when using built-in statsd server
with the -statsd.disableAggregationEnforcement command-line flag or with improperly configured streaming aggregation.
In this case the ingested statsd metrics will be stored to VictoriaMetrics as is without any aggregation.
This may result in high CPU usage during data ingestion, high disk space usage for storing all the unaggregated
statsd metrics and high CPU usage during querying, since all the unaggregated metrics must be read, unpacked and processed
during querying.

P.S. Built-in statsd server can be added to VictoriaMetrics and vmagent after figuring out more ergonomic
specialized configuration for aggregating of statsd metrics. The main requirements for this configuration:

- easy to write, read and update (ideally it should work out of the box for most cases without additional configuration)
- hard to misconfigure (e.g. hard to shoot yourself in the foot)

It would be great if this configuration will be compatible with the configuration of the most widely used statsd server.

In the mean time it is recommended continue using external statsd server.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6265
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5053
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5052
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/206
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4600
2024-07-03 23:57:49 +02:00
..
common app/vmagent/common: use plain sync.Pool instead of a mix of sync.Pool with channel-based pool for PushCtx 2024-04-20 21:31:14 +02:00
csvimport app/vmagent: follow-up for 090cb2c9de 2023-11-25 12:13:39 +02:00
datadogsketches app/{vmagent,vminsert}: follow-up after a1d1ccd6f2 2024-02-07 01:31:52 +02:00
datadogv1 app/{vminsert,vmagent}: preliminary support for /api/v2/series ingestion from new versions of DataDog Agent 2023-12-21 20:50:27 +02:00
datadogv2 lib/protoparser/datadogv2: take into account source_type_name field, since it contains useful value such as kubernetes, docker, system, etc. 2023-12-21 23:05:52 +02:00
deployment
graphite app/{vminsert,vmagent}: preliminary support for /api/v2/series ingestion from new versions of DataDog Agent 2023-12-21 20:50:27 +02:00
influx app/vmagent/influx: replace hybrid channel-based pool + sync.Pool with plain sync.Pool for pushCtx 2024-04-20 21:38:25 +02:00
multiarch deployment: build image for vmagent streamaggr benchmark (#6515) 2024-06-24 16:29:14 +02:00
native app/vmagent: follow-up for 090cb2c9de 2023-11-25 12:13:39 +02:00
newrelic app/vmagent: follow-up for 090cb2c9de 2023-11-25 12:13:39 +02:00
opentelemetry app/{vmagent/insert} fix typo in Firehose 2024-04-03 02:51:57 +03:00
opentsdb app/vmagent: follow-up for 090cb2c9de 2023-11-25 12:13:39 +02:00
opentsdbhttp app/vmagent: follow-up for 090cb2c9de 2023-11-25 12:13:39 +02:00
prometheusimport app/vmagent: follow-up for 090cb2c9de 2023-11-25 12:13:39 +02:00
promremotewrite lib/prompb: change type of Label.Name and Label.Value from []byte to string 2024-01-16 20:41:37 +02:00
remotewrite Revert c6c5a5a186 and b2765c45d0 2024-07-03 23:57:49 +02:00
static/css all: follow-up after 8edb390e21 2022-06-07 01:05:53 +03:00
vmimport app/vmagent: follow-up for 090cb2c9de 2023-11-25 12:13:39 +02:00
main.go Revert c6c5a5a186 and b2765c45d0 2024-07-03 23:57:49 +02:00
Makefile Add build support for loong64 (#6222) 2024-05-10 14:32:05 +02:00
README.md all: replace the outdated url https://docs.victoriametrics.com/vmagent.html with the new one - https://docs.victoriametrics.com/vmagent/ 2024-04-18 01:32:57 +02:00
vmagent.png

See vmagent docs here.

vmagent docs can be edited at docs/vmagent.md.