Aliaksandr Valialkin
8942f290eb
app/vminsert: replace hybrid sync.Pool+channel-based pool scheme for poolCtx with plain sync.Pool
...
This simplifies the code, while doesn't increase memory usage under low and high data ingestion rate.
This is a follow-up for 1decbcf6eb
2024-04-20 21:44:53 +02:00
Aliaksandr Valialkin
7568658c19
lib/protoparser/vmimport: extract stream parsing code into a separate stream package
...
This is a follow-up for 057698f7fb
2023-02-13 10:20:19 -08:00
Aliaksandr Valialkin
c63755c316
lib/writeconcurrencylimiter: improve the logic behind -maxConcurrentInserts limit
...
Previously the -maxConcurrentInserts was limiting the number of established client connections,
which write data to VictoriaMetrics. Some of these connections could be idle.
Such connections do not consume big amounts of CPU and RAM, so there is a little sense in limiting
the number of such connections. So now the -maxConcurrentInserts command-line option
limits the number of concurrently executed insert requests, not including idle connections.
It is recommended removing -maxConcurrentInserts command-line option, since the default value
for this option should work good for most cases.
2023-01-06 22:20:19 -08:00
Nikolay
ad08d9dfc0
changes protoparser apis for accepting reading from io.Reader ( #1624 )
...
adds InsertHandlerForReader apis to vmagent
2021-09-20 14:49:28 +03:00
Aliaksandr Valialkin
dc9eafcd02
app/{vminsert,vmagent}: add -sortLabels command-line option for sorting time series labels before ingesting them in the storage
...
This option can be useful when samples for the same time series are ingested with distinct order of labels.
For example, metric{k1="v1",k2="v2"} and metric{k2="v2",k1="v1"}.
2021-03-31 23:27:58 +03:00
Aliaksandr Valialkin
4146fc4668
all: properly handle CPU limits set on the host system/container
...
This can reduce memory usage on systems with enabled CPU limits.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946
2020-12-08 21:07:29 +02:00
Aliaksandr Valialkin
84227ea2fc
app/{vminsert,vmagent}: take into account all the inserted rows before relabeling in vm_rows_inserted_total and vmagent_rows_inserted_total metrics
2020-10-09 13:29:51 +03:00
Aliaksandr Valialkin
c00627c103
app/vminsert: code prettifying
2020-09-26 04:13:18 +03:00
Aliaksandr Valialkin
f41b36bb9a
app/{vminsert,vmagent}: allow adding extra labels when importing data via Prometheus, CSV and JSON line formats
...
Extra labels may be added to the imported data by passing `extra_label=name=value` query args.
Multiple query args may be passed in order to add multiple extra labels.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/719
2020-09-02 19:43:21 +03:00
Aliaksandr Valialkin
a090627059
app/vminsert: limit memory usage when ingesting data in big packets
2020-07-24 23:32:40 +03:00
Aliaksandr Valialkin
9257eee982
app/vminsert: do not call ApplyRelabeling function if relabeling is disabled
...
This should reduce CPU usage a bit when `-relabelConfig` isn't set
2020-07-23 13:39:44 +03:00
Aliaksandr Valialkin
91b3482894
app/vminsert: add ability to apply relabeling to all the incoming metrics if -relabelConfig command-line arg points to a file with a list of relabel_config entries
...
See https://victoriametrics.github.io/#relabeling
2020-07-02 20:39:28 +03:00
Aliaksandr Valialkin
04762344c6
app/vmagent: initial implementation for vmagent
2020-02-23 13:36:03 +02:00
Aliaksandr Valialkin
42864bb52f
all: do not clash flag description with back-quoted flag types
...
See https://golang.org/pkg/flag/#PrintDefaults for more details.
2020-02-04 15:46:52 +02:00
Aliaksandr Valialkin
4f0a645f77
vendor: update github.com/valyala/fastjson from v1.4.2 to v1.4.5
...
This should fix parsing Inf values in `/api/v1/import`. The previous attempt to fix this in VictoriaMetrics v1.32.1 was unsuccessful.
2020-01-10 23:15:15 +02:00
Aliaksandr Valialkin
68e1cf8942
app/vminsert: add /api/v1/import handler
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6
2019-12-09 20:59:04 +02:00
