461 Commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	f4f1f2f976	docs/vmagent.md: remove the claim that VictoriaMetrics remote write protocol reduces the network bandwidth usage by up to 10x comparing to Prometheus remote write protocol ... The 10x savings are reproduced only on artificial data. The savings on production data are usually in the range 2x-4x.	2023-02-20 19:11:31 -08:00
Aliaksandr Valialkin	a678cbe62e	docs/vmagent.md: mention that Mimir doesnt support backfilling	2023-02-20 19:11:30 -08:00
Aliaksandr Valialkin	76f2c70be3	app/vmagent: add support for VictoriaMetrics remote write protocol, which allows saving up to 10x on network bandwidth costs under high load ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1225	2023-02-20 19:11:30 -08:00
Aliaksandr Valialkin	ccdddf7996	lib/protoparser/promremotewrite: extract stream parsing code into a separate stream package ... This is a follow-up for 057698f7fb	2023-02-13 10:46:54 -08:00
Aliaksandr Valialkin	9be1398b92	lib/protoparser/native: extract stream parsing code into a separate stream package ... This is a follow-up for 057698f7fb	2023-02-13 10:43:05 -08:00
Aliaksandr Valialkin	8830607021	lib/protoparser/graphite: extract stream parsing code into a separate stream package	2023-02-13 10:32:36 -08:00
Aliaksandr Valialkin	a646841c07	lib/protoparser/csvimport: extract stream parsing code into a separate stream package ... This is a follow-up for 057698f7fb	2023-02-13 10:25:46 -08:00
Aliaksandr Valialkin	7568658c19	lib/protoparser/vmimport: extract stream parsing code into a separate stream package ... This is a follow-up for 057698f7fb	2023-02-13 10:20:19 -08:00
Aliaksandr Valialkin	af37717108	lib/protoparser/opentsdbhttp: extract stream parsing code into a separate stream package ... This is a follow-up for 057698f7fb	2023-02-13 10:16:03 -08:00
Aliaksandr Valialkin	7720d403c0	lib/protoparser/opentsdb: extract stream parsing code into a separate stream package ... This is a follow-up for 057698f7fb	2023-02-13 10:03:16 -08:00
Aliaksandr Valialkin	fe196e0b7a	lib/protoparser/influx: extract stream parsing code into a separate stream package ... This is a follow-up for 057698f7fb	2023-02-13 09:58:52 -08:00
Aliaksandr Valialkin	f83d6d69b2	lib/protoparser/datadog: extract stream parsing code into a separate stream package ... This is a follow-up for 057698f7fb	2023-02-13 09:51:47 -08:00
Roman Khavronenko	057698f7fb	lib/protoparser/prometheus: move streamparser to subpackage (#3814) ... `lib/protoparser/prometheus` is used by various applications, such as `app/vmalert`. The recent change to the `lib/protoparser/prometheus` package introduced a new dependency of `lib/writeconcurrencylimiter` which exposes some metrics. Because of the dependency, now all applications which have this dependency also expose these metrics. Creating a new `lib/protoparser/prometheus/stream` package helps to remove these metrics from apps which use `lib/protoparser/prometheus` as dependency. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3761 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-02-13 09:26:07 -08:00
Aliaksandr Valialkin	0e0095d350	all: run apk update && apk upgrade in base Alpine Docker image in order to get all the recent security fixes	2023-02-09 14:01:32 -08:00
Aliaksandr Valialkin	f40661e7b7	docs/vmagent.md: clarify that automatically generated metrics contain all the target-specific labels, including instance and job	2023-02-09 11:04:51 -08:00
Aliaksandr Valialkin	06f6b76521	app/vmagent: properly return 200 response code when importing data via Prometheus PushGateway protocol ... This is the same fix as has been already applied to app/vminsert at cdb6d651e9 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3636 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1415	2023-01-27 14:40:02 -08:00
Aliaksandr Valialkin	0890adde67	docs: update command-line descriptions after 73256fe438	2023-01-27 00:00:37 -08:00
Aliaksandr Valialkin	cb374677a9	app/vmagent/prometheusimport: delete the temporary directory created by vmagent after the test is complete ... This is a follow-up for 1cfa183c2b	2023-01-26 23:21:24 -08:00
Nikolay	73256fe438	lib/netutil: init implimentation of proxy protocol (#3687) ... * lib/netutil: init implimentation of proxy protocol https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3335 * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-26 23:08:35 -08:00
Roman Khavronenko	95ee86b600	docs: specify the time window for series_limit (#3708) ... Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com>	2023-01-25 09:30:20 -08:00
Aliaksandr Valialkin	28f66f0079	docs: update the list of command-line flags according to the latest changes	2023-01-25 09:20:24 -08:00
Aliaksandr Valialkin	d655d6b047	lib/streamaggr: add ability to de-duplicate input samples before aggregation	2023-01-25 09:14:49 -08:00
Aliaksandr Valialkin	c496f06ca3	app/vmagent/{promremotewrite,vmimport}: remove unused functions InsertHandlerForReader() ... Thanks to 1cfa183c2b , where the first such function has been removed	2023-01-23 22:31:29 -08:00
Aliaksandr Valialkin	f7acdb13db	app/{vmagent,vminsert}: follow-up for 1cfa183c2b ... - Call httpserver.GetQuotedRemoteAddr() and httpserver.GetRequestURI() only when the error occurs. This saves CPU time on fast path when there are no parsing errors. - Create a helper function - httpserver.LogError() - for logging the error with the request uri and remote addr context.	2023-01-23 22:26:53 -08:00
Artem Navoiev	1cfa183c2b	add error handler for parsing prometheus text format to vmagent and v… (#3693) ... * add error handler for parsing prometheus text format to vmagent and vminsert Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * fix typo Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * typo Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * fix variables naming and error message Signed-off-by: Artem Navoiev <tenmozes@gmail.com> Signed-off-by: Artem Navoiev <tenmozes@gmail.com>	2023-01-23 22:14:34 -08:00
Aliaksandr Valialkin	95ce1ba6ce	lib/httpserver: directly pass flag value to CheckAuthFlag() ... There is no sense in passing a pointer to flag value there. This is a follow-up for 4225a0bd75	2023-01-10 15:52:23 -08:00
Zakhar Bessarab	4225a0bd75	Use httpAuth.* flags as a fallback for endpoints protected by *AuthKey flags (#3582) ... * {lib/server, app/}: use `httpAuth.*` flag as fallback for `*AuthKey` if it is not set * lib/ingestserver/opentsdbhttp: fix opentdb HTTP handler not respecting `httpAuth.*` flags * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2023-01-10 15:46:13 -08:00
Artem Navoiev	37c52ccaf4	vmagent: add minimal scrape file exampe for vmagent quick start (#3627) ... * vmagent: add minimal scrape file exampe for vmagent quick start Signed-off-by: Artem Navoiev <tenmozes@gmail.com> * replace example with link to your prometheus.yml in docker Signed-off-by: Artem Navoiev <tenmozes@gmail.com> Signed-off-by: Artem Navoiev <tenmozes@gmail.com>	2023-01-10 23:16:10 +01:00
Aliaksandr Valialkin	af263fe881	all: small improvements in error messages and command-line flag descriptions related to concurrency limiters	2023-01-07 00:11:44 -08:00
Aliaksandr Valialkin	c63755c316	lib/writeconcurrencylimiter: improve the logic behind -maxConcurrentInserts limit ... Previously the -maxConcurrentInserts was limiting the number of established client connections, which write data to VictoriaMetrics. Some of these connections could be idle. Such connections do not consume big amounts of CPU and RAM, so there is a little sense in limiting the number of such connections. So now the -maxConcurrentInserts command-line option limits the number of concurrently executed insert requests, not including idle connections. It is recommended removing -maxConcurrentInserts command-line option, since the default value for this option should work good for most cases.	2023-01-06 22:20:19 -08:00
Karan Sharma	48f371a46c	lib/promscrape: add Prometheus-compatible service discovery for Nomad (#3549) ... Add nomad_sd_config support for service discovery	2023-01-05 23:03:58 +01:00
Aliaksandr Valialkin	0dea3b71da	lib/promscrape: pre-fetch metric_relabel_configs rules when debugging metric relabeling for a particular target ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407	2023-01-05 03:26:49 -08:00
Aliaksandr Valialkin	fd175ad80b	docs: update -help outputs for vm* tools	2023-01-03 23:27:06 -08:00
Aliaksandr Valialkin	fa13bbc48a	app/{vmagent,vminsert}: add support for streaming aggregation ... See https://docs.victoriametrics.com/stream-aggregation.html Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3460	2023-01-03 22:19:21 -08:00
Aliaksandr Valialkin	f33e687723	app/vmagent/remotewrite: improve descriptions for -remoteWrite.relabelConfig and -remoteWrite.urlRelabelConfig ... - Cross-reference these command-line flags. - Add a link to https://docs.victoriametrics.com/vmagent.html#relabeling	2023-01-03 22:04:49 -08:00
Artem Navoiev	7d9c4bebc0	update links to grafana dashboards (#3534) ... docs: update links to grafana dashboards Signed-off-by: Artem Navoiev <tenmozes@gmail.com>	2022-12-25 17:36:20 +01:00
Roman Khavronenko	e4e9dfb785	vmagent: respect -usePromCompatibleNaming if no relabeling is set (#3511) ... * vmagent: respect `-usePromCompatibleNaming` if no relabeling is set See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3493 Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmagent: upd test Signed-off-by: hagen1778 <roman@victoriametrics.com> Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-12-20 14:32:45 -08:00
Aliaksandr Valialkin	4de9d35458	lib/flagutil/bytes.go: properly handle values bigger than 2GiB on 32-bit architectures ... This fixes handling of values bigger than 2GiB for the following command-line flags: - -storage.minFreeDiskSpaceBytes - -remoteWrite.maxDiskUsagePerURL	2022-12-14 19:26:31 -08:00
Aliaksandr Valialkin	5d30080555	lib/flagutil: support for TB and TiB suffixes for command-line flags, which accept byte sizes	2022-12-14 17:52:32 -08:00
Aliaksandr Valialkin	231569f89b	docs/vmagent.md: small formatting fix	2022-12-14 14:25:35 -08:00
Aliaksandr Valialkin	7a61bafe59	docs/vmagent.md: clarify that relabeling is actually a debugging at relabel debug section	2022-12-13 15:46:21 -08:00
Aliaksandr Valialkin	88597f187b	app/{vmagent,vminsert}/datadog: make the host label optional in DataDog data ingestion protocol ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3432	2022-12-10 23:32:31 -08:00
Aliaksandr Valialkin	b01607e3fb	docs: clarify that single-node VictoriaMetrics also provides functionality for relabel debugging ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407	2022-12-10 20:49:52 -08:00
Aliaksandr Valialkin	a30ae502ef	lib/promscrape: allow editing relabeling configs and labels at /target-relabel-debug page ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407	2022-12-10 12:44:45 -08:00
Aliaksandr Valialkin	a8b8e23d68	lib/promscrape: implement target-level and metric-level relabel debugging ... Target-level debugging is performed by clicking the 'debug' link at the corresponding target on either http://vmagent:8429/targets page or on http://vmagent:8428/service-discovery page. Metric-level debugging is perfromed at http://vmagent:8429/metric-relabel-debug page. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3407 See https://docs.victoriametrics.com/vmagent.html#relabel-debug	2022-12-10 02:09:44 -08:00
Aliaksandr Valialkin	eeacbaf0b6	all: update Go builder from v1.19.3 to v1.19.4 ... See https://github.com/golang/go/issues?q=milestone%3AGo1.19.4+label%3ACherryPickApproved	2022-12-08 16:41:24 -08:00
Aliaksandr Valialkin	56b8980915	lib/promscrape: allow using sample_limit and series_limit options in stream parsing mode ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3458	2022-12-08 16:33:38 -08:00
Pedro Gonçalves	1e0666abb4	Datadog - Add device as a tag if it's present as a field in the series object (#3431) ... * Datadog - Add device as a tag if it's present as a field in the series object * address PR comments	2022-12-05 23:06:03 -08:00
Aliaksandr Valialkin	14660d4df5	all: typo fix: the the -> the	2022-12-03 21:53:01 -08:00
Aliaksandr Valialkin	654e94f420	lib/promscrape: add exported_ prefix to metric names exported by scrape targets if they clash with automatically generated metrics ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3406	2022-11-28 18:37:09 -08:00