103 Commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	899d2c40fb	lib/promrelabel: optimize action: labelmap for non-trivial regexs	2022-09-30 10:43:31 +03:00
Aliaksandr Valialkin	fda60b3d4d	lib/promrelabel: properly parse regex with escaped $ at the end ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3131 Thanks to @dmitryk-dk for the initial fix at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3179	2022-09-30 08:15:43 +03:00
Aliaksandr Valialkin	6a32a64073	lib/bytesutil: add FastStringTransformer and use it in the rest of the code where needed	2022-09-28 10:41:00 +03:00
Aliaksandr Valialkin	ef435f8cc4	lib/promrelabel: add SanitizeName() function for sanitizing Prometheus metric names and label names ... Optimize this function by using results cache for input strings. Use this function all over the code. This is a follow-up for fcffdba9dc Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113	2022-09-28 10:40:59 +03:00
Aliaksandr Valialkin	a2cd79576f	lib/promrelabel: call PromRegex.MatchString() on a slow path only if it contains non-empty literal prefix ... This should improve slow path speed for regexps without literal prefixes	2022-08-26 21:48:30 +03:00
Aliaksandr Valialkin	f49c9bb700	lib/promrelabel: optimize common regex mismatch cases for action: replace and action: labelmap	2022-08-26 15:45:31 +03:00
Aliaksandr Valialkin	4c6916f32a	lib/promrelabel: use regexutil.PromRegex for regex matching in actions labeldrop,labelkeep,drop and keep ... This makes possible optimizing additional cases inside regexutil.PromRegex	2022-08-26 15:23:45 +03:00
Aliaksandr Valialkin	7afe8450fc	lib/promrelabel: optimize matching for commonly used regex patterns in if option ... The following regex patterns are optimized: - literal string match, e.g. "foo" - prefix match, e.g. "foo.*" and "foo.+" - substring match, e.g. ".*foo.*" and ".+foo.+" - alternate values match, e.g. "foo|bar|baz"	2022-08-26 14:53:06 +03:00
Aliaksandr Valialkin	0ad3bbadd3	lib/regexutil: add Simplify() function for simplifying the regular expression	2022-08-26 11:57:12 +03:00
Aliaksandr Valialkin	b373661988	lib/promrelabel: optimize action: {drop,keep,labeldrop,labelkeep} with anchored regex prefix ... The following commonly used relabeling rules must work faster now: - action: labeldrop regex: "^foo.+$" - action: labeldrop regex: "^bar.*"	2022-08-25 23:23:55 +03:00
Aliaksandr Valialkin	0d4ea03a73	lib/promrelabel: optimize action: {labeldrop,labelkeep,keep,drop} with regex containing alternate values ... For example, the following relabeling rule must work much faster now: - action: labeldrop regex: "foo|bar|baz"	2022-08-24 17:54:29 +03:00
Aliaksandr Valialkin	8550c44e31	app/vmagent: add ability to construct a label from multiple existing labels by referring them in the replacement field during relabeling ... For example: - target_label: composite-label replacement: "{{source_label1}}-{{source_label2}}"	2022-08-21 22:50:01 +03:00
Roman Khavronenko	d3f13ab85b	lib/promrelabel: fix expected test result (#2957) ... follow-up after 68c4ec9472 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-08-08 13:47:29 +03:00
Aliaksandr Valialkin	68c4ec9472	lib/promrelabel: do not split regex into multiple lines if it contains groups ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2928	2022-08-08 03:15:26 +03:00
Aliaksandr Valialkin	814bb1685f	all: fix other typos in the same way as 6f4d9b2a48 does	2022-07-18 12:08:15 +03:00
Aliaksandr Valialkin	e6ed92529b	all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package ... This package already has the same name, so there is no need in explicit name	2022-06-21 20:23:32 +03:00
Aliaksandr Valialkin	c18f8cccfa	lib/promrelabel: support action: graphite relabeling ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2737	2022-06-16 20:24:22 +03:00
Aliaksandr Valialkin	ca689fec54	docs/CHANGELOG.md: follow-up after 2177089f94	2022-06-01 14:51:26 +03:00
Roman Khavronenko	2177089f94	promrelabel: add support of lowercase and uppercase relabeling actions (#2665) ... * promrelabel: add support of `lowercase` and `uppercase` relabeling actions https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2664 Signed-off-by: hagen1778 <roman@victoriametrics.com> * lib/storage: make golangci-lint happy Signed-off-by: hagen1778 <roman@victoriametrics.com> Co-authored-by: Nikolay <nik@victoriametrics.com>	2022-06-01 10:02:37 +02:00
Aliaksandr Valialkin	123aa4c79e	lib/promscrape: properly implement ScrapeConfig.clone() ... Previously ScrapeConfig.clone() was improperly copying promauth.Secret fields - their contents was replaced with `<secret>` value. This led to inability to use passwords and secrets in `-promscrape.config` file. The bug has been introduced in v1.77.0 in the commit 67b10896d2 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2551	2022-05-07 00:05:40 +03:00
Aliaksandr Valialkin	3d0549c982	lib/promscrape: optimize getScrapeWork() function ... Reduce the number of memory allocations in this function. This improves its performance by up to 50%. This should improve service discovery speed when big number of potential targets with big number of meta-labels are generated by service discovery. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2270	2022-04-20 15:37:00 +03:00
Dmytro Kozlov	a3ee275149	lib/promscrape: Enable filters for endpoint and labels (#2466) ... * lib/promscrape: Enable filters for endpoint and labels * lib/promscrape: cleanup * lib/promscrape: update template * lib/promscrape: move logic filter logic to backend * lib/promscrape: updated placeholder * lib/promscrape: updated placeholder * lib/promscrape: use two different fields for filters, updated form, added error on parsing queries * lib/promscrape: rename functions * lib/promscrape: removed unused values * wip * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-19 18:26:21 +03:00
Aliaksandr Valialkin	3f49bdaeff	lib/promrelabel: add support for conditional relabeling via if filter ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998	2022-02-24 02:27:26 +02:00
Aliaksandr Valialkin	afafeb379a	all: typo fix: unexected -> unexpected	2021-12-20 17:39:52 +02:00
Aliaksandr Valialkin	06642d97f5	app: allow specifying http and https urls in the following command-line flags ... * -promscrape.config * -relabelConfig * -remoteWrite.relabelConfig * -remoteWrite.urlRelabelConfig	2021-12-03 00:10:02 +02:00
Aliaksandr Valialkin	207c5760ce	lib/promrelabel: fix parsing regex: true in relabeling rules	2021-09-21 23:00:53 +03:00
Aliaksandr Valialkin	d6bd956930	lib/promrelabel: add keep_metrics and drop_metrics actions to relabeling rules ... These actions simlify metrics filtering. For example, - action: keep_metrics regex: 'foo|bar|baz' would leave only metrics with `foo`, `bar` and `baz` names, while the rest of metrics will be deleted. The commit also makes possible to split long regexps into multiple lines. For example, the following config is equivalent to the config above: - action: keep_metrics regex: - foo - bar - baz	2021-09-09 16:18:21 +03:00
Aliaksandr Valialkin	bb31117555	lib/promrelabel: add tests for verifying that regex works as expected in single quotes and double quotes	2021-07-27 10:50:55 +03:00
Aliaksandr Valialkin	661d2668f8	lib/promrelabel: add tests for labelsToString() function	2021-06-04 20:42:46 +03:00
Aliaksandr Valialkin	78f83dc5ad	app/{vmagent,vminsert}: follow-up after 2fe045e2a4 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1343	2021-06-04 20:27:58 +03:00
jelmd	2fe045e2a4	new feature: debug relabeling (#1344) ... * new feature: relabel logging Use scrape_configs[x].relabel_debug = true to log metric names inkl. labels before and after relabeling. After relabeling related metrics get dropped, i.e. not submitted to servers. * vminsert wants relabel logging, too.	2021-06-04 17:50:23 +03:00
Aliaksandr Valialkin	ec6a284978	lib/promrelabel: add tests for conditional removal of label on another label match ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1294	2021-05-18 00:23:03 +03:00
Aliaksandr Valialkin	9a930720b3	lib/promrelabel: add tests for removing the specified {label="value"} pair	2021-05-03 11:26:53 +03:00
Aliaksandr Valialkin	a939667ce0	lib/promrelabel: remove unneded optimizations for labeldrop and labelkeep actions ... These optimizations may slow down code execution by matching the same label against regexp two times instead of a single time	2021-03-02 17:55:43 +02:00
Aliaksandr Valialkin	f7049e2af7	lib/promrelabel: optimize labeldrop and labelkeep relabeling for prefix.* and prefix.+ regexps	2021-02-24 17:58:28 +02:00
Aliaksandr Valialkin	d136081040	lib/promrelabel: add more optimizations for relabeling for common cases	2021-02-22 16:33:55 +02:00
Aliaksandr Valialkin	dd1e53b119	lib/promrelabel: optimize relabeling performance for common cases	2021-02-22 00:51:13 +02:00
Vasily	6fcbd17bdd	Add omitempty for DisableCompression and DisableKeepAlive fields in ScrapeConfig (#796) ... * Add omitempty for DisableCompression and DisableKeepAlive fields in ScrapeConfig * Add omitempty annotation to all the default/optional values * Fix annotations after review	2020-11-13 16:19:05 +02:00
Aliaksandr Valialkin	83df20b5b5	lib/promscrape: clean references to label name and label value strings after applying per-target relabeling ... This should reduce memory usage when per-target relabeling creates big number of temporary labels with long names and/or values. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825	2020-11-07 16:19:59 +02:00
Aliaksandr Valialkin	f42194d817	lib/promrelabel: make a copy of label with new name for action: labelmap in the same way as Prometheus does ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/812	2020-10-05 16:19:19 +03:00
Aliaksandr Valialkin	c402265e88	all: support %{ENV_VAR} placeholders in yaml configs in all the vm* components ... Such placeholders are substituted by the corresponding environment variable values. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/583	2020-08-13 17:15:25 +03:00
Aliaksandr Valialkin	91b3482894	app/vminsert: add ability to apply relabeling to all the incoming metrics if -relabelConfig command-line arg points to a file with a list of relabel_config entries ... See https://victoriametrics.github.io/#relabeling	2020-07-02 20:39:28 +03:00
Aliaksandr Valialkin	d5dddb0953	all: use %w instead of %s for wrapping errors in fmt.Errorf ... This will simplify examining the returned errors such as httpserver.ErrorWithStatusCode . See https://blog.golang.org/go1.13-errors for details.	2020-06-30 23:05:11 +03:00
Aliaksandr Valialkin	7eb171182b	lib/promrelabel: properly apply ^ and $ anchors to regex value in Prometheus relabeling rules	2020-06-25 17:19:19 +03:00
Aliaksandr Valialkin	2a39ba639d	lib/promrelabel: add support for keep_if_equal and drop_if_equal actions to relabel configs ... These actions may be useful for filtering out unneeded targets and/or metrics if they contain equal label values. For example, the following rule would leave the target only if __meta_kubernetes_annotation_prometheus_io_port equals __meta_kubernetes_pod_container_port_number: - action: keep_if_equal source_labels: [__meta_kubernetes_annotation_prometheus_io_port, __meta_kubernetes_pod_container_port_number]	2020-06-23 17:29:03 +03:00
Aliaksandr Valialkin	5820c0ffb7	lib/promrelabel: allows regex capture groups in target_label like Prometheus does ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/569	2020-06-19 02:21:24 +03:00
Aliaksandr Valialkin	69004a5f67	lib/promscrape: fix tests after the commit 658a8742ac ... The original commit copies `__address__` label to `instance` label when generating per-target labels as Prometheus does. See https://www.robustperception.io/life-of-a-label for details. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/453	2020-05-03 16:56:15 +03:00
DexterZhang	658a8742ac	fix(vmagent): different behavior as how prometheus deal with labels. [Issue#453] (#454)	2020-05-03 16:51:03 +03:00
Aliaksandr Valialkin	3b234d82e5	lib/promscrape: reload only modified scrapers on config changes ... This should improve scrape stability when big number of targets are scraped and these targets are frequently changed. Thanks to @xbsura for the idea and initial implementation attempts at the following pull requests: - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/449 - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/458 - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/459 - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/460	2020-05-03 12:45:40 +03:00
Aliaksandr Valialkin	78863d7066	lib/promscrape: add promrelabel.GetLabelValueByName helper function	2020-04-14 14:12:01 +03:00
Aliaksandr Valialkin	6a75c95194	lib/promscrape: remove labels starting with __meta_ after applying relabel_configs as Prometheus does ... This should reduce CPU load during scraping when target discovery generates big number of `__meta_*` labels (for instance, k8s discovery). See https://www.robustperception.io/life-of-a-label for details.	2020-04-14 12:23:22 +03:00
Aliaksandr Valialkin	986dba5ab3	app/vmagent: do not allow non-supported fields in -remoteWrite.relabelConfig and file_sd_configs ... This should reduce possible confusion like in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/363	2020-03-06 20:19:13 +02:00
Aliaksandr Valialkin	04762344c6	app/vmagent: initial implementation for vmagent	2020-02-23 13:36:03 +02:00