549 Commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	86bce7f5f9	lib/promscrape: add more cases to TestAddRowToTimeseries ... This is a follow-up for 16fdd2af8a	2022-11-09 16:13:56 +02:00
Jeremy PLANCKEEL	16fdd2af8a	test(golang): add test to function addRowToTimeseries (#3282) ... Co-authored-by: jplanckeel-externe <jplanckeel.externe@bedrockstreaming.com>	2022-11-09 15:41:26 +02:00
Aliaksandr Valialkin	7fa5d043f5	lib/promscrape/discovery/consul: add __meta_consul_partition label in the same way as Prometheus does ... See https://github.com/prometheus/prometheus/pull/11482	2022-11-07 15:25:53 +02:00
Aliaksandr Valialkin	76e8888272	lib/promscrape: properly add exported_ prefix to labels, which clash with target labels if honor_labels: true option isn't set. ... The issue was in the `labels := dst[offset:]` line in the beginning of appendExtraLabels() function. The `dst` may be re-allocated when adding extra labels to it. In this case the addition of `exported_` prefix to labels inside `labels` slice become invisible in the returned `dst` labels. While at it, properly handle some corner cases: - Add additional `exported_` prefix to clashing metric labels with already existing `exported_` prefix. - Store scraped metric names in `exported___name__` label if scrape target contains `__name__` label. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3278 Thanks to @jplanckeel for the initial attempt to fix this issue at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/3281	2022-10-28 22:14:26 +03:00
Aliaksandr Valialkin	454baf84d6	lib/promscrape/discovery/kubernetes: do not print an empty kubeconfig_file option in yaml at /config page	2022-10-28 22:14:25 +03:00
Aliaksandr Valialkin	518c340ae3	lib/envtemplate: allow referring env vars from other env vars via %{ENV_VAR} syntax ... This is a follow-up for 02096e06d0	2022-10-26 14:49:33 +03:00
Nikolay	1059c4d84a	lib/promscrape/discovery/kubernetes: correctly wrap error (#3250) ... * lib/promscrape/discovery/kubernetes: correctly wrap error follow-up after 1304824201 * Update docs/CHANGELOG.md Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-10-18 20:37:42 +03:00
Aliaksandr Valialkin	069401a304	all: log error when environment variables referred from -promscrape.config are missing ... This should prevent from using incorrect config files	2022-10-18 10:47:16 +03:00
Aliaksandr Valialkin	50f5eae0e0	lib/promrelabel: remove unconditional sorting of the labels in ParsedConfigs.Apply(), since the sorting isnt needed in many places ... Sort labels explicitly after calling the ParsedConfigs.Apply() when needed. This reduces CPU usage when performing metric-level relabeling, where labels' sorting isn't needed.	2022-10-09 14:51:16 +03:00
Aliaksandr Valialkin	5269b1ad77	lib/promscrape: allow controlling staleness tracking on a per-scrape_config basis ... Add support for no_stale_markers option at scrape_config section. See https://docs.victoriametrics.com/sd_configs.html#scrape_configs and https://docs.victoriametrics.com/vmagent.html#prometheus-staleness-markers	2022-10-07 23:36:14 +03:00
Aliaksandr Valialkin	f9df0cae16	lib/promscrape: allow specifying full target url in __address__ label ... Previously the `__address__` label could contain only `host:port` part of the target url, while the scheme and metrics path were obtained from `__scheme__` and `__metrics_path__` labels. Now it is possible to set the full url in `__address__` label. This makes valid the following scrape config, which is frequently used by novice users: scrape_configs: - job_name: foo static_configs: - targets: - http://host1/metrics1 - https://host2/metrics2	2022-10-07 22:43:04 +03:00
Aliaksandr Valialkin	4998402004	lib/promscrape: add external_labels from global section of -promscrape.config after the relabeling is applied to the scraped metrics ... This aligns with Prometheus behaviour. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3137	2022-10-01 16:13:19 +03:00
Aliaksandr Valialkin	f86070169d	lib/promscrape/discovery/azure: remove unneeded conversion to string	2022-10-01 16:04:37 +03:00
Aliaksandr Valialkin	c54e14cdec	lib/promscrape/discovery/ec2: expose __meta_ec2_region label in the same way as Prometheus 2.39 does ... See https://github.com/prometheus/prometheus/pull/11326	2022-09-30 20:48:32 +03:00
Aliaksandr Valialkin	17289ff481	lib/regexutil: cache MatchString results for unoptimized regexps ... This increases relabeling performance by 3x for unoptimized regexs	2022-09-30 10:41:29 +03:00
Aliaksandr Valialkin	593da3603e	lib/bytesutil: move InternString() from lib/promscrape/discoverytutils to lib/bytesutil ... lib/bytesutil is more appropriate place for InternString() function	2022-09-30 07:44:35 +03:00
Aliaksandr Valialkin	6a32a64073	lib/bytesutil: add FastStringTransformer and use it in the rest of the code where needed	2022-09-28 10:41:00 +03:00
Aliaksandr Valialkin	56ce7ce85b	lib/promscrape: typo fix after 74c00a8762	2022-09-14 15:06:50 +03:00
Aliaksandr Valialkin	74c00a8762	lib/promscrape: read response body into memory in stream parsing mode before parsing it ... This reduces scrape duration for targets returning big responses. The response body was already read into memory in stream parsing mode before this change, so this commit shouldn't increase memory usage.	2022-09-14 13:15:29 +03:00
Aliaksandr Valialkin	ccad651a61	lib/promscrape/discovery/kubernetes: add more context on WatchEvent parse error ... This should improve debugging issues with Kubernetes API server	2022-09-13 19:36:55 +03:00
Bryce Lampe	74f8e12e87	Support "HTTP" and "HTTPS" schemes (#3019) ... * Support "HTTP" and "HTTPS" schemes * Update lib/promscrape/config.go Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2022-08-27 02:22:37 +03:00
Aliaksandr Valialkin	30b8d91727	lib/promscrape/discoveryutils: always store just allocated string to sanitized label names cache ... This is a follow-up for c06e7a142c	2022-08-27 00:28:39 +03:00
Aliaksandr Valialkin	c06e7a142c	lib/promscrape: optimize discoveryutils.SanitizeLabelName() ... Cache sanitized label names and return them next time. This reduces the number of allocations and speeds up the SanitizeLabelName() function for common case when the number of unique label names is smaller than 100k	2022-08-27 00:17:45 +03:00
Aliaksandr Valialkin	1f89278d88	all: subsitute ioutil.ReadAll with io.ReadAll ... ioutil.ReadAll is deprecated since Go1.16 - see https://tip.golang.org/doc/go1.16#ioutil VictoriaMetrics requires at least Go1.18, so it is OK to switch from ioutil.ReadAll to io.ReadAll. This is a follow-up for 02ca2342ab	2022-08-22 00:16:37 +03:00
Aliaksandr Valialkin	9f94c295ab	all: use os.{Read|Write}File instead of ioutil.{Read|Write}File ... The ioutil.{Read|Write}File is deprecated since Go1.16 - see https://tip.golang.org/doc/go1.16#ioutil VictoriaMetrics needs at least Go1.18, so it is safe to remove ioutil usage from source code. This is a follow-up for 02ca2342ab	2022-08-21 23:52:35 +03:00
Aliaksandr Valialkin	7d26414b2e	lib/promscrape: automatically generate additional per-target labels for targets with non-zero series limit ... The following metrics are generated: - scrape_series_limit - scrape_series_current - scrape_series_limit_samples_dropped These metrics simplify alerting on targets, which expose too many time series See https://docs.victoriametrics.com/vmagent.html#automatically-generated-metrics and https://docs.victoriametrics.com/vmagent.html#cardinality-limiter for more details	2022-08-17 13:19:33 +03:00
Aliaksandr Valialkin	bb68ab99fa	lib/promscrape: retry http requests if the server returns 429 status code ... The 429 status code means that the server is overwhelmed with requests. The client can retry the request after some wait time. Implement this strategy for service discovery and scrape requests. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2940	2022-08-16 15:01:08 +03:00
Aliaksandr Valialkin	c3f8481011	lib/promscrape: update links to sd_configs from Prometheus site to https://docs.victoriametrics.com/sd_configs.html	2022-08-15 01:40:20 +03:00
Aliaksandr Valialkin	95d36da358	lib/promscrape/discovery/kubernetes: add __meta_kubernetes_pod_container_image label in the same way as Prometheus 2.38 does ... See https://github.com/prometheus/prometheus/pull/11034	2022-08-15 01:18:23 +03:00
Aliaksandr Valialkin	c4fcd9f1c5	lib/promscrape/discovery/kubernetes: add __meta_kubernetes_service_port_number label to role: service in the same way as Prometheus 2.38 does ... See https://github.com/prometheus/prometheus/pull/11002	2022-08-15 01:06:34 +03:00
Aliaksandr Valialkin	511805d88d	lib/promscrape/discovery/dns: add support for resolving MX records ... See https://github.com/prometheus/prometheus/pull/10099	2022-08-15 00:32:34 +03:00
Aliaksandr Valialkin	46d7792b72	lib/promscrape: follow-up after 2c553d5a2f ... - fix broken tests - cosmetic code cleanup - document the change at https://docs.victoriametrics.com/vmagent.html#multitenancy - document the change at https://docs.victoriametrics.com/CHANGELOG.html	2022-08-08 14:46:26 +03:00
Fury	2c553d5a2f	add support to scrape multi tenant metrics (#2950) ... * add support to scrape multi tenant metrics * add support to scrape multi tenant metrics Co-authored-by: 赵福玉 <zhaofuyu@zhaofuyudeMac-mini.local>	2022-08-08 14:10:18 +03:00
Aliaksandr Valialkin	0ef29ceb14	lib/promscrape/discovery/kubernetes: add missing __meta_kubernetes_ingress_class_name label for role: ingress ... See 7e65ad3e43 and 7e1111ff14	2022-08-05 20:55:00 +03:00
Aliaksandr Valialkin	f2816ef031	lib/promscrape/discovery/ec2: properly handle custom endpoint option in ec2_sd_configs ... This option was ignored since d289ecded1 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287	2022-08-05 18:50:02 +03:00
Aliaksandr Valialkin	3e8890e71b	lib/promscrape/discovery/dockerswarm: properly set __meta_dockerswarm_container_label_* labels instead of __meta_dockerswarm_task_label_* labels ... See https://github.com/prometheus/prometheus/issues/9187	2022-08-05 16:11:28 +03:00
Aliaksandr Valialkin	68de1f4e4a	lib/promscrape/discovery/consul: allow stale responses from Consul service discovery by default ... This aligns with Prometheus behaviour. See `allow_stale` option description at https://prometheus.io/docs/prometheus/latest/configuration/configuration/#consul_sd_config	2022-08-05 14:41:40 +03:00
Aliaksandr Valialkin	02de848c88	lib/promscrape/discovery/yandexcloud: further code cleanup after 83a4abda3f	2022-08-05 10:30:47 +03:00
Aliaksandr Valialkin	83a4abda3f	lib/promscrape/discovery/yandexcloud: follow-up after 6e5ac32fba ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1386	2022-08-04 22:26:43 +03:00
Igor Tiunov	6e5ac32fba	YC service discovery (#2923) ... * YC service discovery https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1386 * Fixed linter suggestions * fixed golint errors	2022-08-04 20:44:16 +03:00
Aliaksandr Valialkin	4c2f9a1a2e	lib/promscrape: set up=0 for partially failed scrape in stream parsing mode ... This behaviour aligns with Prometheus behavior	2022-07-22 13:29:44 +03:00
Aliaksandr Valialkin	0fd86e2364	lib/promscrape: reload all the scrape configs when the global section is changed inside -promscrape.config ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2884	2022-07-18 17:15:07 +03:00
Nikolay	7301aa678c	lib/promscrape: adds azure service discovery (#2743) ... * lib/promscrape: adds azure service discovery Adds azure service discovery mechanism implements authorization with oauth and msi lists virtual machines and virtual machines managed by scaleSet https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1364 * makes linter happy * Apply suggestions from code review Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> * wip Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-13 23:43:18 +03:00
Aliaksandr Valialkin	f97355d9fb	lib/promscrape: properly set Host header when sending requests via http proxy ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2794	2022-07-07 02:27:52 +03:00
Aliaksandr Valialkin	01f55bc66b	lib/promscrape/discovery/kubernetes: properly populate service-level labels for role: endpointslice targets ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2823	2022-07-07 00:32:26 +03:00
Aliaksandr Valialkin	b186b63e07	lib/promscrape/discovery/kubernetes: allow attaching node-level labels to role: endpoints and role: endpointlice targets in the same way as Prometheus does ... See https://github.com/prometheus/prometheus/pull/10759	2022-07-06 23:18:59 +03:00
Aliaksandr Valialkin	e6ba2af7a1	lib/promscrape: fix a test after c66f676f3b	2022-07-06 13:26:35 +03:00
Aliaksandr Valialkin	c66f676f3b	lib/promscrape: push scrape_samples_limit metric to remote storage if sample_limit option is set in scrape_config for this target ... See https://github.com/VictoriaMetrics/operator/issues/497	2022-07-06 12:37:55 +03:00
Aliaksandr Valialkin	855436efd2	lib/promauth: refactor NewConfig in order to improve maintainability ... 1. Split NewConfig into smaller functions 2. Introduce Options struct for simplifying construction of the Config with various options This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2684	2022-07-04 14:31:12 +03:00
Aliaksandr Valialkin	e40b40afe6	Revert "lib/promscrape, vmagent: fix path to files (#2801)" ... This reverts commit 0a8e35835c. Reason for revert: it incorrectly fixes the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2799 See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2799#issuecomment-1171392005	2022-06-30 18:23:56 +03:00
Dmytro Kozlov	0a8e35835c	lib/promscrape, vmagent: fix path to files (#2801) ... vmagent: respect `-pathPrefix` flag for static files and links	2022-06-30 16:22:54 +02:00
ttyv	bdf9f4669a	lib/promscrape: fix vmagent tickerCh reload behaviour (#2786) ... Co-authored-by: Dmitriy <dab@ttyv.ru>	2022-06-30 12:33:01 +02:00
Aliaksandr Valialkin	52eadb729e	lib/promscrape: always send stale markers with the real scrape timestamp ... This guarantees that query won't return data just after the series is disappeared.	2022-06-23 11:34:18 +03:00
Aliaksandr Valialkin	1c4f67c5d2	lib/promauth: add ability to send additional http headers in requests to scrape targets ... This solves https://stackoverflow.com/questions/66032498/prometheus-scrape-metric-with-custom-header	2022-06-22 20:39:43 +03:00
Aliaksandr Valialkin	e6ed92529b	all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package ... This package already has the same name, so there is no need in explicit name	2022-06-21 20:23:32 +03:00
Roman Khavronenko	63b538ecd1	vmagent: update SD duration histogram metric if SD is active (#2677) ... The change updates histogram for registering SD update duration only SD is considered as `active`. SD is active if at least one scraper for this SD has started. This change supposed to reduce metrics cardinality produced by duration histogram which gets updated even if SD isn't configured. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2671 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-06-07 15:46:44 +03:00
Aliaksandr Valialkin	a5814fe16a	lib/promscrape/discovery/kubernetes: use unsupportedFieldError() function instead of errContext string ... This improves code readability and maintainability a bit, since the format string is passed as string literal into fmt.Errorf.	2022-06-07 01:22:07 +03:00
Aliaksandr Valialkin	8608dd093c	all: follow-up after 8edb390e21 ... - Remove unused js bloatware from /targets page. This strips down binary size by more than 100Kb - Add /service-discovery page for API compatibility with Prometheus - Properly load bootstrap.min.css from /prometheus/targets - Serve static contents for /targets page from app/vminsert instead of app/vmselect, because /targets page is served from there	2022-06-07 00:57:09 +03:00
Aliaksandr Valialkin	6f0a0e3072	lib/promscrape/discovery/kubernetes: follow-up after 006b8c7534 ... - make more clear error logs - simplify testing for newKubeConfig by passing only the path to kube_config file instead of SDConfig struct	2022-06-06 14:40:52 +03:00
Aliaksandr Valialkin	0922ed2b7e	lib/promscrape: add -promscrape.cluster.name command-line flag ... This flag is used for proper data de-duplication when the same target is scraped from multiple vmagent clusters. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2679	2022-06-04 00:37:01 +03:00
Dmytro Kozlov	8edb390e21	lib/promscrape: adds service discovery visualization for /targets page(#2675) ... * lib/promscrape: updated template * lib/promscrape: fixed click on unhealthy and all btns * app/vmselect: jquery scripts into static folder Co-authored-by: f41gh7 <nik@victoriametrics.com>	2022-06-03 15:38:45 +02:00
Nikolay	a18914abee	lib/promscrape/discovery/kubernetes: follow-up after 0b5c874911 (#2672)	2022-06-01 20:44:45 +02:00
hadesy	006b8c7534	promscrape/discovery: support kubeconfig (#2533)	2022-06-01 20:34:00 +02:00
Aliaksandr Valialkin	d2567ccdd6	lib/promscrape: use strconv.Atoi instead of strconv.ParseInt for parsing -promscrape.cluster.memberNum ... In this case there is no need in converting int64 to int	2022-06-01 01:42:34 +03:00
Dmytro Kozlov	11f91532c5	issue-2594: use embedded for static files (#2650) ... embed static js and css files from CDN into vmalert, vmagent and vmsingle binaries. Co-authored-by: f41gh7 <nik@victoriametrics.com> https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2594	2022-05-31 01:55:28 +02:00
Dmytro Kozlov	1eb29794e6	removed redundant return (fixed linter) (#2647) ... * removed redundant return * updated lint package version	2022-05-26 16:24:01 +02:00
Aliaksandr Valialkin	796804e4b0	lib/promscrape: add -promscrape.suppressScrapeErrorsDelay command-line flag ... This flag can be used for reducing the amounts of logs when scraping unreliable scrape targets. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2575 The patch is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2576 . Thanks to @jelmd .	2022-05-25 22:59:36 +03:00
Roman Khavronenko	d5eb6afe26	lib/promscrape/discovery/kubernetes: fixes kubernetes service discovery (#2615) ... * lib/promscrape/discovery/kubernetes: properly updates discovered scrape works previously, added or updated scrapeworks may override previuosly discovered. it happens because swosByKey may contain small subset of kubernetes objects with it's labels. It happens for objectsUpdated and objectsAdded maps, which include only changed elements * Properly calculate vm_promscrape_discovery_kubernetes_scrape_works Co-authored-by: f41gh7 <nik@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-05-21 01:01:37 +03:00
Boris Petersen	3df8caca15	Add ability to sign requests for all AWS services (#2604) ... This adds the ability to utilize sigv4 signing for all AWS services not just "aps". When the newly introduced property "service" is not set it will default to "aps". Signed-off-by: Boris Petersen <boris.petersen@idealo.de>	2022-05-18 14:58:31 +02:00
Aliaksandr Valialkin	a0727ab1b1	docs/vmagent.md: typo fix in the description for -promscrape.cluster.replicationFactor command-line flag	2022-05-12 18:50:29 +03:00
Aliaksandr Valialkin	9ea3f0c0d3	lib/awsapi: remove whitelist arg from GetFiltersQueryString(), since it may break new filters in the future ... Let users decide which filters to use. If users start using disallowed filters, then AWS will return an error.	2022-05-09 15:33:22 +03:00
Aliaksandr Valialkin	123aa4c79e	lib/promscrape: properly implement ScrapeConfig.clone() ... Previously ScrapeConfig.clone() was improperly copying promauth.Secret fields - their contents was replaced with `<secret>` value. This led to inability to use passwords and secrets in `-promscrape.config` file. The bug has been introduced in v1.77.0 in the commit 67b10896d2 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2551	2022-05-07 00:05:40 +03:00
Aliaksandr Valialkin	1dc4cc243b	lib/promscrape: rename promscrape_stale_samples_created_total metric to vm_promscrape_stale_samples_created_total, so its name is consistent with the rest of vm_promscrape_ metrics	2022-05-06 15:33:13 +03:00
Aliaksandr Valialkin	d5b55fe22d	lib/promscrape/discovery/ec2: add ability to filter Availability Zones in ec2_sd_config via az_filters section	2022-05-06 12:43:29 +03:00
Aliaksandr Valialkin	97f9c2f667	lib/promscrape/discovery/ec2: properly pass filters to DescribeAvailabilityZones API call ... Previously filters wheren't passed to this call after the commit 0e09fdb8b0 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626	2022-05-05 11:00:23 +03:00
Aliaksandr Valialkin	d285c2fea7	lib/awsapi: pass filtersQueryString arg to GetEC2APIResponse() function, so the caller could decide whether to use the filters during the AWS API query ... The filters shouldn't be passed to DescribeAvailabilityZones API call. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287 Related commits: 0e09fdb8b0 d289ecded1	2022-05-05 10:29:34 +03:00
Nikolay	d289ecded1	{lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite (#2458) ... * {lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite moves aws related code into separate lib from lib/promscrape it allows to write data from vmagent to the AWS managed prometheus (cortex) https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287 * Apply suggestions from code review * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-05-04 20:24:19 +03:00
Nikolay	3575aabeaf	lib/promscrape: adds correct http status codes for redirect (#2530) ... standard http client accepts multiple http status codes as redirect it should fix issue with incorrect redirects https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2482	2022-05-03 13:31:31 +03:00
Dima Lazerka	84683b8569	Fix targetstatus qtpl paths (#2517) ... Ran `make quicktemplate-gen` from the root directory	2022-04-29 10:36:03 +03:00
Aliaksandr Valialkin	6be07e8c25	lib/promscrape/discovery/kubernetes: do not drop pod meta-labels even if the corresponding node objects are missing ... This reflects the logic used in Prometheus. See https://github.com/prometheus/prometheus/pull/10080	2022-04-26 15:26:01 +03:00
Aliaksandr Valialkin	6f79b2b68b	lib/promscrape/discovery/kubernetes: limit the minimum sleep time between updating dependent ScrapeWork objects ... Previously the sleep time could be dropped to nanoseconds, which could result in CPU time waste	2022-04-22 23:14:17 +03:00
Aliaksandr Valialkin	15190fcdae	lib/promscrape/discovery/kubernetes: allow attaching node-level labels and annotations to discovered pod targets in the same way as Prometheus 2.35 does ... See https://github.com/prometheus/prometheus/issues/9510 and https://github.com/prometheus/prometheus/pull/10080	2022-04-22 20:15:41 +03:00
Aliaksandr Valialkin	57a0aa204d	lib/promscrape/discovery/kubernetes: improve the performance of urlWatcher.reloadObjects() on multi-CPU systems ... Parallelize the generation of ScrapeWork objects there. Previously they were generated in a single goroutine.	2022-04-22 13:22:01 +03:00
Aliaksandr Valialkin	67b10896d2	lib/promscrape: prevent from memory leaks on -promscrape.config reload when only a small part of scrape jobs is updated ... This is a follow-up after 26b78ad707	2022-04-22 13:19:43 +03:00
Aliaksandr Valialkin	167d1bea8f	lib/promscrape/discovery/kubernetes: properly update endpoints and endpointslice objects when the related pod or service objects are updated ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240 This is a follow-up for 2341bd48d7	2022-04-21 13:06:22 +03:00
Aliaksandr Valialkin	c75d0095f5	lib/promscrape: remove possible data race when cleaning up internStringsMap	2022-04-20 18:40:53 +03:00
Aliaksandr Valialkin	82e34984dd	lib/promscrape: zero out labels after duplicate removal inside mergeLabels()	2022-04-20 18:35:33 +03:00
Aliaksandr Valialkin	a2de31f8d3	lib/promscrape/discovery/kubernetes: do not pre-allocate memory for ScrapeWork objects ... There is high chance that ScrapeWork objects won't be generated because of relabeling	2022-04-20 16:40:25 +03:00
Aliaksandr Valialkin	2341bd48d7	lib/promscrape: follow-up after 91e290a8ff	2022-04-20 16:11:37 +03:00
Nikolay	91e290a8ff	lib/promscrape: reduce latency for k8s GetLabels (#2454) ... replaces internStringMap with sync.Map - it greatly reduces lock contention concurently reload scrape work for api watcher - each object labels added by dedicated CPU changes can be tested with following script https://gist.github.com/f41gh7/6f8f8d8719786aff1f18a85c23aebf70	2022-04-20 16:09:40 +03:00
Aliaksandr Valialkin	3d0549c982	lib/promscrape: optimize getScrapeWork() function ... Reduce the number of memory allocations in this function. This improves its performance by up to 50%. This should improve service discovery speed when big number of potential targets with big number of meta-labels are generated by service discovery. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2270	2022-04-20 15:37:00 +03:00
Aliaksandr Valialkin	4513893ead	lib/promscrape: use a hash over target labels as a key for dropped targets' map ... This reduces the number of allocations and improves the performance for updating dropped targets' map. This map is exposed at /api/v1/targets as in droppedTargets list.	2022-04-20 15:37:00 +03:00
Dmytro Kozlov	136a44bcfc	lib/promscrape: simply update UI (#2479) ... * lib/promscrape: simply update UI * lib/promscrape: added vm icon	2022-04-20 10:25:04 +02:00
Aliaksandr Valialkin	f6d0e5e74a	all: typo fix: Kuberntes -> Kubernetes	2022-04-20 10:50:49 +03:00
Dmytro Kozlov	a3ee275149	lib/promscrape: Enable filters for endpoint and labels (#2466) ... * lib/promscrape: Enable filters for endpoint and labels * lib/promscrape: cleanup * lib/promscrape: update template * lib/promscrape: move logic filter logic to backend * lib/promscrape: updated placeholder * lib/promscrape: updated placeholder * lib/promscrape: use two different fields for filters, updated form, added error on parsing queries * lib/promscrape: rename functions * lib/promscrape: removed unused values * wip * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-19 18:26:21 +03:00
Nikolay	26b78ad707	lib/promscrape: adds job restart method (#2455) ... * lib/promscrape: adds job restart method it must restart only ScrapeConfig with changed content this change greatly reduce time, that needed for job restart and it should decrease possible data loss when config frequently changed at kubernetes based deployments Apply suggestions from code review Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> * wip Co-authored-by: Roman Khavronenko <roman@victoriametrics.com> Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-16 20:28:46 +03:00
Aliaksandr Valialkin	ebaa1c7ad5	lib/promscrape: follow-up after baa1c24b36	2022-04-16 14:25:54 +03:00
Nikolay	baa1c24b36	lib/promscrape: removes omitempty for ScrapeConfig (#2457) ... This change fixes incorrect marshalling for ScrapeConfig it affects http endpoint and ScrapeConfig checksum. With omitempty, custom Marshaller is not called if field is not a pointer. Previously this issue happened at vmalert	2022-04-16 13:22:11 +03:00
Aliaksandr Valialkin	f3d4671bb6	lib/promscrape: follow-up after 7e79adfb55	2022-04-12 12:36:17 +03:00
Nikolay	7e79adfb55	lib/promscrape: allows to use k8s pod name as clusterMemberNum (#2436) ... * lib/promscrape: allows to use k8s pod name as clusterMemberNum it must improve user expirience and simplify clustering scrapers. it must allow to use vmagent cluster with distroless images https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2359 * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-04-12 12:24:11 +03:00