Aliaksandr Valialkin
ab57b92932
lib/promscrape/discovery/kubernetes: add support for __meta_kubernetes_pod_container_id
...
See https://github.com/prometheus/prometheus/issues/11843
and https://github.com/prometheus/prometheus/pull/11844
2023-01-27 16:34:06 -08:00
Zakhar Bessarab
322d96bfe5
discovery/{consul,nomad}: fix cancelling serviceWatcher in-flight requests ( #3658 )
...
* lib/promscrape/discovery/{consul,nomad}: fix background service update watches not canceling requests on serviceWatcher stop
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
* lib/promscrape/discovery/{consul,nomad}: fix closing serviseWatcher during scrape job restart
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
* wip
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3468
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-17 21:47:11 -08:00
Aliaksandr Valialkin
ec23ab6bc2
lib/promscrape/discovery: missing changes after b4ad3a3b4c
2023-01-11 23:02:45 -08:00
Aliaksandr Valialkin
b4ad3a3b4c
lib/promscrape: follow-up for 8537533beb
...
- Add a comment describing the purpose of the `role` field inside `apiConfig` struct
- Revert changes at lib/promscrape/discovery/dockerswarm/dockerswarm.go ,
since they reduce code readability. E.g. the reader needs to look up the named string constants
in order to get their values.
2023-01-11 22:54:18 -08:00
Zakhar Bessarab
8537533beb
lib/promscrape/discovery/dockerswarm: fix discovery filters being applied to all objects ( #3632 )
...
* lib/promscrape/discovery/dockerswarm: fix discovery filters being applied to all objects
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
* Update docs/CHANGELOG.md
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-11 22:50:34 -08:00
Aliaksandr Valialkin
cbe62f23ba
lib/promscrape/discovery/gce: follow-up for b2ccdaaa2f
...
- Use promutils.Labels.GetLabels() instead of comparing promutils.Labels.Labels to nil.
This make the code more consistent with other places.
- Mention the release where the issue has been introduced at docs/CHANGELOG.md.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3624
2023-01-10 13:51:03 -08:00
Zakhar Bessarab
b2ccdaaa2f
lib/promscrape/discovery/gce: fix crash in case instance does not have any labels set ( #3625 )
...
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-01-10 11:07:11 +01:00
Aliaksandr Valialkin
e5eca54951
lib/promscrape/discovery/nomad: sync nomad_sd_configs fields with the Prometheus implementation
...
See the list of configs supported by Prometheus at f88a0a7d83/discovery/nomad/nomad.go (L76-L84)865ee8d37c/api/api.go (L297)865ee8d37c/api/api.go (L555-L556)865ee8d37c/api/api.go (L294-L296)8aee209c53https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3367
2023-01-09 21:14:48 -08:00
Roman Khavronenko
8aee209c53
lib/promscrape: remove datacenter field from nomad_sd_config ( #3612 )
...
Looks like `datacenter` field isn't part of `/v1/services` API.
See https://developer.hashicorp.com/nomad/api-docs/services#list-services
and https://developer.hashicorp.com/nomad/api-docs/services#read-service
Related issues:
https://github.com/traefik/traefik/issues/9109
https://github.com/prometheus/prometheus/issues/11776
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-01-09 09:07:40 +01:00
Aliaksandr Valialkin
463b957e54
lib/promscrape/discovery/{consul,nomad}: wait until the deleted serviceWatchers are stopped inside updateServices() call
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3468
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3367
2023-01-05 21:52:33 -08:00
Aliaksandr Valialkin
f392913d00
lib/promscrape: follow-up after bced9fb978
...
- Document the bugfix at docs/CHANGELOG.md
- Wait until all the worker goroutines are done in consulWatcher.mustStop()
- Do not log `context canceled` errors when discovering consul serviceNames
- Removed explicit handling of gzipped responses at lib/promscrape/discoveryutils.Client,
since this handling is automatically performed by net/http.Transport.
See DisableCompression option at https://pkg.go.dev/net/http#Transport .
- Remove explicit handling of the proxyURL, since it is automatically handled
by net/http.Transport. See Proxy option at https://pkg.go.dev/net/http#Transport .
- Expliticly set MaxIdleConnsPerHost, since its default value equals to 2.
Such a small value may result in excess tcp connection churn
when more than 2 concurrent requests are processed by lib/promscrape/discoveryutils.Client.
- Do not set explicitly the `Host` request header, since it is automatically set by net/http.Client.
- Backport the bugfix to the recently added nomad_sd_configs - see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3367
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3468
2023-01-05 21:13:06 -08:00
Zakhar Bessarab
bced9fb978
lib/promscrape/discoveryutils: switch to native http client from fasthttp ( #3568 )
2023-01-05 19:34:47 -08:00
Aliaksandr Valialkin
9f348cf8a1
lib/promscrape/discovery/nomad: follow-up after 48f371a46c
...
- Remove undocumented `username` and `password` config options from `nomad_sd_config`.
TODO: probably, remove these options from `consul_sd_config` too?
These options exist there for backwards compatibility purposes.
- Add __meta_nomad_service_alloc_id and __meta_nomad_service_job_id meta-labels
These labels contain AllocID and JobID fields for the discovered Nomad services.
- Various typo fixes.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3367
2023-01-05 18:07:20 -08:00
Karan Sharma
48f371a46c
lib/promscrape: add Prometheus-compatible service discovery for Nomad ( #3549 )
...
Add nomad_sd_config support for service discovery
2023-01-05 23:03:58 +01:00
Zakhar Bessarab
185cdcd813
lib/promscrape/discovery/dockerswarm: fix query encoding of filters ( #3586 )
...
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-01-05 03:34:25 -08:00
Aliaksandr Valialkin
0076422350
lib/promscrape/discovery/azure: typo fix
2022-12-21 21:25:16 -08:00
Aliaksandr Valialkin
0bf3ae9559
lib/promscrape/discovery/consul: expose service tags in individual labels __meta_consul_tag_<tagname>
...
This simplifies copying service tags to target labels with the following relabeling rule:
- action: labelmap
regex: __meta_consul_tag_(.+)
See https://stackoverflow.com/questions/44339461/relabeling-in-prometheus
2022-12-19 13:08:11 -08:00
Aliaksandr Valialkin
2406c0dcfd
docs/CHANGELOG.md: document the bugfix at 05b42601c3
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3247
2022-12-08 18:35:28 -08:00
Zakhar Bessarab
05b42601c3
lib/promscrape/discovery/azure: remove API server from URL returned by azure ( #3403 )
...
* lib/promscrape/discovery/azure: remove API server from URL returned by azure
* lib/promscrape/discovery/azure: validate nextLink contains same URL as apiServer
2022-12-08 18:29:10 -08:00
Aliaksandr Valialkin
f325410c26
lib/promscrape: optimize service discovery speed
...
- Return meta-labels for the discovered targets via promutils.Labels
instead of map[string]string. This improves the speed of generating
meta-labels for discovered targets by up to 5x.
- Remove memory allocations in hot paths during ScrapeWork generation.
The ScrapeWork contains scrape settings for a single discovered target.
This improves the service discovery speed by up to 2x.
2022-11-29 21:26:00 -08:00
Aliaksandr Valialkin
295c84df66
lib/promscrape/discovery: add a benchmark for measuring the performance of creating pod meta-labels
2022-11-29 20:27:48 -08:00
Aliaksandr Valialkin
95f0266558
lib/promscrape/discovery/gce: do not pass filter arg when discovering zones
...
The filter arg isn't supported by zones API in GCE.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3202
2022-11-21 22:32:05 +02:00
Aliaksandr Valialkin
7fa5d043f5
lib/promscrape/discovery/consul: add __meta_consul_partition label in the same way as Prometheus does
...
See https://github.com/prometheus/prometheus/pull/11482
2022-11-07 15:25:53 +02:00
Aliaksandr Valialkin
454baf84d6
lib/promscrape/discovery/kubernetes: do not print an empty kubeconfig_file option in yaml at /config page
2022-10-28 22:14:25 +03:00
Nikolay
1059c4d84a
lib/promscrape/discovery/kubernetes: correctly wrap error ( #3250 )
...
* lib/promscrape/discovery/kubernetes: correctly wrap error
follow-up after 1304824201
2022-10-18 20:37:42 +03:00
Aliaksandr Valialkin
f86070169d
lib/promscrape/discovery/azure: remove unneeded conversion to string
2022-10-01 16:04:37 +03:00
Aliaksandr Valialkin
c54e14cdec
lib/promscrape/discovery/ec2: expose __meta_ec2_region label in the same way as Prometheus 2.39 does
...
See https://github.com/prometheus/prometheus/pull/11326
2022-09-30 20:48:32 +03:00
Aliaksandr Valialkin
ccad651a61
lib/promscrape/discovery/kubernetes: add more context on WatchEvent parse error
...
This should improve debugging issues with Kubernetes API server
2022-09-13 19:36:55 +03:00
Aliaksandr Valialkin
c06e7a142c
lib/promscrape: optimize discoveryutils.SanitizeLabelName()
...
Cache sanitized label names and return them next time.
This reduces the number of allocations and speeds up the SanitizeLabelName()
function for common case when the number of unique label names is smaller than 100k
2022-08-27 00:17:45 +03:00
Aliaksandr Valialkin
1f89278d88
all: subsitute ioutil.ReadAll with io.ReadAll
...
ioutil.ReadAll is deprecated since Go1.16 - see https://tip.golang.org/doc/go1.16#ioutil
VictoriaMetrics requires at least Go1.18, so it is OK to switch from ioutil.ReadAll to io.ReadAll.
This is a follow-up for 02ca2342ab
2022-08-22 00:16:37 +03:00
Aliaksandr Valialkin
9f94c295ab
all: use os.{Read|Write}File instead of ioutil.{Read|Write}File
...
The ioutil.{Read|Write}File is deprecated since Go1.16 -
see https://tip.golang.org/doc/go1.16#ioutil
VictoriaMetrics needs at least Go1.18, so it is safe to remove ioutil usage
from source code.
This is a follow-up for 02ca2342ab
2022-08-21 23:52:35 +03:00
Aliaksandr Valialkin
c3f8481011
lib/promscrape: update links to sd_configs from Prometheus site to https://docs.victoriametrics.com/sd_configs.html
2022-08-15 01:40:20 +03:00
Aliaksandr Valialkin
95d36da358
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_pod_container_image label in the same way as Prometheus 2.38 does
...
See https://github.com/prometheus/prometheus/pull/11034
2022-08-15 01:18:23 +03:00
Aliaksandr Valialkin
c4fcd9f1c5
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_service_port_number label to role: service in the same way as Prometheus 2.38 does
...
See https://github.com/prometheus/prometheus/pull/11002
2022-08-15 01:06:34 +03:00
Aliaksandr Valialkin
511805d88d
lib/promscrape/discovery/dns: add support for resolving MX records
...
See https://github.com/prometheus/prometheus/pull/10099
2022-08-15 00:32:34 +03:00
Aliaksandr Valialkin
0ef29ceb14
lib/promscrape/discovery/kubernetes: add missing __meta_kubernetes_ingress_class_name label for role: ingress
...
See 7e65ad3e437e1111ff14
2022-08-05 20:55:00 +03:00
Aliaksandr Valialkin
f2816ef031
lib/promscrape/discovery/ec2: properly handle custom endpoint option in ec2_sd_configs
...
This option was ignored since d289ecded1https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287
2022-08-05 18:50:02 +03:00
Aliaksandr Valialkin
3e8890e71b
lib/promscrape/discovery/dockerswarm: properly set __meta_dockerswarm_container_label_* labels instead of __meta_dockerswarm_task_label_* labels
...
See https://github.com/prometheus/prometheus/issues/9187
2022-08-05 16:11:28 +03:00
Aliaksandr Valialkin
68de1f4e4a
lib/promscrape/discovery/consul: allow stale responses from Consul service discovery by default
...
This aligns with Prometheus behaviour.
See `allow_stale` option description at https://prometheus.io/docs/prometheus/latest/configuration/configuration/#consul_sd_config
2022-08-05 14:41:40 +03:00
Aliaksandr Valialkin
02de848c88
lib/promscrape/discovery/yandexcloud: further code cleanup after 83a4abda3f
2022-08-05 10:30:47 +03:00
Aliaksandr Valialkin
83a4abda3f
lib/promscrape/discovery/yandexcloud: follow-up after 6e5ac32fba
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1386
2022-08-04 22:26:43 +03:00
Igor Tiunov
6e5ac32fba
YC service discovery ( #2923 )
...
* YC service discovery
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1386
* Fixed linter suggestions
* fixed golint errors
2022-08-04 20:44:16 +03:00
Nikolay
7301aa678c
lib/promscrape: adds azure service discovery ( #2743 )
...
* lib/promscrape: adds azure service discovery
Adds azure service discovery mechanism
implements authorization with oauth and msi
lists virtual machines and virtual machines managed by scaleSet
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1364
* makes linter happy
* Apply suggestions from code review
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
* wip
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-13 23:43:18 +03:00
Aliaksandr Valialkin
01f55bc66b
lib/promscrape/discovery/kubernetes: properly populate service-level labels for role: endpointslice targets
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2823
2022-07-07 00:32:26 +03:00
Aliaksandr Valialkin
b186b63e07
lib/promscrape/discovery/kubernetes: allow attaching node-level labels to role: endpoints and role: endpointlice targets in the same way as Prometheus does
...
See https://github.com/prometheus/prometheus/pull/10759
2022-07-06 23:18:59 +03:00
Aliaksandr Valialkin
855436efd2
lib/promauth: refactor NewConfig in order to improve maintainability
...
1. Split NewConfig into smaller functions
2. Introduce Options struct for simplifying construction of the Config with various options
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2684
2022-07-04 14:31:12 +03:00
Aliaksandr Valialkin
1c4f67c5d2
lib/promauth: add ability to send additional http headers in requests to scrape targets
...
This solves https://stackoverflow.com/questions/66032498/prometheus-scrape-metric-with-custom-header
2022-06-22 20:39:43 +03:00
Aliaksandr Valialkin
a5814fe16a
lib/promscrape/discovery/kubernetes: use unsupportedFieldError() function instead of errContext string
...
This improves code readability and maintainability a bit, since the format string
is passed as string literal into fmt.Errorf.
2022-06-07 01:22:07 +03:00
Aliaksandr Valialkin
6f0a0e3072
lib/promscrape/discovery/kubernetes: follow-up after 006b8c7534
...
- make more clear error logs
- simplify testing for newKubeConfig by passing only the path to kube_config file instead of SDConfig struct
2022-06-06 14:40:52 +03:00
Nikolay
a18914abee
lib/promscrape/discovery/kubernetes: follow-up after 0b5c874911 ( #2672 )
2022-06-01 20:44:45 +02:00
hadesy
006b8c7534
promscrape/discovery: support kubeconfig ( #2533 )
2022-06-01 20:34:00 +02:00
Roman Khavronenko
d5eb6afe26
lib/promscrape/discovery/kubernetes: fixes kubernetes service discovery ( #2615 )
...
* lib/promscrape/discovery/kubernetes: properly updates discovered scrape works
previously, added or updated scrapeworks may override previuosly
discovered.
it happens because swosByKey may contain small subset of kubernetes
objects with it's labels.
It happens for objectsUpdated and objectsAdded maps, which include only changed elements
* Properly calculate vm_promscrape_discovery_kubernetes_scrape_works
Co-authored-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-21 01:01:37 +03:00
Boris Petersen
3df8caca15
Add ability to sign requests for all AWS services ( #2604 )
...
This adds the ability to utilize sigv4 signing for all AWS services not
just "aps". When the newly introduced property "service" is not set it
will default to "aps".
Signed-off-by: Boris Petersen <boris.petersen@idealo.de>
2022-05-18 14:58:31 +02:00
Aliaksandr Valialkin
9ea3f0c0d3
lib/awsapi: remove whitelist arg from GetFiltersQueryString(), since it may break new filters in the future
...
Let users decide which filters to use. If users start using disallowed filters, then AWS will return an error.
2022-05-09 15:33:22 +03:00
Aliaksandr Valialkin
123aa4c79e
lib/promscrape: properly implement ScrapeConfig.clone()
...
Previously ScrapeConfig.clone() was improperly copying promauth.Secret fields -
their contents was replaced with `<secret>` value.
This led to inability to use passwords and secrets in `-promscrape.config` file.
The bug has been introduced in v1.77.0 in the commit 67b10896d2https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2551
2022-05-07 00:05:40 +03:00
Aliaksandr Valialkin
d5b55fe22d
lib/promscrape/discovery/ec2: add ability to filter Availability Zones in ec2_sd_config via az_filters section
2022-05-06 12:43:29 +03:00
Aliaksandr Valialkin
97f9c2f667
lib/promscrape/discovery/ec2: properly pass filters to DescribeAvailabilityZones API call
...
Previously filters wheren't passed to this call after the commit 0e09fdb8b0https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626
2022-05-05 11:00:23 +03:00
Aliaksandr Valialkin
d285c2fea7
lib/awsapi: pass filtersQueryString arg to GetEC2APIResponse() function, so the caller could decide whether to use the filters during the AWS API query
...
The filters shouldn't be passed to DescribeAvailabilityZones API call.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287
Related commits:
0e09fdb8b0d289ecded1
2022-05-05 10:29:34 +03:00
Nikolay
d289ecded1
{lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite ( #2458 )
...
* {lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite
moves aws related code into separate lib from lib/promscrape
it allows to write data from vmagent to the AWS managed prometheus (cortex)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287
* Apply suggestions from code review
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-04 20:24:19 +03:00
Aliaksandr Valialkin
6be07e8c25
lib/promscrape/discovery/kubernetes: do not drop pod meta-labels even if the corresponding node objects are missing
...
This reflects the logic used in Prometheus.
See https://github.com/prometheus/prometheus/pull/10080
2022-04-26 15:26:01 +03:00
Aliaksandr Valialkin
6f79b2b68b
lib/promscrape/discovery/kubernetes: limit the minimum sleep time between updating dependent ScrapeWork objects
...
Previously the sleep time could be dropped to nanoseconds, which could result in CPU time waste
2022-04-22 23:14:17 +03:00
Aliaksandr Valialkin
15190fcdae
lib/promscrape/discovery/kubernetes: allow attaching node-level labels and annotations to discovered pod targets in the same way as Prometheus 2.35 does
...
See https://github.com/prometheus/prometheus/issues/9510
and https://github.com/prometheus/prometheus/pull/10080
2022-04-22 20:15:41 +03:00
Aliaksandr Valialkin
57a0aa204d
lib/promscrape/discovery/kubernetes: improve the performance of urlWatcher.reloadObjects() on multi-CPU systems
...
Parallelize the generation of ScrapeWork objects there. Previously they were generated in a single goroutine.
2022-04-22 13:22:01 +03:00
Aliaksandr Valialkin
167d1bea8f
lib/promscrape/discovery/kubernetes: properly update endpoints and endpointslice objects when the related pod or service objects are updated
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240
This is a follow-up for 2341bd48d7
2022-04-21 13:06:22 +03:00
Aliaksandr Valialkin
a2de31f8d3
lib/promscrape/discovery/kubernetes: do not pre-allocate memory for ScrapeWork objects
...
There is high chance that ScrapeWork objects won't be generated because of relabeling
2022-04-20 16:40:25 +03:00
Aliaksandr Valialkin
2341bd48d7
lib/promscrape: follow-up after 91e290a8ff
2022-04-20 16:11:37 +03:00
Nikolay
91e290a8ff
lib/promscrape: reduce latency for k8s GetLabels ( #2454 )
...
replaces internStringMap with sync.Map - it greatly reduces lock contention
concurently reload scrape work for api watcher - each object labels added by dedicated CPU
changes can be tested with following script https://gist.github.com/f41gh7/6f8f8d8719786aff1f18a85c23aebf70
2022-04-20 16:09:40 +03:00
Aliaksandr Valialkin
f6d0e5e74a
all: typo fix: Kuberntes -> Kubernetes
2022-04-20 10:50:49 +03:00
Roman Khavronenko
69d1893f4c
Consul SD - update services on the watcher's start ( #2202 )
...
* lib/discovery/consul: update services on the watcher's start
Previously, watcher's start was only initing goroutines for discovery
but not waiting for the first iteration to end. It means first Consul
discovery wasn't returning discovered targets until the next iteration.
The change makes the watcher's start blocking until we get first discovery
iteration done and all registries updated.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* vmalert: remove workarounds for consul SD
Now when consul SD lib properly updates services
on the first start, we don't need workarounds in vmalert.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* lib/discovery/consul: update after review
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-21 15:32:45 +02:00
Nikolay
c90c1c4d54
fixes all_tenants query option usage for openstack service discovery ( #2184 )
...
explicit use configuration parametr instead of conditional add
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2182
2022-02-14 13:07:30 +02:00
artifactori
ea153e5f90
Show gce sdconfig zone on vmagent:8429/config ( #2178 )
...
* vmagent: add test for marshalling gce sdconfig with ZoneYAML
* vmagent: implement MarshalYAML for ZoneYAML on gce sdconfig
2022-02-12 00:39:23 +02:00
Aliaksandr Valialkin
3cb72ccc2a
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_endpointslice_{label,annotation}* labels to be consistent with other role values for Kubernetes service discovery
2022-02-11 14:54:47 +02:00
Nikolay
4e7f7f3302
fixes service discovery for kubernetes ( #2173 )
...
* fixes service discovery for kubernetes
now it must take in account all pods that belong to the discovered endpoint and endpointslice
adds simple test for endpoints
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2134
* wip
* docs/CHANGELOG.md: document the change
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-11 13:34:22 +02:00
Aliaksandr Valialkin
80f03177c4
lib/promscrape/discovery/kubernetes: add __meta_kubernetes_node_provider_id label for discovered Kubernetes nodes in the same way as Prometheus does
...
See https://github.com/prometheus/prometheus/pull/9603
2022-01-13 23:16:02 +02:00
Aliaksandr Valialkin
355a63733d
lib/promscrape/discovery/kubernetes: add the ability to limit service discovery to the current namespace
...
See https://github.com/prometheus/prometheus/issues/9782 and https://github.com/prometheus/prometheus/pull/9881
2022-01-13 22:44:35 +02:00
Aliaksandr Valialkin
17eb86a689
lib/promscrape/discovery/dockerswarm: follow up after 68a117a25a
...
- Document the bugfix at docs/CHANGELOG.md
- Set __address__ field after copying commonLabels to the resulting map of discovered labels.
This makes sure that the correct __address__ label is used.
2022-01-11 09:20:10 +02:00
Alexander Shtuchkin
68a117a25a
Fix for #2038 : Make correct __address__ value for dockerswarm promscrape ( #2041 )
2022-01-11 08:59:06 +02:00
Aliaksandr Valialkin
cbfc7b7c92
app/{vminsert,vmagent}: hide passwords and auth tokens by default at /config page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764
2021-11-05 14:41:16 +02:00
Aliaksandr Valialkin
4821adfd95
lib/promscrape: properly show proxy_url option value at /config page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755
2021-10-26 21:23:54 +03:00
Aliaksandr Valialkin
3408a05d12
lib/promscrape/discovery/kubernetes: log a warning if role: endpoints discovers more than 1000 targets per a single endpoint
...
In this case `role: endpointslice` must be used instead.
See the following references:
* https://kubernetes.io/docs/reference/labels-annotations-taints/#endpoints-kubernetes-io-over-capacity
* https://github.com/kubernetes/kubernetes/pull/99975
* https://github.com/prometheus/prometheus/issues/7572#issuecomment-934779398
2021-10-19 13:20:40 +03:00
Aliaksandr Valialkin
f6d33596ff
lib/promscrape/discovery/kubernetes: rename endpointslices.go -> endpointslice.go in order to be consistent with EndpointSlice struct name
...
This is a follow-up for 31b42b30b6
2021-10-15 12:27:12 +03:00
Nikolay
3d17112a7e
changes auth validation for openstack ( #1663 )
...
* changes auth validation for openstack
must fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1655
* Apply suggestions from code review
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-09-29 00:28:49 +03:00
Nikolay
0e09fdb8b0
makes filters optional for ec2 api requests ( #1627 )
...
filters can be applied only for DescribeInstances requests, like prometheus does.
related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626
2021-09-17 18:00:37 +03:00
Aliaksandr Valialkin
fb6ed0ce19
lib/promscrape/discovery/docker: support host networking mode
...
See https://github.com/prometheus/prometheus/issues/9116
2021-09-13 13:30:16 +03:00
Aliaksandr Valialkin
6295861acd
lib/promscrape/discovery/kubernetes: properly use https scheme for wildcard TLS certificates in ingress target discovery
2021-09-13 13:03:42 +03:00
Aliaksandr Valialkin
5c63d69454
lib/promscrape/discovery/kubernetes: return back support role: endpointslices, since it is used by VictoriaMetrics operator
...
This is a follow up commit after 31b42b30b6
2021-08-29 12:37:03 +03:00
Aliaksandr Valialkin
31b42b30b6
lib/promscrape/discovery/kubernetes: rename role: endpointslices to role: endpointslice to be consistent with Prometheus
...
See 2ec6c7dbb8/discovery/kubernetes/kubernetes.go (L99)
2021-08-29 11:23:08 +03:00
Aliaksandr Valialkin
2e001db4de
lib/promscrape/discovery/kubernetes: use v1 API instead of v1beta1 API for role: ingress and role: endpointslices
...
This should fix service discovery for these roles in Kubernetes v1.22 and newer versions.
See https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122
The corresponding change in Prometheus - https://github.com/prometheus/prometheus/pull/9205
2021-08-29 11:16:59 +03:00
Aliaksandr Valialkin
c27ee35c5c
lib/promscrape: expose promscrape_discovery_http_errors_total metric for tracking errors per each http_sd config
2021-08-25 13:05:49 +03:00
Aliaksandr Valialkin
50663ba41f
lib/promscrape/discovery/gce: add __meta_gce_interface_ipv4_<name> labels as in Prometheus 2.29
...
See https://github.com/prometheus/prometheus/pull/8978
2021-08-03 16:11:49 +03:00
Aliaksandr Valialkin
3cad8b4564
lib/promscrape/discovery/ec2: add __meta_ec2_availability_zone_id label as Prometheus 2.29 does
2021-08-03 16:11:49 +03:00
Aliaksandr Valialkin
7088f17494
lib/promscrape/discovery/consul: use case-insensitive comparison for service names
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1424
2021-07-02 14:49:27 +03:00
Aliaksandr Valialkin
9ce211a514
lib/promscrape/discovery/docker: fix golint warning: struct field Id should be ID
2021-06-29 13:12:28 +03:00
Aliaksandr Valialkin
cb5453953f
lib/promscrape: split docker and dockerswarm service discovery code bases, since they have very little in common
...
This is a follow up after c85a5b7fcb
2021-06-25 13:20:20 +03:00
Aliaksandr Valialkin
a69045e440
lib/promscrape: consistently sort service discovery routines
...
This should simplify further maintenance of the code
2021-06-25 12:10:46 +03:00
Lu Jiajing
c85a5b7fcb
Support Docker ServiceDiscovery ( #1402 )
...
* add docker discovery
* add test
* add labels test and add scrape work
* remove TODO
* refactor to merge apiConfig and sdConfig
* apply suggestion
2021-06-25 11:42:47 +03:00
Nikolay
434e33da9b
adds missing MustStop call to do and http sd ( #1404 )
2021-06-25 11:39:18 +03:00
Aliaksandr Valialkin
e8a5bb92b7
lib/promscrape/discovery/consul: properly pass namespace to Consul watcher
...
Follow-up for 58a2989fe7
2021-06-22 17:42:41 +03:00
Aliaksandr Valialkin
ac54f34f9e
lib/promscrape/discovery/http: follow up after e307bbb29a
2021-06-22 13:40:33 +03:00
Aliaksandr Valialkin
755040a171
lib/promscrape/discovery: support generic auth configs in Consul service discovery in the same way as Prometheus 2.28 does
2021-06-22 13:34:02 +03:00
