Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-16 00:41:24 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
7,534 Commits 114 Branches 748 Tags 509 MiB
d52fd73f18
Commit Graph

44 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
15dda54e79
lib/promscrape/discovery/kubernetes: propagate possible errors at newAPIWatcher() to the caller 
This allows substituting FATAL panics with recoverable runtime errors such as missing or invalid TLS CA file
and/or missing/invalid /var/run/secrets/kubernetes.io/serviceaccount/namespace file.
Now these errors are logged instead of PANIC'ing, so they can be fixed by updating the corresponding files
without the need to restart vmagent.

This is a follow-up for 90427abc65
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5243
 2023-10-27 20:27:58 +02:00
Hui Wang
a37125d043
lib/promscrape/discovery/kubernetes: avoid possible panic if given caFile under kubernetes.SDConfig.HTTPClientConfig is not exist (#5243) 
follow up d5a599badc
 2023-10-27 20:27:58 +02:00
Aliaksandr Valialkin
ecc11dc32d
lib/promauth: refactor NewConfig in order to improve maintainability 
1. Split NewConfig into smaller functions
2. Introduce Options struct for simplifying construction of the Config with various options

This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2684
 2022-07-04 14:31:43 +03:00
Aliaksandr Valialkin
3ae6300497
lib/promauth: add ability to send additional http headers in requests to scrape targets 
This solves https://stackoverflow.com/questions/66032498/prometheus-scrape-metric-with-custom-header
 2022-06-22 20:40:50 +03:00
Aliaksandr Valialkin
3dbb19d624
lib/promscrape/discovery/kubernetes: follow-up after 006b8c7534 
- make more clear error logs
- simplify testing for newKubeConfig by passing only the path to kube_config file instead of SDConfig struct
 2022-06-06 14:41:28 +03:00
Nikolay
72e43ef2fe
lib/promscrape/discovery/kubernetes: follow-up after 0b5c874911 (#2672) 2022-06-04 01:11:23 +03:00
hadesy
28d4624f60
promscrape/discovery: support kubeconfig (#2533) 2022-06-04 01:11:23 +03:00
Aliaksandr Valialkin
146c14d879 lib/promscrape/discovery/kubernetes: return back support role: endpointslices, since it is used by VictoriaMetrics operator 
This is a follow up commit after 31b42b30b6
 2021-08-29 12:37:36 +03:00
Aliaksandr Valialkin
ca61d7c82b lib/promscrape/discovery/kubernetes: rename role: endpointslices to role: endpointslice to be consistent with Prometheus 
See 2ec6c7dbb8/discovery/kubernetes/kubernetes.go (L99)
 2021-08-29 11:23:59 +03:00
Nikolay
2780d6dbcd basic OAuth2 support for remoteWrite and scrape targets (#1316) 
* adds OAuth2 support for remoteWrite and scrapping

* adds tests
changes init
 2021-05-22 18:02:01 +03:00
Aliaksandr Valialkin
edd66b7e82 lib/promscrape/discovery/kubernetes: remove superflouos mustStart and mustStop functions 2021-04-05 22:43:49 +03:00
Aliaksandr Valialkin
7eca60694e lib/promscrape/discovery/kubernetes: reduce CPU time spent on registering big number of Kubernetes objects shared among big number of scrape jobs 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1182
 2021-04-05 22:05:02 +03:00
Aliaksandr Valialkin
87700f1259 lib/promscrape: add support for authorization config in -promscrape.config as Prometheus 2.26 does 
See https://github.com/prometheus/prometheus/pull/8512
 2021-04-02 21:20:37 +03:00
Aliaksandr Valialkin
8fc29ffc67 lib/promscrape/discovery/kubernetes: use a single watcher per apiURL 
Previously multiple scrape jobs could create multiple watchers for the same apiURL. Now only a single watcher is used.
This should reduce load on Kubernetes API server when many scrape job configs use Kubernetes service discovery.
 2021-03-11 17:04:14 +02:00
Aliaksandr Valialkin
92ddb8f197 lib/promscrape/discovery/kubernetes: move apiWatcher code to a separate file 2021-03-05 17:32:32 +02:00
Nikolay
7d92ef3acd Fix ingress discovery api (#1110) 2021-03-03 10:45:50 +02:00
Aliaksandr Valialkin
25f453ce1a lib/promscrape/discovery/kubernetes: properly check for nil pointer inside interface 
See https://mangatmodi.medium.com/go-check-nil-interface-the-right-way-d142776edef1

This fixes a panic when the ScrapeWork is filtered out in swcFunc.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1108
 2021-03-03 10:42:54 +02:00
Aliaksandr Valialkin
ac5c47a9f5 lib/promscrape/discovery: properly track vm_promscrape_discovery_kubernetes_objects_removed_total metric 2021-03-02 18:33:29 +02:00
Aliaksandr Valialkin
f9c1fe3852 lib/promscrape/discovery/kubernetes: cache ScrapeWork objects as soon as the corresponding k8s objects are changed 
This should reduce CPU usage and memory usage when Kubernetes contains tens of thousands of objects
 2021-03-02 16:44:19 +02:00
Aliaksandr Valialkin
c3bf72992f lib/promscrape: explicitly stop and cleanup service discovery routines when new config is read from -promscrape.config 
This should reduce memory usage when `-promscrape.config` file frequently changes
 2021-03-01 14:15:16 +02:00
Aliaksandr Valialkin
3e44d9947e lib/promscrape/discovery/kubernetes: properly account the number of objects when watcher is stopped 
A follow-up for b21b110b7a
 2021-02-28 17:06:49 +02:00
Aliaksandr Valialkin
0ef7a94056 lib/promscrape/discovery/kubernetes: add vm_promscrape_discovery_kubernetes_* metrics for monitoring internal state of k8s service discovery 2021-02-28 16:58:45 +02:00
Aliaksandr Valialkin
f52bdbe2a3 lib/promscrape/discovery/kubernetes: remove resourceVersionMatch=NotOlderThan query arg when watching for k8s object changes, since it cannot be used when watch=1 query arg is passed 2021-02-28 16:08:43 +02:00
Aliaksandr Valialkin
5c9e657808 lib/promscrape/discovery/kubernetes: fix deadlock in startWatcherForURL 
reloadObjects must be called without holding aw.mu lock
 2021-02-28 15:25:33 +02:00
Aliaksandr Valialkin
e77f2f8630 lib/promscrape/discovery/kubernetes: typo fix after 241ffd1f3b 2021-02-28 15:15:27 +02:00
Aliaksandr Valialkin
82441537ff lib/promscrape/discovery/kubernetes: pre-populate labelsByKey in reloadObject() 2021-02-28 15:09:43 +02:00
Aliaksandr Valialkin
6a21ef87b7 lib/promscrape: add missing startWatchersForRole() call at the beginning of apiWatcher.getLabelsForRole 2021-02-28 14:00:00 +02:00
Aliaksandr Valialkin
6d0e7fb8b0 lib/promscrape/discovery/kubernetes: reload k8s resources on every error 
This is needed for obtaining fresh resourceVersion
 2021-02-27 01:46:59 +02:00
Aliaksandr Valialkin
fa3ce450fb lib/promscrape: cache ScrapeWork 
This should reduce the time needed for updating big number of scrape targets.
 2021-02-26 21:43:41 +02:00
Aliaksandr Valialkin
efcdf613c2 lib/promscrape/discovery/kubernetes: cache target labels 
This should reduce CPU usage on repeated SDConfig.GetLabels() calls.
 2021-02-26 20:24:29 +02:00
Aliaksandr Valialkin
dc8c045378 lib/promscrape: cleanup after 9b2246c29b 
Main points:

* Revert changes outside lib/promscrape/discovery/kuberntes . These changes can be applied later in a separate commit
* Minimize changes in lib/promscrape/discovery/kubernetes compared to a93e644001
* Corner case fixes.
 2021-02-26 19:09:12 +02:00
Nikolay
cf9262b01f vmagent kubernetes watch stream discovery. (#1082) 
* started work on sd for k8s

* continue work on watch sd

* fixes

* continue work

* continue work on sd k8s

* disable gzip

* fixes typos

* log errror

* minor fix

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2021-02-26 19:09:12 +02:00
Aliaksandr Valialkin
367fc17933 lib/promscrape: code prettifying for 8dd03ecf19 2020-12-24 10:57:20 +02:00
Nikolay
b00f7816e2 adds proxy_url support, (#980) 
* adds proxy_url support,
adds proxy_url to the dockerswarm, eureka, kubernetes and consul service discovery,
adds proxy_url to the scrape_config for targets scrapping,
http based proxy is supported atm,
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/503

* fixes imports
 2020-12-24 10:57:19 +02:00
Aliaksandr Valialkin
d962568e93 all: use %w instead of %s for wrapping errors in fmt.Errorf 
This will simplify examining the returned errors such as httpserver.ErrorWithStatusCode .
See https://blog.golang.org/go1.13-errors for details.
 2020-06-30 23:33:46 +03:00
Aliaksandr Valialkin
d54a93fc81 app/vmagent: fix scraping mTLS targets, which has been broken in v1.35.1 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/470
 2020-05-12 17:23:43 +03:00
Aliaksandr Valialkin
89aa6dbf56 lib/promscrape: add Prometheus-compatible service discovery for Consul aka consul_sd_configs 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/330
 2020-05-04 20:53:06 +03:00
Aliaksandr Valialkin
ed91fe1d9b lib/promscrape: move common code for discovery api config map handling into discoveryutils 2020-05-04 20:52:58 +03:00
Aliaksandr Valialkin
c50fd219dc lib/promscrape/discovery/kubernetes/: unify apiConfig creation 2020-05-04 20:52:53 +03:00
Aliaksandr Valialkin
b6d88bac04 vendor: use github.com/VictoriaMetrics/fasthttp instead of github.com/fasthttp/fasthttp 
The upstream fasthttp may contain issues like 996610f021 ,
plus a code that isn't used by VictoriaMetrics. So let's use a private copy under our control instead.
 2020-04-29 16:43:09 +03:00
Aliaksandr Valialkin
e220f3eeb6 lib/promscrape: move KubernetesSDConfig to lib/promscrape/discovery/kubernetes 2020-04-23 11:34:30 +03:00
Aliaksandr Valialkin
81481abaa9 lib/promscrape/discovery/kubernetes: reuse a client for empty api_server inside different jobs 2020-04-20 17:07:37 +03:00
Aliaksandr Valialkin
391fb0903e lib/promscrape/discovery/kubernetes: remove only unused client for API server during cleaning 2020-04-14 14:19:26 +03:00
Aliaksandr Valialkin
7c4fb038e3 lib/promscrape: add initial support for kubernetes_sd_config 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/334
 2020-04-13 21:03:53 +03:00