Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-16 00:41:24 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
1,493 Commits 114 Branches 748 Tags 509 MiB
6721e47ae9
Commit Graph

89 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
e3439a6cd0 lib/promscrape: optimize per-metric hash calculations 
This increases vmagent performance by up to 10% when scraping big number of metrics
 2020-08-10 19:47:50 +03:00
Aliaksandr Valialkin
e3999ac010 lib/promscrape: show real timestamp and real duration for the scape on /targets page 
Previously the scrape duration may be negative when calculated scrape timestamp drifts away from the real scrape timestamp
 2020-08-10 12:40:49 +03:00
Aliaksandr Valialkin
c09c881264 lib/promscrape: make errcheck happy 2020-08-09 13:17:30 +03:00
Aliaksandr Valialkin
2dfb42a8b4 lib/promscrape: export scrape_samples_added per-target metric like Prometheus does 
This metric may be useful for detecting targets with high churn rate for the exported metrics.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/683
 2020-08-09 12:45:30 +03:00
Aliaksandr Valialkin
d4be3efc60 lib/promscrape: add a test for scrape config for blackbox exporter 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/684
 2020-08-09 12:03:51 +03:00
Roman Khavronenko
a02097e657 Extend metric vm_promscrape_targets with status label (#615) 
The change to `vm_promscrape_targets` metric suppose to improve observability
for `vmagent` so it will be possible to track how many targets are up or down
for every specific scrape group:
```
vm_promscrape_targets{type="static_configs", status="down"} 1
vm_promscrape_targets{type="static_configs", status="up"} 2
```
 2020-07-13 21:54:53 +03:00
Aliaksandr Valialkin
87f8c728bf lib/promscrape: send Accept header similar to Prometheus when scraping targets 
This should fix scraping Spring Boot servers, which return incorrect response
unless `Accept: text/plain` request header is set.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/608
 2020-07-08 19:50:06 +03:00
Aliaksandr Valialkin
6f1d926698 lib/promscrape: use HostClient.DoDeadline instead of HostClient.Do in order to guarantee strict deadline across multiple scrape attempts 2020-07-03 21:33:48 +03:00
Aliaksandr Valialkin
ee03b4ccbd lib/promscrape: prevent from too big deadline misses on scrape retries 
The maximum deadline miss duration is reduced to 2x scrape_interval in the worst case.
By default it is limited to scrape_interval configured for the given scrape target.
 2020-07-03 20:42:09 +03:00
Aliaksandr Valialkin
dfa83a4a35 lib/promscrape: check for nil error before checking for the returned status code when scraping targets 2020-07-03 18:37:25 +03:00
Aliaksandr Valialkin
2361ad8ab4 lib/promscrape: add ability to set disable_compression and disable_keepalive options in scrape_config section of the config passed to -promscrape.config 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/580
 2020-07-02 14:19:34 +03:00
Aliaksandr Valialkin
0f754bea49 lib/promscrape: add -promscrape.disableKeepAlive command-line flag for disabling http keep-alive connections when scraping targets 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/580
 2020-07-01 02:20:46 +03:00
Aliaksandr Valialkin
d962568e93 all: use %w instead of %s for wrapping errors in fmt.Errorf 
This will simplify examining the returned errors such as httpserver.ErrorWithStatusCode .
See https://blog.golang.org/go1.13-errors for details.
 2020-06-30 23:33:46 +03:00
Aliaksandr Valialkin
5a43842bd3 lib/promscrape: add missing label sorting for autogenerated metrics 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/592
 2020-06-29 22:39:40 +03:00
Ween
b42cf33c4d Fix Auto metrics relabeled errors (#593) 
* Fix Auto metrics relabeled errors

* Finalize auto-genenated  Labels

* Fix Test Errors

Co-authored-by: xinyulong <xinyulong@kuaishou.com>
 2020-06-29 22:39:39 +03:00
Aliaksandr Valialkin
de7e585ac8 lib/promscrape: preserve the previously discovered targets on discovery errors per each job_name 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/582
 2020-06-23 15:42:46 +03:00
Aliaksandr Valialkin
a80e852aab lib/promscrape: retry performing the request to the server for up to 3 times before giving up when it closes keep-alive connections 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/580
 2020-06-23 12:34:12 +03:00
Aliaksandr Valialkin
50aa34bcbe lib/promscrape/discovery/consul: reduce load on Consul when discovering big number of targets by using background caching 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574
 2020-06-20 18:20:07 +03:00
Aliaksandr Valialkin
62e1908986 lib/promscrape: reduce default value for -promscrape.discovery.concurrency from 500 to 100 
This should reduce load on Kubernetes API server and Consul when big number of targets are discovered

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574
 2020-06-20 17:53:48 +03:00
Aliaksandr Valialkin
1f2826bae2 lib/promscrape/discovery/ec2: expose __meta_ec2_ami like the next Prometheus release will do 
See b5d61fb66c for details
 2020-06-20 17:45:30 +03:00
Aliaksandr Valialkin
2e5b6220a4 lib/promrelabel: allows regex capture groups in target_label like Prometheus does 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/569
 2020-06-19 02:20:58 +03:00
Aliaksandr Valialkin
b747362936 lib/promscrape: mention about -promscrape.maxScrapeSize in the error message when target returns too big response 2020-05-24 14:41:24 +03:00
Aliaksandr Valialkin
b59e089ac7 app/vmagent: add -dryRun option for checking all the configs mentioned in command-line flags without running vmagent 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/362
 2020-05-21 15:23:18 +03:00
Aliaksandr Valialkin
482bae8466 lib/promscrape: add -promscrape.config.dryRun flag for checking -promscrape.config for errors or unsupported options 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/508
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/362
 2020-05-21 14:54:32 +03:00
Aliaksandr Valialkin
73ec5cf460 lib/promscrape: add -promscrape.discovery.concurrency and -promscrape.discovery.concurrentWaitTime flags for tuning the number of concurrent requests to autodiscovery API servers at Consul or Kubernetes 2020-05-19 17:35:59 +03:00
Aliaksandr Valialkin
3845420a8f lib: extract common code for returning fast unix timestamp into lib/fasttime 2020-05-14 23:06:50 +03:00
Aliaksandr Valialkin
d54a93fc81 app/vmagent: fix scraping mTLS targets, which has been broken in v1.35.1 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/470
 2020-05-12 17:23:43 +03:00
Aliaksandr Valialkin
405cf44aed app/vmagent,lib/promscrape: do not set HostClient.DialDualStack, since it isnt used if HostClient.Dial is set 2020-05-12 15:24:53 +03:00
Aliaksandr Valialkin
9f39e618ed lib/promscrape/discovery/gce: discover per-zone instances for gce_sd_config in parallel. This should reduce discovery latency 2020-05-06 15:00:23 +03:00
Aliaksandr Valialkin
8ab5e47b5c lib/promscrape: add Prometheus-compatible DNS-based service discovery aka dns_sd_configs 2020-05-06 00:02:41 +03:00
Aliaksandr Valialkin
42d563934b lib/promscrape: properly connect to TCP6 addresses if -enableTCP6 is set 2020-05-06 00:02:40 +03:00
Aliaksandr Valialkin
1c8e97c8a0 lib/procutil: add NewSighupChan function, which returns a channel, which is triggered on every SIGHUP 2020-05-05 10:56:15 +03:00
Aliaksandr Valialkin
054457d1f4 lib/promscrape: allow explicitly setting empty token via token: "" in consul_sd_config 2020-05-05 07:49:54 +03:00
Aliaksandr Valialkin
89aa6dbf56 lib/promscrape: add Prometheus-compatible service discovery for Consul aka consul_sd_configs 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/330
 2020-05-04 20:53:06 +03:00
Aliaksandr Valialkin
ed91fe1d9b lib/promscrape: move common code for discovery api config map handling into discoveryutils 2020-05-04 20:52:58 +03:00
Aliaksandr Valialkin
c50fd219dc lib/promscrape/discovery/kubernetes/: unify apiConfig creation 2020-05-04 20:52:53 +03:00
Aliaksandr Valialkin
a5880f17af lib/promscrape: remove debug line left after the commit e4aac6ea40 2020-05-03 17:16:19 +03:00
Aliaksandr Valialkin
1f0e8fdc0d lib/promscrape: fix tests after the commit 658a8742ac 
The original commit copies `__address__` label to `instance` label when generating per-target labels as Prometheus does.

See https://www.robustperception.io/life-of-a-label for details.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/453
 2020-05-03 16:59:29 +03:00
DexterZhang
317688f144 fix(vmagent): different behavior as how prometheus deal with labels. [Issue#453] (#454) 2020-05-03 16:59:28 +03:00
Aliaksandr Valialkin
ab1e6a76bb lib/promscrape: make consistent scrape time offsets across reloads for the same ScrapeURL and Labels 
This should make consistent intervals between data points for scrape targets across reloads.
Previously these intervals were random.
 2020-05-03 14:31:22 +03:00
Aliaksandr Valialkin
f25416984b lib/promscrape: fix TestGetFileSDScrapeWorkSuccess after 3b234d82e5 2020-05-03 14:31:20 +03:00
Aliaksandr Valialkin
f422203e10 lib/promscrape: reload only modified scrapers on config changes 
This should improve scrape stability when big number of targets are scraped and these targets are frequently changed.

Thanks to @xbsura for the idea and initial implementation attempts at the following pull requests:

- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/449
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/458
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/459
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/460
 2020-05-03 12:47:16 +03:00
Aliaksandr Valialkin
de5f923476 lib/promscrape: set 30 seconds timeout for discovery api requests 
Previously such requests could hang for long time. This could make debugging harder.
 2020-04-29 17:29:03 +03:00
Aliaksandr Valialkin
b6d88bac04 vendor: use github.com/VictoriaMetrics/fasthttp instead of github.com/fasthttp/fasthttp 
The upstream fasthttp may contain issues like 996610f021 ,
plus a code that isn't used by VictoriaMetrics. So let's use a private copy under our control instead.
 2020-04-29 16:43:09 +03:00
Aliaksandr Valialkin
53740d0026 lib/promscrape: handle connection reset when targets responds with http redirect 2020-04-28 02:14:32 +03:00
肖贝贝
3e6f29f462 fix: vmagent not follow 301/302 redirect bug (#445) 
Co-authored-by: xiaobeibei <xiaobeibei@bigo.sg>
 2020-04-28 02:14:31 +03:00
Aliaksandr Valialkin
424068f804 lib/promscrape: handle connection reset when targets responds with http redirect 2020-04-28 02:14:26 +03:00
肖贝贝
7d045bf2ca fix: vmagent not follow 301/302 redirect bug (#445) 
Co-authored-by: xiaobeibei <xiaobeibei@bigo.sg>
 2020-04-28 02:14:25 +03:00
Aliaksandr Valialkin
a603a15757 lib/promscrape/discovery/gce: make golangci-lint happy 2020-04-27 19:29:42 +03:00
Aliaksandr Valialkin
86a1d9cb0c lib/promscrape: add initial support for Prometheus-compatible service discovery for Amazon EC2 aka ec2_sd_configs 2020-04-27 19:29:22 +03:00