Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-16 00:41:24 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
7,084 Commits 114 Branches 748 Tags 509 MiB
36a1fdca6c
Commit Graph

131 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
ad445a06cd
lib/promscrape: properly show proxy_url option value at /config page 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755
 2021-10-26 21:24:22 +03:00
Aliaksandr Valialkin
b08f51f5d3
lib/promscrape: do not populate response body to memory in stream parsing mode if -promscrape.noStaleMarkers is set 
The response body isn't used if -promscrape.noStaleMarkers is set after the commit 2876137c92 ,
so there is no sense in pupulating it in memory. This should reduce memory usage when scraping big responses.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1728#issuecomment-949630694
 2021-10-22 16:49:21 +03:00
Aliaksandr Valialkin
a8bcc3c276
lib/promscrape: reduce memory usage if -promscrape.noStaleMarkers command-line flag is passed 
Do not store in memory the response from the last scrape per each target if -promscrape.noStaleMarkers option is enabled.
This should reduce memory usage when the scraped targets return large responses.
 2021-10-22 13:22:08 +03:00
Aliaksandr Valialkin
0a9be5ef9d
lib/promscrape: expose promscrape_series_limit_max_series and promscrape_series_limit_current_series metrics per each scrape target with the enabled unique series limiter 2021-10-16 19:14:13 +03:00
Aliaksandr Valialkin
0f4fda1bda
lib/promscrape: store the last scraped response in compressed form if its size exceeds -promscrape.minResponseSizeForStreamParse 
This should reduce memory usage when scraping targets with big response bodies.
 2021-10-16 13:00:11 +03:00
Aliaksandr Valialkin
0452a8d4e8
lib/promscrape: store the full response in stream parsing mode in scrapeWork.lastScrape byte slice 
This allows sending staleness marks and properly calculate scrape_series_added metric in stream parsing mode
at the cost of the increased memory usage, since now the potentially big response is kept
in the lastScrape byte slice per each scrapeWork.

In practice the memory usage increase shouldn't be big, since the response size
is usually much smaller than the parsed metrics from this response after the relabeling,
which usually adds a big pile of target-specific labels per each metric.
 2021-10-15 15:26:24 +03:00
Aliaksandr Valialkin
25421fa2ae
lib/promscrape: add -promscrape.minResponseSizeForStreamParse command-line option for automatic switching to stream parsing mode when scraping targets with big responses 
This should reduce memory usage when vmagent scrapes targets with non-uniform response sizes.
This is common case in Kubernetes monitoring.
 2021-10-14 12:30:55 +03:00
Ziqi Zhao
1db3aeab36
fix some typos (#1678) 
Co-authored-by: 柘远 <zzq237937@alibaba-inc.com>
 2021-10-06 14:43:56 +03:00
Aliaksandr Valialkin
d90834da70 lib/promscrape: generate scrape_timeout_seconds metric per each scrape target in the same way as Prometheus 2.30 does 
See https://github.com/prometheus/prometheus/pull/9247
 2021-09-12 15:21:26 +03:00
Aliaksandr Valialkin
6c97388dde lib/promscrape: add ability to configure scrape_timeout and scrape_interval via relabeling 
See https://github.com/prometheus/prometheus/pull/8911
 2021-09-12 13:35:20 +03:00
Aliaksandr Valialkin
09670479cd lib/promscrape: reduce CPU usage for common case when calculating scrape_series_added metric 
Also reduce CPU usage when applying `series_limit` to scrape targets with constant set of metrics.

The main idea is to perform the calculations on scrape_series_added and series_limit
only if the set of metrics exposed by the target has been changed.
Scrape targets rarely change the set of exposed metrics,
so this optimization should reduce CPU usage in general case.
 2021-09-12 12:53:45 +03:00
Aliaksandr Valialkin
c339642858 lib/promscrape: add the actual job name to the labels of promscrape_series_limit_rows_dropped_total metric 2021-09-11 11:03:38 +03:00
Aliaksandr Valialkin
5aaaa686a4 lib/promscrape: send stale markers for disappeared metrics like Prometheus does 2021-09-11 11:02:56 +03:00
Aliaksandr Valialkin
b64866e64c lib/promscrape: add scrape_ prefix to job and target labels exported by promscrape_series_limit_rows_dropped_total metric 
This is needed in order to prevent from possible clash with the corresponding (job, target) labels for the job, which scrapes this metric.
 2021-09-09 17:31:04 +03:00
Aliaksandr Valialkin
c4df601f43 lib/promscrape: add the ability to limit the number of unique series per each scrape target 
The number of series per target can be limited with the following options:

* Global limit with `-promscrape.maxSeriesPerTarget` command-line option.
* Per-target limit with `max_series: N` option in `scrape_config` section.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1561
 2021-09-01 16:08:12 +03:00
Aliaksandr Valialkin
67bc407747 lib/promscrape: reduce memory and CPU usage when Prometheus staleness tracking is enabled for metrics from deleted / disappeared scrape targets 
Store the scraped response body instead of storing the parsed and relabeld metrics.
This should reduce memory usage, since the response body takes less memory than the parsed and relabeled metrics.
This is especially true for Kubernetes service discovery, which adds many long labels for all the scraped metrics.

This should also reduce CPU usage, since the marshaling of the parsed
and relabeld metrics has been substituted by response body copying.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
 2021-08-21 21:24:07 +03:00
Aliaksandr Valialkin
c3b24882a7 lib/promscrape: use scrapeTimestamp when storing stale markers for failed scrape 
This will make timestamps for stale markers more consistent for timestamps for other samples
 2021-08-19 14:19:54 +03:00
Aliaksandr Valialkin
8ee575dee9 lib/promscrape: send stale markers for the previously scraped metrics on failed scrapes like Prometheus does 2021-08-18 22:00:46 +03:00
Aliaksandr Valialkin
5d92fafc40 app/vmselect: add -search.noStaleMarkers command-line flag for disabling stale markers handling in queries 
This option allows reducing CPU usage a bit when VictoriaMetrics is used
for collecting and processing non-Prometheus data. For example, InfluxDB line protocol, Graphite, OpenTSDB, CSV, etc.
 2021-08-18 13:58:06 +03:00
Aliaksandr Valialkin
f21fad53b4 lib/promscrape: add ability to disable sending Prometheus staleness markers with -promscrape.disableStaleMarkers command-line flag 
This option can be useful when vmagent consumes too much additional memory
for staleness markers functionality and when staleness markers aren't needed.
 2021-08-18 13:58:05 +03:00
Aliaksandr Valialkin
c1f81f08d4 all: add support for Prometheus staleness markers 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845
 2021-08-13 12:13:15 +03:00
Aliaksandr Valialkin
b877538622 app/vmagent: follow-up after fe445f753b 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1491
 2021-08-05 09:51:00 +03:00
Omar Ghader
fe445f753b
feature: Add multitenant for vmagent (#1505) 
* feature: Add multitenant for vmagent

* Minor fix

* Fix rcs index out of range

* Minor fix

* Fix multi Init

* Fix multi Init

* Fix multi Init

* Add default multi

* Adjust naming

* Add TenantInserted metrics

* Add TenantInserted metrics

* fix: remove unused metrics for vmagent

* fix: remove unused metrics for vmagent

Co-authored-by: mghader <marc.ghader@ubisoft.com>
Co-authored-by: Sebastian YEPES <syepes@gmail.com>
 2021-08-05 09:44:29 +03:00
Aliaksandr Valialkin
5f91a701fa lib/promscrape: show the number of samples collected during the last scrape at /targets and /api/v1/targets pages 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1377
 2021-06-14 14:04:35 +03:00
Aliaksandr Valialkin
269e35d676 app/{vmagent,vminsert}: follow-up after 2fe045e2a4 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1343
 2021-06-04 20:33:22 +03:00
jelmd
d8b46908db new feature: debug relabeling (#1344) 
* new feature: relabel logging

Use scrape_configs[x].relabel_debug = true to log metric names inkl.
labels before and after relabeling. After relabeling related metrics
get dropped, i.e. not submitted to servers.

* vminsert wants relabel logging, too.
 2021-06-04 20:33:21 +03:00
faceair
b801b299f0 lib/promscrape: apply body size & sample limit to stream parse (#1331) 
* lib/promscrape: apply body size limit to stream parse

Signed-off-by: faceair <git@faceair.me>

* lib/promscrape: apply sample limit to stream parse

Signed-off-by: faceair <git@faceair.me>
 2021-05-27 15:04:11 +03:00
Aliaksandr Valialkin
2825a1e86d lib/promscrape: add follow_redirect option to scrape_configs section like Prometheus does 
See https://github.com/prometheus/prometheus/pull/8546
 2021-04-02 21:20:37 +03:00
Aliaksandr Valialkin
7f52aae20c lib/promscrape: an attempt to reduce memory usage when vmagent scrapes targets with varying number of metrics 
Do not cache too big byte buffers and too big writeRequestCtx objects,
since it is cheaper to re-create them instead of wasting RAM for their caching.

This reverts 7f6f350ee1
 2021-03-15 11:49:29 +02:00
Aliaksandr Valialkin
33cd6c26d3 lib/promscrape: return back the logic for flushing big buffers to storage from the commit 3fd8653b40 
This should reduce memory usage when vmagent scrapes targets with big number of metrics and `-promscrape.streamParse` isn't enabled
 2021-03-14 22:25:37 +02:00
Aliaksandr Valialkin
60e0280a94 lib/promscrape: add ability to configure proxy options via proxy_tls_config, proxy_basic_auth, proxy_bearer_token and proxy_bearer_token_file 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1116
 2021-03-12 03:36:11 +02:00
Aliaksandr Valialkin
0554430d7e lib/promscrape: apply sample_limit after metric relabeling is applied as Prometheus does 
See the description for `sample_limit` option from Prometheus docs:

Per-scrape limit on number of scraped samples that will be accepted.
If more than this number of samples are present after metric relabeling
the entire scrape will be treated as failed. 0 means no limit.

https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config
 2021-03-09 15:52:41 +02:00
Aliaksandr Valialkin
502fab797a lib/promscrape: add scrape_offset option to scrape_config 
This option can be used for specifying the particular offset per each scrape interval for target scraping
 2021-03-08 11:59:32 +02:00
Aliaksandr Valialkin
9a2bf65134 lib/promscrape: add ability to spread scrape targets among multiple vmagent instances 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1084
 2021-02-28 18:40:42 +02:00
Aliaksandr Valialkin
197ecca426 lib/promrelabel: add more optimizations for relabeling for common cases 2021-02-22 16:36:54 +02:00
Aliaksandr Valialkin
f09b46b2b1 lib/promscrape: typo fix after the commit f26162ec99 2021-02-19 00:34:18 +02:00
Aliaksandr Valialkin
502d0e2524 lib/promscrape: add scrape_align_interval config option into scrape config 
This option allows aligning scrapes to a particular intervals.
 2021-02-18 23:53:04 +02:00
Aliaksandr Valialkin
18c2075159 lib/promscrape: remove vm_promscrape_scrapes_failed_per_url_total and vm_promscrape_scrapes_skipped_by_sample_limit_per_url_total metrics 
These metrics may result in big number of time series when vmagent scrapes thousands of targets and these targets constantly changes.

* It is better using `up == 0` query for determining failing targets.
* It is better using the following query for determining targets with exceeded limit on the number of metrics:

  scrape_samples_scraped > 0 if up == 0
 2021-02-12 05:23:27 +02:00
Aliaksandr Valialkin
ffec5131ae lib/promscrape: export vm_promscrape_scrapes_failed_per_url_total and vm_promscrape_scrapes_skipped_by_sample_limit_per_url_total metrics 
These metrics could be useful for determining imporperly working scrape targets.
Note that these metrics are exported only for failing scrape targets. They aren't exposed for normally working targets.
 2021-01-27 00:40:39 +02:00
Aliaksandr Valialkin
4b324da947 all: consistently use timers from timerpool 2021-01-27 00:40:39 +02:00
Nikolay
9f0a4fd00e Fixes error handling for promscrape.streamParse (#1009) 
properly return error if client cannot read data,
properly suppress scraper errors
 2021-01-12 13:35:09 +02:00
Aliaksandr Valialkin
367fc17933 lib/promscrape: code prettifying for 8dd03ecf19 2020-12-24 10:57:20 +02:00
Nikolay
b00f7816e2 adds proxy_url support, (#980) 
* adds proxy_url support,
adds proxy_url to the dockerswarm, eureka, kubernetes and consul service discovery,
adds proxy_url to the scrape_config for targets scrapping,
http based proxy is supported atm,
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/503

* fixes imports
 2020-12-24 10:57:19 +02:00
Aliaksandr Valialkin
1ee5a234dc lib/promscrape: remove ID field from ScrapeWork struct. Use a pointer to ScrapeWork as a key in targetStatusMap 
This simplifies the code a bit.
 2020-12-17 14:31:55 +02:00
Aliaksandr Valialkin
eddc2bd017 lib/promscrape: properly handle scrape errors when stream parsing is enabled 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/967
 2020-12-15 14:10:52 +02:00
Aliaksandr Valialkin
9d79d11a6c lib/promscrape: mention in scrape error message that scrape errors can be disabled by -promscrape.suppressScrapeErrors command-line flag 2020-12-06 23:27:07 +02:00
Aliaksandr Valialkin
af667c59c1 all: typo fix: thouthand->thousand 2020-11-26 13:34:05 +02:00
Aliaksandr Valialkin
535fea3d11 lib/promscrape: eliminate data race in stream parse mode 
Previously `-promscrape.streamParse` mode could result in garbage labels for the scraped metrics because of data race.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825#issuecomment-723198247
 2020-11-07 12:45:52 +02:00
Aliaksandr Valialkin
c046735571 lib/promscrape: reduce memory allocations in promLabelsString() function 
This should help with reducing memory usage in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878
 2020-11-04 10:38:59 +02:00
Aliaksandr Valialkin
ed724d25ba lib/promscrape: add stream parse mode for efficient scraping of targets that expose millions of metrics 2020-11-01 23:12:26 +02:00
Aliaksandr Valialkin
0d44e371f3 lib/promscrape: add -promscrape.suppressDuplicateScrapeTargetErrors command-line flag in order to suppress duplicate scrape target errors 
Show also original labels for duplicate targets in error message in order to simplify debugging the issue.

Now `/targets` endpoint accepts optional `show_original_labels=1` query arg, which shows original labels for each target.
This may simplify debugging for target relabeling.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/651
 2020-10-08 18:59:25 +03:00
Aliaksandr Valialkin
ca08161b54 lib/promscrape: typo fix 2020-09-12 00:14:15 +03:00
Aliaksandr Valialkin
e53235ac5c lib/promscrape: do not reset the remaining rows when pushing a part of data to remote storage during big scrapes 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/753

Thanks to @PerGon and @clmssz for help with debugging.
 2020-09-11 23:38:17 +03:00
Aliaksandr Valialkin
95ce89e7d7 lib/promscrape: use the number of parsed rows as a basis for writeRequestCtxPool leveling 
The previous basis on `cap(sw.labels)` doesn't work anymore after 7785869ccc ,
because `sw.labels` may be reset multiple times when processing big number of rows.
 2020-09-02 18:46:55 +03:00
Aliaksandr Valialkin
a01c56104a lib/promscrape: fix applying sample_limit when scraping targets with big number of metrics 
This has been broken at 7785869ccc
 2020-09-01 11:09:25 +03:00
Aliaksandr Valialkin
deff8d419a lib/promscrape: reduce memory usage when scraping targets with millions of metrics 
This should help when scraping /federate endpoints from Prometheus instances,
which scrape millions of metrics. See https://prometheus.io/docs/prometheus/latest/federation/
 2020-09-01 10:55:24 +03:00
Aliaksandr Valialkin
5f2277624a lib/{promscrape,leveledbytebufferpool}: rename getPoolIdAndCapacity to getPoolIDAndCapacity in order to make golint happy 2020-08-28 09:49:22 +03:00
Aliaksandr Valialkin
9a77ae9d1c lib/promscrape: reduce memory usage when scraping targets with big number of metrics alongside targets with small number of labels 
Previously targets with big number of metrics and/or labels could generated too big buffers,
which then could be re-used when scraping targets with small number of metrics.
This resulted in memory waste.

Now big buffers are used only for targets with big number of metrics / labels,
while small buffers are used for targets with small number of metrics / labels.
 2020-08-16 22:30:34 +03:00
Aliaksandr Valialkin
9e67343756 lib/promscrape: use a hint on body length instead of body capacity 
This should reduce memory usage for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/689
 2020-08-14 01:17:46 +03:00
Aliaksandr Valialkin
b4119bb51e lib/promscrape: reduce memory usage when scraping big number of targets 
Thanks to @dxtrzhang for the original idea at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/688

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/689
 2020-08-14 01:05:04 +03:00
Aliaksandr Valialkin
e3439a6cd0 lib/promscrape: optimize per-metric hash calculations 
This increases vmagent performance by up to 10% when scraping big number of metrics
 2020-08-10 19:47:50 +03:00
Aliaksandr Valialkin
e3999ac010 lib/promscrape: show real timestamp and real duration for the scape on /targets page 
Previously the scrape duration may be negative when calculated scrape timestamp drifts away from the real scrape timestamp
 2020-08-10 12:40:49 +03:00
Aliaksandr Valialkin
c09c881264 lib/promscrape: make errcheck happy 2020-08-09 13:17:30 +03:00
Aliaksandr Valialkin
2dfb42a8b4 lib/promscrape: export scrape_samples_added per-target metric like Prometheus does 
This metric may be useful for detecting targets with high churn rate for the exported metrics.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/683
 2020-08-09 12:45:30 +03:00
Roman Khavronenko
a02097e657 Extend metric vm_promscrape_targets with status label (#615) 
The change to `vm_promscrape_targets` metric suppose to improve observability
for `vmagent` so it will be possible to track how many targets are up or down
for every specific scrape group:
```
vm_promscrape_targets{type="static_configs", status="down"} 1
vm_promscrape_targets{type="static_configs", status="up"} 2
```
 2020-07-13 21:54:53 +03:00
Aliaksandr Valialkin
2361ad8ab4 lib/promscrape: add ability to set disable_compression and disable_keepalive options in scrape_config section of the config passed to -promscrape.config 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/580
 2020-07-02 14:19:34 +03:00
Aliaksandr Valialkin
5a43842bd3 lib/promscrape: add missing label sorting for autogenerated metrics 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/592
 2020-06-29 22:39:40 +03:00
Ween
b42cf33c4d Fix Auto metrics relabeled errors (#593) 
* Fix Auto metrics relabeled errors

* Finalize auto-genenated  Labels

* Fix Test Errors

Co-authored-by: xinyulong <xinyulong@kuaishou.com>
 2020-06-29 22:39:39 +03:00
Aliaksandr Valialkin
de7e585ac8 lib/promscrape: preserve the previously discovered targets on discovery errors per each job_name 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/582
 2020-06-23 15:42:46 +03:00
Aliaksandr Valialkin
a5880f17af lib/promscrape: remove debug line left after the commit e4aac6ea40 2020-05-03 17:16:19 +03:00
Aliaksandr Valialkin
ab1e6a76bb lib/promscrape: make consistent scrape time offsets across reloads for the same ScrapeURL and Labels 
This should make consistent intervals between data points for scrape targets across reloads.
Previously these intervals were random.
 2020-05-03 14:31:22 +03:00
Aliaksandr Valialkin
f422203e10 lib/promscrape: reload only modified scrapers on config changes 
This should improve scrape stability when big number of targets are scraped and these targets are frequently changed.

Thanks to @xbsura for the idea and initial implementation attempts at the following pull requests:

- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/449
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/458
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/459
- https://github.com/VictoriaMetrics/VictoriaMetrics/pull/460
 2020-05-03 12:47:16 +03:00
Aliaksandr Valialkin
d86640d609 lib/promscrape: suppress scrape errors if -promscrape.suppressScrapeErrors flag is set 2020-04-16 23:41:52 +03:00
Aliaksandr Valialkin
70104f3fb1 lib/promscrape: print all the labels for the target on error message for failed scrape 
This should improve debuggability.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/420
 2020-04-16 23:35:10 +03:00
Aliaksandr Valialkin
636e1578de lib/promscrape: add promrelabel.GetLabelValueByName helper function 2020-04-14 14:12:15 +03:00
Aliaksandr Valialkin
3945bf9dec lib/promscrape: mention job name in error messages when target cannot be scraped 
This should improve debuggability
 2020-04-14 13:33:18 +03:00
Aliaksandr Valialkin
88366cad15 lib/promscrape: properly expose statuses for targets with duplicate scrape urls at /targets page 
Previously targets with duplicate scrape urls were merged into a single line on the page.
Now each target with duplicate scrape url is displayed on a separate line.
 2020-04-14 13:10:06 +03:00
Aliaksandr Valialkin
f58d15f27c lib/promscrape: rename 'scrape_config->scrape_limit' to 'scrape_config->sample_limit' 
`scrape_config` block from Prometheus config contains `sample_limit` field,
while in `vmagent` this field was mistakenly named as `scrape_limit`.
 2020-04-14 12:00:03 +03:00
Aliaksandr Valialkin
7fbfef2aee lib/promscrape: extract common auth code to lib/promauth 2020-04-13 12:59:22 +03:00
Aliaksandr Valialkin
c189104be7 lib/promscrape: reduce timestamp jitter when scraping targets 
This should improve compression for timestamps
 2020-04-01 16:13:01 +03:00
Aliaksandr Valialkin
7ee7614e90 app/vmagent: initial implementation for vmagent 2020-02-23 17:31:54 +02:00