Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-28 09:55:36 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,016 Commits 122 Branches 748 Tags 389 MiB
5009b25a03
Commit Graph

736 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
5009b25a03 lib/promscrape: code cleanup after c6dee6c52d 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574
 2020-12-03 19:50:53 +02:00
Nikolay
c6dee6c52d
Changes consul discovery api (#921) 
* adds consul watch api,
it must reduce load on consul service with blocking wait requests,
changed discoveryClient api with fetchResponseMeta callback.

* small fix

* fix after master merge

* adds watch client at discovery utils

* fixes consul watcher,
changes namings,
fixes data race

* small typo fix

* sanity fix

* fix naming and service node update
 2020-12-03 19:47:40 +02:00
Aliaksandr Valialkin
357f886f97 Revert "lib/storage: add missing (AccountID, ProjectID) in MetricName.String() test" 
This reverts commit f3e80eb70e493620e42a0cc22a62c9af75076c77, since it isn't needed for single-node version of VictoriaMetrics
 2020-11-29 01:27:25 +02:00
Aliaksandr Valialkin
ace969d595 lib/storage: add missing (AccountID, ProjectID) in MetricName.String() test 2020-11-29 01:26:04 +02:00
Aliaksandr Valialkin
32869e4c0f lib/promscrape: fix failing tests after a906b3862f 2020-11-29 01:26:03 +02:00
Aliaksandr Valialkin
a906b3862f lib/protoparser/prometheus: properly parse OpenMetrics timestamps 
OpenMetrics timestamps are floating-point numbers, that represent Unix timestamp in seconds.
This differs from Prometheus exposition format, where timestamps are integer numbers representing Unix timestamp in milliseconds.
 2020-11-27 14:54:29 +02:00
Aliaksandr Valialkin
3a32789352 lib/promscrape: reduce memory allocations when unpacking gzipped responses received from scrape targets 2020-11-26 18:32:06 +02:00
Aliaksandr Valialkin
2cea4d403f all: typo fix: thouthand->thousand 2020-11-26 13:33:46 +02:00
Aliaksandr Valialkin
b0a5c382ee lib/promscrape: release http response non-200 status code 2020-11-26 13:25:17 +02:00
Aliaksandr Valialkin
b7fcdb528d app/{vmagent,victoria-metrics}: add -dryRun option and make more clear handling for -promscrape.config.dryRun 2020-11-25 22:59:13 +02:00
Aliaksandr Valialkin
1c669a69a8 lib/mergeset: tune the number of rawItemsBlocks to merge at once 
512 blocks give higher ingestion performance and slightly lower memory usage
 2020-11-25 21:52:52 +02:00
Aliaksandr Valialkin
7119f294f3 lib/mergeset: help GC by removing refereces to slices in inmemoryBlock.Reset 2020-11-25 21:19:43 +02:00
Aliaksandr Valialkin
8a057e705a lib/storage: log metric name plus all its labels when the metric timestamp is outside the configured retention 
This should simplify debugging when the source of the metric with unexpected timestamp must be found.
 2020-11-25 14:41:37 +02:00
Aliaksandr Valialkin
b65236530c lib/storage: typo fix in error message: allowd->allowed 2020-11-25 14:15:42 +02:00
Aliaksandr Valialkin
ae04378424 lib/protoparser/prometheus: properly parse "infinity" values in OpenMetrics format 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/924
 2020-11-24 19:03:38 +02:00
Aliaksandr Valialkin
bf95fbfc1d lib/logger: disable rate limiting for error and warn logs by default 2020-11-24 12:42:25 +02:00
Aliaksandr Valialkin
78d2715d04 all: spelling fix: superflouos->superfluous. This is a follow-up for 0acdab3ab9 2020-11-24 12:42:22 +02:00
Aliaksandr Valialkin
b7f4fc6e0d lib/protoparser/prometheus: properly parse metrics with exemplars 
Examplars have been introduced in OpenMetrics - see https://github.com/OpenObservability/OpenMetrics/blob/master/OpenMetrics.md#exemplars-1
Previously VictoriaMetrics couldn't parse the following metric

    foo{bar="baz"} 123 # exemplar here

This commit fixes this. Note that VictoriaMetrics ignores the exemplar as for now.
 2020-11-24 12:34:56 +02:00
Aliaksandr Valialkin
aa90b93778 lib/promscrape: expose __meta_ec2_ipv6_addresses label for ec2_sd_config like Prometheus will do in the next release 2020-11-23 16:56:42 +02:00
Aliaksandr Valialkin
de523c81b9 lib/promscrape: add filters option to dockerswarm_sd_config like Prometheus did in v2.23.0 2020-11-23 16:27:40 +02:00
Aliaksandr Valialkin
f765985947 lib/fs: replace fs.OpenReaderAt with fs.MustOpenReaderAt 
All the callers for fs.OpenReaderAt expect that the file will be opened.
So it is better to log fatal error inside fs.MustOpenReaderAt instead of leaving this to the caller.
 2020-11-23 09:57:21 +02:00
Aliaksandr Valialkin
9d160f9048 lib/promscrape: hint that -enableTCP6 command-line flag can be used for connecting to IPv6 addresses 2020-11-21 14:39:00 +02:00
Aliaksandr Valialkin
d7932775cc lib/promscrape/discovery/eureka: follow-up after eec76718e9 2020-11-20 14:00:12 +02:00
Nikolay
eec76718e9
Adds eureka service discovery (#913) 
* Adds eureka service discovery
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/851
Netflix service discovery for AWS

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-11-20 13:38:12 +02:00
Aliaksandr Valialkin
97a0c80904 lib/logger: follow-up for 09105ff49c 2020-11-19 12:37:00 +02:00
Nikolay
09105ff49c
Adds log suppression per caller (#908) 
* Adds log suppression per caller
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/905

* fixes style and report message
 2020-11-19 12:17:23 +02:00
Aliaksandr Valialkin
a6b2b2c005 lib/logger: add -loggerWarnsPerSecondLimit command-line flag for rate limiting of WARN log messages 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/905
 2020-11-18 03:43:37 +02:00
Aliaksandr Valialkin
266788be14 app/vmselect: use storage.NewSearchQuery() instead of constructing storage.SearchQuery in-place 
This should prevent from bugs when AccountID and ProjectID aren't set in storage.SearchQuery.
 2020-11-16 18:24:00 +02:00
Aliaksandr Valialkin
0208d8c103 lib/storage: add a test for Storage.SearchMetricNames 2020-11-16 13:15:16 +02:00
Aliaksandr Valialkin
465923b181 app/vmselect/graphite: add /tags/findSeries handler from Graphite Tags API 
See https://graphite.readthedocs.io/en/stable/tags.html#exploring-tags
 2020-11-16 12:53:13 +02:00
Aliaksandr Valialkin
48d033a198 app/vminsert: add /tags/tagSeries and /tags/tagMultiSeries handlers from Graphite Tags API 
See https://graphite.readthedocs.io/en/stable/tags.html#adding-series-to-the-tagdb
 2020-11-16 02:39:58 +02:00
Aliaksandr Valialkin
4aaee33860 lib/storage: do not show artifically created label for reverse Graphite labels at /api/v1/labels page 2020-11-16 00:44:35 +02:00
Aliaksandr Valialkin
0deabbbb4a lib/protoparser/promremotewrite: log the time spent on unsuccessful data read from the network 
This should help with debugging `connection timed out` errors.
 2020-11-13 17:49:12 +02:00
Vasily
6fcbd17bdd Add omitempty for DisableCompression and DisableKeepAlive fields in ScrapeConfig (#796) 
* Add omitempty for DisableCompression and DisableKeepAlive fields in ScrapeConfig

* Add omitempty annotation to all the default/optional values

* Fix annotations after review
 2020-11-13 16:19:05 +02:00
Aliaksandr Valialkin
cdf1e6684b lib/protoparser/opentsdbhttp: increment errors counter on unmarshal errors 
This is a follow-up for 149c0c4a6d
 2020-11-13 13:23:17 +02:00
Aliaksandr Valialkin
149c0c4a6d lib/protoparser: propagate callback error to the caller of ParseStream for every supported data ingestion protocols 
The caller of ParseStream then can generate HTTP 503 responses for non-nil errors occured in callbacks when processing incoming requests.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/896
 2020-11-13 13:05:24 +02:00
Aliaksandr Valialkin
4f8a3af061 lib/protoparser/promremotewrite: synchronously process Prometheus remote_write requests 
There is no reason in processing these requests asynchronously in the face of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/896
Synchronous processing code is easier to read and understand than the previous async code
 2020-11-13 12:17:25 +02:00
Aliaksandr Valialkin
57a4af98fa lib/protoparser/promremotewrite: forward errors, which can occur during data ingestion, to the caller of ParseStream, so it could properly return HTTP 503 status code on non-nil error 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/896
 2020-11-13 11:01:07 +02:00
Aliaksandr Valialkin
47a038401b all: consistently return text-based HTTP responses with charset=utf-8 
This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897
 2020-11-13 10:35:41 +02:00
immerrr again
51c529a2b6
app/vmstorage: add "/internal/force_flush" endpoint (#893) 2020-11-11 14:40:27 +02:00
Aliaksandr Valialkin
e205975716 lib/promscrape: make a copy of ScrapeWork from discovered []ScrapeWork slice instead of referring to an item in this slice 
This should prevent from holding previously discovered []ScrapeWork slices when a part of discovered targets changes over time.
This should reduce memory usage for the case when big number of discovered scrape targets changes over time.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825
 2020-11-10 16:13:05 +02:00
Aliaksandr Valialkin
6e668fd480 lib/promscrape: pre-allocate slice for discovered targets based on previously discovered targets 
This should reduce load on GC a bit when discovering big number of scrape targets
 2020-11-10 15:56:51 +02:00
Aliaksandr Valialkin
bcd139362b lib/promscrape: add -promscrape.dropOriginalLabels command-line flag for reducing memory usage when discovering big number of scrape targets 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825
 2020-11-10 00:19:57 +02:00
Aliaksandr Valialkin
6c24c5caa3 lib/promscrape: further reduce memory usage for per-scrape target labels by making a copy of actually used labels 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825
 2020-11-09 10:54:42 +02:00
Aliaksandr Valialkin
83df20b5b5 lib/promscrape: clean references to label name and label value strings after applying per-target relabeling 
This should reduce memory usage when per-target relabeling creates big number of temporary labels
with long names and/or values.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825
 2020-11-07 16:19:59 +02:00
Aliaksandr Valialkin
9e83335ca9 lib/promscrape/discovery/kubernetes: go fmt 2020-11-07 13:03:49 +02:00
Aliaksandr Valialkin
5407eed2f6 lib/promscrape/discovery/kubernetes: reduce memory usage for labels when discovering big number of scrape targets by using string concatenation instead of fmt.Sprintf 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825
 2020-11-07 13:03:08 +02:00
Aliaksandr Valialkin
188325f0fc lib/promscrape: eliminate data race in stream parse mode 
Previously `-promscrape.streamParse` mode could result in garbage labels for the scraped metrics because of data race.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825#issuecomment-723198247
 2020-11-07 12:45:17 +02:00
Aliaksandr Valialkin
368b69b4c4 app/vmselect: properly handle errors in GetLabelsOnTimeRange and GetLabelValuesOnTimeRange 2020-11-05 01:38:38 +02:00
Aliaksandr Valialkin
1cb78ba1a0 lib/storage: remove data race when updating rowsDeleted 2020-11-05 01:12:21 +02:00