Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-11-28 18:05:24 +01:00
Code Issues Actions 9 Packages Projects Releases Wiki Activity
2,318 Commits 122 Branches 748 Tags 389 MiB
f9084611bd
Commit Graph

168 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
edcdc39eb3 app/vmagent/remotewrite: cleanup after 1d1ba889fe 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
 2021-02-17 21:42:55 +02:00
Nikolay
1d1ba889fe
adds pushback for fastqueue, (#1075) 
during shutdown currently sending block was lost,
now its pushed back to fast queue and will be flushed on disk,
it may lead to data duplication.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
 2021-02-17 21:23:38 +02:00
Aliaksandr Valialkin
802fabf0d7 deployment/docker: embed tzdata into prod Go app instead of installing it into base docker image 
While this increases app size by 700Kb, this allows using -loggerTimezone in a scratch base image
See https://github.com/golang/go/issues/38017
 2021-02-12 04:54:27 +02:00
Aliaksandr Valialkin
f9902b3372 deployment/docker: use docker buildx for creating multiarch builds 
See https://github.com/docker/buildx/
 2021-02-12 04:31:22 +02:00
Aliaksandr Valialkin
8dc6095749 app/vmagent: add Advanced usage section with the description for all the command-line flags 2021-02-03 21:02:59 +02:00
Aliaksandr Valialkin
b577cdd855 docs: increase heading sizes in vmagent, vmauth, vmbackup and vmrestore docs, so they match the heading sizes in VictoriaMetrics docs 2021-02-01 19:44:00 +02:00
Aliaksandr Valialkin
b2aa80e74b app/vmagent: add -remoteWrite.roundDigits command-line option for limiting the number of digits after the point for stored values 
This commit also adds --vm-round-digits command-line option to vmctl tool.
 2021-02-01 14:27:09 +02:00
Aliaksandr Valialkin
de3c662e8a all: consistently use timers from timerpool 2021-01-27 00:39:26 +02:00
Aliaksandr Valialkin
419ad74269 app/vmagent: add -remoteWrite.rateLimit command-line flag for limiting data rate to remote storage 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1035
 2021-01-27 00:39:26 +02:00
Aliaksandr Valialkin
75f7c51cab docs/vmagent.md: follow-up for 184a659c5f 2021-01-13 13:53:14 +02:00
Nikolay
dc9d7aedd5
adds extra_label to all import apis (#1007) 
* adds extra_label to all import apis,
changes priority for extra_label - now it has priority over original labels

* Update README.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>

* Update README.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>

* adds extra labels to vmagent  import api
changes order for adding labels, now its added after user values

* adds tests for extra_label

* import fix

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2021-01-13 00:52:50 +02:00
Nikolay
7976c22797
Fixes error handling for promscrape.streamParse (#1009) 
properly return error if client cannot read data,
properly suppress scraper errors
 2021-01-12 13:31:47 +02:00
Aliaksandr Valialkin
24ffad74c1 all: use net.Dial instead of fasthttp.Dial, because fasthttp.Dial limits the number of concurrent dials to 1000 2021-01-11 12:53:30 +02:00
Aliaksandr Valialkin
aa56b9217e app/vmagent: add vmagent_remotewrite_blocks_sent_total and vmagent_remotewrite_bytes_sent_total metrics per each -remoteWrite.url 2020-12-15 20:39:57 +02:00
Aliaksandr Valialkin
b10ad44692 docs/vmagent.md: typo fix: pearsed->parsed 2020-12-15 19:03:01 +02:00
Aliaksandr Valialkin
1eabbc0e27 docs/vmagent.md: mention that sample_limit option has no sense when stream parsing is enabled 2020-12-15 18:43:41 +02:00
Nikolay
95c9b630cc
adds new Array Flags (#965) 
* adds ArrayDuration and ArrayBool flags,
makes sendTimeout and tlsInsecure configurable per remoteWrite url

* added backward compatibility testcases for ArrayDuration and ArrayBool

* fixes bool flag

* fixes test cases
 2020-12-15 12:51:12 +02:00
Aliaksandr Valialkin
069c9ade52 app/{vmagent,vminsert}: follow-up for ce8c2dd1f1: return /targets page in HTML when requested via web browser 2020-12-14 14:06:00 +02:00
Nikolay
ce8c2dd1f1
Changes targets api (#961) 
* changes /targets api
adds html response if requester accepts text/html,
adds quick template for /targets api,
fixes pathPrefix for / requests

* changes namings

* renamed targets file

* Update app/victoria-metrics/main.go

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>

* adds trimspace to qtpl,
moves content-type for targets response closer to writer

* fixes bug with prefix

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-12-14 13:36:48 +02:00
Aliaksandr Valialkin
9d42546a27 docs: consistently use links to https://victoriametrics.github.io for documentation references 2020-12-11 21:08:18 +02:00
Aliaksandr Valialkin
4146fc4668 all: properly handle CPU limits set on the host system/container 
This can reduce memory usage on systems with enabled CPU limits.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946
 2020-12-08 21:07:29 +02:00
Aliaksandr Valialkin
b6b1b06d70 app/{vmalert,vmagent}: skip empty values in -remoteWrite.label and -label lists 2020-12-08 14:55:13 +02:00
Aliaksandr Valialkin
9d787f9edd all: do not print usage info for all the flags when incorrect command-line flag is passed 
This should improve usability for VictoriaMetrics apps that have big number of command-line flags,
i.e. all the apps.
 2020-12-03 21:47:37 +02:00
Aliaksandr Valialkin
b7fcdb528d app/{vmagent,victoria-metrics}: add -dryRun option and make more clear handling for -promscrape.config.dryRun 2020-11-25 22:59:13 +02:00
Aliaksandr Valialkin
dabbf930d8 app/vmagent: do not enable -promscrape.config.strictParse when -dryRun command-line flag is set 
Users can specify -promscrape.config.strictParse if -promscrape.config shouldn't contain unknown config entries
 2020-11-25 22:26:25 +02:00
Nikolay
eec76718e9
Adds eureka service discovery (#913) 
* Adds eureka service discovery
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/851
Netflix service discovery for AWS

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-11-20 13:38:12 +02:00
Aliaksandr Valialkin
47a038401b all: consistently return text-based HTTP responses with charset=utf-8 
This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897
 2020-11-13 10:35:41 +02:00
faceair
077f8cbe1c
add charset on targets response (#897) 2020-11-13 10:17:37 +02:00
Aliaksandr Valialkin
4057305148 docs/vmagent.md: added a link to https://valyala.medium.com/how-to-use-relabeling-in-prometheus-and-victoriametrics-8b90fc22c4b2 into Relabeling section 2020-11-12 12:27:06 +02:00
Aliaksandr Valialkin
bb06b98202 docs/vmagent.md: typo fix 2020-11-11 16:04:46 +02:00
Aliaksandr Valialkin
4adb96161a docs/vmagent.md: add Configuration update section 2020-11-11 16:01:15 +02:00
Aliaksandr Valialkin
bcd139362b lib/promscrape: add -promscrape.dropOriginalLabels command-line flag for reducing memory usage when discovering big number of scrape targets 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825
 2020-11-10 00:19:57 +02:00
Aliaksandr Valialkin
83df20b5b5 lib/promscrape: clean references to label name and label value strings after applying per-target relabeling 
This should reduce memory usage when per-target relabeling creates big number of temporary labels
with long names and/or values.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825
 2020-11-07 16:19:59 +02:00
Aliaksandr Valialkin
381ad564a2 docs/vmagent.md: update after 4c808d58bf 2020-11-04 20:31:43 +02:00
Nikolay
4c808d58bf
Adds ready probe (#874) 
* adds leading forward slash check for scrapeURL path
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/835

* adds ready probe for scrape config initialization,
it should prevent metrics loss during vmagent rolling update,
/ready api will return 425 http code, if some scrape config still waits for initialization.

* updates docs

* Update app/vmagent/README.md

* renames var

* Update app/vmagent/README.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-11-04 20:29:18 +02:00
Nikolay
e4182dd896
reduces memory usage for vmagent, (#880) 
* reduces memory usage for vmagent,
limits count of droppedTarget, that can be stored for /api/v1/targets page up to 999 items,
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878

* Update app/vmagent/README.md

* Update app/vmagent/README.md

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-11-04 17:03:43 +02:00
Aliaksandr Valialkin
600f225cff Revert "docs/vmagent.md: mention about -promscrape.dropOriginalLabels" 
This reverts commit bd81f926a4.
 2020-11-04 11:44:57 +02:00
Aliaksandr Valialkin
bd81f926a4 docs/vmagent.md: mention about -promscrape.dropOriginalLabels 2020-11-04 11:16:33 +02:00
Aliaksandr Valialkin
e0a91ef163 app/vmagent/remotewrite: drop packets only on 409 status code, since there are other valid 4xx status codes, which shouldnt result in packet drop 2020-11-03 14:25:10 +02:00
Aliaksandr Valialkin
0b35da159c app/vmagent/remotewrite: drop packets if remote storage returns 4xx status code 
This makes consistent the behaviour with Prometheus.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/873
 2020-11-02 00:45:09 +02:00
Aliaksandr Valialkin
e277c3d07b lib/promscrape: add stream parse mode for efficient scraping of targets that expose millions of metrics 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
b7638f04a7 app/vmagent: expose /api/v1/targets page according to https://prometheus.io/docs/prometheus/latest/querying/api/#targets 
This page is exposed by vmagent and by a single-node VictoriaMetrics

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/643
 2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin
63c4999e06 lib/promscrape: code prettifying after 9bd9f67718 2020-10-12 16:12:36 +03:00
Nikolay Khramchikhin
9bd9f67718
Adds dockerswarm sd (#818) 
* adds dockerswarm service discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/656

 Following roles supported: services, tasks and nodes.
 Basic, token and tls auth supported.
 Added tests for labels generation.

* added unix socket support to discovery utils

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-10-12 13:38:21 +03:00
Aliaksandr Valialkin
84227ea2fc app/{vminsert,vmagent}: take into account all the inserted rows before relabeling in vm_rows_inserted_total and vmagent_rows_inserted_total metrics 2020-10-09 13:29:51 +03:00
Aliaksandr Valialkin
6105d61d11 docs/vmagent.md: clarify -promscrape.suppressDuplicateScrapeTargetErrors command-line flag usage 2020-10-08 19:24:31 +03:00
Aliaksandr Valialkin
71ea4935de lib/promscrape: add -promscrape.suppressDuplicateScrapeTargetErrors command-line flag in order to suppress duplicate scrape target errors 
Show also original labels for duplicate targets in error message in order to simplify debugging the issue.

Now `/targets` endpoint accepts optional `show_original_labels=1` query arg, which shows original labels for each target.
This may simplify debugging for target relabeling.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/651
 2020-10-08 18:58:30 +03:00
Aliaksandr Valialkin
d16dbfd639 app/vmagent: add a link to https://victoriametrics.github.io/vmagent.html from main page 2020-10-06 15:29:49 +03:00
Nikolay Khramchikhin
cbe3cf683b
Adds openstack sd (#811) 
* adds openstack service discovery

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/728

 implemented hypervisors and instance discovery with openstack v3 api.
 Added tests for labeling and data parsing.
 Added token refresh.

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-10-05 16:45:33 +03:00
Aliaksandr Valialkin
127537d631 app/vmagent/remotewrite: do not show -remoteWrite.url in logs if -remoteWrite.showURL isn't set 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/773
 2020-09-29 19:49:12 +03:00