Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-25 20:00:06 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
4,160 Commits 113 Branches 748 Tags 707 MiB
7810375c5f
Commit Graph

308 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
f3d4671bb6
lib/promscrape: follow-up after 7e79adfb55 2022-04-12 12:36:17 +03:00
Aliaksandr Valialkin
569b0d444c
app/vmagent: properly initialize stdDialer 
This is a follow-up commit for 7da20a4b3f

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1699
 2022-04-06 13:57:20 +03:00
Aliaksandr Valialkin
7da20a4b3f
app/vmagent: reduce the probability of TLS handshake timeout when dialing the remote storage 
The following actions are taken:

- Increase the TLS hashdshake timeout from 5 seconds to 10 seconds
- Increase dial timeout from 5 seconds to 30 seconds
- Specify DialContext instead of Dial in http.Transport. This allows properly handling
  the Context arg during dialing the remote storage

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1699
 2022-04-06 12:34:25 +03:00
Nikolay
0c0efc7781
vmctl verify-blocks command (#2390) 
* lib/protoparser: changes ParseStream for native format
uses reader instead of http.Request
updates app/vmagent and app/vmagent method usage

* app/vmctl: add verify-block subcommand
it allows to check exported from VictoriaMetrics data block in native format
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2362

Update app/vmctl/README.md

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
 2022-04-05 16:01:32 +02:00
Aliaksandr Valialkin
c8f356a6a8
app: sync Markdown changes from a8de1ab000 2022-03-22 14:11:18 +02:00
Aliaksandr Valialkin
5ab6c350ec
docs/CHANGELOG.md: document a1e17e91f8 2022-03-21 15:34:49 +02:00
Aliaksandr Valialkin
ec03dec72d
app/vmagent/remotewrite: prevent from infinite recursion panic when pushing a time series with big number of samples to remote storage 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2335
 2022-03-18 19:06:22 +02:00
Aliaksandr Valialkin
3f49bdaeff
lib/promrelabel: add support for conditional relabeling via if filter 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998
 2022-02-24 02:27:26 +02:00
Aliaksandr Valialkin
979d89f5dd
app/vmagent: add -influxDBLabel in the same way as for app/vminsert 
This is a follow-up for 3d19fa6932
 2022-02-21 14:09:20 +02:00
Roman Khavronenko
3ac3124eed
docs: make profiling commands more copy-friendly (#2213) 
* docs: make profiling commands more copy-friendly

The change adds `copy text` snippet to code examples
and replaces hostname placeholders with `0.0.0.0`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-18 12:23:36 +02:00
Aliaksandr Valialkin
ee5da826e9
docs: update -help output for VictoriaMetrics components 2022-02-15 21:08:22 +02:00
Nikolay
75e84144c7
adds release build for macos darwin amd64 and arm64 (#2185) 
* adds release build for macos darwin amd64 and arm64

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1896
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1851

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2022-02-14 17:28:56 +02:00
Aliaksandr Valialkin
a56b77db5b
docs/vmagent.md: add unsupported Prometheus config sections chapter 2022-02-12 01:08:10 +02:00
Aliaksandr Valialkin
9bb60ab00f
lib/promscrape: set -promscrape.config.strictParse to true by default 
This allows detecting long-living silent errors in -promscrape.config
 2022-02-08 15:41:43 +02:00
Aliaksandr Valialkin
2968779f16
lib/promscrape: provide the ability to fetch target responses on behalf of vmagent or single-node VictoriaMetrics 
This feature may be useful when debugging metrics for the given target located in isolated environment
 2022-02-03 19:00:55 +02:00
Aliaksandr Valialkin
e4e36383e2
lib/promscrape: do not send staleness markers on graceful shutdown 
This follows Prometheus behavior.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2013#issuecomment-1006994079
 2022-01-07 01:17:57 +02:00
Denys Holius
ae89b4e818
Old links replaced for newest (#2033) 
* replaced old links to the website

* fixed deletion main README.md file

* fix: added docs files after docs-sync
 2022-01-05 16:30:13 +02:00
Yurii Kravets
f0c331c724
Update README.md (#1996) 
* Update README.md

go 1.16 -> 1.17

* Update README.md

* Update README.md

* Update Cluster-VictoriaMetrics.md

* Update Single-server-VictoriaMetrics.md

* Update vmauth.md

* Update vmbackup.md

* Update vmrestore.md

* Update vmagent.md

* Update vmctl.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md
 2021-12-23 12:09:59 +02:00
Aliaksandr Valialkin
ce333f28d8
all: use logger.WithThrottler() where appropriate 2021-12-21 17:03:25 +02:00
Roman Khavronenko
34fdc8881b
vmagent: add error log for skipped data block when rejected by receiv… (#1956) 
* vmagent: add error log for skipped data block when rejected by receiving side

Previously, rejected data blocks were silently dropped - only metrics were update.
From operational perspective, having an additional logging for such cases is preferable.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1911

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmagent: throttle log messages about skipped blocks

The new type of logger was added to logger pacakge.
This new type supposed to control number of logged messages
by time.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* lib/logger: make LogThrottler public, so its methods can be inspected by external packages

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2021-12-21 16:36:09 +02:00
Aliaksandr Valialkin
afafeb379a
all: typo fix: unexected -> unexpected 2021-12-20 17:39:52 +02:00
Aliaksandr Valialkin
06642d97f5
app: allow specifying http and https urls in the following command-line flags 
* -promscrape.config
* -relabelConfig
* -remoteWrite.relabelConfig
* -remoteWrite.urlRelabelConfig
 2021-12-03 00:10:02 +02:00
Aliaksandr Valialkin
90c542af12
app: use relative paths instead of absolute paths for the supported http handlers on the main page 
This allows hiding VictoriaMetrics components behind proxies, which serve pages at different path prefixes

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1858
 2021-12-02 13:52:39 +02:00
Aliaksandr Valialkin
e5ac9d8e57
all: consistently return application/json content-type without charset=utf-8 
The `application/json` content-type has utf-8 encoding by default.
See https://stackoverflow.com/questions/9254891/what-does-content-type-application-json-charset-utf-8-really-mean

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897
 2021-11-09 18:04:44 +02:00
Aliaksandr Valialkin
a67518fc6d
docs: mention that graphs on the official dashboards contain useful hints 2021-11-08 19:54:10 +02:00
Aliaksandr Valialkin
c43bcdb5fb
app/vmagent: allow bigger number of in-memory blocks for big values of -remoteWrite.queues 
This should improve the maximum data ingestion speed for highly-loaded vmagent instances
which run on beefy servers with many CPU cores and big amounts of RAM
 2021-11-05 15:16:05 +02:00
Aliaksandr Valialkin
cbfc7b7c92
app/{vminsert,vmagent}: hide passwords and auth tokens by default at /config page 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764
 2021-11-05 14:41:16 +02:00
Aliaksandr Valialkin
4839d07f34
app/vmagent/remotewrite: fix parallel data sending to remote storage systems at e0d2ba5608 2021-11-04 16:58:28 +02:00
Aliaksandr Valialkin
a69264e885
app/vmagent: add -remoteWrite.maxRowsPerBlock command-line option, which may be used for improving data ingestion performance under high load 2021-11-04 15:39:14 +02:00
Aliaksandr Valialkin
e0d2ba5608
app/vmagent/remotewrite: send data to remote storage systems in parallel 
This should improve data ingestion speed when many `-remoteWrite.url` command-line flags are configured
 2021-11-04 15:04:16 +02:00
Aliaksandr Valialkin
aa534c2582
lib/promscrape: add -promscrape.maxResponseHeadersSize command-line flag for tuning the maximum http response headers size from Prometheus scrape targets 2021-11-03 22:26:56 +02:00
Aliaksandr Valialkin
d1eb87c831
app/{vmagent,vminsert}: add ability to restrict access to /config page with authKey query arg 
The authKey can be configured via `-configAuthKey` command-line flag.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764
 2021-11-01 16:44:54 +02:00
Aliaksandr Valialkin
4821adfd95
lib/promscrape: properly show proxy_url option value at /config page 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755
 2021-10-26 21:23:54 +03:00
Aliaksandr Valialkin
2876137c92
lib/promscrape: reduce memory usage if -promscrape.noStaleMarkers command-line flag is passed 
Do not store in memory the response from the last scrape per each target if -promscrape.noStaleMarkers option is enabled.
This should reduce memory usage when the scraped targets return large responses.
 2021-10-22 13:10:29 +03:00
Aliaksandr Valialkin
8ad95f0db7
lib/httpserver: expose command-line flags at /flags page 
This should simplify debugging.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695
 2021-10-20 00:45:09 +03:00
Aliaksandr Valialkin
9866dd95c1
lib/promscrape: store the full response in stream parsing mode in scrapeWork.lastScrape byte slice 
This allows sending staleness marks and properly calculate scrape_series_added metric in stream parsing mode
at the cost of the increased memory usage, since now the potentially big response is kept
in the lastScrape byte slice per each scrapeWork.

In practice the memory usage increase shouldn't be big, since the response size
is usually much smaller than the parsed metrics from this response after the relabeling,
which usually adds a big pile of target-specific labels per each metric.
 2021-10-15 15:39:23 +03:00
Aliaksandr Valialkin
bbd34fa15e
lib/promscrape: add -promscrape.minResponseSizeForStreamParse command-line option for automatic switching to stream parsing mode when scraping targets with big responses 
This should reduce memory usage when vmagent scrapes targets with non-uniform response sizes.
This is common case in Kubernetes monitoring.
 2021-10-14 12:29:35 +03:00
Aliaksandr Valialkin
1a7287c408
lib/promscrape: return error if sample_limit or series_limit options are set when stream parsing mode is enabled 2021-10-14 12:11:23 +03:00
Aliaksandr Valialkin
5a58c041c2
app/vmagent: expose -promscrape.config contents at /config page as Prometheus does 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695
 2021-10-12 16:25:37 +03:00
Aliaksandr Valialkin
c7c966d0e9
docs/vmagent.md: update docs after 3e9a939a990c8b608414388c96f68eb062364ae7 2021-10-05 10:23:33 +03:00
Aliaksandr Valialkin
9515e58e28
docs/vmagent.md: document how to write data to Kafka 2021-09-30 17:45:53 +03:00
Aliaksandr Valialkin
396e233ac1
docs/vmagent.md: update Telegraf config in the section about Kafka 2021-09-29 11:21:15 +03:00
Aliaksandr Valialkin
0e5ab52908
docs/vmagent.md: add docs about reading metrics from Kafka 2021-09-29 01:46:12 +03:00
Nikolay
cc72f9428d
changes vmagent api (#1656) 
* changes vmagent api
adds auth.Token to promremotewrite InsertHandlerReader
changes remoteWrite client constructor, allows to use multiple remoteWriteUrl schemes, like kafka://
changes url path concatenation for tenant remoteWrite

Update app/vmagent/remotewrite/client.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>

* Update app/vmagent/remotewrite/remotewrite.go

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2021-09-29 00:52:07 +03:00
Aliaksandr Valialkin
91b3c601bc
app/{vminsert,vmagent}: add ability to ingest data via DataDog "submit metrics" API 
See https://docs.datadoghq.com/api/latest/metrics/#submit-metrics

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/206
 2021-09-29 00:13:08 +03:00
Aliaksandr Valialkin
c5bb95a417
docs: make docs-sync 2021-09-23 20:51:35 +03:00
Aliaksandr Valialkin
94148d5ad7 docs/vmagent.md: typo fix 2021-09-20 16:49:20 +03:00
Aliaksandr Valialkin
51657b1e04 docs/vmagent.md: typo fixes in Prometheus staleness markers docs 2021-09-20 16:44:09 +03:00
Nikolay
ad08d9dfc0
changes protoparser apis for accepting reading from io.Reader (#1624) 
adds InsertHandlerForReader apis to vmagent
 2021-09-20 14:49:28 +03:00
Aliaksandr Valialkin
07b9c7994f docs/vmagent.md: mention out of order sample errors, which are typically emitted by Thanos, Cortex or Prometheus 2021-09-13 19:36:31 +03:00