Aliaksandr Valialkin
983b465757
app/vmagent: add missing _total suffix to vmagent_remotewrite_global_rows_pushed_before_relabel_total counter
...
This is a follow up for c536139d0b
2022-05-06 15:51:34 +03:00
Aliaksandr Valialkin
2baa9e8f48
app/vmagent: expose vmagent_remotewrite_global_rows_pushed_before_relabel and vmagent_remotewrite_rows_pushed_after_relabel_total metrics
2022-05-06 15:30:10 +03:00
Aliaksandr Valialkin
9c89ea1c90
app/vmagent: rename vmagent_remote_write_rate_limit_reached_total to vmagent_remotewrite_rate_limit_reached_total for the sake of consistency with other vmagent_remotewrite_ metrics
2022-05-06 15:02:18 +03:00
Aliaksandr Valialkin
873f55bac5
lib/awsapi: pass filtersQueryString arg to GetEC2APIResponse() function, so the caller could decide whether to use the filters during the AWS API query
...
The filters shouldn't be passed to DescribeAvailabilityZones API call.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1626
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287
Related commits:
0e09fdb8b0d289ecded1
2022-05-05 10:29:47 +03:00
Aliaksandr Valialkin
9fea63d1c8
app/vmagent: rename -remoteWrite.useSigv4 command-line flag to -remoteWrite.aws.useSigv4, so its name is consistent with the other -remoteWrite.aws.* command-line flags
2022-05-04 20:41:45 +03:00
Nikolay
7e58cba6cf
{lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite ( #2458 )
...
* {lib/promscrape,app/vmagent}: adds sigv4 support for vmagent remoteWrite
moves aws related code into separate lib from lib/promscrape
it allows to write data from vmagent to the AWS managed prometheus (cortex)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1287
* Apply suggestions from code review
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-05-04 20:28:37 +03:00
Gard Rimestad
ceadcb7f8e
app/vmagent add metric for rate limit ( #2521 )
...
This adds a metric for the rate limit.
The limit is present as a flag currently:
`flag{name="remoteWrite.rateLimit", value="500000", is_set="true"} 1`
We are running many instances of vmagent and when creating alerts it is harder than it needs to be when extracting the value from the flag.
With this change it should be easier to monitor how close to the limit we are.
`((100/vmagent_remotewrite_rate_limit{account="account"})*sum (rate(vmagent_remotewrite_conn_bytes_written_total{account="account"}))) and ON (account) flag{name="remoteWrite.rateLimit"} == 1`
2022-05-02 22:25:47 +03:00
Aliaksandr Valialkin
917bb702ef
app/vmagent: substitute hard-to-read 500000000 with 500MB in -remoteWrite.maxDiskUsagePerURL description
2022-04-26 15:48:44 +03:00
Aliaksandr Valialkin
d0bac8e224
all: typo fix: Kuberntes -> Kubernetes
2022-04-20 10:51:41 +03:00
Aliaksandr Valialkin
7debf57ca6
lib/httpserver: clarify that -tls flag enables TLS for http requests to -httpListenAddr
2022-04-16 16:59:41 +03:00
Aliaksandr Valialkin
6bd032a6d3
docs: sync docs with the latest changes
2022-04-16 16:00:27 +03:00
Aliaksandr Valialkin
70ad171070
lib/promscrape: follow-up after 7e79adfb55
2022-04-12 12:37:03 +03:00
Aliaksandr Valialkin
a57d5dd415
app/vmagent: properly initialize stdDialer
...
This is a follow-up commit for 7da20a4b3fhttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/1699
2022-04-06 14:00:50 +03:00
Aliaksandr Valialkin
f082e64e0c
app/vmagent: reduce the probability of TLS handshake timeout when dialing the remote storage
...
The following actions are taken:
- Increase the TLS hashdshake timeout from 5 seconds to 10 seconds
- Increase dial timeout from 5 seconds to 30 seconds
- Specify DialContext instead of Dial in http.Transport. This allows properly handling
the Context arg during dialing the remote storage
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1699
2022-04-06 12:35:14 +03:00
Nikolay
7eb49d204f
vmctl verify-blocks command ( #2390 )
...
* lib/protoparser: changes ParseStream for native format
uses reader instead of http.Request
updates app/vmagent and app/vmagent method usage
* app/vmctl: add verify-block subcommand
it allows to check exported from VictoriaMetrics data block in native format
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2362
Update app/vmctl/README.md
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-04-05 17:46:36 +03:00
Aliaksandr Valialkin
ba7cfd7b25
app: sync Markdown changes from a8de1ab000
2022-03-22 14:12:03 +02:00
Aliaksandr Valialkin
84b3234f3d
docs/CHANGELOG.md: document a1e17e91f8
2022-03-21 15:37:03 +02:00
Aliaksandr Valialkin
9d91ad7124
app/vmagent/remotewrite: prevent from infinite recursion panic when pushing a time series with big number of samples to remote storage
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2335
2022-03-18 19:07:27 +02:00
Aliaksandr Valialkin
2431c9cf81
lib/promrelabel: add support for conditional relabeling via if filter
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1998
2022-02-24 02:35:13 +02:00
Aliaksandr Valialkin
4a658cf955
app/vmagent: add -influxDBLabel in the same way as for app/vminsert
...
This is a follow-up for 3d19fa6932
2022-02-21 14:09:40 +02:00
Roman Khavronenko
f59685e5ff
docs: make profiling commands more copy-friendly ( #2213 )
...
* docs: make profiling commands more copy-friendly
The change adds `copy text` snippet to code examples
and replaces hostname placeholders with `0.0.0.0`.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-18 12:26:35 +02:00
Aliaksandr Valialkin
7e8596edb8
docs: update -help output for VictoriaMetrics components
2022-02-15 21:02:36 +02:00
Nikolay
48a9e068be
adds release build for macos darwin amd64 and arm64 ( #2185 )
...
* adds release build for macos darwin amd64 and arm64
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1896
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1851
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-02-14 17:42:33 +02:00
Aliaksandr Valialkin
216b13186d
docs/vmagent.md: add unsupported Prometheus config sections chapter
2022-02-12 01:08:34 +02:00
Aliaksandr Valialkin
eed66b6640
lib/promscrape: set -promscrape.config.strictParse to true by default
...
This allows detecting long-living silent errors in -promscrape.config
2022-02-08 15:42:33 +02:00
Aliaksandr Valialkin
678b3e71db
lib/promscrape: provide the ability to fetch target responses on behalf of vmagent or single-node VictoriaMetrics
...
This feature may be useful when debugging metrics for the given target located in isolated environment
2022-02-03 19:02:12 +02:00
Aliaksandr Valialkin
fa89f3e5a5
lib/promscrape: do not send staleness markers on graceful shutdown
...
This follows Prometheus behavior.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2013#issuecomment-1006994079
2022-01-07 01:19:06 +02:00
Denys Holius
3a522c591a
Old links replaced for newest ( #2033 )
...
* replaced old links to the website
* fixed deletion main README.md file
* fix: added docs files after docs-sync
2022-01-05 16:32:54 +02:00
Yurii Kravets
f6e2e0dbd1
Update README.md ( #1996 )
...
* Update README.md
go 1.16 -> 1.17
* Update README.md
* Update README.md
* Update Cluster-VictoriaMetrics.md
* Update Single-server-VictoriaMetrics.md
* Update vmauth.md
* Update vmbackup.md
* Update vmrestore.md
* Update vmagent.md
* Update vmctl.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
2021-12-23 12:11:30 +02:00
Aliaksandr Valialkin
727797a6fd
all: use logger.WithThrottler() where appropriate
2021-12-21 17:10:54 +02:00
Roman Khavronenko
23e1de06ee
vmagent: add error log for skipped data block when rejected by receiv… ( #1956 )
...
* vmagent: add error log for skipped data block when rejected by receiving side
Previously, rejected data blocks were silently dropped - only metrics were update.
From operational perspective, having an additional logging for such cases is preferable.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1911
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* vmagent: throttle log messages about skipped blocks
The new type of logger was added to logger pacakge.
This new type supposed to control number of logged messages
by time.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* lib/logger: make LogThrottler public, so its methods can be inspected by external packages
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-12-21 16:42:38 +02:00
Aliaksandr Valialkin
053e85ff3d
all: typo fix: unexected -> unexpected
2021-12-20 17:40:13 +02:00
Aliaksandr Valialkin
d40441947a
app: allow specifying http and https urls in the following command-line flags
...
* -promscrape.config
* -relabelConfig
* -remoteWrite.relabelConfig
* -remoteWrite.urlRelabelConfig
2021-12-03 00:11:47 +02:00
Aliaksandr Valialkin
c540235470
app: use relative paths instead of absolute paths for the supported http handlers on the main page
...
This allows hiding VictoriaMetrics components behind proxies, which serve pages at different path prefixes
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1858
2021-12-02 13:54:15 +02:00
Aliaksandr Valialkin
4fb19fe34b
all: consistently return application/json content-type without charset=utf-8
...
The `application/json` content-type has utf-8 encoding by default.
See https://stackoverflow.com/questions/9254891/what-does-content-type-application-json-charset-utf-8-really-mean
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897
2021-11-09 18:07:22 +02:00
Aliaksandr Valialkin
57aaf914ac
docs: mention that graphs on the official dashboards contain useful hints
2021-11-08 19:54:25 +02:00
Aliaksandr Valialkin
e702b7a92f
app/vmagent: allow bigger number of in-memory blocks for big values of -remoteWrite.queues
...
This should improve the maximum data ingestion speed for highly-loaded vmagent instances
which run on beefy servers with many CPU cores and big amounts of RAM
2021-11-05 15:16:50 +02:00
Aliaksandr Valialkin
847004fa77
app/{vminsert,vmagent}: hide passwords and auth tokens by default at /config page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764
2021-11-05 14:42:13 +02:00
Aliaksandr Valialkin
9edc12fa99
app/vmagent/remotewrite: fix parallel data sending to remote storage systems at e0d2ba5608
2021-11-04 16:58:53 +02:00
Aliaksandr Valialkin
63c29099ba
app/vmagent: add -remoteWrite.maxRowsPerBlock command-line option, which may be used for improving data ingestion performance under high load
2021-11-04 15:39:55 +02:00
Aliaksandr Valialkin
5f13ba8631
app/vmagent/remotewrite: send data to remote storage systems in parallel
...
This should improve data ingestion speed many `-remoteWrite.url` options are configured
2021-11-04 15:04:02 +02:00
Aliaksandr Valialkin
d445d22c0c
lib/promscrape: add -promscrape.maxResponseHeadersSize command-line flag for tuning the maximum http response headers size from Prometheus scrape targets
2021-11-03 22:27:55 +02:00
Aliaksandr Valialkin
551a94ba7f
app/{vmagent,vminsert}: add ability to restrict access to /config page with authKey query arg
...
The authKey can be configured via `-configAuthKey` command-line flag.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764
2021-11-01 16:47:30 +02:00
Aliaksandr Valialkin
ad445a06cd
lib/promscrape: properly show proxy_url option value at /config page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755
2021-10-26 21:24:22 +03:00
Aliaksandr Valialkin
a8bcc3c276
lib/promscrape: reduce memory usage if -promscrape.noStaleMarkers command-line flag is passed
...
Do not store in memory the response from the last scrape per each target if -promscrape.noStaleMarkers option is enabled.
This should reduce memory usage when the scraped targets return large responses.
2021-10-22 13:22:08 +03:00
Aliaksandr Valialkin
5705f4b6d1
lib/httpserver: expose command-line flags at /flags page
...
This should simplify debugging.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695
2021-10-20 00:46:54 +03:00
Aliaksandr Valialkin
0452a8d4e8
lib/promscrape: store the full response in stream parsing mode in scrapeWork.lastScrape byte slice
...
This allows sending staleness marks and properly calculate scrape_series_added metric in stream parsing mode
at the cost of the increased memory usage, since now the potentially big response is kept
in the lastScrape byte slice per each scrapeWork.
In practice the memory usage increase shouldn't be big, since the response size
is usually much smaller than the parsed metrics from this response after the relabeling,
which usually adds a big pile of target-specific labels per each metric.
2021-10-15 15:26:24 +03:00
Aliaksandr Valialkin
25421fa2ae
lib/promscrape: add -promscrape.minResponseSizeForStreamParse command-line option for automatic switching to stream parsing mode when scraping targets with big responses
...
This should reduce memory usage when vmagent scrapes targets with non-uniform response sizes.
This is common case in Kubernetes monitoring.
2021-10-14 12:30:55 +03:00
Aliaksandr Valialkin
bee130cc78
lib/promscrape: return error if sample_limit or series_limit options are set when stream parsing mode is enabled
2021-10-14 12:30:54 +03:00
Aliaksandr Valialkin
aeedfe2fe2
app/vmagent: expose -promscrape.config contents at /config page as Prometheus does
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695
2021-10-12 16:27:37 +03:00
