Commit Graph

737 Commits

Author SHA1 Message Date
hagen1778
95edeffbc6
docs: add link to sandbox to the Grafana section
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-12-22 16:42:33 +01:00
Aliaksandr Valialkin
fb90a56de2
app/{vminsert,vmagent}: preliminary support for /api/v2/series ingestion from new versions of DataDog Agent
This commit adds only JSON support - https://docs.datadoghq.com/api/latest/metrics/#submit-metrics ,
while recent versions of DataDog Agent send data to /api/v2/series in undocumented Protobuf format.
The support for this format will be added later.

Thanks to @AndrewChubatiuk for the initial implementation at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5094

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4451
2023-12-21 20:50:55 +02:00
Dmytro Kozlov
df012f1553
docs: remove default value from the maxConcurrentInserts flag (#5494) 2023-12-19 15:57:33 +01:00
Aliaksandr Valialkin
326a77c697
all: add -metrics.exposeMetadata command-line flag, which can be used for adding TYPE and HELP metadata for metrics exposed at /metrics page
This may be needed for systems, which require this metadata such as Google Cloud Managed Prometheus.
See https://cloud.google.com/stackdriver/docs/managed-prometheus/troubleshooting#missing-metric-type
2023-12-19 03:20:40 +02:00
Aliaksandr Valialkin
4b529562ce
lib/pushmetrics: add -pushmetrics.header and -pushmetrics.disableCompression command-line flags 2023-12-17 19:56:46 +02:00
Aliaksandr Valialkin
6f15ca4a16
docs/Single-server-VictoriaMetrics.md: cross-link HA docs with the corresponding chapter at vmauth docs 2023-12-13 23:06:46 +02:00
Yury Molodov
1a5cdb4790
vmui: autocomplete usability improvements (#5422)
* vmui: add show quick tip for autocomplete

* vmui: auto-completion usability improvements #5348

* vmui: add const for min symbols in autocomplete

* Use proper queries to VictoriaMetrics

* vmui: fix comments for autocomplete

* app/vmselect: run `make vmui-update`

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-12-13 00:32:41 +02:00
Aliaksandr Valialkin
a5bc9d93cc
docs: sync -help output for VictoriaMetrics components after recent changes 2023-12-10 01:13:41 +02:00
Aliaksandr Valialkin
3d3b0e31e0
app/vmselect: add -search.maxResponseSeries command-line flag for limiting the number of time series a single response can return
This limit can be used for preventing from high memory usage at Grafana when the response returns too many series.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5372
2023-12-10 00:54:42 +02:00
Aliaksandr Valialkin
adc69b872c
docs: run make docs-images-to-webp after 02a0a7f428
This reduces added image sizes by 3x

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5437
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5370
2023-12-07 16:40:37 +02:00
Alexander Marshalov
02a0a7f428
added field version to the response for /api/v1/status/buildinfo API for using more efficient API in Grafana for receiving label values, added additional info about setup Grafana datasource (#5370) (#5437) 2023-12-07 16:37:36 +02:00
Artem Navoiev
17e2b4f814
fix anchor in docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-12-05 13:10:43 +01:00
Aliaksandr Valialkin
06c73df55a
Revert "add datadog /api/v2/series and /api/beta/sketches support (#5094)"
This reverts commit 543f218fe9.

Reason for revert: https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5094#issuecomment-1839789080
2023-12-05 02:26:22 +02:00
Andrii Chubatiuk
48228031e4
docs: sync mistakenly deleted docs from 543f218fe9
543f218fe9
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-28 16:46:08 +01:00
hagen1778
8a0bb4bf17
docs: mention loadbalancer in Monitoring chapter
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-28 11:14:17 +01:00
hagen1778
f42ec79958
docs: fix link for cache reset on vmselects
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-28 09:29:10 +01:00
Aliaksandr Valialkin
2cd9cda12c
docs: make more visible that the maximum JSON line length, which is accepted by /api/v1/import, is limited by -import.maxLineLen command-line flag value
This is a follow-up for 0cf55ded34

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5364
2023-11-24 13:12:51 +02:00
Roman Khavronenko
0cf55ded34
lib/protoparser: decrease import.maxLineLen from 100MB to 10MB (#5364)
Tests showed that importing a single line with 70MB size takes 5.3GiB
RSS memory for VictoriaMetrics single-node.
In the scenario when user exports and imports data from one VM to another,
it could possibly lead to OOM exception for destination VM.

Importing a single line with 16MB size taks 1.3GiB RSS memory.
Hence, the limit for `import.maxLineLen` was decreased from 100MB to 10MB
to improve reliability of VictoriaMetrics during imports.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-11-24 12:53:04 +02:00
Aliaksandr Valialkin
334a739ff6
docs: convert png images to webp in all the docs except of docs/operator/*
This reduces the size of docs/* folder from 33MB to 18MB

Images inside docs/operator/* must be converted at the https://github.com/VictoriaMetrics/operator/tree/master/docs
and then the updated images must be automatically propagated to the docs/operator/*

This is a follow-up for d3f919df3e

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5206
2023-11-22 19:21:00 +02:00
hagen1778
3ffa8975d4
docs: follow-up after d3f919df3e
d3f919df3e
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-20 11:52:09 +01:00
Dmytro Kozlov
d3f919df3e
docs/managed-victoriametrics: use webp format to reduce image size (#5206) 2023-11-20 11:28:47 +01:00
Aliaksandr Valialkin
7d4873bcef
docs: mention that VictoriaMetrics and vmagent support data ingestion via New Relic protocol now
This is a follow-up for f60c08a7bd
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3520
2023-11-15 22:54:29 +01:00
Artem Navoiev
5d61a7327d
docs: update Grafana setup section, use more direct link and add noti… (#5287) 2023-11-14 14:34:01 +01:00
Aliaksandr Valialkin
230230cf0b
lib/logger: add -loggerMaxArgLen command-line flag for fine-tuning the maximum length of logged args 2023-11-11 12:30:08 +01:00
Aliaksandr Valialkin
010dc15d16
lib/blockcache: do not cache entries, which were attempted to be accessed 1 or 2 times
Previously entries which were accessed only 1 time weren't cached.
It has been appeared that some rarely executed heavy queries may read indexdb block twice
in a row instead of once. There is no need in caching such a block then.
This change should eliminate cache size spikes for indexdb/dataBlocks when such heavy queries are executed.

Expose -blockcache.missesBeforeCaching command-line flag, which can be used for fine-tuning
the number of cache misses needed before storing the block in the caching.
2023-11-10 22:28:03 +01:00
Aliaksandr Valialkin
a9a26c20b5
docs/Single-server-VictoriaMetrics.md: make High availability section more clear 2023-11-10 20:58:32 +01:00
PhracturedBlue
2474281f1b
Support building images via podman (#4978) 2023-11-09 00:50:21 -08:00
Yury Molodov
f90d2ec843
vmui: display query error on Explore metrics page (#5272)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5202
2023-11-03 16:23:19 +01:00
Aliaksandr Valialkin
815fda8995
docs: update -help output after recent changes to VictoriaMetrics components 2023-11-02 20:27:10 +01:00
Aliaksandr Valialkin
65db6609eb
docs/CHANGELOG.md: update the description of the optimization for SLO/SLI-like queries according to latest changes
See commits 4497a08e3d and 92826b0b4a
2023-11-02 20:05:05 +01:00
Aliaksandr Valialkin
536c1301fd
docs/Single-server-VictoriaMetrics.md: document why data inside <-storageDataPath>/snapshots directory should be manipulated only via snapshot API 2023-11-02 08:30:39 +01:00
Aliaksandr Valialkin
a71efce784
docs/Single-server-VictoriaMetrics.md: cosmetic fixes after 23369321f1 2023-10-31 16:42:29 +01:00
Aliaksandr Valialkin
4ac95b6f49
docs/CHANGELOG.md: move the description for -http.header.* command-line flags from SECURITY to FEATURE
The SECURITY label should be applied only to changes, which fix security issues.
The change at ad839aa492 adds new command-line flags, which can be used
for improving security in some cases. They do not fix any security issues.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5111
2023-10-31 16:23:08 +01:00
Roman Khavronenko
a950873fff
app/vmselect: expose vm_memory_intensive_queries_total counter metric (#5208)
The new metric gets increased each time `-search.logQueryMemoryUsage` memory limit
is exceeded by a query. This metric should help to identify expensive and heavy queries
without inspecting the logs.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-31 13:31:09 +01:00
Roman Khavronenko
23369321f1
docs: mention information loss when downsampling gauges (#5204)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-30 15:29:06 +01:00
Dima Lazerka
ad839aa492
lib/httpserver: add flags to specify HSTS / Frame-Options / CSP headers for httpserver (#5111)
support `Strict-Transport-Security`, `Content-Security-Policy` and `X-Frame-Options`
HTTP headers in all VictoriaMetrics components. 
The values for headers can be specified by users via the following flags: 
`-http.header.hsts`, `-http.header.csp` and `-http.header.frameOptions`.

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-10-30 11:33:38 +01:00
Aliaksandr Valialkin
2f21c0c119
docs: use https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest instead of https://github.com/VictoriaMetrics/VictoriaMetrics/releases link where needed
The https://github.com/VictoriaMetrics/VictoriaMetrics/releases link may show non-latest
releases at the top, such as LTS releases or VictoriaLogs releases.
So it is better to use https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest link,
which always redirect to the latest available release of VictoriaMetrics.
2023-10-18 20:06:25 +02:00
Roman Khavronenko
b8b6e120ff
app/vmselect: limit the number of parallel workers by 32 (#5195)
* app/vmselect: limit the number of parallel workers by 32

The change should improve performance and memory usage during query processing
on machines with big number of CPU cores. The number of parallel workers for
query processing is controlled via `-search.maxWorkersPerQuery` command-line flag.
By default, the number of workers is limited by the number of available CPU cores,
but not more than 32. The limit can be increased via `-search.maxWorkersPerQuery`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

- The `-search.maxWorkersPerQuery` command-line flag doesn't limit resource usage,
  so move it from the `resource usage limits` to `troubleshooting` chapter at docs/Single-server-VictoriaMetrics.md

- Make more clear the description for the `-search.maxWorkersPerQuery` command-line flag

- Add the description of `-search.maxWorkersPerQuery` to docs/Cluster-VictoriaMetrics.md

- Limit the maximum value, which can be passed to `-search.maxWorkersPerQuery`, to GOMAXPROCS,
  because bigger values may worsen query performance and increase CPU usage

- Improve the the description of the change at docs/CHANGELOG.md. Mark it as FEATURE instead of BUGFIX,
  since it is closer to a feature than to a bugfix.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5087

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-10-18 19:51:37 +02:00
Aliaksandr Valialkin
b8c267075e
lib/promscrape: add a link to https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets in descriptions for -promscrape.cluster.* command-line flags
This should help users figuring out the purpose of -promscrape.cluster.* command-line flags
2023-10-16 14:46:22 +02:00
Aliaksandr Valialkin
2c334ed953
app/{vmagent,vminsert}: follow-up for NewRelic data ingestion protocol support
This is a follow-up for f60c08a7bd

Changes:

- Make sure all the urls related to NewRelic protocol start from /newrelic . Previously some urls were started from /api/v1/newrelic

- Remove /api/v1 part from NewRelic urls, since it has no sense

- Remove automatic transformation from CamelCase to snake_case for NewRelic labels and metric names,
  since it may complicate the transition from NewRelic to VictoriaMetrics. Preserve all the metric names and label names,
  so users could query metrics and labels by the same names which are used in NewRelic.
  The automatic transformation from CamelCase to snake_case can be added later as a special action for relabeling rules if needed.

- Properly update per-tenant data ingestion stats at app/vmagent/newrelic/request_handler.go . Previously it was always zero.

- Fix NewRelic urls in vmagent when multitenant data ingestion is enabled. Previously they were mistakenly started from `/`.

- Document NewRelic data ingestion url at docs/Cluster-VictoriaMetrics.md

- Remove superflouos memory allocations at lib/protoparser/newrelic

- Improve tests at lib/protoparser/newrelic/*

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3520
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4712
2023-10-16 00:25:25 +02:00
Aliaksandr Valialkin
ddbe713470
docs/Single-server-VictoriaMetrics.md: add a link to the original issue, which describes how to run VictoriaMetrics as Windows service
This is a follow-up for cc7d5b7bab

The original issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3781 contains
up-to-date information on how to run VictoriaMetrics components as Windows service,
plus it may contain additional information about this case such as https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3781#issuecomment-1708092680 ,
so it is better to refer this issue from the docs.
2023-10-15 19:33:39 +02:00
Artem Navoiev
8dff1c696f
docs fix broken links
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-14 15:39:49 +02:00
Artem Navoiev
1ca0b86397
docs remove (( from the link to fix them
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-14 00:14:41 +02:00
Aliaksandr Valialkin
3f5a41e35e
lib/license: cleanups and prettifications for log messages and docs related to licensing
- Make more clear the docs at docs/enterprise.md, so readers could figure out faster
  on how to obtain enterprise key and how to pass it to VictoriaMetrics Enterprise components.

- Fix examples at docs/enterprise.md, which were referring to non-existing `-license-file` command-line flag.
  The `-licenseFile` command-line flag must be used instead.

- Improve the description of `-license*` command-line flags, so users could understand
  faster how to use them.

- Improve the warning message, which is emitted when the deprecated -eula command-line flag is passed,
  so the user could figure out how to switch faster to -license* command-line flags.

- Disallow running VictoriaMetrics components with both -license and -licenseFile command-line flags.

- Disallow running VictoriaMetrics components when -licensFile points to an empty file.

- Consistently use the phrase "This flag is available only in Enterprise binaries" across
  all the enterprise-specific command-line flags.

- Remove unneeded level of indirection for `noLicenseMessage` and `expiredMessage` string contants
  in order to improve code readability and maintainability.

- Remove unneded `return` statements after `logger.Fatalf()` calls, since these calls exit the app and never return.

- Make sure that the info log message about successful license verification is emitted
  when the license is verified successfully. Previously the error message could be logged
  when the license payload is invalid or if it misses some required features.
2023-10-13 14:56:09 +02:00
Artem Navoiev
36b0cea3f3
docs: mention the free trial for enterprise
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 636d4ea196)
2023-10-13 14:31:37 +02:00
Aliaksandr Valialkin
964e6ccc17
docs/Single-server-VictoriaMetrics.md: remove question mark added by accident in the commit 3d5d62e38a 2023-10-12 20:01:18 +02:00
hagen1778
b57756734e
docs: follow-up after 636d4ea196
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-10 11:25:23 +02:00
Artem Navoiev
b3cc22b159 docs: update the license flags description
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-05 10:41:30 -07:00
Dmytro Kozlov
f60c08a7bd
app/(vminsert|vmagent): add support for new relic infrastructure agent (#4712)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-10-05 14:39:51 +02:00
Zakhar Bessarab
cc7d5b7bab
docs: add example service configuration for windows (#5129)
Use service configuration example from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3781#issuecomment-1424671624

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-10-05 13:55:27 +02:00