Commit Graph

2931 Commits

Author SHA1 Message Date
Haleygo
130e0ea5f0
vmalert-tool: implement unittest (#4789)
1. split package rule under /app/vmalert, expose needed objects
2. add vmalert-tool with unittest subcmd

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2945
2023-10-16 14:12:06 +02:00
Aliaksandr Valialkin
d6f585ec0f
docs/vmbackup.md: clarify documentation about -deleteAllObjectVersions command-line flag
Updates 2fc7e9f47e
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5121
2023-10-16 14:02:37 +02:00
Aliaksandr Valialkin
4278b00a66
app/vmselect/promql: do not use unsafe conversion from bytes slice to string when storing a value by map key
The assigned map key shouldn't change over time, otherwise the map won't work properly.

This is a follow-up for 1f91f22b5f
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5087
2023-10-16 13:55:40 +02:00
Aliaksandr Valialkin
815e9bf892
app/{vmagent,vminsert}: follow-up for NewRelic data ingestion protocol support
This is a follow-up for f60c08a7bd

Changes:

- Make sure all the urls related to NewRelic protocol start from /newrelic . Previously some urls were started from /api/v1/newrelic

- Remove /api/v1 part from NewRelic urls, since it has no sense

- Remove automatic transformation from CamelCase to snake_case for NewRelic labels and metric names,
  since it may complicate the transition from NewRelic to VictoriaMetrics. Preserve all the metric names and label names,
  so users could query metrics and labels by the same names which are used in NewRelic.
  The automatic transformation from CamelCase to snake_case can be added later as a special action for relabeling rules if needed.

- Properly update per-tenant data ingestion stats at app/vmagent/newrelic/request_handler.go . Previously it was always zero.

- Fix NewRelic urls in vmagent when multitenant data ingestion is enabled. Previously they were mistakenly started from `/`.

- Document NewRelic data ingestion url at docs/Cluster-VictoriaMetrics.md

- Remove superflouos memory allocations at lib/protoparser/newrelic

- Improve tests at lib/protoparser/newrelic/*

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3520
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4712
2023-10-16 13:55:04 +02:00
hagen1778
709a2bad66
docs: remove extra / in the end of the link
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-16 13:24:30 +02:00
Aliaksandr Valialkin
dfc67aedcb
lib/license: cleanups and prettifications for log messages and docs related to licensing
- Make more clear the docs at docs/enterprise.md, so readers could figure out faster
  on how to obtain enterprise key and how to pass it to VictoriaMetrics Enterprise components.

- Fix examples at docs/enterprise.md, which were referring to non-existing `-license-file` command-line flag.
  The `-licenseFile` command-line flag must be used instead.

- Improve the description of `-license*` command-line flags, so users could understand
  faster how to use them.

- Improve the warning message, which is emitted when the deprecated -eula command-line flag is passed,
  so the user could figure out how to switch faster to -license* command-line flags.

- Disallow running VictoriaMetrics components with both -license and -licenseFile command-line flags.

- Disallow running VictoriaMetrics components when -licensFile points to an empty file.

- Consistently use the phrase "This flag is available only in Enterprise binaries" across
  all the enterprise-specific command-line flags.

- Remove unneeded level of indirection for `noLicenseMessage` and `expiredMessage` string contants
  in order to improve code readability and maintainability.

- Remove unneded `return` statements after `logger.Fatalf()` calls, since these calls exit the app and never return.

- Make sure that the info log message about successful license verification is emitted
  when the license is verified successfully. Previously the error message could be logged
  when the license payload is invalid or if it misses some required features.
2023-10-16 12:51:37 +02:00
Aliaksandr Valialkin
b86bec8109
app/vmui: small UX enhancements
- Reduce vertical space usage, so more information is available on the screen without the need to scroll.
- Show information for lines with higher values at the top of the legend under the graph.
  This should simplify graph analysis when it contains many lines.
2023-10-16 12:39:32 +02:00
Aliaksandr Valialkin
ca35e0d66a
deployment/docker: update Go builder from Go1.21.1 to Go1.21.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.21.2+label%3ACherryPickApproved
and https://github.com/golang/go/issues?q=milestone%3AGo1.21.3+label%3ACherryPickApproved
2023-10-16 12:38:14 +02:00
Aliaksandr Valialkin
348c1bcec0
app/{vmselect,vlselect}: enable caching of static contents from /vmui/static/ folder at client side
This should improve repated VMUI page load times on slow networks

See https://developer.chrome.com/docs/lighthouse/performance/uses-long-cache-ttl/
2023-10-16 12:36:34 +02:00
Zakhar Bessarab
a353fb7e00
docs/vmbackupmaanger: reference permanent objects deletion (#5157)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-10-16 12:14:43 +02:00
Zakhar Bessarab
8b42a1733c
lib/backup: add -deleteAllObjectVersions command-line flag (#5147)
New flag enforces removal of all versions of the object in remote object storage.

See:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5121
- https://docs.victoriametrics.com/vmbackup.html#permanent-deletion-of-objects-in-s3-compatible-storages

(cherry picked from commit 2fc7e9f47e)
2023-10-10 14:14:21 +02:00
Yury Molodov
65a9f3da03
vmui: transfer Top Queries time interval #5097 (#5145)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5097
(cherry picked from commit 6dc5306c9b)
2023-10-10 14:14:20 +02:00
Nikolay
4a50e9400c
app/vmselect: reduce lock contention for heavy aggregation requests (#5119)
reduce lock contention for heavy aggregation requests
previously lock contetion may happen on machine with big number of CPU due to enabled string interning. sync.Map was a choke point for all aggregation requests.
Now instead of interning, new string is created. It may increase CPU and memory usage for some cases.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5087
2023-10-10 13:44:02 +02:00
Haleygo
b52f1d1f0a
vmalert: add evalAlignment for rule group and fix evalutaion timstamp (#5066)
* vmalert: add `query_time_alignment` for rule group

1. add `eval_alignment` attribute for group which by default is true. So group rule query stamp will be aligned with interval and propagated to ALERT metrics and the messages for alertmanager;
2. deprecate `datasource.queryTimeAlignment` flag.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5049
(cherry picked from commit 2aa0f5fc41)
2023-10-10 12:45:37 +02:00
Dmytro Kozlov
1cc6cd3d4f
app/vmalert: hide sensetive info in the vmalert (#5059)
Strip sensitive information such as auth headers or passwords from datasource, remote-read,
remote-write or notifier URLs in log messages or UI. This behavior is by default and is controlled via
`-datasource.showURL`, `-remoteRead.showURL`, `remoteWrite.showURL` or `-notifier.showURL` cmd-line flags.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5044
(cherry picked from commit 244c887825)
2023-10-10 12:45:36 +02:00
hagen1778
a906837403
docs: follow-up after 636d4ea196
Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit b57756734e)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-10 12:45:36 +02:00
Yury Molodov
29487700d9
vmui: enhancement of autocomplete feature (#5051)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4993
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3006
(cherry picked from commit c5044cdba9)
2023-10-10 12:45:08 +02:00
Artem Navoiev
65b2a0ce60
docs: update the license flags description
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

(cherry picked from commit b3cc22b159)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-10 12:45:06 +02:00
hagen1778
a41c2aeafb
app/(vminsert|vmagent): fix label names for newrelic metrics
Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 7cf615a73b)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-05 15:08:07 +02:00
Dmytro Kozlov
a89cf4ce12
app/(vminsert|vmagent): add support for new relic infrastructure agent (#4712)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit f60c08a7bd)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-05 15:06:05 +02:00
Aliaksandr Valialkin
e275be05b0
docs/vmbackup.md: mention that -filestream.disableFadvise command-line flag can be used for reducing CPU usage on systems with big number of CPU cores
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5120

This is a follow-up for 75dd7b30ba
2023-10-04 16:38:53 +02:00
Zakhar Bessarab
dbf0aa4b87
docs/vmbackupmanager: add missing step for restore flow in k8s (#5116)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-10-04 16:38:53 +02:00
Aliaksandr Valialkin
4e5d3d38f8
lib/filestream: add -filestream.disableFadvise syscall for unconditional disabling of fadvise syscall
This may be needed in rare cases when performing backups on systems with big number of CPU cores
and big value passed to -concurrency command-line flag.

See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5120
2023-10-04 16:30:19 +02:00
Aliaksandr Valialkin
b5812e2457
app/{vmselect,vlselect}: run make vmui-update vmui-logs-update 2023-10-02 21:44:21 +02:00
Yury Molodov
f5521ce4d4
vmui: add storage for query history (#5022)
* vmui: add storage for query history

* docs/vmui: add storage for query history
2023-10-02 21:41:32 +02:00
Roman Khavronenko
1f2cb594d9
lib/promscrape: make concurrency control optional (#5073)
* lib/promscrape: make concurrency control optional

Before, `-maxConcurrentInserts` was limiting all calls to `promscrape.Parse`
function: during ingestion and scraping. This behavior is incorrect.
Cmd-line flag `-maxConcurrentInserts` should have effect onl on ingestion.

Since both pipelines use the same `promscrape.Parse` function, we extend it
to make concurrency limiter optional. So caller can decide whether concurrency
should be limited or not.

This commit makes c53b5788b4
obsolete.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Revert "dashboards: move `Concurrent inserts` panel to Troubleshooting section"

This reverts commit c53b5788b4.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-02 21:34:41 +02:00
Yury Molodov
40973d37da
vmui: improve the appearance of the trace (#5091) 2023-10-02 21:24:51 +02:00
Dmytro Kozlov
d8d003af24
app/vmagent: fix check of the DataDog agent path requests when requests have trailing slashes (#5106)
* app/vmagent: fix check of the DataDog agent path requests when requests have trailing slashes

* app/vmagent: fix CHANGELOG.md description

* wip

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-10-02 21:18:47 +02:00
Aliaksandr Valialkin
5fd79f47f1
app/vmselect/promql: follow-up for 896c85a4a4
- Clarify the description of the change at docs/CHANGELOG.md
- Make sure that bitmap_*(X, NaN) returns NaN

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4996
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5021
2023-10-02 21:07:46 +02:00
Aliaksandr Valialkin
120f3bc467
lib/logstorage: follow-up for 8a23d08c21
- Compare the actual free disk space to the value provided via -storage.minFreeDiskSpaceBytes
  directly inside the Storage.IsReadOnly(). This should work fast in most cases.
  This simplifies the logic at lib/storage.

- Do not take into account -storage.minFreeDiskSpaceBytes during background merges, since
  it results in uncontrolled growth of small parts when the free disk space approaches -storage.minFreeDiskSpaceBytes.
  The background merge logic uses another mechanism for determining whether there is enough
  disk space for the merge - it reserves the needed disk space before the merge
  and releases it after the merge. This prevents from out of disk space errors during background merge.

- Properly handle corner cases for flushing in-memory data to disk when the storage
  enters read-only mode. This is better than losing the in-memory data.

- Return back Storage.MustAddRows() instead of Storage.AddRows(),
  since the only case when AddRows() can return error is when the storage is in read-only mode.
  This case must be handled by the caller by calling Storage.IsReadOnly()
  before adding rows to the storage.
  This simplifies the code a bit, since the caller of Storage.MustAddRows() shouldn't handle
  errors returned by Storage.AddRows().

- Properly store parsed logs to Storage if parts of the request contain invalid log lines.
  Previously the parsed logs could be lost in this case.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4737
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4945
2023-10-02 20:38:00 +02:00
hagen1778
25a006099d
app/vlinsert/loki: make fmt
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-02 20:35:45 +02:00
Aliaksandr Valialkin
5f1492d978
docs/vmalert.md: refer to -evaluationInterval command-line flag instead of evaluation_interval option, which isnt supported by vmalert
This is follow-up for 5c42c1218a
2023-10-02 20:32:02 +02:00
Dmytro Kozlov
90b189dab8
app/vmselect: fix bitmap_*() functions behavior (#5021)
Related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4996

Signed-off-by: dmitryk-dk d.kozlov@victoriametrics.com

Signed-off-by: dmitryk-dk d.kozlov@victoriametrics.com
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-10-02 20:13:27 +02:00
Zakhar Bessarab
dfdada055c
lib/logstorage: switch to read-only mode when running out of disk space (#4945)
* lib/logstorage: switch to read-only mode when running out of disk space

Added support of `--storage.minFreeDiskSpaceBytes` command-line flag to allow graceful handling of running out of disk space at `--storageDataPath`.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4737
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/logstorage: fix error handling logic during merge

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/logstorage: fix log level

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-10-02 17:09:57 +02:00
Dmytro Kozlov
10371eac60
vmui: update information about tsdb usage in cluster version (#5004)
* vmui: update information about tsdb usage in cluster version

* vmui: cleanup

* vmui: add CHANGELOG.md

* vmui: cleanup

* vmui: update logic, move information to the visible place

* app/vmui: remove values fetch, update documentation for cardinality explorer

* app/vmui: update CHANGELOG.md
2023-10-01 21:30:44 +02:00
Roman Khavronenko
d39c8525e2
Docker add vmauth (#5057)
* docker-compose: add vmauth to cluster env

vmauth acts as a balancer and used as an example of how to interconnect
VM components via vmauth.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docker-compose: add vmauth to cluster env

vmauth acts as a balancer and used as an example of how to interconnect
VM components via vmauth.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-10-01 21:24:01 +02:00
Aliaksandr Valialkin
15645c8a94
app/vmagent: follow-up for cfef814750
- Properly handle /insert/multitenant/api/put url for opentsdb handler at vmagent
- Document that the bug has been introduced in v1.93.2 at docs/CHANGELOG.md
- Add a link to multitenant url docs in bugfix description

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5061
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4910
2023-10-01 20:52:29 +02:00
Alexander Marshalov
cfef814750
fixed ingestion via multitenant url for opentsdbhttp (#5061) (#5063) 2023-09-26 10:47:49 +02:00
Aliaksandr Valialkin
e341128096
docs/vmagent.md: make VictoriaMetrics remove_write protocol more visible by mentioning it at the top of the page 2023-09-25 17:42:25 +02:00
Aliaksandr Valialkin
a421db5977
lib/storage: stop exposing vm_merge_need_free_disk_space metric
This metric confuses users and has no any useful information.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686#issuecomment-1733844128
2023-09-25 17:00:14 +02:00
Aliaksandr Valialkin
538dc6058d
app/vmselect/promql: run make fmt after 3b9605dba5 2023-09-25 16:15:58 +02:00
Aliaksandr Valialkin
b43ff80d21
app/vmselect/promql: do not sort q1 or q2 results
This makes sure that `q2` series are returned after `q1` series in the same way as Prometheus does

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763
2023-09-25 16:15:02 +02:00
Aliaksandr Valialkin
c954019e43
app/vmselect/promql: completely substitute median_over_time() WITH template with regular median_over_time() rollup function
This is a follow-up for 34d7a670d0

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5034
2023-09-25 15:31:25 +02:00
Zakhar Bessarab
fd6ca57c14
app/vmselect/promql: add implementation of median_over_time for rollup functions list (#5042)
`median_over_time` is handled by predefined WITH template in MetricsQL library which translates it to `quantile_over_time(0.5)`
This makes it impossble to use `median_over_time` as a usual rollup function for `aggr_over_time`.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5034

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-25 15:31:25 +02:00
Zakhar Bessarab
86eaf6906b
docs/vmbackup: update docs for different authentication options, add examples (#5046)
Updates: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5023

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-22 11:42:34 +02:00
Aliaksandr Valialkin
11ebcf1f9b
app/vmauth: consistently use '%w' for formatting errors in fmt.Errorf() 2023-09-21 11:05:26 +02:00
Roman Khavronenko
c9f121e694
app/vmauth: update config reload routine (#5019)
* expose metrics `vmauth_config_last_reload_*` for tracking the state of config reloads, similarly to vmagent/vmalert components.
* do not print logs like `SIGHUP received...` once per configured `-configCheckInterval` cmd-line flag. This log will be printed only if config reload was invoked manually.
*  prevent configuration reloading if there were no changes in config. This improves memory usage when `-configCheckInterval` cmd-line flag is configured and config has extensive list of regexp expressions requiring additional memory on parsing.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-09-21 11:05:26 +02:00
Aliaksandr Valialkin
cc8f2bee0d
app/vlinsert: follow-up for d570763c91
- Switch from summary to histogram for vl_http_request_duration_seconds metric.
  This allows calculating request duration quantiles across multiple hosts
  via histogram_quantile(0.99, sum(vl_http_request_duration_seconds_bucket) by (vmrange)).
- Take into account only successfully processed data ingestion requests
  when updating vl_http_request_duration_seconds histogram.
  Failed requests are ignored, since they may significantly skew measurements.
- Clarify the description of the change at docs/VictoriaLogs/CHANGELOG.md.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4934
2023-09-19 00:45:28 +02:00
Aliaksandr Valialkin
9e50ea6cce
app/vlinsert/insertutils: cosmetic changes after 8d3e574c31 2023-09-19 00:35:51 +02:00
crossoverJie
fb13887573
app/vlinsert: Add vl_http_request_duration_seconds metrics (#4934) 2023-09-19 00:10:24 +02:00
Konstantin
c1a8a2d54c
app/vmselect: return +Inf as null in graphite render api (#5009)
Signed-off-by: Konstantin Kulikov <k.kulikov2@gmail.com>
2023-09-18 16:41:39 +02:00
Zakhar Bessarab
b0c8f7f718
app/vlinsert: add flag to limit amount of fields per line (#4976)
Adding limit on ingestion allows to avoid issues like this one https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762
Such issues are often caused by misconfigurtion on log persing/ingestion side and preventing such rows from being ingested allows to avoid performance implications created by storing such log rows.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-18 12:00:02 +02:00
Dmytro Kozlov
5477b52991
vmagent: add validation of MetricsQL functions (#4991)
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-09-15 13:16:22 +02:00
Aliaksandr Valialkin
9c3a37597c
app/vmselect/netstorage: run make fmt after 58326dbf25 2023-09-10 15:18:15 +02:00
Aliaksandr Valialkin
a5a953fe1e
app/vmagent/remotewrite: fix data race when extra labels are added to samples before sending them to multiple remote storage systems
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4972
2023-09-08 23:26:40 +02:00
Aliaksandr Valialkin
23bb38ef80
app/vmauth: add ability to specify response status codes for retrying requests during load-balancing
Response status codes for retrying can be specified via retry_status_codes list

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893
2023-09-08 23:26:36 +02:00
Zakhar Bessarab
b842a0cd25
docs: sync description for license flags (#4977)
- update eula flag to add deprecation notice
- add new license flags description

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-08 23:26:25 +02:00
Roman Khavronenko
548acce6b3
vmalert: correctly add duplicated params to the query (#4955)
Fix the bug when Group's `params` fields with multiple values were
overriding each other instead of adding up.
The bug was introduced in this commit eccecdf177
 starting from v1.91.1 https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.91.1

 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4908

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6351d07da8)
2023-09-08 09:33:45 +02:00
Aliaksandr Valialkin
3257fcfd8d
app/vmauth: retry requests at other backends on 5xx response status codes
This should allow implementing high availability scheme described at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4792#issuecomment-1674338561

See also https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893
2023-09-08 00:47:57 +02:00
Aliaksandr Valialkin
58326dbf25
app/vmselect: return 503 status code when partial responses are denied and some of vmstorage nodes are temporarily unavailable
This should help detecting this case and automatic retrying the query at healthy cluster replica
in another availability zone.

This commit is needed as a preparation for automatic query retry at another backend at vmauth on 5xx errors
as described at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4792#issuecomment-1674338561
2023-09-07 16:07:06 +02:00
Aliaksandr Valialkin
0bc0d2610c
go.mod: increase the minimum supported Go version from Go1.19 to Go1.20 2023-09-07 12:18:12 +02:00
Aliaksandr Valialkin
677612c6cb
all: update Go builder from Go1.21.0 to Go1.21.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.21.1+label%3ACherryPickApproved
2023-09-07 11:36:49 +02:00
Haleygo
0212219f6c
vmalert: add eval_offset for group (#4693)
Adds `eval_offset` attribute for Groups.
If specified, Group will be evaluated at the exact time offset on the range of [0...evaluationInterval].
The setting might be useful for cron-like rules which must be evaluated at specific moments of time.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3409

Signed-off-by: Haley Wang <pipilong.25@gmail.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 45c0e4bb31)
2023-09-07 10:59:14 +02:00
Aliaksandr Valialkin
4d3c24492c
app/vmselect: run make vmui-update 2023-09-06 10:29:59 +02:00
Yury Molodov
6c96bebc23
vmui: fix render heatmap (#4957) 2023-09-06 10:29:58 +02:00
Artem Navoiev
696f143eb5
use correct abbriviation for ESA legal doc
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-09-05 16:59:47 +02:00
Artem Navoiev
40c795a1e3
change link to the enterprise legal doc
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-09-05 16:58:51 +02:00
Yury Molodov
30db332866
feat: add the option to see the latest queries (#4718) (#4759)
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-09-05 16:56:53 +02:00
Nikolay
7c19d01e9a
app/vminsert: properly close vmstorage connection (#4935)
* app/vminsert: properly close vmstorage connection
previously vmstorage may stuck in broken state until vminsert restarts
since vmstorage was marked as read-only and connection was broken to it.
checkReadonly function never marked connection as broken
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4870

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-09-01 17:56:41 +02:00
Aliaksandr Valialkin
b9b2fbc7cd
app/vmselect: run make vmui-update after c112dd7367 2023-09-01 10:54:22 +02:00
Yury Molodov
9bd71b0f12
vmui: support for Prometheus data on the cardinality page (#4713)
* feat: add cardinality support for prometheus (#4320)

* docs/CHANGELOG.md: add cardinality support for prometheus

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-09-01 10:54:21 +02:00
Aliaksandr Valialkin
d8afd7fe98
Makefile: update golangci-lint from v1.51.2 to v1.54.2
See https://github.com/golangci/golangci-lint/releases/tag/v1.54.2
2023-09-01 10:25:49 +02:00
Dima Lazerka
1d60c236b1
Add flagutil.Duration to avoid conversion bugs (#4835)
* Introduce flagutil.Duration

To avoid conversion bugs

* Fix tests

* Comment why not .Seconds()
2023-09-01 09:30:30 +02:00
Aliaksandr Valialkin
953699d59a
app/vmauth: add tests for ResponseHeaders
This is a follow-up for b18eed3427

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4825
2023-09-01 09:22:01 +02:00
Aliaksandr Valialkin
33e64d3207
app/vmctl: remove superflouos whitespace after 2853fac3f5 2023-09-01 08:59:19 +02:00
Alexander Marshalov
9ba352da0f
vmauth: added ability to set and remove response headers (#4825) (#4914)
* added ability to set and clear response headers (#4825)

Signed-off-by: Alexander Marshalov <_@marshalov.org>

* added ability to set and clear response headers (#4825)

Signed-off-by: Alexander Marshalov <_@marshalov.org>

* fix review comment

Signed-off-by: Alexander Marshalov <_@marshalov.org>

---------

Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-09-01 08:57:52 +02:00
Dmytro Kozlov
986ad3d112
docs: follow up after 939952068b (#4923) 2023-09-01 08:52:45 +02:00
Zakhar Bessarab
82060727ec
app/vlinsert/loki: add handler for healthcheck endpoint (#4885)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-08-30 16:37:54 +02:00
Nikolay
fbe2795670
app/vminsert: fixes readonly check (#4892)
* app/vminsert: fixes readonly check
previously vminsert doesn't check readOnly state for vmstorage, since check was never performed for nil buffer
In this case every 30 second storage node loss readonly state and received some data.
It caused re-routing and possible slow down for ingestion
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4870

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-08-30 16:24:24 +02:00
Aliaksandr Valialkin
1ca3b660f0
app/vmselect/promql: add support for _ delimiters in numeric values
For example, 1_234_567_890 is equivalent to 1234567890,
while 1.234_567_890 is equivalent to 1.234567890
2023-08-30 14:35:58 +02:00
Aliaksandr Valialkin
3a2d035283
lib/auth: add NewTokenPossibleMultitenant() for parsing auth token, which can be multitenant
Disallow parsing multitenant token at auth.NewToken().

Use auth.NewTokenPossibleMultitenant() at vminsert only. All the other callers should call auth.NewToken(),
since they do not support multitenant token.

This is a follow-up for f0c06b428e

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4910
2023-08-30 14:13:51 +02:00
hagen1778
bda9699657
app/vmselect: follow-up after f0c06b428e
Remove extra error message when auth token is nil. The default message
about unsupported path should be more clear to the user who mistakenly
requested /multitenant path.

f0c06b428e
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-29 21:52:50 +02:00
Zakhar Bessarab
f0c06b428e
app/vmselect: fix panic when using /select/multitenant endpoint (#4912)
app/vmselect: fix panic when using `/select/multitenant` endpoint

Such requests must be rejected as not found since vmselect does not support multitenant endpoint.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4910

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-08-29 21:48:12 +02:00
Aliaksandr Valialkin
5e8dfcf65e
app/{vmselect,vlselect}: run make vmui-update vmui-logs-update after recent changes to app/vmui 2023-08-29 12:58:58 +02:00
Aliaksandr Valialkin
19d61737c1
app/{vminsert,vmselect}: follow-up after 2b7b3293c1
- Document the change at docs/CHANGELOG.md
- Set the default value for -vmstorageUserTimeout to 3 seconds. This is much better
  than the 0 value, which means that TCP connection to unreachable vmstorage could block
  for up to 16 minutes.
- Document -vmstorageUserTimeout at docs/Cluster-VictoriaMetrics.md
2023-08-29 12:17:39 +02:00
Will Jordan
2b7b3293c1
Add vmstorageUserTimeout flags to configure TCP user timeout (Linux) (#4423)
`TCP_USER_TIMEOUT` (since Linux 2.6.37) specifies the maximum amount of
time that transmitted data may remain unacknowledged before TCP will
forcibly close the connection and return `ETIMEDOUT` to the application.

Setting a low TCP user timeout allows RPC connections quickly reroute
around unavailable storage nodes during network interruptions.
2023-08-29 11:46:39 +02:00
Aliaksandr Valialkin
1159b31270
app/vmagent/remotewrite: do not retry request immediately on io.ErrUnexpectedEOF, since this error isn't returned on stale connection
Also, mention the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4139 in comments to the code
in order to simplify further maintenance of this code.

This is a follow-up for 992a1c0a3a
2023-08-29 09:48:49 +02:00
Aliaksandr Valialkin
453077808d
docs/vmauth.md: consistently prepend command-line flags with a single - 2023-08-28 09:28:24 +02:00
Yury Molodov
2aff0ce327
vmui: ui improvements (#4872)
* vmui: chart refactoring to enhance code structure

* vmui: improve ui

(cherry picked from commit 72167a697e)
2023-08-27 09:05:55 +02:00
hagen1778
ff20d3e97c
docs: update remote-read api docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f4577005be)
2023-08-27 09:05:51 +02:00
Dmytro Kozlov
939952068b
docs: add migration guide from Promscale (#4865)
Signed-off-by: dmitryk-dk d.kozlov@victoriametrics.com
(cherry picked from commit 0b0e0bb50e)
2023-08-27 09:05:44 +02:00
hagen1778
b42187cf82
docs: typo fix for vmauth
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit ffbebfdfe6)
2023-08-27 09:05:41 +02:00
hagen1778
49327b584d
docs: mention that vmauth doesn't follow redirects
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4868

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 4d316a23ae)
2023-08-27 09:05:37 +02:00
hagen1778
33bf28e1bd
app/vmagent: fix comment typo after 992a1c0a3a
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 757ae4275b)
2023-08-27 09:05:04 +02:00
Roman Khavronenko
b9a2512ac3
vmagent: retry failed write request on the closed connection (#4857)
* vmagent: retry failed write request on the closed connection

 Retry failed write request on the closed connection immediately,
 without waiting for backoff. This should improve data delivery speed
 and reduce amount of error logs emitted by vmagent when using idle connections.

 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4139

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmagent: retry failed write request on the closed connection

Re-instantinate request before retry as body could have been already spoiled.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
(cherry picked from commit 992a1c0a3a)
2023-08-27 09:04:59 +02:00
Roman Khavronenko
68150655d2
vmalert: correctly re-instantinate HTTP req on retries (#4864)
* vmalert: correctly re-instantinate HTTP req on retries

Previosly, request retry to datasource re-used existing HTTP request.
But if request object was already partially processed (body was read),
then retry will be unsuccessful.

The change re-instantinates HTTP request object before retry.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: review fix

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit ddf87b32ed)
2023-08-27 09:04:42 +02:00
Aliaksandr Valialkin
c813b5e4b1
lib/promscrape: add -promscrape.cluster.memberLabel command-line flag
This flag allows specifying an additional label to add to all the scraped metrics.
The flag must contain label name to add. The label value will be equal to -promscrape.cluster.memberNum.

This functionality can help when there is a need to differentiate metrics scraped
by distinct vmagent instances in the cluster according to https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4247

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4247#issuecomment-1692279393
2023-08-24 22:04:34 +02:00
Yury Molodov
c527d2a549
vmui: change warning display for text fields (#4848) (#4863)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4848
(cherry picked from commit ca44b8da1f)
2023-08-21 15:50:20 +02:00
Yury Molodov
931c63f602
vmui: chart refactoring to enhance code structure (#4830)
(cherry picked from commit 8287749c05)
2023-08-21 15:50:19 +02:00
hagen1778
f48962e834
vmselect: follow-up after 7349f18c55
7349f18c55
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit ea2fbcf0e6)
2023-08-21 15:50:19 +02:00
Lapo Luchini
44ce25565a
Fix typo in vmauth docs.
(cherry picked from commit 297ef605ef)
2023-08-21 15:50:19 +02:00
Tamara Vashchuk
6a59737e96
vmui: Add button to prettify query (#4694)
* Add button to prettify query

Just capitalizes query text for now

* Add /prettify-query API handler

* Replace UI pretiffier using prettifier API

* Add showing server errors

Had to pass setQueryErrors from useFetchQuery.ts

* Use serverUrl from global AppState

* Change icon to AutoAwsome icon + added style change color when button is active

* Add sync/await to prettifyQuery function

* Doc public function for lint

* Minor async fix

* Removed extra blank lines

* Extract usePrettifyQuery hook

* Made more generic style for :active button

* Refactor usePrettifyQuery

However, prettify errors don't clean up query errors, but should

* Add prettyQuery functionality to CHANGELOG.md

* Reuse queryErrors

* Unhide errors on start

---------

Co-authored-by: Tamara <toma.vashchuk@gmail.com>

(cherry picked from commit 7349f18c55)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-21 15:50:17 +02:00
Aliaksandr Valialkin
50820ab1aa
app/vmagent/remotewrite: follow-up after a27c2f3773
- Fix Prometheus-compatible naming after applying the relabeling if -usePromCompatibleNaming command-line flag is set.
  This should prevent from possible Prometheus-incompatible metric names and label names generated by the relabeling.
- Do not return anything from relabelCtx.appendExtraLabels() function, since it cannot change the number of time series
  passed to it. Append labels for the passed time series in-place.
- Remove promrelabel.FinalizeLabels() call after adding extra labels to time series, since this call has been already
  made at relabelCtx.applyRelabeling(). It is user's responsibility if he passes labels with double underscore prefixes
  to -remoteWrite.label.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4247
2023-08-17 14:44:35 +02:00
Alexander Marshalov
73287a7c3a
vmagent: fixed premature release of the context (after #4247 / #4824) (#4849)
Follow-up after a27c2f3773

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4247

Signed-off-by: Alexander Marshalov <_@marshalov.org>
(cherry picked from commit 1e1a30ed7f)
2023-08-17 12:16:04 +02:00
Dmytro Kozlov
15bc40216d
app/vmctl: fix migration process if tenant have no data (#4799)
app/vmctl: don't interrupt migration process if tenant has no data

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Alexander Marshalov <_@marshalov.org>
(cherry picked from commit 39623ae428)
2023-08-16 14:56:52 +02:00
Roman Khavronenko
505ef72e69
vmbackup: correctly check if specified -dst belongs to specified -storageDataPath (#4841)
See this issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4837

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6da32a27ac)
2023-08-16 14:56:52 +02:00
Alexander Marshalov
58cf862b05
fixed applying remoteWrite.label for pushed metrics (#4247) (#4824)
vmagent: properly add extra labels before sending data to remote storage

labels from `remoteWrite.label` are now added to sent metrics just before they
 are pushed to `remoteWrite.url` after all relabelings, including stream aggregation relabelings (#4247)

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4247

Signed-off-by: Alexander Marshalov <_@marshalov.org>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit a27c2f3773)
2023-08-15 13:48:19 +02:00
Aliaksandr Valialkin
4b1f01e45d
lib/promrelabel: properly replace : char with _ in metric names when -usePromCompatibleNaming command-line flag is set
This addresses https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3113#issuecomment-1275077071 comment from @johnseekins
2023-08-14 16:18:17 +02:00
Aliaksandr Valialkin
bde876f7c9
all: refer to https://docs.victoriametrics.com/#resource-usage-limits in the error message about -search.max* limit
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4827
2023-08-14 02:02:12 -07:00
Aliaksandr Valialkin
1361239393
app/vmbackup: add ability to make server-side copying of existing backups 2023-08-13 17:26:26 -07:00
Aliaksandr Valialkin
f7f29a1840
app/vmctl: document that -vm-native-step-interval command-line option now supports week value
This is a follow-up for d322ee4b35

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4738
2023-08-12 07:33:13 -07:00
Nikolay
476286385f
opentelemetry: return human readable error for json encoding. (#4822)
Opentelemetry parser supports only protobuf atm.

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-08-12 05:06:19 -07:00
Aliaksandr Valialkin
5c80b11c15
app/vmselect: prevent from panic when lookbehind window inside rollup function is parsed into negative value
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4795
2023-08-12 04:49:56 -07:00
Aliaksandr Valialkin
0ee8a9120a
lib/flagutil: add defaultValue arg to NewArray{Int,Bytes,Duration} functions
The defaultValue is printed in the flag description when passing -help to the app.

This is a follow-up for aef31f201a

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4776
2023-08-12 04:19:34 -07:00
Aliaksandr Valialkin
37af7d4ed3
app/{vmselect,vlselect}: run make vmui-update vmui-logs-update after 86f1459ca6 2023-08-11 07:01:15 -07:00
Zakhar Bessarab
bae701e612
app/vlinsert/elasticsearch: add a command-line flag to provide ES version (#4778)
* app/vlinsert/elasticsearch: add a command-line flag to provide ES version

Adds a flag which will allow to change version which will be reported by ES endpoint for compatibility checks performed by external logs shippers(such as filebeat).
See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4777

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Document the -elasticsearch.version command-line flag

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4777

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-08-11 06:57:01 -07:00
Aliaksandr Valialkin
4470856cc4
all: update Go builder from Go1.20.7 to Go1.21.0
See https://tip.golang.org/doc/go1.21
and https://go.dev/blog/go1.21
2023-08-11 06:26:27 -07:00
Aliaksandr Valialkin
02a54dbe63
app/vmagent/remotewrite: go fmt 2023-08-11 06:26:12 -07:00
Aliaksandr Valialkin
b9e34a1386
docs/CHANGELOG.md: add a link to stream aggregation for the description of the bugfix at a4a1884237
This makes the description more clear.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4804
2023-08-11 05:48:10 -07:00
Aliaksandr Valialkin
d02fb47c2d
app/vmagent/remotewrite: keep in sync the default value for -remoteWrite.sendTimeout option in the description with the actually used timeout
This is a follow-up for aef31f201a

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4776
2023-08-11 05:46:27 -07:00
Aliaksandr Valialkin
2601930f20
app/vmctl: follow-up after 5aed369132
- Fix default value for --remote-read-disable-path-append
- Clarify description for the change at docs/CHANGELOG.md

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4655

TODO: address the comment at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4744
2023-08-11 05:45:22 -07:00
Zakhar Bessarab
bffec2fc02
{vmagent/remotewrite,vminsert/common}: fix dropInput and keepInput flags inconsistency (#4809)
{vmagent/remotewrite,vminsert/common}: fix dropInput and keepInput flags inconsistency

Sync behavior for dropInput and keepInput flags between single-node and vmagent.
Fix vmagent not respecting dropInput flag and reverse logic for keepInput.
2023-08-11 05:40:06 -07:00
Yury Molodov
088d08ac4a
vmui: hide "Logs Explorer" for the base build (#4761)
LogsExplorer should be a part of VictoriaLogs binaries, as well as VMUI is now part of VictoriaMetrics binaries.
2023-08-11 05:00:08 -07:00
Yury Molodov
cf0077b552
vmui: allow displaying the full error message on click (#4760)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4719
2023-08-11 04:59:34 -07:00
Abirdcfly
70780306b3
vmalert: fix vmalert_remotewrite_send_duration_seconds_total metric value (#4801)
The deferred call's arguments are evaluated immediately, but the function call is not executed until the surrounding function returns.

Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2023-08-11 04:58:00 -07:00
hagen1778
1786a703fd
vmalert: mention vmalert_iteration_duration_seconds metric in README
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-11 04:57:24 -07:00
Haleygo
74d5622606
vmalert: fix redundant clean up move (#4803)
Follow-up after 55ae2c2d57
2023-08-11 04:56:57 -07:00
Roman Khavronenko
4c91773a15
vmalert: cleanup config reload metrics handling (#4790)
* rename `configErr` to `lastConfigErr` to reduce confusion
* add tests to verify metrics and msg are set properly
* fix mistake when config success metric wasn't restored after an error

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-11 04:56:21 -07:00
Haleygo
c0fdd73313
vmalert: fix uncleaned tmp files in tests (#4788) 2023-08-11 04:55:42 -07:00
Zakhar Bessarab
0747ca0595
docs: make phrase about dedup and evaluation interval relation less obscure (#4781)
Value of `-dedup.minScrapeInterval` comand-line flag must be higher than `evaluation_interval` in order to make sure that only one sample on each evaluation will be left after deduplication.
Moreover, value of `-dedup.minScrapeInterval` must be a multiple of vmalert's `evaluation_interval` in order to make sure that samples will be aligned between deduplication window periods.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4774#issuecomment-1663940811

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-08-11 04:54:43 -07:00
Alexander Marshalov
d90dae2a68
add info about remoteWrite.sendTimeout default value (#4776)
Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-08-11 04:53:16 -07:00
Roman Khavronenko
7220f70506
security: bump go version from 1.20.6 to 1.20.7 (#4773)
The update includes a security fix to the crypto/tls package,
as well as bug fixes to the assembler and the compiler.

See the list of issues addressed in Go1.20.7 here:
https://github.com/golang/go/issues?q=milestone%3AGo1.20.7+label%3ACherryPickApproved

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-11 04:44:01 -07:00
SunKyu Lee
1d2f0e8c7b
vmauth: add broken backend backoff flag (#4416)
vmauth: allow configuring deadline for a backend to be excluded from the rotation

The new flag `-failTimeout` allows overriding default time for a bad backend
to be excluded from rotation. The override option could be useful for systems
where it is expected for backends to be off for significant periods of time. 

Co-authored-by: Zakhar Bessarab <zekker6@gmail.com>
2023-08-11 04:30:23 -07:00
Yury Molodov
d443ab5ae1
vmui: display partial response warning (#4742)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4721
2023-08-11 04:17:27 -07:00
Dmytro Kozlov
dabce3cc5d
app/vmctl: add flag where use can define path to the source remote read protocol (#4744)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4655
2023-08-11 03:56:34 -07:00
Roman Khavronenko
5a5785292c
vmctl: allow disabling binary export protocol (#4716)
Binary export API protocol can be disabled via `-vm-native-disable-binary-protocol` cmd-line flag when migrating data from VictoriaMetrics. Disabling binary protocol
can be useful for deduplication of the exported data before ingestion.
For this, deduplication need to be configured at `-vm-native-src-addr` side
and `-vm-native-disable-binary-protocol` should be set on vmctl side.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-11 03:19:20 -07:00
Nikolay
b0977b07fb
app/vminsert: adds note for dropSamplesOnOverload flag (#4797)
Adds note for dropSamplesOnOverload flag that are samples dropped before replication
2023-08-10 12:18:29 +02:00
Dmytro Kozlov
8d0576c714
app/vmctl: add support the week step for time-based chunks (#4743)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4738
2023-07-31 08:00:11 -07:00
Dmytro Kozlov
2d507a8757
app/vmctl: remove ping from remote read protocol (#4749)
Ping method was never used, so we remove it.
2023-07-31 07:59:03 -07:00
Roman Khavronenko
02dba5f728
vmalert: remove deprecated in v1.79.0 web links with */status suffix (#4747)
Links of form `/api/v1/<groupID>/<alertID>/status` were deprecated
in favour of `/api/v1/alerts?group_id=<>&alert_id=<>` links in
v1.79.0. See more details here https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2825

This change removes code responsible for deprecated functionality.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-31 07:58:37 -07:00
hagen1778
c6acb5b6bc
docs: rm typo of naming vmalert as a stateless service
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-31 07:57:42 -07:00
Aliaksandr Valialkin
8b4bf5d269
app/vlstorage: expose vl_data_size_bytes metric at /metrics page for tracking the on-disk data size (both indexdb and the data itself) 2023-07-31 07:56:16 -07:00
Roman Khavronenko
85eb62a2ec
vmalert: remove deprecated in v1.61.0 -rule.configCheckInterval (#4745)
Use `-configCheckInterval` command-line flag instead.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-31 07:55:54 -07:00
Damon07
4c509c0b89
{app/vmselect,docs}: support share_eq_over_time#4441 (#4725)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4441

Co-authored-by: wangm <wangmm@tuya.com>
2023-07-31 07:51:09 -07:00
Aliaksandr Valialkin
7b01e28371
app/vmalert: use proper timestamp in setConfigSuccess() 2023-07-28 11:16:14 -07:00
Roman Khavronenko
303d3616ec
vmalert: revert unittest feature (#4734)
* Revert "vmalert: unittest support stale datapoint (#4696)"

This reverts commit 0b44df7ec8.

* Revert "docs: specify min version and limitations for vmalert's unit tests"

This reverts commit a24541bd

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Revert "vmalert: init unit test (#4596)"

This reverts commit da60a68d

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs: mention unittest revert in changelog

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 9f1b9b86cc)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-28 11:09:52 +02:00
Aliaksandr Valialkin
63d9a92d3d
docs/vmalert.md: fix broken links to Web chapter 2023-07-27 18:05:25 -07:00
Nikolay
85de94e85c
lib/protoparser: adds opentelemetry parser (#2570)
* lib/protoparser: adds opentelemetry parser
app/{vmagent,vminsert}: adds opentelemetry ingestion path

Adds ability to ingest data with opentelemetry protocol
protobuf and json encoding is supported
data converted into prometheus protobuf timeseries
each data type has own converter and it may produce multiple timeseries
from single datapoint (for summary and histogram).
only cumulative aggregationFamily is supported for sum(prometheus
counter) and histogram.

Apply suggestions from code review

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

updates deps

fixes tests

wip

wip

wip

wip

lib/protoparser/opentelemetry: moves to vtprotobuf generator

go mod vendor

lib/protoparse/opentelemetry: reduce memory allocations

* wip

- Remove support for JSON parsing, since it is too fragile and is rarely used in practice.
  The most clients send OpenTelemetry metrics in protobuf.
  The JSON parser can be added in the future if needed.
- Remove unused code from lib/protoparser/opentelemetry/pb and lib/protoparser/opentelemetry/proto
- Do not re-use protobuf message between ParseStream() calls, since there is high chance
  of high fragmentation of the re-used message because of too complex nested structure of the message.

* wip

* wip

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-07-27 13:37:15 -07:00
Aliaksandr Valialkin
ce18e9b2c4
app/vmalert: make golangci-lint happy after ae0e4a8c90 2023-07-27 13:27:36 -07:00
Haleygo
3c297e0253
vmalert: add keep_firing_for field for alerting rule (#4669)
vmalert: support `keep_firing_for` field for alerting rule

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4529

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-07-27 13:00:45 -07:00