Commit Graph

3469 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
2db1a664e1
deployment: update Go builder from Go1.21.3 to Go1.21.4
See https://github.com/golang/go/issues?q=milestone%3AGo1.21.4+label%3ACherryPickApproved
2023-11-10 22:28:44 +01:00
Aliaksandr Valialkin
010dc15d16
lib/blockcache: do not cache entries, which were attempted to be accessed 1 or 2 times
Previously entries which were accessed only 1 time weren't cached.
It has been appeared that some rarely executed heavy queries may read indexdb block twice
in a row instead of once. There is no need in caching such a block then.
This change should eliminate cache size spikes for indexdb/dataBlocks when such heavy queries are executed.

Expose -blockcache.missesBeforeCaching command-line flag, which can be used for fine-tuning
the number of cache misses needed before storing the block in the caching.
2023-11-10 22:28:03 +01:00
Aliaksandr Valialkin
22498c5087
docs/Articles.md: sort third-party articles by importance 2023-11-10 21:26:51 +01:00
Aliaksandr Valialkin
dc668b8246
docs/Articles.md: add a link to https://blog.cloudflare.com/introducing-http-traffic-anomalies-notifications/ 2023-11-10 20:59:57 +01:00
Aliaksandr Valialkin
a9a26c20b5
docs/Single-server-VictoriaMetrics.md: make High availability section more clear 2023-11-10 20:58:32 +01:00
Zakhar Bessarab
73a1862182
docs/changelog: document vmbackupmanager bugfix (#5303)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-11-08 18:51:14 +01:00
Artem Navoiev
930d26b2ff
docs: vmagent change the codeblock languages
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-11-08 18:12:29 +01:00
Github Actions
2b15f2fc8c
Automatic update Grafana datasource docs from VictoriaMetrics/grafana-datasource@cc18249 (#5305) 2023-11-08 18:19:12 +04:00
Github Actions
746cc93e95
Automatic update Grafana datasource docs from VictoriaMetrics/grafana-datasource@52bdb4a (#5304) 2023-11-08 17:59:10 +04:00
Roman Khavronenko
bffd30b57a
app/vmalert: update remote-write process (#5284)
* app/vmalert: update remote-write process

* automatically retry remote-write requests on closed connections. The change should reduce the amount of logs produced in environments with short-living connections or environments without support of keep-alive on network balancers.
* increment `vmalert_remotewrite_errors_total` metric if all retries to send remote-write request failed. Before, this metric was incremented only if remote-write client's buffer is overloaded.
* increment `vmalert_remotewrite_dropped_rows_total` amd `vmalert_remotewrite_dropped_bytes_total` metrics if remote-write client's buffer is overloaded. Before, these metrics were incremented only after unsuccessful HTTP calls.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Update docs/CHANGELOG.md

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Hui Wang <haley@victoriametrics.com>
2023-11-08 14:53:07 +08:00
Artem Navoiev
b51d16e74c
docs: url example change the title h2->h3 h3->h4 for better indexing in search
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-11-07 09:54:32 +01:00
Artem Navoiev
e4f44bad91
docs: fix formatting in stream aggregation more
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-11-07 09:31:21 +01:00
Artem Navoiev
ea4ca70be1
docs: fix formatting in stream aggregation
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-11-07 09:27:44 +01:00
Github Actions
4d2b98b755
Automatic update operator docs from VictoriaMetrics/operator@b4b79da (#5291) 2023-11-06 16:04:12 +08:00
hagen1778
c07dc45786
app/vmalert: fix typo in remoteWrite.concurrency description
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-03 22:04:50 +01:00
Yury Molodov
f90d2ec843
vmui: display query error on Explore metrics page (#5272)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5202
2023-11-03 16:23:19 +01:00
hagen1778
054367c421
docs: make docs-sync after 323f3720ed
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-03 16:22:23 +01:00
Zakhar Bessarab
323f3720ed
app/vmauth: add option to skip TLS verification (#5256)
Add `tls_insecure_skip_verify` option on per-user basis which allows to disable TLS verification for all requests to backend on behalf of this user.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5240

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-11-03 12:04:17 +01:00
Aliaksandr Valialkin
815fda8995
docs: update -help output after recent changes to VictoriaMetrics components 2023-11-02 20:27:10 +01:00
Aliaksandr Valialkin
65db6609eb
docs/CHANGELOG.md: update the description of the optimization for SLO/SLI-like queries according to latest changes
See commits 4497a08e3d and 92826b0b4a
2023-11-02 20:05:05 +01:00
Roman Khavronenko
b5254199c6
app/vmalert: add label file pointing to the group's filename to metrics (#5281)
The filename should help identifying alerting rules belonging to specific groups
with identical names but different filenames.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5267

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-11-02 16:01:31 +01:00
Hui Wang
90d45574bf
vmalert: reduce restore query request for each alerting rule (#5265)
reduce the number of queries for restoring alerts state on start-up. 
The change should speed up the restore process and reduce pressure on `remoteRead.url`.
2023-11-02 15:22:13 +01:00
Aliaksandr Valialkin
dd33fc0c76
docs/CHANGELOG.md: typo fix: tis -> this 2023-11-02 08:33:40 +01:00
Aliaksandr Valialkin
536c1301fd
docs/Single-server-VictoriaMetrics.md: document why data inside <-storageDataPath>/snapshots directory should be manipulated only via snapshot API 2023-11-02 08:30:39 +01:00
Aliaksandr Valialkin
87a86ec9db
docs/CHANGELOG.md: document v1.93.7 LTS release 2023-11-02 08:21:00 +01:00
Aliaksandr Valialkin
ed70a40669
app/vmagent/remotewrite: add -remoteWrite.shardByURL.labels command-line flag
This command-line flag can be used for specifying a list of labels used for sharding
among -remoteWrite.url entries when -remoteWrite.shardByURL command-line flag is set.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4942
2023-11-01 23:08:54 +01:00
Alexander Marshalov
828ddd4e4f
vmauth: add browser authorization request for http requests without… (#5234)
* vmauth: add browser authorization request for http requests without credentials to a route that is not in the `unauthorized_user` section (when `unauthorized_user` is specified).

* add link to issue in CHANGELOG

* Extend vmauth docs

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-11-01 20:59:46 +01:00
Github Actions
b44b74d118
Automatic update operator docs from VictoriaMetrics/operator@49826be (#5270)
Co-authored-by: Alexander Marshalov <_@marshalov.org>
2023-11-01 17:45:56 +01:00
Artem Navoiev
324ecbeed6
add Try new Docs button in the current docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-11-01 17:22:49 +01:00
Aliaksandr Valialkin
da887b49e7
app/vmui: show query execution duration in the header of query input field
This should simplify the process of query optimization
2023-11-01 16:43:51 +01:00
Hui Wang
e482eeff58
vmalert: support specifying full http url in notifier static_configs target (#5261)
* vmalert: support specifying full http or https urls in notifier static_configs target address
* show right label results in ui
2023-11-01 19:53:50 +08:00
Github Actions
3e76c3bd50
Automatic update operator docs from VictoriaMetrics/operator@57c1bf6 (#5266) 2023-11-01 16:33:25 +08:00
Aliaksandr Valialkin
c4c6ee9485
app/vmui: fix non-working Disable cache checkbox at JSON and Table views 2023-10-31 22:58:06 +01:00
Aliaksandr Valialkin
ea81f6fc36
app/vmselect/promql: add outliers_iqr(q) and outlier_iqr_over_time(m[d]) functions
These functions allow detecting anomalies in series and samples using Interquartile range method.
See Outliers section at https://en.wikipedia.org/wiki/Interquartile_range for more details.
2023-10-31 22:10:31 +01:00
Aliaksandr Valialkin
41a0fdaf39
app/vmselect/promql: optimize repeated SLI-like instant queries with lookbehind windows >= 1d
Repeated instant queries with long lookbehind windows, which contain one of the following rollup functions,
are optimized via partial result caching:

- sum_over_time()
- count_over_time()
- avg_over_time()
- increase()
- rate()

The basic idea of optimization is to calculate

  rf(m[d] @ t)

as

  rf(m[offset] @ t) + rf(m[d] @ (t-offset)) - rf(m[offset] @ (t-d))

where rf(m[d] @ (t-offset)) is cached query result, which was calculated previously

The offset may be in the range of up to 1 hour.
2023-10-31 19:25:23 +01:00
Aliaksandr Valialkin
c96fc05f3e
docs/Cluster-VictoriaMetrics.md: sync with cluster branch after 9d8f93050c 2023-10-31 19:13:11 +01:00
Aliaksandr Valialkin
714af89b13
lib/httpserver: follow-up for 0638bbe69c
- Replace spaces with underscores in the `reason` label value for the vm_http_request_errors_total metric
  in order be consistent with Prometheus-like naming

- Clarify the description for the change at docs/CHANGELOG.md

Updates https://github.com/victoriaMetrics/victoriaMetrics/issues/4590
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5166
2023-10-31 18:52:39 +01:00
Aliaksandr Valialkin
aade16f534
docs/Cluster-VictoriaMetrics.md: clarify the description on why -dosnwampling.period must be set at both vmstorage and vmselect
This is a follow-up for ca7457d906
2023-10-31 16:53:46 +01:00
Aliaksandr Valialkin
a71efce784
docs/Single-server-VictoriaMetrics.md: cosmetic fixes after 23369321f1 2023-10-31 16:42:29 +01:00
Aliaksandr Valialkin
4ac95b6f49
docs/CHANGELOG.md: move the description for -http.header.* command-line flags from SECURITY to FEATURE
The SECURITY label should be applied only to changes, which fix security issues.
The change at ad839aa492 adds new command-line flags, which can be used
for improving security in some cases. They do not fix any security issues.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5111
2023-10-31 16:23:08 +01:00
hagen1778
f6208965ce
dashboards/cluster: fix description about max threshold for Concurrent selects panel.
Before, it was mistakenly implying that `max` is equal to the double of available CPUs.

Addresses https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5214

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-31 16:05:33 +01:00
Roman Khavronenko
a950873fff
app/vmselect: expose vm_memory_intensive_queries_total counter metric (#5208)
The new metric gets increased each time `-search.logQueryMemoryUsage` memory limit
is exceeded by a query. This metric should help to identify expensive and heavy queries
without inspecting the logs.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-31 13:31:09 +01:00
hagen1778
a8051d48c4
docs: follow-up for 0638bbe69c
0638bbe69c
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-31 12:54:30 +01:00
hagen1778
aaf9e3d526
dashboards/vmalert: add new panel Missed evaluations
The new panel supposed to indicate alerting groups that miss their evaluations.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-31 10:35:19 +01:00
hagen1778
9866974a53
deployment/alerts: add TooManyMissedIterations alerting rule
The new rule for vmalert supposed to detect groups that miss their
evaulations due to slow queries.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-31 10:35:18 +01:00
hagen1778
8874b525b7
dashboards: fix Errors rate to Alertmanager filter
The panel `Errors rate to Alertmanager` had `group` label filter
applied to the expression, while the metric `vmalert_alerts_send_errors_total`
doesn't have that label. This resulted into always empty results.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-31 10:16:45 +01:00
Roman Khavronenko
ca7457d906
docs: explain motivation behind having -downsampling.period on vmselect (#5205)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-30 19:03:36 +01:00
Roman Khavronenko
23369321f1
docs: mention information loss when downsampling gauges (#5204)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-30 15:29:06 +01:00
Hui Wang
abcb21aa5e
vmalert: fix alert firing state in replay mode (#5192)
fix possible missing firing states for alerting rules in replay mode
Before if one firing stage is bigger than single query request range, like rule with a big `for`, alerting rule won't able to be detected as firing.

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-10-30 13:54:18 +01:00
hagen1778
e964df8039
docs/troubleshooting: mention issue with un-ordered labels
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5219#issuecomment-1773441711

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-30 13:53:14 +01:00
hagen1778
a64b37cf24
docs: rm mention of default values for security HTTP headers
The headers, their corresponding flags are mentioned at
https://docs.victoriametrics.com/#security

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-30 11:46:17 +01:00
Dima Lazerka
ad839aa492
lib/httpserver: add flags to specify HSTS / Frame-Options / CSP headers for httpserver (#5111)
support `Strict-Transport-Security`, `Content-Security-Policy` and `X-Frame-Options`
HTTP headers in all VictoriaMetrics components. 
The values for headers can be specified by users via the following flags: 
`-http.header.hsts`, `-http.header.csp` and `-http.header.frameOptions`.

Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-10-30 11:33:38 +01:00
Roman Khavronenko
29cebd82fb
lib/storage: log warning about RO mode only on state change (#5191)
Before, vmstorage would log the same message each second producing excessive
amount of logs.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5159

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-30 10:52:57 +01:00
Aliaksandr Valialkin
632d788b63
lib/promscrape/discovery/kubernetes: stop all the url watchers, which belong to a particular groupWatcher, at once
Previously url watchers for pod, service and node objects could be mistakenly closed
when service discovery was set up only for endpoints and endpointslice roles,
since watchers for these roles may start start pod, service and node url watchers
with nil apiWatcher passed to groupWatcher.startWatchersForRole().

Now all the url watchers, which belong to a particular groupWatcher, are stopped at once
when this groupWatcher has no apiWatcher subscribers.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5216

The issue has been introduced in v1.93.5 when addressing https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4850
2023-10-27 13:51:35 +02:00
Hui Wang
7c90ce39cb
do not print redundant error logs when failed to scrape consul or no… (#5239)
* do not print redundant error logs when failed to scrape consul or nomad target
prometheus performs the same because it uses consul lib which just drops the error(1806bcb38c/api/api.go (L1134))
2023-10-27 13:31:55 +08:00
Daria Karavaieva
b60bb1d98a
model list - isolation forest (#5235)
* model list - isolation forest

* curse of dimensionality

* isol forest definition change, minor fixes

* blank line fix
2023-10-26 12:25:54 +02:00
Aliaksandr Valialkin
d5a599badc
lib/promauth: follow-up for e16d3f5639
- Make sure that invalid/missing TLS CA file or TLS client certificate files at vmagent startup
  don't prevent from processing the corresponding scrape targets after the file becomes correct,
  without the need to restart vmagent.
  Previously scrape targets with invalid TLS CA file or TLS client certificate files
  were permanently dropped after the first attempt to initialize them, and they didn't
  appear until the next vmagent reload or the next change in other places of the loaded scrape configs.

- Make sure that TLS CA is properly re-loaded from file after it changes without the need to restart vmagent.
  Previously the old TLS CA was used until vmagent restart.

- Properly handle errors during http request creation for the second attempt to send data to remote system
  at vmagent and vmalert. Previously failed request creation could result in nil pointer dereferencing,
  since the returned request is nil on error.

- Add more context to the logged error during AWS sigv4 request signing before sending the data to -remoteWrite.url at vmagent.
  Previously it could miss details on the source of the request.

- Do not create a new HTTP client per second when generating OAuth2 token needed to put in Authorization header
  of every http request issued by vmagent during service discovery or target scraping.
  Re-use the HTTP client instead until the corresponding scrape config changes.

- Cache error at lib/promauth.Config.GetAuthHeader() in the same way as the auth header is cached,
  e.g. the error is cached for a second now. This should reduce load on CPU and OAuth2 server
  when auth header cannot be obtained because of temporary error.

- Share tls.Config.GetClientCertificate function among multiple scrape targets with the same tls_config.
  Cache the loaded certificate and the error for one second. This should significantly reduce CPU load
  when scraping big number of targets with the same tls_config.

- Allow loading TLS certificates from HTTP and HTTPs urls by specifying these urls at `tls_config->cert_file` and `tls_config->key_file`.

- Improve test coverage at lib/promauth

- Skip unreachable or invalid files specified at `scrape_config_files` during vmagent startup, since these files may become valid later.
  Previously vmagent was exitting in this case.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4959
2023-10-25 23:19:37 +02:00
Aliaksandr Valialkin
eed5206376
lib/promauth: properly parse string contents for ca, cert and key fields at tls_config
Previously yaml parser wasn't accepting string values for these fields,
because it was mistakenly expecting a list of uint8 values instead.
2023-10-25 23:12:21 +02:00
hagen1778
a216fe6728
app/vmalert: follow-up after c9375cac5e
c9375cac5e

Descriptions were updated in attempt to make it more clear for readers,
re-phrasing and linking missing docs.

`eval_delay` was added to tests to verify it can be unmarshalled.

`eval_delay` is now applied before timestamp alignment to make it more predictable.
Before, if delay < interval the timestamp won't be aligned.

`eval_delay` and `eval_offset` was added to API output.

`PreviouslySentSeriesToRW` converted to private `previouslySentSeriesToRW`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-25 13:07:13 +02:00
Hui Wang
c9375cac5e
vmalert: add -rule.evalDelay flag and eval_delay as group attribute (#5185)
Also mark `-datasource.lookback` as will be deprecated, see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5155.
2023-10-25 11:54:18 +02:00
hagen1778
003ef3a518
deployment/alerts: make TooHighMemoryUsage more tolerable to spikes
Using `min_over_time` should reduce the amount of false positives when
component is running in near-the-threshold state. Now it should trigger
only if all collected samples were above the threshold on 10m interval.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-24 09:39:46 +02:00
Alexander Marshalov
33484d3365
lib/streamaggr: respect streamAgg.dropInput with empty stream aggr config (#5213)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5207
2023-10-20 15:55:58 +02:00
krakazyabra
239849db5d
docs/case-studies: update Wedos info (#5211) 2023-10-19 22:38:40 +02:00
Aliaksandr Valialkin
2f21c0c119
docs: use https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest instead of https://github.com/VictoriaMetrics/VictoriaMetrics/releases link where needed
The https://github.com/VictoriaMetrics/VictoriaMetrics/releases link may show non-latest
releases at the top, such as LTS releases or VictoriaLogs releases.
So it is better to use https://github.com/VictoriaMetrics/VictoriaMetrics/releases/latest link,
which always redirect to the latest available release of VictoriaMetrics.
2023-10-18 20:06:25 +02:00
Roman Khavronenko
b8b6e120ff
app/vmselect: limit the number of parallel workers by 32 (#5195)
* app/vmselect: limit the number of parallel workers by 32

The change should improve performance and memory usage during query processing
on machines with big number of CPU cores. The number of parallel workers for
query processing is controlled via `-search.maxWorkersPerQuery` command-line flag.
By default, the number of workers is limited by the number of available CPU cores,
but not more than 32. The limit can be increased via `-search.maxWorkersPerQuery`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

- The `-search.maxWorkersPerQuery` command-line flag doesn't limit resource usage,
  so move it from the `resource usage limits` to `troubleshooting` chapter at docs/Single-server-VictoriaMetrics.md

- Make more clear the description for the `-search.maxWorkersPerQuery` command-line flag

- Add the description of `-search.maxWorkersPerQuery` to docs/Cluster-VictoriaMetrics.md

- Limit the maximum value, which can be passed to `-search.maxWorkersPerQuery`, to GOMAXPROCS,
  because bigger values may worsen query performance and increase CPU usage

- Improve the the description of the change at docs/CHANGELOG.md. Mark it as FEATURE instead of BUGFIX,
  since it is closer to a feature than to a bugfix.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5087

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-10-18 19:51:37 +02:00
Aliaksandr Valialkin
38b8872a47
docs/Articles.md: add an article https://rtfm.co.ua/en/victoriametrics-vmauth-proxy-authentication-and-authorization/ 2023-10-18 18:29:26 +02:00
Aliaksandr Valialkin
ad871dd9ed
docs/FAQ.md: add questions on why VictoriaMetrics doesnt rebalance data and doesnt restore replication factor between vmstorage nodes 2023-10-18 18:06:26 +02:00
Aliaksandr Valialkin
ea4758f5cd
docs/FAQ.md: refresh the answer to the question about how does VictoriaMetrics compare to competing solutions
- Mention Grafana Mimir
- Fix broken links
2023-10-18 09:13:57 +02:00
Github Actions
0a6932acfe
Automatic update operator docs from VictoriaMetrics/operator@2c826bb (#5188) 2023-10-17 15:57:11 +02:00
hagen1778
fd2d07ba33
lib/storage: follow-up after 188cfe3a85
188cfe3a85

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5159

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-17 15:45:14 +02:00
Hui Wang
e16d3f5639
fix inconsistent behaviors with prometheus when scraping (#5153)
* fix inconsistent behaviors with prometheus when scraping

1. address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4959. skip job with wrong syntax in `scrape_configs` with error logs instead of exiting;
2. show error messages on vmagent /targets ui if there are wrong auth configs in `scrape_configs`, previously will print error logs and do scrape without auth header;
3. don't send requests if there are wrong auth configs in:
    1. vmagent remoteWrite;
    2. vmalert datasource/remoteRead/remoteWrite/notifier.

* add changelogs

* address review comments

* fix ut
2023-10-17 17:58:19 +08:00
hagen1778
3e2f09541e
docs: mention key concepts as querying example
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5169

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-17 11:01:17 +02:00
hagen1778
c2d252c045
dashboards/vmalert: respect job and instance filters in No data errors
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-17 09:40:39 +02:00
hagen1778
edba9f6266
dashboards/vmalert: use desc sorting for tooltips on panels
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-17 09:31:09 +02:00
Aliaksandr Valialkin
f89051f83f
docs/Articles.md: add newly appeared articles about VictoriaMetrics at medium.com
- https://sarthak-acoustic.medium.com/solving-metrics-at-scale-with-victoriametrics-ac9c306826c3
- https://medium.com/@seifeddinerajhi/victoriametrics-a-comprehensive-guide-comparing-it-to-prometheus-and-implementing-kubernetes-03eb8feb0cc2
2023-10-17 00:55:35 +02:00
Aliaksandr Valialkin
14f3d844fe
docs/CHANGELOG.md: document v1.93.6 LTS release
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.6
2023-10-17 00:53:18 +02:00
Aliaksandr Valialkin
daaf2b0e61
docs/CHANGELOG.md: document v1.87.10 release
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.87.10
2023-10-16 23:25:38 +02:00
Aliaksandr Valialkin
da77f4deeb
app/vmselect/promql: add labels_equal(q, "label1", "label2", ...) function
This function returns q series, which have identical values for the listed labels
"label1", "label2", ...

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5148
2023-10-16 21:50:11 +02:00
Aliaksandr Valialkin
484b5ed12f
docs/MetricsQL.md: typo fix after bdb743c88d
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5071
2023-10-16 21:09:36 +02:00
Aliaksandr Valialkin
6c3dd16a16
app/vmagent/remotewrite: move sas var initialization closer to the place where it is used
This makes the code sligthtly easier to understand.

This is a follow-up for 1d3d989be5

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5170
2023-10-16 20:52:56 +02:00
Aliaksandr Valialkin
bdb743c88d
app/vmselect/promql: add drop_empty_series() function for dropping empty series before performing additional calculations
This can be useful in the following queries:

   drop_empty_series(temperature <= 30) default 40

This query drops temperature series with all the values bigger than 30 on the selected time range,
while replacing gaps in the remaining series with 40.

The query without drop_empty_series:

  (temperature <= 30) default 40

would leave all the temperature series with all the values bigger than 30 on the selected time range,
and replace all their values with 40. This is not what could be epxected in some cases
like here - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5071
2023-10-16 20:44:56 +02:00
hagen1778
1d3d989be5
app/vmagent/remotewrite: follow-up after 4f102ff945
4f102ff945
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-16 16:00:24 +02:00
Aliaksandr Valialkin
b8c267075e
lib/promscrape: add a link to https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets in descriptions for -promscrape.cluster.* command-line flags
This should help users figuring out the purpose of -promscrape.cluster.* command-line flags
2023-10-16 14:46:22 +02:00
Aliaksandr Valialkin
6f98b9c221
Revert "docs victorialogs use relative links"
This reverts commit 3d7a77bf82.

Reason for revert: relative links do not work properly at GitHub code
and at GitHub wiki. For example, the following page contains broken links
before reverting this commit:

https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/VictoriaLogs/CHANGELOG.md

It is always better to use absolute links thank relative links, since the page contents
can be copy-n-pasted to other pages, which are located in vastly different directories,
and all the links will remain working.
2023-10-16 13:40:43 +02:00
Aliaksandr Valialkin
56b9c0b717
docs/CaseStudies.md: typ fix: vmgent -> vmagent
This is a follow-up for f5c46b8176
2023-10-16 13:33:04 +02:00
Aliaksandr Valialkin
07150359b2
docs/vmbackup.md: clarify documentation about -deleteAllObjectVersions command-line flag
Updates 2fc7e9f47e
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5121
2023-10-16 12:09:46 +02:00
Alexander Marshalov
b248413a07
fixed error when creating a full backup using the -origin flag (#5180)
* fixed error when creating a full backup using the `-origin` flag (#5144)

* Update docs/CHANGELOG.md

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-10-16 12:02:51 +02:00
Github Actions
7abdbbc8e2
Automatic update operator docs from VictoriaMetrics/operator@79298bf (#5177) 2023-10-16 10:45:46 +08:00
Aliaksandr Valialkin
2c334ed953
app/{vmagent,vminsert}: follow-up for NewRelic data ingestion protocol support
This is a follow-up for f60c08a7bd

Changes:

- Make sure all the urls related to NewRelic protocol start from /newrelic . Previously some urls were started from /api/v1/newrelic

- Remove /api/v1 part from NewRelic urls, since it has no sense

- Remove automatic transformation from CamelCase to snake_case for NewRelic labels and metric names,
  since it may complicate the transition from NewRelic to VictoriaMetrics. Preserve all the metric names and label names,
  so users could query metrics and labels by the same names which are used in NewRelic.
  The automatic transformation from CamelCase to snake_case can be added later as a special action for relabeling rules if needed.

- Properly update per-tenant data ingestion stats at app/vmagent/newrelic/request_handler.go . Previously it was always zero.

- Fix NewRelic urls in vmagent when multitenant data ingestion is enabled. Previously they were mistakenly started from `/`.

- Document NewRelic data ingestion url at docs/Cluster-VictoriaMetrics.md

- Remove superflouos memory allocations at lib/protoparser/newrelic

- Improve tests at lib/protoparser/newrelic/*

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3520
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4712
2023-10-16 00:25:25 +02:00
Aliaksandr Valialkin
ddbe713470
docs/Single-server-VictoriaMetrics.md: add a link to the original issue, which describes how to run VictoriaMetrics as Windows service
This is a follow-up for cc7d5b7bab

The original issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3781 contains
up-to-date information on how to run VictoriaMetrics components as Windows service,
plus it may contain additional information about this case such as https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3781#issuecomment-1708092680 ,
so it is better to refer this issue from the docs.
2023-10-15 19:33:39 +02:00
Roman Khavronenko
3594214a16
lib/vmselect: bump maxSearchQuerySize to 5MB (#5158)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5154#issuecomment-1757216612

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5154

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-15 19:24:38 +02:00
Artem Navoiev
8dff1c696f
docs fix broken links
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-14 15:39:49 +02:00
Artem Navoiev
a200aaf0ef
docs fix broken links in operator
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-14 15:17:17 +02:00
Artem Navoiev
b808fe959a
docs vlogs changelog fix broken link
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-14 15:10:28 +02:00
Artem Navoiev
065a6b020b
docs fix 404 external links
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-14 15:03:13 +02:00
Artem Navoiev
f5c46b8176
docs fix bad links
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-14 14:56:06 +02:00
Artem Navoiev
3d7a77bf82 docs victorialogs use relative links
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-14 05:42:29 -07:00
Artem Navoiev
83d64aa621
docs/vlogs/dataingetion remoev unknown language from codeblock
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-14 13:54:41 +02:00
Artem Navoiev
8d3b732ce0
docs/vmlogs fix aliases
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-14 13:45:57 +02:00
hagen1778
60eab8e2ef
docs: fix typo
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-14 07:46:00 +02:00
hagen1778
fe8cc573d1
docs: remove extra / in the end of the link
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-14 07:43:40 +02:00
Artem Navoiev
1ca0b86397
docs remove (( from the link to fix them
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-14 00:14:41 +02:00
Artem Navoiev
84cf96228f docs/alert-tools add yaml output type for the correct displaying
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-13 09:39:27 -07:00
Aliaksandr Valialkin
d1638db849
docs/Cluster-VictoriaMetrics.md: follow-up after f42c12e69a
Return back accidentally deleted text regarding KISS principles used in VictoriaMetrics development
2023-10-13 15:00:09 +02:00
Aliaksandr Valialkin
3f5a41e35e
lib/license: cleanups and prettifications for log messages and docs related to licensing
- Make more clear the docs at docs/enterprise.md, so readers could figure out faster
  on how to obtain enterprise key and how to pass it to VictoriaMetrics Enterprise components.

- Fix examples at docs/enterprise.md, which were referring to non-existing `-license-file` command-line flag.
  The `-licenseFile` command-line flag must be used instead.

- Improve the description of `-license*` command-line flags, so users could understand
  faster how to use them.

- Improve the warning message, which is emitted when the deprecated -eula command-line flag is passed,
  so the user could figure out how to switch faster to -license* command-line flags.

- Disallow running VictoriaMetrics components with both -license and -licenseFile command-line flags.

- Disallow running VictoriaMetrics components when -licensFile points to an empty file.

- Consistently use the phrase "This flag is available only in Enterprise binaries" across
  all the enterprise-specific command-line flags.

- Remove unneeded level of indirection for `noLicenseMessage` and `expiredMessage` string contants
  in order to improve code readability and maintainability.

- Remove unneded `return` statements after `logger.Fatalf()` calls, since these calls exit the app and never return.

- Make sure that the info log message about successful license verification is emitted
  when the license is verified successfully. Previously the error message could be logged
  when the license payload is invalid or if it misses some required features.
2023-10-13 14:56:09 +02:00
Artem Navoiev
36b0cea3f3
docs: mention the free trial for enterprise
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 636d4ea196)
2023-10-13 14:31:37 +02:00
Haleygo
dc28196237
vmalert-tool: implement unittest (#4789)
1. split package rule under /app/vmalert, expose needed objects
2. add vmalert-tool with unittest subcmd

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2945
2023-10-13 13:54:33 +02:00
Artem Navoiev
98a5007d32
fix title for managed qs guide
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-13 11:10:58 +02:00
Aliaksandr Valialkin
964e6ccc17
docs/Single-server-VictoriaMetrics.md: remove question mark added by accident in the commit 3d5d62e38a 2023-10-12 20:01:18 +02:00
Aliaksandr Valialkin
930a36df40
app/vmui: small UX enhancements
- Reduce vertical space usage, so more information is available on the screen without the need to scroll.
- Show information for lines with higher values at the top of the legend under the graph.
  This should simplify graph analysis when it contains many lines.
2023-10-12 19:54:19 +02:00
Aliaksandr Valialkin
d984598e30
deployment/docker: update Go builder from Go1.21.1 to Go1.21.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.21.2+label%3ACherryPickApproved
and https://github.com/golang/go/issues?q=milestone%3AGo1.21.3+label%3ACherryPickApproved
2023-10-12 09:41:41 +02:00
Aliaksandr Valialkin
31f7ef0811
app/{vmselect,vlselect}: enable caching of static contents from /vmui/static/ folder at client side
This should improve repated VMUI page load times on slow networks

See https://developer.chrome.com/docs/lighthouse/performance/uses-long-cache-ttl/
2023-10-12 09:33:40 +02:00
Zakhar Bessarab
8c4ce186d3
docs/vmbackupmaanger: reference permanent objects deletion (#5157)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-10-11 12:38:05 +02:00
hagen1778
d43566605b
dasbhoards: fix vminsert/vmstorage/vmselect metrics filtering
Fix vminsert/vmstorage/vmselect metrics filtering when dashboard is used
to display data from many sub-clusters with unique job names.
Before, only one specific job could have been accounted for component-specific panels,
instead of all available jobs for the component.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-11 12:09:04 +02:00
Zakhar Bessarab
2fc7e9f47e
lib/backup: add -deleteAllObjectVersions command-line flag (#5147)
New flag enforces removal of all versions of the object in remote object storage.

See:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5121
- https://docs.victoriametrics.com/vmbackup.html#permanent-deletion-of-objects-in-s3-compatible-storages
2023-10-10 14:13:23 +02:00
Yury Molodov
6dc5306c9b
vmui: transfer Top Queries time interval #5097 (#5145)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5097
2023-10-10 13:58:39 +02:00
Dmytro Kozlov
a6d06474f4
docs/managed-victoriametrics: update documentation (#5135)
Co-authored-by: Ivan Yatskevich <ivan@yatskevich.com>
2023-10-10 13:48:22 +02:00
Nikolay
1f91f22b5f
app/vmselect: reduce lock contention for heavy aggregation requests (#5119)
reduce lock contention for heavy aggregation requests
previously lock contetion may happen on machine with big number of CPU due to enabled string interning. sync.Map was a choke point for all aggregation requests.
Now instead of interning, new string is created. It may increase CPU and memory usage for some cases.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5087
2023-10-10 13:45:20 +02:00
Haleygo
2aa0f5fc41
vmalert: add evalAlignment for rule group and fix evalutaion timstamp (#5066)
* vmalert: add `query_time_alignment` for rule group

1. add `eval_alignment` attribute for group which by default is true. So group rule query stamp will be aligned with interval and propagated to ALERT metrics and the messages for alertmanager;
2. deprecate `datasource.queryTimeAlignment` flag.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5049
2023-10-10 12:41:19 +02:00
Dmytro Kozlov
244c887825
app/vmalert: hide sensetive info in the vmalert (#5059)
Strip sensitive information such as auth headers or passwords from datasource, remote-read, 
remote-write or notifier URLs in log messages or UI. This behavior is by default and is controlled via 
`-datasource.showURL`, `-remoteRead.showURL`, `remoteWrite.showURL` or `-notifier.showURL` cmd-line flags. 

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5044
2023-10-10 11:40:27 +02:00
hagen1778
b57756734e
docs: follow-up after 636d4ea196
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-10 11:25:23 +02:00
Yury Molodov
c5044cdba9
vmui: enhancement of autocomplete feature (#5051)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4993
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3006
2023-10-10 10:38:08 +02:00
Artem Navoiev
636d4ea196 docs: mention the free trial for enterprise
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-10 01:31:48 -07:00
Zakhar Bessarab
7183621d84
docs/enterprise: fix rendering of example alert (#5137)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-10-09 18:57:25 +04:00
Artem Navoiev
dc79b25771
docs: fix markdown for k8s managed guide
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-09 16:25:45 +02:00
hagen1778
cb13ce9147
docs: add YT video to "How to reduce expenses on monitoring"
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-09 11:14:51 +02:00
Artem Navoiev
b3cc22b159 docs: update the license flags description
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-05 10:41:30 -07:00
Dmytro Kozlov
f60c08a7bd
app/(vminsert|vmagent): add support for new relic infrastructure agent (#4712)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-10-05 14:39:51 +02:00
Zakhar Bessarab
cc7d5b7bab
docs: add example service configuration for windows (#5129)
Use service configuration example from https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3781#issuecomment-1424671624

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-10-05 13:55:27 +02:00
Denys Holius
3d5d62e38a
docs: update social links 2023-10-05 13:53:08 +02:00
Github Actions
b576acff5e
Automatic update operator docs from VictoriaMetrics/operator@26d1b3b (#5130) 2023-10-05 12:43:28 +02:00
Aliaksandr Valialkin
09e6606b38
docs/VictoriaLogs/Roadmap.md: add integration with Grafana 2023-10-05 11:40:45 +02:00
Zakhar Bessarab
490694bb01
docs/enterprise: add info about license flags (#5085)
* docs/enterprise: add info about license flags

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/enterprise: add more examples on how to run enterprise components

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/enterprise: address review feedback

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/enterprise: specify release version

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/enterprise: fix image tag in example

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/enterprise: add docker-compose example, fix secret name

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-10-05 13:05:57 +04:00
Artem Navoiev
8b90bb3bc0
docs: victorialogs change the doc ordering.2
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-05 10:40:55 +02:00
Artem Navoiev
c606d3efde
docs: victorialogs change the doc ordering
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-05 10:24:19 +02:00
Github Actions
590bd50df4
Automatic update operator docs from VictoriaMetrics/operator@5da4bf6 (#5127) 2023-10-04 18:51:53 +02:00
Aliaksandr Valialkin
10677af9df
docs/VictoriaLogs/CHANGELOG.md: add release date for v0.4.1-victorialogs 2023-10-04 17:57:04 +02:00
Github Actions
cd0683a3af
Automatic update operator docs from VictoriaMetrics/operator@9fb79f4 (#5124) 2023-10-04 17:03:50 +02:00
Aliaksandr Valialkin
ade64b5783
docs/vmbackup.md: mention that -filestream.disableFadvise command-line flag can be used for reducing CPU usage on systems with big number of CPU cores
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5120

This is a follow-up for 75dd7b30ba
2023-10-04 16:38:15 +02:00
Zakhar Bessarab
d863d11b5a
docs/vmbackupmanager: add missing step for restore flow in k8s (#5116)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-10-04 16:34:18 +02:00
Aliaksandr Valialkin
75dd7b30ba
lib/filestream: add -filestream.disableFadvise syscall for unconditional disabling of fadvise syscall
This may be needed in rare cases when performing backups on systems with big number of CPU cores
and big value passed to -concurrency command-line flag.

See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5120
2023-10-04 16:19:46 +02:00
Artem Navoiev
94bdc91a56
docs: fix wieghts for logs folders
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-04 15:48:54 +02:00
Artem Navoiev
ce17f81b00
docs: remove id clashes for pages, properly cover victorialogs changelog page (#5123)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-04 15:46:28 +02:00
Artem Navoiev
c21863a956
add hugo front matter for operator docs (#5122)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-10-04 15:27:40 +02:00
Aliaksandr Valialkin
61935e3387
deployment/docker: update VictoriaLogs from v0.4.0-victorialogs to v0.4.1-victorialogs
See https://docs.victoriametrics.com/VictoriaLogs/CHANGELOG.html#v041
2023-10-04 14:43:42 +02:00
f41gh7
960f72368a
cut v0.4.1-victorialogs release 2023-10-04 14:29:41 +02:00
Aliaksandr Valialkin
2dd9b69101
docs/Single-server-VictoriaMetrics.md: follow-up for 612ec2df6b
Move the description of JSON line format into a separate chapter

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5083
2023-10-03 17:47:21 +02:00
hagen1778
de651165bd
alerting: account for vmauth component for alerts ServiceDown and TooManyRestarts
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-03 16:45:33 +02:00
Alexander Marshalov
7b35eaa853
hide deprecated docs from menu (#5095) 2023-10-03 14:28:33 +02:00
Zakhar Bessarab
e330ab7ad9
docs/victorialogs: changelog followup for 650d72f3 (#5114)
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-10-03 14:26:03 +02:00
hagen1778
612ec2df6b
docs: add more details explaining JSON line format
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5083

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-03 11:10:49 +02:00
Aliaksandr Valialkin
3d99d4b6c0
deployment: update VictoriaLogs from v0.3.0-victorialogs to v0.4.0-victorialogs
See https://docs.victoriametrics.com/VictoriaLogs/CHANGELOG.html#v040
2023-10-03 02:33:47 +02:00
Aliaksandr Valialkin
53d2e9ac98
docs/VictoriaLogs/CHANGELOG.md: cut v0.4.0-victorialogs 2023-10-03 01:46:33 +02:00
Aliaksandr Valialkin
f13a96f42c
docs/CHANGELOG.md: cut v1.94.0 2023-10-02 22:33:35 +02:00
Aliaksandr Valialkin
0387b0b39f
docs/Articles.md: add a video from GopherCon 2023 for Writing a TSDB from scratch: performance optimization talk 2023-10-02 22:24:57 +02:00
Yury Molodov
f39045eca6
vmui: add storage for query history (#5022)
* vmui: add storage for query history

* docs/vmui: add storage for query history
2023-10-02 21:41:03 +02:00
Roman Khavronenko
a4bd73ec7e
lib/promscrape: make concurrency control optional (#5073)
* lib/promscrape: make concurrency control optional

Before, `-maxConcurrentInserts` was limiting all calls to `promscrape.Parse`
function: during ingestion and scraping. This behavior is incorrect.
Cmd-line flag `-maxConcurrentInserts` should have effect onl on ingestion.

Since both pipelines use the same `promscrape.Parse` function, we extend it
to make concurrency limiter optional. So caller can decide whether concurrency
should be limited or not.

This commit makes c53b5788b4
obsolete.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Revert "dashboards: move `Concurrent inserts` panel to Troubleshooting section"

This reverts commit c53b5788b4.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-02 21:32:11 +02:00
Dmytro Kozlov
34961dd4b8
app/vmagent: fix check of the DataDog agent path requests when requests have trailing slashes (#5106)
* app/vmagent: fix check of the DataDog agent path requests when requests have trailing slashes

* app/vmagent: fix CHANGELOG.md description

* wip

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-10-02 21:18:03 +02:00
Aliaksandr Valialkin
859977d591
Revert "lib/promscrape: add metric vm_promscrape_scrapes_skipped_total (#5074)"
This reverts commit 74301cdbf5.

Reason for revert:

vmagent already provides better approach for detecting slow scrape targets via the following query:

    scrape_duration_seconds / scrape_timeout_seconds > 1

This query depends on automatically generated per-target metrics.
See https://docs.victoriametrics.com/vmagent.html#automatically-generated-metrics for more details.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5074
2023-10-02 20:59:56 +02:00
Aliaksandr Valialkin
71668637ce
app/vmselect/promql: follow-up for 896c85a4a4
- Clarify the description of the change at docs/CHANGELOG.md
- Make sure that bitmap_*(X, NaN) returns NaN

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4996
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5021
2023-10-02 20:08:26 +02:00
Aliaksandr Valialkin
10f297e1e7
docs/Cluster-VictoriaMetrics.md: increase the minimum supported version of Go builder from 1.18 to 1.20
See the related commit 3da493ff62
2023-10-02 19:34:17 +02:00
Aliaksandr Valialkin
8dce4eb189
lib/logstorage: follow-up for 94627113db
- Move uniqueFields from rows to blockStreamMerger struct.
  This allows localizing all the references to uniqueFields inside blockStreamMerger.mustWriteBlock(),
  which should improve readability and maintainability of the code.

- Remove logging of the event when blocks cannot be merged because they contain more than maxColumnsPerBlock,
  since the provided logging didn't provide the solution for the issue with too many columns.
  I couldn't figure out the proper solution, which could be helpful for end user,
  so decided to remove the logging until we find the solution.

This commit also contains the following additional changes:

- It truncates field names longer than 128 chars during logs ingestion.
  This should prevent from ingesting bogus field names.
  This also should prevent from too big columnsHeader blocks,
  which could negatively affect search query performance,
  since columnsHeader is read on every scan of the corresponding data block.

- It limits the maximum length of const column value to 256.
  Longer values are stored in an ordinary columns.
  This helps limiting the size of columnsHeader blocks
  and improving search query performance by avoiding
  reading too long const columns on every scan of the corresponding data block.

- It deduplicates columns with identical names during data ingestion
  and background merging. Previously it was possible to pass columns with duplicate names
  to block.mustInitFromRows(), and they were stored as is in the block.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4969
2023-10-02 19:19:08 +02:00
Dmytro Kozlov
60694606f1
docs: add clarification of the retention filter usage (#5103)
docs: add clarification of the retention filter usage

Updated documentation regarding retention filter usage if duration is set lower than
`-retentionPeriod` flag value.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-10-02 17:17:16 +02:00
Roman Khavronenko
74301cdbf5
lib/promscrape: add metric vm_promscrape_scrapes_skipped_total (#5074)
* lib/promscrape: add metric `vm_promscrape_scrapes_skipped_total`

add metric `vm_promscrape_scrapes_skipped_total`to show whether vmagent skips the scrapes.
This could happen if vmagent is overloaded or target is responding too slow for configured `scrape_interval`.

The follow-up commit should add a corresponding alerting rule and panel to vmagent dashboard.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* deployment/docker: add `TooManyScrapeSkips` alerting rule for vmagent

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* dashboards: add panels `Scrape duration 0.99 quantile` and `Skipped scrapes` to vmagent dashboard

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-10-02 17:12:12 +02:00
Github Actions
256d800200
Automatic update operator docs from VictoriaMetrics/operator@44bdc27 (#5104) 2023-10-02 15:50:18 +02:00
Github Actions
62de314510
Automatic update operator docs from VictoriaMetrics/operator@c7125bd (#5102) 2023-10-02 14:50:08 +02:00
Aliaksandr Valialkin
bf6ebc86fd
docs/VictoriaLogs/CHANGELOG.md: remove duplicate lines about vl_http_request_duration_seconds metric
This is a follow-up after 8a23d08c21

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4945
2023-10-01 23:32:05 +02:00
Aliaksandr Valialkin
7373d04d54
docs/Single-server-VictoriaMetrics.md: refer to active queries and top queries pages at VMUI instead of refering to the corresponding HTTP endpoints
"Active queries" and "Top queries" pages at VMUI are user-friendly than the corresponding HTTP endpoints
2023-10-01 21:54:37 +02:00
Aliaksandr Valialkin
7af9be92cc
docs/vmalert.md: refer to -evaluationInterval command-line flag instead of evaluation_interval option, which isnt supported by vmalert
This is follow-up for 5c42c1218a
2023-10-01 21:54:29 +02:00
Aliaksandr Valialkin
cc3b1267f5
docs/Troubleshooting.md: describe how to optimize SLI/SLO queries with long lookbehind windows 2023-10-01 21:54:23 +02:00
Aliaksandr Valialkin
5e49b72126
docs/CHANGELOG.md: follow-up for f0e33700fc
Mention that the statistic inaccuracy is related to cardinality explorer
2023-10-01 21:33:31 +02:00
Aliaksandr Valialkin
859859aa1c
app/vmagent: follow-up for cfef814750
- Properly handle /insert/multitenant/api/put url for opentsdb handler at vmagent
- Document that the bug has been introduced in v1.93.2 at docs/CHANGELOG.md
- Add a link to multitenant url docs in bugfix description

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5061
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4910
2023-10-01 21:09:32 +02:00
Dmytro Kozlov
896c85a4a4
app/vmselect: fix bitmap_*() functions behavior (#5021)
Related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4996

Signed-off-by: dmitryk-dk d.kozlov@victoriametrics.com

Signed-off-by: dmitryk-dk d.kozlov@victoriametrics.com
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-09-29 12:03:01 +02:00
Zakhar Bessarab
94627113db
lib/logstorage: prevent from panic during background merge (#4969)
* lib/logstorage: prevent from panic during background merge

Fixes panic during background merge when resulting block would contain more columns than maxColumnsPerBlock.
Buffered data will be flushed and replaced by the next block.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/logstorage: clarify field description and comment

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-29 11:58:20 +02:00
Zakhar Bessarab
8a23d08c21
lib/logstorage: switch to read-only mode when running out of disk space (#4945)
* lib/logstorage: switch to read-only mode when running out of disk space

Added support of `--storage.minFreeDiskSpaceBytes` command-line flag to allow graceful handling of running out of disk space at `--storageDataPath`.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4737
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/logstorage: fix error handling logic during merge

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/logstorage: fix log level

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-09-29 11:55:38 +02:00
Dmytro Kozlov
f0e33700fc
vmui: update information about tsdb usage in cluster version (#5004)
* vmui: update information about tsdb usage in cluster version

* vmui: cleanup

* vmui: add CHANGELOG.md

* vmui: cleanup

* vmui: update logic, move information to the visible place

* app/vmui: remove values fetch, update documentation for cardinality explorer

* app/vmui: update CHANGELOG.md
2023-09-29 11:47:45 +02:00
Zakhar Bessarab
0adec48182 doc: address review feedback
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-28 06:21:03 -07:00
Zakhar Bessarab
1a834b4210 doc: mention InfluxDB v2 HTTP API support
Address: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5076
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-28 06:21:03 -07:00
Github Actions
af53799332
Automatic update operator docs from VictoriaMetrics/operator@958ce2b (#5070) 2023-09-27 09:04:33 +08:00
hagen1778
c53b5788b4
dashboards: move Concurrent inserts panel to Troubleshooting section
Moved because this panel is related to both: scraped and ingested data.
Before, it could have give a misleading impression that it is related to ingested metrics only.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-09-26 14:26:40 +02:00
Alexander Marshalov
34a9d1f818
fixed ingestion via multitenant url for opentsdbhttp (#5061) (#5064) 2023-09-26 11:18:34 +02:00
Roman Khavronenko
4d1b572f46
Docker add vmauth (#5057)
* docker-compose: add vmauth to cluster env

vmauth acts as a balancer and used as an example of how to interconnect
VM components via vmauth.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docker-compose: add vmauth to cluster env

vmauth acts as a balancer and used as an example of how to interconnect
VM components via vmauth.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-09-26 10:50:10 +02:00
Aliaksandr Valialkin
f897d5241d
docs/vmagent.md: make VictoriaMetrics remove_write protocol more visible by mentioning it at the top of the page 2023-09-25 17:42:16 +02:00
Aliaksandr Valialkin
89a5e27216
docs: run make docs-sync after 8e722e10ee 2023-09-25 17:35:29 +02:00
Aliaksandr Valialkin
8e722e10ee
docs/CaseStudies.md: add Criteo case study
This is a follow-up for bdbe616408

See https://medium.com/criteo-engineering/victoriametrics-a-prometheus-remote-storage-solution-57081a3d8e61
2023-09-25 17:34:19 +02:00
Aliaksandr Valialkin
717c53af27
lib/storage: stop exposing vm_merge_need_free_disk_space metric
This metric confuses users and has no any useful information.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686#issuecomment-1733844128
2023-09-25 16:52:39 +02:00
Aliaksandr Valialkin
3b9605dba5
app/vmselect/promql: do not sort q1 or q2 results
This makes sure that `q2` series are returned after `q1` series in the same way as Prometheus does

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4763
2023-09-25 16:14:16 +02:00
Aliaksandr Valialkin
a740159541
app/vmselect/promql: completely substitute median_over_time() WITH template with regular median_over_time() rollup function
This is a follow-up for 34d7a670d0

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5034
2023-09-25 15:28:12 +02:00
Zakhar Bessarab
34d7a670d0
app/vmselect/promql: add implementation of median_over_time for rollup functions list (#5042)
`median_over_time` is handled by predefined WITH template in MetricsQL library which translates it to `quantile_over_time(0.5)`
This makes it impossble to use `median_over_time` as a usual rollup function for `aggr_over_time`.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5034

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-25 14:01:00 +02:00
Roman Khavronenko
ec50375991
docs/changelog: add link to sandbox (#5050)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-09-25 14:00:41 +02:00
Github Actions
33c17a5a2b
Automatic update operator docs from VictoriaMetrics/operator@587ea54 (#5054) 2023-09-25 12:42:05 +02:00
hagen1778
dd98385a10
docs/articles: add link to "How to reduce expenses on monitoring" slides
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-09-22 17:21:32 +02:00
Aliaksandr Valialkin
72960732c2
docs/Cluster-VictoriaMetrics.md: update -help output for enterprise components 2023-09-22 13:51:32 +02:00
Zakhar Bessarab
8d99c12a7d
lib/promscrape/discovery/kubernetes: supress context.Cancelled error in logs (#5048)
lib/promscrape/discovery/kubernetes: supress context.Cancelled error in logs

It is possible that context.Cancelled will appear after k8s watcher was closed due to reload(see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4850).

Logging an error misinforms user and looks like vmagent discovery will stop working even though this does not affect discovery.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-22 13:01:33 +02:00
Zakhar Bessarab
455077cd67
docs/vmbackup: update docs for different authentication options, add examples (#5046)
Updates: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5023

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-22 11:19:19 +02:00
Github Actions
71a17dfcbe
Automatic update operator docs from VictoriaMetrics/operator@9d65e09 (#5040) 2023-09-21 13:20:55 +02:00
Zakhar Bessarab
760cdcec68
lib/backup: fix issue with inconsistent copying of appliedRetention.txt (#5027)
* lib/backup: fix issue with inconsistent copying of appliedRetention.txt

appliedRetention.txt can be modified in place, so it should be always copied just the same as parts.json

Updates: https://github.com/victoriaMetrics/victoriaMetrics/issues/5005
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: add changelog entry for appliedRetention.txt copying fix

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-21 11:25:19 +02:00
Roman Khavronenko
462c918251
app/vmauth: update config reload routine (#5019)
* expose metrics `vmauth_config_last_reload_*` for tracking the state of config reloads, similarly to vmagent/vmalert components.
* do not print logs like `SIGHUP received...` once per configured `-configCheckInterval` cmd-line flag. This log will be printed only if config reload was invoked manually.
*  prevent configuration reloading if there were no changes in config. This improves memory usage when `-configCheckInterval` cmd-line flag is configured and config has extensive list of regexp expressions requiring additional memory on parsing.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-09-20 15:04:52 +02:00
Aliaksandr Valialkin
28aed4d098
docs/CHANGELOG.md: publish changes for v1.93.5 2023-09-19 10:50:25 +02:00
hagen1778
bdbe616408
docs/articles: add https://medium.com/criteo-engineering/victoriametrics-a-prometheus-remote-storage-solution-57081a3d8e61
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-09-19 10:03:40 +02:00
Aliaksandr Valialkin
582f1f8fda
docs/CHANGELOG.md: clarify the description of bugfixes at f7dda12b4d and b6ad581b45
This is a follow-up for 8b01bc4a5c

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4999
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5009
2023-09-19 00:22:32 +02:00
Aliaksandr Valialkin
e9647bb669
app/vlinsert: follow-up for d570763c91
- Switch from summary to histogram for vl_http_request_duration_seconds metric.
  This allows calculating request duration quantiles across multiple hosts
  via histogram_quantile(0.99, sum(vl_http_request_duration_seconds_bucket) by (vmrange)).
- Take into account only successfully processed data ingestion requests
  when updating vl_http_request_duration_seconds histogram.
  Failed requests are ignored, since they may significantly skew measurements.
- Clarify the description of the change at docs/VictoriaLogs/CHANGELOG.md.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4934
2023-09-19 00:02:43 +02:00
Aliaksandr Valialkin
76af32d869
lib/promscrape/discovery/kubernetes: follow-up after eeb862f3ff
- Move the bugfix description to the correct place in docs/CHANGELOG.md
- Prevent from logging of 'context canceled' errors after the url watcher is stopped,
  since these errors are expected and may confuse users.
- Remove unused urlWatcher.refCount field.
- Remove unused urlWatcher.close() method.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4850
2023-09-18 17:06:39 +02:00
Aliaksandr Valialkin
4d01bc6d52
lib/backup: properly copy parts.json files inside indexdb directory additional to data directory
This is a follow-up for 264ffe3fa1

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5005
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5006
2023-09-18 16:16:50 +02:00
Aliaksandr Valialkin
f267733d9a
app/vlinsert/insertutils: cosmetic changes after 8d3e574c31 2023-09-18 11:59:46 +02:00
Nikolay
8b01bc4a5c
docs: reflect recent changes at change logs (#5015) 2023-09-18 08:22:10 +02:00
crossoverJie
d570763c91
app/vlinsert: Add vl_http_request_duration_seconds metrics (#4934) 2023-09-16 15:10:29 +02:00
Zakhar Bessarab
eeb862f3ff
lib/promscrape/discovery/kubernetes: fix leaking api watcher (#4861)
* lib/promscrape/discovery/kubernetes: fix leaking api watcher

goroutine which was polling k8s API had no execution control. This leaded to leaking goroutines during config reload.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4850
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape/discovery/kubernetes: use reference counting for urlWatcher cleanup

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape/discovery/kubernetes: remove waitgroup sync for goroutines polling API server

This is unnecessary since context will is cancelled and new requests will not be sent. Also, using waitgroup will increase time required to perform reload which might result in missed scrapes.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/promscrape/discovery/kubernetes: clarify comment

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Apply suggestions from code review

* lib/promscrape/discovery/kubernetes: address review feedback

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-09-15 19:40:13 +02:00
Zakhar Bessarab
264ffe3fa1
lib/backup: force copying of parts.json (#5006)
* lib/backup: force copying of parts.json

Copying of parts.json is required because `part.key()` comparison can create same key value for files with different contents. This will result in inconsistent backup being created or restored.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5005
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/backup: ensure parts.json is only copied once

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2023-09-15 19:04:38 +02:00
Zakhar Bessarab
8d3e574c31
app/vlinsert: add flag to limit amount of fields per line (#4976)
Adding limit on ingestion allows to avoid issues like this one https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762
Such issues are often caused by misconfigurtion on log persing/ingestion side and preventing such rows from being ingested allows to avoid performance implications created by storing such log rows.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-15 15:18:38 +02:00
Zakhar Bessarab
2a362e7397
docs: add changelog entry for downsampling.period and dedup.minScrapeInterval verification (#5000)
* docs: add changelog entry for downsampling.period and dedup.minScrapeInterval verification

- added changelog entry
- documented requirements for dedup.minScrapeInterval and downsampling.period being multiples of each other

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: `make docs-sync`

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-15 15:14:16 +02:00
Dmytro Kozlov
d5f9619984
vmagent: add validation of MetricsQL functions (#4991)
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-09-15 13:15:23 +02:00
Github Actions
9f3f085d7f
Automatic update operator docs from VictoriaMetrics/operator@49d245d (#5002) 2023-09-13 22:33:00 +08:00
Github Actions
d6cdfd231a
Automatic update operator docs from VictoriaMetrics/operator@a9c54f4 (#4998) 2023-09-13 06:03:46 +08:00
Alexander Marshalov
eebef296d4
Updates in release guide (#4956)
new helm release flow and operator release flow

Signed-off-by: Alexander Marshalov <_@marshalov.org>
2023-09-11 09:55:30 +02:00
Github Actions
9902418524
Automatic update operator docs from VictoriaMetrics/operator@21f7203 (#4988) 2023-09-11 09:40:46 +02:00
Aliaksandr Valialkin
151f363552
docs/CHANGELOG.md: document v1.87.9 2023-09-10 21:41:23 +02:00
Aliaksandr Valialkin
bb8eda0b0f
docs/CHANGELOG.md: document v1.93.4 2023-09-10 19:47:38 +02:00
Aliaksandr Valialkin
9bdcf483a7
docs/vmrestore.md: run make docs-sync after bf4936bbf0 2023-09-08 23:28:44 +02:00
Aliaksandr Valialkin
0bbc6a5b43
app/vmagent/remotewrite: fix data race when extra labels are added to samples before sending them to multiple remote storage systems
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4972
2023-09-08 23:24:00 +02:00
Aliaksandr Valialkin
a315694dd9
app/vmauth: add ability to specify response status codes for retrying requests during load-balancing
Response status codes for retrying can be specified via retry_status_codes list

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893
2023-09-08 23:23:15 +02:00
Zakhar Bessarab
bf4936bbf0
docs: sync description for license flags (#4977)
- update eula flag to add deprecation notice
- add new license flags description

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-09-08 17:09:34 +04:00
Roman Khavronenko
6351d07da8
vmalert: correctly add duplicated params to the query (#4955)
Fix the bug when Group's `params` fields with multiple values were
overriding each other instead of adding up.
The bug was introduced in this commit eccecdf177
 starting from v1.91.1 https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.91.1

 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4908

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-09-08 09:32:48 +02:00
Aliaksandr Valialkin
b80d338287
app/vmauth: retry requests at other backends on 5xx response status codes
This should allow implementing high availability scheme described at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4792#issuecomment-1674338561

See also https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4893
2023-09-08 00:46:37 +02:00
Aliaksandr Valialkin
dd10f94951
app/vmselect: return 503 status code when partial responses are denied and some of vmstorage nodes are temporarily unavailable
This should help detecting this case and automatic retrying the query at healthy cluster replica
in another availability zone.

This commit is needed as a preparation for automatic query retry at another backend at vmauth on 5xx errors
as described at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4792#issuecomment-1674338561
2023-09-07 16:11:39 +02:00
Aliaksandr Valialkin
9de440c803
lib/logger: increase the maximum log arg size from 200 to 500
The 200 chars limit has been appeared too small for typical log messages emitted by VictoriaMetrics components

This is a follow-up for 87fea7d8ac
2023-09-07 16:11:08 +02:00
Aliaksandr Valialkin
87fea7d8ac
lib/logger: limit the maximum arg length, which can be emitted to log lines
This should prevent from emitting too long lines when too long args are passed to logger.* functions.
For example, too long MetricsQL queries or too long data samples.
2023-09-07 15:22:46 +02:00
Github Actions
78dfbe1622
Automatic update operator docs from VictoriaMetrics/operator@83c07ed (#4970) 2023-09-07 15:09:28 +02:00
Aliaksandr Valialkin
3da493ff62
go.mod: increase the minimum supported Go version from Go1.19 to Go1.20 2023-09-07 13:01:38 +02:00
Github Actions
72d3063bef
Automatic update operator docs from VictoriaMetrics/operator@9c461d2 (#4968) 2023-09-07 12:26:02 +02:00
Aliaksandr Valialkin
9bccc5aab2
docs/CHANGELOG.md: return back accidentally deleted line at 45c0e4bb31 2023-09-07 12:03:04 +02:00
Aliaksandr Valialkin
2dc33e0ddc
all: update Go builder from Go1.21.0 to Go1.21.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.21.1+label%3ACherryPickApproved
2023-09-07 11:36:16 +02:00
Aliaksandr Valialkin
5f85dd7f80
docs/CHANGELOG.md: clarify the scope of recent bugfixes 2023-09-07 11:25:11 +02:00
Aliaksandr Valialkin
448baf12a3
deployment/docker: properly build armv5 production builds for GOARCH=arm
Pass GOARM=5 when building GOARCH=arm production builds, since the default value for this env var
has been changed to GOARM=6 since Go1.21.0.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4965
and https://github.com/golang/go/issues/62475
2023-09-07 11:18:53 +02:00
hagen1778
40c94b26dd
docs: mention that quantiles can't be used in sharded mode
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4942
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-09-07 10:06:09 +02:00
Haleygo
45c0e4bb31
vmalert: add eval_offset for group (#4693)
Adds `eval_offset` attribute for Groups. 
If specified, Group will be evaluated at the exact time offset on the range of [0...evaluationInterval]. 
The setting might be useful for cron-like rules which must be evaluated at specific moments of time. 

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3409

Signed-off-by: Haley Wang <pipilong.25@gmail.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2023-09-06 16:29:59 +02:00
Aliaksandr Valialkin
138e02da37
docs/CHANGELOG.md: document the bugfix at 7db72dd7e6
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4947
2023-09-06 12:17:21 +02:00
Yury Molodov
7b92f1d038
vmui: fix render heatmap (#4957) 2023-09-06 10:26:45 +02:00
Aliaksandr Valialkin
2e04ddbd32
docs/CaseStudies.md: update Grammarly case study with the newly published article https://www.grammarly.com/blog/engineering/monitoring-with-victoriametrics/
Follow-up c0246b2e17
2023-09-05 17:03:40 +02:00
Artem Navoiev
7be3848ee6
use correct abbriviation for ESA legal doc
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-09-05 11:01:45 +02:00
Artem Navoiev
42baf5fe3f change link to the enterprise legal doc
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-09-05 01:53:36 -07:00
Github Actions
e04c435875
Automatic update operator docs from VictoriaMetrics/operator@b63f6e9 (#4946) 2023-09-04 18:19:38 +02:00
hagen1778
f9e47a9abe
docs: fix broken link in vmctl references
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-09-04 12:45:46 +02:00
Yury Molodov
d19072a2d9
feat: add the option to see the latest queries (#4718) (#4759)
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-09-04 11:29:11 +02:00
hagen1778
c0246b2e17
docs: add How Grammarly Improved Monitoring by Over 10x with VictoriaMetrics
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-09-04 10:18:35 +02:00
Dima Lazerka
0c7d46d637
flagutil: Make .Msecs private (#4906)
* Introduce flagutil.Duration

To avoid conversion bugs

* Fix tests

* Clarify documentation re. month=31 days

* Add fasttime.UnixTime() to obtain time.Time

The goal is to refactor out the last usage of `.Msecs`.

* Use fasttime for time.Now()

* wip

- Remove fasttime.UnixTime(), since it doesn't improve code readability and maintainability
- Run `make docs-sync` for syncing changes from README.md to docs/ folder
- Make lib/flagutil.Duration.Msec private
- Rename msecsPerMonth const to msecsPer31Days in order to be consistent with retention31Days

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-09-03 10:33:37 +02:00
Github Actions
26cee36499
Automatic update operator docs from VictoriaMetrics/operator@add9045 (#4941) 2023-09-02 17:19:32 +02:00
Github Actions
ff4bc9d18e
Automatic update operator docs from VictoriaMetrics/operator@c32931b (#4940) 2023-09-02 16:20:46 +02:00
Aliaksandr Valialkin
7a716095dc
docs/CHANGELOG.md: document 1.93.3 release 2023-09-02 10:21:20 +02:00
Aliaksandr Valialkin
82ccae1c02
docs/CHANGELOG.md: document v1.87.8 2023-09-02 01:54:07 +02:00