Commit Graph

9168 Commits

Author SHA1 Message Date
Artem Navoiev
19911bb458
docs: clarify operator name
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-09-20 17:35:27 +02:00
Hui Wang
d2b98245ea
vmalert: fix variable $activeAt value when templating rule annotation in replay mode
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-09-20 17:34:54 +02:00
hagen1778
e3a1eaab87
docs: fix more typos in the changelog
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-20 17:34:28 +02:00
hagen1778
a99cf73eac
docs: rm update node about loggerMaxArgLen as it doesn't have incompatibility effect
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-20 17:34:27 +02:00
hagen1778
5b1015de4c
docs: fix typo in link in change line about NaN
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-20 17:33:43 +02:00
Thomas Danielsson
3e5b80244a
docs: fix typo in the changelog
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-09-20 17:33:43 +02:00
Aliaksandr Valialkin
472b6b326e
lib/logstorage: make sure that getCommonTokens returns common tokens in the original order of tokens inside tokenSets arg
This fixes flaky test TestGetCommonTokensForOrFilters:

    filter_or_test.go:143: unexpected tokens for field "_msg"; got ["foo" "bar"]; want ["bar" "foo"]
2024-09-19 16:00:21 +02:00
hagen1778
d3e2e55f3f
app/{vmselect,vlselect}: run make vmui-update vmui-logs-update
Executed after https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6972
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6900

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit c00b64726c)
2024-09-19 15:48:40 +02:00
Yury Molodov
47b4f25c7b
vmui: optimize public directory by cleaning up files ()
### Describe Your Changes

### Pull Request Description:

1. **HTML File Structure Optimization**: Adjusted the location of HTML
files for different builds to prevent redundant files in the final
output. See issue 
2. **Metadata Fixes**: Corrected metadata in HTML files for each build
configuration.
3. **Favicon Update**: Replaced PNG favicon (`14 KB` and `1.58 KB`) with
SVG (`1.35 KB`).
4. **Social Media Optimization**: Optimized the social preview image,
reducing its size by `60.2 KB`.
5. **Git Ignore Update**: Added `public/index.html` to `.gitignore` as
it is dynamically generated during the build process.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 4e976f66f3)
2024-09-19 15:48:10 +02:00
Thomas Danielsson
4f93422064
docs: typo filebeat -> fluent-bit ()
### Describe Your Changes

Fix a typo; `filebeat.yml` -> `fluent-bit.conf`

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Thomas Danielsson <thomas@elajt.se>
(cherry picked from commit 69b976f08e)
2024-09-19 15:48:09 +02:00
Yury Molodov
5a905e2e94
vmui: change the query_range request method from GET to POST ()
### Describe Your Changes

change the `/query_range` and `/query` requests method from `GET` to
`POST`. See .

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit b0bdb92729)
2024-09-19 15:48:09 +02:00
Roman Khavronenko
e6dac18db3
lib/logger: increase default value of -loggerMaxArgLen cmd-line fla… ()
…g from 1e3 to 5e3

This should improve visibility on errors produced by very long queries.

The change is classified as BUG in order to port it to LTS releases.

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Mathias Palmersheim <mathias@victoriametrics.com>
(cherry picked from commit e115b85770)
2024-09-19 15:48:09 +02:00
Yury Molodov
0f0d5002e6
vmui: update dependencies in package.json to latest versions ()
Update dependencies in `package.json` to latest versions

(cherry picked from commit 7491f49e9e)
2024-09-19 15:48:08 +02:00
Yury Molodov
868fa988b3
vmui/logs: add auto refresh ()
### Describe Your Changes

Add auto refresh


![image](https://github.com/user-attachments/assets/20ed1102-d5e4-4d3f-9c24-7d298d93400a)

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit bc9cb69170)
2024-09-19 15:48:08 +02:00
Thomas Danielsson
066137feaa
typo: added missing '{' ()
### Describe Your Changes

Added missing `{` in vmalert rule.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 75cc7922c3)
2024-09-19 15:48:08 +02:00
Aliaksandr Valialkin
7e0ff1ee46
app/vlselect/logsql: call Query.Optimize() on the cloned query in order to replace * filter with filterNoop inside getLastNQueryResults()
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785

(cherry picked from commit e86891b010)
2024-09-19 15:48:07 +02:00
Aliaksandr Valialkin
4e00e4428e
app/vmselect/promql: properly calculate c1 and c2 and c1 or c2 by upgrading github.com/VictoriaMetrics/metricsql to v0.79.0
The fix is in the https://github.com/VictoriaMetrics/metricsql/pull/34
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6637

(cherry picked from commit b82e2cabc5)
2024-09-19 15:48:06 +02:00
Github Actions
8e5e772af4
Automatic update operator docs from VictoriaMetrics/operator@fe90f57 ()
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
(cherry picked from commit 8bc30b68cd)
2024-09-19 15:48:06 +02:00
Dmytro Kozlov
ab86aad292
docs/victoriametrics-cloud: updated pictures and descriptions with new functionality ()
Improved VictoriaMetrics documentation for cloud

Related issue: https://github.com/VictoriaMetrics/cloud/issues/2143

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 4c228f1e18)
2024-09-19 15:48:06 +02:00
Alexander Marshalov
c0e764d4a4
[vmcloud]: Cloud tiers assumptions docs ()
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit c0272463d9)
2024-09-19 15:46:53 +02:00
Artem Navoiev
bcab5861ae
victorialogs: add HA example for logstash and fluentbit ()
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 277fed9990)
2024-09-19 15:46:52 +02:00
Artem Navoiev
1ad6e8df74
docs: adjust image size
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 153cceb124)
2024-09-19 15:46:52 +02:00
Artem Navoiev
382444aebb
docs: fix links to docker
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit ae5f98f46b)
2024-09-19 15:46:52 +02:00
Artem Navoiev
947b9d3e8d
docs: victorialogs just mention telegraf as we support not only http but more
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 71df11dfbb)
2024-09-19 15:46:51 +02:00
Artem Navoiev
c073206f0c
docs: update roadmap, mark telegraf and otel as done and link the docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 47cd7bb4c1)
2024-09-19 15:46:51 +02:00
f41gh7
65798f0f8d
docs/changelog: mention vmagent kafka consumer bugfix
Changes were made to the enteprise repository
2024-09-19 15:36:04 +02:00
Nikolay
6f99dcc7c1
lib/storage: consistently check for missing metricID index records ()
* Previously, only metricID->metricName missing index records were
tracked with deadline But it was possible a case for missing
metricID->TSID index records. IndexDB metrics fix exposed misleading
metric for such missing records.

* This commit adds check for metricID->TSID missing index records. And
delete missing metricID entry if it hit 60 second deadline.

Related issue
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6931

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-16 13:07:37 +02:00
Nikolay
c32032ac1b
lib/fs: properly call windows APIs ()
Previously we manually imported system windows DDLs
and made direct syscall.

 But golang exposes syscall wrappers with sys/windows package.
It seems, that direct syscall was broken at 1.23 golang release. It was
`GetDiskFreeSpace` syscall in our case.

This commit replaces all manual syscalls with wrappers

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6973

Related golang issue:
https://github.com/golang/go/issues/69029

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-13 13:19:04 +02:00
Dima Lazerka
465c7ad045
docs: fixes misspelled typos
Also tried to make it catch "Authorisation" in the future, fixed a lot
of other misspells along the way, but didn't make it catch
"Authorisation" anyway.

- Fix misspelled "Authorization" header name
- Fix misspelled "organization"
- Fix more misspells
2024-09-13 13:19:03 +02:00
Aliaksandr Valialkin
213cd045fc
docs/VictoriaLogs/querying/README.md: typo fixes: use field_values instead field_names where needed 2024-09-11 09:42:42 +02:00
Hui Wang
c7fc0d0d2f
vmalert: do not send message to alertmanager when alert has no label … ()
…pair

`alert_relabel_configs` in [notifier
config](https://docs.victoriametrics.com/vmalert/#notifier-configuration-file)
can drop alert labels when used to filter different tenant alert message
to different notifier.
alertmanager would report error like `msg="Failed to validate alerts"
err="at least one label pair required"` in this case, but the rest of
the alerts inside one request would still be valid in alertmanager, so
it's not severe.

(cherry picked from commit ae4d376e41)
2024-09-09 16:06:44 +02:00
Artem Navoiev
d5cfce693a
docs: Add VictoriaLogs HA example for single nodes ()
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit f9a8c09fe8)
2024-09-09 16:06:43 +02:00
hagen1778
30f34b28d6
docs: mention -dryRun flag next to config example in vmalert
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 7983ea4edb)
2024-09-09 16:06:43 +02:00
Aliaksandr Valialkin
c68c7c0d1a
deployment: update VictoriaLogs from v0.28.0-victorialogs to v0.29.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.29.0-victorialogs
2024-09-08 21:23:29 +02:00
Aliaksandr Valialkin
32c0e02958
docs/VictoriaLogs/CHANGELOG.md: cut v0.29.0 2024-09-08 21:11:34 +02:00
Aliaksandr Valialkin
21dfc36398
vendor: run make vendor-update 2024-09-08 21:05:23 +02:00
Aliaksandr Valialkin
5a9abe9783
Makefile: upgrade go mod tidy compatibility mode from Go1.22 to Go1.23
This is a follow-up for 1b9f3b39b4
2024-09-08 21:05:22 +02:00
Aliaksandr Valialkin
69171b3075
Makefile: remove '-d' flag from 'go get', since this flag is deprecated
See https://go.dev/doc/go-get-install-deprecation
2024-09-08 20:11:24 +02:00
Aliaksandr Valialkin
8eef397d29
deployment/docker: update base Alpine docker image from 3.20.2 to 3.20.3
See https://alpinelinux.org/posts/Alpine-3.17.10-3.18.9-3.19.4-3.20.3-released.html
2024-09-08 19:27:05 +02:00
Aliaksandr Valialkin
cad236003b
app/vlselect: consistently reuse the original query timestamp when executing /select/logsql/query with positive limit=N query arg
Previously the query could return incorrect results, since the query timestamp was updated with every Query.Clone() call
during iterative search for the time range with up to limit=N rows.

While at it, optimize queries, which find low number of matching logs, while spend a lot of CPU time for searching
across big number of logs. The optimization reduces the upper bound of the time range to search if the current time range
contains zero matching rows.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785
2024-09-08 14:34:46 +02:00
Aliaksandr Valialkin
297301e8c0
lib/logstorage: preserve the order of tokens to check against bloom filters in AND filters
Previously tokens from AND filters were extracted in random order. This could slow down
checking them agains bloom filters if the most specific tokens go at the beginning of the AND filters.
Preserve the original order of tokens when matching them against bloom filters,
so the user could control the performance of the query by putting the most specific AND filters
at the beginning of the query.

While at it, add tests for getCommonTokensForAndFilters() and getCommonTokensForOrFilters().

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6554
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6556
2024-09-08 12:28:34 +02:00
Aliaksandr Valialkin
4b49b62a58
lib/logstorage: improve error logging for incorrect queries passed to /select/logsql/stats_query and /select/logsql/stats_query_range functions 2024-09-08 12:28:33 +02:00
Aliaksandr Valialkin
edb1afe804
lib/logstorage: properly extract common tokens from unsupported OR filters
Previously the following query could miss rows matching !bar if these rows do not contain foo:

   foo OR !bar

This is because of incorrect detection of common tokens for OR filters - all the unsupported filters
were skipped (including the NOT filter (aka `!`)), while in this case zero common tokens must be returned.

While at it, move repetiteve code in TestFilterAnd and TestFilterOr into f function.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6554
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6556
2024-09-08 12:28:33 +02:00
Aliaksandr Valialkin
c448189f69
app/vlselect: add /select/logsql/stats_query_range endpoint for building time series panels in VictoriaLogs plugin for Grafana
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6943
Updates https://github.com/VictoriaMetrics/victorialogs-datasource/issues/61
2024-09-07 00:44:34 +02:00
Aliaksandr Valialkin
01c8e12370
app/vlselect: add /select/logsql/stats_query endpoint, which is going to be used by vmalert
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6942
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6706
2024-09-06 23:00:58 +02:00
Aliaksandr Valialkin
e90e809c00
deployment: update Go builder from Go1.23.0 to Go1.23.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.23.1+label%3ACherryPickApproved
2024-09-06 22:57:56 +02:00
Aliaksandr Valialkin
c5badeea08
lib/logstorage: substitute | operator with or operator at math pipe
This is needed for avoiding confusion between the `|` operator at `math` pipe and `|` pipe delimiter.
For example, the following query was parsed unexpectedly:

   * | math foo / bar | fields x

as

   * | math foo / (bar | fields) as x

Substituting `|` with `or` inside `math` pipe fixes this ambiguity.
2024-09-06 22:43:29 +02:00
f41gh7
395894688c
app/*/multiarch: return back empty value for TARGETARCH
follow-up after 91456ab5bb

docker buildx uses special variables, such as TARGETARCH and it shouldn't be overwritten.

 See this article for details
https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-06 18:15:22 +02:00
f41gh7
b9d9aad85a
docs/changelog: mention storage changes
After a5424e95b3

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-09-06 18:13:48 +02:00
Artem Fetishev
85bf768013
lib/storage: adds metrics that count records that failed to insert
### Describe Your Changes

Add storage metrics that count records that failed to insert:

- `RowsReceivedTotal`: the number of records that have been received by
the storage from the clients
- `RowsAddedTotal`: the number of records that have actually been
persisted. This value must be equal to `RowsReceivedTotal` if all the
records have been valid ones. But it will be smaller otherwise. The
values of the metrics below should provide the insight of why some
records hasn't been added
-   `NaNValueRows`: the number of records whose value was `NaN`
- `StaleNaNValueRows`: the number of records whose value was `Stale NaN`
- `InvalidRawMetricNames`: the number of records whose raw metric name
has failed to unmarshal.

The following metrics existed before this PR and are listed here for
completeness:

- `TooSmallTimestampRows`: the number of records whose timestamp is
negative or is older than retention period
- `TooBigTimestampRows`: the number of records whose timestamp is too
far in the future.
- `HourlySeriesLimitRowsDropped`: the number of records that have not
been added because the hourly series limit has been exceeded.
- `DailySeriesLimitRowsDropped`: the number of records that have not
been added because the daily series limit has been exceeded.

---
Signed-off-by: Artem Fetishev <wwctrsrx@gmail.com>
2024-09-06 18:13:48 +02:00