Commit Graph

4793 Commits

Author SHA1 Message Date
Zakhar Bessarab
372ce74d62
docs/guides-vmgateway-grafana: update guide (#7347)
Some checks are pending
publish-docs / Build (push) Waiting to run
### Describe Your Changes

- update to recent versions of components
- add information about the license key
- add example configuration for remote write with oAuth identity for
vmagent

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-10-24 23:01:57 +04:00
Andrii Chubatiuk
fc537bea00
lib/promscrape/discovery/kubernetes: support kubernetes native sidecars (#7324)
Some checks are pending
build / Build (push) Waiting to run
CodeQL Go / Analyze (push) Waiting to run
main / lint (push) Waiting to run
main / test (test-full) (push) Blocked by required conditions
main / test (test-full-386) (push) Blocked by required conditions
main / test (test-pure) (push) Blocked by required conditions
publish-docs / Build (push) Waiting to run
This commit adds Kubernetes Native Sidecar support. 

It's the special type of init containers, that have restartPolicy == "Always" and continue to run after container initialization. 


related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7287
2024-10-24 17:04:12 +02:00
Zakhar Bessarab
837d0d136d
lib/mergeset: add sparse indexdb cache (#7269)
Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7182

- add a separate index cache for searches which might read through large
amounts of random entries. Primary use-case for this is retention and
downsampling filters, when applying filters background merge needs to
fetch large amount of random entries which pollutes an index cache.
Using different caches allows to reduce effect on memory usage and cache
efficiency of the main cache while still having high cache hit rate. A
separate cache size is 5% of allowed memory.

- reduce size of indexdb/dataBlocks cache in order to free memory for
new sparse cache. Reduced size by 5% and moved this to a separate cache.

- add a separate metricName search which does not cache metric names -
this is needed in order to allow disabling metric name caching when
applying downsampling/retention filters. Applying filters during
background merge accesses random entries, this fills up cache and does
not provide an actual improvement due to random access nature.


Merge performance and memory usage stats before and after the change:

- before

![image](https://github.com/user-attachments/assets/485fffbb-c225-47ae-b5c5-bc8a7c57b36e)


- after

![image](https://github.com/user-attachments/assets/f4ba3440-7c1c-4ec1-bc54-4d2ab431eef5)

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-10-24 15:21:17 +02:00
Andrii Chubatiuk
5fecb77f69
app/vmctl: fix match expression for vm-native protocol with --vm-native-disable-per-metric-migration flag enabled (#7310)
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7309

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-24 14:57:58 +02:00
hagen1778
53b7288e0d
docs: clarify that auto generated metrics can't be relabeled in scrape config
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-24 12:09:30 +02:00
Github Actions
ec0abe736a
Automatic update helm docs from VictoriaMetrics/helm-charts@f3aac52 (#7335)
Some checks are pending
publish-docs / Build (push) Waiting to run
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
Co-authored-by: Hui Wang <haley@victoriametrics.com>
2024-10-24 12:20:06 +08:00
Github Actions
6434fa2c4e
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@12fc7d5 (#7340)
Some checks are pending
publish-docs / Build (push) Waiting to run
2024-10-23 13:02:18 -07:00
hagen1778
7e53324f5d
docs: clarify that vminsert also supports exponential histogram parsing
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-23 11:47:50 +02:00
Github Actions
5ee3bc98d6
Automatic update helm docs from VictoriaMetrics/helm-charts@6237358 (#7327)
Some checks are pending
publish-docs / Build (push) Waiting to run
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-10-23 03:27:32 +04:00
Roman Khavronenko
0204ce942d
app/vmalert: update -remoteWrite.concurrency and -remoteWrite.flushInterval (#7272)
Some checks are pending
build / Build (push) Waiting to run
CodeQL Go / Analyze (push) Waiting to run
main / lint (push) Waiting to run
main / test (test-full) (push) Blocked by required conditions
main / test (test-full-386) (push) Blocked by required conditions
main / test (test-pure) (push) Blocked by required conditions
publish-docs / Build (push) Waiting to run
Auto-adjust `-remoteWrite.concurrency` cmd-line flags with the number of
available CPU cores in the same way as vmagent does. With this change
the default behavior of vmalert in high-loaded installation should
become more resilient. This change also reduces
`-remoteWrite.flushInterval` from `5s` to `2s` to provide better data
freshness.


---------
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-10-22 14:43:55 +02:00
Antoine Deschênes
d656934d22
vmalert: properly set group_name and file fields for recording rules (#7298)
This commit properly adds `group_name` and `file` fields for recording rules web api response   at `/api/v1/rules`.
Previously these fields were blank.

Related issue https://github.com/victoriaMetrics/victoriaMetrics/issues/7297

Signed-off-by: Antoine Deschênes <antoine.deschenes@linux.com>
2024-10-22 14:13:56 +02:00
Github Actions
ac82b5aea6
Automatic update helm docs from VictoriaMetrics/helm-charts@999d44f (#7316)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-10-22 14:04:48 +02:00
Github Actions
e8c7d6373e
Automatic update operator docs from VictoriaMetrics/operator@b357f60 (#7319)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-10-22 14:04:24 +02:00
Fred Navruzov
b17fce3e4b
docs/vmanomaly-release-1.17.2 (#7322)
### Describe Your Changes

- release 1.17.2 updates
- added sections on logging and CLI args to docs

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-22 13:42:37 +03:00
Andrii Chubatiuk
7ecf68093f
docs: updated cmd flags highlight style (#7312)
### Describe Your Changes

Changed highlight style for cmd flags

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-22 03:00:07 -07:00
Zakhar Bessarab
05f6ea621d
app/vmselect: add retention and downsampling filters debug pages (#776)
Some checks are pending
publish-docs / Build (push) Waiting to run
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6304

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-21 09:52:24 +02:00
Github Actions
1cb32ee6c8
Automatic update helm docs from VictoriaMetrics/helm-charts@1e789d9 (#7307)
Some checks failed
publish-docs / Build (push) Has been cancelled
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
2024-10-19 02:44:35 +04:00
Fred Navruzov
7a538bbe78
docs/vmanomaly: release 1.17.1 (#7302)
### Describe Your Changes

docs/vmanomaly: release 1.17.1

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-18 21:07:24 +04:00
hagen1778
d553d101b2
docs/CHANGELOG.md: cut v1.102.5
Some checks failed
CodeQL JS/TS / Analyze (push) Waiting to run
publish-docs / Build (push) Waiting to run
build / Build (push) Has been cancelled
CodeQL Go / Analyze (push) Has been cancelled
main / lint (push) Has been cancelled
main / test (test-full) (push) Has been cancelled
main / test (test-full-386) (push) Has been cancelled
main / test (test-pure) (push) Has been cancelled
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-18 14:31:24 +02:00
hagen1778
73b073e298
docs/CHANGELOG.md: cut v1.97.10
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-18 14:30:26 +02:00
hagen1778
361afaec5b
docs/CHANGELOG.md: cut v1.105.0
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-18 14:28:14 +02:00
hagen1778
f9c79eba30
docs: re-order changes by priority in log
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-18 14:24:20 +02:00
Zhu Jiekun
8c50c38a80
vmstorage: auto calculate maxUniqueTimeseries based on resources (#6961)
### Describe Your Changes

Add support for
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6930

Calculate `-search.maxUniqueTimeseries` by
`-search.maxConcurrentRequests` and remaining memory if it's **not set**
or **less equal than 0**.

The remaining memory is affected by `-memory.allowedPercent`,
`-memory.allowedBytes` and cgroup memory limit.
### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

(cherry picked from commit 85f60237e2)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-18 14:00:14 +02:00
Andrii Chubatiuk
965a33c893
lib/promscrape: fixed reload on max_scrape_size change (#7282)
### Describe Your Changes

fixed reload on max_scrape_size change
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7260

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-18 11:35:23 +02:00
Hui Wang
c4fe23794a
vmalert: fix blocking hot-reload process if the old rule group hasn't started yet (#7258)
Group
[sleeps](daa7183749/app/vmalert/rule/group.go (L320))
random duration before start the evaluation, and during the sleep,
`g.updateCh <- new` will be blocked since there is no `<-g.updateCh`
waiting.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-18 11:18:24 +02:00
hagen1778
41e0bbb6d1
docs/vmctl: clarify the meaning of the comment
The comment was ambiguous and not clear to the readers.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-18 09:46:54 +02:00
Aliaksandr Valialkin
025eec2cb0
deployment: update VictoriaLogs from v0.36.0-victorialogs to v0.37.0-victorialogs
Some checks are pending
CodeQL Go / Analyze (push) Waiting to run
CodeQL JS/TS / Analyze (push) Waiting to run
main / lint (push) Waiting to run
main / test (test-full) (push) Blocked by required conditions
main / test (test-full-386) (push) Blocked by required conditions
main / test (test-pure) (push) Blocked by required conditions
publish-docs / Build (push) Waiting to run
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.37.0-victorialogs
2024-10-18 02:40:13 +02:00
Aliaksandr Valialkin
51cd3ba02b
docs/VictoriaLogs/CHANGELOG.md: cut v0.37.0-victorialogs release 2024-10-18 02:32:37 +02:00
Yury Molodov
423df09d7d
vmui/logs: add ability to hide hits chart (#7206)
### Describe Your Changes

**Added ability to hide the hits chart**

- Users can now hide or show the hits chart by clicking the "eye" icon
located in the upper-right corner of the chart.
- When the chart is hidden, it will stop sending requests to
`/select/logsql/hits`.
- Upon displaying the chart again, it will automatically refresh. If a
relative time range is set, the chart will update according to the time
period of the logs currently being displayed.

**Hits chart visible:**

![image](https://github.com/user-attachments/assets/577e877b-6417-4b83-8d84-c55e3d39864a)

**Hits chart hidden:**

![image](https://github.com/user-attachments/assets/068b1143-d140-4d72-8d65-663900124f32)

Related issue: #7117

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-10-18 02:30:56 +02:00
Yury Molodov
36a86c3aaf
vmui/logs: fix display of hits chart (#7167)
### Describe Your Changes

Fixed the display of hits chart in VictoriaLogs.
See #7133

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-18 02:28:23 +02:00
Aliaksandr Valialkin
064b9a6314
docs/VictoriaLogs/CHANGELOG.md: remove "index.html" trailer from the link to docs for the sake of consistency with other links to docs
This is a follow-up for 3538869942
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7252
2024-10-18 02:26:01 +02:00
Aliaksandr Valialkin
2023f017b1
lib/logstorage: optimize performance for queries, which select all the log fields for logs containing hundreds of log fields (aka "wide events")
Unpack the full columnsHeader block instead of unpacking meta-information per each individual column
when the query, which selects all the columns, is executed. This improves performance when scanning
logs with big number of fields.
2024-10-18 02:22:42 +02:00
Aliaksandr Valialkin
78c6fb0883
lib/logstorage: improve performance of top and field_values pipes on systems with many CPU cores
- Parallelize mering of per-CPU results.
- Parallelize writing the results to the next pipe.
2024-10-18 02:22:42 +02:00
Aliaksandr Valialkin
c4b2fdff70
lib/logstorage: optimize 'stats by(...)' calculations for by(...) fields with millions of unique values on multi-CPU systems
- Parallelize merging of per-CPU `stats by(...)` result shards.
- Parallelize writing `stats by(...)` results to the next pipe.
2024-10-18 02:22:41 +02:00
Aliaksandr Valialkin
192c07f76a
lib/logstorage: optimize performance for top pipe when it is applied to a field with millions of unique values
- Use parallel merge of per-CPU shard results. This improves merge performance on multi-CPU systems.
- Use topN heap sort of per-shard results. This improves performance when results contain millions of entries.
2024-10-18 02:21:56 +02:00
hagen1778
98fcd95438
docs/vmagent: distinguish between metrics, samples and series
Some checks are pending
publish-docs / Build (push) Waiting to run
Before, doc incorrectly used `metric` instead of `sample` or `series`.
This commit aligns description with https://docs.victoriametrics.com/keyconcepts/#structure-of-a-metric

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-17 19:21:52 +02:00
Artem Fetishev
d6bafe31d3
docs/troubleshooting: add reduce_mem_usage=1 to export query (#7286)
### Describe Your Changes

When debugging unexpected query results, add reduce_mem_usage=1 param to
export query to preserve duplicates.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
2024-10-17 18:50:25 +02:00
Fred Navruzov
bc65c9f399
docs/vmanomaly: release v1.17.0 (#7285)
### Describe Your Changes

docs/vmanomaly: release v1.17.0

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-17 18:56:04 +03:00
Nikolay
635bdd130b
lib/storage: properly unmarshal SearchQuery (#7277)
After adding multitenant query feature at v1.104.0, searchQuery wasn't
properly unmarshalled at bottom vmselect in multi-level cluster setup.
It resulted into empty query responses.

This commit adds fallback to Unmarshal method of SearchQuery to fill
TenantTokens. It allows to properly execute search requests
at vmselect side.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7270

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2024-10-17 10:52:35 -03:00
Zakhar Bessarab
d036063c78
docs/vmbackup: add information about cluster backups (#7244)
Some checks are pending
build / Build (push) Waiting to run
CodeQL Go / Analyze (push) Waiting to run
main / lint (push) Waiting to run
main / test (test-full) (push) Blocked by required conditions
main / test (test-full-386) (push) Blocked by required conditions
main / test (test-pure) (push) Blocked by required conditions
publish-docs / Build (push) Waiting to run
### Describe Your Changes

Add more detailed information about performing backups for
VictoriaMetrics cluster setup.
More detailed explanation should help to address questions similar to
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7225

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-10-17 13:56:23 +02:00
hagen1778
aa6c237603
docs: follow-up after f0d1db81dc
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-17 13:49:51 +02:00
Hui Wang
ab0d31a7b0
vmagent: fix type of command-line flag -streamAggr.dedupInterval (#7081)
Previously unit `m` is not correctly supported.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-17 13:27:59 +02:00
Zakhar Bessarab
65e9d19f3c
lib/flagutil/dict: properly update default value in case there is no key value set (#7211)
### Describe Your Changes

If a dict flag has only one value without a prefix it is supposed to
replace default value.

Previously, when flag was set to `-flag=2` and the default value in
`NewDictInt` was set to 1 the resulting value for any `flag.Get()` call
would be 1 which is not expected.

This commit updates default value for the flag in case there is only one
entry for flag and the entry is a number without a key.

This affects cluster version and specifically `replicationFactor` flag
usage with vmstorage [node
groups](https://docs.victoriametrics.com/cluster-victoriametrics/#vmstorage-groups-at-vmselect).
Previously, the following configuration would effectively be ignored:
```
/path/to/vmselect \
 -replicationFactor=2 \
 -storageNode=g1/host1,g1/host2,g1/host3 \
 -storageNode=g2/host4,g2/host5,g2/host6 \
 -storageNode=g3/host7,g3/host8,g3/host9
```

Changes from this PR will force default value for `replicationFactor`
flag to be set to `2` which is expected as the result of this
configuration.


---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-10-17 12:05:47 +02:00
hagen1778
23f8ab6f81
docs/contirubting: clarify the type of changelog line we expect
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-17 11:59:55 +02:00
Andrii Chubatiuk
3538869942
vlogs: added basic alerts (#7252)
### Describe Your Changes

Added basic VLogs alerts

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-17 11:33:06 +02:00
Hui Wang
4984e71da6
vmalert-tool: add more syntax checks for input_series and exp_samples (#7263)
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7224,
allow using
```
        exp_samples:
          - labels: '{}'
```
for prometheus compatibility.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-17 11:00:34 +02:00
Hui Wang
c90adf566e
vmalert-tool: reduce victoriametrics health check interval (#7256)
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6970.
This reduces the hard limit on duration for completing the test when
users run vmalert-tool on slow hosts.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-17 10:51:12 +02:00
Github Actions
c5fb281019
Automatic update helm docs from VictoriaMetrics/helm-charts@845bc1f (#7268)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-10-17 10:42:41 +02:00
Aliaksandr Valialkin
a72e1155b9
docs/VictoriaLogs/CHANGELOG.md: add missing part of the sentence
Some checks failed
build / Build (push) Waiting to run
CodeQL Go / Analyze (push) Waiting to run
main / lint (push) Waiting to run
main / test (test-full) (push) Blocked by required conditions
main / test (test-full-386) (push) Blocked by required conditions
main / test (test-pure) (push) Blocked by required conditions
publish-docs / Build (push) Waiting to run
CodeQL JS/TS / Analyze (push) Has been cancelled
2024-10-16 20:22:19 +02:00
Aliaksandr Valialkin
677f1cd1be
docs/VictoriaLogs/CHANGELOG.md: typo fix: refer the correct endpoints for stats results 2024-10-16 20:19:22 +02:00