Commit Graph

9331 Commits

Author SHA1 Message Date
danish-mehmood
34c4b4ab0c
docs: fix typos and format in case study (#7374)
### Describe Your Changes

- made small typo fix in case studies

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 8dc4e2b5a5)
2024-10-30 13:13:05 +01:00
cangqiaoyuzhuo
07cf3189f8
chore: fix function name (#7381)
### Describe Your Changes

 fix function name

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 45896fb477)
2024-10-30 13:13:05 +01:00
hagen1778
97ff8a8fdf
docs: consistently update twitter.com to x.com
See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7383

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit cfba770c8e)
2024-10-30 13:13:05 +01:00
Lubov66
5330b473c7
docs: replace twitter.com with x.com (#7383)
Replacing the name and link of the social network

(cherry picked from commit 702ff923fe)
2024-10-30 13:13:04 +01:00
Github Actions
369cc724c7
Automatic update helm docs from VictoriaMetrics/helm-charts@87ea94d (#7385)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
(cherry picked from commit b2cf8685e5)
2024-10-30 13:13:04 +01:00
hagen1778
bdfef9700d
docs: add frequently asked questions to vmalert integration with vlogs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 2c93353173)
2024-10-30 13:13:04 +01:00
hagen1778
8d64847f5e
docs: mark alerting feature as done in VictoriaLogs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit b789a9dc83)
2024-10-30 13:13:04 +01:00
hagen1778
f923bcba5e
docs: update version placeholder to point to the next version
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 77b690ab26)
2024-10-30 13:13:03 +01:00
Artem Navoiev
19327643b4
docs: logs vmalert add identifier
Some checks failed
build / Build (push) Waiting to run
CodeQL Go / Analyze (push) Waiting to run
main / lint (push) Waiting to run
main / test (test-full) (push) Blocked by required conditions
main / test (test-full-386) (push) Blocked by required conditions
main / test (test-pure) (push) Blocked by required conditions
CodeQL JS/TS / Analyze (push) Has been cancelled
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 3473440d26)
2024-10-30 09:52:54 +01:00
hagen1778
8d42bbcaa7
docs: rm unused vmanomaly assets
These images copies are present in vmanomaly subfolder.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 2ac07aa813)
2024-10-30 09:52:54 +01:00
hagen1778
7c1b297772
docs: add step for updating VM version in docs
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7388

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit a49eb7d4f5)
2024-10-30 09:52:54 +01:00
hagen1778
d7d5023acc
docs: update VM versions to the latest version
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7388

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit b86b0dd910)
2024-10-30 09:52:53 +01:00
Aliaksandr Valialkin
d5b2ef3095
docs/VictoriaLogs/CHANGELOG.md: remove unnededed with prefix in front of rank at top pipe example
(cherry picked from commit c963d7d10d)
2024-10-30 09:52:53 +01:00
Aliaksandr Valialkin
16be99ccab
deployment/docker: update VictoriaLogs from v0.37.0-victorialogs to v0.38.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.38.0-victorialogs

(cherry picked from commit c43a6ce0eb)
2024-10-30 09:52:53 +01:00
Aliaksandr Valialkin
4266fdf145
docs/VictoriaLogs/CHANGELOG.md: cut v0.38.0 release
(cherry picked from commit 12223cf5d0)
2024-10-30 09:52:52 +01:00
Aliaksandr Valialkin
cadee3e254
app/vlselect/vmui: run make vmui-logs-update after dd89745a34
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7294
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7279
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7290

(cherry picked from commit 4f057e5669)
2024-10-30 09:52:52 +01:00
Aliaksandr Valialkin
1dd01b8a8f
lib/logstorage: follow-up for af831a6c906158f371f1b6810706fa0a54b78386
Sync the code between top and sort pipes regarding the code related to rank.

(cherry picked from commit 7a623c225f)
2024-10-30 09:52:52 +01:00
Aliaksandr Valialkin
48b707a929
app/vlogscli: print hint on how to see available commands when starting vlogscli
This should improve the first-time experience with vlogscli

(cherry picked from commit 8faee6b446)
2024-10-30 09:52:51 +01:00
Aliaksandr Valialkin
329d9a46ee
lib/logstorage: add an ability to return rank from top pipe results
(cherry picked from commit 3c06d083ea)
2024-10-30 09:52:51 +01:00
Aliaksandr Valialkin
fe5f16b662
lib/logstorage: dynamically adjust the number of (bloom, values) shards in a part depending on the number of non-const columns
This allows reducing the amounts of data, which must be read during queries over logs with big number of fields (aka "wide events").
This, in turn, improves query performance when the data, which needs to be scanned during the query, doesn't fit OS page cache.

(cherry picked from commit 7a62eefa34)
2024-10-30 09:52:51 +01:00
Aliaksandr Valialkin
40be393f5a
docs/VictoriaLogs/README.md: add tuning chapter
(cherry picked from commit 67b4059aa4)
2024-10-30 09:52:50 +01:00
Aliaksandr Valialkin
76b21c8560
lib/logstorage: avoid reading columnsHeader data when field_values pipe is applied directly to log filters
This improves performance of `field_values` pipe when it is applied to large number of data blocks.
This also improves performance of /select/logsql/field_values HTTP API.

(cherry picked from commit 8d968acd0a)
2024-10-30 09:52:50 +01:00
Hui Wang
9616814728
vmalert: integrate with victorialogs (#7255)
Some checks are pending
build / Build (push) Waiting to run
CodeQL Go / Analyze (push) Waiting to run
main / lint (push) Waiting to run
main / test (test-full) (push) Blocked by required conditions
main / test (test-full-386) (push) Blocked by required conditions
main / test (test-pure) (push) Blocked by required conditions
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6706.
See
https://github.com/VictoriaMetrics/VictoriaMetrics/blob/vmalert-support-vlog-ds/docs/VictoriaLogs/vmalert.md.

Related fix
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7254.

Note: in this pull request, vmalert doesn't support
[backfilling](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/vmalert-support-vlog-ds/docs/VictoriaLogs/vmalert.md#rules-backfilling)
for rules with a customized time filter. It might be added in the
future, see [this
issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7289)
for details.

Feature can be tested with image
`victoriametrics/vmalert:heads-vmalert-support-vlog-ds-0-g420629c-scratch`.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 68bad22fd2)
2024-10-29 16:32:00 +01:00
Fred Navruzov
61657e56e8
docs/vmanomaly - release 1.18.0 (#7378)
### Describe Your Changes

docs/vmanomaly - release 1.18.0

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 5d73b8b866)
2024-10-29 16:31:59 +01:00
Zakhar Bessarab
517bd9392c
lib/storage/partition: prevent panic in case resulting in-memory part is empty after merge (#7329)
Some checks failed
build / Build (push) Has been cancelled
CodeQL Go / Analyze (push) Has been cancelled
CodeQL JS/TS / Analyze (push) Has been cancelled
main / lint (push) Has been cancelled
main / test (test-full) (push) Has been cancelled
main / test (test-full-386) (push) Has been cancelled
main / test (test-pure) (push) Has been cancelled
It is possible for in-memory part to be empty if ingested samples are
removed by retention filters. In this case, data will not be discarded
due to retention before creating in memory part. After in-memory parts
merge samples will be removed resulting in creating completely empty
part at destination.

 This commit checks for resulting part and skips it, if it's empty.

---------
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-10-27 20:42:42 +01:00
Zhu Jiekun
3d55605ae5
lib/promscrape: adds support for PuppetDB service discovery
This commit adds support for
[PuppetDB](https://www.puppet.com/docs/puppetdb/8/overview.html) service
discovery to the `vmagent` and `victoria-metrics-single` components.

Related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5744
2024-10-27 20:42:42 +01:00
Andrii Chubatiuk
da14c7b26e
app/vlinsert: adds journald ingestion support
This commit allows to ingest logs with journald format. 

https://www.freedesktop.org/software/systemd/man/latest/systemd-journal-remote.service.html

related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4618
2024-10-27 20:42:41 +01:00
Dan Dascalescu
64218cc0ad
Fix "loosing" typo in README.md (#7368)
### Checklist

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/)
2024-10-27 20:25:17 +01:00
Zhu Jiekun
57ddb51089
dashboards: fix query for full ETA vm_free_disk_space_bytes - vm_free_disk_space_limit_bytes (#7355)
### Describe Your Changes

Fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7334

available disk space should be 
```
(vm_free_disk_space_bytes{job=~...} - vm_free_disk_space_limit_bytes{job=~...})
```
instead of 
```
vm_free_disk_space_bytes{job=~...}
```

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-27 20:25:17 +01:00
Hui Wang
451bd164f0
docs: clarify flags -search.maxxxDuration can only be overridden to… (#7227)
… a smaller value with `timeout` arg
2024-10-27 20:25:17 +01:00
Github Actions
8629331316
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@ed19341 (#7352) 2024-10-27 20:25:17 +01:00
Github Actions
d11300874f
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@9e6db28 (#7350) 2024-10-27 20:25:17 +01:00
Yury Molodov
4b748750e7
vmui/logs: fix query and limit update issue (#7294)
### Describe Your Changes

Fixes issues with incorrect updating of query and limit fields, and
resolves the problem where the display tab resets.

Related issue: #7279 and #7290

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-10-27 20:25:16 +01:00
Zakhar Bessarab
c295c5a4e2
docs/guides-vmgateway-grafana: update guide (#7347)
### Describe Your Changes

- update to recent versions of components
- add information about the license key
- add example configuration for remote write with oAuth identity for
vmagent

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-10-27 20:25:16 +01:00
Andrii Chubatiuk
f6f4884ba6
lib/promscrape/discovery/kubernetes: support kubernetes native sidecars (#7324)
This commit adds Kubernetes Native Sidecar support. 

It's the special type of init containers, that have restartPolicy == "Always" and continue to run after container initialization. 


related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7287
2024-10-27 20:25:15 +01:00
Zakhar Bessarab
8198e7241d
lib/mergeset: add sparse indexdb cache (#7269)
Some checks failed
build / Build (push) Has been cancelled
CodeQL Go / Analyze (push) Has been cancelled
main / lint (push) Has been cancelled
main / test (test-full) (push) Has been cancelled
main / test (test-full-386) (push) Has been cancelled
main / test (test-pure) (push) Has been cancelled
Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7182

- add a separate index cache for searches which might read through large
amounts of random entries. Primary use-case for this is retention and
downsampling filters, when applying filters background merge needs to
fetch large amount of random entries which pollutes an index cache.
Using different caches allows to reduce effect on memory usage and cache
efficiency of the main cache while still having high cache hit rate. A
separate cache size is 5% of allowed memory.

- reduce size of indexdb/dataBlocks cache in order to free memory for
new sparse cache. Reduced size by 5% and moved this to a separate cache.

- add a separate metricName search which does not cache metric names -
this is needed in order to allow disabling metric name caching when
applying downsampling/retention filters. Applying filters during
background merge accesses random entries, this fills up cache and does
not provide an actual improvement due to random access nature.

Merge performance and memory usage stats before and after the change:

- before

![image](https://github.com/user-attachments/assets/485fffbb-c225-47ae-b5c5-bc8a7c57b36e)

- after

![image](https://github.com/user-attachments/assets/f4ba3440-7c1c-4ec1-bc54-4d2ab431eef5)

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 837d0d136d)
2024-10-24 12:43:06 -03:00
Andrii Chubatiuk
04dec639fe
app/vmctl: fix match expression for vm-native protocol with --vm-native-disable-per-metric-migration flag enabled (#7310)
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7309

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 5fecb77f69)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-24 15:49:11 +02:00
hagen1778
ef08faefea
docs: clarify that auto generated metrics can't be relabeled in scrape config
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 53b7288e0d)
2024-10-24 15:48:21 +02:00
Github Actions
fefe3c8d00
Automatic update helm docs from VictoriaMetrics/helm-charts@f3aac52 (#7335)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
Co-authored-by: Hui Wang <haley@victoriametrics.com>
(cherry picked from commit ec0abe736a)
2024-10-24 15:48:21 +02:00
Github Actions
9331ee8b06
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@12fc7d5 (#7340)
(cherry picked from commit 6434fa2c4e)
2024-10-24 15:48:20 +02:00
Artem Fetishev
a722ddbdd9
lib/storage: Fix flaky test: TestStorageRotateIndexDB (#7267)
This commit fixes the TestStorageRotateIndexDB flaky test reported at:
#6977. Sample test failure: https://pastebin.com/bTSs8HP1

The test fails because one goroutine adds items to the indexDB table
while another goroutine is closing that table. This may happen if
indexDB rotation happens twice during one Storage.add() operation:
-  Storage.add() takes the current indexDB and adds index recods to it
- First index db rotation makes the current index DB a previous one
(still ok at this point)
- Second index db rotation removes the indexDB that was current two
rotations earlier. It does this by setting the mustDrop flag to true and
decrementing the ref counter. The ref counter reaches zero which cases
the underlying indexdb table to release its resources gracefully.
Graceful release assumes that the table is not written anymore. But
Storage.add() still adds items to it.

The solution is to increment the indexDB ref counters while it is used
inside add().
The unit test has been changed a little so that the test fails reliably.
The idea is to make add() function invocation to last much longer,
therefore the test inserts not just one record at a time but thouthands
of them.

To see the test fail, just replace the idbsLocked() func with:

```go
unc (s *Storage) idbsLocked2() (*indexDB, *indexDB, func()) {
       return s.idbCurr.Load(), s.idbNext.Load(), func() {}
}
```

---------

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>

(cherry picked from commit 6b9f57e5f7)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-24 15:45:54 +02:00
hagen1778
81a4d7c37d
docs: clarify that vminsert also supports exponential histogram parsing
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 7e53324f5d)
2024-10-24 15:29:36 +02:00
Github Actions
60e0730b8d
Automatic update helm docs from VictoriaMetrics/helm-charts@6237358 (#7327)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: AndrewChubatiuk <3162380+AndrewChubatiuk@users.noreply.github.com>
(cherry picked from commit 5ee3bc98d6)
2024-10-24 15:29:33 +02:00
Roman Khavronenko
fdb898f249
app/vmalert: update -remoteWrite.concurrency and -remoteWrite.flushInterval (#7272)
Some checks failed
build / Build (push) Has been cancelled
CodeQL Go / Analyze (push) Has been cancelled
main / lint (push) Has been cancelled
main / test (test-full) (push) Has been cancelled
main / test (test-full-386) (push) Has been cancelled
main / test (test-pure) (push) Has been cancelled
Auto-adjust `-remoteWrite.concurrency` cmd-line flags with the number of
available CPU cores in the same way as vmagent does. With this change
the default behavior of vmalert in high-loaded installation should
become more resilient. This change also reduces
`-remoteWrite.flushInterval` from `5s` to `2s` to provide better data
freshness.


---------
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-10-22 17:28:51 +02:00
Antoine Deschênes
85e0441ddd
vmalert: properly set group_name and file fields for recording rules (#7298)
This commit properly adds `group_name` and `file` fields for recording rules web api response   at `/api/v1/rules`.
Previously these fields were blank.

Related issue https://github.com/victoriaMetrics/victoriaMetrics/issues/7297

Signed-off-by: Antoine Deschênes <antoine.deschenes@linux.com>
2024-10-22 17:28:50 +02:00
Github Actions
1a0c6026a5
Automatic update helm docs from VictoriaMetrics/helm-charts@999d44f (#7316)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-10-22 17:28:50 +02:00
Github Actions
1fd04f9329
Automatic update operator docs from VictoriaMetrics/operator@b357f60 (#7319)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: Github Actions <133988544+victoriametrics-bot@users.noreply.github.com>
Co-authored-by: f41gh7 <18450869+f41gh7@users.noreply.github.com>
2024-10-22 17:28:50 +02:00
Fred Navruzov
63ba28fdc2
docs/vmanomaly-release-1.17.2 (#7322)
### Describe Your Changes

- release 1.17.2 updates
- added sections on logging and CLI args to docs

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-22 17:28:50 +02:00
Andrii Chubatiuk
3bd6397991
docs: updated cmd flags highlight style (#7312)
### Describe Your Changes

Changed highlight style for cmd flags

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-10-22 17:28:50 +02:00
f41gh7
6d2349af63
deployment: bump VM to v1.105.0
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-10-22 17:28:49 +02:00