Aliaksandr Valialkin
5733e56e40
docs/VictoriaLogs/querying: refer to vlogscli from docs for /select/logsql/query and /select/logsql/tail HTTP endpoints
...
publish-docs / Build (push) Has been cancelled
These endpoints are easier to query via vlogscli
2024-11-02 12:31:40 +01:00
Aliaksandr Valialkin
37a58677fa
docs/VictoriaLogs/data-ingestion: mention Grafana Agent and Grafana Alloy additionally to Promtail as log collectors for Grafana Loki
2024-11-02 11:36:22 +01:00
Aliaksandr Valialkin
c7e242b8ce
docs/VictoriaLogs/LogsQL.md: remove misleading information that min and max stats functions work only for numeric values
...
publish-docs / Build (push) Waiting to run
build / Build (push) Has been cancelled
CodeQL Go / Analyze (push) Has been cancelled
main / lint (push) Has been cancelled
main / test (test-full) (push) Has been cancelled
main / test (test-full-386) (push) Has been cancelled
main / test (test-pure) (push) Has been cancelled
These functions work for both numeric and string values after ad505a7a9a
2024-11-01 20:25:33 +01:00
Aliaksandr Valialkin
a86df52adf
docs/VictoriaLogs/LogsQL.md: typo fix in the link to description for sum_len stats function: sum-len-stats -> sum_len-stats
...
This is a follow-up for 364f084b43
2024-11-01 20:13:47 +01:00
Aliaksandr Valialkin
4478e48eb6
app/vlinsert: implement the ability to add extra fields to the ingested logs
...
This can be done via extra_fields query arg or via VL-Extra-Fields HTTP header.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7354#issuecomment-2448671445
2024-11-01 20:06:17 +01:00
Aliaksandr Valialkin
b3e2db5647
docs/VictoriaLogs/keyConcepts.md: remove duplicate sentence about missing timezone information in the ingested _time field values
2024-11-01 17:20:28 +01:00
Github Actions
955f3660de
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@3ecaebd ( #7395 )
2024-10-31 14:08:35 +01:00
Aliaksandr Valialkin
24b6e117dd
deployment/docker: update VictoriaLogs from v0.39.0-victorialogs to v0.40.0-victorialogs
...
build / Build (push) Waiting to run
CodeQL Go / Analyze (push) Waiting to run
main / lint (push) Waiting to run
main / test (test-full) (push) Blocked by required conditions
main / test (test-full-386) (push) Blocked by required conditions
main / test (test-pure) (push) Blocked by required conditions
publish-docs / Build (push) Waiting to run
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.40.0-victorialogs
2024-10-30 23:43:20 +01:00
Aliaksandr Valialkin
1a7b55009b
docs/VictoriaLogs/CHANGELOG.md: cut v0.40.0-victorialogs release
2024-10-30 23:38:06 +01:00
Aliaksandr Valialkin
f88e2ae9fb
docs/VictoriaLogs/README.md: follow-up for 0f6b9e9490: consistently use canonical url for Grafana dashboards
...
See also 0a5ffb3bc1
2024-10-30 23:29:23 +01:00
Aliaksandr Valialkin
0f6b9e9490
docs/VictoriaLogs/README.md: recommend installing Grafana dashboard for VictoriaLogs at Monitoring chapter
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6886
2024-10-30 23:20:36 +01:00
Aliaksandr Valialkin
c5d08d317c
lib/logstorage: properly reset cached output fields for extract and extract_regexp pipes after the log entry matches if(...) condition
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7162
2024-10-30 22:29:52 +01:00
Aliaksandr Valialkin
2e635a42d8
lib/logstorage: properly cache replace() and replace_regexp() results for identical adjacent field values
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7162
2024-10-30 22:29:52 +01:00
Aliaksandr Valialkin
f62502a943
docs/VictoriaLogs/querying/README.md: typo fix after f2cd284cf4a7429848fb4be1522483a9b93a43e7: Alls -> All
build / Build (push) Waiting to run
CodeQL Go / Analyze (push) Waiting to run
main / lint (push) Waiting to run
main / test (test-full) (push) Blocked by required conditions
main / test (test-full-386) (push) Blocked by required conditions
main / test (test-pure) (push) Blocked by required conditions
publish-docs / Build (push) Waiting to run
2024-10-30 19:08:57 +01:00
Aliaksandr Valialkin
7603446850
app/vlselect: add support for extra_filters and extra_stream_filters query args across all the HTTP querying APIs
...
These query args are going to be used for quick filtering on field values at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7365
2024-10-30 17:59:49 +01:00
hagen1778
6494606924
deployment/alerts: consistently update path to alerting rules
...
Follow-up after 68bad22fd2
2024-10-30 16:06:57 +01:00
Aliaksandr Valialkin
25bca94d04
deployment/docker: update VictoriaLogs from v0.38.0-victorialogs to v0.39.0-victorialogs
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.39.0-victorialogs
2024-10-30 15:11:17 +01:00
Aliaksandr Valialkin
13822f6d11
docs/VictoriaLogs/CHANGELOG.md: cut v0.39.0-victorialogs release
2024-10-30 15:06:00 +01:00
Aliaksandr Valialkin
16ee470da6
app/vlinsert: accept logs with empty _msg field
...
In this case the _msg field is set to the value specified in the -defaultMsgValue command-line flag.
This should simplify first-time migration to VictoriaLogs from other systems.
2024-10-30 14:59:38 +01:00
Aliaksandr Valialkin
96466562b6
docs/VictoriaLogs/data-ingestion/README.md: formatting fixes after 8b36529b32
...
- Remove leading whitespace from the first lines in 'HTTP parameters' chapter.
This whitespace isn't needed for the markdown formatting.
- Add leading whitespace for the second sentence in the list bullet describing AccountID and ProjectID HTTP headers.
This fixes markdown formatting for this list bullet.
2024-10-30 14:23:54 +01:00
Aliaksandr Valialkin
ed73f8350b
app/vlinsert: allow specifying comma-separated list of fields containing log message via _msg_field query arg and VL-Msg-Field HTTP request header
...
This msy be useful when ingesting logs from different sources, which store the log message in different fields.
For example, `_msg_field=message,event.data,some_field` will get log message from the first non-empty field:
`message`, `event.data` and `some_field`.
2024-10-30 14:17:33 +01:00
Aliaksandr Valialkin
102e9d4f4e
lib/logstorage: make sure that the number of output (bloom, values) shards is bigger than zero.
...
If the number of output (bloom, values) shards is zero, then this may lead to panic
as shown at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7391 .
This panic may happen when parts with only constant fields with distinct values are merged into
output part with non-constant fields, which should be written to (bloom, values) shards.
2024-10-30 13:39:28 +01:00
hagen1778
2c93353173
docs: add frequently asked questions to vmalert integration with vlogs
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-30 12:38:47 +01:00
hagen1778
b789a9dc83
docs: mark alerting feature as done in VictoriaLogs
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-30 12:20:00 +01:00
hagen1778
77b690ab26
docs: update version placeholder to point to the next version
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-30 12:19:44 +01:00
Artem Navoiev
3473440d26
docs: logs vmalert add identifier
...
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-10-30 09:45:28 +01:00
Aliaksandr Valialkin
c963d7d10d
docs/VictoriaLogs/CHANGELOG.md: remove unnededed with prefix in front of rank at top pipe example
2024-10-29 18:27:40 +01:00
Aliaksandr Valialkin
c43a6ce0eb
deployment/docker: update VictoriaLogs from v0.37.0-victorialogs to v0.38.0-victorialogs
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.38.0-victorialogs
2024-10-29 18:16:16 +01:00
Aliaksandr Valialkin
12223cf5d0
docs/VictoriaLogs/CHANGELOG.md: cut v0.38.0 release
2024-10-29 18:08:36 +01:00
Aliaksandr Valialkin
7a623c225f
lib/logstorage: follow-up for af831a6c906158f371f1b6810706fa0a54b78386
...
Sync the code between top and sort pipes regarding the code related to rank.
2024-10-29 16:44:46 +01:00
Aliaksandr Valialkin
3c06d083ea
lib/logstorage: add an ability to return rank from top pipe results
2024-10-29 16:44:45 +01:00
Aliaksandr Valialkin
7a62eefa34
lib/logstorage: dynamically adjust the number of (bloom, values) shards in a part depending on the number of non-const columns
...
This allows reducing the amounts of data, which must be read during queries over logs with big number of fields (aka "wide events").
This, in turn, improves query performance when the data, which needs to be scanned during the query, doesn't fit OS page cache.
2024-10-29 16:44:45 +01:00
Aliaksandr Valialkin
67b4059aa4
docs/VictoriaLogs/README.md: add tuning chapter
2024-10-29 16:44:44 +01:00
Aliaksandr Valialkin
8d968acd0a
lib/logstorage: avoid reading columnsHeader data when field_values pipe is applied directly to log filters
...
This improves performance of `field_values` pipe when it is applied to large number of data blocks.
This also improves performance of /select/logsql/field_values HTTP API.
2024-10-29 16:44:44 +01:00
Hui Wang
68bad22fd2
vmalert: integrate with victorialogs ( #7255 )
...
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6706 .
See
https://github.com/VictoriaMetrics/VictoriaMetrics/blob/vmalert-support-vlog-ds/docs/VictoriaLogs/vmalert.md .
Related fix
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7254 .
Note: in this pull request, vmalert doesn't support
[backfilling](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/vmalert-support-vlog-ds/docs/VictoriaLogs/vmalert.md#rules-backfilling )
for rules with a customized time filter. It might be added in the
future, see [this
issue](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7289 )
for details.
Feature can be tested with image
`victoriametrics/vmalert:heads-vmalert-support-vlog-ds-0-g420629c-scratch`.
---------
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-10-29 16:30:39 +01:00
Andrii Chubatiuk
7e60afb6fc
app/vlinsert: adds journald ingestion support
...
This commit allows to ingest logs with journald format.
https://www.freedesktop.org/software/systemd/man/latest/systemd-journal-remote.service.html
related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4618
2024-10-27 20:36:33 +01:00
Hui Wang
0172e65b8d
docs: clarify flags -search.maxxxDuration can only be overridden to… ( #7227 )
...
publish-docs / Build (push) Waiting to run
build / Build (push) Has been cancelled
CodeQL Go / Analyze (push) Has been cancelled
CodeQL JS/TS / Analyze (push) Has been cancelled
main / lint (push) Has been cancelled
main / test (test-full) (push) Has been cancelled
main / test (test-full-386) (push) Has been cancelled
main / test (test-pure) (push) Has been cancelled
… a smaller value with `timeout` arg
2024-10-25 11:11:09 +02:00
Github Actions
2224424136
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@9e6db28 ( #7350 )
2024-10-25 01:16:42 -07:00
Yury Molodov
dd89745a34
vmui/logs: fix query and limit update issue ( #7294 )
...
### Describe Your Changes
Fixes issues with incorrect updating of query and limit fields, and
resolves the problem where the display tab resets.
Related issue: #7279 and #7290
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/ ).
---------
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-10-25 09:32:20 +02:00
Github Actions
6434fa2c4e
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@12fc7d5 ( #7340 )
publish-docs / Build (push) Waiting to run
2024-10-23 13:02:18 -07:00
Aliaksandr Valialkin
025eec2cb0
deployment: update VictoriaLogs from v0.36.0-victorialogs to v0.37.0-victorialogs
...
CodeQL Go / Analyze (push) Waiting to run
CodeQL JS/TS / Analyze (push) Waiting to run
main / lint (push) Waiting to run
main / test (test-full) (push) Blocked by required conditions
main / test (test-full-386) (push) Blocked by required conditions
main / test (test-pure) (push) Blocked by required conditions
publish-docs / Build (push) Waiting to run
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.37.0-victorialogs
2024-10-18 02:40:13 +02:00
Aliaksandr Valialkin
51cd3ba02b
docs/VictoriaLogs/CHANGELOG.md: cut v0.37.0-victorialogs release
2024-10-18 02:32:37 +02:00
Yury Molodov
423df09d7d
vmui/logs: add ability to hide hits chart ( #7206 )
...
### Describe Your Changes
**Added ability to hide the hits chart**
- Users can now hide or show the hits chart by clicking the "eye" icon
located in the upper-right corner of the chart.
- When the chart is hidden, it will stop sending requests to
`/select/logsql/hits`.
- Upon displaying the chart again, it will automatically refresh. If a
relative time range is set, the chart will update according to the time
period of the logs currently being displayed.
**Hits chart visible:**
**Hits chart hidden:**
Related issue: #7117
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/ ).
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-10-18 02:30:56 +02:00
Yury Molodov
36a86c3aaf
vmui/logs: fix display of hits chart ( #7167 )
...
### Describe Your Changes
Fixed the display of hits chart in VictoriaLogs.
See #7133
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/ ).
2024-10-18 02:28:23 +02:00
Aliaksandr Valialkin
064b9a6314
docs/VictoriaLogs/CHANGELOG.md: remove "index.html" trailer from the link to docs for the sake of consistency with other links to docs
...
This is a follow-up for 3538869942https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7252
2024-10-18 02:26:01 +02:00
Aliaksandr Valialkin
2023f017b1
lib/logstorage: optimize performance for queries, which select all the log fields for logs containing hundreds of log fields (aka "wide events")
...
Unpack the full columnsHeader block instead of unpacking meta-information per each individual column
when the query, which selects all the columns, is executed. This improves performance when scanning
logs with big number of fields.
2024-10-18 02:22:42 +02:00
Aliaksandr Valialkin
78c6fb0883
lib/logstorage: improve performance of top and field_values pipes on systems with many CPU cores
...
- Parallelize mering of per-CPU results.
- Parallelize writing the results to the next pipe.
2024-10-18 02:22:42 +02:00
Aliaksandr Valialkin
c4b2fdff70
lib/logstorage: optimize 'stats by(...)' calculations for by(...) fields with millions of unique values on multi-CPU systems
...
- Parallelize merging of per-CPU `stats by(...)` result shards.
- Parallelize writing `stats by(...)` results to the next pipe.
2024-10-18 02:22:41 +02:00
Aliaksandr Valialkin
192c07f76a
lib/logstorage: optimize performance for top pipe when it is applied to a field with millions of unique values
...
- Use parallel merge of per-CPU shard results. This improves merge performance on multi-CPU systems.
- Use topN heap sort of per-shard results. This improves performance when results contain millions of entries.
2024-10-18 02:21:56 +02:00
hagen1778
aa6c237603
docs: follow-up after f0d1db81dc
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-10-17 13:49:51 +02:00
