Commit Graph

376 Commits

Author SHA1 Message Date
Yury Molodov
7bc20086ec
vmui: add the ability to cancel running queries (#7204)
### Describe Your Changes

- Added functionality to cancel running queries on the Explore Logs and
Query pages.
- The loader was changed from a spinner to a top bar within the block.
This still indicates loading, but solves the issue of the spinner
"flickering," especially during graph dragging.

Related issue: #7097

https://github.com/user-attachments/assets/98e59aeb-905b-4b9d-bbb2-688223b22a82

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 6c9772b101)
2024-10-15 14:53:27 +02:00
Aliaksandr Valialkin
0881e5fd5c
app/vlselect: do not show empty fields in query results
Empty fields are treated as non-existing fields by VictoriaLogs data model.
So there is no sense in returning empty fields in query results, since they may mislead and confuse users.

(cherry picked from commit bac193e50b)
2024-10-15 11:49:32 +02:00
Andrii Chubatiuk
822d6bbdbc
docs/victorialogs: fixed HA examples links (#7249)
### Describe Your Changes

Fixed VictoriaLogs HA examples references in docs

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 41e0b62099)
2024-10-15 11:49:31 +02:00
Aliaksandr Valialkin
f627d7f686
app/vlstorage: add support for forced merge via /internal/force_merge HTTP endpoint
(cherry picked from commit 3c73dbbacc)
2024-10-15 11:49:31 +02:00
Aliaksandr Valialkin
ac2b6e8704
lib/logstorage: make a copy of s.partitions slice when performing queries over the selected partitions
s.partitions can be changed when new partition is registered or when old partition is dropped.
This could lead to data races and panics when s.partitions slice is accessed by concurrently executed queries.

The fix is to make a copy of the selected partitions under s.partitionsLock before performing the query.

(cherry picked from commit b4b79a4961)
2024-10-15 11:49:31 +02:00
Aliaksandr Valialkin
fe9e6b7495
docs/VictoriaLogs: replace incorrect usafe of VictoriaMetrics with VictoriaLogs
(cherry picked from commit 91f5417572)
2024-10-15 11:49:30 +02:00
Aliaksandr Valialkin
f7496c17e4
docs/VictoriaLogs/Roadmap.md: remove OpenTelemetry, Fluentd and Telegraf from the list of data ingestion protocols, which are going to be supported by VictoriaLogs
These protocols are already supported:

- OpenTelemetry protocol - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4839 ,
  https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6218 and 01430a155c

- Fluentd protocol - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/7098
  and 05a64a8c14

- Telegraf protocol - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5310 ,
  https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6443 and 1731c0eabf

(cherry picked from commit 200d723b9a)
2024-10-15 11:49:29 +02:00
Aliaksandr Valialkin
e581338b84
lib/logstorage: make sure that bs.br is non-nil before checking br.bs.bsw.bh.rowsCount there
br.bs may be nil when br contains the block with additional filters applied during pipe calculations.
For example, `* | count() if (error) errors`.

(cherry picked from commit 867f671cc4)
2024-10-15 11:49:29 +02:00
Zakhar Bessarab
4b1fe0d656
docs/victoria-logs: fix typo (#7245)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
(cherry picked from commit 2239f5829f)
2024-10-15 11:49:29 +02:00
Aliaksandr Valialkin
d2352fd84f
docs/VictoriaLogs/querying/vlogscli.md: add the list of features of vlogscli
(cherry picked from commit b9a8c1ff3a)
2024-10-11 14:28:07 +02:00
n4mine
e2f0fe8315
docs: fix typo in docs/VictoriaLogs/data-ingestion/Vector.md (#7222)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit f060b67da5)
2024-10-11 14:28:07 +02:00
Aliaksandr Valialkin
0fe533e0be
deployment: update VictoriaLogs from v0.34.0-victorialogs to v0.35.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.35.0-victorialogs

(cherry picked from commit baa87b5b36)
2024-10-11 14:27:46 +02:00
Aliaksandr Valialkin
ff63816b06
docs/VictoriaLogs: cut v0.35.0 release
(cherry picked from commit 252aa792f7)
2024-10-11 14:27:46 +02:00
Aliaksandr Valialkin
8d4c1cb1d8
docs/VictoriaLogs: make vlogscli more visible
(cherry picked from commit 9413b2de91)
2024-10-11 14:27:46 +02:00
Aliaksandr Valialkin
4bb5f588bc
app/vlogscli: add -accountID and -projectID command-line flags for querying the given tenants
(cherry picked from commit ad5d8097da)
2024-10-11 14:27:45 +02:00
Aliaksandr Valialkin
d07e09b1e4
app/vlogscli: add support for live tailing
(cherry picked from commit e31625e0b2)
Signed-off-by: hagen1778 <roman@victoriametrics.com>

# Conflicts:
#	Makefile
2024-10-11 14:27:26 +02:00
n4mine
fb47859c59
docs: fix typo in docs/VictoriaLogs/data-ingestion/Vector.md (#7208)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 344d61da79)
2024-10-11 14:25:22 +02:00
Aliaksandr Valialkin
00bd1c68a2
deployment: update VictoriaLogs image tag from v0.33.0-victorialogs to v0.34.0-victorialogs
Some checks failed
build / Build (push) Has been cancelled
CodeQL Go / Analyze (push) Has been cancelled
main / lint (push) Has been cancelled
main / test (test-full) (push) Has been cancelled
main / test (test-full-386) (push) Has been cancelled
main / test (test-pure) (push) Has been cancelled
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.34.0-victorialogs
2024-10-08 12:30:49 +02:00
Aliaksandr Valialkin
db75455fbd
docs/VictoriaLogs/CHANGELOG.md: cut v0.34.0 release 2024-10-08 12:21:50 +02:00
Aliaksandr Valialkin
efe5935497
app/vlogscli: add ability to display query results in logfmt, single-line and multi-line json modes
(cherry picked from commit 492190885d)
2024-10-07 14:46:21 +02:00
Aliaksandr Valialkin
2e5dbd6f91
app/vlogscli: return back sorting result fields by name
This simplifies locating the needed field when the number of fields per each returned result is big

(cherry picked from commit daad96b3a5)
2024-10-07 14:46:20 +02:00
Aliaksandr Valialkin
026560df73
app/vlogscli: preserve the original order of fields in the displayed responses
Some checks failed
build / Build (push) Has been cancelled
CodeQL Go / Analyze (push) Has been cancelled
main / lint (push) Has been cancelled
main / test (test-full) (push) Has been cancelled
main / test (test-full-386) (push) Has been cancelled
main / test (test-pure) (push) Has been cancelled
2024-10-05 21:30:10 +02:00
Aliaksandr Valialkin
3809efc1ad
docs/VictoriaLogs/LogsQL.md: remove redundant replace word in examples for conditional replace
Thanks to @p5i for the bugreport at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7168
2024-10-05 21:30:06 +02:00
Aliaksandr Valialkin
7a44614e0b
lib/logstorage: add len pipe for calculating byte length of log field values
(cherry picked from commit 364f084b43)
2024-10-04 10:42:51 +02:00
Aliaksandr Valialkin
ff151f2caa
docs/VictoriaLogs/querying/vlogscli.md: typo fixes 2024-10-01 21:11:51 +02:00
Aliaksandr Valialkin
856c189688
docs/VictoriaLogs/querying/vlogscli.md: provide instructions on how to download run vlogscli 2024-10-01 16:13:35 +02:00
Aliaksandr Valialkin
ee784f9334
deployment: update VictoriaLogs docker image from v0.32.1-victorialogs to v0.33.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.33.0-victorialogs
2024-10-01 13:58:36 +02:00
Aliaksandr Valialkin
b2c3dbef09
docs/VictoriaLogs/CHANGELOG.md: cut v0.33.0-victorialogs release 2024-10-01 13:42:27 +02:00
Aliaksandr Valialkin
81f3e07e1e
lib/logstorage: do not count dictionary values which have no matching logs in count_uniq stats function
Create blockResultColumn.forEachDictValue* helper functions for visiting matching
dictionary values. These helper functions should prevent from counting dictionary values
without matching logs in the future.

This is a follow-up for 0c0f013a60
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7152
2024-10-01 13:36:27 +02:00
Aliaksandr Valialkin
8c55b699f4
app/vlogscli: add interactive command-line tool for querying VictoriaLogs 2024-10-01 12:24:53 +02:00
Aliaksandr Valialkin
285a4dde11
docs/VictoriaLogs/FAQ.md: clarify answers a bit 2024-10-01 12:24:03 +02:00
Aliaksandr Valialkin
975f0f3610
deployment/docker: update VictoriaLogs from v0.32.0-victorialogs to v0.32.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.32.1-victorialogs
2024-09-30 14:38:27 +02:00
Aliaksandr Valialkin
c07746c1af
docs/VictoriaLogs/CHANGELOG.md: cut v0.32.1-victorialogs release 2024-09-30 14:30:34 +02:00
Aliaksandr Valialkin
dbcf06cd85
lib/logstorage: skip values with zero hits for 'uniq', 'top' and 'field_values' pipes
See https://github.com/VictoriaMetrics/victorialogs-datasource/issues/72#issuecomment-2352078483
2024-09-30 14:16:21 +02:00
Aliaksandr Valialkin
bf746f9975
deployment: update VictoriaLogs docker image from v0.31.0-victorialogs to v0.32.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.32.0-victorialogs
2024-09-29 14:54:42 +02:00
Aliaksandr Valialkin
3babcb0bbd
docs/VictoriaLogs/CHANGELOG.md: cut v0.32.0-victorialogs 2024-09-29 14:48:36 +02:00
Aliaksandr Valialkin
58d1e517de
lib/logstorage: clear hits slice obtained from encoding.GetUint64s() before updating it with hits for valueTypeDict column
encoding.GetUint64s() returns uninitialized slice, which may contain arbitrary values.
So values in this slice must be reset to zero before using it for counting hits in `uniq` and `top` pipes.
2024-09-29 10:29:50 +02:00
Aliaksandr Valialkin
7f8b1300a9
lib/logstorage: add non-empty if (...) condition to automatically generated result names in stats pipe
This allows executing queries with `stats` pipe, which calculate multiple results with the same functions,
but with different `if (...)` conditions. For example:

  _time:5m | count(), count() if (error)

Previously such queries couldn't be executed becasue automatically generated name for the second result
didn't include `if (error)`, so names for both results were identical - `count(*)`.
2024-09-29 09:52:19 +02:00
Aliaksandr Valialkin
04c73d54d4
lib/logstorage: support order alias for sort pipe
Now the following queries are equivalents:

    _time:5s | sort by (_time)

    _time:5s | order by (_time)

This is needed for convenience, since `order by` is commonly used in other query languages such as SQL.
2024-09-29 09:52:18 +02:00
Aliaksandr Valialkin
0f1b3852dd
app/vlinsert: support unix timestamps in seconds and milliseconds in JSON stream data ingestion API 2024-09-28 21:57:19 +02:00
Aliaksandr Valialkin
b8fa213310
app/vlinsert: accept unix timestamp in seconds additionally to milliseconds at ElasticSearch bulk API
Timestamps in seconds are sometimes used for data ingestion via ElasticSearch bulk API
2024-09-28 21:21:19 +02:00
Andrii Chubatiuk
6adb3f0dc2
victorialogs: marked fluentd support in roadmap, added syslog example (#7098)
### Describe Your Changes

Marked fluentd in victorialogs roadmap
Added fluentd syslog example setup

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 05a64a8c14)
2024-09-27 14:43:04 +02:00
Aliaksandr Valialkin
7ef9600da9
deployment: update VictoriaLogs docker image from v0.30.1-victorialogs to v0.31.0-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.31.0-victorialogs
2024-09-27 13:59:49 +02:00
Aliaksandr Valialkin
8c62845211
docs/VictoriaLogs/CHANGELOG.md: cut v0.31.0-victorialogs release 2024-09-27 13:54:24 +02:00
Yury Molodov
64793ff5f0
vmui/logs: improve graph usability (#7025)
### Describe Your Changes

- Show the time range in the tooltip when hovering over staircase
graphs.
- Use bolder lines for staircase graphs.
- Increase the number of steps on the staircase graph to 100.
- Reduce the maximum width of the tooltip to 1/3 of the screen.
- Insert only the label name under the cursor into the query input field
when `Ctrl`-clicking the line legend.

See [this
comment](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545#issuecomment-2336805237).

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-09-27 13:20:18 +02:00
Aliaksandr Valialkin
1a6313ca68
lib/logstorage: allow using - instead of ! as a shorthand for NOT operator in LogsQL 2024-09-27 13:15:55 +02:00
Aliaksandr Valialkin
b60cb98377
lib/logstorage: support skipping _stream: prefix for stream filters
'_stream:{...}' can be written as '{...}'

This simplifies writing queries with stream filters, and makes them more familier to Loki users.
2024-09-27 13:15:55 +02:00
Yury Molodov
b95af2accf
vmui: add functionality to preserve selected columns (#7037)
### Describe Your Changes

1) Changed table settings from a popup to a modal window to simplify
future functionality additions.
2) Added functionality to save selected columns when data is modified or
the page is reloaded. See #7016.

<details>
  <summary>Example screenshots</summary>

<img alt="demo-1" width="600"
src="https://github.com/user-attachments/assets/a5d9a910-363c-4931-8b12-18ea8b3d97d8"/>

</details>

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit c896bf340d)
2024-09-27 12:40:52 +02:00
Aliaksandr Valialkin
fc6ddad117
deployment: update VictoriaLogs docker image tag from v0.30.0-victorialogs to v0.30.1-victorialogs
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.30.1-victorialogs
2024-09-27 11:26:31 +02:00
Aliaksandr Valialkin
ebb2c605f9
docs/VictoriaLogs/CHANGELOG.md: cut v0.30.1-victorialogs release 2024-09-27 11:21:28 +02:00