Commit Graph

4265 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
b8a8d3d6f1
lib/logstorage: drop all the pipes from the query when calculating the number of matching logs at /select/logsql/hits API 2024-07-10 00:39:16 +02:00
Yury Molodov
33bd5ccbab
vmui/logs: add spinner to bar chart (#6577)
Add a spinner to the bar chart

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6558

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 662e026279)
2024-07-09 18:27:23 +02:00
Hui Wang
6f602a4ef5
security: upgrade base docker image (Alpine) from 3.20.0 to 3.20.1
See https://www.alpinelinux.org/posts/Alpine-3.20.1-released.html

>including security fixes for:
OPENSSL
[CVE-2024-4741](https://security.alpinelinux.org/vuln/CVE-2024-4741)
BUSYBOX
[CVE-2023-42364](https://security.alpinelinux.org/vuln/CVE-2023-42364)
[CVE-2023-42365](https://security.alpinelinux.org/vuln/CVE-2023-42365)

(cherry picked from commit 8e9f98e725)
2024-07-09 11:38:44 +02:00
Phuong Le
a0920d2f27
docs: use white version of logo for dark theme (#6610)
The logo on the README is hard to recognize in dark mode on GitHub, so I
made it responsive based on user preferences.

(cherry picked from commit c341369fc1)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-09 11:20:40 +02:00
hagen1778
f33f7d60ed
docs: follow-up 0e1dbdee28
* restore old anchor names to keep links compatibility.
See https://docs.victoriametrics.com/#documentation requirements
* consistently use the same format for commands `sh` as it makes it better
renderred and automatically adds `copy` button to fileds with commands
* simplify the text by removing extra points in the list
* add recommendations for installing the cluster setup
* explicitly mention the ports services are listening on
* add description for `storageNode` cmd-line flag to inform the reader what
values need to be put into it
* fix the incorrect vmui link in cluster installation recommendation
* rename component anchors to be more unique, because URL doesn't respect
hierarchy for the anchored links and may result into conflicts in future

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6e17255ec0)
2024-07-09 10:53:07 +02:00
Artem Navoiev
b73214cb35
add alt to cluster logo, this allows to hide it on our docs site
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 878c727a3a)
2024-07-09 10:53:04 +02:00
Mathias Palmersheim
3ec15e9ba8
Added Binary Install Docs (#6598)
### Describe Your Changes

Updated Quickstart guide for VIctoriaMetrics and VictoriaMetrics Cluster to include instructions for installing the binaries by hand

### Checklist

The following checks are **mandatory**:

- [ x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 0e1dbdee28)
2024-07-09 10:53:00 +02:00
Github Actions
3eb8068b77
Automatic update operator docs from VictoriaMetrics/operator@9ef3d95 (#6596)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
(cherry picked from commit f41f5ef62e)
2024-07-09 10:52:57 +02:00
Daria Karavaieva
0897887d07
docs/vmanomaly:custom model guide fix (#6594)
### Describe Your Changes

Fixed Custom Model guide according to newer `vmanomaly` versions

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 7478d2de4c)
2024-07-09 10:52:55 +02:00
Github Actions
5fd4edbf8c
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@e77b503 (#6601)
(cherry picked from commit 108a5e10d8)
2024-07-09 10:52:53 +02:00
omahs
efc6b00b2c
docs: fix typos (#6600)
### Describe Your Changes

docs: fix typos

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 8786a08d27)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-09 10:52:47 +02:00
Aliaksandr Valialkin
f3be3573e7
lib/promscrape/discovery/vultr: follow-up after 17e3d019d2
- Sort the discovered labels in alphabetical order at https://docs.victoriametrics.com/sd_configs/#vultr_sd_configs
- Rename VultrConfigs to VultrSDConfigs to be consistent with the naming for other SD configs.
- Prepare query arg filters for `list instances API` at newAPIConfig() instead of passing them in a separate listParams struct.
  This simplifies the code a bit.
- Return error when bearer token isn't set at vultr_sd_configs, since this token is mandatory
  according to https://docs.victoriametrics.com/sd_configs/#vultr_sd_configs
- Remove unused fields from the parsed response from Vultr list instances API in order to simplify the code a bit.
- Remove double logging of errors inside getInstances() function, since these errors must be already logged by the caller.
- Simplify tests, so they are easier to maintain.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6041
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6068
2024-07-05 17:40:39 +02:00
Github Actions
67a7cf37c2
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@0c2f8ef (#6592)
(cherry picked from commit 5275b36380)
2024-07-05 15:16:14 +02:00
Artem Navoiev
0d7bed212f
docs: remove templates
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit d5e4857a27)
2024-07-05 15:14:01 +02:00
Github Actions
6178e48e01
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@e0a0b2d (#6591)
(cherry picked from commit fe42884b1b)
2024-07-05 15:13:56 +02:00
hagen1778
819f1da12b
docs: clarify multitenancy via labels for vmagent
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 4d92f875fd)
2024-07-05 13:58:22 +02:00
Github Actions
6f12ca71eb
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@47c26c4 (#6586)
(cherry picked from commit 7e781f0f78)
2024-07-05 09:49:32 +02:00
hagen1778
7d98c200eb
docs: update refereneces to victoriametrics-datasource plugin
The plugin was renamed in https://github.com/VictoriaMetrics/victoriametrics-datasource/pull/178

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit ee66fb4387)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-05 09:49:28 +02:00
Yury Molodov
7fc9912d15
vmui: add compact JSON display (#6582)
### Describe Your Changes
If a JSON element has only one field, it will be displayed on a single
line.
 #6559

| Old Display | New Display |
|-------------|-------------|
|
![image](https://github.com/VictoriaMetrics/VictoriaMetrics/assets/29711459/8866517b-a49d-450f-904c-19117397a078)
|
![image](https://github.com/VictoriaMetrics/VictoriaMetrics/assets/29711459/8e222b43-a4cb-4f32-9a79-6199778404d3)
|

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 959a4383c5)
2024-07-05 09:49:12 +02:00
Github Actions
9e324cc28a
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@b326811 (#6580)
(cherry picked from commit 7e169a767d)
2024-07-05 09:49:10 +02:00
Github Actions
8df41f37e6
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@4ed20bd (#6578)
(cherry picked from commit 11989db586)
2024-07-05 09:49:08 +02:00
Hui Wang
bbd49a1a61
vmalert: allow omitting -replay.timeTo in replay mode, default valu… (#6575)
…e is the current timestamp

address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6492

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 3169524fb7)
2024-07-05 09:49:06 +02:00
Roman Khavronenko
b13c363f12
app/vmalert: add examples for source override (#6561)
The change adds a new docs section with examples on how source can be
overridden. It should address questions like
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6536

While there, fix the example in `external.alert.source` cmd-line flag
and docker-compose examples.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit c429bbf889)
2024-07-05 09:49:03 +02:00
Aliaksandr Valialkin
210e5b8957
docs/VictoriaLogs/CHANGELOG.md: cut v0.27.1-victorialogs 2024-07-05 01:22:50 +02:00
Aliaksandr Valialkin
6397c38a0a
lib/logstorage: use quicktemplate.AppendJSONString instead of strconv.AppendQuote for encoding JSON strings
The strconv.AppendQuote improperly encodes special chars such as \x1b . They must be encoded as \u001b .

See https://github.com/VictoriaMetrics/victorialogs-datasource/issues/24
2024-07-05 01:22:49 +02:00
Github Actions
3d6e3ee98f
Automatic update operator docs from VictoriaMetrics/operator@0c311bb (#6574) 2024-07-04 11:46:16 +02:00
Aliaksandr Valialkin
172ae1adf7
Revert c6c5a5a186 and b2765c45d0
Reason for revert:

There are many statsd servers exist:

- https://github.com/statsd/statsd - classical statsd server
- https://docs.datadoghq.com/developers/dogstatsd/ - statsd server from DataDog built into DatDog Agent ( https://docs.datadoghq.com/agent/ )
- https://github.com/avito-tech/bioyino - high-performance statsd server
- https://github.com/atlassian/gostatsd - statsd server in Go
- https://github.com/prometheus/statsd_exporter - statsd server, which exposes the aggregated data as Prometheus metrics

These servers can be used for efficient aggregating of statsd data and sending it to VictoriaMetrics
according to https://docs.victoriametrics.com/#how-to-send-data-from-graphite-compatible-agents-such-as-statsd (
the https://github.com/prometheus/statsd_exporter can be scraped as usual Prometheus target
according to https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter ).

Adding support for statsd data ingestion protocol into VictoriaMetrics makes sense only if it provides
significant advantages over the existing statsd servers, while has no significant drawbacks comparing
to existing statsd servers.

The main advantage of statsd server built into VictoriaMetrics and vmagent - getting rid of additional statsd server.
The main drawback is non-trivial and inconvenient streaming aggregation configs, which must be used for the ingested statsd metrics (
see https://docs.victoriametrics.com/stream-aggregation/ ). These configs are incompatible with the configs for standalone statsd servers.
So you need to manually translate configs of the used statsd server to stream aggregation configs when migrating
from standalone statsd server to statsd server built into VictoriaMetrics (or vmagent).

Another important drawback is that it is very easy to shoot yourself in the foot when using built-in statsd server
with the -statsd.disableAggregationEnforcement command-line flag or with improperly configured streaming aggregation.
In this case the ingested statsd metrics will be stored to VictoriaMetrics as is without any aggregation.
This may result in high CPU usage during data ingestion, high disk space usage for storing all the unaggregated
statsd metrics and high CPU usage during querying, since all the unaggregated metrics must be read, unpacked and processed
during querying.

P.S. Built-in statsd server can be added to VictoriaMetrics and vmagent after figuring out more ergonomic
specialized configuration for aggregating of statsd metrics. The main requirements for this configuration:

- easy to write, read and update (ideally it should work out of the box for most cases without additional configuration)
- hard to misconfigure (e.g. hard to shoot yourself in the foot)

It would be great if this configuration will be compatible with the configuration of the most widely used statsd server.

In the mean time it is recommended continue using external statsd server.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6265
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5053
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5052
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/206
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4600
2024-07-03 23:57:49 +02:00
Github Actions
bc0303bca3
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@6182902 (#6572) 2024-07-03 15:22:50 +02:00
Github Actions
730088caaf
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@a04844a (#6571) 2024-07-03 15:22:50 +02:00
Github Actions
a72d0310c0
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@a53ccd7 (#6570) 2024-07-03 14:20:56 +02:00
Aliaksandr Valialkin
4268a310c1
app/vmagent/remotewrite/remotewrite.go: make remoteWriteCtx.TryPush code easier to follow
Move the code responsible for relabelCtx clearing into deferred function.
This allows making more clear the remoteWriteCtx.TryPush code.

This is a follow-up for 879771808b

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6205

While at it, clarify the description of the bugfix at docs/CHANGELOG.md
2024-07-03 14:18:51 +02:00
Aliaksandr Valialkin
f406764ccc
app/vmagent/remotewrite/streamaggr.go: clarify the description for -remoteWrite.streamAggr.* command-line flags, so they are applied to the corresponding -remoteWrite.url 2024-07-03 14:18:51 +02:00
Artem Navoiev
f327eac81b
add file placeholder grafana logs datasource docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-03 14:18:50 +02:00
Artem Navoiev
7ddaf51414
remove unsued image
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-03 14:18:50 +02:00
Github Actions
3a11c84ee8
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@dae8560 (#6568)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-03 14:18:49 +02:00
Aliaksandr Valialkin
f9b64cdfe0
docs/MetricsQL.md: document which metric types are usually passed to which rollup functions
This should reduce incorrect usage of rollup functions like this one - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3974#issuecomment-2205574667
2024-07-03 11:36:58 +02:00
Aliaksandr Valialkin
547aa6108e
docs: update VictoriaMetrics release from v1.100.1 to v1.101.0 across all the docs
This is a follow-up for 5e8c087d42

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6194
2024-07-03 01:07:07 +02:00
Aliaksandr Valialkin
bb7406e9c0
app/vmselect/promql: follow-up for dd0d2c77c8 and 6149adbe10
Use metricsql.IsLikelyInvalid() function for determining whether the given query is likely invalid,
e.g. there is high change the query is incorrectly written, so it will return unexpected results.

The query is invalid most of the time if it passes something other than series selector into rollup function.
For example:

- rate(sum(foo))
- rate(foo + bar)
- rate(foo > bar)

Improtant note: the query is considered valid if it misses the lookbehind window in square brackes inside rollup function,
e.g. rate(foo), since this is very convenient MetricsQL extention to PromQL, and this query returns the expected results
most of the time.

Other unsafe query types can be added in the future into metricsql.IsLikelyInvalid().

TODO: probably, the -search.disableImplicitConversion command-line flag must be set by default in the future releases of VictoriaMetrics.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4338
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6180
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6450
2024-07-03 00:46:56 +02:00
Aliaksandr Valialkin
82748b2b9d
deployment/docker: update Go builder from Go1.22.4 to Go1.22.5
See https://github.com/golang/go/issues?q=milestone%3AGo1.22.5+label%3ACherryPickApproved
2024-07-03 00:07:55 +02:00
Aliaksandr Valialkin
f8779d1ed2
lib/streamaggr: follow-up for the commit c0e4ccb7b5
- Clarify docs for `Ignore aggregation intervals on start` feature.

- Make more clear the code dealing with ignoreFirstIntervals at aggregator.runFlusher() functions.
  It is better from readability and maintainability PoV using distinct a.flush() calls
  for distinct cases instead of merging them into a single a.flush() call.

- Take into account the first incomplete interval when tracking the number of skipped aggregation intervals,
  since this behaviour is easier to understand by the end users.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6137
2024-07-02 21:34:48 +02:00
hagen1778
e423c4b72c
docs: mention graphite.sanitizeMetricName in cluster docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit ffb49c677b)
2024-07-02 17:18:07 +02:00
Andrii Chubatiuk
252aa5a3ab
lib/protoparser/graphite: added -graphite.sanitizeMetricName flag (#6489)
### Describe Your Changes

Added flag to sanitize graphite metrics
fixes #6077

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 476faf5578)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-02 17:16:00 +02:00
Zakhar Bessarab
4ece1747d3
vmbackupmanager: fix state restore (#773)
* app/vmbackupmanager: fix state restore on startup

Fix improperly treating completed backups as "failed" in metrics.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/changelog: document the fix

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f3831bdd13)
2024-07-02 14:37:15 +02:00
LHHDZ
c8431c8e4d
app/vmauth: reader pool to reduce gc & mem alloc (#6533)
follow up https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6446

issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6445

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
(cherry picked from commit 4d66e042e3)
2024-07-02 14:37:15 +02:00
Github Actions
3fda4cc4de
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@e9bd079 (#6564)
(cherry picked from commit dd97dd6373)
2024-07-02 14:37:15 +02:00
Aliaksandr Valialkin
fed596b202
docs/VictoriaLogs/CHANGELOG.md: cut v0.27.0-victorialogs 2024-07-02 01:43:54 +02:00
Aliaksandr Valialkin
0912a652d5
app/vlinsert/insertutils: flush the ingested logs from in-memory buffer to storage every second
Previously the in-memory buffer could remain unflushed for long periods of time under low ingestion rate.
The ingested logs weren't visible for search during this time.
2024-07-02 01:39:45 +02:00
Aliaksandr Valialkin
ab28a1f93e
app/vlinsert/syslog: add an ability to use log ingestion time as the _time field 2024-07-02 01:39:45 +02:00
Aliaksandr Valialkin
f4dd9bd988
docs/VictoriaLogs/CHANGELOG.md: use new url https://docs.victoriametrics.com/victorialogs/querying/ instead of old one https://docs.victoriametrics.com/VictoriaLogs/querying/
(cherry picked from commit 387b3b7fb7)
2024-07-01 16:40:44 +02:00
Andrii Chubatiuk
65c742d976
deployment: remove snap packages support (#6543)
### Describe Your Changes

Removed snap packages support as it requires time for maintenance and
it's not popular at all

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 0a42c8fd8b)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-01 16:40:39 +02:00