Commit Graph

4312 Commits

Author SHA1 Message Date
Phuong Le
a0920d2f27
docs: use white version of logo for dark theme (#6610)
The logo on the README is hard to recognize in dark mode on GitHub, so I
made it responsive based on user preferences.

(cherry picked from commit c341369fc1)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-09 11:20:40 +02:00
hagen1778
f33f7d60ed
docs: follow-up 0e1dbdee28
* restore old anchor names to keep links compatibility.
See https://docs.victoriametrics.com/#documentation requirements
* consistently use the same format for commands `sh` as it makes it better
renderred and automatically adds `copy` button to fileds with commands
* simplify the text by removing extra points in the list
* add recommendations for installing the cluster setup
* explicitly mention the ports services are listening on
* add description for `storageNode` cmd-line flag to inform the reader what
values need to be put into it
* fix the incorrect vmui link in cluster installation recommendation
* rename component anchors to be more unique, because URL doesn't respect
hierarchy for the anchored links and may result into conflicts in future

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6e17255ec0)
2024-07-09 10:53:07 +02:00
Artem Navoiev
b73214cb35
add alt to cluster logo, this allows to hide it on our docs site
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 878c727a3a)
2024-07-09 10:53:04 +02:00
Mathias Palmersheim
3ec15e9ba8
Added Binary Install Docs (#6598)
### Describe Your Changes

Updated Quickstart guide for VIctoriaMetrics and VictoriaMetrics Cluster to include instructions for installing the binaries by hand

### Checklist

The following checks are **mandatory**:

- [ x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 0e1dbdee28)
2024-07-09 10:53:00 +02:00
Github Actions
3eb8068b77
Automatic update operator docs from VictoriaMetrics/operator@9ef3d95 (#6596)
Automated changes by
[create-pull-request](https://github.com/peter-evans/create-pull-request)
GitHub action

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
(cherry picked from commit f41f5ef62e)
2024-07-09 10:52:57 +02:00
Daria Karavaieva
0897887d07
docs/vmanomaly:custom model guide fix (#6594)
### Describe Your Changes

Fixed Custom Model guide according to newer `vmanomaly` versions

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 7478d2de4c)
2024-07-09 10:52:55 +02:00
Github Actions
5fd4edbf8c
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@e77b503 (#6601)
(cherry picked from commit 108a5e10d8)
2024-07-09 10:52:53 +02:00
omahs
efc6b00b2c
docs: fix typos (#6600)
### Describe Your Changes

docs: fix typos

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 8786a08d27)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-09 10:52:47 +02:00
Aliaksandr Valialkin
f3be3573e7
lib/promscrape/discovery/vultr: follow-up after 17e3d019d2
- Sort the discovered labels in alphabetical order at https://docs.victoriametrics.com/sd_configs/#vultr_sd_configs
- Rename VultrConfigs to VultrSDConfigs to be consistent with the naming for other SD configs.
- Prepare query arg filters for `list instances API` at newAPIConfig() instead of passing them in a separate listParams struct.
  This simplifies the code a bit.
- Return error when bearer token isn't set at vultr_sd_configs, since this token is mandatory
  according to https://docs.victoriametrics.com/sd_configs/#vultr_sd_configs
- Remove unused fields from the parsed response from Vultr list instances API in order to simplify the code a bit.
- Remove double logging of errors inside getInstances() function, since these errors must be already logged by the caller.
- Simplify tests, so they are easier to maintain.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6041
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6068
2024-07-05 17:40:39 +02:00
Github Actions
67a7cf37c2
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@0c2f8ef (#6592)
(cherry picked from commit 5275b36380)
2024-07-05 15:16:14 +02:00
Artem Navoiev
0d7bed212f
docs: remove templates
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit d5e4857a27)
2024-07-05 15:14:01 +02:00
Github Actions
6178e48e01
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@e0a0b2d (#6591)
(cherry picked from commit fe42884b1b)
2024-07-05 15:13:56 +02:00
hagen1778
819f1da12b
docs: clarify multitenancy via labels for vmagent
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 4d92f875fd)
2024-07-05 13:58:22 +02:00
Github Actions
6f12ca71eb
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@47c26c4 (#6586)
(cherry picked from commit 7e781f0f78)
2024-07-05 09:49:32 +02:00
hagen1778
7d98c200eb
docs: update refereneces to victoriametrics-datasource plugin
The plugin was renamed in https://github.com/VictoriaMetrics/victoriametrics-datasource/pull/178

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit ee66fb4387)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-05 09:49:28 +02:00
Yury Molodov
7fc9912d15
vmui: add compact JSON display (#6582)
### Describe Your Changes
If a JSON element has only one field, it will be displayed on a single
line.
 #6559

| Old Display | New Display |
|-------------|-------------|
|
![image](https://github.com/VictoriaMetrics/VictoriaMetrics/assets/29711459/8866517b-a49d-450f-904c-19117397a078)
|
![image](https://github.com/VictoriaMetrics/VictoriaMetrics/assets/29711459/8e222b43-a4cb-4f32-9a79-6199778404d3)
|

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 959a4383c5)
2024-07-05 09:49:12 +02:00
Github Actions
9e324cc28a
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@b326811 (#6580)
(cherry picked from commit 7e169a767d)
2024-07-05 09:49:10 +02:00
Github Actions
8df41f37e6
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@4ed20bd (#6578)
(cherry picked from commit 11989db586)
2024-07-05 09:49:08 +02:00
Hui Wang
bbd49a1a61
vmalert: allow omitting -replay.timeTo in replay mode, default valu… (#6575)
…e is the current timestamp

address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6492

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 3169524fb7)
2024-07-05 09:49:06 +02:00
Roman Khavronenko
b13c363f12
app/vmalert: add examples for source override (#6561)
The change adds a new docs section with examples on how source can be
overridden. It should address questions like
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6536

While there, fix the example in `external.alert.source` cmd-line flag
and docker-compose examples.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit c429bbf889)
2024-07-05 09:49:03 +02:00
Aliaksandr Valialkin
210e5b8957
docs/VictoriaLogs/CHANGELOG.md: cut v0.27.1-victorialogs 2024-07-05 01:22:50 +02:00
Aliaksandr Valialkin
6397c38a0a
lib/logstorage: use quicktemplate.AppendJSONString instead of strconv.AppendQuote for encoding JSON strings
The strconv.AppendQuote improperly encodes special chars such as \x1b . They must be encoded as \u001b .

See https://github.com/VictoriaMetrics/victorialogs-datasource/issues/24
2024-07-05 01:22:49 +02:00
Github Actions
3d6e3ee98f
Automatic update operator docs from VictoriaMetrics/operator@0c311bb (#6574) 2024-07-04 11:46:16 +02:00
Aliaksandr Valialkin
172ae1adf7
Revert c6c5a5a186 and b2765c45d0
Reason for revert:

There are many statsd servers exist:

- https://github.com/statsd/statsd - classical statsd server
- https://docs.datadoghq.com/developers/dogstatsd/ - statsd server from DataDog built into DatDog Agent ( https://docs.datadoghq.com/agent/ )
- https://github.com/avito-tech/bioyino - high-performance statsd server
- https://github.com/atlassian/gostatsd - statsd server in Go
- https://github.com/prometheus/statsd_exporter - statsd server, which exposes the aggregated data as Prometheus metrics

These servers can be used for efficient aggregating of statsd data and sending it to VictoriaMetrics
according to https://docs.victoriametrics.com/#how-to-send-data-from-graphite-compatible-agents-such-as-statsd (
the https://github.com/prometheus/statsd_exporter can be scraped as usual Prometheus target
according to https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter ).

Adding support for statsd data ingestion protocol into VictoriaMetrics makes sense only if it provides
significant advantages over the existing statsd servers, while has no significant drawbacks comparing
to existing statsd servers.

The main advantage of statsd server built into VictoriaMetrics and vmagent - getting rid of additional statsd server.
The main drawback is non-trivial and inconvenient streaming aggregation configs, which must be used for the ingested statsd metrics (
see https://docs.victoriametrics.com/stream-aggregation/ ). These configs are incompatible with the configs for standalone statsd servers.
So you need to manually translate configs of the used statsd server to stream aggregation configs when migrating
from standalone statsd server to statsd server built into VictoriaMetrics (or vmagent).

Another important drawback is that it is very easy to shoot yourself in the foot when using built-in statsd server
with the -statsd.disableAggregationEnforcement command-line flag or with improperly configured streaming aggregation.
In this case the ingested statsd metrics will be stored to VictoriaMetrics as is without any aggregation.
This may result in high CPU usage during data ingestion, high disk space usage for storing all the unaggregated
statsd metrics and high CPU usage during querying, since all the unaggregated metrics must be read, unpacked and processed
during querying.

P.S. Built-in statsd server can be added to VictoriaMetrics and vmagent after figuring out more ergonomic
specialized configuration for aggregating of statsd metrics. The main requirements for this configuration:

- easy to write, read and update (ideally it should work out of the box for most cases without additional configuration)
- hard to misconfigure (e.g. hard to shoot yourself in the foot)

It would be great if this configuration will be compatible with the configuration of the most widely used statsd server.

In the mean time it is recommended continue using external statsd server.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6265
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5053
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5052
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/206
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4600
2024-07-03 23:57:49 +02:00
Github Actions
bc0303bca3
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@6182902 (#6572) 2024-07-03 15:22:50 +02:00
Github Actions
730088caaf
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@a04844a (#6571) 2024-07-03 15:22:50 +02:00
Github Actions
a72d0310c0
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@a53ccd7 (#6570) 2024-07-03 14:20:56 +02:00
Aliaksandr Valialkin
4268a310c1
app/vmagent/remotewrite/remotewrite.go: make remoteWriteCtx.TryPush code easier to follow
Move the code responsible for relabelCtx clearing into deferred function.
This allows making more clear the remoteWriteCtx.TryPush code.

This is a follow-up for 879771808b

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6205

While at it, clarify the description of the bugfix at docs/CHANGELOG.md
2024-07-03 14:18:51 +02:00
Aliaksandr Valialkin
f406764ccc
app/vmagent/remotewrite/streamaggr.go: clarify the description for -remoteWrite.streamAggr.* command-line flags, so they are applied to the corresponding -remoteWrite.url 2024-07-03 14:18:51 +02:00
Artem Navoiev
f327eac81b
add file placeholder grafana logs datasource docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-03 14:18:50 +02:00
Artem Navoiev
7ddaf51414
remove unsued image
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-03 14:18:50 +02:00
Github Actions
3a11c84ee8
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@dae8560 (#6568)
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-03 14:18:49 +02:00
Aliaksandr Valialkin
f9b64cdfe0
docs/MetricsQL.md: document which metric types are usually passed to which rollup functions
This should reduce incorrect usage of rollup functions like this one - https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3974#issuecomment-2205574667
2024-07-03 11:36:58 +02:00
Aliaksandr Valialkin
547aa6108e
docs: update VictoriaMetrics release from v1.100.1 to v1.101.0 across all the docs
This is a follow-up for 5e8c087d42

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6194
2024-07-03 01:07:07 +02:00
Aliaksandr Valialkin
bb7406e9c0
app/vmselect/promql: follow-up for dd0d2c77c8 and 6149adbe10
Use metricsql.IsLikelyInvalid() function for determining whether the given query is likely invalid,
e.g. there is high change the query is incorrectly written, so it will return unexpected results.

The query is invalid most of the time if it passes something other than series selector into rollup function.
For example:

- rate(sum(foo))
- rate(foo + bar)
- rate(foo > bar)

Improtant note: the query is considered valid if it misses the lookbehind window in square brackes inside rollup function,
e.g. rate(foo), since this is very convenient MetricsQL extention to PromQL, and this query returns the expected results
most of the time.

Other unsafe query types can be added in the future into metricsql.IsLikelyInvalid().

TODO: probably, the -search.disableImplicitConversion command-line flag must be set by default in the future releases of VictoriaMetrics.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4338
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6180
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6450
2024-07-03 00:46:56 +02:00
Aliaksandr Valialkin
82748b2b9d
deployment/docker: update Go builder from Go1.22.4 to Go1.22.5
See https://github.com/golang/go/issues?q=milestone%3AGo1.22.5+label%3ACherryPickApproved
2024-07-03 00:07:55 +02:00
Aliaksandr Valialkin
f8779d1ed2
lib/streamaggr: follow-up for the commit c0e4ccb7b5
- Clarify docs for `Ignore aggregation intervals on start` feature.

- Make more clear the code dealing with ignoreFirstIntervals at aggregator.runFlusher() functions.
  It is better from readability and maintainability PoV using distinct a.flush() calls
  for distinct cases instead of merging them into a single a.flush() call.

- Take into account the first incomplete interval when tracking the number of skipped aggregation intervals,
  since this behaviour is easier to understand by the end users.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6137
2024-07-02 21:34:48 +02:00
hagen1778
e423c4b72c
docs: mention graphite.sanitizeMetricName in cluster docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit ffb49c677b)
2024-07-02 17:18:07 +02:00
Andrii Chubatiuk
252aa5a3ab
lib/protoparser/graphite: added -graphite.sanitizeMetricName flag (#6489)
### Describe Your Changes

Added flag to sanitize graphite metrics
fixes #6077

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 476faf5578)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-02 17:16:00 +02:00
Zakhar Bessarab
4ece1747d3
vmbackupmanager: fix state restore (#773)
* app/vmbackupmanager: fix state restore on startup

Fix improperly treating completed backups as "failed" in metrics.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/changelog: document the fix

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f3831bdd13)
2024-07-02 14:37:15 +02:00
LHHDZ
c8431c8e4d
app/vmauth: reader pool to reduce gc & mem alloc (#6533)
follow up https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6446

issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6445

---------

Signed-off-by: f41gh7 <nik@victoriametrics.com>
Co-authored-by: f41gh7 <nik@victoriametrics.com>
(cherry picked from commit 4d66e042e3)
2024-07-02 14:37:15 +02:00
Github Actions
3fda4cc4de
Automatic update Grafana datasource docs from VictoriaMetrics/victoriametrics-datasource@e9bd079 (#6564)
(cherry picked from commit dd97dd6373)
2024-07-02 14:37:15 +02:00
Aliaksandr Valialkin
fed596b202
docs/VictoriaLogs/CHANGELOG.md: cut v0.27.0-victorialogs 2024-07-02 01:43:54 +02:00
Aliaksandr Valialkin
0912a652d5
app/vlinsert/insertutils: flush the ingested logs from in-memory buffer to storage every second
Previously the in-memory buffer could remain unflushed for long periods of time under low ingestion rate.
The ingested logs weren't visible for search during this time.
2024-07-02 01:39:45 +02:00
Aliaksandr Valialkin
ab28a1f93e
app/vlinsert/syslog: add an ability to use log ingestion time as the _time field 2024-07-02 01:39:45 +02:00
Aliaksandr Valialkin
f4dd9bd988
docs/VictoriaLogs/CHANGELOG.md: use new url https://docs.victoriametrics.com/victorialogs/querying/ instead of old one https://docs.victoriametrics.com/VictoriaLogs/querying/
(cherry picked from commit 387b3b7fb7)
2024-07-01 16:40:44 +02:00
Andrii Chubatiuk
65c742d976
deployment: remove snap packages support (#6543)
### Describe Your Changes

Removed snap packages support as it requires time for maintenance and
it's not popular at all

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 0a42c8fd8b)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-01 16:40:39 +02:00
Hui Wang
87cb132f53
app/vmselect/netstorage: do not retry request when complexity limit i… (#6469)
…s already exceeded

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-01 16:38:15 +02:00
Andrii Chubatiuk
937ae2ca90
lib/streamaggr: added stale samples metric, added metrics labels (#6462)
### Describe Your Changes

- added stale metrics counters for input and output samples
- added labels for aggregator metrics =>
`name="{rwctx}:{aggrId}:{aggrSuffix}"`
   - rwctx - global or number starting from 1
   - aggrid - aggregator id starting from 1
   - aggrSuffix - <interval>_(by|without)_label1_label2_labeln
   e.g: `name="global:1:1m_without_instance_pod"`

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 861852f262)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-07-01 15:01:49 +02:00
hagen1778
69625aa8a1
docs: mark without optional in stream aggr docs
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f8eea0f2c9)
2024-07-01 14:56:46 +02:00
Aliaksandr Valialkin
5143e93534
docs/VictoriaLogs/CHANGELOG.md: cut v0.26.1-victorialogs 2024-07-01 02:33:02 +02:00
Aliaksandr Valialkin
208a624d4d
lib/logstorage: properly search for the surrounding logs in stream_context pipe
The set of log fields in the found logs may differ from the set of log fields present in the log stream.
So compare only the log fields in the found logs when searching for the matching log entry in the log stream.

While at it, return _stream field in the delimiter log entry, since this field is used by VictoriaLogs Web UI
for grouping logs by log streams.
2024-07-01 02:33:00 +02:00
Aliaksandr Valialkin
516c7b2ca0
docs/VictoriaLogs: remove "preview" warning - VictoriaLogs is ready for prod 2024-07-01 01:52:40 +02:00
Aliaksandr Valialkin
0e6ee55a95
docs/VictoriaLogs/CHANGELOG.md: cut v0.26.0-victorialogs 2024-07-01 01:49:55 +02:00
Aliaksandr Valialkin
76a58ae08d
lib/logstorage: add ability to store sorted log position into a separate field with sort ... rank <fieldName> syntax 2024-07-01 01:46:03 +02:00
Aliaksandr Valialkin
d0dca7b8c5
lib/logstorage: add delimiter between log chunks returned from | stream_context pipe 2024-07-01 01:46:02 +02:00
Artem Navoiev
2ac2d0919d
operator: sync docs with operator repo, add alias
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-06-29 17:19:49 +02:00
Aliaksandr Valialkin
76053a0ef0
docs/VictoriaLogs: typo fixes 2024-06-28 19:26:32 +02:00
Aliaksandr Valialkin
75c7b2c07a
docs/VictoriaLogs/CHANGELOG.md: cut v0.25.0-victorialogs 2024-06-28 19:18:21 +02:00
Aliaksandr Valialkin
4b3477e62b
lib/logstorage: add stream_context pipe, which allows selecting surrounding logs for the matching logs 2024-06-28 19:15:19 +02:00
Aliaksandr Valialkin
c9fc8079c4
app/vlinsert/syslog: properly skip empty lines in Syslog protocol
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6548
2024-06-28 14:09:45 +02:00
Aliaksandr Valialkin
bb6424aeca
app/vlselect/logsql: add optional fields_limit query arg to /select/logsql/hits HTTP endpoint
This query arg is needed for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6545
in order to return top N groups with the biggest number of hits.
2024-06-28 03:10:05 +02:00
Aliaksandr Valialkin
3eecc3de8c
docs/VictoriaLogs: typo fixes 2024-06-28 03:10:05 +02:00
Aliaksandr Valialkin
5a24ea6cc3
docs/VictoriaLogs: document how to export logs from VictoriaLogs
While at it, mention vmctl-like tool for migrating logs from other systems to VictoriaLogs at docs/VictoriaLogs/Roadmap.md

Thanks to the question from Xavier Pestel at https://www.linkedin.com/feed/update/urn:li:activity:7212093021301927937?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A7212093021301927937%2C7212171550651731969%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287212171550651731969%2Curn%3Ali%3Aactivity%3A7212093021301927937%29
2024-06-28 02:06:24 +02:00
Aliaksandr Valialkin
05365999ea
docs/VictoriaLogs/querying/README.md: clarify live tailing docs 2024-06-27 19:12:54 +02:00
Aliaksandr Valialkin
4391ef25e9
docs/VictoriaLogs/Roadmap.md: remove Web UI from roadmap, since it is already implemented
See https://docs.victoriametrics.com/victorialogs/querying/#web-ui

While at it, add DataDog protocol for logs into 'data ingestion protocols' roadmap.
2024-06-27 19:12:54 +02:00
Aliaksandr Valialkin
00899506f7
docs/VictoriaLogs/querying: remove unused and outdated vmui.webp image
The image has been deleted from the docs/VictoriaLogs/querying/README.md file
in the commit 87f1c8bd6c
2024-06-27 15:16:43 +02:00
Aliaksandr Valialkin
13df75a28b
docs/VictoriaLogs/querying/README.md: add a link to live tailing docs from querying log docs 2024-06-27 15:12:59 +02:00
Aliaksandr Valialkin
b26acec9a8
app/vlselect: properly return live tailing results 2024-06-27 15:06:15 +02:00
Aliaksandr Valialkin
dd62a2b9d6
lib/logstorage: work-in-progress 2024-06-27 14:21:03 +02:00
Aliaksandr Valialkin
0352d98d74
docs/goals.md: clarify that the usability of the existing functionality must be improved the first 2024-06-27 09:55:40 +02:00
Josh Keife
bdebc128f7
Update vmalert.md (#6539)
Fix typo

### Describe Your Changes

Fix typo in the vmalert docs. In the docs it states rules at `VMAgent`'s
namespace when it should be `VMAlert`'s namespace.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 8d01dd0514)
2024-06-27 09:32:07 +02:00
Aliaksandr Valialkin
911891da95
docs/goals.md: clarify that the listed goals are related to development and support, not sales and marketing 2024-06-26 19:12:36 +02:00
Fred Navruzov
ed41f18c5a
docs/vmanomaly: quickstart page improvements (#6538)
### Describe Your Changes

Small improvements to a QuickStart guide of `vmanomaly`

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-06-26 19:12:35 +02:00
Andrii Chubatiuk
580d02c3f8
added IMDSv2 for YC SD (#6524)
### Describe Your Changes

Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5513

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-06-26 19:12:35 +02:00
Andrii Chubatiuk
f79df2aa8b
app/vmauth: allow dropping host header (#6525)
### Describe Your Changes

Fixes #6453

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-06-26 19:12:35 +02:00
rtm0
48a5c4cb01
Fix Date metricid cache consistency under concurrent use (#6534)
### Describe Your Changes

Fix Date metricid cache consistency under concurrent use.
When one goroutine calls Has() and does not find the cache entry in the
immutable map it will acquire a lock and check the mutable map. And it
is possible that before that lock is acquired, the entry is moved from
the mutable map to the immutable map by another goroutine causing a
cache miss.

The fix is to check the immutable map again once the lock is acquired. 

### Checklist

The following checks are **mandatory**:

- [x ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Fetishev <wwctrsrx@gmail.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-06-26 19:12:34 +02:00
Aliaksandr Valialkin
3a2c0b0a61
docs/goals.md: heading fixes 2024-06-26 16:41:21 +02:00
Aliaksandr Valialkin
d2be74b4a1
docs/goals.md: add VictoriaMetrics proverbs 2024-06-26 14:01:15 +02:00
Aliaksandr Valialkin
75d45068a5
docs/VictoriaLogs/README.md: document the relation between -retention.maxDiskSpaceUsageBytes and -retentionPeriod command-line flags 2024-06-26 12:03:29 +02:00
Aliaksandr Valialkin
db5a5a4f6e
docs/goals.md: add additional non-goals, which frequently become false goals according to the development history of VictoriaMetrics 2024-06-26 12:03:28 +02:00
Yury Molodov
6bde0196d8
vmui/logs: fix the update of the relative time range (#6517)
### Describe Your Changes

- Fixed the update of the relative time range when `Execute Query` is
clicked
- Optimized server requests: now, if an error occurs in the `/query`
request, the `/hits` request will not be executed.

#6345 (duplicates: #6440, #6312)

(cherry picked from commit 43342745ac)
2024-06-26 11:26:08 +02:00
James Rhoat
8249de986e
Updating documentation around OTEL (#6519)
Updating documentation around the opentelemetry endpoint for metrics and
the "How to use OpenTelemetry metrics with VictoriaMetrics" guide so
that it shows not only how to directly write but also how to write to
the otel collector and view metrics in vmui.

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

(cherry picked from commit 6652fb630f)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-26 11:26:05 +02:00
Yury Molodov
904ec020ed
vmui: fix input cursor position reset (#6530)
### Describe Your Changes

This PR addresses the issue where the cursor jumps to the end of the
input fields in the modal settings window after each keystroke.

### Before fix:

![ezgif-7-4c69805cea](https://github.com/VictoriaMetrics/VictoriaMetrics/assets/29711459/2e99e833-09e3-4b44-89aa-fc1bd3c4346d)

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit e9b71a2883)
2024-06-26 11:25:47 +02:00
hagen1778
6ad66b3317
docs: rm mentions of vmrecover as it still doesn't exist
While there, add a hint about truncated error messages.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 6775a00622)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-26 11:25:42 +02:00
Daria Karavaieva
affde1794f
docs/vmanomaly:quickstart example fix (#6535)
- Fixed config example in QuickStart vmanomaly docs for 1.13 version
compatibility

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit 4b77778ff5)
2024-06-26 11:25:15 +02:00
Aliaksandr Valialkin
2dfbc40563
docs/goals.md: add VictoriaMetrics goals and non-goals page
Everybody who works on VictoriaMetrics must be aware of these goals and non-goals
2024-06-25 18:00:55 +02:00
Aliaksandr Valialkin
d5cbda3424
app/vlstorage: add -retention.maxDiskSpaceUsageBytes command-line flag for limiting the retention at VictoriaLogs by disk space usage 2024-06-25 17:30:46 +02:00
Aliaksandr Valialkin
252ff71a49
docs/VictoriaLogs/data-ingestion/README.md: document that Rsyslog and Syslog-ng can be used for ingesting logs into VictoriaLogs 2024-06-25 15:15:53 +02:00
Aliaksandr Valialkin
0a88a55ae0
docs/VictoriaLogs/CHANGELOG.md: cut v0.23.0-victorialogs release 2024-06-25 14:59:31 +02:00
Aliaksandr Valialkin
f24123a776
lib/logstorage: parse syslog structured data into separate fields in order to simplify further querying of this data 2024-06-25 14:54:25 +02:00
Aliaksandr Valialkin
1716c4e609
lib/logstorage: properly parse timezone offset at TryParseTimestampRFC3339Nano()
The TryParseTimestampRFC3339Nano() must properly parse RFC3339 timestamps with timezone offsets.

While at it, make tryParseTimestampISO8601 function private in order to prevent
from improper usage of this function from outside the lib/logstorage package.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6508
2024-06-25 14:54:24 +02:00
hagen1778
26d58f8768
docs: add missing ticket links to the recent releases
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-25 14:54:24 +02:00
hagen1778
841305da8a
deployment: fix incorrect link to vmui from VictoriaMetrics plugin in Grafana
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 0dcb3acbf7)
2024-06-25 13:58:59 +02:00
Nikolay
bf1464fc33
dashboards: add dashboard and alerts for vmauth (#6491)
Signed-off-by: f41gh7 <nik@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
(cherry picked from commit 14b9ef1e4d)
2024-06-25 11:17:04 +02:00
hagen1778
8237b63fe3
docs: make docs-sync
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit cc6a1da0d7)
2024-06-25 11:17:04 +02:00
Roman Khavronenko
e3da10a7c1
docs: add section for docs modification (#6528)
While there, remove the obsolete files from the previous version of
docs.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 87edfa61c1)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-06-25 11:17:03 +02:00
Aliaksandr Valialkin
a5445e09c2
Revert "app/vmselect: fix the way of counting raw samples in single query (#6464)"
This reverts commit 5ecf439078.

Reason for revert: the previous logic was correct.

The purpose of `-search.maxSamplesPerQuery` command-line flag is to limit the amounts of CPU resources,
which could be taken by a single query - see https://docs.victoriametrics.com/#resource-usage-limits .
VictoriaMetrics processes samples in blocks during querying - it reads the block, then unpacks it,
then filters out samples outside the selected time range. This means that it _spends CPU time_
on reading and unpacking of _all the samples_ in every block on the requested time range,
even if only a single sample per each block matches the given time range.

The previous logic was effectively limiting CPU time a single query could take.
The new logic fails limiting CPU time a single query could take in some pathological cases
when only a small fraction of samples per each requested block fit the requested time range.
This allows performing multiplication DoS-attacks by querying very narrow time ranges over historical blocks,
which tend to be full. For example, if the `-search.maxSamplesPerQuery` equals to a billion,
and the query requests a single sample out of 8K samples per each block, this means that the query
may unpack a billion of such blocks without exceeding the limit, e.g. it may unpack and process 8K*1e9=8e12 samples.
This is not what the resource usage limits were created for originally - see https://docs.victoriametrics.com/#resource-usage-limits

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5851
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6464
2024-06-25 02:55:43 +02:00
Aliaksandr Valialkin
813a7aec6c
docs: return back spellcheck fixes from the commit 92b22581e6, which were accidentally reverted in 45cf83cd8f
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6470
2024-06-25 02:33:59 +02:00
Aliaksandr Valialkin
7f2b491748
Revert "docs: Add spellcheck workflow (#6470)"
This reverts commit 92b22581e6.

Reason for revert: too complex and slow approach for spellchecking task.

This approach may significantly slow down development pace. It also may take non-trivial
amounts of additional time and resources at CI/CD because of all this npm shit at cspell directory.

Note to @arkid15r : the idea with the ability to run spellchecker on all the VictoriaMetrics
codebase is great. But this shouldn't be mandatory pre-commit check. It is enough to have
a Makefile rule like `make spellcheck`, which could be run manually whenever spellcheck is needed
(e.g. once per month or once per quarter).
2024-06-25 02:28:38 +02:00