Commit Graph

8378 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
590160ddbb
lib/slicesutil: add helper functions for setting slice length and extending its capacity
The added helper functions - SetLength() and ExtendCapacity() - replace error-prone code with simple function calls.
2024-05-12 11:32:17 +02:00
Aliaksandr Valialkin
f20d452196
lib/storage: remove outdated misleading comments 2024-05-12 10:24:04 +02:00
Aliaksandr Valialkin
92de6ea340
app/vmselect: use strings.EqualFold instead of strings.ToLower where appropriate
Strings.EqualFold doesn't allocate memory contrary to strings.ToLower if the input string contains uppercase chars
2024-05-12 10:20:41 +02:00
Aliaksandr Valialkin
95608885ea
app/vmselect/promql: properly estimate the needed amounts of memory for executing aggregate function over rollup function in incremental mode
Incremental aggregation processes only GOMAXPROCS time series at a time, so its' memory usage doesn't depend
on the number of input time series.

The issue has been introduced in 5138eaeea0

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3203
2024-05-12 10:14:11 +02:00
Aliaksandr Valialkin
de7fc743ca
README.md: mention that -tlsCertFile and -tlsKeyFile options aren't needed when automatic issuing of tls certificates is enabled with -tlsAutoCertHosts flag 2024-05-12 09:48:40 +02:00
Aliaksandr Valialkin
f4051dd1e0
docs/Single-server-VictoriaMetrics.md and docs/README.md: sync with README.md with make docs-sync after the commit c6c5a5a186 2024-05-12 09:40:27 +02:00
Roman Khavronenko
87fd400dfc
Feature allow configuring disableOnDiskQueue and dropSamplesOnOverload per url (#6248)
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html):
allow configuring `-remoteWrite.disableOnDiskQueue` and
`-remoteWrite.dropSamplesOnOverload` cmd-line flags per each
`-remoteWrite.url`. See this [pull
request](https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6065).
Thanks to @rbizos for implementaion!
* FEATURE: [vmagent](https://docs.victoriametrics.com/vmagent.html): add
labels `path` and `url` to metrics
`vmagent_remotewrite_push_failures_total` and
`vmagent_remotewrite_samples_dropped_total`. Now number of failed pushes
and dropped samples can be tracked per `-remoteWrite.url`.

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Raphael Bizos <r.bizos@criteo.com>
2024-05-10 12:09:21 +02:00
Github Actions
c87ce86d96
Automatic update operator docs from VictoriaMetrics/operator@0829591 (#6250) 2024-05-10 10:57:10 +08:00
qiangxuhui
80f3644ee3
Add build support for loong64 (#6222)
### Describe Your Changes

Added makefile rule for `GOARCH=loong64` to support building all
VictoriaMetrics components on the `loongarch64` platform.


### Checklist

The following checks are **mandatory**:
 
* [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: qiangxuhui <qiangxuhui@loongson.cn>
2024-05-09 14:22:03 +02:00
Github Actions
a8d0c1a62d
Automatic update operator docs from VictoriaMetrics/operator@4a51b37 (#6245) 2024-05-09 11:40:53 +08:00
hagen1778
56531abd56
app/vmselect/vmui: add missing static files
These files weren't added to the git after `make vmui-build vmui-update` command
in commit 7fd9325e62 (diff-50d9a4b91bdad190f2db92553736267103ab4225dfb6642b675fb4b8196e6560)

Related to https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6224

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-08 14:22:34 +02:00
Roman Khavronenko
8a03e987cb
lib/streamaggr: set correct suffix <output>_prometheus (#6228)
Set correct suffix `<output>_prometheus` for aggregation outputs
`increase_prometheus` and `total_prometheus`
Before, outputs `total` and `total_prometheus` or `increase` and
`increase_prometheus` had the same suffix.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-08 13:11:30 +02:00
Andrii Chubatiuk
a9283e06a3
streamaggr: made labels compressor shared (#6173)
Though labels compressor is quite resource intensive, each aggregator
and deduplicator instance has it's own compressor. Made it shared across
all aggregators to consume less resources while using multiple
aggregators.

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
2024-05-08 13:10:53 +02:00
Zhu Jiekun
02851d7800
chore: [deployment] upgrade from go 1.22.2 to 1.22.3 to include security fixes (#6238)
### Describe Your Changes

upgrade from go 1.22.2 to 1.22.3 to include security fixes. Also see:
- https://go.dev/doc/devel/release
-
https://github.com/golang/go/issues?q=milestone%3AGo1.22.3+label%3ACherryPickApproved

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Jiekun <jiekun.dev@gmail.com>
2024-05-08 10:02:22 +02:00
Zhu Jiekun
17e3d019d2
feature: [vmagent] Add service discovery support for Vultr (#6068)
### Describe Your Changes
related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6041

#### Added
- Added service discovery support for Vultr.

#### Docs
- `CHANGELOG.md`, `sd_configs.md`, `vmagent.md` are updated.

#### Note
- Useful links: 
- Vultr API:
https://www.vultr.com/api/#tag/instances/operation/list-instances
    - Vultr client SDK: https://github.com/vultr/govultr
- Prometheus SD:
https://github.com/prometheus/prometheus/tree/main/discovery/vultr

---
### Checklist

The following checks are mandatory:

- [X] I have read the [Contributing
Guidelines](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/CONTRIBUTING.md)
- [x] All commits are signed and include `Signed-off-by` line. Use `git
commit -s` to include `Signed-off-by` your commits. See this
[doc](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) about
how to sign your commits.
- [x] Tests are passing locally. Use `make test` to run all tests
locally.
- [x] Linting is passing locally. Use `make check-all` to run all
linters locally.

Further checks are optional for External Contributions:

- [X] Include a link to the GitHub issue in the commit message, if issue
exists.
- [x] Mention the change in the
[Changelog](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/CHANGELOG.md).
Explain what has changed and why. If there is a related issue or
documentation change - link them as well.

  Tips for writing a good changelog message::

* Write a human-readable changelog message that describes the problem
and solution.
* Include a link to the issue or pull request in your changelog message.
* Use specific language identifying the fix, such as an error message,
metric name, or flag name.
* Provide a link to the relevant documentation for any new features you
add or modify.

- [ ] After your pull request is merged, please add a message to the
issue with instructions for how to test the fix or try the feature you
added. Here is an
[example](https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4048#issuecomment-1546453726)
- [x] Do not close the original issue before the change is released.
Please note, in some cases Github can automatically close the issue once
PR is merged. Re-open the issue in such case.
- [x] If the change somehow affects public interfaces (a new flag was
added or updated, or some behavior has changed) - add the corresponding
change to documentation.

Signed-off-by: Jiekun <jiekun.dev@gmail.com>
2024-05-08 10:01:48 +02:00
Oleg
c6c5a5a186
Statsd protocol compatibility (#5053)
In this PR I added compatibility with [statsd
protocol](https://github.com/b/statsd_spec) with tags to be able to send
metrics directly from statsd clients to vmagent or directly to VM.
For example its compatible with
[statsd-instrument](https://github.com/Shopify/statsd-instrument) and
[dogstatsd-ruby](https://github.com/DataDog/dogstatsd-ruby) gems

Related issues: #5052, #206, #4600
2024-05-07 21:46:08 +02:00
Github Actions
55c7dafb35
Automatic update operator docs from VictoriaMetrics/operator@2789953 (#6237) 2024-05-07 21:34:57 +02:00
Alexander Marshalov
3d4988ecf6
fix typo in scrape config examples (#6234) 2024-05-07 16:52:44 +02:00
Github Actions
134dcaef33
Automatic update operator docs from VictoriaMetrics/operator@6271553 (#6233) 2024-05-07 16:51:39 +02:00
Ted Possible
5a3abfa041
Exemplar support (#5982)
This code adds Exemplars to VMagent and the promscrape parser adhering
to OpenMetrics Specifications. This will allow forwarding of exemplars
to Prometheus and other third party apps that support OpenMetrics specs.

---------

Signed-off-by: Ted Possible <ted_possible@cable.comcast.com>
2024-05-07 12:09:44 +02:00
hagen1778
2561a132ee
docs: mention influxListenAddr in URLs format doc
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-05-06 15:29:24 +02:00
Andrii Chubatiuk
879771808b
app/vmagent/remotewrite: do not cleanup timeseries which are used in multiple remote write contexts (#6206)
When at least one remote write has deduplication configured it cleans up
timeseries while they can be in use by another remote write without
deduplication

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6205
---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-05-06 12:09:51 +02:00
hagen1778
c0050beadc
docs: mention actual version in update nodes
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-30 18:38:04 +02:00
Yury Molodov
046a4a5ecf
vmui: fix issue preventing first query trace expansion (#6197)
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6186
2024-04-30 13:32:29 +02:00
Hui Wang
e3c226cf92
docs: update vmalert and vmagent docs (#6207)
* restore and actualize doc section explaining duplicated labels error
* rm misleading comment about post-aggregation in stream aggregation
2024-04-30 10:27:06 +02:00
Corporte Gadfly
8bca4d2de4
deployment: minor grammatical fixes in alert descriptions (#6199) 2024-04-30 10:24:31 +02:00
Roman Khavronenko
e2590b339d
app/vmauth: add test for LeastLoaded balance policy (#6144)
Check if least-loaded works correctly.
related to
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6136

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-30 10:22:17 +02:00
Zakhar Bessarab
329c3cbdf0
lib/mergeset: improve test coverage (#6118)
Add test to cover the code path with overflowing shards buffers and
triggering merge to partition.

This test covers the code path which leaded to
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5959

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-04-30 10:21:37 +02:00
Github Actions
e908effd22
Automatic update operator docs from VictoriaMetrics/operator@8f025b3 (#6200)
Co-authored-by: Hui Wang <haley@victoriametrics.com>
2024-04-30 16:01:09 +08:00
hagen1778
d386a68b59
dashboards: add new panel Concurrent selects to vmstorage row
The panel will show how many ongoing select queries are processed by vmstorage
and should help to identify resource bottlenecks. See panel description for more details.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-29 13:55:19 +02:00
hagen1778
9e18724036
deployment: update per-tenant-statistic dashboard to be compatible with Grafana 10
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-29 12:38:56 +02:00
hagen1778
5917ac003e
deployment: update backupmanager dashboard to be compatible with Grafana 10
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-29 12:35:25 +02:00
hagen1778
f0c4d372bd
deployment: update operator dashboard to be compatible with Grafana 10
- Use TimeSeries panel instead of deprecated Graph
- Update panel styles
- Fix version panel

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-29 12:32:19 +02:00
hagen1778
9256df17fa
deployment: bump Grafana version to 10.4.2
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-29 12:10:24 +02:00
hagen1778
8606b48ce5
dashboards: add Network Usage panel to Resource Usage row
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4478
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-29 11:54:17 +02:00
Dima Lazerka
564463259a
deployment/dashboards: properly show version for non-stable docker images (#6150)
re: .*-(?:tags|heads)-(.*)-(?:0|dirty)-.*

cases:
victoria-metrics-20240419-160209-heads-enterprise-single-node-0-g08f933ab0c
enterprise-single-node

victoria-metrics-20240201-133950-tags-v1.97.1-enterprise-0-g760a8733b
v1.97.1-enterprise

victoria-metrics-20240419-160209-heads-rotation-part-2-0-ge2367b6d1-dirty-848b54cd
rotation-part-2-0-ge2367b6d1

victoria-metrics-20240419-160209-heads-lts-1.93-enterprise-search-contention-0-g30ef4aad21-amd64
lts-1.93-enterprise-search-contention

victoria-metrics-20240425-150852-tags-v1.101.0-enterprise-0-g718138c64
v1.101.0-enterprise

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Dzmitry Lazerka <dlazerka@gmail.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-04-29 11:11:28 +02:00
Zakhar Bessarab
6b493582da
dashboards/victoria-metrics-single: allow selecting multiple instance values (#5870)
Allowing to select multiple instance IPs makes it much easier to view
metrics for longer periods of time in dynamic environments such as
Kubernetes. In k8s update will also cause IP to change making it harder
to use dashboard to check the status.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5869

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-04-29 10:34:36 +02:00
Roman Khavronenko
5e8c087d42
deployment: bump to 1.101.0 (#6194)
Step 17:
> Bump VictoriaMetrics version at deployment/docker/docker-compose.yml
and at deployment/docker/docker-compose-cluster.yml.

from https://docs.victoriametrics.com/release-guide/

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-26 13:18:51 +02:00
Github Actions
3a669b4b87
Automatic update operator docs from VictoriaMetrics/operator@bbf847c (#6189) 2024-04-26 10:14:40 +02:00
hagen1778
5334f0c2ce
docs/CHANGELOG.md: cut v1.101.0
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-25 16:25:09 +02:00
hagen1778
7fd9325e62
app/vmselect: run make vmui-update
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-25 15:51:03 +02:00
hagen1778
a62551f773
docs: add change for c0e4ccb7b5
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-25 15:37:37 +02:00
Hui Wang
dd0d2c77c8
app/vmselect: implement cmd-line flags -search.disableImplicitConversions and -search.logImplicitConversions (#6180)
address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4338
support disable or log [implicit
conversions](https://docs.victoriametrics.com/metricsql/#implicit-query-conversions)
for subquery with cmd-line flags `-search.disableImplicitConversion` and
`-search.logImplicitConversion`

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-04-25 12:54:42 +02:00
Yury Molodov
57b7d16259
vmui: improve error message for server response issues (#6177)
Updates error messages for better clarity and guidance on server
response issues.
2024-04-25 12:52:13 +02:00
Yury Molodov
6193fa3dcf
vmui: trigger auto-suggestion at any cursor position (#6155)
- Implemented auto-suggestion triggers for mid-string cursor positions
in vmui.
- Improved the suggestion list positioning to appear directly beneath
the active text editing area.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5864
2024-04-25 12:48:49 +02:00
hagen1778
6aaf1768f4
Revert "docs: removed code-style highlighting for commanad-line flags of VM components (#6147)"
This reverts commit 9bedbcfa2f.
2024-04-25 11:23:02 +02:00
Andrii Chubatiuk
07496d7d92
deployment: update makefile package-* targets (#6172)
Updated package targets in a same manner, how it's done for publish ones
in
7958f38864
2024-04-24 18:29:32 +02:00
hagen1778
2d9bbe1934
docs/changelog: mention downsampling fixes for ENT version of VM
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-24 17:03:21 +02:00
hagen1778
679844feaf
Revert "app/vmbackup: introduce new flag type URL (#6152)"
This reverts commit 029060af60.
2024-04-24 13:47:57 +02:00
Roman Khavronenko
029060af60
app/vmbackup: introduce new flag type URL (#6152)
The new flag type is supposed to be used for specifying URL values which
could contain sensitive information such as auth tokens in GET params or
HTTP basic authentication.

The URL flag also allows loading its value from files if `file://`
prefix is specified. As example, the new flag type was used in
app/vmbackup as it requires specifying `authKey` param for making the
snapshot.

See related issue
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5973

Thanks to @wasim-nihal for initial implementation
https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6060

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-04-24 10:57:54 +02:00