Commit Graph

580 Commits

Author SHA1 Message Date
Nikolay
bb399518db
app/vmselect: properly return binary pow function result (#7619)
Previously, for `^` aka pow function calls, VictoriaMetrics returned `1`
if left arg was Nan. For example, given query=`(hour()==2)^1` returns 1
for NaN produced by hour() == 2 function. It added additional non-exist
datapoints to the timeseries.

This commit port bugfix from `metricql` package and adds test for it.
Now, VictoriaMetrics
correctly returns `NaN` for such cases.

Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7359

Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-11-21 15:16:28 +01:00
Artem Fetishev
7c60d82215
tests: integration tests for vmsingle (#7434)
### Describe Your Changes

This PR continues the implementation of integration tests (#7199). It
adds the support for vm-single:

-    A vmsingle app wrapper has been added
- Sample vmsingle tests that test the VM documentation related to
querying data (#7435)
- The tests use the go-cmp/{cmp,/cmpopts} packages, therefore they have
been added to ./vendor
-    Minor refactoring: data objects have been moved to model.go

Advice on porting things to cluster branch:

- The build rule must include tests that start with TestVmsingle
(similarly to how TestCluster tests are skipped in master branch)
- The build rule must depend on `vmstorage vminsert vmselect` instead of
`victoria-metrics`
- The query_test.go can actually be implemented for cluster as well. To
do this the tests need to be renamed to start with TestCluster and the
tests must instantiace vm{storage,insert,select} instead of vmsingle.

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

---------

Signed-off-by: Artem Fetishev <rtm@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-11-07 12:58:37 +01:00
Andrii Chubatiuk
7e60afb6fc
app/vlinsert: adds journald ingestion support
This commit allows to ingest logs with journald format. 

https://www.freedesktop.org/software/systemd/man/latest/systemd-journal-remote.service.html

related issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4618
2024-10-27 20:36:33 +01:00
Aliaksandr Valialkin
630211cfed
app/vlogscli: add interactive command-line tool for querying VictoriaLogs 2024-10-01 12:23:07 +02:00
Aliaksandr Valialkin
f65b976eda
vendor: run make vendor-update 2024-09-26 22:33:05 +02:00
Aliaksandr Valialkin
b82e2cabc5
app/vmselect/promql: properly calculate c1 and c2 and c1 or c2 by upgrading github.com/VictoriaMetrics/metricsql to v0.79.0
The fix is in the https://github.com/VictoriaMetrics/metricsql/pull/34
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6637
2024-09-18 17:38:19 +02:00
Aliaksandr Valialkin
c6faab77f9
vendor: run make vendor-update 2024-09-08 21:05:06 +02:00
Aliaksandr Valialkin
5261a84119
deployment: update Go builder from Go1.23.0 to Go1.23.1
See https://github.com/golang/go/issues?q=milestone%3AGo1.23.1+label%3ACherryPickApproved
2024-09-06 22:51:15 +02:00
Roman Khavronenko
1b9f3b39b4
deployment/docker: update Go builder from Go1.22.5 to Go1.23.0 (#6861)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-08-22 23:55:50 +02:00
Nikolay
9feee15493
lib/promscrape: fixes proxy autorization (#6783)
* Adds custom dial func for HTTP-Connect and socks5 proxy tunnels.
  Standard golang http.transport exposes GetProxyConnectHeader function,
  but it doesn't allow to use separate tls config for proxy.
  It also not possible to enforce HTTP-Connect with standard http lib.
* For http scrape targets, by default http.Transport.Proxy function must
  be used. Since it has special case with full uri forward.
* Adds proxy.URL json methods that allow to properly copy internal
fields, like User/Password.
It should fix bug with proxy_url. When credentials specified at URL was
ignored.
* Adds tests for scrape client proxy requests

related issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6771
2024-08-19 22:31:18 +02:00
f41gh7
7b21471f78
vendor: updates metricsql to v0.77.0 with bugfix
Fixes panic if incorrect metricsql expression passed to the prettifier API.

Prettify function had misleading panic for duration expression formatting. It expected all WITH templates to be already parsed.
But WITH expression expand was removed.

Bug was introduced at e712a49898
and present at v1.98.0+ releases

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6736
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-08-01 10:50:05 +02:00
Aliaksandr Valialkin
1e3c7f0ef5
vendor: run make vendor-update 2024-07-27 13:52:48 +02:00
Aliaksandr Valialkin
66a34acc3a
vendor: run make vendor-update 2024-07-17 20:47:20 +02:00
Aliaksandr Valialkin
f5a89aea1e
vendor: update github.com/VictoriaMetrics/metrics from v1.35.0 to v1.35.1 2024-07-16 13:18:38 +02:00
Aliaksandr Valialkin
202e5704e6
vendor: update github.com/VictoriaMetrics/metrics from v1.34.1 to v1.35.0
Fix potential memory leaks across VictoriaMetrics codebase after metrics.UnregisterSet(s) call
because of missing s.UnregisterAllMetrics() call.

This is a follow-up for 6a6e34ab8e . It is OK if some vmauth metrics
aren't visible for a few microseconds when the previous metrics are unregistered and new metrics
weren't registered yet.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6247
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4690
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6252
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5805
2024-07-15 10:43:37 +02:00
Nikolay
14ac8a09ea
make vendor-update (#6627)
make vendor-update
2024-07-10 17:14:51 +02:00
justinrush
5fd3aef549
lib/backup: add support for Azure Managed Identity (#6518)
### Describe Your Changes

These changes support using Azure Managed Identity for the `vmbackup`
utility. It adds two new environment variables:

* `AZURE_USE_DEFAULT_CREDENTIAL`: Instructs the `vmbackup` utility to
build a connection using the [Azure Default
Credential](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity@v1.5.2#NewDefaultAzureCredential)
mode. This causes the Azure SDK to check for a variety of environment
variables to try and make a connection. By default, it tries to use
managed identity if that is set up.

This will close
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5984

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

### Testing

However you normally test the `vmbackup` utility using Azure Blob should
continue to work without any changes. The set up for that is environment
specific and not listed out here.

Once regression testing has been done you can set up [Azure Managed
Identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview)
so your resource (AKS, VM, etc), can use that credential method. Once it
is set up, update your environment variables according to the updated
documentation.

I added unit tests to the `FS.Init` function, then made my changes, then
updated the unit tests to capture the new branches.

I tested this in our environment, but with SAS token auth and managed
identity and it works as expected.

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Justin Rush <jarush@epic.com>
Co-authored-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
2024-07-10 11:52:05 +02:00
Aliaksandr Valialkin
f928298f03
vendor: update github.com/valyala/quicktemplate from v1.7.0 to v1.8.0 2024-07-05 01:22:23 +02:00
Aliaksandr Valialkin
f5518b2adc
deployment/docker: update Go builder from Go1.22.4 to Go1.22.5
See https://github.com/golang/go/issues?q=milestone%3AGo1.22.5+label%3ACherryPickApproved
2024-07-03 00:07:09 +02:00
Aliaksandr Valialkin
f15ba5eac3
vendor: run make vendor-update 2024-07-03 00:00:09 +02:00
Aliaksandr Valialkin
5c55722db4
vendor: run make vendor-update 2024-06-24 23:08:47 +02:00
Nikolay
1af13208c2
vendor update: updates metrics to v1.34.0 (#6523)
mention change for
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6457

### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-06-24 14:51:34 +02:00
Aliaksandr Valialkin
b12864f905
vendor: run make vendor-update 2024-06-07 16:40:42 +02:00
Aliaksandr Valialkin
43cf221681
lib/logstorage: work-in-progress 2024-06-05 03:18:12 +02:00
Aliaksandr Valialkin
ce1e0610b0
vendor: run make vendor-update 2024-05-22 21:58:38 +02:00
Aliaksandr Valialkin
51de9f30fc
vendor: run make vendor-update 2024-05-12 16:17:38 +02:00
Aliaksandr Valialkin
75398cd0f8
go.mod: update the required Go version from 1.21 to 1.22
This is a follow-up for 95222b2079
2024-05-12 16:07:11 +02:00
Aliaksandr Valialkin
184e4ef4d7
vendor: run make vendor-update 2024-04-17 22:54:56 +02:00
Aliaksandr Valialkin
37b04bbb81
vendor: run make vendor-update 2024-04-17 20:59:48 +02:00
Aliaksandr Valialkin
7c0003d8a4
vendor: run make vendor-update 2024-04-11 09:46:22 +02:00
Aliaksandr Valialkin
b212c9d6f5
vendor: run make vendor-update 2024-04-04 01:34:44 +03:00
Aliaksandr Valialkin
4cb70ee9a3
vendor: update github.com/VictoriaMetrics/metrics and github.com/VictoriaMetrics/metricsql to newer versions
This is needed for updating broken links to MetricsQL docs:

https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/MetricsQL -> https://docs.victoriametrics.com/metricsql/

This is a follow-up for 7e3511ffbd
2024-03-30 04:44:19 +02:00
Aliaksandr Valialkin
f95e9f13ae
vendor: run make vendor-update 2024-03-18 00:51:41 +02:00
Aliaksandr Valialkin
d8688c9e82
vendor: run make vendor-update 2024-03-06 21:24:42 +02:00
Aliaksandr Valialkin
a304372d88
vendor: run make vendor-update 2024-03-01 00:55:47 +02:00
Aliaksandr Valialkin
62498a1e68
deployment/docker: downgrade Go builder from 1.22.0 to 1.21.7
Go1.22.0 contains the bug https://github.com/golang/go/issues/65705 ,
which prevents vmagent from normal operation.
2024-02-29 13:52:26 +02:00
Aliaksandr Valialkin
aae71832e5
vendor: retain v0.46.0 for github.com/prometheus/common , since v0.48.0 leads to build error
The error is:

vendor/github.com/prometheus/client_golang/prometheus/testutil/promlint/promlint.go:71:38: undefined: expfmt.FmtText
2024-02-25 02:20:24 +02:00
Aliaksandr Valialkin
35f592a02c
app/vmselect/promql: properly handle args in count_values_over_time() function
Prevsiously they were swapped - the first arg should be the label name and the second arg should be label filters

This is a follow-up for e389b7b959e8144fdff5075bf7a5a39b2b0c6dd3

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5847
2024-02-25 01:48:18 +02:00
Aliaksandr Valialkin
4bdb3d9fd9
vendor: run make vendor-update 2024-02-24 03:22:39 +02:00
Aliaksandr Valialkin
7e1dd8ab9d
lib: consistently use atomic.* types instead of atomic.* functions
See ea9e2b19a5
2024-02-24 02:07:53 +02:00
Aliaksandr Valialkin
f7c3dee1c3
app/vmselect/promql: add count_values_over_time() MetricsQL function
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5847
2024-02-22 13:39:29 +02:00
Aliaksandr Valialkin
2e30842582
vendor: update github.com/VictoriaMetrics/metricsql from v0.72.1 to v0.73.0
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5383
2024-02-18 18:41:41 +02:00
Aliaksandr Valialkin
6a6ea89da5
vendor: update github.com/VictoriaMetrics/metrics from v1.31.0 to v1.32.0 2024-02-18 12:42:22 +02:00
Aliaksandr Valialkin
9c9021fd8b
vendor: run make vendor-update 2024-02-14 15:44:03 +02:00
Aliaksandr Valialkin
d6e22f2888
app/vmselect: add sum_eq_over_time, sum_gt_over_time and sum_le_over_time functions to MetricsQL
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4641
2024-02-13 23:40:07 +02:00
Aliaksandr Valialkin
2f3091460f
vendor: update github.com/VictoriaMetrics/metricsql from v0.70.1 to v0.71.0
This adds an ability to propagate label filters across label_set() and alias() functions.

This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827#issuecomment-1654095358
2024-02-13 06:36:59 +02:00
Aliaksandr Valialkin
c5f9d9f0d6
vendor: run make vendor-update 2024-02-12 22:31:30 +02:00
Aliaksandr Valialkin
95222b2079
all: upgrade Go builder from Go1.21.7 to Go1.22.0
See https://go.dev/doc/go1.22
2024-02-12 21:59:51 +02:00
Aliaksandr Valialkin
541b644d3d
app/{vmagent,vminsert}: follow-up after a1d1ccd6f2
- Document the change at docs/CHANGELOG.md
- Copy changes from docs/Single-server-VictoriaMetrics.md to README.md
- Add missing handler for processing multitenant requests ( https://docs.victoriametrics.com/vmagent/#multitenancy )
- Substitute github.com/stretchr/testify dependency with 3 lines of code in the added tests
- Comment unclear code at lib/protoparser/datadogsketches/parser.go , so @AndrewChubatiuk could update it
  and add permalinks to the original source code there.
- Various code cleanups

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5584
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3091
2024-02-07 01:28:05 +02:00
Andrii Chubatiuk
a1d1ccd6f2
support datadog /api/beta/sketches API (#5584)
Co-authored-by: Andrew Chubatiuk <andrew.chubatiuk@motional.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-02-06 20:58:11 +00:00