Dmytro Kozlov
96ecec877d
vmselect/{promql, prometheus}: show flag names which user can update in error message ( #3049 )
...
* vmselect/{promql, prometheus}: show flag names which user can update in error message
* vmselect/{promql, prometheus}: fix typo
2022-09-06 14:48:20 +03:00
Aliaksandr Valialkin
c6b74148cf
app/vmselect/promql: consistently calculate rate_over_sum(m[d]) as sum_over_time(m[d])/d
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3045
2022-09-02 23:19:05 +03:00
Aliaksandr Valialkin
024e2f18da
app/vmselect/promql: evaluate union() args in parallel in order to increase query performance
...
Note that the parallel execution of `union()` args may take more memory and CPU time
than the sequential execution if args contain heavy queries, which may load all the available CPU,
disk and memory resources and vmselect and vmstorage levels.
2022-09-02 21:01:04 +03:00
Aliaksandr Valialkin
8aaaf221cc
app/vmselect/promql: follow-up after 2d71b4859c
...
- Use getScalar() function for obtaining the expected scalar from phi arg
- Reduce the error message returned to the user when incorrect phi is passed to histogram_quantiles
- Improve the description of this bugfix in the docs/CHANGELOG.md
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3026
2022-08-27 01:38:17 +03:00
Dmytro Kozlov
2cf29cc4d1
vmselect/promql: fix panic in histogram_quantiles function ( #3029 )
...
* vmselect/promql: fix panic in histogram_quantiles function
* Update docs/MetricsQL.md
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
2022-08-27 01:34:54 +03:00
Dmytro Kozlov
d32a6359b0
vmselect/promql: enable search.maxPointsSubqueryPerTimeseries for sub-queries ( #2963 )
...
* vmselect/promql: enable search.maxPointsPerTimeSeriesSubquery for sub-queries
* vmselect/promql: cleanup
* vmselect/promql: rename config flag
* vmselect/promql: add tests
* vmselect/promql: use test object instead of log
* vmselect/promql: fix posible panic is subquery has more points. add description
* vmselect/promql: update tests descriptions
* vmselect/promql: update doInternal validation
* vmselect/promql: fix linter
* vmselect/promql: fix linter
* vmselect/promql: update documentation and release notes
* wip
- Properly apply -search.maxPointsSubqueryPerTimeseries limit to subqueries.
Previously the -search.maxPointsPerTimeseries limit was unexpectedly applied to subqueries
if it was smaller than the -search.maxPointsSubqueryPerTimeseries .
- Clarify docs for -search.maxPointsSubqueryPerTimeseries command-line flag .
- Document -search.maxPointsPerTimeseries and -search.maxPointsSubqueryPerTimeseries flags at https://docs.victoriametrics.com/#resource-usage-limits .
- Update docs/CHANGELOG.md .
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2922
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-24 15:27:41 +03:00
Aliaksandr Valialkin
06f6de6d47
all: use os.{Read|Write}File instead of ioutil.{Read|Write}File
...
The ioutil.{Read|Write}File is deprecated since Go1.16 -
see https://tip.golang.org/doc/go1.16#ioutil
VictoriaMetrics needs at least Go1.18, so it is safe to remove ioutil usage
from source code.
This is a follow-up for 02ca2342ab
2022-08-21 23:55:20 +03:00
Aliaksandr Valialkin
7d7cf2b6fd
app/vmselect: follow-up after 63e0f16062
...
* Explicitly store a pointer to UserReadableError in the error interface.
Previously Go automatically converted the value to a pointer before storing in the error interface.
* Add Unwrap() method to UserReadableError, so it can be used transparently with the other code,
which calls errors.Is() and errors.As().
* Document the change in docs/CHANGELOG.md
2022-08-15 13:53:19 +03:00
Roman Khavronenko
8a26ec435d
vmselect: introduce UserReadableError type of error ( #2894 )
...
When read query fails, VM returns rich error message with
all the details. While these details might be useful
for debugging specific cases, they're usually too verbose
for users.
Introducing a new error type `UserReadableError` is supposed
to allow to return to user only the most important parts
of the error trace. This supposed to improve error readability
in web interfaces such as VMUI or Grafana.
The full error trace is still logged with the full context
and can be found in vmselect logs.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-08-15 13:53:18 +03:00
Aliaksandr Valialkin
21f588e9cf
app/vmselect/promql: fix TestVmrangeBucketsToLE test after 8516670582
...
Remove 'AccountID=0, ProjectID=0, ' prefix for cluster version
2022-08-07 00:15:10 +03:00
Dmytro Kozlov
ab645b5fab
vmselect/promql: add tests for vmrangeBucketsToLE ( #2907 )
...
* vmselect/promql: add tests for vmrangeBucketsToLE
* vmselect/promql: cleanup
* vmselect/promql: cleanup
* vmselect/promql: fix panic tests want result
* vmselect/promql: cleanup
* vmselect/promql: update test name
* vmselect/promql: fix linter error
* vmselect/promql: refactor testcases
* vmselect/promql: cleanup
* vmselect/promql: remove unused reassign to workers, fix typo
* wip
* wip
* wip
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-26 20:43:43 +03:00
Aliaksandr Valialkin
8a83c59956
app/vmselect/promql: reduce the diff for f148cffc8a
...
This is a follow-up for c826f06366
2022-07-26 19:21:16 +03:00
Alan Liang
f97de473f4
vmselect: fix vmrangeBucketsToLE func may panic when ts value equal zero ( #2902 )
...
Co-authored-by: alanwzliang <alanwzliang@tencent.com>
2022-07-25 10:49:48 +03:00
Roman Khavronenko
04c4f8bafd
vmselect: return correct error for second part of expression ( #2893 )
...
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-21 20:40:47 +03:00
Aliaksandr Valialkin
f992f96a88
app/vmselect/promql: execute q1 and q2 from q1 op q2 in parallel if labels pushdown cannot be applied
...
This should improve query performance if VictoriaMetrics has enough resources for processing `q1` and `q2` in parallel.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2886
2022-07-19 14:29:41 +03:00
Aliaksandr Valialkin
d920b0afec
app/vmselect/promql: properly return q1 series from q1 ifnot q2 when q2 returns nothing
2022-07-18 14:24:23 +03:00
Aliaksandr Valialkin
f933d5f142
app/vmselect/promql/transform.go: reuse evalNumber() function for constructing timezone_offset() results
2022-07-18 14:24:22 +03:00
Aliaksandr Valialkin
919939ee9f
app/vmselect/promql: consistency update after 93fbd0c54b
2022-07-13 12:34:28 +03:00
Roman Khavronenko
214e01499c
promql: return step as scrapeInterval when it can't be calculated ( #2865 )
...
The change allows to specify default value for `getScrapeInterval`
function when actual interval can't be calculated.
Before the change, function were returning `maxSilenceInterval` (5m)
in such cases, which may be not correct for instant queries processing.
The specific scenario where using `maxSilenceInterval` caused issues
is the following:
1. Series becomes stale;
2. Client (in this case vmalert) continues to request series every 15s;
3. Database returns empty results as expected;
4. But at some specific moment of time database returns datapoints from `now()-5m`,
because lookback window was extended to `maxSilenceInterval`.
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-07-13 12:34:28 +03:00
Aliaksandr Valialkin
5867708b3d
app/vmselect/promql: validate function name before evaluating its arguments
...
This avoids unneeded evaluation of args for unknown functions
2022-07-12 19:49:54 +03:00
Roman Khavronenko
78da074ad3
metricsql: properly evaluate timezone_offset over time interval ( #2842 )
...
* metricsql: properly evaluate `timezone_offset` over time interval
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771
Signed-off-by: hagen1778 <roman@victoriametrics.com>
* Update docs/CHANGELOG.md
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-07-08 14:04:40 +03:00
Aliaksandr Valialkin
f4df43f7cc
app/vmselect/netstorage: remove unused auth.Token arg
2022-07-06 12:46:21 +03:00
Aliaksandr Valialkin
10c63f5853
app/vmselect: make fmt after f3ece83e67
2022-07-05 14:56:53 +03:00
Aliaksandr Valialkin
8516670582
app/vmselect/promql: properly calculate histogram_quantile over unexpected le buckets
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819
2022-07-05 13:20:41 +03:00
Aliaksandr Valialkin
fa08220d27
app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2787
2022-06-30 22:40:26 +03:00
Aliaksandr Valialkin
4fb0f15322
all: readability improvements for query traces
...
- show dates in human-readable format, e.g. 2022-05-07, instead of a numeric value
- limit the maximum length of queries and filters shown in trace messages
2022-06-30 18:19:43 +03:00
Aliaksandr Valialkin
daefb64f38
app/vmselect: expose additional histograms at /metrics page, which may help get more insights for the query workload
...
This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792
2022-06-28 20:18:31 +03:00
Aliaksandr Valialkin
9adff0b686
app/vmselect/promql: show the number of scanned samples in the query trace
2022-06-28 19:32:03 +03:00
Aliaksandr Valialkin
399d4c36ae
app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day
2022-06-28 12:55:20 +03:00
Aliaksandr Valialkin
6386f117c8
all: show timeRange in traces in human-readable format instead of timestamps in milliseconds
2022-06-27 13:42:57 +03:00
Aliaksandr Valialkin
fe2269b999
all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package
...
This package already has the same name, so there is no need in explicit name
2022-06-21 20:24:28 +03:00
Aliaksandr Valialkin
a9ea3fee38
lib/querytracer: make it easier to use by passing trace context message to New and NewChild
...
The context message can be extended by calling Donef.
If there is no need to extend the message, then just call Done.
2022-06-08 21:16:12 +03:00
Aliaksandr Valialkin
afced37c0b
all: add initial support for query tracing
...
See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403
2022-06-01 02:31:44 +03:00
Aliaksandr Valialkin
68bfa05437
app/vmselect/promql: do not return values from label_value() if the original time series has no values at the selected timestamps
2022-05-09 17:58:39 +03:00
Aliaksandr Valialkin
6eb1580158
app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d
2022-04-27 10:58:40 +03:00
Aliaksandr Valialkin
dac24aa342
app/vmselect/promql: properly handle scalar default vector, scalar if vector and scalar ifnot vector queries
...
Previously `vector` time series could be unexpectedly returned from such queries
2022-04-21 15:34:14 +03:00
Aliaksandr Valialkin
bc54ae9608
app/vmselect/promql: fix comparison to nan
...
The comparison to nan has been broken in d335cc886chttps://github.com/VictoriaMetrics/VictoriaMetrics/issues/150
2022-04-21 14:56:41 +03:00
Aliaksandr Valialkin
a5cfe5d13e
app/vmselect/promql: add drop_common_labels() function
2022-04-21 14:20:36 +03:00
Aliaksandr Valialkin
7ca08945fd
app/vmselect/promql: fix q default b where b may have empty time series
2022-04-21 11:43:09 +03:00
Aliaksandr Valialkin
25b841c6ed
app/vmselect/promql: fix duplicate time series error on joins against time series filtered by values
...
This should prevent from `duplicate time series` errors when executing the following query:
kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1)
where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series
2022-04-20 22:21:20 +03:00
Aliaksandr Valialkin
0ef7a05fc0
app/vmselect/promql: rename removeNaNs() to more clear removeEmptySeries()
2022-04-20 19:53:24 +03:00
Aliaksandr Valialkin
3605e1743b
app/vmselect/promql: allow calling InitRollupResultCache+StopRollupResultCache multiple times during tests
2022-04-11 12:32:23 +03:00
Aliaksandr Valialkin
b843f0e229
app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs
2022-03-26 11:28:14 +02:00
Aliaksandr Valialkin
99ee1c13c0
app/vmselect/promql: reduce the maximum number of label values, which can be propagated from one side of the binary operation to another side of the binary operation from 10K to 1K
...
There are user reports that 10K unique values in a single label filter may lead to performance and memory usage issues
2022-02-24 04:03:28 +02:00
Aliaksandr Valialkin
244c23ea2c
lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes
...
This should reduce memory usage under high time series churn rate
2022-02-23 13:42:27 +02:00
Aliaksandr Valialkin
989668beba
app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130
2022-02-12 15:46:43 +02:00
Aliaksandr Valialkin
1173964d8d
app/vmselect/promql: clarify comments on why the right side of if and and operators are executed at first
2022-02-03 00:26:42 +02:00
Aliaksandr Valialkin
10367d7e4c
app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values
...
Such filters may slow down time series search, so just skip them.
This is a follow-up for e7f1ceeb84https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827
2022-02-02 23:42:25 +02:00
Aliaksandr Valialkin
13c2692ca1
app/vmselect/promql: properly handle foo or bar queries
...
Such queries may miss `bar` results after the commit e7f1ceeb84
2022-02-01 17:43:10 +02:00
Aliaksandr Valialkin
02b2bfcff3
lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations
...
Follow-up for f4989edd96https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007
2022-02-01 11:20:20 +02:00
