414 Commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	21f588e9cf	app/vmselect/promql: fix TestVmrangeBucketsToLE test after 8516670582 ... Remove 'AccountID=0, ProjectID=0, ' prefix for cluster version	2022-08-07 00:15:10 +03:00
Dmytro Kozlov	ab645b5fab	vmselect/promql: add tests for vmrangeBucketsToLE (#2907) ... * vmselect/promql: add tests for vmrangeBucketsToLE * vmselect/promql: cleanup * vmselect/promql: cleanup * vmselect/promql: fix panic tests want result * vmselect/promql: cleanup * vmselect/promql: update test name * vmselect/promql: fix linter error * vmselect/promql: refactor testcases * vmselect/promql: cleanup * vmselect/promql: remove unused reassign to workers, fix typo * wip * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-26 20:43:43 +03:00
Aliaksandr Valialkin	8a83c59956	app/vmselect/promql: reduce the diff for f148cffc8a ... This is a follow-up for c826f06366	2022-07-26 19:21:16 +03:00
Alan Liang	f97de473f4	vmselect: fix vmrangeBucketsToLE func may panic when ts value equal zero (#2902) ... Co-authored-by: alanwzliang <alanwzliang@tencent.com>	2022-07-25 10:49:48 +03:00
Roman Khavronenko	04c4f8bafd	vmselect: return correct error for second part of expression (#2893) ... Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-21 20:40:47 +03:00
Aliaksandr Valialkin	f992f96a88	app/vmselect/promql: execute q1 and q2 from q1 op q2 in parallel if labels pushdown cannot be applied ... This should improve query performance if VictoriaMetrics has enough resources for processing `q1` and `q2` in parallel. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2886	2022-07-19 14:29:41 +03:00
Aliaksandr Valialkin	d920b0afec	app/vmselect/promql: properly return q1 series from q1 ifnot q2 when q2 returns nothing	2022-07-18 14:24:23 +03:00
Aliaksandr Valialkin	f933d5f142	app/vmselect/promql/transform.go: reuse evalNumber() function for constructing timezone_offset() results	2022-07-18 14:24:22 +03:00
Aliaksandr Valialkin	919939ee9f	app/vmselect/promql: consistency update after 93fbd0c54b	2022-07-13 12:34:28 +03:00
Roman Khavronenko	214e01499c	promql: return step as scrapeInterval when it can't be calculated (#2865) ... The change allows to specify default value for `getScrapeInterval` function when actual interval can't be calculated. Before the change, function were returning `maxSilenceInterval` (5m) in such cases, which may be not correct for instant queries processing. The specific scenario where using `maxSilenceInterval` caused issues is the following: 1. Series becomes stale; 2. Client (in this case vmalert) continues to request series every 15s; 3. Database returns empty results as expected; 4. But at some specific moment of time database returns datapoints from `now()-5m`, because lookback window was extended to `maxSilenceInterval`. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2022-07-13 12:34:28 +03:00
Aliaksandr Valialkin	5867708b3d	app/vmselect/promql: validate function name before evaluating its arguments ... This avoids unneeded evaluation of args for unknown functions	2022-07-12 19:49:54 +03:00
Roman Khavronenko	78da074ad3	metricsql: properly evaluate timezone_offset over time interval (#2842) ... * metricsql: properly evaluate `timezone_offset` over time interval https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2771 Signed-off-by: hagen1778 <roman@victoriametrics.com> * Update docs/CHANGELOG.md Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-07-08 14:04:40 +03:00
Aliaksandr Valialkin	f4df43f7cc	app/vmselect/netstorage: remove unused auth.Token arg	2022-07-06 12:46:21 +03:00
Aliaksandr Valialkin	10c63f5853	app/vmselect: make fmt after f3ece83e67	2022-07-05 14:56:53 +03:00
Aliaksandr Valialkin	8516670582	app/vmselect/promql: properly calculate histogram_quantile over unexpected le buckets ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2819	2022-07-05 13:20:41 +03:00
Aliaksandr Valialkin	fa08220d27	app/vmselect/promql: properly handle partial counter resets in rate(), irate(), increase() and remove_resets() functions ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2787	2022-06-30 22:40:26 +03:00
Aliaksandr Valialkin	4fb0f15322	all: readability improvements for query traces ... - show dates in human-readable format, e.g. 2022-05-07, instead of a numeric value - limit the maximum length of queries and filters shown in trace messages	2022-06-30 18:19:43 +03:00
Aliaksandr Valialkin	daefb64f38	app/vmselect: expose additional histograms at /metrics page, which may help get more insights for the query workload ... This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2792	2022-06-28 20:18:31 +03:00
Aliaksandr Valialkin	9adff0b686	app/vmselect/promql: show the number of scanned samples in the query trace	2022-06-28 19:32:03 +03:00
Aliaksandr Valialkin	399d4c36ae	app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day	2022-06-28 12:55:20 +03:00
Aliaksandr Valialkin	6386f117c8	all: show timeRange in traces in human-readable format instead of timestamps in milliseconds	2022-06-27 13:42:57 +03:00
Aliaksandr Valialkin	fe2269b999	all: remove explicit "xxhash" name when importing github.com/cespare/xxhash/v2 package ... This package already has the same name, so there is no need in explicit name	2022-06-21 20:24:28 +03:00
Aliaksandr Valialkin	a9ea3fee38	lib/querytracer: make it easier to use by passing trace context message to New and NewChild ... The context message can be extended by calling Donef. If there is no need to extend the message, then just call Done.	2022-06-08 21:16:12 +03:00
Aliaksandr Valialkin	afced37c0b	all: add initial support for query tracing ... See https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#query-tracing Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1403	2022-06-01 02:31:44 +03:00
Aliaksandr Valialkin	68bfa05437	app/vmselect/promql: do not return values from label_value() if the original time series has no values at the selected timestamps	2022-05-09 17:58:39 +03:00
Aliaksandr Valialkin	6eb1580158	app/vmselect/promql: add tlast_change_over_time(m[d]) function, which returns the timestamp for the last change of m on the given lookbehind window d	2022-04-27 10:58:40 +03:00
Aliaksandr Valialkin	dac24aa342	app/vmselect/promql: properly handle scalar default vector, scalar if vector and scalar ifnot vector queries ... Previously `vector` time series could be unexpectedly returned from such queries	2022-04-21 15:34:14 +03:00
Aliaksandr Valialkin	bc54ae9608	app/vmselect/promql: fix comparison to nan ... The comparison to nan has been broken in d335cc886c Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150	2022-04-21 14:56:41 +03:00
Aliaksandr Valialkin	a5cfe5d13e	app/vmselect/promql: add drop_common_labels() function	2022-04-21 14:20:36 +03:00
Aliaksandr Valialkin	7ca08945fd	app/vmselect/promql: fix q default b where b may have empty time series	2022-04-21 11:43:09 +03:00
Aliaksandr Valialkin	25b841c6ed	app/vmselect/promql: fix duplicate time series error on joins against time series filtered by values ... This should prevent from `duplicate time series` errors when executing the following query: kube_pod_container_resource_requests{resource="cpu"} * on (namespace,pod) group_left() (kube_pod_status_phase{phase=~"Pending|Running"}==1) where `kube_pod_status_phase{phase=~"Pending|Running"}==1` filters out diplicate time series	2022-04-20 22:21:20 +03:00
Aliaksandr Valialkin	0ef7a05fc0	app/vmselect/promql: rename removeNaNs() to more clear removeEmptySeries()	2022-04-20 19:53:24 +03:00
Aliaksandr Valialkin	3605e1743b	app/vmselect/promql: allow calling InitRollupResultCache+StopRollupResultCache multiple times during tests	2022-04-11 12:32:23 +03:00
Aliaksandr Valialkin	b843f0e229	app/vmselect: add fine-grained limits for the number of returned/scanned time series for various APIs	2022-03-26 11:28:14 +02:00
Aliaksandr Valialkin	99ee1c13c0	app/vmselect/promql: reduce the maximum number of label values, which can be propagated from one side of the binary operation to another side of the binary operation from 10K to 1K ... There are user reports that 10K unique values in a single label filter may lead to performance and memory usage issues	2022-02-24 04:03:28 +02:00
Aliaksandr Valialkin	244c23ea2c	lib/workingsetcache: reduce the default cache rotation period from hour to 20 minutes ... This should reduce memory usage under high time series churn rate	2022-02-23 13:42:27 +02:00
Aliaksandr Valialkin	989668beba	app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130	2022-02-12 15:46:43 +02:00
Aliaksandr Valialkin	1173964d8d	app/vmselect/promql: clarify comments on why the right side of if and and operators are executed at first	2022-02-03 00:26:42 +02:00
Aliaksandr Valialkin	10367d7e4c	app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values ... Such filters may slow down time series search, so just skip them. This is a follow-up for e7f1ceeb84 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827	2022-02-02 23:42:25 +02:00
Aliaksandr Valialkin	13c2692ca1	app/vmselect/promql: properly handle foo or bar queries ... Such queries may miss `bar` results after the commit e7f1ceeb84 because common label filters from `foo` could be mistakenly applied to `bar`.	2022-02-01 17:43:10 +02:00
Aliaksandr Valialkin	02b2bfcff3	lib/bytesutil: split Resize* funcs to MayOverallocate and NoOverallocate for more fine-grained control over memory allocations ... Follow-up for f4989edd96 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007	2022-02-01 11:20:20 +02:00
Aliaksandr Valialkin	784cdcb080	app/vmselect/promql: check for binary operation in case-insensitive manner when deciding which side of the operation to perform the first ... PromQL and MetricsQL operators are case-insensitive	2022-01-31 22:07:35 +02:00
Aliaksandr Valialkin	04d6596298	app/vmselect/promql: optimize queries, which join on _info metrics. ... Automatically add common filters from one side of binary operation to the other side before sending the query to storage subsystem. See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/ and https://www.robustperception.io/exposing-the-software-version-to-prometheus	2022-01-31 20:25:15 +02:00
Aliaksandr Valialkin	6232eaa938	lib/bytesutil: split Resize() into ResizeNoCopy() and ResizeWithCopy() functions ... Previously bytesutil.Resize() was copying the original byte slice contents to a newly allocated slice. This wasted CPU cycles and memory bandwidth in some places, where the original slice contents wasn't needed after slize resizing. Switch such places to bytesutil.ResizeNoCopy(). Rename the original bytesutil.Resize() function to bytesutil.ResizeWithCopy() for the sake of improved readability. Additionally, allocate new slice with `make()` instead of `append()`. This guarantees that the capacity of the allocated slice exactly matches the requested size. The `append()` could return a slice with bigger capacity as an optimization for further `append()` calls. This could result in excess memory usage when the returned byte slice was cached (for instance, in lib/blockcache). Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007	2022-01-25 15:28:42 +02:00
Aliaksandr Valialkin	cc76174387	app/vmselect/promql: properly keep metric names when optimized path is used for aggregate function calculations ... For example, `sum(rate(...) keep_metric_names) by (__name__)` didn't leave the original metric name because of this issue. This is a follup-up for 1bdc71d917 Udates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949	2022-01-17 15:33:39 +02:00
Aliaksandr Valialkin	5a1e4a140f	app/vmselect/promql: implement keep_metric_names modifier for transform and rollup functions ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949	2022-01-14 04:15:35 +02:00
Aliaksandr Valialkin	53c9a81f2b	app/vmselect/promql: add stale_samples_over_time() function	2022-01-14 01:49:24 +02:00
Aliaksandr Valialkin	0580a58feb	app/vmselect/promql: add support for @ modifier ... Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier Extend the support with the following features: * Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()` * Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)` returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348	2022-01-13 22:15:19 +02:00
Aliaksandr Valialkin	7d635a2311	app/vmselect/promql: fix limit_offset() test ... The test has been broken in addae7fc6a	2022-01-13 16:58:47 +02:00
Aliaksandr Valialkin	3396f08874	app/vmselect/promql: follow-up after 177e345d8a ... * Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions. * Simplify their implementation * Mention these functions in docs/CHANGELOG.md	2021-12-20 13:15:06 +02:00