Commit Graph

213 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
262cf81757 app/vmselect: properly parse negative combined offsets such as -1h2m3s
Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.
2020-12-19 01:25:03 +02:00
Aliaksandr Valialkin
d7be1a0124 Upgrade github.com/valyala/gozstd from v1.8.3 to v1.9.0 2020-12-17 15:13:35 +02:00
Aliaksandr Valialkin
5a846f3e52 vendor: make vendor-update 2020-12-11 23:32:50 +02:00
Aliaksandr Valialkin
8dab061f51 vendor: make vendor-update 2020-12-05 12:45:41 +02:00
Aliaksandr Valialkin
d242c2f2bd app/vmselect/promql: add count_eq_over_time(m[d], N) and count_ne_over_time(m[d], N) for calculating the number of samples in m over d that are equal / not equal to N 2020-12-05 12:31:01 +02:00
Aliaksandr Valialkin
8cf76d8747 app/vmselect/promql: add label_uppercase(q, label1, ... labelN) and label_lowercase(q, label1, ... labelN) functions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/936
2020-12-03 21:46:18 +02:00
Aliaksandr Valialkin
732d354b90 vendor: make vendor-update 2020-12-03 20:21:54 +02:00
Aliaksandr Valialkin
a8c5e2f0c5 vendor: upgrade github.com/VictoriaMetrics/metricsql from v0.7.2 to v0.7.3
This fixes parsing of hex numbers in MetricsQL such as 0x3b

The bug has been detected at https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/
2020-12-02 08:11:42 +02:00
Aliaksandr Valialkin
3de102bcf1 vendor: update github.com/VictoriaMetrics/fasthttp from v1.0.8 to v1.0.9 2020-11-26 13:27:23 +02:00
Aliaksandr Valialkin
e6447e7588 vendor: update github.com/VictoriaMetrics/fasthttp from v1.0.7 to v1.0.8 2020-11-26 12:13:09 +02:00
Aliaksandr Valialkin
26e699c440 lib/protoparser/prometheus: properly parse "infinity" values in OpenMetrics format
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/924
2020-11-24 19:02:50 +02:00
Aliaksandr Valialkin
59aa57efbc vendor: make vendor-update 2020-11-19 19:21:23 +02:00
Aliaksandr Valialkin
d29997dab6 vendor: make vendor-update 2020-11-16 20:52:18 +02:00
Aliaksandr Valialkin
4d9501a0c4 vendor: make vendor-update 2020-11-13 13:08:41 +02:00
Aliaksandr Valialkin
45958d16f6 vendor: make vendor-update 2020-11-06 13:57:06 +02:00
Aliaksandr Valialkin
dbd1789479 vendor: make vendor-update 2020-11-02 22:04:35 +02:00
Aliaksandr Valialkin
9df946517c vendor: make vendor-update 2020-11-02 02:40:17 +02:00
Aliaksandr Valialkin
cde26141c0 vendor: make vendor-update 2020-10-16 15:05:54 +03:00
Aliaksandr Valialkin
8c027ba8a4 vendor: update github.com/VictoriaMetrics/metricsql from v0.7.1 to v0.7.2
The new release of github.com/VictoriaMetrics/metricsql adds more optimizations for `foo{filters1} op bar{filters2}`:

* rollup_func(foo[d]) op bar{filters}
* transform_func(foo) op bar{filters}
* num_or_scalar op bar op baz{filters}
2020-10-16 12:55:27 +03:00
Aliaksandr Valialkin
7786ed95b5 vendor: make vendor-update 2020-10-08 11:52:43 +03:00
Aliaksandr Valialkin
3e2bf32872 vendor: update github.com/VictoriaMetrics/metricsql from v0.7.0 to v0.7.1 2020-10-08 11:47:23 +03:00
Aliaksandr Valialkin
f6ee6efc34 app/vmselect/promql: add missing label filters to binary operands before query execution
This implements the optimization described at https://utcc.utoronto.ca/~cks/space/blog/sysadmin/PrometheusLabelNonOptimization

See also https://github.com/cortexproject/cortex/issues/3253
2020-10-07 21:17:11 +03:00
Aliaksandr Valialkin
812c670d60 vendor: make vendor-update 2020-10-05 23:19:49 +03:00
Aliaksandr Valialkin
af7a0f7aff vendor: make vendor-update 2020-09-30 08:59:53 +03:00
Aliaksandr Valialkin
0548f0c5c8 vendor: make vendor-update 2020-09-28 21:59:20 +03:00
Aliaksandr Valialkin
265d892a8c vendor: make vendor-update 2020-09-23 14:23:51 +03:00
Aliaksandr Valialkin
4ebd2fa560 vendor: make vendor-update 2020-09-22 17:08:54 +03:00
Aliaksandr Valialkin
5bd30171b3 vendor: update github.com/valyala/quicktemplate from v1.6.2 to v1.6.3 2020-09-18 13:10:55 +03:00
Aliaksandr Valialkin
e25b90849f vendor: udpate github.com/VictoriaMetrics/fasthttp from v1.0.5 to v1.0.7 2020-09-18 12:20:48 +03:00
Aliaksandr Valialkin
7429fb2301 vendor: make vendor-update 2020-09-17 01:44:16 +03:00
Aliaksandr Valialkin
cd87ca303f lib/protoparser: report more errors for incorrect timestamps and/or values
Previously certain errors in timestamps and/or values could be silently skipped,
which could lead to samples with zero values stored in the database.

Updates https://github.com/VictoriaMetrics/vmctl/issues/25
2020-09-16 02:16:15 +03:00
Aliaksandr Valialkin
5c4e111b43 lib/protoparser/graphite: return error when value or timestamp cannot be properly parsed
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/99
2020-09-16 02:16:13 +03:00
Aliaksandr Valialkin
2a86e2fb98 vendor: update github.com/valyala/fastjson from v1.5.4 to v1.5.5
This should properly parse `+Inf` values when importing JSON lines via `/api/v1/import`

Updates https://github.com/VictoriaMetrics/vmctl/issues/25
2020-09-16 00:09:05 +03:00
Aliaksandr Valialkin
b776a93608 app/vmselect/promql: support composite durations like Prometheus 2.21 does
The following durations are supported now: `1h5m35s` or `1s543ms`

See https://github.com/prometheus/prometheus/releases/tag/v2.21.0
and https://github.com/prometheus/prometheus/pull/7713
2020-09-11 22:17:24 +03:00
Aliaksandr Valialkin
87f916a2fb vendor: make vendor-update 2020-09-10 09:50:00 +03:00
Aliaksandr Valialkin
df6519c190 vendor: make vendor-update 2020-09-08 15:18:59 +03:00
Aliaksandr Valialkin
fcde009e11 vendor: update github.com/VictoriaMetrics/metricsql from v0.4.3 to v0.5.1
The new version of the package supports binary operations on string literals:

    * "foo" + "bar"     => "foobar"
    * "foo" == "bar"    => NaN
    * "foo" == "foo"    => 1
    * "foo" >bool "bar" => 1
    * "foo" < "bar"     => NaN

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/717
2020-09-03 16:33:23 +03:00
Aliaksandr Valialkin
478d8f8393 app/vmselect/promql: add count_le_over_time(m[d], le) and count_gt_over_time(m[d], gt) functions
These functions returns the number of raw samples that don't exceed `le` or are bigger than `gt`.
These functions are complement to already existing `share_le_over_time(m[d], le)` and `share_gt_over_time(m[d], gt)`.
2020-09-03 15:28:58 +03:00
Aliaksandr Valialkin
c3a2d4ee6f vendor: update github.com/VictoriaMetrics/metricsql from v0.4.1 to v0.4.2
The new version of this package properly supports escaped identifiers.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/743
2020-09-03 14:51:06 +03:00
Aliaksandr Valialkin
74ba0a6271 vendor: make vendor-update 2020-09-01 17:43:27 +03:00
Aliaksandr Valialkin
98217e4c40 vendor: upgrade github.com/valyala/gozstd from v1.7.1 to v1.8.3 2020-08-15 15:11:50 +03:00
Aliaksandr Valialkin
008a769434 vendor: downgrade github.com/valyala/gozstd from v1.8.1 to v1.7.1 until https://github.com/facebook/zstd/issues/2222 is fixed 2020-08-15 14:51:16 +03:00
Aliaksandr Valialkin
7fb942308c vendor: update github.com/valyala/gozstd from v1.7.0 to v1.8.1 2020-08-15 13:46:34 +03:00
Aliaksandr Valialkin
f4c90449dc deployment/dm: update images 2020-08-14 02:27:55 +03:00
Aliaksandr Valialkin
0a34f56b39 vendor: update github.com/VictoriaMetrics/fasthttp from v1.0.4 to v1.0.5 2020-08-14 02:19:56 +03:00
Aliaksandr Valialkin
d615ae81e5 vendor: update github.com/klauspost/compress from v1.10.10 to v1.10.11 2020-08-14 02:16:39 +03:00
Aliaksandr Valialkin
60c7397be5 all: support %{ENV_VAR} placeholders in yaml configs in all the vm* components
Such placeholders are substituted by the corresponding environment variable values.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/583
2020-08-13 17:17:06 +03:00
Aliaksandr Valialkin
43830b1699 vendor: make vendor-update 2020-08-09 15:14:09 +03:00
Aliaksandr Valialkin
d8aa433c4d vendor: update github.com/VictoriaMetrics/metrics from v1.12.2 to v1.12.3 2020-08-07 13:02:35 +03:00
Aliaksandr Valialkin
95a8c492ef app/vmselect/promql: properly handle -n^m like Prometheus does
`-n^m` must be handled as `-(n^m)` instead of `(-n)^m`.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/675
2020-08-07 07:42:42 +03:00
Aliaksandr Valialkin
8835004a4c app/vmselect: properly handle PromQL queries like scalar1 < metric < scalar2 like Prometheus does
This fixes some cases from https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 23:21:14 +03:00
Aliaksandr Valialkin
3947307363 vendor: update github.com/VictoriaMetrics/metricsql from v0.2.10 to v0.3.0
This adds support for special integers in MetricsQL that start from 0x, 0b, 0o.
This improves compatibility with PromQL - see https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 21:45:33 +03:00
Aliaksandr Valialkin
76064ba9e7 Perform conversion from string to []byte according to rule #6 at https://golang.org/pkg/unsafe/#Pointer 2020-08-05 11:55:12 +03:00
Aliaksandr Valialkin
9038df211f vendor: make vendor-update 2020-08-05 11:15:47 +03:00
Aliaksandr Valialkin
bdb881c43b app/vmselect/promql: add zscore-related functions: zscore_over_time(m[d]) and zscore(q) by (...) 2020-08-03 21:52:15 +03:00
Aliaksandr Valialkin
bbc7583015 vendor: update github.com/valyala/quicktemplate from v1.5.2 to v1.6.0 2020-07-30 23:39:48 +03:00
Aliaksandr Valialkin
f068ea74d0 vendor: make vendor-update 2020-07-29 09:35:34 +03:00
Aliaksandr Valialkin
ab47fa2300 vendor: update github.com/VictoriaMetrics/metrics from v1.12.1 to v1.12.2 2020-07-28 22:01:41 +03:00
Aliaksandr Valialkin
ce7c8898af vendor: update github.com/VictoriaMetrics/metrics from v1.12.0 to v1.12.1 2020-07-28 00:21:06 +03:00
Aliaksandr Valialkin
cf69b1ea6f app/vmselect/promql: add buckets_limit(k, buckets) function, which limits the number of buckets per time series to k
This function works with both Prometheus-style and VictoriaMetrics-style buckets.
The function removes buckets with the lowest values in order to reserve the highest precision.
The function is useful for building heatmaps in Grafana from too big number of buckets.
2020-07-24 16:14:12 +03:00
Aliaksandr Valialkin
8d1721d128 app/vmselect/promql: add rate_over_sum(m[d]) function to MetricsQL, which returns rate over sum of m values over d duration
Something like `sum_over_time(m[d]) / d`, but more accurate.
2020-07-24 01:17:15 +03:00
Aliaksandr Valialkin
edbc777e91 vendor: make vendor-update 2020-07-22 16:54:02 +03:00
Aliaksandr Valialkin
77523af9fc vendor: update github.com/VictoriaMetrics/metrics from v1.11.3 to v1.12.0 2020-07-20 16:56:44 +03:00
Aliaksandr Valialkin
4d2011a87d app/vmselect/promql: add mode() aggregate function 2020-07-20 15:30:11 +03:00
Aliaksandr Valialkin
427fa43ce2 app/vmselect/promql: add mode_over_time(m[d]) function
See https://en.wikipedia.org/wiki/Mode_(statistics) and https://stackoverflow.com/questions/61134078/promql-query-to-return-the-value-from-a-range-vector-which-occurs-maximum-no-of
2020-07-17 18:29:10 +03:00
Aliaksandr Valialkin
fc8fe38a82 app/vmselect/promql: add group() aggregate function to MetricsQL
This function has been added in Prometheus 2.20. See https://github.com/prometheus/prometheus/pull/7480
2020-07-17 15:17:38 +03:00
Aliaksandr Valialkin
74fb0b293d vendor: make vendor-update 2020-07-14 20:30:57 +03:00
Aliaksandr Valialkin
5910207d61 vendor: update github.com/valyala/quicktemplate from v1.5.0 to v1.5.1
This should fix incorrect encoding for json strings with char codes below 0x20

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/613
2020-07-10 12:58:40 +03:00
Aliaksandr Valialkin
fd4d593c75 vendor: make vendor-update 2020-07-08 19:24:59 +03:00
Aliaksandr Valialkin
fb86071552 app/vmselect: add /api/v1/status/active_queries page with the list of currently running queries
This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/575

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/528
2020-07-08 19:09:31 +03:00
DexterZhang
9930ce1fa9
Feat/query list vmselect (#575)
* feat(vmselect): add support for listing current running queries and canceling specific query

* fix(vmselect): change current queries' pid from int64 counter to uuid

* feat(vmselect): add auth to internal operations like `/resetRollupResultCache`, `/query/list` and `/query/kill`. add flag `internalAuthKey` for these auth

* fix(vmselect): add more info to current queries

* review: delete some unnecessary code and use function instead of init

* review: returen *queriesMap in newQueriesMap

* review: delete unused var in struct queriesMap, add comments to exported functions

* review: add return if error occurs

* feat(vmselect): truncate query string in current running query list API since the size of query string might be large;
                use query string's pointer in struct `query` for the same reason;
		add query info API to get full access of query's info;
2020-07-08 19:04:29 +03:00
Aliaksandr Valialkin
f10e8809c0 app/vmselect: add interpolate function for filling gaps with linearly interpolated values
See https://stackoverflow.com/q/62565021/274937 for details
2020-07-02 14:54:46 +03:00
Aliaksandr Valialkin
618bcc818c vendor: make vendor-update 2020-07-01 01:04:05 +03:00
Aliaksandr Valialkin
9b06c83cd6 vendor: make vendor-update 2020-06-25 23:44:33 +03:00
Aliaksandr Valialkin
b2b17589fa vendor: update github.com/valyala/fastjson from v1.5.1 to v1.5.2 2020-06-25 23:34:24 +03:00
Aliaksandr Valialkin
6d9c5ad422 vendor: update github.com/klauspost/compress from v1.10.9 to v1.10.10 2020-06-23 13:47:00 +03:00
Aliaksandr Valialkin
ca4c9023e3 vendor: make vendor-update 2020-06-19 02:40:36 +03:00
Aliaksandr Valialkin
60b8ce47ad vendor: make vendor-update 2020-06-06 00:00:40 +03:00
Aliaksandr Valialkin
356845d716 vendor: update github.com/klauspost/compress from v1.10.7 to v1.10.8 2020-06-05 23:52:08 +03:00
Aliaksandr Valialkin
91cebdccde vendor: update github.com/klauspost/compress from v1.10.6 to v1.10.7 2020-06-03 00:11:25 +03:00
Aliaksandr Valialkin
fc32881105 vendor: make vendor-update 2020-05-27 18:41:21 +03:00
Aliaksandr Valialkin
97feac596f vendor: make vendor-update 2020-05-25 00:06:22 +03:00
Aliaksandr Valialkin
d83c68ca03 app/vmselect/promql: add ascent_over_time(m[d]) and descent_over_time(m[d]) functions
These functions could be useful in GPS tracking apps for calculating the summary for height gain/loss
over the given duration `d`.
2020-05-21 12:06:34 +03:00
Aliaksandr Valialkin
071d58864b vendor: update github.com/VictoriaMetrics/metrics from v1.11.2 to v1.11.3 2020-05-20 02:55:17 +03:00
Aliaksandr Valialkin
fc81ea38d4 app/vmselect/promql: add outliersk(N, m) aggregate function for anomaly detection across groups of similar time series 2020-05-19 13:52:44 +03:00
Aliaksandr Valialkin
bdbb5f6cfe vendor: make vendor-update 2020-05-15 15:03:11 +03:00
Aliaksandr Valialkin
408ade27a9 app/vmselect/promql: add any(x) by (y) aggregate function, which returns any time series from q for each group y 2020-05-12 19:50:29 +03:00
Aliaksandr Valialkin
21c2982ac8 app/vmselect/promql: support for sum(x) by (y) limit N syntax in order to limit the number of output time series after aggregation 2020-05-12 19:50:12 +03:00
Aliaksandr Valialkin
cc825c483b vendor: make vendor-update 2020-05-12 14:26:29 +03:00
Aliaksandr Valialkin
fd739808f3 make vendor-update 2020-05-05 00:53:41 +03:00
Aliaksandr Valialkin
54414fefef vendor: update github.com/valyala/quicktemplate from v1.4.1 to v1.5.0 2020-05-04 01:37:34 +03:00
Aliaksandr Valialkin
b6d88bac04 vendor: use github.com/VictoriaMetrics/fasthttp instead of github.com/fasthttp/fasthttp
The upstream fasthttp may contain issues like 996610f021 ,
plus a code that isn't used by VictoriaMetrics. So let's use a private copy under our control instead.
2020-04-29 16:43:09 +03:00
Aliaksandr Valialkin
9ed4951ec8 lib/metricsql: move it to a separate repository - github.com/VictoriaMetrics/metrics 2020-04-28 15:30:06 +03:00
Aliaksandr Valialkin
c66daf1f0a vendor: make vendor-update 2020-04-24 15:28:37 +03:00
Aliaksandr Valialkin
b0f6d3244c vendor: update github.com/aws/aws-sdk-go from v1.30.8 to v1.30.12 2020-04-23 12:36:14 +03:00
Aliaksandr Valialkin
36f6935ddd vendor: update github.com/valyala/fastjson from v1.5.0 to v1.5.1 2020-04-21 00:04:14 +03:00
Aliaksandr Valialkin
76c4140da7 vendor: update github.com/valyala/gozstd from v1.6.4 to v1.7.0 2020-04-20 23:04:20 +03:00
Aliaksandr Valialkin
da05904638 vendor: make vendor-update 2020-04-17 13:25:18 +03:00
Aliaksandr Valialkin
9fd1827824 vendor: make vendor-update 2020-04-15 14:51:51 +03:00
Aliaksandr Valialkin
1ce6c311dd vendor: make vendor-update 2020-04-10 18:42:02 +03:00