VictoriaMetrics/app/vmselect
Zakhar Bessarab af3922b1df
lib/storage: add ability to use downsampling for the given series filter (#733)
* lib/storage: add ability to use downsampling for the given series filter

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: add information about downsampling filters

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: fix MetricsQL filter

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage/downsampling: treat missing downsampling filter as a bug

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage/part_header: verify correctness of downsampling filters when opening partition

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage/downsampling: save only appliable rules in part metadata

Filter and save only rules which are appliable to partition based on MinTimestamp of stored data.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage/downsampling: update log messages for final dedup

Properly specify a reason of re-running deduplication for partition.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage: consistently use MaxTimestamp to determine deduplication/downsampling rules

Using MinTimestamp leads to applying downsampling to parts which are only partially covered by downsampling rule.
For example, partition covers range [1000-2000]. At t=2100 and rule offset 500 data with t=2100-500 => 1600 must be downsampled. The range check against MinTimestamp evaluates to true even though partition contains range which must not be downsampled - [1600:2000].

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* Follow-up

- Apply the first matching downsampling period if multiple filters match the given time series.
  This allows fine-tuning the downsampling config for the specific needs.
- Take into account downsampling filters during search queries.
- Reduce the difference between community and enterprise branches. This should simplify further maintenance of these branches.
- Properly parse series filters with colons inside them.
- Document the feature at docs/CHANGELOG.md.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4960

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2024-03-30 04:12:23 +02:00
..
graphite adding AggregateSeriesLists graphite function (#5809) 2024-03-26 15:13:34 +01:00
graphiteql app,lib: fix typos in comments (#3804) 2023-02-13 13:27:13 +01:00
netstorage lib/storage: add ability to use downsampling for the given series filter (#733) 2024-03-30 04:12:23 +02:00
prometheus app/vmselect: use GetDeadlineForLabelsAPI for LabelAPI requests (#5992) 2024-03-25 15:07:34 +01:00
promql app/vmselect/promql: use unsafe.Slice instead of deprecated reflect.SliceHeader 2024-02-29 17:50:07 +02:00
querystats app/vmselect: do not track queries with less than 1ms execution time at /api/v1/status/top_queries 2021-07-15 16:44:28 +03:00
searchutils docs: follow-up for 70eaa06f08 2024-03-25 15:30:56 +01:00
vmui fix typo in kyiv city name 2024-03-23 21:56:31 +01:00
main.go vmsingle/vmselect returns http status 429 (TooManyRequests) instead of 503 (ServiceUnavailable) when max concurrent requests limit is reached. (#5682) 2024-01-24 17:55:06 +01:00
README.md app/vmui: move source code from https://github.com/VictoriaMetrics/vmui to app/vmui 2021-07-09 17:15:23 +03:00
vmui.go app/vmselect: add missing whitespace into the description for -vmui.defaultTimezone command-line flag 2024-02-01 14:49:51 +02:00

vmselect performs the incoming queries and fetches the required data from vmstorage.

The vmui directory contains static contents built from app/vmui package with make vmui-update command. The vmui page is available at http://<victoria-metrics>:8428/vmui/.