VictoriaMetrics/app
Zakhar Bessarab 401ae72587
app/vmselect/promql: propagate lower bucket values when fixing a histogram (#6547)
### Describe Your Changes

In most cases histograms are exposed in sorted manner with lower buckets
being first. This means that during scraping buckets with lower bounds
have higher chance of being updated earlier than upper ones.

Previously, values were propagated from upper to lower bounds, which
means that in most cases that would produce results higher than expected
once all buckets will become updated.
Propagating from upper bound effectively limits highest value of
histogram to the value of previous scrape. Once the data will become
consistent in the subsequent evaluation this causes spikes in the
result.

Changing propagation to be from lower to higher buckets reduces value
spikes in most cases due to nature of the original inconsistency.

 See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4580

An example histogram with previous(red) and updated(blue) versions:

![1719565540](https://github.com/VictoriaMetrics/VictoriaMetrics/assets/1367798/605c5e60-6abe-45b5-89b2-d470b60127b8)

This also makes logic of filling nan values with lower buckets values: [1 2 3 nan nan nan] => [1 2 3 3 3 3] obsolete.
Since buckets are now fixed from lower ones to upper this happens in the main loop, so there is no need in a second one.

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Andrii Chubatiuk <andrew.chubatiuk@gmail.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6a4bd5049b)
2024-07-10 15:17:08 +02:00
..
victoria-logs deployment: build image for vmagent streamaggr benchmark (#6515) 2024-06-24 16:29:14 +02:00
vlinsert app/vlinsert/loki: use easyproto instead for parsing Loki protobuf messages 2024-07-10 03:05:55 +02:00
vlogsgenerator lib/logstorage: work-in-progress 2024-05-25 00:31:55 +02:00
vlselect app/vlselect/vmui: run make vmui-logs-update after 662e026279 2024-07-10 03:05:55 +02:00
vlstorage lib/logstorage: work-in-progress 2024-06-27 14:21:03 +02:00
vmagent all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
vmalert all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
vmalert-tool deployment: build image for vmagent streamaggr benchmark (#6515) 2024-06-24 16:29:14 +02:00
vmauth all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
vmbackup deployment: build image for vmagent streamaggr benchmark (#6515) 2024-06-24 16:29:14 +02:00
vmbackupmanager all: replace old https://docs.victoriametrics.com/vmbackupmanager.html url with the new one - https://docs.victoriametrics.com/vmbackupmanager/ 2024-04-18 02:04:39 +02:00
vmctl all: consistently use 'any' instead of 'interface{}' 2024-07-10 00:23:26 +02:00
vmgateway all: replace old https://docs.victoriametrics.com/vmgateway.html url with the new one - https://docs.victoriametrics.com/vmgateway/ 2024-04-18 02:08:53 +02:00
vminsert deployment: build image for vmagent streamaggr benchmark (#6515) 2024-06-24 16:29:14 +02:00
vmrestore deployment: build image for vmagent streamaggr benchmark (#6515) 2024-06-24 16:29:14 +02:00
vmselect app/vmselect/promql: propagate lower bucket values when fixing a histogram (#6547) 2024-07-10 15:17:08 +02:00
vmstorage lib/mergeset: adds tracking for indexdb records drop (#6297) 2024-05-24 16:08:34 +02:00
vmui vmui/logs: add spinner to bar chart (#6577) 2024-07-09 18:27:23 +02:00