VictoriaMetrics/app
Roman Khavronenko 562edb72ea
app/vmalert: fix data race during hot-config reload (#5698)
* app/vmalert: fix data race during hot-config reload

During hot-reload, the logic evokes the group update and rules evaluation
interruption simultaneously. Falsely assuming that interruption happens before
the update. However, it could happen that group will be updated first and only
after the rules evaluation will be cancelled. Which will result in permanent
interruption for all rules within the group.

The fix caches the cancel context function into local variable first. And only after
performs the group update. With cached cancel function we can safely call it without
worrying that we cancel the evaluation for already updated group.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* Revert "app/vmalert: fix data race during hot-config reload"

This reverts commit a4bb7e8932.

* app/vmalert: fix data race during hot-config reload

During hot-reload, the logic evokes the group update and rules evaluation
interruption simultaneously. Falsely assuming that interruption happens before
the update. However, it could happen that group will be updated first and only
after the rules evaluation will be cancelled. Which will result in permanent
interruption for all rules within the group.

The fix cancels the evaulation context before applying the update, making sure
that the context will be cancelled for old group always.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2024-01-26 22:43:02 +01:00
..
victoria-logs lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop() 2024-01-16 21:18:22 +02:00
vlinsert Add _stream fields log (#5068) 2023-11-17 16:04:13 +01:00
vlselect vmui: reduced the number of server requests (#5253) 2023-11-14 01:50:57 +01:00
vlstorage lib/logstorage: follow-up for 8a23d08c21 2023-10-02 20:38:00 +02:00
vmagent all: add up to 10% random jitter to the interval between periodic tasks performed by various components 2024-01-22 18:39:16 +02:00
vmalert app/vmalert: fix data race during hot-config reload (#5698) 2024-01-26 22:43:02 +01:00
vmalert-tool docs: convert png images to webp in all the docs except of docs/operator/* 2023-11-22 19:29:47 +02:00
vmauth app/vmauth: return 503 service unavailable status code when the backend returns response with unsupported status code, but the request cannot be re-tried. 2024-01-26 21:10:57 +01:00
vmbackup lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop() 2024-01-16 21:18:22 +02:00
vmbackupmanager docs: convert png images to webp in all the docs except of docs/operator/* 2023-11-22 19:29:47 +02:00
vmctl app/vmctl/backoff: fix flaky test 2024-01-22 18:42:05 +02:00
vmgateway docs: convert png images to webp in all the docs except of docs/operator/* 2023-11-22 19:29:47 +02:00
vminsert app/vminsert/clusternative: explain why lower-level vminsert doesnt compress responses to upper-level vminsert 2024-01-23 18:14:19 +02:00
vmrestore lib/pushmetrics: wait until the background goroutines, which push metrics, are stopped at pushmetrics.Stop() 2024-01-16 21:18:22 +02:00
vmselect app/vmselect/netstorage: initialize tmpBlocksFileWrapper at goroutine, which continues using it 2024-01-26 21:29:30 +01:00
vmstorage lib/{mergeset,storage}: make background merge more responsive and scalable 2024-01-26 22:19:52 +01:00
vmui vmui: fix Enter key in query field (#5667) (#5681) 2024-01-26 22:38:51 +01:00