VictoriaMetrics/app/vlinsert/elasticsearch
Aliaksandr Valialkin 120f3bc467
lib/logstorage: follow-up for 8a23d08c21
- Compare the actual free disk space to the value provided via -storage.minFreeDiskSpaceBytes
  directly inside the Storage.IsReadOnly(). This should work fast in most cases.
  This simplifies the logic at lib/storage.

- Do not take into account -storage.minFreeDiskSpaceBytes during background merges, since
  it results in uncontrolled growth of small parts when the free disk space approaches -storage.minFreeDiskSpaceBytes.
  The background merge logic uses another mechanism for determining whether there is enough
  disk space for the merge - it reserves the needed disk space before the merge
  and releases it after the merge. This prevents from out of disk space errors during background merge.

- Properly handle corner cases for flushing in-memory data to disk when the storage
  enters read-only mode. This is better than losing the in-memory data.

- Return back Storage.MustAddRows() instead of Storage.AddRows(),
  since the only case when AddRows() can return error is when the storage is in read-only mode.
  This case must be handled by the caller by calling Storage.IsReadOnly()
  before adding rows to the storage.
  This simplifies the code a bit, since the caller of Storage.MustAddRows() shouldn't handle
  errors returned by Storage.AddRows().

- Properly store parsed logs to Storage if parts of the request contain invalid log lines.
  Previously the parsed logs could be lost in this case.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4737
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4945
2023-10-02 20:38:00 +02:00
..
bulk_response.qtpl app/victoria-logs: initial code release 2023-07-06 17:30:05 -07:00
bulk_response.qtpl.go app/victoria-logs: initial code release 2023-07-06 17:30:05 -07:00
elasticsearch_test.go lib/logstorage: follow-up for 8a23d08c21 2023-10-02 20:38:00 +02:00
elasticsearch_timing_test.go lib/logstorage: follow-up for 8a23d08c21 2023-10-02 20:38:00 +02:00
elasticsearch.go lib/logstorage: follow-up for 8a23d08c21 2023-10-02 20:38:00 +02:00