Aliaksandr Valialkin 0efd37cec1 lib/{mergeset,storage}: reduce the number of fsync calls on data ingestion path on systems with many cpu cores ... VictoriaMetrics maintains a buffer per CPU core for the ingested data. These buffers are flushed to disk every second. These buffers are flushed to disk in parallel starting from the commit 56b6b893ce . This resulted in increased write disk IO usage on systems with many cpu cores as described at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338#issuecomment-863046999 . This commit merges the per-CPU buffers into bigger in-memory buffers before flushing them to disk. This should reduce the rate of fsync syscalls and, consequently, the write disk IO on systems with many CPU cores. This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338 See also https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1244		2021-06-17 13:51:42 +03:00
..
block_header.go	lib/{mergeset,storage}: do not use pools for indexBlock and inmemoryBlock during their caching, since this results in higher memory usage in production without any performance gains	2021-02-21 21:43:37 +02:00
block_stream_reader_test.go	lib/mergeset: reduce memory usage for inmemoryBlock by using more compact items representation	2021-02-21 22:09:10 +02:00
block_stream_reader.go	lib/mergeset: reduce memory usage for inmemoryBlock by using more compact items representation	2021-02-21 22:09:10 +02:00
block_stream_writer.go	all: use %w instead of %s for wrapping errors in fmt.Errorf	2020-06-30 23:33:46 +03:00
encoding_test.go	lib/mergeset: reduce memory usage for inmemoryBlock by using more compact items representation	2021-02-21 22:09:10 +02:00
encoding_timing_test.go	all: open-sourcing single-node version	2019-05-23 00:18:06 +03:00
encoding.go	Revert "lib/mergeset: remove a pool for inmemoryBlock structs"	2021-05-28 01:11:22 +03:00
inmemory_part.go	lib/mergeset: reduce memory usage for inmemoryBlock by using more compact items representation	2021-02-21 22:09:10 +02:00
merge_test.go	lib/mergeset: reduce memory usage for inmemoryBlock by using more compact items representation	2021-02-21 22:09:10 +02:00
merge.go	lib/mergeset: reduce memory usage for inmemoryBlock by using more compact items representation	2021-02-21 22:09:10 +02:00
metaindex_row.go	lib: dump compressed block contents on error during decompression	2020-08-15 14:51:14 +03:00
part_header.go	all: use %w instead of %s for wrapping errors in fmt.Errorf	2020-06-30 23:33:46 +03:00
part_search_test.go	lib/mergeset: unconditionally cache indexdb blocks	2021-02-09 00:49:59 +02:00
part_search.go	Revert "lib/mergeset: remove a pool for inmemoryBlock structs"	2021-05-28 01:11:22 +03:00
part.go	lib/{mergeset,storage}: do not use pools for indexBlock and inmemoryBlock during their caching, since this results in higher memory usage in production without any performance gains	2021-02-21 21:43:37 +02:00
table_search_test.go	lib/mergeset: unconditionally cache indexdb blocks	2021-02-09 00:49:59 +02:00
table_search_timing_test.go	lib/mergeset: reduce memory usage for inmemoryBlock by using more compact items representation	2021-02-21 22:09:10 +02:00
table_search.go	lib/mergeset: unconditionally cache indexdb blocks	2021-02-09 00:49:59 +02:00
table_test.go	lib/mergeset: reduce memory usage for inmemoryBlock by using more compact items representation	2021-02-21 22:09:10 +02:00
table.go	lib/{mergeset,storage}: reduce the number of fsync calls on data ingestion path on systems with many cpu cores	2021-06-17 13:51:42 +03:00