Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-05 01:01:09 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
fd175ad80b
VictoriaMetrics/lib/storage
History
Aliaksandr Valialkin 1b16118e17
lib/{storage,mergeset}: tune the threshold for assisted merge 
The https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3425#issuecomment-1359117221
reveals that CPU usage for incoming queries may significantly increase when the number
of in-memory parts becomes too big.

This commit reduces the maximum number of in-memory parts before starting the assisted merge
during data ingestion. This should reduce CPU usage for incoming queries,
since they need to inspect lower number of in-memory parts.

This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3425
2022-12-28 14:39:24 -08:00
..
block_header_test.go lib/storage: typo fix: umarshal -> unmarshal 2021-03-02 20:47:59 +02:00
block_header.go lib/storage: verify that timestamps in block are in the range specified by blockHeader.{Min,Max}Timestamp when upacking the block 2022-09-06 13:08:09 +03:00
block_stream_merger.go lib/storage: skip blocks outside the configured retention during search 2022-10-24 02:52:44 +03:00
block_stream_reader_test.go
block_stream_reader_timing_test.go
block_stream_reader.go lib/storage: remove logging redundant path values in a single error message 2022-12-03 22:13:13 -08:00
block_stream_writer_timing_test.go lib/{mergeset,storage}: pass compressLevel to blockStreamWriter.InitFromInmemoryPart 2022-12-03 22:46:48 -08:00
block_stream_writer.go lib/{mergeset,storage}: pass compressLevel to blockStreamWriter.InitFromInmemoryPart 2022-12-03 22:46:48 -08:00
block_test.go lib/storage: verify that timestamps in block are in the range specified by blockHeader.{Min,Max}Timestamp when upacking the block 2022-09-06 13:08:09 +03:00
block.go lib/storage: validate timestamps in the block only if they use encoding, which needs validation 2022-10-21 00:52:32 +03:00
dedup_test.go lib/storage: follow-up after 7c0ae3a86a 2022-12-08 18:16:57 -08:00
dedup_timing_test.go lib/storage: follow-up after 7c0ae3a86a 2022-12-08 18:16:57 -08:00
dedup.go lib/storage: follow-up after 7c0ae3a86a 2022-12-08 18:16:57 -08:00
index_db_test.go lib/storage: search for TSIDs for the given metricIDs in the previous indexdb if they aren't found in the current indexdb 2022-12-19 12:03:09 -08:00
index_db_timing_test.go lib/storage: properly take into account already registered series when -storage.maxHourlySeries or -storage.maxDailySeries limits are enabled 2022-06-20 13:47:47 +03:00
index_db.go lib/storage: clear the err if it is set to io.EOF when searching for the TSID by metricID 2022-12-20 14:05:29 -08:00
inmemory_part_test.go
inmemory_part_timing_test.go
inmemory_part.go all: add -inmemoryDataFlushInterval command-line flag for controlling the frequency of saving in-memory data to disk 2022-12-05 15:16:14 -08:00
merge_test.go lib/{mergeset,storage}: pass compressLevel to blockStreamWriter.InitFromInmemoryPart 2022-12-03 22:46:48 -08:00
merge_timing_test.go lib/{mergeset,storage}: pass compressLevel to blockStreamWriter.InitFromInmemoryPart 2022-12-03 22:46:48 -08:00
merge.go lib/storage: optimization: do not scan block for rows outside retention if it is covered by the retention 2022-12-03 22:14:12 -08:00
metaindex_row_test.go
metaindex_row.go all: subsitute ioutil.ReadAll with io.ReadAll 2022-08-22 00:16:37 +03:00
metric_name_test.go app/vminsert: add support for data ingestion via other vminsert nodes 2021-05-08 19:52:57 +03:00
metric_name.go lib/regexutil: add Simplify() function for simplifying the regular expression 2022-08-26 11:57:12 +03:00
part_header_test.go
part_header.go lib/fs: add canOverwrite arg to WriteFileAtomically when it is allowed to overwrite the file atomically if it already exists 2022-10-26 01:07:34 +03:00
part_search_test.go app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day 2022-06-28 13:02:47 +03:00
part_search_timing_test.go lib/storage: skip missing tsids in the current block header by using binary search 2022-12-14 22:06:51 -08:00
part_search.go lib/storage: optimize partSearch.searchBHS() for common case when the TSID for the current block header is bigger or equal to the current tsid 2022-12-19 10:28:03 -08:00
part.go lib/mergeset: tune caches size limits for indexdb/dataBlocks and indexdb/indexBlocks 2022-01-21 12:45:43 +02:00
partition_search_test.go all: add -inmemoryDataFlushInterval command-line flag for controlling the frequency of saving in-memory data to disk 2022-12-05 15:16:14 -08:00
partition_search.go all: make fmt via the upcoming Go1.19 2022-07-11 19:22:15 +03:00
partition_test.go lib/{mergeset,storage}: improve the detection of the needed free space for background merge 2021-08-25 09:35:44 +03:00
partition.go lib/{storage,mergeset}: tune the threshold for assisted merge 2022-12-28 14:39:24 -08:00
raw_block.go
raw_row.go lib/{mergeset,storage}: pass compressLevel to blockStreamWriter.InitFromInmemoryPart 2022-12-03 22:46:48 -08:00
search_test.go app/vmselect: optimize /api/v1/series a bit for time ranges smaller than one day 2022-06-28 13:02:47 +03:00
search.go lib/storage: skip blocks outside the configured retention during search 2022-10-24 02:52:44 +03:00
storage_test.go all: add -inmemoryDataFlushInterval command-line flag for controlling the frequency of saving in-memory data to disk 2022-12-05 15:16:14 -08:00
storage_timing_test.go app/vmstorage: add ability to limit series cardinality via -storage.maxHourlySeries and -storage.maxDailySeries command-line flags 2021-05-20 14:15:19 +03:00
storage.go lib/flagutil/bytes.go: properly handle values bigger than 2GiB on 32-bit architectures 2022-12-14 19:26:31 -08:00
table_search_test.go all: add -inmemoryDataFlushInterval command-line flag for controlling the frequency of saving in-memory data to disk 2022-12-05 15:16:14 -08:00
table_search_timing_test.go all: add -inmemoryDataFlushInterval command-line flag for controlling the frequency of saving in-memory data to disk 2022-12-05 15:16:14 -08:00
table_search.go lib/storage: do not pass retentionMsecs and isReadOnly args explicitly - access them via Storage arg 2022-10-24 01:31:04 +03:00
table_test.go lib/storage: do not pass retentionMsecs and isReadOnly args explicitly - access them via Storage arg 2022-10-24 01:31:04 +03:00
table_timing_test.go all: add -inmemoryDataFlushInterval command-line flag for controlling the frequency of saving in-memory data to disk 2022-12-05 15:16:14 -08:00
table.go all: add -inmemoryDataFlushInterval command-line flag for controlling the frequency of saving in-memory data to disk 2022-12-05 15:16:14 -08:00
tag_filters_test.go lib/regexutil: add Simplify() function for simplifying the regular expression 2022-08-26 11:57:12 +03:00
tag_filters_timing_test.go lib/regexutil: add Simplify() function for simplifying the regular expression 2022-08-26 11:57:12 +03:00
tag_filters.go lib/storage: optimize matching speed for non-trivial regexp filters 2022-10-01 12:06:06 +03:00
time_test.go
time.go all: readability improvements for query traces 2022-06-30 18:20:33 +03:00
tsid_test.go
tsid.go