Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-15 08:23:34 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
c3b1d9ee21
VictoriaMetrics/lib/storage
History
Roman Khavronenko c3b1d9ee21
lib/storage: introduce per-day MetricName=>TSID index (#4252) 
The new index substitutes global MetricName=>TSID index
used for locating TSIDs on ingestion path.
For installations with high ingestion and churn rate, global
MetricName=>TSID index can grow enormously making
index lookups too expensive. This also results into bigger
than expected cache growth for indexdb blocks.

New per-day index supposed to be much smaller and more efficient.
This should improve ingestion speed and reliability during
re-routings in cluster.

The negative outcome could be occupied disk size, since
per-day index is more expensive comparing to global index.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-05-16 23:18:11 -07:00
..
block_header_test.go
block_header.go lib/storage: verify that timestamps in block are in the range specified by blockHeader.{Min,Max}Timestamp when upacking the block 2022-09-06 13:07:49 +03:00
block_stream_merger.go lib/storage: skip blocks outside the configured retention during search 2022-10-24 02:56:13 +03:00
block_stream_reader_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
block_stream_reader_timing_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
block_stream_reader.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
block_stream_writer_timing_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
block_stream_writer.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
block_test.go lib/storage: use deterministic random generator in tests 2023-01-23 20:12:32 -08:00
block.go lib/storage: validate timestamps in the block only if they use encoding, which needs validation 2022-10-21 00:54:37 +03:00
dedup_test.go lib/storage: follow-up after 7c0ae3a86a 2022-12-08 18:18:49 -08:00
dedup_timing_test.go lib/storage: follow-up after 7c0ae3a86a 2022-12-08 18:18:49 -08:00
dedup.go app,lib: fix typos in comments (#3804) 2023-02-13 09:32:35 -08:00
filenames.go lib/storage: consistently use OS-independent separator in file paths 2023-03-25 14:34:36 -07:00
index_db_test.go lib/storage: introduce per-day MetricName=>TSID index (#4252) 2023-05-16 23:18:11 -07:00
index_db_timing_test.go lib/fs: add MustReadDir() function 2023-04-14 22:11:40 -07:00
index_db.go lib/storage: introduce per-day MetricName=>TSID index (#4252) 2023-05-16 23:18:11 -07:00
inmemory_part_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
inmemory_part_timing_test.go lib/storage: use deterministic random generator in tests 2023-01-23 20:12:32 -08:00
inmemory_part.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
merge_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
merge_timing_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
merge.go lib/storage: optimization: do not scan block for rows outside retention if it is covered by the retention 2022-12-03 22:14:20 -08:00
metaindex_row_test.go
metaindex_row.go all: subsitute ioutil.ReadAll with io.ReadAll 2022-08-22 00:16:04 +03:00
metric_name_test.go app/vminsert: add support for data ingestion via other vminsert nodes 2021-05-08 19:53:45 +03:00
metric_name.go lib/storage: log original labels set when label value is truncated (#3952) 2023-03-14 16:11:02 -07:00
part_header.go lib/storage: fix a bug, which prevents from reading pre-v1.90.0 parts 2023-04-14 22:33:29 -07:00
part_search_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
part_search_timing_test.go lib/storage: skip missing tsids in the current block header by using binary search 2022-12-14 22:07:55 -08:00
part_search.go lib/storage: optimize partSearch.searchBHS() for common case when the TSID for the current block header is bigger or equal to the current tsid 2022-12-19 10:31:39 -08:00
part.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
partition_search_test.go lib/storage: replace OpenStorage() with MustOpenStorage() 2023-04-14 23:04:42 -07:00
partition_search.go lib/storage: fixes finalDedup for backfilled data (#3737) 2023-02-01 09:57:02 -08:00
partition_test.go lib/storage: remove unused code (#4050) 2023-04-02 21:23:24 -07:00
partition.go lib/storage: reduce the unimportant logging during Storage start / stop 2023-05-16 15:32:35 -07:00
raw_block.go
raw_row.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
search_test.go lib/storage: replace OpenStorage() with MustOpenStorage() 2023-04-14 23:04:42 -07:00
search.go app,lib: fix typos in comments (#3804) 2023-02-13 09:32:35 -08:00
storage_test.go lib/storage: follow-up after a50d63c376 (#4289) 2023-05-16 10:13:20 -07:00
storage_timing_test.go lib/storage: replace OpenStorage() with MustOpenStorage() 2023-04-14 23:04:42 -07:00
storage.go lib/storage: introduce per-day MetricName=>TSID index (#4252) 2023-05-16 23:18:11 -07:00
table_search_test.go lib/storage: replace OpenStorage() with MustOpenStorage() 2023-04-14 23:04:42 -07:00
table_search_timing_test.go lib/storage: replace OpenStorage() with MustOpenStorage() 2023-04-14 23:04:42 -07:00
table_search.go lib/storage: do not pass retentionMsecs and isReadOnly args explicitly - access them via Storage arg 2022-10-24 01:32:56 +03:00
table_test.go lib/storage: replace OpenStorage() with MustOpenStorage() 2023-04-14 23:04:42 -07:00
table_timing_test.go lib/storage: replace OpenStorage() with MustOpenStorage() 2023-04-14 23:04:42 -07:00
table.go lib/storage: replace OpenStorage() with MustOpenStorage() 2023-04-14 23:04:42 -07:00
tag_filters_test.go lib/regexutil: add Simplify() function for simplifying the regular expression 2022-08-26 11:57:43 +03:00
tag_filters_timing_test.go lib/regexutil: add Simplify() function for simplifying the regular expression 2022-08-26 11:57:43 +03:00
tag_filters.go lib/storage: optimize matching speed for non-trivial regexp filters 2022-10-01 12:07:18 +03:00
time_test.go
time.go app,lib: fix typos in comments (#3804) 2023-02-13 09:32:35 -08:00
tsid_test.go
tsid.go