VictoriaMetrics/lib/logstorage
Zakhar Bessarab bea3431ed1
lib/storage/partition: add check to ensure parts exist on disk (#5017)
* lib/storage/partition: add check to ensure parts exist on disk

If part exists in parts.json but is missing on disk there will be a misleading error similar to "unexpected number of substrings in the part name".

This change forces verification of part existence and throws a correct error in case it is missing on disk.

Such issue can be result of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5005 or disk corruption.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage/partition: use filepath.Join instead of string concatenation

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* lib/storage/partition: add action points for error message

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* all: add a check for missing part in lib/mergeset and lib/logstorage

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-09-19 11:17:41 +02:00
..
arena.go
block_data_test.go
block_data.go
block_header_test.go
block_header.go
block_search.go Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
block_stream_merger.go
block_stream_reader.go
block_stream_writer.go
block_test.go
block_timing_test.go
block.go
bloomfilter_test.go
bloomfilter.go
consts.go
datadb_test.go
datadb.go lib/storage/partition: add check to ensure parts exist on disk (#5017) 2023-09-19 11:17:41 +02:00
encoding_test.go lib/logstorage: fix make test-pure tests 2023-07-04 13:14:30 -07:00
encoding_timing_test.go
encoding.go
filenames.go
filters_test.go
filters.go Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
hash128_test.go
hash128_timing_test.go
hash128.go
index_block_header_test.go
index_block_header.go
indexdb_test.go
indexdb.go app/vlstorage: expose vl_data_size_bytes metric at /metrics page for tracking the on-disk data size (both indexdb and the data itself) 2023-07-31 07:56:53 -07:00
inmemory_part_test.go lib/logstorage: go fmt 2023-07-04 14:13:14 -07:00
inmemory_part_timing_test.go
inmemory_part.go
log_rows_timing_test.go
log_rows.go app/vlinsert/loki: follow-up after 09df5b66fd 2023-07-20 16:48:21 -07:00
parser_test.go lib/logstorage: go fmt after a8000b74c5 2023-07-18 16:04:51 -07:00
parser.go lib/logstorage: properly encode "offset" search word just after _time filter 2023-07-18 16:00:06 -07:00
part_header_test.go
part_header.go
part.go
partition_test.go
partition.go
rows_test.go
rows.go
storage_search_test.go
storage_search.go
storage_test.go Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:16:42 +02:00
storage.go lib/logstorage: eliminate data race when clearing s.ptwHot after deleting the corresponding partition 2023-08-29 11:09:55 +02:00
stream_filter.go
stream_id_test.go
stream_id.go
stream_tags.go
tenant_id_test.go app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 10:10:55 +02:00
tenant_id.go app/vlinsert/loki: follow-up after 09df5b66fd 2023-07-20 16:48:21 -07:00
tokenizer_test.go
tokenizer_timing_test.go
tokenizer.go
u128_test.go
u128.go
values_encoder_test.go lib/logstorage: fix TestValuesEncoder() on 32-bit architectures 2023-07-13 11:27:13 -07:00
values_encoder_timing_test.go
values_encoder.go