Artem Fetishev 85bf768013 lib/storage: adds metrics that count records that failed to insert ... ### Describe Your Changes Add storage metrics that count records that failed to insert: - `RowsReceivedTotal`: the number of records that have been received by the storage from the clients - `RowsAddedTotal`: the number of records that have actually been persisted. This value must be equal to `RowsReceivedTotal` if all the records have been valid ones. But it will be smaller otherwise. The values of the metrics below should provide the insight of why some records hasn't been added - `NaNValueRows`: the number of records whose value was `NaN` - `StaleNaNValueRows`: the number of records whose value was `Stale NaN` - `InvalidRawMetricNames`: the number of records whose raw metric name has failed to unmarshal. The following metrics existed before this PR and are listed here for completeness: - `TooSmallTimestampRows`: the number of records whose timestamp is negative or is older than retention period - `TooBigTimestampRows`: the number of records whose timestamp is too far in the future. - `HourlySeriesLimitRowsDropped`: the number of records that have not been added because the hourly series limit has been exceeded. - `DailySeriesLimitRowsDropped`: the number of records that have not been added because the daily series limit has been exceeded. --- Signed-off-by: Artem Fetishev <wwctrsrx@gmail.com>		2024-09-06 18:13:48 +02:00
..
block_header_test.go	lib/storage: typo fix: umarshal -> unmarshal	2021-03-02 20:48:44 +02:00
block_header.go	lib/encoding: optimize UnmarshalVarUint64, UnmarshalVarInt64 and UnmarshalBytes a bit	2024-05-14 01:30:25 +02:00
block_stream_merger.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:23:26 +02:00
block_stream_reader_test.go	Makefile: update golangci-lint from v1.51.2 to v1.54.2	2023-09-01 10:25:49 +02:00
block_stream_reader_timing_test.go	lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart	2023-04-14 15:47:20 -07:00
block_stream_reader.go	lib/storage: avoid superflouos copy of block header data	2024-02-23 01:11:31 +02:00
block_stream_writer_timing_test.go	lib/storage: consistently use atomic.* types instead of atomic.* function calls on ordinary types	2024-02-24 00:33:07 +02:00
block_stream_writer.go	lib/storage: consistently use atomic.* types instead of atomic.* function calls on ordinary types	2024-02-24 00:33:07 +02:00
block_test.go	all: consistently use %w instead of %s in when error is passed to fmt.Errorf()	2023-10-26 09:44:40 +02:00
block.go	lib/encoding: optimize UnmarshalVarUint64, UnmarshalVarInt64 and UnmarshalBytes a bit	2024-05-14 01:30:25 +02:00
dedup_test.go	lib/storage: follow-up after 7c0ae3a86a	2022-12-08 18:18:49 -08:00
dedup_timing_test.go	lib/storage: follow-up after 7c0ae3a86a	2022-12-08 18:18:49 -08:00
dedup.go	docs: mention staleNaN handling during deduplication	2024-01-16 20:11:45 +02:00
filenames.go	lib/storage: consistently use OS-independent separator in file paths	2023-03-25 14:34:36 -07:00
index_db_test.go	Fix inconsistent error handling in Storage.AddRows() (#6583)	2024-07-17 12:55:07 +02:00
index_db_timing_test.go	lib/storage: replace the remaining atomic.* functions with atomic.* types for the sake of consistency	2024-02-24 00:51:03 +02:00
index_db.go	lib/storage: fix metric vm_object_references{type="indexdb"} (#6937)	2024-09-05 16:57:48 +02:00
inmemory_part_test.go	lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart	2023-04-14 15:47:20 -07:00
inmemory_part_timing_test.go	lib/storage: use deterministic random generator in tests	2023-01-23 20:12:32 -08:00
inmemory_part.go	lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart	2023-04-14 15:47:20 -07:00
merge_test.go	lib/storage: consistently use atomic.* types instead of atomic.* function calls on ordinary types	2024-02-24 00:33:07 +02:00
merge_timing_test.go	lib/storage: consistently use atomic.* types instead of atomic.* function calls on ordinary types	2024-02-24 00:33:07 +02:00
merge.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:23:26 +02:00
metaindex_row_test.go	lib/storage: correctly use maxBlockSize in various checks	2020-09-24 18:13:15 +03:00
metaindex_row.go	all: subsitute ioutil.ReadAll with io.ReadAll	2022-08-22 00:16:04 +03:00
metric_name_test.go	app/vminsert: add support for data ingestion via other vminsert nodes	2021-05-08 19:53:45 +03:00
metric_name.go	lib/storage: change default value for maxLabelValueLen to 1024 (#6313)	2024-05-22 21:55:21 +02:00
part_header.go	lib/storage: add ability to use downsampling for the given series filter (#733)	2024-04-03 02:38:37 +03:00
part_search_test.go	lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart	2023-04-14 15:47:20 -07:00
part_search_timing_test.go	all: consistently use %w instead of %s in when error is passed to fmt.Errorf()	2023-10-26 09:44:40 +02:00
part_search.go	lib/storage: optimize partSearch.searchBHS() for common case when the TSID for the current block header is bigger or equal to the current tsid	2022-12-19 10:31:39 -08:00
part.go	lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart	2023-04-14 15:47:20 -07:00
partition_search_test.go	lib/storage: do not keep rawRows buffer across flush() calls	2024-02-23 01:06:09 +02:00
partition_search.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:23:26 +02:00
partition_test.go	lib/storage: stop exposing vm_merge_need_free_disk_space metric	2023-09-25 17:00:14 +02:00
partition.go	Revert "refactor(vmstorage): Refactor the code to reduce the time complexity of MustAddRows and improve readability (#6629)"	2024-07-25 14:43:00 +02:00
raw_block.go	all: open-sourcing single-node version	2019-05-23 00:18:06 +03:00
raw_row.go	lib/storage: consistently use atomic.* types instead of atomic.* function calls on ordinary types	2024-02-24 00:33:07 +02:00
search_test.go	Fix inconsistent error handling in Storage.AddRows() (#6583)	2024-07-17 12:55:07 +02:00
search.go	lib/storage: fix SearchQuery.Unmarshal() after 32193b6059	2024-05-14 01:39:01 +02:00
storage_test.go	lib/storage: adds metrics that count records that failed to insert	2024-09-06 18:13:48 +02:00
storage_timing_test.go	Fix inconsistent error handling in Storage.AddRows() (#6583)	2024-07-17 12:55:07 +02:00
storage.go	lib/storage: adds metrics that count records that failed to insert	2024-09-06 18:13:48 +02:00
table_search_test.go	lib/storage: properly free up resources from newTestStorage() by calling stopTestStorage()	2023-07-13 17:13:34 -07:00
table_search_timing_test.go	lib/storage: replace the remaining atomic.* functions with atomic.* types for the sake of consistency	2024-02-24 00:51:03 +02:00
table_search.go	all: consistently use 'any' instead of 'interface{}'	2024-07-10 00:23:26 +02:00
table_test.go	flagutil: Make .Msecs private (#4906)	2023-09-03 10:37:57 +02:00
table_timing_test.go	lib/storage: properly free up resources from newTestStorage() by calling stopTestStorage()	2023-07-13 17:13:34 -07:00
table.go	Revert "refactor(vmstorage): Refactor the code to reduce the time complexity of MustAddRows and improve readability (#6629)"	2024-07-25 14:43:00 +02:00
tag_filters_test.go	lib/logstorage: work-in-progress	2024-05-24 03:07:07 +02:00
tag_filters_timing_test.go	all: fix golangci-lint(revive) warnings after 0c0ed61ce7	2024-04-03 03:00:45 +03:00
tag_filters.go	lib/logstorage: work-in-progress	2024-05-24 03:07:07 +02:00
time_test.go	all: open-sourcing single-node version	2019-05-23 00:18:06 +03:00
time.go	app,lib: fix typos in comments (#3804)	2023-02-13 09:32:35 -08:00
tsid_test.go	all: open-sourcing cluster version	2019-05-23 00:25:38 +03:00
tsid.go	lib/storage: optimize TSID comparison	2019-09-26 14:20:02 +03:00