Aliaksandr Valialkin
c7dccebaef
lib/storage: optimize search by label filters matching big number of time series
2021-02-10 00:44:54 +02:00
Aliaksandr Valialkin
a4140de9e6
lib/mergeset: unconditionally cache indexdb blocks
...
Production workloads show that indexdb blocks must be cached unconditionally for reducing CPU usage.
This shouldn't increase memory usage too much, since unused blocks are removed from the cache every two minutes.
2021-02-09 00:47:50 +02:00
Aliaksandr Valialkin
157c02622b
app/vmselect: add ability to set Graphite-compatible filter via {__graphite__="foo.*.bar"}
syntax
2021-02-03 01:21:54 +02:00
Aliaksandr Valialkin
4aaee33860
lib/storage: do not show artifically created label for reverse Graphite labels at /api/v1/labels page
2020-11-16 00:44:35 +02:00
Aliaksandr Valialkin
b378cd6ed8
app/vmselect: optimize querying for /api/v1/labels
and /api/v1/label/<name>/values
when start
and end
args are set
2020-11-05 01:01:33 +02:00
Aliaksandr Valialkin
fe289331dd
lib/storage: remove obsolete code
2020-11-02 19:11:59 +02:00
faceair
d2960a20e0
evaluate the execution cost of all tag filters ( #824 )
...
* evaluate the execution cost of all tag filters
* fix suffixes typo
2020-10-17 00:46:55 +03:00
Aliaksandr Valialkin
764dc2499f
lib/storage: code cleanup after 10f2eedee0
...
Remove the code that uses metricIDs caches for the current and the previous hour during metricIDs search,
since this code became unused after implementing per-day inverted index almost a year ago.
While at it, fix a bug, which could prevent from finding time series with names containing dots (aka Graphite-like names
such as `foo.bar.baz`).
2020-10-01 19:06:23 +03:00
Aliaksandr Valialkin
10f2eedee0
lib/storage: imrpove cache effectiveness for time series ids matching the given filters
...
Previously the maximum cache lifetime has been limited by 10 seconds. Now it is extended up to a day.
This should reduce CPU usage in the following cases:
* when querying recently added data with small churn rate for time series
* when querying historical data
2020-10-01 14:38:25 +03:00
Aliaksandr Valialkin
a69234ed18
lib/storage: code prettifying after be5e1222f3
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/781
2020-09-22 00:36:45 +03:00
faceair
be5e1222f3
add filter to getMetricIDs ( #783 )
...
* add getMetricIDs filter
* check nil filter before use
2020-09-22 00:33:43 +03:00
Aliaksandr Valialkin
f6bc608e86
app/vmselect: initial implementation of Graphite Metrics API
...
See https://graphite-api.readthedocs.io/en/latest/api.html#the-metrics-api
2020-09-11 00:30:01 +03:00
Aliaksandr Valialkin
9043a509a3
lib/storage: properly check timeouts and pace limits
...
Previously they were checked on every iteration for small number of iterations
2020-08-07 08:40:37 +03:00
Aliaksandr Valialkin
8e44fba76d
lib/storage: reduce the frequency (and overhead) for timeout and pace limiter checks by 4x
2020-08-06 18:45:55 +03:00
Aliaksandr Valialkin
54ef2d8112
lib/storage: slightly reduce code difference between single-node and cluster versions
2020-07-24 00:31:16 +03:00
Aliaksandr Valialkin
039c9d2441
lib/storage: respect -search.maxQueryDuration
when searching for time series in inverted index
...
Previously the time spent on inverted index search could exceed the configured `-search.maxQueryDuration`.
This commit stops searching in inverted index on query timeout.
2020-07-23 21:21:42 +03:00
Aliaksandr Valialkin
2a45871823
lib/storage: add more fine-grained pace limiting for search
2020-07-23 19:26:08 +03:00
Aliaksandr Valialkin
d3442b40b2
lib/uint64set: optimize adding items to the set via Set.AddMulti
2020-07-21 20:56:59 +03:00
Aliaksandr Valialkin
e1107fec10
lib/storage: reset MetricName->TSID
cache after marking metricIDs as deleted
...
This is a follow-up commit after 12b16077c4
,
which didn't reset the `tsidCache` in all the required places.
This could result in indefinite errors like:
missing metricName by metricID ...; this could be the case after unclean shutdown; deleting the metricID, so it could be re-created next time
Fix this by resetting the cache inside deleteMetricIDs function.
2020-07-14 14:06:32 +03:00
Aliaksandr Valialkin
0b2086b7a5
app/vminsert: prevent from adding and/or selecting labels with empty values
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/600
2020-07-02 23:14:11 +03:00
Aliaksandr Valialkin
d5dddb0953
all: use %w instead of %s for wrapping errors in fmt.Errorf
...
This will simplify examining the returned errors such as httpserver.ErrorWithStatusCode .
See https://blog.golang.org/go1.13-errors for details.
2020-06-30 23:05:11 +03:00
Aliaksandr Valialkin
ae1cc0fc4b
lib/storage: properly match {tag!="|foo"}
filters
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/546
2020-06-10 19:35:56 +03:00
Aliaksandr Valialkin
d186472081
lib/storage: improve search speed for time series matching Graphite whildcards such as foo.*.bar.baz
...
Add index for reverse Graphite-like metric names with dots. Use this index during search for filters
like `__name__=~"foo\\.[^.]*\\.bar\\.baz"` which end with non-empty suffix with dots, i.e. `.bar.baz` in this case.
This change may "hide" historical time series during queries. The workaround is to add `[.]*` to the end of regexp label filter,
i.e. "foo\\.[^.]*\\.bar\\.baz" should be substituted with "foo\\.[^.]*\\.bar\\.baz[.]*".
2020-05-27 21:45:52 +03:00
Aliaksandr Valialkin
4fc33163c4
lib/storage: optimize ingestion pefrormance for new time series
2020-05-15 13:24:37 +03:00
Aliaksandr Valialkin
0afd48d2ee
lib: extract common code for returning fast unix timestamp into lib/fasttime
2020-05-14 23:02:07 +03:00
Aliaksandr Valialkin
827a3a7866
lib/storage: document that getnerateUniqueMetricID should return dense ids
2020-05-14 14:08:45 +03:00
Aliaksandr Valialkin
cc00a2c453
lib/storage: typo fixes in error messages: or -> of
2020-05-12 12:12:42 +03:00
Aliaksandr Valialkin
83aca79137
lib/storage: recover when metricID->metricName entry is missing in the inverted index after unclean shutdown
...
Newly added index entries can be missing after unclean shutdown, since they didn't flush to persistent storage yet.
Log about this and delete the corresponding metricID, so it could be re-created next time.
2020-04-28 12:00:33 +03:00
Aliaksandr Valialkin
491b31b369
lib/storage: postpone label filters matching too many time series instead of giving up with error
...
This should reduce the frequency of the following errors:
cannot find tag filter matching less than N time series; either increase -search.maxUniqueTimeseries or use more specific tag filters
more than N time series found on the time range [...]; either increase -search.maxUniqueTimeseries or shrink the time range
2020-04-24 21:13:50 +03:00
Aliaksandr Valialkin
364db13c9c
app/vmselect: add /api/v1/status/tsdb
page with useful stats for locating root cause for high cardinality issues
...
See https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-stats
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/425
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/268
2020-04-22 22:03:43 +03:00
Aliaksandr Valialkin
c1de3f67b4
lib/storage: skip metricID if the corresponding metricID->metricName is missing in inverted index during search
...
This case is possible when the corresponding metricID->metricName entry didn't propagate to inverted index yet.
This should fix the following error:
error when searching tsids for tfss [...]: cannot find metricName by metricID 1582417212213420669: EOF
2020-04-15 00:06:43 +03:00
Aliaksandr Valialkin
e0d0348f36
lib/storage: add missing reset for tagFilter.matchesEmptyValue on tagFilter.Init
2020-04-01 17:42:44 +03:00
Aliaksandr Valialkin
972713bd79
lib/storage: add fast path for the previous indexdb search if it doesn't contain per-day inverted index yet
2020-03-31 12:51:21 +03:00
Aliaksandr Valialkin
5d99ca6cfc
lib/storage: optimize per-day inverted index search for tag filters matching big number of time series
...
- Sort tag filters in the ascending number of matching time series
in order to apply the most specific filters first.
- Fall back to metricName search for filters matching big number of time series
(usually this are negative filters or regexp filters).
2020-03-31 00:48:35 +03:00
Aliaksandr Valialkin
df91d2d91f
lib/storage: remove obsolete code
2020-03-13 22:48:17 +02:00
Aliaksandr Valialkin
f9289b804a
lib/storage: reduce memory allocations when merging metricID sets
2020-01-17 22:10:44 +02:00
Aliaksandr Valialkin
a247236f61
lib/storage: fall back to global inverted index if a filter match too many time series in per-day index
...
Previously this resulted to error message. The query may succeed via search in global index.
2019-12-03 14:48:31 +02:00
Aliaksandr Valialkin
f52874dab4
lib/storage: optimize regexp filter search
2019-12-03 00:43:12 +02:00
Aliaksandr Valialkin
20812008a7
lib/storage: remove metricID with missing metricID->metricName entry
...
The metricID->metricName entry can be missing in the indexdb after unclean shutdown
when only a part of entries for new time series is written into indexdb.
Recover from such a situation by removing the broken metricID. New metricID
will be automatically created for time series with the given metricName
when new data point will arive to it.
2019-12-02 20:46:44 +02:00
Aliaksandr Valialkin
da98703748
app/vmselect/promql: optimize binary search over big number of samples during rollup calculations
2019-11-25 14:01:46 +02:00
Aliaksandr Valialkin
f652c0f40f
lib/storage: move non-matching tag filters to the top at matchTagFilters
...
This should reduce the amount of useless work needed for matching the next metricNames.
2019-11-21 21:35:13 +02:00
Aliaksandr Valialkin
b8cde6cce1
lib/storage: speed up time series search for queries with multiple filters
...
Use optimized specialized binary search for uint64 metricIDs instead of generic sort.Search.
2019-11-21 18:43:17 +02:00
Aliaksandr Valialkin
2ab4cea5e5
lib/storage: always start using per-day inverted index on the next day after its creation
...
The current day could miss entries for already stopped time series before
enabling per-day index.
This fixes the issue when queries return empty results during the first hour after
upgrading to v1.29.*
2019-11-16 12:11:25 +02:00
Aliaksandr Valialkin
86a1cd700b
lib/storage: remove inmemory index for recent hour, since it uses too much memory
...
Production workload shows that the index requires ~4Kb of RAM per active time series.
This is too much for high number of active time series, so let's delete this index.
Now the queries should fall back to the index for the current day instead of the index
for the recent hour. The query performance for the current day index should be good enough
given the 100M rows/sec scan speed per CPU core.
2019-11-13 17:58:07 +02:00
Aliaksandr Valialkin
33895d4a0f
lib/storage: add missing increment for recentHourInvertedIndexSearchCalls
2019-11-13 15:13:51 +02:00
Aliaksandr Valialkin
c57eb0ff83
lib/storage: add -disableRecentHourIndex
flag for disabling inmemory index for recent hour
...
This may be useful for saving RAM on high number of time series aka high cardinality
2019-11-13 15:02:51 +02:00
Aliaksandr Valialkin
ca259864e2
lib/storage: return back inmemory inverted index for recent hour
...
Issues fixed:
- Slow startup times. Now the index is loaded from cache during start.
- High memory usage related to superflouos index copies every 10 seconds.
2019-11-13 13:11:04 +02:00
Aliaksandr Valialkin
01bb3c06c7
lib/storage: remove inmemory inverted index for recent hours
...
Production load with >10M active time series showed it could
slow down VictoriaMetrics startup times and could eat
all the memory leading to OOM.
Remove inmemory inverted index for recent hours until thorough
testing on production data shows it works OK.
2019-11-13 10:45:53 +02:00
Aliaksandr Valialkin
8e8f98f712
lib/storage: add tests for dateMetricIDCache
2019-11-11 13:21:57 +02:00
Aliaksandr Valialkin
3956003dd0
lib/storage: reorganize the code in getStartDateForPerDayInvertedIndex according to golangci-lint
2019-11-10 00:38:59 +02:00
Aliaksandr Valialkin
ee7765b10d
lib/storage: implement per-day inverted index
2019-11-10 00:02:46 +02:00
Aliaksandr Valialkin
5810ba57c2
lib/storage: use specialized cache for (date, metricID) entries
...
This improves ingestion performance.
2019-11-09 23:06:11 +02:00
Aliaksandr Valialkin
e573ef2126
lib/storage: remove unused code from getMetricIDsForTimeRange: it is expected that time range is always non-zero
2019-11-09 19:03:34 +02:00
Aliaksandr Valialkin
823fa085ef
lib/storage: properly set time range when deleting time series
2019-11-09 18:49:49 +02:00
Aliaksandr Valialkin
695c1dc5eb
lib/storage: obtain all the time series ids from (tag->metricIDs) rows instead of (metricID->TSID) rows, since this much faster
2019-11-09 18:04:33 +02:00
Aliaksandr Valialkin
cdbe848102
lib/storage: small code prettifying
2019-11-09 14:19:52 +02:00
Aliaksandr Valialkin
6ad7fe8eeb
lib/storage: export vm_new_timeseries_created_total
metric for determining time series churn rate
2019-11-08 21:21:07 +02:00
Aliaksandr Valialkin
d888b21657
lib/storage: add inmemory inverted index for the last hour
...
It should improve performance for `last N hours` dashboards with update intervals smaller than 1 hour.
2019-11-08 21:21:07 +02:00
Aliaksandr Valialkin
e472f0b23b
lib/storage: substitute error message about unsorted items in the index block after metricIDs merge with counter
...
The origin of the error has been detected and documented in the code,
so it is enough to export a counter for such errors at `vm_index_blocks_with_metric_ids_incorrect_order_total`,
so it could be monitored and alerted on high error rates.
Export also the counter for processed index blocks with metricIDs - `vm_index_blocks_with_metric_ids_processed_total`,
so its' rate could be compared to `rate(vm_index_blocks_with_metric_ids_incorrect_order_total)`.
2019-11-06 14:28:11 +02:00
Aliaksandr Valialkin
c51ca04a43
lib/storage: take into account the requested time range when caching TSIDs for the given tag filters
2019-11-06 14:28:11 +02:00
Aliaksandr Valialkin
e37f06dc52
lib/storage: dump incorrectly sorted items on a single line; this should simplify error reporting
2019-11-05 18:44:22 +02:00
Aliaksandr Valialkin
885ba17905
lib/storage: separate the max inverted index scan loops per metric into fast and slow loops
...
Slow loops could require seeks and expensive regexp matching, while fast loops just scans
all the metricIDs for the given `tag=value` prefix. So these operations must have separate
max loops multiplier.
2019-11-05 17:27:48 +02:00
Aliaksandr Valialkin
b9a06e8e74
lib/storage: skip repeated useless work when intersection of metricIDs with the given filter is too expensive
...
This should improve performance for query filters over big number of time series.
2019-11-05 14:19:13 +02:00
Aliaksandr Valialkin
30c8301b11
lib/storage: reduce the maximum inverted index scans before giving up to label filters matching by metric name
...
The new value reduces the amount of wasted work during index scans over big number of time series.
2019-11-05 14:19:06 +02:00
Aliaksandr Valialkin
e53f9e553d
lib/storage: try potentially faster tag filters at first, then apply slower tag filters
...
The fastest tag filters are non-negative non-regexp, since they are the most specific.
The slowest tag filters are negative regexp, since they require scanning
all the entries for the given label.
2019-11-05 14:19:01 +02:00
Aliaksandr Valialkin
02e0b19a62
lib/storage: tune the returned value from adjustMaxMetricsAdaptive
2019-11-04 00:44:37 +02:00
Aliaksandr Valialkin
6be4456d88
lib/{storage,uint64set}: add Set.Union() function and use it
2019-11-04 00:44:37 +02:00
Aliaksandr Valialkin
97ce4e03a5
all: add support for GOARCH=386 and fix all the issues related to 32-bit architectures such as GOARCH=arm
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/212
2019-10-17 18:23:23 +03:00
Aliaksandr Valialkin
f6334bffa1
lib/storage: harden the check that the original items are sorted after mergeTagToMetricIDsRows fails to preserve sort order
2019-10-09 12:13:17 +03:00
Aliaksandr Valialkin
c1cf7d9f93
lib/storage: add tests for mergeTagToMetricIDsRows and return the original items if the function breaks items` ordering.
...
This should save from data corruption issues revealed in the previous releases up to v1.28.0-beta5.
2019-10-08 16:27:35 +03:00
Aliaksandr Valialkin
c39355921e
lib/storage: verify whether items are sorted in the end of call to mergeTagToMetricIDsRows
...
This should prevent from inverted index corruption if bug in mergeTagToMetricIDsRows is discovered.
2019-09-26 13:13:41 +03:00
Aliaksandr Valialkin
2444433d83
lib/storage: add missing break in removeDuplicateMetricIDs
2019-09-25 18:23:43 +03:00
Aliaksandr Valialkin
ea4c828bae
lib/storage: remove duplicate MetricIDs in tag->metricIDs
items before writing them into inverted index
2019-09-25 17:55:13 +03:00
Aliaksandr Valialkin
aebc45ad26
lib/{mergeset,storage}: do not cache inverted index blocks containing tag->metricIDs
items
...
This should reduce the amounts of used RAM during queries with filters over big number of time series.
2019-09-25 14:02:15 +03:00
Aliaksandr Valialkin
b986516fbe
lib/storage: create and use lib/uint64set
instead of map[uint64]struct{}
...
This should improve inverted index search performance for filters matching big number of time series,
since `lib/uint64set.Set` is faster than `map[uint64]struct{}` for both `Add` and `Has` calls.
See the corresponding benchmarks in `lib/uint64set`.
2019-09-24 21:17:55 +03:00
Aliaksandr Valialkin
ef2296e420
lib/storage: typo fix: return dstData instead of data from mergeTagToMetricIDsRows
2019-09-24 19:32:34 +03:00
Aliaksandr Valialkin
a6086cde78
lib/storage: limit the number of metricIDs in tag->metricIDs row
...
This reduces the overhead on index and metaindex in lib/mergeset
2019-09-24 00:49:51 +03:00
Aliaksandr Valialkin
c9063ece66
lib/storage: share tsids across all the partSearch instances
...
This should reduce memory usage when big number of time series matches the given query.
2019-09-23 22:35:15 +03:00
Aliaksandr Valialkin
4e26ad869b
lib/{storage,mergeset}: verify PrepareBlock callback results
...
Do not touch the first and the last item passed to PrepareBlock
in order to preserve sort order of mergeset blocks.
2019-09-23 20:43:13 +03:00
Aliaksandr Valialkin
0adebae1f8
lib/storage: generate the first tag->metricIDs item in a mergeset block with a single metricID
...
The first item from each mergeset block goes into index (lib/mergeset.blockHeader),
so it must be short in order to reduce index size.
2019-09-22 19:21:33 +03:00
Aliaksandr Valialkin
0686ac52c3
lib/{storage,mergeset}: merge tag->metricID
rows into tag->metricIDs
rows for common tag
values
...
This should improve lookup performance if the same `label=value` pair exists
in big number of time series.
This should also reduce memory usage for mergeset data cache, since `tag->metricIDs` rows
occupy less space than the original `tag->metricID` rows.
2019-09-20 22:06:41 +03:00
Aliaksandr Valialkin
a544f49c2b
lib/storage: optimize selecting all the metricIDs by scanning MetricID->TSID entries instead of tag->MetricID entries
...
The number of MetricID->TSID entries is smaller than the number of tag->MetricID entries
and MetricID->TSID entries are usually shorter than tag->MetricID entries.
This should improve performance when selecting all the metricIDs.
2019-09-20 11:54:10 +03:00
Aliaksandr Valialkin
a84fe76677
lib/storage: use sort.Sort instead of sort.slice in getSortedMetricIDs
2019-09-19 20:07:22 +03:00
Aliaksandr Valialkin
3a697a935a
lib/storage: skip duplicate call to intersectMetricIDsWithTagFilter on zero successful intersects
2019-09-19 17:49:56 +03:00
Aliaksandr Valialkin
3d83f5d334
lib/storage: mark tag filter returning errFallbackToMetricNameMatch as useless
...
This will save CPU on subsequent calls for this filter
2019-09-18 19:10:32 +03:00
Aliaksandr Valialkin
8d35718dc6
lib/storage: properly construct keys for uselessTagFiltersCache and register useless negative tag filters there
2019-09-17 23:20:27 +03:00
Aliaksandr Valialkin
e1d76ec1f3
lib/storage: invalidate tagFilters -> TSIDS
cache when newly added index data becomes visible to search
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/163
2019-08-29 15:08:35 +03:00
Aliaksandr Valialkin
bf915fc0db
lib/storage: report proper maxMetrics limit when more than -search.maxUniqueTimeseries series match the given filters
2019-08-27 14:21:42 +03:00
Aliaksandr Valialkin
4f738c8a15
lib/storage: try slower path for searching the tag filter with the minimum number of matching time series before giving up with increase -search.maxUniqueTimeseries
error
2019-08-19 16:04:21 +03:00
Aliaksandr Valialkin
5b41122292
lib/storage: properly cache tagFilters -> TSIDs entries from historical index
2019-08-14 02:29:58 +03:00
Aliaksandr Valialkin
964c296f96
lib/storage: compress contents of cache for tagFilters -> TSIDs
...
This should increase cache capacity
2019-08-14 02:29:52 +03:00
Aliaksandr Valialkin
09fc6e22e5
all: use workingsetcache instead of fastcache
...
This should reduce the amount of RAM required for processing time series
with non-zero churn rate.
The previous cache behavior can be restored with `-cache.oldBehavior` command-line flag.
2019-08-13 21:39:34 +03:00
Aliaksandr Valialkin
a77e88db7d
lib/storage: fix matching against tag filter with empty name
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/137
2019-07-30 15:15:09 +03:00
Aliaksandr Valialkin
e75d5f47c4
lib/storage: remove unused function isTooBigTimeRangeForDateMetricIDs
2019-07-12 02:28:23 +03:00
Aliaksandr Valialkin
fc90ebf43c
lib/storage: do not reduce maxMetrics
on time ranges exceeding maxDaysForDateMetricIDs
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/95
2019-07-12 02:20:34 +03:00
Aliaksandr Valialkin
43ea4ce428
lib/storage: remember and skip individual tag filters matching too many metrics
...
This saves CPU time by skipping useless matching for individual tag filters.
2019-07-11 14:48:30 +03:00
Aliaksandr Valialkin
1fe6d784d8
all: consistency renaming: bytesSize -> sizeBytes
2019-07-10 00:47:36 +03:00
Aliaksandr Valialkin
2ecb117082
lib/storage: skip non-matching metricIDs in sortedFilter
...
This should improve performance for big sorteFilter lists.
2019-06-29 13:48:32 +03:00
Aliaksandr Valialkin
c1be1e4342
lib/storage: optimize time series search by regexp filter
...
This should improve search speed on label filters like `{foo=~"bar.+baz"}`
2019-06-27 16:17:43 +03:00
Aliaksandr Valialkin
683bf2a11f
lib/storage: make sure non-nil args are passed to openIndexDB
2019-06-25 20:10:04 +03:00
Aliaksandr Valialkin
eb2283a029
lib/storage: reduce too big maxMetrics in getTagFilterWithMinMetricIDsCountAdaptive
...
This should improve performance on inverted index search for big amount of unique time series
when big -search.maxUniqueTimeseries is set.
2019-06-25 19:55:27 +03:00
Aliaksandr Valialkin
e8377011ab
lib/storage: free up memory from caches owned by indexDB when it is deleted
2019-06-25 14:42:44 +03:00
Aliaksandr Valialkin
33ea2120c3
lib/storage: use unversioned keys for tag cache in extDB
...
Data in ExtDB cannot be changed, so it is OK to use unversioned keys for tag cache.
This should improve performance for index lookups over big amount of time series.
2019-06-25 13:08:58 +03:00
Aliaksandr Valialkin
cf63669303
lib/storage: skip searching in extDB if it doesn't contain items for the given time range
...
This should improve inverted index search performance for big amount
of unique time series when the search is performed only on recent data.
2019-06-25 13:00:37 +03:00
Aliaksandr Valialkin
80db24386e
lib/storage: typo fixes found by golangci-lint; updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/69
2019-06-20 14:37:55 +03:00
Aliaksandr Valialkin
419197ba08
lib/fs: consolidate *RemoveAll* funcs into a single MustRemoveAll func
...
The func syncs parent dir in order to persist directory removal
in the event of power loss
2019-06-12 01:53:46 +03:00
Aliaksandr Valialkin
ac7b186f13
all: try hard removing directory with contents
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
2019-06-11 01:57:59 +03:00
Aliaksandr Valialkin
7b6623558f
lib/storage: skip adaptive searching for tag filter matching the minimum number of metrics if the identical previous search didn't found such filter
...
This should improve speed for searching metrics among high number of time series
with high churn rate like in big Kubernetes clusters with frequent deployments.
2019-06-10 14:07:39 +03:00
Aliaksandr Valialkin
a1351bbaee
lib/storage: factor out getTagFilterWithMinMetricIDsCountAdaptive from updateMetricIDsForTagFilters
2019-06-10 13:26:44 +03:00
Aliaksandr Valialkin
b4d707d9bb
lib/storage: give clearer names to more functions
2019-06-10 13:01:23 +03:00
Aliaksandr Valialkin
bee7298f81
lib/storage: give more clear names to functions
2019-06-10 12:50:44 +03:00
Aliaksandr Valialkin
4d936b1524
lib/storage: make getSeriesCount func indexSearch method
2019-06-10 12:29:11 +03:00
Aliaksandr Valialkin
d37924900b
lib/storage: optimize time series lookup for recent hours when the db contains many millions of time series with high churn rate (aka frequent deployments in Kubernetes)
2019-06-09 19:13:56 +03:00
Aliaksandr Valialkin
54fb8b21f9
all: fix misspellings
2019-05-25 21:51:11 +03:00
Aliaksandr Valialkin
1836c415e6
all: open-sourcing single-node version
2019-05-23 00:18:06 +03:00