Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-20 23:46:23 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
3,034 Commits 114 Branches 748 Tags 625 MiB
d05cac6c98
Commit Graph

3034 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Aliaksandr Valialkin
4ba86f501a vendor: update github.com/VictoriaMetrics/metrics from v1.7.1 to v1.7.2 2019-10-06 11:20:45 +03:00
Aliaksandr Valialkin
fdc5cfd838 lib/mergeset: reduce the maximum number of cached blocks, since there are reports on OOMs due to too big caches 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/189
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/195
 2019-09-30 12:25:40 +03:00
Artem Navoiev
a116f5e7c1 Add regression test for query apo (#194) 
Part of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/187
cover:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/184
 2019-09-30 11:25:54 +03:00
Aliaksandr Valialkin
4e9e1ca0f7 app/vmselect/netstorage: hint the OS that tmpBlocksFile is read almost sequentially 
This became the case after b7ee2e7af2 .
 2019-09-30 00:11:14 +03:00
Aliaksandr Valialkin
c1d3705be0 app/vmselect/netstorage: marshal block outside tmpBlocksFile.WriteBlock 
This allows re-using the destination buffer for marshaling in the outer loop.
 2019-09-28 21:07:13 +03:00
Aliaksandr Valialkin
b7ee2e7af2 app/vmselect/netstorage: reduce the number of disk seeks when the query processes big number of time series 2019-09-28 21:07:09 +03:00
Aliaksandr Valialkin
67d44b0845 app/vmselect/promql: do not generate timestamps for NaN values in timestamp function according to Prometheus logic 2019-09-27 18:54:43 +03:00
Artem Navoiev
1e6ae9eff4 Add regression test for duplicated labels and series 
Part of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/187
cover:
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/155
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/172
 2019-09-27 16:52:16 +03:00
Aliaksandr Valialkin
fa81f82714 deployment/docker: switch Go builder image from v1.13.0 to v1.13.1 2019-09-26 17:09:40 +03:00
Aliaksandr Valialkin
0fa6df94a2 lib/storage: optimize TSID comparison 2019-09-26 14:16:02 +03:00
Aliaksandr Valialkin
c39355921e lib/storage: verify whether items are sorted in the end of call to mergeTagToMetricIDsRows 
This should prevent from inverted index corruption if bug in mergeTagToMetricIDsRows is discovered.
 2019-09-26 13:13:41 +03:00
Artem Navoiev
cf4786f34a add test for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/161 2019-09-26 12:45:19 +03:00
Aliaksandr Valialkin
3e67862676 README.md: typo fix 2019-09-26 11:03:14 +03:00
Aliaksandr Valialkin
0db9fcedd5 lib/storage: properly match labels against regexp with (?i) flag 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/161
 2019-09-26 11:03:10 +03:00
Aliaksandr Valialkin
391530bb74 README.md: mention recommended ext4 options for mkfs.ext4 when creating multi-TB partition 2019-09-25 23:52:43 +03:00
Aliaksandr Valialkin
60c5b368bc README.md: tiny updates 2019-09-25 23:29:55 +03:00
Aliaksandr Valialkin
26dc21cf64 app/vmselect/promql: add increases_over_time and decreases_over_time functions 
`increases_over_time(q[d])` returns the number of `q` increases during the given duration `d`.
`decreases_over_time(q[d])` returns the number of `q` decreases during the given duration `d`.
 2019-09-25 20:38:44 +03:00
Aliaksandr Valialkin
2444433d83 lib/storage: add missing break in removeDuplicateMetricIDs 2019-09-25 18:23:43 +03:00
Aliaksandr Valialkin
ea4c828bae lib/storage: remove duplicate MetricIDs in tag->metricIDs items before writing them into inverted index 2019-09-25 17:55:13 +03:00
Aliaksandr Valialkin
aebc45ad26 lib/{mergeset,storage}: do not cache inverted index blocks containing tag->metricIDs items 
This should reduce the amounts of used RAM during queries with filters over big number of time series.
 2019-09-25 14:02:15 +03:00
Aliaksandr Valialkin
2cb811b42f lib/uint64set: optimize Set.AppendTo 2019-09-25 00:34:17 +03:00
Aliaksandr Valialkin
b986516fbe lib/storage: create and use lib/uint64set instead of map[uint64]struct{} 
This should improve inverted index search performance for filters matching big number of time series,
since `lib/uint64set.Set` is faster than `map[uint64]struct{}` for both `Add` and `Has` calls.
See the corresponding benchmarks in `lib/uint64set`.
 2019-09-24 21:17:55 +03:00
Aliaksandr Valialkin
ef2296e420 lib/storage: typo fix: return dstData instead of data from mergeTagToMetricIDsRows 2019-09-24 19:32:34 +03:00
Aliaksandr Valialkin
a6086cde78 lib/storage: limit the number of metricIDs in tag->metricIDs row 
This reduces the overhead on index and metaindex in lib/mergeset
 2019-09-24 00:49:51 +03:00
Aliaksandr Valialkin
c9063ece66 lib/storage: share tsids across all the partSearch instances 
This should reduce memory usage when big number of time series matches the given query.
 2019-09-23 22:35:15 +03:00
Aliaksandr Valialkin
4e26ad869b lib/{storage,mergeset}: verify PrepareBlock callback results 
Do not touch the first and the last item passed to PrepareBlock
in order to preserve sort order of mergeset blocks.
 2019-09-23 20:43:13 +03:00
Aliaksandr Valialkin
0772191975 lib/mergeset: detect whether we are in test by executable suffix 2019-09-22 23:12:15 +03:00
Aliaksandr Valialkin
48999e5396 lib/workingsetcache: remove data race when resetting c.misses 2019-09-22 19:36:49 +03:00
Aliaksandr Valialkin
0adebae1f8 lib/storage: generate the first tag->metricIDs item in a mergeset block with a single metricID 
The first item from each mergeset block goes into index (lib/mergeset.blockHeader),
so it must be short in order to reduce index size.
 2019-09-22 19:21:33 +03:00
Aliaksandr Valialkin
267efde5ae README.md: update troubleshooting and tuning sections according to recent questions from our users 2019-09-22 19:12:24 +03:00
Aliaksandr Valialkin
0686ac52c3 lib/{storage,mergeset}: merge tag->metricID rows into tag->metricIDs rows for common tag values 
This should improve lookup performance if the same `label=value` pair exists
in big number of time series.
This should also reduce memory usage for mergeset data cache, since `tag->metricIDs` rows
occupy less space than the original `tag->metricID` rows.
 2019-09-20 22:06:41 +03:00
Aliaksandr Valialkin
68722c3c74 lib/encoding: optimize UnmarshalUint* and UnmarshalInt* 2019-09-20 13:08:16 +03:00
Aliaksandr Valialkin
a544f49c2b lib/storage: optimize selecting all the metricIDs by scanning MetricID->TSID entries instead of tag->MetricID entries 
The number of MetricID->TSID entries is smaller than the number of tag->MetricID entries
and MetricID->TSID entries are usually shorter than tag->MetricID entries.
This should improve performance when selecting all the metricIDs.
 2019-09-20 11:54:10 +03:00
Aliaksandr Valialkin
d32f88c378 app/vminsert/opentsdbhttp: remove FATAL prefix from logger.Fatalf errors for the sake of consistency with other logger.Fatalf calls 2019-09-19 22:15:59 +03:00
Aliaksandr Valialkin
00cfb2d2b9 lib/mergeset: rename misleading mergeSmallParts to mergeExistingParts 2019-09-19 21:48:20 +03:00
Aliaksandr Valialkin
37dc223e25 lib/mergeset: use sort.IsSorted instead of sort.SliceIsSorted in inmemoryBlock.isSorted in order to reduce memory allocations 2019-09-19 20:13:08 +03:00
Aliaksandr Valialkin
a84fe76677 lib/storage: use sort.Sort instead of sort.slice in getSortedMetricIDs 2019-09-19 20:07:22 +03:00
Aliaksandr Valialkin
3a697a935a lib/storage: skip duplicate call to intersectMetricIDsWithTagFilter on zero successful intersects 2019-09-19 17:49:56 +03:00
Aliaksandr Valialkin
51a21c7d4b lib/mergeset: fill partHeader.firstItem on first block flush 2019-09-19 17:48:09 +03:00
Aliaksandr Valialkin
3d83f5d334 lib/storage: mark tag filter returning errFallbackToMetricNameMatch as useless 
This will save CPU on subsequent calls for this filter
 2019-09-18 19:10:32 +03:00
Aliaksandr Valialkin
6f3b2fd600 deployment/docker/docker-compose.yml: update Prometheus and Grafana image tags 
Prometheus: from v2.10.0 to v2.12.0
Grafana: v6.2.1 from to v6.3.5
 2019-09-18 18:29:09 +03:00
Aliaksandr Valialkin
8d35718dc6 lib/storage: properly construct keys for uselessTagFiltersCache and register useless negative tag filters there 2019-09-17 23:20:27 +03:00
Aliaksandr Valialkin
33975513d0 vendor: update github.com/valyala/gozstd from v1.6.1 to v1.6.2 2019-09-16 21:50:49 +03:00
Aliaksandr Valialkin
63f2b539df vendor: make vendor-update 2019-09-13 22:48:56 +03:00
Aliaksandr Valialkin
9428ec9c9f deployment/docker: remove file system paths from the compiled binary 2019-09-13 22:45:59 +03:00
Aliaksandr Valialkin
0c8057924f lib/mergeset: properly check for sorted block headers 
Fix a typo for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/181
 2019-09-13 21:59:29 +03:00
Aliaksandr Valialkin
d4218d27e6 app/vmselect/promql: properly handle subqueries like aggr_func(rollup_func(metric[window:step])) 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/184
 2019-09-13 21:41:04 +03:00
hanzai
e2274714b1 lib/workingsetcache: adjust switching from mode=split to mode=whole smoothly and load cachefile successfully 2019-09-13 19:13:01 +03:00
Aliaksandr Valialkin
4d636c244d app/vmselect/promql: binary operation fixes according to Prometheus behaviour 
The follosing issues were fixed:
- VictoriaMetrics could leave superflouos labels when using `on` or `ignoring` modifiers
- VictoriaMetrics could return `duplicate timeseries` error when using `group_left` or `group_right` with non-empty label list
 2019-09-13 17:42:52 +03:00
Aliaksandr Valialkin
bad53e4207 lib/mergeset: dynamically calculate the maximum number of items per part, which can be cached in OS page cache 2019-09-11 14:53:45 +03:00