Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-16 17:21:04 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
252 Commits 114 Branches 748 Tags 517 MiB
5b8526e925
Commit Graph

71 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
c6bec48927 lib/storage: add metrics for calculating skipped rows outside the retention 
The metrics are:

    - vm_too_big_timestamp_rows_total
    - vm_too_small_timestamp_rows_total
 2019-07-26 14:11:56 +03:00
Aliaksandr Valialkin
f2e8d54fb0 lib/encoding/zstd: go fmt 2019-07-25 01:37:57 +03:00
Aliaksandr Valialkin
97b5dc7122 lib/encoding/zstd: disable CRC checks in pure Go build 
This should give slightly better compression and decompressions performance.
Additionally this shaves off 4 bytes per each compressed block.
 2019-07-24 19:17:32 +03:00
Aliaksandr Valialkin
54f035d4ce all: small updates after PR #114 2019-07-24 17:43:43 +03:00
Aliaksandr Valialkin
7a133567fb lib/encoding: small fixes in tests after the PR #114 2019-07-24 17:43:39 +03:00
Roman Khavronenko
fcf09aaa3c all: add Pure Go build (pull request #114) 
Updates #94
 2019-07-24 17:43:32 +03:00
Aliaksandr Valialkin
3fae34eeb4 lib/encoding: improve gauge series detection 
- Series with negative values are always gauges
- Counters may only have increasing values with possible counter resets

This should improve compression ratio for gauge series which
were previously mistakenly detected as counters.
 2019-07-20 14:05:25 +03:00
Aliaksandr Valialkin
7abb96b454 lib/netutil: do not count timeouts as network errors 2019-07-15 23:06:13 +03:00
Aliaksandr Valialkin
73a47d2a53 lib/storage: remove unused function isTooBigTimeRangeForDateMetricIDs 2019-07-12 02:28:40 +03:00
Aliaksandr Valialkin
97f9397687 lib/storage: do not reduce maxMetrics on time ranges exceeding maxDaysForDateMetricIDs 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/95
 2019-07-12 02:21:52 +03:00
Aliaksandr Valialkin
4ca66344ee lib/storage: do not pollute inverted index with data for samples outside the retention period 2019-07-11 17:11:33 +03:00
Aliaksandr Valialkin
0522efb2d6 lib/storage: add missing tagFilter.Marshal func 2019-07-11 15:01:01 +03:00
Aliaksandr Valialkin
12b1d67b41 lib/storage: use fast path for orSuffix when searching for metricIDs against plain tag value 2019-07-11 14:48:51 +03:00
Aliaksandr Valialkin
bf2e1b0ac1 lib/storage: remember and skip individual tag filters matching too many metrics 
This saves CPU time by skipping useless matching for individual tag filters.
 2019-07-11 14:48:47 +03:00
Aliaksandr Valialkin
ba8195c58e all: consistency renaming: bytesSize -> sizeBytes 2019-07-10 00:47:42 +03:00
Aliaksandr Valialkin
41f512af1c all: add vm_data_size_bytes metrics for easy monitoring of on-disk data size and on-disk inverted index size 2019-07-04 19:43:04 +03:00
Aliaksandr Valialkin
6682a35731 lib/memory: attempt #3 to determine memory limit for LXC container 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/84
 2019-07-01 14:01:57 +03:00
Aliaksandr Valialkin
b6ea1a7d5e lib/mergeset: make fmt 2019-06-29 14:25:46 +03:00
Aliaksandr Valialkin
ffc1bb00f6 lib/storage: skip non-matching metricIDs in sortedFilter 
This should improve performance for big sorteFilter lists.
 2019-06-29 13:49:40 +03:00
Aliaksandr Valialkin
2257dcd278 lib/mergeset: speed up binarySearchKey by skipping the first item during binary search 2019-06-29 13:49:32 +03:00
Aliaksandr Valialkin
6ea12a079e lib/memory: remove TestReadLXCMemoryLimit, since it doesnt work in Travis 2019-06-28 18:23:06 +03:00
Aliaksandr Valialkin
d0732d3137 lib/memory: attempt #2 to determine memory limit inside LXC container 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/84
 2019-06-28 18:08:57 +03:00
Aliaksandr Valialkin
ad436757c3 lib/memory: an attempt to read proper memory limit inside LXC container 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/84
 2019-06-28 15:35:02 +03:00
Aliaksandr Valialkin
416d27ef11 lib/storage: optimize time series search by regexp filter 
This should improve search speed on label filters like `{foo=~"bar.+baz"}`
 2019-06-27 16:18:00 +03:00
Aliaksandr Valialkin
ee23a143b9 lib/storage: make sure non-nil args are passed to openIndexDB 2019-06-25 20:10:08 +03:00
Aliaksandr Valialkin
8b0a63722f lib/storage: reduce too big maxMetrics in getTagFilterWithMinMetricIDsCountAdaptive 
This should improve performance on inverted index search for big amount of unique time series
when big -search.maxUniqueTimeseries is set.
 2019-06-25 19:57:31 +03:00
Aliaksandr Valialkin
0263cb0adc lib/storage: free up memory from caches owned by indexDB when it is deleted 2019-06-25 14:41:16 +03:00
Aliaksandr Valialkin
362e187011 lib/storage: use unversioned keys for tag cache in extDB 
Data in ExtDB cannot be changed, so it is OK to use unversioned keys for tag cache.
This should improve performance for index lookups over big amount of time series.
 2019-06-25 13:15:42 +03:00
Aliaksandr Valialkin
51e2f3b48f lib/storage: skip searching in extDB if it doesn't contain items for the given time range 
This should improve inverted index search performance for big amount
of unique time series when the search is performed only on recent data.
 2019-06-25 12:57:56 +03:00
Aliaksandr Valialkin
fb9358635d lib/storage: mention source parts on merge error 
This should improve determining broken source part.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/76
 2019-06-24 14:09:46 +03:00
Aliaksandr Valialkin
9164c223ec all: initial stubs for Windows support; see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70 2019-06-20 20:07:41 +03:00
Aliaksandr Valialkin
786beb8fc8 Makefile: enable golangci-lint in make check_all 2019-06-20 15:00:58 +03:00
Aliaksandr Valialkin
9cac11db64 lib/storage: typo fixes found by golangci-lint; updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/69 2019-06-20 14:38:45 +03:00
Aliaksandr Valialkin
7778030f9f lib/netutil: remove unused TCPListener.name; updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/69 2019-06-20 14:36:19 +03:00
Aliaksandr Valialkin
a0c22a6830 app/vmstorage: add vm_cache_entries{type="storage/hour_metric_ids"} metric for tracking active time series count 2019-06-19 18:37:38 +03:00
Aliaksandr Valialkin
24ae3ef532 lib/prompb: remove superflouos bytes copying in ReadSnappy 2019-06-18 21:02:02 +03:00
Aliaksandr Valialkin
e40224d5de lib/flagutil: add NewArray helper func 2019-06-18 10:44:09 +03:00
Aliaksandr Valialkin
f9e1d32168 lib/storage: persist metric ids for the current and the previous hour on graceful shutdown 
This should improve performance after restart when the db contains a lot of time series
with high time series churn (i.e. metrics from Kubernetes with many pods and frequent deployments)
 2019-06-14 07:55:09 +03:00
Aliaksandr Valialkin
b0b93e3d50 lib/fs: sync parent dir in MustRemoveAll only if it exists 
The parent directory may be non-existing when the deleted directory
didn't exist before the MustRemoveAll call
 2019-06-12 02:16:15 +03:00
Aliaksandr Valialkin
18d6f293f7 lib/fs: consolidate *RemoveAll* funcs into a single MustRemoveAll func 
The func syncs parent dir in order to persist directory removal
in the event of power loss
 2019-06-12 01:55:18 +03:00
Aliaksandr Valialkin
28d9904efc lib/fs: panic with fatal error when directories cannot be removed 
Unremoved directories may lead to inconsistent data directory,
so VictoriaMetrics will fail to start next time.

So panic on the first error when trying to remove directory in order
to simplify recover process.
 2019-06-12 01:20:10 +03:00
Aliaksandr Valialkin
d897bc3f08 lib/fs: attempt #2 to work around NFS issue with directory removal 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
 2019-06-12 01:07:29 +03:00
Aliaksandr Valialkin
51e2e255a6 lib/fs: consistency renaming SyncPath -> MustSyncPath, since it doesnt return error 2019-06-11 23:13:45 +03:00
Aliaksandr Valialkin
3fa4c28f6b lib/fs: make sure the created directory remains visible in the fs in the event of power loss 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/63
 2019-06-11 23:08:17 +03:00
Aliaksandr Valialkin
0b7f751f60 lib/fs: use filepath.Dir instead of filepath.Split, since the filename is unused 2019-06-11 22:54:23 +03:00
Aliaksandr Valialkin
b491045a4b lib/{storage,mergeset}: sync filenames inside part when finalizing the part 
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/63
 2019-06-11 21:51:19 +03:00
Aliaksandr Valialkin
3437c30180 all: try hard removing directory with contents 
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61
 2019-06-11 01:58:08 +03:00
Aliaksandr Valialkin
75a0acf72d app/vmselect: add /api/v1/labels/count handler for quick detection of labels with the maximum number of distinct values 2019-06-10 19:54:55 +03:00
Aliaksandr Valialkin
0ccedbdfd2 lib/storage: mention the accountID and projectID in error message when filtering out other (accountID, projectID) entries 2019-06-10 14:43:53 +03:00
Aliaksandr Valialkin
d54f5fec0b lib/storage: skip adaptive searching for tag filter matching the minimum number of metrics if the identical previous search didn't found such filter 
This should improve speed for searching metrics among high number of time series
with high churn rate like in big Kubernetes clusters with frequent deployments.
 2019-06-10 14:07:47 +03:00