Commit Graph

380 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
89234f395d lib/storage: use sort.Sort instead of sort.slice in getSortedMetricIDs 2019-09-19 20:08:13 +03:00
Aliaksandr Valialkin
6e586fa09c lib/storage: skip duplicate call to intersectMetricIDsWithTagFilter on zero successful intersects 2019-09-19 17:51:10 +03:00
Aliaksandr Valialkin
410f993bf6 lib/mergeset: fill partHeader.firstItem on first block flush 2019-09-19 17:48:22 +03:00
Aliaksandr Valialkin
c05885fb5f lib/storage: mark tag filter returning errFallbackToMetricNameMatch as useless
This will save CPU on subsequent calls for this filter
2019-09-18 19:11:44 +03:00
Aliaksandr Valialkin
e041a196a7 deployment/docker/docker-compose.yml: update Prometheus from v2.3.2 to v2.12.0 2019-09-18 18:30:02 +03:00
Aliaksandr Valialkin
db71c940ea lib/storage: properly construct keys for uselessTagFiltersCache and register useless negative tag filters there 2019-09-17 23:18:37 +03:00
Artem Navoiev
ccb6dc6925 [deployment] clean up helm (#185) 2019-09-16 21:58:16 +03:00
Aliaksandr Valialkin
491b1317f4 vendor: update github.com/valyala/gozstd from v1.6.1 to v1.6.2 2019-09-16 21:50:02 +03:00
Aliaksandr Valialkin
5666112de2 deployment: switch docker image tag from v1.27.2-cluster to v1.27.3-cluster 2019-09-14 11:33:18 +03:00
Aliaksandr Valialkin
ba21622b78 vendor: make vendor-update 2019-09-13 22:49:34 +03:00
Aliaksandr Valialkin
020341d13a deployment/docker: remove file system paths from the compiled binary 2019-09-13 22:46:07 +03:00
Aliaksandr Valialkin
550a12415a app/vminsert/netstorage: log network errors when sending data to vmstorage nodes
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/175
2019-09-13 22:26:24 +03:00
Aliaksandr Valialkin
41ef6b060e lib/mergeset: properly check for sorted block headers
Fix a typo for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/181
2019-09-13 21:59:38 +03:00
Aliaksandr Valialkin
ee4585db33 app/vmselect/promql: properly handle subqueries like aggr_func(rollup_func(metric[window:step]))
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/184
2019-09-13 21:42:11 +03:00
hanzai
08cde5e3f4 lib/workingsetcache: adjust switching from mode=split to mode=whole smoothly and load cachefile successfully 2019-09-13 19:13:16 +03:00
Aliaksandr Valialkin
828e5f6d26 app/vmselect/promql: binary operation fixes according to Prometheus behaviour
The follosing issues were fixed:
- VictoriaMetrics could leave superflouos labels when using `on` or `ignoring` modifiers
- VictoriaMetrics could return `duplicate timeseries` error when using `group_left` or `group_right` with non-empty label list
2019-09-13 17:43:09 +03:00
Artem Navoiev
62b424bc4c [ci] github actions - run pipeline on pull request. Fix running of test in external PR from forks 2019-09-11 14:54:45 +03:00
Aliaksandr Valialkin
ed50b8792b app/vminsert/netstorage: reduce the maximum buffer size for rerouted rows, so it occupies less RAM 2019-09-11 14:50:30 +03:00
Aliaksandr Valialkin
b101064f8b all: report the number of bytes read on io.ReadFull error
This should simplify error investigation similar to https://github.com/VictoriaMetrics/VictoriaMetrics/issues/175
2019-09-11 14:50:24 +03:00
Aliaksandr Valialkin
2f4c950fe9 app/vminsert/netstorage: send per-storageNode bufs to vmstorage nodes in parallel
This should improve the maximum ingestion throughput.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/175
2019-09-11 14:50:19 +03:00
Aliaksandr Valialkin
694cc59ed1 app/vminsert/netstorage: dynamically adjust timeouts for sending packets from vminsert to vmstorage depending on packet size
Bigger packets will have more chances to be sent to vmstorage.
2019-09-11 14:50:14 +03:00
Aliaksandr Valialkin
568ff61dcf lib/mergeset: dynamically calculate the maximum number of items per part, which can be cached in OS page cache 2019-09-09 11:42:45 +03:00
Artem Navoiev
dc6e4151b0 [ci] bump version of go to 1.13 in github actions config 2019-09-08 19:52:05 +03:00
Aliaksandr Valialkin
9b8af27786 vendor: update github.com/klauspost/compress from v1.7.6 to v1.8.2 2019-09-06 00:49:57 +03:00
Aliaksandr Valialkin
b71d828e84 vendor: update golang.org/x/sys 2019-09-06 00:49:57 +03:00
Aliaksandr Valialkin
1f4e0b722d deployment: switch docker image tag from v1.27.1-cluster to v1.27.2-cluster 2019-09-05 12:30:03 +03:00
Aliaksandr Valialkin
2c654258ef lib/fs: add MustStopDirRemover for waiting until pending directories are removed on graceful shutdown
This patch is mainly required for laggy NFS. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/162
2019-09-05 11:17:17 +03:00
Aliaksandr Valialkin
d0953e9f02 app/vmselect/promql: ignore grouping by destination label in count_values, since such a grouping is performed automatically 2019-09-04 19:59:02 +03:00
Aliaksandr Valialkin
2c2bd897dd lib/storage: remove duplicate tag keys on MetricName.Marshal call
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/172
2019-09-04 18:13:51 +03:00
Aliaksandr Valialkin
5a9b1d85bb deployment/docker: switch Go builder from Go 1.12.9 to Go 1.13.0 2019-09-04 17:17:52 +03:00
Aliaksandr Valialkin
f78ffe565f app/vmselect/promql: do not return artificial points beyond the last point in time series 2019-09-04 16:34:29 +03:00
Aliaksandr Valialkin
a7d5d611fe app/vmselect/prometheus: do not adjust start and end args in /api/v1/query_range if nocache=1 arg is set
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/171
2019-09-04 13:10:17 +03:00
Aliaksandr Valialkin
82bfe818d0 lib/fs: try harder with directory removal on NFS in the event of temporary lock
Do not give up after 11 attempts of directory removal on laggy NFS.

Add `vm_nfs_dir_remove_failed_attempts_total` metric for counting the number of failed attempts
on directory removal.

Log failed attempts on directory removal after long sleep times.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/162
2019-09-04 12:24:41 +03:00
Alfred Krohmer
7cde25bac4 Don't render extraLabels in ServiceMonitor if they are not given
This produced invalid YAML before.
2019-09-03 22:11:24 +03:00
Aliaksandr Valialkin
3182e2a66b deployment: update docker images from v1.27.0-cluster to v1.27.1-cluster 2019-09-03 21:05:50 +03:00
Aliaksandr Valialkin
b08f085082 app/vmselect/promql: reset timeseries name on group_left and group_right as Prometheus does 2019-09-03 20:43:29 +03:00
Aliaksandr Valialkin
458d412bb6 app/vmselect/netstorage: adaptively adjust the maximum inmemory file size for storing temporary blocks
The maximum inmemory file size now depends on `-memory.allowedPercent`.
This should improve performance and reduce the number of filesystem calls
on machines with big amounts of RAM when performing heavy queries
over big number of samples and time series.
2019-09-03 13:32:18 +03:00
Aliaksandr Valialkin
0b0153ba3d lib/storage: invalidate tagFilters -> TSIDS cache when newly added index data becomes visible to search
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/163
2019-08-29 15:08:44 +03:00
Aliaksandr Valialkin
8504a38214 lib/prombp: apply ba06b47c16
The following commands used:

gofmt -r '(uint64(x)&0x7F)<<shift -> uint64(x&0x7F)<<shift' -w ./lib/prompb/
gofmt -r '(int64(x)&0x7F)<<shift -> int64(x&0x7F)<<shift' -w ./lib/prompb/
2019-08-29 13:35:54 +03:00
Aliaksandr Valialkin
fb719bfb23 deployment: update docker images from v1.26.0-cluster to v1.27.0-cluster 2019-08-29 00:09:51 +03:00
Aliaksandr Valialkin
8f81908b1f .github/workflows: added GitHub actions 2019-08-28 23:11:26 +03:00
Aliaksandr Valialkin
604a4312f9 all: port to FreeBSD on GOARCH=amd64 2019-08-28 01:46:09 +03:00
Aliaksandr Valialkin
5893a9f9a3 app/vmstorage: increase default values for search.maxTagKeys, search.maxTagValues and search.maxUniqueTimeseries 2019-08-27 14:28:26 +03:00
Aliaksandr Valialkin
da07a6fb38 lib/storage: go fmt 2019-08-27 14:28:24 +03:00
Aliaksandr Valialkin
a63b69e9e2 lib/storage: report proper maxMetrics limit when more than -search.maxUniqueTimeseries series match the given filters 2019-08-27 14:21:31 +03:00
Aliaksandr Valialkin
82e813bad3 lib/storage: properly handle (?i) in the tag filter regexp
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/161
2019-08-26 00:44:56 +03:00
Aliaksandr Valialkin
e2eac858b5 lib/storage: calculate the maximum number of rows per small part from -memory.allowedPercent
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/159

This simplifies error detection additionally to the `vm_rows_ignored_total` counters.
2019-08-25 15:29:09 +03:00
Aliaksandr Valialkin
0a8dd9cc9a lib/storage: calculate the maximum number of rows per small part from -memory.allowedPercent
This should improve query speed over recent data on machines with big amounts of RAM
2019-08-25 14:41:32 +03:00
Aliaksandr Valialkin
bc576fb386 lib/storage: properly limit the number of output rows in small and big parts storage
Previously small parts storage didn't take into account the available disk space for big parts.
2019-08-25 14:41:32 +03:00
Aliaksandr Valialkin
947decb3dd lib/storage: remove outdated comment on maxRowsPerSmallPart
The commend became outdated after the commit ed6ac1a5df027f0dfc22448e3b27c26b6f77c67a,
which stops merging of small parts on graceful shutdown instead of waiting
for their completion.
2019-08-25 13:46:10 +03:00