Commit Graph

1669 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
c23b66a1ad lib/storage: pre-allocate memory for blockHeader slice in unmarshalBlockHeaders
This reduces memory usage and memory fragmentation when working with big number of time series
2019-08-19 12:46:33 +03:00
Aliaksandr Valialkin
be39414f9c deployment/docker: switch Go builder from go1.12.8 to go1.12.9 2019-08-18 22:07:58 +03:00
Aliaksandr Valialkin
e74fb23189 app/vmselect/promql: add scrape_interval(q[d]) function, which would return scrape interval for q over d 2019-08-18 21:08:26 +03:00
Aliaksandr Valialkin
582fdc059a app/vmselect/promql: hande comparisons with NaN similar to Prometheus
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150
2019-08-18 00:25:50 +03:00
Aliaksandr Valialkin
1c108fc494 app/vmselect/promql: add lifetime(q[d]) function, which returns the lifetime of q over d in seconds.
This function is useful for determining time series lifetime.
`d` must exceed the expected lifetime of the time series, otherwise
the function would return values close to `d`.
2019-08-16 11:59:32 +03:00
Aliaksandr Valialkin
d6b5ed6d39 app/vmselect/promql: fix corner-case calculation for ideriv 2019-08-16 11:59:28 +03:00
Aliaksandr Valialkin
639b14e8ab app/vmselect/promql: properly handle corner cases for rollup functions 2019-08-15 23:29:59 +03:00
Aliaksandr Valialkin
483de1cc06 lib/workingsetcache: automatically detect when it is better to double cache capacity 2019-08-15 22:57:55 +03:00
Aliaksandr Valialkin
9e0896055d deployment/docker: switch Go builder from go1.12.7 to go1.12.8 2019-08-15 20:43:36 +03:00
Aliaksandr Valialkin
5bb61b8b38 vendor: update github.com/valyala/gozstd from v1.5.1 to v1.6.0 2019-08-15 12:56:42 +03:00
Aliaksandr Valialkin
75a58dee02 README.md: typo fix 2019-08-14 03:28:07 +03:00
Aliaksandr Valialkin
5b41122292 lib/storage: properly cache tagFilters -> TSIDs entries from historical index 2019-08-14 02:29:58 +03:00
Aliaksandr Valialkin
964c296f96 lib/storage: compress contents of cache for tagFilters -> TSIDs
This should increase cache capacity
2019-08-14 02:29:52 +03:00
Aliaksandr Valialkin
9ecb994671 app/vmselect/promql: store compressed results in the cache
This should increase rollup results cache capacity.
2019-08-14 02:29:45 +03:00
Aliaksandr Valialkin
9d41e0dcae README.md: reduce the recommended max_shards value according to test results
See https://github.com/prometheus/prometheus/issues/5803#issuecomment-520973662
2019-08-13 22:33:10 +03:00
Aliaksandr Valialkin
09fc6e22e5 all: use workingsetcache instead of fastcache
This should reduce the amount of RAM required for processing time series
with non-zero churn rate.

The previous cache behavior can be restored with `-cache.oldBehavior` command-line flag.
2019-08-13 21:39:34 +03:00
Aliaksandr Valialkin
99c37c2c96 lib/fs: add test for IsTemporaryFileName 2019-08-13 21:33:45 +03:00
Aliaksandr Valialkin
06c2c25544 Makefile: consistency renaming: check_all -> check-all 2019-08-13 21:31:19 +03:00
Aliaksandr Valialkin
ec1b185991 lib/storage: remove broken BenchmarkIndexDBSearchTSIDs 2019-08-13 20:22:08 +03:00
Aliaksandr Valialkin
0967683ae9 lib: move common code for creating flock.lock file into fs.CreateFlockFile 2019-08-13 01:45:46 +03:00
Aliaksandr Valialkin
ad8a43b4e1 README.md: fix metric names in influx line protocol example
Default separator between `measurement` and `field_name` is `_`.
2019-08-12 15:58:34 +03:00
Aliaksandr Valialkin
7346982763 README.md: mention that Influx line protocol accepts timestamps in nanoseconds by default 2019-08-12 15:31:52 +03:00
Aliaksandr Valialkin
5d8d110010 lib/fs: atomically create file with the given contents on WriteFileAtomically
This should prevent from `transaction` and `metadata.json` files corruption
on unclean shutdown such as OOM, `kill -9`, power loss, etc.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/148
2019-08-12 15:02:55 +03:00
Aliaksandr Valialkin
0b488f1e37 lib/storage: do not change timestamps to constant rate if values are constant or have constant delta
This breaks the original timestamps, which results in issues like
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/120 and
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/141 .
2019-08-06 15:40:07 +03:00
Aliaksandr Valialkin
b8bb74ffc6 app/vmstorage: add vm_concurrent_addrows_* metrics for tracking concurrency for Storage.AddRows calls
Track also the number of dropped rows due to the exceeded timeout
on concurrency limit for Storage.AddRows. This number is tracked in `vm_concurrent_addrows_dropped_rows_total`
2019-08-06 15:08:33 +03:00
Aliaksandr Valialkin
5c9e48417a vendor: update github.com/VictoriaMetrics/metrics to v1.7.1 2019-08-05 19:21:36 +03:00
Aliaksandr Valialkin
5c83f8e203 app: add vm_concurrent_ metrics for visibility in concurrency limiters for vminsert and vmselect 2019-08-05 18:30:57 +03:00
Aliaksandr Valialkin
05713469c3 vendor: make vendor-update 2019-08-05 10:33:21 +03:00
Aliaksandr Valialkin
8822079b77 lib/storage: properly reset partSearch.fetchData in partSearch.reset 2019-08-05 09:56:06 +03:00
Aliaksandr Valialkin
99e048c9df app/vmselect: allow passing match[], start and time to /api/v1/label/<label_name>/values
`/api/v1/label/<label_name>/values?match[]=q` emulates emulates `label_values(q, <label_name>)`
call in Grafana templating.
2019-08-04 23:09:21 +03:00
Aliaksandr Valialkin
47e4b50112 app/vmselect: optimize /api/v1/series by skipping storage data
Fetch and process only time series metainfo.
2019-08-04 23:01:28 +03:00
Aliaksandr Valialkin
241170dc05 app/vmselect/prometheus: prevent from fetching and scanning all the data on /api/v1/searies call by default 2019-08-04 19:42:36 +03:00
Aliaksandr Valialkin
1c69f4eadc app/vmselect/promql: tune automatic window adjustement
Increase the windows adjustement for small scrape intervals,
since they usually have higher jitter.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/139
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/134
2019-08-04 19:34:05 +03:00
Aliaksandr Valialkin
8d93b15b86 app/vmselect/promql: further increase the allowed jitter for scrape interval
Real-world production data shows higher jitter than 1/8 of scrape interval.
This may results in gaps on the graph. So increase the allowed jitter to 1/4
of scrape interval in order to reduce the probability of gaps on the graphs
over time series with high jitter for scrape_interval.
2019-08-02 20:10:23 +03:00
Aliaksandr Valialkin
fcc166622a README.md: mention that monitoring is recommended for VictoriaMetrics 2019-08-02 15:27:10 +03:00
Aliaksandr Valialkin
a9f39168d2 app/vminsert/influx: round automatically generated timestamp according to the given precision arg 2019-08-02 00:24:06 +03:00
Aliaksandr Valialkin
f090b2e917 app/vmselect/promql: tolerate higher jitter in scrape interval
Allow jitter for up to 1/8 instead of 1/16 for the scrape interval.
This should imrpove graphs when `step` is smaller than the `scrape_interval`.
2019-08-01 23:26:00 +03:00
Aliaksandr Valialkin
10caad4728 lib/decimal: modernize tests a bit 2019-07-31 21:10:03 +03:00
Aliaksandr Valialkin
3b90c2a99a
Add CODE_OF_CONDUCT.md 2019-07-31 15:44:26 +03:00
Aliaksandr Valialkin
57ec4f5f92 Update issue templates
Add a template for feature request
2019-07-31 15:41:57 +03:00
Aliaksandr Valialkin
01cb15b6f5 Update issue templates
Add a template for bug report.
2019-07-31 15:39:41 +03:00
Aliaksandr Valialkin
b9256511e8
README.md: add join slack badge 2019-07-31 15:27:11 +03:00
Aliaksandr Valialkin
3a38b23fa3 app/vmselect/promql: add vm_slow_queries_total metric for counting slow queries
The query is slow if its execution time exceeds `-search.logSlowQueryDuration`
2019-07-31 03:36:37 +03:00
Aliaksandr Valialkin
8bd6f1f6df app/vmselect/promql: return NaN from histogram_quantile if at least a single bucket is broken 2019-07-31 01:18:07 +03:00
Aliaksandr Valialkin
4aaa5c2efc app/vmselect/promql: allow adjusting window for default rollup function
Default rollup function is `last_over_time`. It must support adjusting
the provided window in order to prevent from gaps on the graph
for window values smaller than scrape interval.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/134
2019-07-31 00:45:54 +03:00
Aliaksandr Valialkin
10f5a26bec app/vmselect/promql: return NaN values if invalid bucket counts are passed to histogram_quantile
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/136
2019-07-30 22:05:10 +03:00
Aliaksandr Valialkin
c14fd6c43f lib/storage: typo fixes after a77e88db7d 2019-07-30 15:38:52 +03:00
Aliaksandr Valialkin
a77e88db7d lib/storage: fix matching against tag filter with empty name
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/137
2019-07-30 15:15:09 +03:00
Aliaksandr Valialkin
aad7236e5d README.md: formatting fixes 2019-07-28 22:02:42 +03:00
Artem Navoiev
5e5de6be9a Create CONTRIBUTING.md 2019-07-28 20:42:32 +03:00