Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-20 15:36:31 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
381 Commits 114 Branches 748 Tags 619 MiB
4a2cd85b92
Commit Graph

151 Commits

Author SHA1 Message Date
Artem Navoiev
22e3fabefd Add OpenTSDB and Prometheus integration tests (#168) 
* [WIP] open tsdb and prometheus integration tests

* app/victoria-metrics: fix race condition on parallel tests
 2019-09-05 17:55:38 +03:00
Aliaksandr Valialkin
88f8670ede lib/fs: add MustStopDirRemover for waiting until pending directories are removed on graceful shutdown 
This patch is mainly required for laggy NFS. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/162
 2019-09-05 11:13:17 +03:00
Aliaksandr Valialkin
6954e126fc app/vmselect/promql: ignore grouping by destination label in count_values, since such a grouping is performed automatically 2019-09-04 19:58:01 +03:00
Aliaksandr Valialkin
305e7bc981 app/vmselect/promql: do not return artificial points beyond the last point in time series 2019-09-04 16:35:34 +03:00
Aliaksandr Valialkin
9721d06c6a app/vmselect/prometheus: do not adjust start and end args in /api/v1/query_range if nocache=1 arg is set 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/171
 2019-09-04 13:10:09 +03:00
Aliaksandr Valialkin
db4560ca31 app/vmselect/promql: reset timeseries name on group_left and group_right as Prometheus does 2019-09-03 20:42:54 +03:00
Aliaksandr Valialkin
1575a560f0 app/vmselect/netstorage: adaptively adjust the maximum inmemory file size for storing temporary blocks 
The maximum inmemory file size now depends on `-memory.allowedPercent`.
This should improve performance and reduce the number of filesystem calls
on machines with big amounts of RAM when performing heavy queries
over big number of samples and time series.
 2019-09-03 13:32:09 +03:00
Aliaksandr Valialkin
3685fc18d5 Makefile: extract app-local and app-local-pure build rules 2019-08-28 01:34:58 +03:00
Aliaksandr Valialkin
ede7ad3703 app/victoria-metrics: add missing victoria-metrics prefix to --version output when building with make victoria-metrics 2019-08-28 01:28:08 +03:00
Aliaksandr Valialkin
9196c085a7 all: port to FreeBSD on GOARCH=amd64 2019-08-28 01:19:23 +03:00
Aliaksandr Valialkin
603a79b357 app/vmstorage: increase default values for search.maxTagKeys, search.maxTagValues and search.maxUniqueTimeseries 2019-08-27 14:29:53 +03:00
Aliaksandr Valialkin
fb909cf710 app/vminsert/influx: set db label only if Influx line doesnt have db tag 2019-08-24 13:52:48 +03:00
Aliaksandr Valialkin
9c9221d1b2 app/vminsert: skip empty tags 2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
70ca018a57 app/vminsert/opentsdbhttp: skip invalid rows and continue parsing the remaining rows 
Invalid rows are logged and counted in `vm_rows_invalid_total{type="opentsdb-http"}` metric
 2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
4266091e4f app/vminsert/opentsdb: skip invalid rows and continue parsing the remaining rows 
Invalid rows are logged and counted in `vm_rows_invalid_total{type="opentsdb"}` metric
 2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
8001d29b6e app/vminsert/graphite: skip invalid rows and continue parsing the remaining rows 
Invalid rows are logged and counted in `vm_rows_invalid_total{type="graphite"}` metric
 2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
9d3f1fcbb9 app/vminsert/influx: skip invalid rows and continue parsing the remaining rows 
Invalid influx lines are logged and counted in `vm_rows_invalid_total{type="influx"}` metric.
 2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
ba7b3806be app/vminsert/influx: do not allow escaping newline char, since they dont occur in real life 
The prefious report with escaped newline chars in influx line protocol was false alarm.
 2019-08-23 18:42:05 +03:00
Aliaksandr Valialkin
7fa88c6efc app/vminsert/opentsdbhttp: allow timestamp as float64 and as string, since it occurs in real life 2019-08-23 18:35:41 +03:00
Aliaksandr Valialkin
4da34b11f8 app/vminsert/influx: handle \r\n aka crlf influx line endings from windows world 
Such lines exist in real life.
 2019-08-23 18:28:49 +03:00
Aliaksandr Valialkin
a18317adbc app/vminsert/influx: allow escaping newline char 
Though newline char isn't mentioned in escape rules at https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_tutorial/ ,
there are reports that such chars occur in real life
 2019-08-23 15:14:46 +03:00
Aliaksandr Valialkin
44d7fc599d app/vminsert/influx: skip comments starting with # in influx line protocol 2019-08-23 14:43:09 +03:00
Aliaksandr Valialkin
ac004665b5 all: return 503 http error if service is temporarily unavailable 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/156
 2019-08-23 09:55:07 +03:00
Aliaksandr Valialkin
8c03a8c4b4 app/vminsert: allow setting the maximum number of labels per time series via -maxLabelsPerTimeseries 2019-08-23 08:45:26 +03:00
Aliaksandr Valialkin
1272e407b2 app/vmselect/promql: attempt to repair invalid bucket counts passed to histogram_quantile 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/136
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/154
 2019-08-22 14:39:46 +03:00
Aliaksandr Valialkin
5f33fc8e46 app/vminsert: add ability to ingest data via HTTP OpenTSDB /api/put requests 
This is manual merge of the https://github.com/VictoriaMetrics/VictoriaMetrics/pull/152
Thanks to nustinov@gmail.com for the initial pull request.
 2019-08-22 12:28:32 +03:00
Aliaksandr Valialkin
ec8125606d app/vminsert/opentsdb: fix BenchmarkRowsUnmarshal by adding missing put prefixes to each line 2019-08-21 19:14:47 +03:00
Aliaksandr Valialkin
f4a38f7fb1 app/vmselect/promql: fix panic on -search.disableCache 
Reset the cache if it is disabled instead of stopping, since it is stopped on graceful shutdown.
 2019-08-21 17:11:52 +03:00
Aliaksandr Valialkin
ab740afd0d app/vmselect/promql: explain why empty timeseries arent removed in transformLabelValue 2019-08-21 11:29:24 +03:00
Aliaksandr Valialkin
7b5168adfb app/vmselect/promql: remove NaNs from /api/v1/query_range output like Prometheus does 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/153
 2019-08-20 23:01:41 +03:00
Aliaksandr Valialkin
a0d480fbf3 app/vmselect/promql: pre-allocate memory for map for checking for duplicate timeseries 
This should reduce memory allocations for big number of timeseries
 2019-08-20 23:01:39 +03:00
Aliaksandr Valialkin
d3fd113a80 app/vmselect/promql: add label_value(q, label_name) func, which returns numeric value labels with name label_name in q 2019-08-20 00:28:34 +03:00
Aliaksandr Valialkin
dd86e6130c app/vmselect/promql: independently track offset hints for tStart and tEnd 
This should improve performance if timeseries starts or ends on the selected time range
 2019-08-19 13:40:14 +03:00
Aliaksandr Valialkin
6a27657d73 app/vmselect/promql: optimize search for timestamp boundaries in rollupConfig.Do 
This should improve the performance of queries over big number of time series
with big number of output points.
 2019-08-19 13:03:29 +03:00
Aliaksandr Valialkin
e74fb23189 app/vmselect/promql: add scrape_interval(q[d]) function, which would return scrape interval for q over d 2019-08-18 21:08:26 +03:00
Aliaksandr Valialkin
582fdc059a app/vmselect/promql: hande comparisons with NaN similar to Prometheus 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150
 2019-08-18 00:25:50 +03:00
Aliaksandr Valialkin
1c108fc494 app/vmselect/promql: add lifetime(q[d]) function, which returns the lifetime of q over d in seconds. 
This function is useful for determining time series lifetime.
`d` must exceed the expected lifetime of the time series, otherwise
the function would return values close to `d`.
 2019-08-16 11:59:32 +03:00
Aliaksandr Valialkin
d6b5ed6d39 app/vmselect/promql: fix corner-case calculation for ideriv 2019-08-16 11:59:28 +03:00
Aliaksandr Valialkin
639b14e8ab app/vmselect/promql: properly handle corner cases for rollup functions 2019-08-15 23:29:59 +03:00
Aliaksandr Valialkin
9ecb994671 app/vmselect/promql: store compressed results in the cache 
This should increase rollup results cache capacity.
 2019-08-14 02:29:45 +03:00
Aliaksandr Valialkin
09fc6e22e5 all: use workingsetcache instead of fastcache 
This should reduce the amount of RAM required for processing time series
with non-zero churn rate.

The previous cache behavior can be restored with `-cache.oldBehavior` command-line flag.
 2019-08-13 21:39:34 +03:00
Aliaksandr Valialkin
b8bb74ffc6 app/vmstorage: add vm_concurrent_addrows_* metrics for tracking concurrency for Storage.AddRows calls 
Track also the number of dropped rows due to the exceeded timeout
on concurrency limit for Storage.AddRows. This number is tracked in `vm_concurrent_addrows_dropped_rows_total`
 2019-08-06 15:08:33 +03:00
Aliaksandr Valialkin
5c83f8e203 app: add vm_concurrent_ metrics for visibility in concurrency limiters for vminsert and vmselect 2019-08-05 18:30:57 +03:00
Aliaksandr Valialkin
99e048c9df app/vmselect: allow passing match[], start and time to /api/v1/label/<label_name>/values 
`/api/v1/label/<label_name>/values?match[]=q` emulates emulates `label_values(q, <label_name>)`
call in Grafana templating.
 2019-08-04 23:09:21 +03:00
Aliaksandr Valialkin
47e4b50112 app/vmselect: optimize /api/v1/series by skipping storage data 
Fetch and process only time series metainfo.
 2019-08-04 23:01:28 +03:00
Aliaksandr Valialkin
241170dc05 app/vmselect/prometheus: prevent from fetching and scanning all the data on /api/v1/searies call by default 2019-08-04 19:42:36 +03:00
Aliaksandr Valialkin
1c69f4eadc app/vmselect/promql: tune automatic window adjustement 
Increase the windows adjustement for small scrape intervals,
since they usually have higher jitter.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/139
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/134
 2019-08-04 19:34:05 +03:00
Aliaksandr Valialkin
8d93b15b86 app/vmselect/promql: further increase the allowed jitter for scrape interval 
Real-world production data shows higher jitter than 1/8 of scrape interval.
This may results in gaps on the graph. So increase the allowed jitter to 1/4
of scrape interval in order to reduce the probability of gaps on the graphs
over time series with high jitter for scrape_interval.
 2019-08-02 20:10:23 +03:00
Aliaksandr Valialkin
a9f39168d2 app/vminsert/influx: round automatically generated timestamp according to the given precision arg 2019-08-02 00:24:06 +03:00
Aliaksandr Valialkin
f090b2e917 app/vmselect/promql: tolerate higher jitter in scrape interval 
Allow jitter for up to 1/8 instead of 1/16 for the scrape interval.
This should imrpove graphs when `step` is smaller than the `scrape_interval`.
 2019-08-01 23:26:00 +03:00