Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-15 16:30:55 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
362 Commits 114 Branches 748 Tags 504 MiB
b101064f8b
Commit Graph

157 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
b101064f8b all: report the number of bytes read on io.ReadFull error 
This should simplify error investigation similar to https://github.com/VictoriaMetrics/VictoriaMetrics/issues/175
 2019-09-11 14:50:24 +03:00
Aliaksandr Valialkin
2f4c950fe9 app/vminsert/netstorage: send per-storageNode bufs to vmstorage nodes in parallel 
This should improve the maximum ingestion throughput.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/175
 2019-09-11 14:50:19 +03:00
Aliaksandr Valialkin
694cc59ed1 app/vminsert/netstorage: dynamically adjust timeouts for sending packets from vminsert to vmstorage depending on packet size 
Bigger packets will have more chances to be sent to vmstorage.
 2019-09-11 14:50:14 +03:00
Aliaksandr Valialkin
2c654258ef lib/fs: add MustStopDirRemover for waiting until pending directories are removed on graceful shutdown 
This patch is mainly required for laggy NFS. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/162
 2019-09-05 11:17:17 +03:00
Aliaksandr Valialkin
d0953e9f02 app/vmselect/promql: ignore grouping by destination label in count_values, since such a grouping is performed automatically 2019-09-04 19:59:02 +03:00
Aliaksandr Valialkin
f78ffe565f app/vmselect/promql: do not return artificial points beyond the last point in time series 2019-09-04 16:34:29 +03:00
Aliaksandr Valialkin
a7d5d611fe app/vmselect/prometheus: do not adjust start and end args in /api/v1/query_range if nocache=1 arg is set 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/171
 2019-09-04 13:10:17 +03:00
Aliaksandr Valialkin
b08f085082 app/vmselect/promql: reset timeseries name on group_left and group_right as Prometheus does 2019-09-03 20:43:29 +03:00
Aliaksandr Valialkin
458d412bb6 app/vmselect/netstorage: adaptively adjust the maximum inmemory file size for storing temporary blocks 
The maximum inmemory file size now depends on `-memory.allowedPercent`.
This should improve performance and reduce the number of filesystem calls
on machines with big amounts of RAM when performing heavy queries
over big number of samples and time series.
 2019-09-03 13:32:18 +03:00
Aliaksandr Valialkin
604a4312f9 all: port to FreeBSD on GOARCH=amd64 2019-08-28 01:46:09 +03:00
Aliaksandr Valialkin
5893a9f9a3 app/vmstorage: increase default values for search.maxTagKeys, search.maxTagValues and search.maxUniqueTimeseries 2019-08-27 14:28:26 +03:00
Aliaksandr Valialkin
38711526d3 app/vminsert/influx: set db label only if Influx line doesnt have db tag 2019-08-24 13:55:01 +03:00
Aliaksandr Valialkin
1ee536f9fd app/vminsert: skip empty tags 2019-08-24 13:36:41 +03:00
Aliaksandr Valialkin
a283023d16 app/vminsert/opentsdbhttp: skip invalid rows and continue parsing the remaining rows 
Invalid rows are logged and counted in `vm_rows_invalid_total{type="opentsdb-http"}` metric
 2019-08-24 13:36:41 +03:00
Aliaksandr Valialkin
38b9615c53 app/vminsert/opentsdb: skip invalid rows and continue parsing the remaining rows 
Invalid rows are logged and counted in `vm_rows_invalid_total{type="opentsdb"}` metric
 2019-08-24 13:36:41 +03:00
Aliaksandr Valialkin
2a8fc41bab app/vminsert/graphite: skip invalid rows and continue parsing the remaining rows 
Invalid rows are logged and counted in `vm_rows_invalid_total{type="graphite"}` metric
 2019-08-24 13:36:41 +03:00
Aliaksandr Valialkin
22685ef94d app/vminsert/influx: skip invalid rows and continue parsing the remaining rows 
Invalid influx lines are logged and counted in `vm_rows_invalid_total{type="influx"}` metric.
 2019-08-24 13:36:41 +03:00
Aliaksandr Valialkin
425a81a6c7 app/vminsert/influx: do not allow escaping newline char, since they dont occur in real life 
The prefious report with escaped newline chars in influx line protocol was false alarm.
 2019-08-23 18:43:00 +03:00
Aliaksandr Valialkin
8da8dd0876 app/vminsert/opentsdbhttp: allow timestamp as float64 and as string, since it occurs in real life 2019-08-23 18:35:52 +03:00
Aliaksandr Valialkin
0ea21eb9dc app/vminsert/influx: handle \r\n aka crlf influx line endings from windows world 
Such lines exist in real life.
 2019-08-23 18:28:54 +03:00
Aliaksandr Valialkin
b3502b2b39 app/vminsert/influx: allow escaping newline char 
Though newline char isn't mentioned in escape rules at https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_tutorial/ ,
there are reports that such chars occur in real life
 2019-08-23 15:14:58 +03:00
Aliaksandr Valialkin
f1f8fce4f7 app/vminsert/influx: skip comments starting with # in influx line protocol 2019-08-23 14:43:24 +03:00
Aliaksandr Valialkin
697de90893 app/vminsert: do not drop data in reroutedBuf if all the storage nodes are unhealthy 2019-08-23 10:38:19 +03:00
Aliaksandr Valialkin
a5dc54efc3 app/vminsert: properly limit the size of reroutedBuf 2019-08-23 10:29:51 +03:00
Aliaksandr Valialkin
c197641978 all: return 503 http error if service is temporarily unavailable 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/156
 2019-08-23 09:49:50 +03:00
Aliaksandr Valialkin
e734076f0f app/vminsert: allow setting the maximum number of labels per time series via -maxLabelsPerTimeseries 2019-08-23 08:47:18 +03:00
Aliaksandr Valialkin
e9db22a551 app/vmselect/promql: attempt to repair invalid bucket counts passed to histogram_quantile 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/136
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/154
 2019-08-22 14:39:24 +03:00
Aliaksandr Valialkin
0697164b4f app/vminsert: add ability to ingest data via HTTP OpenTSDB /api/put requests 
This is manual merge of the https://github.com/VictoriaMetrics/VictoriaMetrics/pull/152
Thanks to nustinov@gmail.com for the initial pull request.
 2019-08-22 12:46:54 +03:00
Aliaksandr Valialkin
4d555c7c87 app/vminsert/opentsdb: fix BenchmarkRowsUnmarshal by adding missing put prefixes to each line 2019-08-21 19:15:04 +03:00
Aliaksandr Valialkin
90a4b00b10 app/vmselect/promql: fix panic on -search.disableCache 
Reset the cache if it is disabled instead of stopping, since it is stopped on graceful shutdown.
 2019-08-21 17:12:01 +03:00
Aliaksandr Valialkin
491b1762c8 app/vmselect/promql: explain why empty timeseries arent removed in transformLabelValue 2019-08-21 11:29:41 +03:00
Aliaksandr Valialkin
db1de4277c app/vmselect/promql: remove NaNs from /api/v1/query_range output like Prometheus does 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/153
 2019-08-20 23:01:59 +03:00
Aliaksandr Valialkin
99331606e1 app/vmselect/promql: pre-allocate memory for map for checking for duplicate timeseries 
This should reduce memory allocations for big number of timeseries
 2019-08-20 23:01:57 +03:00
Aliaksandr Valialkin
1101765adb app/vmselect/promql: add label_value(q, label_name) func, which returns numeric value labels with name label_name in q 2019-08-20 00:28:44 +03:00
Aliaksandr Valialkin
940349ccb9 app/vmselect/promql: independently track offset hints for tStart and tEnd 
This should improve performance if timeseries starts or ends on the selected time range
 2019-08-19 13:40:24 +03:00
Aliaksandr Valialkin
6ae4b4190f app/vmselect/promql: optimize search for timestamp boundaries in rollupConfig.Do 
This should improve the performance of queries over big number of time series
with big number of output points.
 2019-08-19 13:03:38 +03:00
Aliaksandr Valialkin
005aabd305 app/vmselect/promql: add scrape_interval(q[d]) function, which would return scrape interval for q over d 2019-08-18 21:08:15 +03:00
Aliaksandr Valialkin
218cb4623a app/vmselect/promql: hande comparisons with NaN similar to Prometheus 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/150
 2019-08-18 00:25:58 +03:00
Aliaksandr Valialkin
dcce92c63c app/vmselect/promql: add lifetime(q[d]) function, which returns the lifetime of q over d in seconds. 
This function is useful for determining time series lifetime.
`d` must exceed the expected lifetime of the time series, otherwise
the function would return values close to `d`.
 2019-08-16 11:59:51 +03:00
Aliaksandr Valialkin
0cb66a8f95 app/vmselect/promql: fix corner-case calculation for ideriv 2019-08-16 11:59:50 +03:00
Aliaksandr Valialkin
1b5b9ced27 app/vmselect/promql: properly handle corner cases for rollup functions 2019-08-15 23:31:28 +03:00
Aliaksandr Valialkin
b8bbe92de1 app/vmselect/promql: store compressed results in the cache 
This should increase rollup results cache capacity.
 2019-08-14 02:32:16 +03:00
Aliaksandr Valialkin
8c2158af24 all: use workingsetcache instead of fastcache 
This should reduce the amount of RAM required for processing time series
with non-zero churn rate.

The previous cache behavior can be restored with `-cache.oldBehavior` command-line flag.
 2019-08-13 21:40:28 +03:00
Aliaksandr Valialkin
f56c1298ad app/vmstorage: add vm_concurrent_addrows_* metrics for tracking concurrency for Storage.AddRows calls 
Track also the number of dropped rows due to the exceeded timeout
on concurrency limit for Storage.AddRows. This number is tracked in `vm_concurrent_addrows_dropped_rows_total`
 2019-08-06 15:08:43 +03:00
Aliaksandr Valialkin
8e05758ff5 app: add vm_concurrent_ metrics for visibility in concurrency limiters for vminsert and vmselect 2019-08-05 18:30:29 +03:00
Aliaksandr Valialkin
53c8f56436 app/vmselect: allow passing match[], start and time to /api/v1/label/<label_name>/values 
`/api/v1/label/<label_name>/values?match[]=q` emulates emulates `label_values(q, <label_name>)`
call in Grafana templating.
 2019-08-04 23:07:00 +03:00
Aliaksandr Valialkin
880b1d80b1 app/vmselect: optimize /api/v1/series by skipping storage data 
Fetch and process only time series metainfo.
 2019-08-04 23:00:46 +03:00
Aliaksandr Valialkin
7f5afae1e3 app/vmselect/prometheus: prevent from fetching and scanning all the data on /api/v1/searies call by default 2019-08-04 19:42:45 +03:00
Aliaksandr Valialkin
000c154641 app/vmselect/promql: tune automatic window adjustement 
Increase the windows adjustement for small scrape intervals,
since they usually have higher jitter.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/139
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/134
 2019-08-04 19:34:11 +03:00
Aliaksandr Valialkin
1d4ddadbb1 app/vmselect/promql: further increase the allowed jitter for scrape interval 
Real-world production data shows higher jitter than 1/8 of scrape interval.
This may results in gaps on the graph. So increase the allowed jitter to 1/4
of scrape interval in order to reduce the probability of gaps on the graphs
over time series with high jitter for scrape_interval.
 2019-08-02 20:16:41 +03:00