Aliaksandr Valialkin
7d0743422b
lib/storage: properly calculate global metrics in UpdateStats()
2020-07-23 00:35:31 +03:00
Aliaksandr Valialkin
6afdcf8a20
lib/mergeset: properly calculate global metrics in UpdateStats()
...
Previously these metrics could be calculated multiple times for multiple mergeset.Table instances.
2020-07-23 00:35:29 +03:00
Aliaksandr Valialkin
23fa44e56e
lib/storage: reorder mergeBlockStreams() args in order to make them more consistent
2020-07-22 21:58:25 +03:00
Aliaksandr Valialkin
754eac676d
lib/storage: prevent possible race condition when all the goroutines exit Storage.AddRows, before goroutines other goroutines are blocked on searchTSIDsCond inside Storage.searchTSIDs
...
This condition may occur after the following sequence of events:
1) A goroutine enters the loop body when len(addRowsConcurrencyCh) == cap(addRowsConcurrencyCh) inside Storage.searchTSIDs.
2) All the goroutines return from Storage.AddRows.
3) The goroutine from step 1 blocks on searchTSIDsCond.Wait() inside the loop body.
The goroutine remains blocked until the next call to Storage.AddRows, which calls searchTSIDsCond.Signal().
This may take indefinite time.
2020-07-22 21:52:42 +03:00
Aliaksandr Valialkin
71c3266fca
docs/Single-server-VictoriaMetrics.md: mention that it is recommended inspecting logs during troubleshooting
2020-07-22 18:21:06 +03:00
Aliaksandr Valialkin
edbc777e91
vendor: make vendor-update
2020-07-22 16:54:02 +03:00
Aliaksandr Valialkin
20d0c41ac5
app/vmselect/prometheus: support d
, w
and y
suffixes for durations passed to step
in /api/v1/query_range
like Prometheus does
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/641
2020-07-22 16:27:27 +03:00
Aliaksandr Valialkin
bd4299fafe
app/vmselect/netstorage: reduce memory allocations when unpacking time series data by using a pool for unpackWork entries
...
This should slightly reduce load on GC when processing queries that touch big number of time series.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/646 according to the provided memory profile
2020-07-22 15:04:42 +03:00
Roman Khavronenko
9460bf782e
dashboards/victoriametrics: update Concurrent inserts
panel #632 ( #645 )
...
Panel `Concurrent inserts` was moved to `vminsert` row. Its metrics and description
was updated accordingly to #632 issue.
2020-07-22 12:43:23 +03:00
Aliaksandr Valialkin
a3f48e395e
app/vmagent: add -remoteWrite.decimalPlaces
command-line flag, which may be used for reducing disk space usage on the remote storage
2020-07-21 21:55:42 +03:00
Aliaksandr Valialkin
67be79a0bc
lib/uint64set: optimize adding items to the set via Set.AddMulti
2020-07-21 20:57:05 +03:00
Aliaksandr Valialkin
5bb4fe1ba4
app/vmselect: take into account the time spent in wait queue before query execution as time spent on the query
2020-07-21 19:00:00 +03:00
Aliaksandr Valialkin
0755cb3b50
app/vmselect/promql: skip the first value in time series passed to increase()
if it exceeds by more than 10x the delta between the next value and the first value
...
This should prvent from inflated `increase()` results for time series that start from big initial values.
Such cases may occur when a label value changes in a metric without counter reset.
2020-07-21 17:24:28 +03:00
Aliaksandr Valialkin
71eba8dcf5
app/vmselect: log the total available memory for concurrent requests on not enough memory
errors
...
This should simplify root cause analysis
2020-07-20 19:51:58 +03:00
Aliaksandr Valialkin
3b246aa569
app/vmagent: add -remoteWrite.proxyURL
command-line option
...
This option allows writing data to `-remoteWrite.url` via http, https or socks5 proxy.
This is similar to `proxy_url` option in `remote_write` section of Prometheus.
See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write
2020-07-20 19:31:08 +03:00
Aliaksandr Valialkin
8bee3ef91b
docs/vmagent.md: sync with app/vmagent/README.md
2020-07-20 17:09:30 +03:00
Roman Khavronenko
8949ec961d
app/vmagent: mention grafana dashboard in README ( #639 )
2020-07-20 17:09:27 +03:00
Aliaksandr Valialkin
77523af9fc
vendor: update github.com/VictoriaMetrics/metrics from v1.11.3 to v1.12.0
2020-07-20 16:56:44 +03:00
Aliaksandr Valialkin
86b54f3768
app/vmagent/remotewrite: allow passing empty -remoteWrite.urlRelabelConfig
entries
2020-07-20 15:49:13 +03:00
Aliaksandr Valialkin
141e84b5a4
app/vmselect/prometheus: do not return time series with empty list of datapoints from /api/v1/query_range
...
This matches Prometheus behaviour.
This should fix https://github.com/jacksontj/promxy/issues/329
2020-07-20 15:30:13 +03:00
Aliaksandr Valialkin
4d2011a87d
app/vmselect/promql: add mode()
aggregate function
2020-07-20 15:30:11 +03:00
Aliaksandr Valialkin
31ef39e8da
lib/httpserver: log remote address in error message from httpserver.Errorf
...
This should improve detection of the root cause of errors.
Thanks to Anant for the idea.
2020-07-20 14:06:29 +03:00
Aliaksandr Valialkin
427fa43ce2
app/vmselect/promql: add mode_over_time(m[d])
function
...
See https://en.wikipedia.org/wiki/Mode_(statistics) and https://stackoverflow.com/questions/61134078/promql-query-to-return-the-value-from-a-range-vector-which-occurs-maximum-no-of
2020-07-17 18:29:10 +03:00
Aliaksandr Valialkin
eb402a17bd
app/vmselect/promql: optimize group(rollup(m))
calculations
2020-07-17 16:47:30 +03:00
Aliaksandr Valialkin
ea8dc85ba8
app/vmselect/promql: check that any()
doesn't touch metric name
2020-07-17 16:23:11 +03:00
Aliaksandr Valialkin
b8b13e82e0
deployment/docker: update Go builder from v1.14.5 to v1.14.6
...
This fixes runtime issues found in Go since v1.14.5. See https://github.com/golang/go/issues?q=milestone%3AGo1.14.6+label%3ACherryPickApproved
2020-07-17 15:21:12 +03:00
Aliaksandr Valialkin
fc8fe38a82
app/vmselect/promql: add group()
aggregate function to MetricsQL
...
This function has been added in Prometheus 2.20. See https://github.com/prometheus/prometheus/pull/7480
2020-07-17 15:17:38 +03:00
Aliaksandr Valialkin
c64914a7e4
app/vmselect/promql: keep all labels for time series from any()
call
2020-07-17 15:17:37 +03:00
Roman Khavronenko
21cf6a1ec4
deployment/docker: replace Prometheus with vmagent ( #635 )
...
* replace Prometheus with vmagent in docker compose env;
* cluster dashboard: exclude vmagent from job list;
* cluster dasbhoard: reference datasource var instead of datasource name.
2020-07-17 02:18:03 +03:00
Roman Khavronenko
87946dcc53
vmagent: update grafana dashboard ( #634 )
...
* reference datasource variable instead of datasource name;
* change unit from `bytes` to `bits/s` for Network panel.
2020-07-17 02:12:20 +03:00
Aliaksandr Valialkin
f9b38f7f2d
app/vminsert/influx: properly handle the case when certain labels with empty values are removed by ApplyRelabeling() call
...
Previously this could lead to `out of range` panic
2020-07-17 00:05:24 +03:00
Aliaksandr Valialkin
14dc426b45
app/vmselect: fix nil pointer dereference
panic when unsuccessfully querying vmstorage
2020-07-16 19:15:18 +03:00
Aliaksandr Valialkin
490a42f592
deployment/docker: update Go builder from v1.14.4 to v1.14.5
...
This should fix the following issues in Go - https://github.com/golang/go/issues?q=milestone%3AGo1.14.5+label%3ACherryPickApproved
2020-07-16 18:55:37 +03:00
Roman Khavronenko
cb4c433260
vmagent: add grafana dashboard ( #629 )
...
`vmagent` Grafana dashboard suppose to provide basic observability over multiple
`vmagent` instances. Dashboard is saved in Grafana export format so it can be easily
imported. It was also integrated into docker-compose environment.
2020-07-15 13:58:30 +03:00
Aliaksandr Valialkin
ce381b3868
app/vmalert: consistently use "%w" instead of "%s" in fmt.Errorf
when wrapping errors
2020-07-15 13:55:13 +03:00
Aliaksandr Valialkin
e6d96bb0bd
docs/vmagent.md: make filtering rules for init container pods less confusing
2020-07-14 20:33:19 +03:00
Aliaksandr Valialkin
74fb0b293d
vendor: make vendor-update
2020-07-14 20:30:57 +03:00
Aliaksandr Valialkin
8e7c7a6fbd
docs/Single-server-VictoriaMetrics.md: remove Roadmap
chapter, since it became outdated
2020-07-14 19:07:06 +03:00
Aliaksandr Valialkin
c2b4b9138d
app/vmagent/remotewrite: return proper value from tssRelabelPool.New
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/599
2020-07-14 14:28:14 +03:00
Aliaksandr Valialkin
3365f6867b
docs/Single-server-VictoriaMetrics.md: sync with README.md
2020-07-14 14:19:53 +03:00
Aliaksandr Valialkin
86044f6561
app/{vminsert,vmagent}: add -influxSkipMeasurement
command-line flag for using field name as metric name
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/626
2020-07-14 14:18:40 +03:00
Aliaksandr Valialkin
be0ab4fbfe
lib/storage: reset MetricName->TSID
cache after marking metricIDs as deleted
...
This is a follow-up commit after 12b16077c4
,
which didn't reset the `tsidCache` in all the required places.
This could result in indefinite errors like:
missing metricName by metricID ...; this could be the case after unclean shutdown; deleting the metricID, so it could be re-created next time
Fix this by resetting the cache inside deleteMetricIDs function.
2020-07-14 14:05:19 +03:00
Aliaksandr Valialkin
0e7b2008b2
app/vmselect/prometheus: do not adjust last points in time series with timestamps exceeding the current time
...
Such timestamps usually mean that the query contains `offset`.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/625
2020-07-14 12:56:21 +03:00
Aliaksandr Valialkin
a4c96d9e6d
lib/protoparser: properly update vm_protoparser_rows_read_total{type="promscrape"}
metric
2020-07-14 12:15:56 +03:00
Seva Poliakov
a5e713b6e0
add vm_protoparser_rows_read_total metrics to promscrape ( #624 )
...
* add vm_protoparser_rows_read_total metrics to promscrape
move vm_protoparser_rows_read_total for promscrape to better place
move vm_protoparser_rows_read_total for promscrape to better place
* remove possibility of infinity loop at prometheus parser
2020-07-14 12:02:25 +03:00
Roman Khavronenko
207e93b50d
lib/flagutil: specify additional description for all Array type flags ( #620 )
...
Array type flag is now defined as `value` type in flag description when printed.
This change adds additional description to every Array type flag so it would be
clear what exact type is used:
```
-remoteWrite.urlRelabelConfig array
Optional path to relabel config for the corresponding -remoteWrite.url
Supports array of values separated by comma or specified via multiple flags.
```
2020-07-13 22:00:03 +03:00
Roman Khavronenko
605711bde5
lib/persistentqueue: add vm_persistentqueue_bytes_pending
metric ( #619 )
...
Metric `vm_persistentqueue_bytes_pending` is a gauge that shows current amount
of bytes in persistentqueue flushed on disk as a difference between write and read
offsets. This metric is very similar to `vmagent_remotewrite_pending_data_bytes`
except of accounting for bytes in-memory.
2020-07-13 21:54:54 +03:00
Roman Khavronenko
a02097e657
Extend metric vm_promscrape_targets
with status
label ( #615 )
...
The change to `vm_promscrape_targets` metric suppose to improve observability
for `vmagent` so it will be possible to track how many targets are up or down
for every specific scrape group:
```
vm_promscrape_targets{type="static_configs", status="down"} 1
vm_promscrape_targets{type="static_configs", status="up"} 2
```
2020-07-13 21:54:53 +03:00
Aliaksandr Valialkin
3898cc0285
app/vmselect/prometheus: minimize the diff for the change 1033dc7e2a
over 619b0a25c9
2020-07-13 21:41:17 +03:00
faceair
bf39e67ade
fix empty response template ( #617 )
2020-07-13 21:41:15 +03:00