Aliaksandr Valialkin
67cacb22ac
lib/httpserver: add -tls
, -tlsCertFile
and -tlsKeyFile
command-line flags in every vm binary
...
This makes such binaries compatible with binaries from `master` branch (aka single-node version)
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/677
2020-08-07 10:57:32 +03:00
Aliaksandr Valialkin
307281e922
lib/storage: slow down concurrent searches when the number of concurrent inserts reaches the limit
...
This should improve data ingestion performance when heavy searches are executed
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/618
2020-08-07 08:49:13 +03:00
Aliaksandr Valialkin
dd1d59f57a
lib/storage: properly check timeouts and pace limits
...
Previously they were checked on every iteration for small number of iterations
2020-08-07 08:40:56 +03:00
Aliaksandr Valialkin
2704722b6d
deployment/docker: update Go builder from v1.14.6 to v1.14.7
2020-08-07 08:29:25 +03:00
Aliaksandr Valialkin
87d7710c8e
docs/MetricsQL.md: mention that MetricsQL removes all the NaN values from results
2020-08-07 07:51:40 +03:00
Aliaksandr Valialkin
95a8c492ef
app/vmselect/promql: properly handle -n^m
like Prometheus does
...
`-n^m` must be handled as `-(n^m)` instead of `(-n)^m`.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/675
2020-08-07 07:42:42 +03:00
Aliaksandr Valialkin
7f93d61a56
app/vmselect/promql: remove metric name after applying clamp_min
and clamp_max
functions in order to be consistent with Prometheus
...
This improves VictoriaMetrics score at https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 23:42:55 +03:00
Aliaksandr Valialkin
01000505a0
app/vmselect/promql: remove metric name after applying ceil
, floor
and round
functions in order to be more consistent with Prometheus
...
This improves VictoriaMetrics score at https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 23:34:03 +03:00
Aliaksandr Valialkin
75bff1a567
app/vmselect/promql: remove metric name from results of certain rollup functions in order to be consistent with Prometheus
...
Rollup functions:
- avg_over_time
- min_over_time
- max_over_time
- quantile_over_time
This improves VictoriaMetrics results at https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 23:29:18 +03:00
Aliaksandr Valialkin
8835004a4c
app/vmselect: properly handle PromQL queries like scalar1 < metric < scalar2
like Prometheus does
...
This fixes some cases from https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 23:21:14 +03:00
Aliaksandr Valialkin
3947307363
vendor: update github.com/VictoriaMetrics/metricsql from v0.2.10 to v0.3.0
...
This adds support for special integers in MetricsQL that start from 0x, 0b, 0o.
This improves compatibility with PromQL - see https://promlabs.com/promql-compliance-test-results-victoriametrics/
2020-08-06 21:45:33 +03:00
Aliaksandr Valialkin
a2039b3bbc
app/vmselect: return the upper bound on the number of found time series from storage.Search.Init
...
This is used by a single-node version in order to reduce memory allocations during search.
See bc8381613d
for details.
2020-08-06 19:20:31 +03:00
Aliaksandr Valialkin
b690eeff53
lib/storage: reduce the frequency (and overhead) for timeout and pace limiter checks by 4x
2020-08-06 18:45:47 +03:00
Aliaksandr Valialkin
6c0a92a1ee
lib/pacelimiter: increase scalability for multi-CPU system
2020-08-06 18:33:07 +03:00
Aliaksandr Valialkin
14ddb8a34e
app/vmselect/netstorage: reduce CPU contention when upacking time series blocks by unpacking batches of such blocks instead of a single block
...
This should improve query performance on systems with big number of CPU cores (16 and more)
2020-08-06 17:50:13 +03:00
Aliaksandr Valialkin
46c98cd97a
app/vmselect/netstorage: reduce contention on unpackworkCh and timeseriesWorkCh for multi-CPU system by providing more capacity for these chans
2020-08-06 17:22:39 +03:00
Aliaksandr Valialkin
13f8644f8e
lib/storage: optimize prefetching metric names for the given metricIDs
2020-08-06 16:52:58 +03:00
Aliaksandr Valialkin
a455930ab4
app/vmstorage: rename vm_cache_size_entries{type="storage/prefetchedMetricIDs"}
to vm_cache_entries{type="storage/prefetchedMetricIDs"}
to be consistent with other vm_cache_entries
metrics
2020-08-06 16:34:18 +03:00
Aliaksandr Valialkin
f789e0fa44
lib/fs: export vm_nfs_pending_dirs_to_remove
metric for monitoring the number of pending directories that couldn't be removed due to NFS lock
2020-08-06 15:31:50 +03:00
Aliaksandr Valialkin
a3e91c593b
lib/storage: limit the number of concurrent calls to storage.searchTSIDs to GOMAXPROCS*2
...
This should limit the maximum memory usage and reduce CPU trashing on vmstorage
when multiple heavy queries are executed.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648
2020-08-05 18:27:21 +03:00
Aliaksandr Valialkin
76064ba9e7
Perform conversion from string to []byte according to rule #6 at https://golang.org/pkg/unsafe/#Pointer
2020-08-05 11:55:12 +03:00
Aliaksandr Valialkin
9038df211f
vendor: make vendor-update
2020-08-05 11:15:47 +03:00
Aliaksandr Valialkin
a930460236
app/vmagent: tune http client for sending data to remote storage in order to disable closing keep-alive connections
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/663
2020-08-04 21:01:40 +03:00
Aliaksandr Valialkin
a04f4a3d9a
app/vmselect: use warning level instead of info level for logging slow queries that take longer than -search.logSlowQueryDuration
2020-08-04 20:24:38 +03:00
Aliaksandr Valialkin
265206a6c3
docs/Single-server-VictoriaMetrics.md: add a chapter about data updates
2020-08-04 13:54:29 +03:00
Aliaksandr Valialkin
8cc2e01386
lib/backup: allow using ~/.aws/config
without region
...
Use us-west-2 for determining bucket region.
2020-08-04 13:08:05 +03:00
Aliaksandr Valialkin
bdb881c43b
app/vmselect/promql: add zscore-related functions: zscore_over_time(m[d])
and zscore(q) by (...)
2020-08-03 21:52:15 +03:00
Aliaksandr Valialkin
94471a1273
app: remove duplicate *-pure makefile rules
2020-07-31 20:01:30 +03:00
Aliaksandr Valialkin
a2aa3a60eb
app/vmselect: show X-Forwarded-For
contents on /api/v1/status/active_queries
page
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/659
2020-07-31 20:01:09 +03:00
Aliaksandr Valialkin
3149af624d
lib/storage: reduce the maximum number of concurrent merge workers to GOMAXPROCS/2
...
Previously the limit has been raised to GOMAXPROCS, but it has been appeared that this
increases query latencies since more CPUs are busy with merges.
While at it, substitute `*MergeConcurrencyLimitCh` channels with simple integer limits.
2020-07-31 17:53:13 +03:00
Aliaksandr Valialkin
106e302d7a
all: add mssing APP_NAME to vm*-GOARCH builds
2020-07-31 13:45:32 +03:00
Aliaksandr Valialkin
945645f38f
docs/{vmagent,vmalert}: add instruction on how to build for ARM
2020-07-31 09:25:41 +03:00
Aliaksandr Valialkin
a0eec52e6c
docs/Single-server-VictoriaMetrics.md: mention that downgrade is also safe to perform
2020-07-31 09:21:03 +03:00
Aliaksandr Valialkin
bbc7583015
vendor: update github.com/valyala/quicktemplate from v1.5.2 to v1.6.0
2020-07-30 23:39:48 +03:00
Aliaksandr Valialkin
0c00fe70cf
app/vmselect: do not adjust start
and end
query args passed to /api/v1/query_range
when -search.disableCache
command-line flag is set
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/563
2020-07-30 23:14:56 +03:00
Aliaksandr Valialkin
e9860b2fa3
docs/vmalert.md: sync with app/vmalert/README.md
2020-07-30 21:57:20 +03:00
Aliaksandr Valialkin
29bbab0ec9
lib/storage: remove prioritizing of merging small parts over merging big parts, since it doesn't work as expected
...
The prioritizing could lead to big merge starvation, which could end up in too big number of parts that must be merged into big parts.
Multiple big merges may be initiated after the migration from v1.39.0 or v1.39.1. It is OK - these merges should be finished soon,
which should return CPU and disk IO usage to normal levels.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/618
2020-07-30 20:02:22 +03:00
Aliaksandr Valialkin
96039dcb40
lib/storage: properly update vm_slow_row_inserts_total
metric when importing multiple data points per time series at once
...
Previously the `vm_slow_row_inserts_total` metric may be incremented multiple times for different data points per a single time series,
while only a single increment is needed when inserting the first data point for this time series.
2020-07-30 16:17:19 +03:00
Aliaksandr Valialkin
bafd475f2c
vendor: update github.com/valyala/quicktemplate from v1.5.1 to v1.5.2
2020-07-29 18:20:19 +03:00
Aliaksandr Valialkin
1e067401ba
lib/httpserver: emit X-Forwarded-For additionally to remoteAddr in error logs
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/659
2020-07-29 13:12:35 +03:00
Aliaksandr Valialkin
338ee47d60
app/vmselect/promql: return non-empty value from rate_over_sum(m[d])
even if a single data point is located in the given [d]
window
...
Just divide the data point value by the window duration in this case.
2020-07-29 12:37:34 +03:00
Aliaksandr Valialkin
717c554fb0
app/vmselect/promql: remove rollupFuncArg.realPrevValue handling, since the corner case in increase()
is handled in another way now
...
See e00cfc854d
for the approach used now.
2020-07-29 12:37:34 +03:00
Aliaksandr Valialkin
d9037b3970
app/vmselect/promql: fill gaps with 0 in rate_over_sum
response when the last value before the selected time window isnt empty
2020-07-29 12:37:34 +03:00
Aliaksandr Valialkin
f068ea74d0
vendor: make vendor-update
2020-07-29 09:35:34 +03:00
Aliaksandr Valialkin
ab47fa2300
vendor: update github.com/VictoriaMetrics/metrics from v1.12.1 to v1.12.2
2020-07-28 22:01:41 +03:00
Aliaksandr Valialkin
f6d4275087
app/{vmagent,vminsert}: properly preserve db
tag from query string passed to Influx line protocol query
...
Previously `db` tag from the query string wasn't added to metrics after encountering `db` tag in the Influx line
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/653
2020-07-28 21:25:49 +03:00
Aliaksandr Valialkin
baebe86844
app/vmagent/remotewrite: add missing resp.Body.Close()
after pushing data to remote storage
...
Missing body close could disable HTTP keep-alive connections.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/653
2020-07-28 21:00:25 +03:00
Aliaksandr Valialkin
0f6f0d30d3
app/vmselect: show query origin (aka remote_addr or client address) on the /api/v1/status/active_queries
page for every query
2020-07-28 15:14:40 +03:00
Roman Khavronenko
ec6ed467c6
app/vmalert: support external.label
to specify global labelset for all rules #622 ( #652 )
...
`external.label` flag supposed to help to distinguish alert or recording rules
source in situations when more than one `vmalert` runs for the same datasource
or AlertManager.
2020-07-28 14:23:04 +03:00
Aliaksandr Valialkin
9dccedc599
app/vmselect/promql: return empty values from group()
if all the time series have no values at the given timestamp
...
This aligns `group()` behaviour to Prometheus
2020-07-28 13:41:04 +03:00