Aliaksandr Valialkin
020bd8685e
lib/storage: removed duplicate checks for empty parts during merge - another check is in the beginning of mergeParts functions
2020-09-17 01:49:03 +03:00
Aliaksandr Valialkin
f2a449983d
vendor: make vendor-update
2020-09-17 01:43:19 +03:00
Aliaksandr Valialkin
8674963f6a
docs/Single-server-VictoriaMetrics.md: document that /api/v1/series/count may count delete time series
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/770
2020-09-17 01:38:17 +03:00
Aliaksandr Valialkin
ab53cb6f7b
app/vmagent: substitute -remoteWrite.url
with secret-url
value in logs, since it may contain sensitive info such as passwords or auth tokens
...
Pass `-remoteWrite.showURL` command-line flag in order to see real `-remoteWrite.url` values in logs and at `/metrics` page.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/773
2020-09-16 22:36:25 +03:00
Aliaksandr Valialkin
9f79bcf64a
app/vmselect: improve description for -search.maxQueryDuration
2020-09-16 21:15:41 +03:00
Aliaksandr Valialkin
39dee12ed7
lib/persistentqueue: code simplification after d455764a6f
2020-09-16 21:14:19 +03:00
Aliaksandr Valialkin
d455764a6f
lib/persistentqueue: make the persistent queue more durable against unclean shutdown (kill -9, OOM, hard reset)
...
The strategy is:
- Periodical flushing of inmemory blocks to files, so they aren't lost on unclean shutdown.
- Periodical syncing of metadata for persisted queues, so the metadata remains in sync with the persisted data.
- Automatic adjusting of too big chunk size when opening the queue. The chunk size may be bigger than the writer offset after unclean shutdown.
- Skipping of broken chunk file if it cannot be read.
- Fsyncing finalized chunk files.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/687
2020-09-16 18:13:44 +03:00
Aliaksandr Valialkin
ffadf035fa
lib/protoparser/vmimport: add more testcases for invalid timestamps and values
...
Updates https://github.com/VictoriaMetrics/vmctl/issues/25
2020-09-16 02:22:06 +03:00
Aliaksandr Valialkin
d8183c3124
lib/protoparser: report more errors for incorrect timestamps and/or values
...
Previously certain errors in timestamps and/or values could be silently skipped,
which could lead to samples with zero values stored in the database.
Updates https://github.com/VictoriaMetrics/vmctl/issues/25
2020-09-16 02:14:18 +03:00
Aliaksandr Valialkin
9bc8484ab6
lib/protoparser/graphite: return error when value or timestamp cannot be properly parsed
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/99
2020-09-16 01:35:12 +03:00
Aliaksandr Valialkin
26fa94ba8d
vendor: update github.com/valyala/fastjson from v1.5.4 to v1.5.5
...
This should properly parse `+Inf` values when importing JSON lines via `/api/v1/import`
Updates https://github.com/VictoriaMetrics/vmctl/issues/25
2020-09-16 00:07:56 +03:00
Aliaksandr Valialkin
0bccb58e80
docs/Single-server-VictoriaMetrics.md: mention that VictoriaMetrics ignores NaN and Inf values during data ingestion
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/752
2020-09-15 23:40:28 +03:00
Aliaksandr Valialkin
1fec47a289
app/vmselect/netstorage: reduce memory usage when the time range from query touches big number of samples per each time series
2020-09-15 21:08:28 +03:00
Aliaksandr Valialkin
8c3d7c1a59
app/vmselect: typo fix in -search.maxStalenessInterval description
2020-09-15 14:24:27 +03:00
Aliaksandr Valialkin
fa01169c3d
lib/promscrape: add a link to troubleshooting docs to error message when duplicate scrape target with identical labels is skipped
2020-09-15 14:16:05 +03:00
Aliaksandr Valialkin
51598bd718
docs/Articles.md: add a link to https://medium.com/miro-engineering/prometheus-high-availability-and-fault-tolerance-strategy-long-term-storage-with-victoriametrics-82f6f3f0409e
2020-09-15 12:29:10 +03:00
Aliaksandr Valialkin
ba74d0c14c
lib/promscrape: typo fix
2020-09-12 00:14:21 +03:00
Aliaksandr Valialkin
7d893a234c
lib/promscrape: do not reset the remaining rows when pushing a part of data to remote storage during big scrapes
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/753
Thanks to @PerGon and @clmssz for help with debugging.
2020-09-11 23:39:13 +03:00
Aliaksandr Valialkin
0e533d1a9c
app/vmselect/promql: support composite durations like Prometheus 2.21 does
...
The following durations are supported now: `1h5m35s` or `1s543ms`
See https://github.com/prometheus/prometheus/releases/tag/v2.21.0
and https://github.com/prometheus/prometheus/pull/7713
2020-09-11 23:39:13 +03:00
Aliaksandr Valialkin
0e19f35af5
lib/promscrape/discovery/dns: add __meta_dns_srv_record_target
and __meta_dns_srv_record_port
labels
...
This syncs dns service discovery with Prometheus 2.21 - see https://github.com/prometheus/prometheus/releases
and https://github.com/prometheus/prometheus/pull/7678 .
2020-09-11 23:39:13 +03:00
Roman Khavronenko
6ad6480400
vmalert: add Group name as label to generated alerts and timeseries ( #761 )
...
Solves #611
2020-09-11 20:52:56 +01:00
Roman Khavronenko
4cdffb04a4
vmalert: update groups on config reload only if changes detected ( #759 )
...
On config reload event `vmalert` reloads configuration for every group. While
it works for simple configurations, the more complex and heavy installations may
suffer from frequent config reloads.
The change introduces the `checksum` field for every group and is set to md5 hash
of yaml configuration. The checksum will change if on any change to group
definition like rules order or annotation change. Comparing the `checksum` field
on config reload event helps to detect if group should be updated.
The groups update is now done concurrently, so reload duration will be limited by
the slowest group now.
Partially solves #691 by improving config reload speed.
2020-09-11 20:14:30 +01:00
Aliaksandr Valialkin
ca856284e4
app/vmagent: allow setting multiple identical -remoteWrite.url
values
...
This may be useful when each url is authenticated via different `-remoteWrite.basicAuth.username`.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/755
2020-09-11 15:17:22 +03:00
Aliaksandr Valialkin
62fde80490
lib/protoparser/common: do not read request body when parsing timestamp
query arg
...
This was preventing from reading data via /api/v1/prometheus/import .
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/750
2020-09-11 14:44:58 +03:00
Aliaksandr Valialkin
5a90a92378
lib/storage: do not store inf values, since they may lead to significant precision loss for previously stored values
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/752
2020-09-11 14:44:53 +03:00
Aliaksandr Valialkin
a2f647d142
app/vmselect/prometheus: typo fix in the description for -search.latencyOffset
command-line flag
2020-09-11 14:16:46 +03:00
Aliaksandr Valialkin
f95eea60d1
lib/protoparser: accept timestamp in milliseconds instead of seconds at /api/v1/import/prometheus
...
This improves consistency with timestamps in Prometheus text exposition format
2020-09-11 14:04:46 +03:00
Aliaksandr Valialkin
2380e9b017
app/{vminsert,vmagent}: allow passing timestamp via timestamp
query arg when ingesting data to /api/v1/import/prometheus
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/750
2020-09-11 13:27:14 +03:00
Aliaksandr Valialkin
f0005c3007
app/vmselect: move Deadline from netstorage to searchutils
...
This removes dependency on netstorage from searchutils.
2020-09-11 13:27:13 +03:00
Aliaksandr Valialkin
2114179e19
app/vmselect: substitute inf values at smooth_exponential with the previous values
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/757
2020-09-11 12:24:14 +03:00
Nikolay Khramchikhin
6c80ae0da8
Added endpointslices discovery to k8s api ( #760 )
...
This is similar to https://github.com/prometheus/prometheus/pull/6838 , which will be added in Prometheus v2.21.
See https://github.com/prometheus/prometheus/releases/tag/v2.21.0-rc.1
* Added endpointslices discovery to k8s api
Started from 1.17 k8s version endpointslices is beta,
it allows to query k8s api for endpoints more efficient.
It presents at scrape_config.yaml as separate role for kubernetes_sd_config.
kubernetes_sd_config:
- role: endpointslices
* fixed typos, changed EndpointConditions signature - with values instead of pointers
2020-09-11 12:16:45 +03:00
Aliaksandr Valialkin
204ec415b4
app/vmselect: skip infinite values when calculating smooth_exponential
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/757
2020-09-11 11:29:58 +03:00
Aliaksandr Valialkin
8a8b5a73d3
app/vmselect/graphite: typo fix in label name for vm_request_duration_seconds metric
2020-09-11 01:58:28 +03:00
John Belmonte
c9d0905b17
fix typo on outliersk() doc ( #758 )
2020-09-11 00:55:53 +03:00
Aliaksandr Valialkin
f6bc608e86
app/vmselect: initial implementation of Graphite Metrics API
...
See https://graphite-api.readthedocs.io/en/latest/api.html#the-metrics-api
2020-09-11 00:30:01 +03:00
Aliaksandr Valialkin
3eccecd5fd
vendor: make vendor-update
2020-09-10 09:49:13 +03:00
Aliaksandr Valialkin
b3dcaf0cd7
deployment/docker: update Go builder from v1.15.1 to v1.15.2
...
This fixes the following issues in Go runtime - see https://github.com/golang/go/issues?q=milestone%3AGo1.15.2+label%3ACherryPickApproved
2020-09-10 09:36:43 +03:00
Aliaksandr Valialkin
9d8fdff6c5
lib/storage: reuse timestamp blocks for adjancent metric blocks with identical timestamps
...
This should reduce disk space usage when scraping targets containing metrics with identical names
such as `node_cpu_seconds_total`, histograms, quantiles, etc.
Expose `vm_timestamps_blocks_merged_total` and `vm_timestamps_bytes_saved_total` metrics for monitoring
the effectiveness of timestamp blocks merging.
2020-09-09 23:59:32 +03:00
Aliaksandr Valialkin
d7c04db1fc
docs: sync docs for vmalert, vmauth, vmbackup and vmrestore
2020-09-09 21:10:34 +03:00
Aliaksandr Valialkin
e5ed8c8d75
docs/Articles.md: add links to recently published third-party articles and talks about VictoriaMetrics
2020-09-09 20:15:27 +03:00
Aliaksandr Valialkin
9d431a4b45
docs/Single-server-VictoriaMetrics.md: typo fix
2020-09-09 01:21:45 +03:00
Aliaksandr Valialkin
4739dff6f0
docs/Single-server-VictoriaMetrics.md: typo fix
2020-09-09 00:59:37 +03:00
Aliaksandr Valialkin
11eaa37111
docs/vmagent.md: clarified the case when -remoteWrite.queues
must be tuned
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/745
2020-09-08 20:15:27 +03:00
Aliaksandr Valialkin
df169b1ebd
lib/httpserver: add a jitter to connection timeouts in order to protect from Thundering herd problem
2020-09-08 19:55:09 +03:00
Aliaksandr Valialkin
9d61d24142
vendor: make vendor-update
2020-09-08 15:20:01 +03:00
Aliaksandr Valialkin
62919eaf7e
app/vmselect/promql: go fmt
2020-09-08 15:19:59 +03:00
Aliaksandr Valialkin
e6da63dffe
app/vmselect/promql: adjust integrate()
calculations to be more similar to calculations from InfluxDB: attempt #2
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/701
2020-09-08 14:35:50 +03:00
Aliaksandr Valialkin
8e85b56737
app/vmselect/promql: adjust integrate()
calculations to be more similar to calculations from InfluxDB
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/701
2020-09-08 14:23:39 +03:00
Aliaksandr Valialkin
c0343a661b
app/vmselect/promql: increase floating point calculations accuracy by dividing by 1e3
instead of multiplying by 1e-3
2020-09-08 14:00:47 +03:00
Aliaksandr Valialkin
1bca6160a3
docs/Single-server-VictoriaMetrics.md: make docs-sync
2020-09-07 21:58:06 +03:00