Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-15 16:30:55 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
912 Commits 114 Branches 748 Tags 504 MiB
a53e332a93
Commit Graph

912 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Aliaksandr Valialkin
a53e332a93 app/vmstorage: add missing shutdown for http server on graceful shutdown 
This could result in the following panic during graceful shutdown when `/metrics` page is requested:

http: panic serving 10.101.66.5:57366: runtime error: invalid memory address or nil pointer dereference
goroutine 2050 [running]:
net/http.(*conn).serve.func1(0xc00ef22000)
	net/http/server.go:1772 +0x139
panic(0xa0fc00, 0xe91d80)
	runtime/panic.go:973 +0x3e3
github.com/VictoriaMetrics/VictoriaMetrics/lib/workingsetcache.(*Cache).UpdateStats(0x0, 0xc0000516c8)
	github.com/VictoriaMetrics/VictoriaMetrics/lib/workingsetcache/cache.go:224 +0x37
github.com/VictoriaMetrics/VictoriaMetrics/lib/storage.(*indexDB).UpdateMetrics(0xc00b931d00, 0xc02c41acf8)
	github.com/VictoriaMetrics/VictoriaMetrics/lib/storage/index_db.go:258 +0x9f
github.com/VictoriaMetrics/VictoriaMetrics/lib/storage.(*Storage).UpdateMetrics(0xc0000bc7e0, 0xc02c41ac00)
	github.com/VictoriaMetrics/VictoriaMetrics/lib/storage/storage.go:413 +0x4c5
main.registerStorageMetrics.func1(0x0)
	github.com/VictoriaMetrics/VictoriaMetrics/app/vmstorage/main.go:186 +0xd9
main.registerStorageMetrics.func3(0xc00008c380)
	github.com/VictoriaMetrics/VictoriaMetrics/app/vmstorage/main.go:196 +0x26
main.registerStorageMetrics.func7(0xc)
	github.com/VictoriaMetrics/VictoriaMetrics/app/vmstorage/main.go:211 +0x26
github.com/VictoriaMetrics/metrics.(*Gauge).marshalTo(0xc000010148, 0xaa407d, 0x20, 0xb50d60, 0xc005319890)
	github.com/VictoriaMetrics/metrics@v1.11.2/gauge.go:38 +0x3f
github.com/VictoriaMetrics/metrics.(*Set).WritePrometheus(0xc000084300, 0x7fd56809c940, 0xc005319860)
	github.com/VictoriaMetrics/metrics@v1.11.2/set.go:51 +0x1e1
github.com/VictoriaMetrics/metrics.WritePrometheus(0x7fd56809c940, 0xc005319860, 0xa16f01)
	github.com/VictoriaMetrics/metrics@v1.11.2/metrics.go:42 +0x41
github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver.writePrometheusMetrics(0x7fd56809c940, 0xc005319860)
	github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver/metrics.go:16 +0x44
github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver.handlerWrapper(0xb5a120, 0xc005319860, 0xc005018f00, 0xc00002cc90)
	github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver/httpserver.go:154 +0x58d
github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver.gzipHandler.func1(0xb5a120, 0xc005319860, 0xc005018f00)
	github.com/VictoriaMetrics/VictoriaMetrics/lib/httpserver/httpserver.go:119 +0x8e
net/http.HandlerFunc.ServeHTTP(0xc00002d110, 0xb5a660, 0xc0044141c0, 0xc005018f00)
	net/http/server.go:2012 +0x44
net/http.serverHandler.ServeHTTP(0xc004414000, 0xb5a660, 0xc0044141c0, 0xc005018f00)
	net/http/server.go:2807 +0xa3
net/http.(*conn).serve(0xc00ef22000, 0xb5bf60, 0xc010532080)
	net/http/server.go:1895 +0x86c
created by net/http.(*Server).Serve
	net/http/server.go:2933 +0x35c
 2020-04-02 21:09:55 +03:00
Aliaksandr Valialkin
bf43ad1d4f docs/Articles.md: move Percona article to third-party 2020-04-02 15:43:19 +03:00
Aliaksandr Valialkin
b6ff251884 docs/Articles.md: add a link to https://blog.cloudera.com/benchmarking-time-series-workloads-on-apache-kudu-using-tsbs/ 2020-04-02 15:41:30 +03:00
Aliaksandr Valialkin
cfea171930 docs/CaseStudies.md: add Adsterra case 2020-04-02 00:49:49 +03:00
Aliaksandr Valialkin
3b744f3c32 app/vmstorage: typo fix 2020-04-01 23:43:09 +03:00
Aliaksandr Valialkin
f838cdc86e app/vmstorage: add vm_free_disk_space_bytes metric for monitoring the remaining disk space at -storageDataPath 2020-04-01 23:10:44 +03:00
Aliaksandr Valialkin
8a02e01210 docs/Single-server-VictoriaMetrics.md: re-organize chapters 2020-04-01 22:41:00 +03:00
Aliaksandr Valialkin
84fa146792 lib/httpserver: remove unnecessary http.HandlerFunc wrapper in gzipHandler 2020-04-01 18:14:47 +03:00
Aliaksandr Valialkin
b3cb188c59 docs/Single-server-VictoriaMetrics.md: small fixes and updates 2020-04-01 18:11:10 +03:00
Aliaksandr Valialkin
d5180dbe78 docs/Cluster-VictoriaMetrics.md: small fixes and updates 2020-04-01 18:06:01 +03:00
Aliaksandr Valialkin
120d452002 docs/Cluster-VictoriaMetrics.md: swap production build and development build chapters 2020-04-01 17:48:43 +03:00
Aliaksandr Valialkin
0ad7aaf535 lib/storage: add missing reset for tagFilter.matchesEmptyValue on tagFilter.Init 2020-04-01 17:40:27 +03:00
Aliaksandr Valialkin
c189104be7 lib/promscrape: reduce timestamp jitter when scraping targets 
This should improve compression for timestamps
 2020-04-01 16:13:01 +03:00
Aliaksandr Valialkin
4c56acbafa lib/storage: remove duplicate data points on 7/8*minScrapeInterval interval instead of 1/2*minScrapeInterval 
This should reduce storage usage and should improve deduplication accuracy
 2020-04-01 15:47:04 +03:00
Aliaksandr Valialkin
29d5fbfcd8 docs/Single-server-VictoriaMetrics.md: mention that environment vars may be prefixed with -envflag.prefix 2020-03-31 22:38:26 +03:00
Aliaksandr Valialkin
5792f7296a README.md: mention that response cache must be reset after import historical data 2020-03-31 19:34:06 +03:00
Aliaksandr Valialkin
504ea876f2 lib/storage: handle errors returned from TagFilters.Add when cloning TagFilters with negative filter 2020-03-31 16:18:34 +03:00
Aliaksandr Valialkin
5270b7a097 app/victoria-metrics/testdata: add a test for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/395 2020-03-31 12:51:46 +03:00
Aliaksandr Valialkin
ef714e01c1 lib/storage: add fast path for the previous indexdb search if it doesn't contain per-day inverted index yet 2020-03-31 12:35:15 +03:00
Aliaksandr Valialkin
7e755b4bac lib/storage: optimize per-day inverted index search for tag filters matching big number of time series 
- Sort tag filters in the ascending number of matching time series
  in order to apply the most specific filters first.
- Fall back to metricName search for filters matching big number of time series
  (usually this are negative filters or regexp filters).
 2020-03-31 00:53:29 +03:00
Aliaksandr Valialkin
d450249955 lib/storage: properly handle {label=~"foo|"} filters as Prometheus does 
Such filters must match all the time series with `label="foo"` plus all the time series without `label`

Previously only time series with `label="foo"` were matched.
 2020-03-30 20:21:47 +03:00
Aliaksandr Valialkin
b47444e69d lib/envflag: add -envflag.prefix for setting optional prefix for environment vars 2020-03-30 15:51:44 +03:00
Aliaksandr Valialkin
e6e321f542 vendor: make vendor-update 2020-03-30 15:08:00 +03:00
Aliaksandr Valialkin
f4c3a71139 go.mod: update the minimum required Go version from go1.12 to go1.13 2020-03-30 14:57:59 +03:00
Aliaksandr Valialkin
c6cbc0bd19 app/vmselect/prometheus: allow passing relative time to start, end and time args of /api/v1/* queries 2020-03-29 21:56:52 +03:00
Aliaksandr Valialkin
cb8696699a app/vmselect/prometheus: code simplification: (d.Seconds()/1e3) -> d.Milliseconds() 2020-03-29 21:50:35 +03:00
kreedom
f058efb3d1 [vmalert] config parser (#393) 
* [vmalert] config parser

* make linter be happy

* fix test

* fix sprintf add test for rule validation
 2020-03-29 01:49:40 +02:00
Aliaksandr Valialkin
c66a13bf0f docs: add robots.txt 2020-03-28 23:23:20 +02:00
Aliaksandr Valialkin
ceb6d1459f docs/vmagent.md: add prometheus remote_write proxy use case 2020-03-28 23:17:41 +02:00
Aliaksandr Valialkin
8d55af4e75 docs/CaseStudies.md: add Brandwatch case study 2020-03-28 20:58:46 +02:00
Aliaksandr Valialkin
253844b74c deployment/docker: run docker apps under default user (0, root) in order to preserve backwards compatibility 
If docker app is upgraded from root to non-root, then the data pointed by `-storageDataPath` or similar flags
becomes denied to non-root user after the upgrade. This breaks upgrade path. So revert back to default root user
for docker apps.

Users may explicitly execute `docker run --user <non_root_user>` for running docker apps under non-root user.
 2020-03-28 19:23:13 +02:00
Roman Khavronenko
a2767fe86f bump Prometheus and Grafana images (#389) 2020-03-28 01:19:19 +02:00
Roman Khavronenko
9373a62f8a
Update dashboard according to new Grafana version and some metric renames. (#392) 
The list of changes is following:
* fix Uptime panel column styles according to changes introduced in 6.7 Grafana version
* fix panel `vminsert/Rows per insert` due to metric rename - see #336
* change default datasource to VictoriaMetrics since dashboard now uses MetricsQL for `vminsert/Rows per insert` panel
 2020-03-28 01:17:38 +02:00
Dmitry Naumov
b84071fc25
Rootless docker images by default (#358) 
* Rootless docker images by default

* Migrate to rootless base image

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
 2020-03-27 21:18:32 +02:00
Aliaksandr Valialkin
b803bcca6b vendor: make vendor-update 2020-03-27 20:17:26 +02:00
Aliaksandr Valialkin
42c290ce9f lib/httpserver: add -http.maxGracefulShutdownDuration command-line flag for tuning the maximum duration required for graceful shutdown of http server 2020-03-27 20:10:05 +02:00
Aliaksandr Valialkin
8fa80a2dbc lib/uint64set: remove zero buckets after Set.Intersect 2020-03-27 01:16:34 +02:00
Aliaksandr Valialkin
7a35447031 lib/uint64set: small code cleanup and perf tuning 
* Remember the last accessed bucket on Has() call.
* Inline fast paths inside Add() and Has() calls.
* Remove fragile code with maxUnsortedBuckets inside bucket32.
 2020-03-25 15:29:59 +02:00
Aliaksandr Valialkin
19d93e1a2e deployment/docker: update Go builder from Go1.14.0 to Go1.14.1 2020-03-24 22:35:46 +02:00
Aliaksandr Valialkin
cce936de5b lib/uint64set: go fmt 2020-03-24 22:28:09 +02:00
Aliaksandr Valialkin
7cdac6634c lib/storage: serialize snapshot creation process with mutex 
This guarantees that the snapshot contains all the recently added data
from inmemory buffers when multiple concurrent calls to Storage.CreateSnapshot are performed.
 2020-03-24 22:27:28 +02:00
Aliaksandr Valialkin
c31b956355 lib/uint64set: added more tests 2020-03-24 22:27:26 +02:00
Aliaksandr Valialkin
8fa9066b98 docs/CaseStudies.md: added a case study from MHI Vestas Offshore Wind 2020-03-14 13:22:41 +02:00
Aliaksandr Valialkin
31a533656e lib/storage: remove obsolete code 2020-03-13 22:42:42 +02:00
Aliaksandr Valialkin
58cb7fc476 app/vmselect: adjust label_map() handling for corner cases 
The following corner cases now supported:
* label_map(q, "label", "", "foo") - adds `label="foo"` to series with missing `label`
* label_map(q, "label", "foo", "") - removes `label="foo"` from series

All the unmatched labels are kept unchanged.
 2020-03-13 18:41:52 +02:00
Aliaksandr Valialkin
5c5a30734e vendor: update github.com/VictoriaMetrics/metrics from v1.11.0 to v1.11.2 
This fixes data race in Histogram
 2020-03-13 12:40:15 +02:00
Aliaksandr Valialkin
bf1869d33d lib/promscrape: allow overriding external_labels as Prometheus does 
Prometheus docs at https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config say:

> In communication with external systems, they are always applied only
> when a time series does not have a given label yet and are ignored otherwise.

Though this may result in consistency chaos when scrape targets override `external_labels`,
let's stick with Prometheus behavior for the sake of backwards compatibility.

There is last resort in vmagent with `-remoteWrite.label`, which consistently
sets the configured labels to all the metrics before sending them to remote storage.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/366
 2020-03-12 20:25:27 +02:00
Aliaksandr Valialkin
0e7a71a245 app/vmselect: add label_map(q, label, srcValue1, dstValue1, ... srcValueN, dstValueN) function to MetricsQL 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/369
 2020-03-12 19:13:56 +02:00
Aliaksandr Valialkin
39a977b5aa vendor: update google.golang.org/genproto from fc8f55426688 to da6875a35672 2020-03-12 18:12:46 +02:00
Aliaksandr Valialkin
9ef7bba17b vendor: update golang.org/x/tools from 26f6a1b6802d to 5e2df02acb1e 2020-03-12 18:08:02 +02:00