Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-26 20:30:10 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
2,337 Commits 113 Branches 748 Tags 719 MiB
f4135b0d14
Commit Graph

2337 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Aliaksandr Valialkin
ddabc13796 app/vmagent: properly flush big blocks of data 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/741

Thanks to @IceRain00 for the investigation and initial attempt to fix the issue
at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/742
 2020-09-03 12:12:39 +03:00
Aliaksandr Valialkin
7a839b461f app/vmagent: fix data race when accessing writeRequest.lastFlushTime 2020-09-03 12:12:37 +03:00
Nikolay Khramchikhin
764b3d4fda
changed vmalert behaviour (#738) 
* VMAlert start with empty rules dir

There are some applications (operator for instance), that generates alerts configuration at runtime
and vmalert must start correctly without rules to support this behaviour.
Later application will add rules files and send SIGHUP to vmalert,
which will trigger reading rules files and start rules exectuion.

Removing rules files with SIGHUP signal must stop rules execution and
vmalert will wait for new rules.

* imports sorted

* added test cases for empty rules, removed blank line

* fixed imports conflict

* updated tests
 2020-09-03 11:04:42 +03:00
Aliaksandr Valialkin
b4afc6ee2f docs/Single-server-VictoriaMetrics.md: add missing link to Prometheus text exposition format 2020-09-03 01:10:11 +03:00
Aliaksandr Valialkin
5f16ceb294 app/vmalert: imrovements over 3f932c2db1 2020-09-03 01:00:55 +03:00
DexterZhang
3f932c2db1
feat: spread load of rule evaluation by group when starting new groups (#724) 
* feat: spread load of rule evaluation by group when starting new groups

* review: reduce the resulting diff.

* Update app/vmalert/group.go

Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>
 2020-09-03 00:58:54 +03:00
Aliaksandr Valialkin
f41b36bb9a app/{vminsert,vmagent}: allow adding extra labels when importing data via Prometheus, CSV and JSON line formats 
Extra labels may be added to the imported data by passing `extra_label=name=value` query args.
Multiple query args may be passed in order to add multiple extra labels.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/719
 2020-09-02 19:43:21 +03:00
Aliaksandr Valialkin
038358b777 lib/promscrape: use the number of parsed rows as a basis for writeRequestCtxPool leveling 
The previous basis on `cap(sw.labels)` doesn't work anymore after 7785869ccc ,
because `sw.labels` may be reset multiple times when processing big number of rows.
 2020-09-02 18:46:01 +03:00
Roman Khavronenko
ed899ca9e8
Single dashboards update (#736) 
* dashboard: rename var `datasource` to `ds` for consistency reason

Dasbhoards for cluster version or vmagent operate with datasource variable
named `ds`. For consistency sake we rename this variable in single node version
as well.

* dashboard: add instance variable picker

See dashboard reviews here https://grafana.com/grafana/dashboards/10229/reviews

* dashboard: limit number of buckets in histogram to 12 for vmagent dashboard

* dashboard: bump version requirement in description for single version

* dashboard: drop extra series override for single version

* dashboard: set Y-min to zero for most of panels in vmagent dashboard
 2020-09-02 15:16:40 +03:00
Aliaksandr Valialkin
e9196655dd deployment/docker: update Go builder from v1.15.0 to v1.15.1 2020-09-02 15:10:15 +03:00
Aliaksandr Valialkin
821df709d3 vendor: make vendor-update 2020-09-02 15:05:16 +03:00
John Belmonte
67277abecf
use Y-min 0 on Grafana dashboard graphs (#732) 2020-09-01 19:56:56 +01:00
Aliaksandr Valialkin
c2ff8de456 lib/httpserver: add -http.idleConnTimeout command-line flag for tuning the timeout for incoming idle http connections 2020-09-01 15:33:24 +03:00
Aliaksandr Valialkin
b059f194e4 lib/promscrape: fix applying sample_limit when scraping targets with big number of metrics 
This has been broken at 7785869ccc
 2020-09-01 11:08:13 +03:00
Aliaksandr Valialkin
7785869ccc lib/promscrape: reduce memory usage when scraping targets with millions of metrics 
This should help when scraping /federate endpoints from Prometheus instances,
which scrape millions of metrics. See https://prometheus.io/docs/prometheus/latest/federation/
 2020-09-01 10:57:07 +03:00
Aliaksandr Valialkin
5af777469a app/vmagent: log unsuccessful attempt number when sending data to -remoteWrite.url 2020-08-30 21:40:22 +03:00
Aliaksandr Valialkin
2149733bd2 app/vmagent: apply sane limits to -remoteWrite.queues 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/707
 2020-08-30 21:25:37 +03:00
Aliaksandr Valialkin
dd20784d06 docs/Single-server-VictoriaMetrics.md: mention that VictoriaMetrics accepts relative times at time, start and end query args 2020-08-28 10:13:16 +03:00
Aliaksandr Valialkin
de6970e828 docs/vmalert.md: sync with app/vmalert/README.md via make docs-update 2020-08-28 09:51:48 +03:00
Aliaksandr Valialkin
4a415620d3 docs/Articles.md: add a link to https://medium.com/@romanhavronenko/victoriametrics-how-to-migrate-data-from-prometheus-filtering-and-modifying-time-series-6d40cea4bf21 2020-08-28 09:51:26 +03:00
Aliaksandr Valialkin
acbcad1ece lib/{promscrape,leveledbytebufferpool}: rename getPoolIdAndCapacity to getPoolIDAndCapacity in order to make golint happy 2020-08-28 09:49:32 +03:00
Aliaksandr Valialkin
f4c4ab811b lib/cgroup: limit the maximum GOMAXPROCS value to the number of available CPU cores 
There is no sense in setting GOMAXPROCS to value higher than the number of available CPU cores.
 2020-08-28 09:49:32 +03:00
Roman Khavronenko
10601bc652
vmalert: update -rule flag description to enforce quotes using (#709) 
Description for `-rule` flag uses as example specific chars like asterisks
which could be interpreted wrong by different shells. To avoid this, description
now contains quoted flag values.

See also #708
 2020-08-20 22:36:38 +01:00
Roman Khavronenko
f2c004d1ae
lib/flagutil: avoid int overflow for arch 386 (#710) 
Arch 386 is a 32-bit architecture and interprets int type for numbers as an explicit int32,
whereas on most modern CPUs int is implicitly an int64. This makes tests to fail with
`int overflow` error.
 2020-08-20 22:27:37 +01:00
Aliaksandr Valialkin
efc730863b lib/promscrape: reduce memory usage when scraping targets with big number of metrics alongside targets with small number of labels 
Previously targets with big number of metrics and/or labels could generated too big buffers,
which then could be re-used when scraping targets with small number of metrics.
This resulted in memory waste.

Now big buffers are used only for targets with big number of metrics / labels,
while small buffers are used for targets with small number of metrics / labels.
 2020-08-16 22:29:51 +03:00
Aliaksandr Valialkin
d6967319b6 lib/leveledbytebufferpool: allocate byte buffers with capacity rounded to the upper boundary for the given bucket 
This should reduce the number of resizings for the returned byte buffers.
 2020-08-16 22:13:30 +03:00
Roman Khavronenko
f5f59896ec
lib/decimal: rename significant decimal digits to significant figures (#698) 
The previous notion was inconsistent with what `decimal.Round` does.
According to [wiki](https://en.wikipedia.org/wiki/Significant_figures) rounding
applied to all significant figures, not just decimal ones.
 2020-08-16 17:21:35 +03:00
Aliaksandr Valialkin
147c35ebd4 all: allow using KB, MB, GB, KiB, MiB and GiB suffixes in command-line flag values related to byte sizes or byte rates 2020-08-16 17:05:52 +03:00
Aliaksandr Valialkin
7c0d6a8b88 lib/memory: improve log message about the memory allowed to use by VictoriaMetrics 2020-08-16 16:04:11 +03:00
Aliaksandr Valialkin
ed00eb3f33 lib/protoparser: removed unnecessary call to SetReadDeadline when reading a stream of data 
The OS should return any buffered data in the stream without the need to set the read timeout.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/696
 2020-08-15 15:38:08 +03:00
Aliaksandr Valialkin
7615a3ab8d vendor: upgrade github.com/valyala/gozstd from v1.7.1 to v1.8.3 2020-08-15 15:11:56 +03:00
Aliaksandr Valialkin
7be9bedaf9 vendor: downgrade github.com/valyala/gozstd from v1.8.1 to v1.7.1 until https://github.com/facebook/zstd/issues/2222 is fixed 2020-08-15 14:46:32 +03:00
Aliaksandr Valialkin
00b1659dde lib: dump compressed block contents on error during decompression 
This should improve detecting root cause for https://github.com/facebook/zstd/issues/2222
 2020-08-15 14:44:33 +03:00
Aliaksandr Valialkin
528e25bdde vendor: update github.com/valyala/gozstd from v1.7.0 to v1.8.1 2020-08-15 13:46:43 +03:00
Aliaksandr Valialkin
b3849a90fd lib/leveledbytebufferpool: pre-allocate byte slice with the given capacity if the pool is empty 
This should reduce memory allocations and copying when the byte slice is growing.
 2020-08-15 01:40:54 +03:00
Aliaksandr Valialkin
7d89fafe1a app/vmselect/promql: allow passing multiple args to aggregate functions such as avg(q1, q2, q3) 2020-08-15 01:15:09 +03:00
Aliaksandr Valialkin
cd96248480 docs/vmagent.md: mention that gaps in remote storage may appear if vmagent cannot keep up with data ingestion 2020-08-14 20:47:57 +03:00
Aliaksandr Valialkin
7554be172d lib/protoparser: move common code for detecting timeouts to ReadLinesBlockExt 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/696
 2020-08-14 20:40:15 +03:00
Aliaksandr Valialkin
4beab7ad39 lib/protoparser: prevent from busy loop on repeated timeout errors when reading streams of ingested data 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/696
 2020-08-14 20:14:11 +03:00
Aliaksandr Valialkin
41d23f84ed docs/Cluster-VictoriaMetrics.md: sync with upstream 2020-08-14 19:15:29 +03:00
Aliaksandr Valialkin
184670fb9b docs: update docs 2020-08-14 19:13:42 +03:00
Aliaksandr Valialkin
52791fd1c0 lib/memory: add -memory.allowedBytes command-line flag for setting absolute memory limit for VictoriaMetrics caches 2020-08-14 19:13:38 +03:00
Aliaksandr Valialkin
576da0fe46 app/{vminsert,vmagent}: improve documentation for -influxListenAddr command-line flag 2020-08-14 18:04:44 +03:00
Aliaksandr Valialkin
215967437d lib/protoparser/prometheus: typo fix in error message 2020-08-14 11:04:23 +03:00
Aliaksandr Valialkin
d1ad3adcbe vendor: make vendor-update 2020-08-14 02:29:02 +03:00
Aliaksandr Valialkin
42960feff4 vendor: update github.com/VictoriaMetrics/fasthttp from v1.0.4 to v1.0.5 2020-08-14 02:19:36 +03:00
Aliaksandr Valialkin
07246bc31c vendor: update github.com/klauspost/compress from v1.10.10 to v1.10.11 2020-08-14 02:17:07 +03:00
Aliaksandr Valialkin
e646674b23 lib/promscrape: use a hint on body length instead of body capacity 
This should reduce memory usage for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/689
 2020-08-14 01:17:52 +03:00
Aliaksandr Valialkin
4628deecd1 lib/promscrape: reduce memory usage when scraping big number of targets 
Thanks to @dxtrzhang for the original idea at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/688

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/689
 2020-08-14 01:04:53 +03:00
Aliaksandr Valialkin
eead3ee8ec lib/promscrape: properly retry requests on the server closed connection before returning the first response byte error during service discover API calls and target scrapes 2020-08-13 22:31:52 +03:00