Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-15 00:13:30 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
2,925 Commits 114 Branches 748 Tags 504 MiB
4e87638877
Commit Graph

323 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
4b25e627f8 lib/workingsetcache: properly switch to whole mode 
Previously the switch from `split` to `whole` mode had been performed too early,
e.g. when the current cache size became bigger than 1/4 of the allowed cache size.

Now it is performed when the current cache size becomes bigger than 1/2 of the allowed cache size.

This change can reduce memory usage for data ingestion path when big number of active time series are ingested.
 2021-07-05 15:15:39 +03:00
Aliaksandr Valialkin
28a778dc9f docs/CHANGELOG.md: document the bugfix for vm_merge_need_free_disk_space metric at 9a83e9018d 2021-07-05 12:01:24 +03:00
Aliaksandr Valialkin
1c12c0f79c lib/promauth: reload TLS certificates from disk on every mTLS connection as Prometheus does 
This allows updating client certificates without the need to restart vmagent and/or single-node VictoriaMetrics.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1420
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/470
 2021-07-01 15:43:43 +03:00
Aliaksandr Valialkin
c1b15c7764 docs/CHANGELOG.md: document ae485c2bfd 2021-07-01 11:52:47 +03:00
Aliaksandr Valialkin
71c856beb8 lib/workingsetcache: reset the cache mode when the cache is reset 
This should reduce memory usage if the working set is reduced after the cache reset.
 2021-07-01 11:52:47 +03:00
Aliaksandr Valialkin
0c4c630839 lib/promscrape: typo fix in /targets output 
The typo has been introduced in fb72a2133f

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1408
 2021-06-28 21:27:22 +03:00
Aliaksandr Valialkin
db66ea9a5f docs/CHANGELOG.md: cut v1.62.0 2021-06-25 13:29:48 +03:00
Aliaksandr Valialkin
97d1ccfc8e lib/promscrape: split docker and dockerswarm service discovery code bases, since they have very little in common 
This is a follow up after c85a5b7fcb
 2021-06-25 13:22:16 +03:00
Aliaksandr Valialkin
1b6850bab3 docs/CHANGELOG.md: document the bugfix in increase_pure() function from the commit fb4f758715 2021-06-24 12:06:15 +03:00
Aliaksandr Valialkin
b84aea1e6e lib/protoparser/clusternative: do not pool unmarshalWork structs, since they can occupy big amounts of memory (more than 100MB per each struct) 
This should reduce memory usage for vmstorage under high ingestion rate when the vmstorage runs on a system with big number of CPU cores
 2021-06-23 15:45:08 +03:00
Aliaksandr Valialkin
856aecae05 app/vmselect/promql: return the last timestamp for the max / min value from tmax_over_time() and tmin_over_time() function as most users expect 2021-06-23 14:18:37 +03:00
Aliaksandr Valialkin
dffa2afefe docs/CHANGELOG.md: document the bugfix for incorrect stats collection for concurrently executed tag filter 
Follow up for c22114c6f0
 2021-06-23 14:06:33 +03:00
Aliaksandr Valialkin
c18017a9c3 app/vminsert/netstorage: sort the -storageNode list passed to vminsert nodes 
This should reduce resource usage (CPU, RAM, disk IO) at vmstorage nodes
if the addresses of vmstorage nodes are passed in random order to vminsert nodes.
 2021-06-23 14:00:08 +03:00
Aliaksandr Valialkin
4adf6c9766 lib/promscrape/discovery/http: follow up after e307bbb29a 2021-06-22 13:42:10 +03:00
Aliaksandr Valialkin
3ab3902f17 lib/promscrape/discovery: support generic auth configs in Consul service discovery in the same way as Prometheus 2.28 does 2021-06-22 13:18:51 +03:00
Aliaksandr Valialkin
61fc9b98e5 docs/CHANGELOG.md: document the support for Consul namepsace 
See 58a2989fe7
 2021-06-22 12:56:12 +03:00
Aliaksandr Valialkin
23fcafd437 docs/CHANGELOG.md: typo fixes 2021-06-18 19:15:29 +03:00
Aliaksandr Valialkin
b92d110cad app/vmselect: log slow requests to all the /api/v1/* handlers if their execution time exceeds -search.logSlowQueryDuration 2021-06-18 19:07:03 +03:00
Aliaksandr Valialkin
4acc4602b3 app/vmctl: limit JSON line size by 10K samples (#1394) 
This should reduce the maximum memory usage at VictoriaMetrics when importing time series with big number of samples.
 2021-06-18 15:41:34 +03:00
Aliaksandr Valialkin
cd697b88c5 docs/CHANGELOG.md: document the reduced disk write IO usage 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338
 2021-06-18 14:03:42 +03:00
Aliaksandr Valialkin
f9069ba32a lib/promscrape: show jobs with empty scrape targets on /targets page 2021-06-18 10:54:12 +03:00
Aliaksandr Valialkin
644102b03b docs/CHANGELOG.md: document the changed -remoteWrite.queues value 
This is a follow-up for 0a796f7c3a

See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1385
 2021-06-16 12:37:55 +03:00
Aliaksandr Valialkin
b133de1e37 lib/storage: move deletedMetricIDs set from indexDB to Storage 
This makes consitent the list of deleted metricIDs when it is used from both the current indexDB and the previous indexDB (aka extDB).
This should fix the issue, which could lead to storing new samples under deleted metricIDs after indexDB rotation.
See more details at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347#issuecomment-861232136 .

Thanks to @tangqipengleoo for the initial analysis and the pull request - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 .

This commit resolves the issue in more generic way compared to https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 .

The downside of the commit is the deletedMetricIDs set isn't cleaned from the metricIDs outside the retention. It needs app restart.
This should be OK in most cases.
 2021-06-15 15:07:54 +03:00
Aliaksandr Valialkin
5f91a701fa lib/promscrape: show the number of samples collected during the last scrape at /targets and /api/v1/targets pages 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1377
 2021-06-14 14:04:35 +03:00
Aliaksandr Valialkin
5cd50d840f docs/CHANGELOG.md: document the addition of DigitalOcean service discovery 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1367
 2021-06-14 13:19:31 +03:00
Aliaksandr Valialkin
541429a9af docs/CHANGELOG.md: cut v1.61.1 2021-06-11 13:02:04 +03:00
Aliaksandr Valialkin
ce10bdc82a lib/storage: reset cache on disk during series deletion and during indexdb rotation 
This should prevent from inconsistent behavior (aka partially missing data for some time series) after unclean shutdown.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347
 2021-06-11 12:54:36 +03:00
Aliaksandr Valialkin
d5973d3180 docs/CHANGELOG.md: document the bugfix from 7adfe878e1 2021-06-11 11:30:07 +03:00
Aliaksandr Valialkin
83adc2f3ac docs/CHANGELOG.md: cut v1.61.0 2021-06-09 19:04:59 +03:00
Aliaksandr Valialkin
1e13deaa2c docs/CHANGELOG.md: document the enterprise bugfix for the target property in Graphite Render API 2021-06-09 13:51:32 +03:00
Aliaksandr Valialkin
490783696a docs/CHANGELOG.md: document improvements in re-routing handling in vminsert 
See the following commits:

* 1c09e71f5b
* 0d067eb112
* 2c6b917749
 2021-06-09 13:42:12 +03:00
Aliaksandr Valialkin
f3749dedba docs: document rules replay feature for vmalert 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/836

This is a follow-up for 2a259ef5e7
 2021-06-09 12:30:54 +03:00
Aliaksandr Valialkin
8ad445474a docs/CHANGELOG.md: document the bugfix, which prevents panics for aborted http requests in vmauth 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1353

This is a follow-up for 6b29b955c0
 2021-06-09 12:12:47 +03:00
Aliaksandr Valialkin
269e35d676 app/{vmagent,vminsert}: follow-up after 2fe045e2a4 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1343
 2021-06-04 20:33:22 +03:00
Aliaksandr Valialkin
67cfc07004 docs/CHANGELOG.md: document the bugfix from 6f19bb23a1 2021-06-04 11:56:06 +03:00
Aliaksandr Valialkin
c53a90e5fc docs/CHANGELOG.md: document that it is possible to build VictoriaMetrics components for Solaris 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1322

This is a follow-up for ddc8022702
 2021-06-04 04:41:12 +03:00
Aliaksandr Valialkin
1c09e71f5b app/vminsert: add -disableRerouting command-line flag for disabling re-routing if some vmstorage nodes have lower performance than the others 
Refactor the rerouting mechanism and make it more resilient to cases when some of vmstorage nodes are temporarily unavailable.

Reduce the probability of rerouting storm.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/791
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1054
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1165
 2021-06-04 04:33:52 +03:00
Aliaksandr Valialkin
8cdecfc52c app/vmauth: allow balancing the load among multiple backend nodes by specifying multiple urls in url_prefix config 2021-05-29 01:04:22 +03:00
Aliaksandr Valialkin
97de72054e docs: document f0c21b6300 2021-05-27 15:04:13 +03:00
Aliaksandr Valialkin
820ac6cd0c docs/CHANGELOG.md: document changes from 2233d6ed8a and d210958fd0 2021-05-26 12:24:28 +03:00
Aliaksandr Valialkin
25ed1f0c4f docs/CHANGELOG.md: cut v1.60.0 2021-05-24 15:55:31 +03:00
Aliaksandr Valialkin
402a8ca710 lib/storage: do not populate MetricID->MetricName cache during data ingestion 
This cache isn't needed during data ingestion, so there is no need in spending RAM on it.

This reduces RAM usage on data ingestion path by 30%
 2021-05-24 03:06:40 +03:00
Aliaksandr Valialkin
68c3901ebd docs/CHANGELOG.md: small typo fix 2021-05-23 14:15:49 +03:00
Aliaksandr Valialkin
8ec3e876be docs/CHANGELOG.md: document the addition of extra_filter_labels at 84cc0513e1 2021-05-23 14:15:49 +03:00
Aliaksandr Valialkin
71ff7ee18d lib/promauth: follow-up after 5b8176c68e 2021-05-22 18:02:03 +03:00
Aliaksandr Valialkin
23355ca34c lib/fs: wait for a while before giving up on NFS file removal if the removal queue is full 
This should reduce the probability of the panic on a highly loaded VictoriaMetrics
accepting millions of samples per second.

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1313
 2021-05-21 17:21:35 +03:00
Aliaksandr Valialkin
f664f7fb1d docs/CHANGELOG.md: mention the bugfix from d626c5c2a9 
Updates https://github.com/VictoriaMetrics/operator/issues/243
 2021-05-21 16:38:20 +03:00
Aliaksandr Valialkin
d77db9d813 all: do not skip SIGHUP signal during service initialization 
This can lead to stale or incomplete configs like in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240
 2021-05-21 16:38:20 +03:00
Aliaksandr Valialkin
9aa22cccf0 docs/CHANGELOG.md: move tip to proper place 2021-05-20 17:58:24 +03:00
Aliaksandr Valialkin
165a9f9200 app/vmstorage: add ability to limit series cardinality via -storage.maxHourlySeries and -storage.maxDailySeries command-line flags 2021-05-20 15:31:57 +03:00