Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-16 00:41:24 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
2,430 Commits 114 Branches 748 Tags 509 MiB
f023e92f1a
Commit Graph

2430 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Aliaksandr Valialkin
1adbbe7617 vendor: update github.com/VictoriaMetrics/metrics from v1.14.0 to v1.15.0 
The v1.15.0 exports the following additional metrics:

    process_io_read_bytes_total - the number of bytes read via io syscalls such as read and pread
    process_io_written_bytes_total - the number of bytes written via io syscalls such as write and pwrite
    process_io_read_syscalls_total - the number of read syscalls such as read and pread
    process_io_write_syscalls_total - the number of write syscalls such as write and pwrite
    process_io_storage_read_bytes_total - the number of bytes read from storage layer
    process_io_storage_written_bytes_total - the number of bytes written to storage layer

These metrics can be used for monitoring process io
 2021-02-21 22:53:54 +02:00
Aliaksandr Valialkin
587132555f lib/mergeset: reduce memory usage for inmemoryBlock by using more compact items representation 
This also should reduce CPU time spent by GC, since inmemoryBlock.items don't have pointers now,
so GC doesn't need visiting them.
 2021-02-21 22:09:10 +02:00
Aliaksandr Valialkin
bd3bcdc43c lib/storage: do not re-calculate stats for heavy tag filters 
This should reduce the number of slow queries when stats for heavy tag filters was recalculated.
 2021-02-21 21:43:37 +02:00
Aliaksandr Valialkin
b8a5ee2e93 lib/{mergeset,storage}: allow merging smaller number of small parts 
While this may increase CPU and disk IO usage needed for background merge,
this also recudes CPU usage during queries in production. This is because
such queries tend to read recently added data and it is better to have lower number
of parts for such data in order to reduce CPU usage.

This partially reverts ebf8da3730
 2021-02-21 21:43:37 +02:00
Aliaksandr Valialkin
34195218e1 lib/{mergeset,storage}: do not use pools for indexBlock and inmemoryBlock during their caching, since this results in higher memory usage in production without any performance gains 2021-02-21 21:43:37 +02:00
Roman Khavronenko
14323af8c9
Add Labels limit exceeded panel to dashboard (#1076) 
New panel supposed to display events when VM drops extra label
on exceeding `maxLabelsPerTimeseries` limit.
 2021-02-19 00:50:21 +02:00
Aliaksandr Valialkin
f09b46b2b1 lib/promscrape: typo fix after the commit f26162ec99 2021-02-19 00:34:18 +02:00
Aliaksandr Valialkin
72eef964d9 app/vmagent: properly perform graceful shutdown, which was broken in the commit 1d1ba889fe 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
 2021-02-19 00:34:17 +02:00
Aliaksandr Valialkin
41d3ff4f2b app/vmagent: fix scrape config example for scrape_align_interval option 2021-02-18 23:57:45 +02:00
Aliaksandr Valialkin
502d0e2524 lib/promscrape: add scrape_align_interval config option into scrape config 
This option allows aligning scrapes to a particular intervals.
 2021-02-18 23:53:04 +02:00
Aliaksandr Valialkin
bd1d906eee app/vmselect/promql: reduce the probability of duplicate time series errors when querying Kubernetes metrics 2021-02-18 22:08:13 +02:00
Aliaksandr Valialkin
3a8b2eed58 docs/CHANGELOG.md: cut v1.54.1 2021-02-18 19:10:57 +02:00
Aliaksandr Valialkin
10ccb92e4d lib/storage: use composite index for a query with a name filter and negative filters 2021-02-18 18:57:45 +02:00
Aliaksandr Valialkin
418de71509 lib/storage: properly handle queries containing a filter on metric name plus any number of negative filters and zero non-negative filters 
Example: `node_cpu_seconds_total{mode!="idle"}`
 2021-02-18 18:33:05 +02:00
Aliaksandr Valialkin
902a4f6486 app/victoria-metrics: remove accidentally added single-node tests 2021-02-18 18:33:03 +02:00
Aliaksandr Valialkin
0af9fbe326 docs/Articles.md: add a link to https://www.percona.com/blog/2021/02/12/tame-kubernetes-costs-with-percona-monitoring-and-management-and-prometheus-operator/ 2021-02-18 16:37:22 +02:00
Aliaksandr Valialkin
901710b9e2 app/vmalert: add missing multiarch Dockerfile 2021-02-18 15:23:57 +02:00
Aliaksandr Valialkin
be93e02085 docs/CHANGELOG.md: cut v1.54.0 2021-02-18 14:52:52 +02:00
Aliaksandr Valialkin
628b8eb55e lib/storage: prevent from running identical heavy tag filters in concurrent queries when measuring the number of loops for such tag filter. 
This should reduce CPU usage spikes when measuring the number of loops needed for heavy tag filters
 2021-02-18 14:01:18 +02:00
Aliaksandr Valialkin
fd41f070db lib/storage: sort tag filters by the number of loops they need for the execution 
This metric should work better than the filter execution duration, since it cannot be distorted
by concurrently running queries.
 2021-02-18 12:52:29 +02:00
Aliaksandr Valialkin
a5bfd831fb docs/Articles.md: add a link to https://medium.com/alteos-tech-blog/observability-availability-and-doras-research-program-85deb6680e78 2021-02-18 01:25:54 +02:00
Aliaksandr Valialkin
1dad7ecf54 app/victoria-metrics/testdata: add a test for {__graphite__="foo.*.bar"} selector 2021-02-17 21:53:24 +02:00
Aliaksandr Valialkin
9c81429299 app/vmagent/remotewrite: cleanup after 1d1ba889fe 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
 2021-02-17 21:43:48 +02:00
Nikolay
40973eda1c adds pushback for fastqueue, (#1075) 
during shutdown currently sending block was lost,
now its pushed back to fast queue and will be flushed on disk,
it may lead to data duplication.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065
 2021-02-17 21:43:46 +02:00
Aliaksandr Valialkin
57d192a2c2 docs/CHANGELOG.md: mention that prod binaries are built now with Go1.16 
This is a follow-up for 4edfe76bef
 2021-02-17 21:07:48 +02:00
Aliaksandr Valialkin
80699405aa docs/CaseStudies.md: actualize Wix numbers 2021-02-17 21:03:42 +02:00
Aliaksandr Valialkin
9566015a36 Revert "lib/mergeset: tune lifetime for entries inside block caches" 
This reverts commit 458c89324d.

Production testing revealed zero improvements for memory usage with reduced lifetime for entries in block caches.
 2021-02-17 20:42:15 +02:00
Aliaksandr Valialkin
b4c7d5992b lib/storage: move composite filters to the top during sorting 2021-02-17 20:26:32 +02:00
Aliaksandr Valialkin
2005d8212a lib/storage: return back filter arg to getMetricIDsForTagFilter function 
The filter arg has been removed in the commit c7ee2fabb8
because it was preventing from caching the number of matching time series per each tf.

Now the cache contains duration for tf execution, so the filter shouldn't break such caching.
 2021-02-17 19:33:15 +02:00
Aliaksandr Valialkin
83da939947 app/vmstorage: export vm_composite_filter_success_conversions_total and vm_composite_filter_missing_conversions_total metrics 2021-02-17 19:13:49 +02:00
Aliaksandr Valialkin
0c5bb2a397 lib/storage: revert ecf132933e, since negative filters require the same amount of work as non-negative filters 2021-02-17 18:56:13 +02:00
Aliaksandr Valialkin
bbc287ea6a lib/storage: tag filters sorting... 2021-02-17 18:56:12 +02:00
Aliaksandr Valialkin
f5e841c1e9 lib/storage: further tune tag filters sorting 2021-02-17 17:28:36 +02:00
Aliaksandr Valialkin
45e5cdb631 deployment/dm: update Go builder image from v1.15.8 to v1.16.0 
See release notes for Go1.16 at https://golang.org/doc/go1.16
 2021-02-17 15:18:45 +02:00
Aliaksandr Valialkin
9d1f14d94c lib/storage: tune the logic for sorting tag filters according the their exeuction times 2021-02-17 15:02:19 +02:00
Aliaksandr Valialkin
1a19702d92 lib/storage: make sure that nobody uses partitions when closing the table 2021-02-17 15:02:18 +02:00
Aliaksandr Valialkin
31e04e0d45 docs/CHANGELOG.md: document new per-tenant metrics 2021-02-16 23:33:59 +02:00
Aliaksandr Valialkin
3062ff0fdb app/vmselect: export per-tenant stats on the number of requests and the cumulative request duration 
The metrics are:
- vm_vmselect_http_requests_total{accountID="...",projectID="..."} - the total number of select requests per each tenant
- vm_vmselect_http_duration_ms_total{accountID="...",projectID="..."} - the total duration in milliseconds for per-tenant select requests

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/932
 2021-02-16 23:30:29 +02:00
Aliaksandr Valialkin
2d5587d65c vendor: make vendor-update 2021-02-16 22:25:02 +02:00
Aliaksandr Valialkin
3b8596bfec lib/httpserver: make errcheck happy 2021-02-16 22:05:59 +02:00
Aliaksandr Valialkin
667a7594b7 docs: rename vmbackuper to vmbackupmanager 2021-02-16 22:01:21 +02:00
Aliaksandr Valialkin
1ab7a8dfd5 lib/storage: more tuning for tag filters sorting according the time they take 2021-02-16 21:27:27 +02:00
Aliaksandr Valialkin
35a23234ca lib/mergeset: tune lifetime for entries inside block caches 
This should reduce memory usage in general case without significant CPU usage increase
 2021-02-16 18:12:32 +02:00
Aliaksandr Valialkin
500acb958c lib/mergeset: clarify comments in the code a bit 2021-02-16 18:03:11 +02:00
Aliaksandr Valialkin
7dee57da03 deployment/docker: properly publish latest tag during make publish-via-docker 
This has been broken in f9902b3372
 2021-02-16 17:45:12 +02:00
Aliaksandr Valialkin
73c9da16b8 app/vmselect/netstorage: reuse timeseriesWork objects in order to reduce memory allocations 2021-02-16 16:09:00 +02:00
Aliaksandr Valialkin
a3a09a3c6e app/vmselect/netstorage: use unsafe string as a key for a map when the map already contains the given key 
This should prevent from a memory allocation and a string copy.
 2021-02-16 15:46:31 +02:00
Aliaksandr Valialkin
da05c10544 lib/uint64set: remove memory allocation in bucket16.appendTo when sorting smallPool 2021-02-16 15:31:59 +02:00
Aliaksandr Valialkin
18305caadb lib/httpserver: cache /metrics output for a second 
This should reduce CPU load when `/metrics` output is scraped with a frequency exceeding a request per second
 2021-02-16 14:57:13 +02:00
Aliaksandr Valialkin
7869d38043 lib/protoparser/influx: make sure that escaped whitespace can be put in measurement, tag names and field names 2021-02-16 13:59:11 +02:00