Commit Graph

4792 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
288d13af8d
lib/netutil: parallelize background pings for remote addresses
This should improve the time needed for determining unavailale remote addresses
across big numer of ConnPool's.

This is a follow-up for a1629bd3be

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2022-06-21 13:32:27 +03:00
Aliaksandr Valialkin
24097f2417
docs/CHANGELOG.md: cut v1.78.0 2022-06-20 18:11:04 +03:00
Yurii Kravets
14397ba23e
Changed the level type in alerts.yml for TooManyLogs alert ()
alerts: filter out non error log messages for `TooManyLogs`

Info and Warn error levels aren't always a result of malfunctioning
or faulty state. So we filter them out.
2022-06-20 16:45:52 +02:00
Aliaksandr Valialkin
a1629bd3be
lib/netutil.ConnPool: skip dialing remote address if the previous dial attempt was unsuccessful
If the previous dial attempt was unsuccessful, then all the new dial attempts are skipped
until the background goroutine determines that the given address can be successfully dialed.

This reduces query latency when some of vmstorage nodes are unavailable and dialing them is slow.

This should help with https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711

This commit is based on ideas from the https://github.com/VictoriaMetrics/VictoriaMetrics/pull/2756

The main differences are:

- The check for healthy/unhealthy storage nodes is moved one level lower from app/vmselect/netstorage to lib/netutil.ConnPool.
  This makes possible re-using this feature everywhere lib/netutil.ConnPool is used.
- The check doesn't take into account handshake errors for already established connections.
  Handshake errors usually mean improperly configured VictoriaMetrics cluster, so they shouldn't be ignored.
2022-06-20 17:33:54 +03:00
Aliaksandr Valialkin
45e9732764
docs: follow-up after e4d6b750f6 2022-06-20 17:15:52 +03:00
Nikolay
15662c0f29
lib/httpserver: adds flagsAuthKey command-line flag ()
* lib/httpserver: adds flagsAuthKey command-line flag
It protects /flags endpoint with authKey.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2753O

* Apply suggestions from code review

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-20 17:15:51 +03:00
Aliaksandr Valialkin
b28c6febf9
app/{vminsert,vmselect}: add -vmstorageDialTimeout command-line flag for tuning the maximum time needed for establishing connections to vmstorage
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711
2022-06-20 15:17:34 +03:00
Aliaksandr Valialkin
0abf46d66a
docs/Articles.md: add a link to https://www.sobyte.net/post/2022-05/victoriametrics-bloomfilter/ 2022-06-20 14:44:11 +03:00
Aliaksandr Valialkin
032d4fdf7d
vendor: make vendor-update 2022-06-20 14:31:57 +03:00
Aliaksandr Valialkin
079fdd3158
all: update Go builder for production builds from 1.18.2 to 1.18.3
See https://github.com/golang/go/issues?q=milestone%3AGo1.18.3+label%3ACherryPickApproved
2022-06-20 14:30:48 +03:00
Aliaksandr Valialkin
b2cfb8faf7
app/vmstorage/transport: call vmselectRequestCtx.readSearchQuery() in processVMSelectDeleteMetrics
Previously the processVMSelectDeleteMetrics was calling separate functions from readSearchQuery().
It is better from readability and maintenance PoV to substitute it with readSearchQuery call.
2022-06-20 14:23:17 +03:00
Aliaksandr Valialkin
270ad39359
lib/storage: properly take into account already registered series when -storage.maxHourlySeries or -storage.maxDailySeries limits are enabled
The commit 5fb45173ae takes into account only newly registered series
when applying cardinality limits. This means that the cardinality limit could be exceeded with already registered series.
This commit returns back accounting for already registered series when applying cardinality limits.
2022-06-20 13:53:41 +03:00
Roman Khavronenko
3ada676879
docs: reference links from key concepts ()
Signed-off-by: hagen1778 <roman@victoriametrics.com>

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-19 23:14:30 +03:00
Aliaksandr Valialkin
fe9f59fcd6
all: replace bash with console blocks in all the *.md files
This is a follow-up for 954a7a6fc6
2022-06-19 23:02:02 +03:00
Artem Navoiev
42c69ae74e
docs: replace bash code block type with console () 2022-06-19 23:02:00 +03:00
Aliaksandr Valialkin
4ae1c5655f
docs/CHANGELOG.md: document ef7f52e0e6 2022-06-19 22:49:11 +03:00
Roman Khavronenko
3e45e1ff63
Vmalert notifiers ()
* vmalert: remove head of line blocking for sending alerts

This change makes sending alerts to notifiers concurrent instead
of sequential. This eliminates head of line blocking, where first
faulty notifier address prevents the rest of notifiers from
receiving notifications.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: make default timeout for sending alerts 10s

Previous value of 1m was too high and was inconsistent
with default timeout defined for notifiers via
configuration file.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmalert: linter checks fix

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-19 22:49:10 +03:00
Aliaksandr Valialkin
7a79e7c0ef
lib/storage: create per-day indexes together with global indexes when registering new time series
Previously the creation of per-day indexes and global indexes
for the newly registered time series was decoupled.

Now global indexes and per-day indexes for the current day are created toghether for new time series.
This should speed up registering new time series a bit.
2022-06-19 22:32:41 +03:00
Aliaksandr Valialkin
88e1221b35
lib/storage: do not register new series if -storage.maxHourlySeries or -storage.maxDailySeries limits are exceeded
Previously samples for new series weren't added as expected when series limits were reached,
but new series were still registered in indexdb.
2022-06-19 22:03:02 +03:00
Aliaksandr Valialkin
c5ac176153
lib/storage: reset metric id caches for the previous and the current hour
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2698
2022-06-19 22:02:51 +03:00
Aliaksandr Valialkin
149c5c9381
docs: update docs on how to add tags to metrics collected by DataDog agent
Follow-up for f16072c3c1
2022-06-17 13:12:21 +03:00
Dmytro Kozlov
5bc13e2fe8
vmui: added focusLabel, enable cardinality app configuratior ()
* vmui: added focusLabel, enable app configuratior

* vmui: set focusLabel if {labelName!=""}

* wip

* docs/CHANGELOG.md: mention about focusLabel feature in cardinality explorer

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2730

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-17 13:04:09 +03:00
Dmytro Kozlov
82440e76c5
doc: added workaround for datadog agent ()
* Added workaround for datadog agent

* docs: update datadog workaround

* doc: update doc description

* Apply suggestions from code review

* docs: `make docs-sync`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-16 22:24:17 +03:00
Roman Khavronenko
287bb96758
docs: mention sandbox update in release procedure ()
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-16 20:51:41 +03:00
Roman Khavronenko
cbe39bbb8d
vmselect: limit end param max value by 2d in future ()
* vmselect: limit `end` param max value by 2d in future

The change is applied only to service handlers like `/labels` or `/series`
and limits the `end` param by max value <= now() + 2 days. The same limit
is applied for the ingested data, so no reason to allow to request data
in future far than that.

The change is also needed for corner cases like https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2669
where too high `end` value triggers inefficient global index search.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs/CHANGELOG.md: document the bugfix

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-16 20:51:40 +03:00
Aliaksandr Valialkin
cda3ce09e5
docs/vmagent.md: typo fix: configued -> configured 2022-06-16 20:51:40 +03:00
Roman Khavronenko
246d2df361
dashboards: add cpu usage panels per each component type ()
The change adds extra panel per each component, showing
the amount of used CPU cores and the limit (summary of all instances).

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2696

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-16 20:49:55 +03:00
Aliaksandr Valialkin
450aa0ae5a
lib/promrelabel: support action: graphite relabeling
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2737
2022-06-16 20:25:49 +03:00
Roman Khavronenko
ba7ece02c4
docs: add multiple-remote-writes topology to vmalert ()
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-16 20:21:12 +03:00
Aliaksandr Valialkin
da1d1e83df
app/{vmselect,vmstorage}: properly pass seriesCountByLabelName and seriesCountByFocusLabelValue entries from vmstorage to vmselect 2022-06-16 10:44:29 +03:00
Aliaksandr Valialkin
b4e75a0b89
docs/CHANGELOG.md: document dd327bfa9e2c69fe21ab1d92c14636733d7c5620 2022-06-15 18:41:26 +03:00
Aliaksandr Valialkin
14ce7b0e25
docs/CHANGELOG.md: document 00719e5779a3e4eeedb74cb3d25a9ecfe0e16063 2022-06-15 18:09:23 +03:00
Aliaksandr Valialkin
ee9954082f
app/vmselect/netstorage: properly aggregate seriesCountByLabelName and seriesCountByFocusLabelValue obtained from multiple vmselect nodes at /api/v1/status/tsdb 2022-06-15 16:48:40 +03:00
Aliaksandr Valialkin
45fa9d798d
app/vmselect: accept focusLabel query arg at /api/v1/status/tsdb 2022-06-14 18:39:00 +03:00
Aliaksandr Valialkin
fb77843639
lib/storage: show top labels with the highest number of series in cardinality explorer 2022-06-14 16:34:13 +03:00
Dmytro Kozlov
ed9b4c6f6d
vmui: refactor Cardinality panel ()
* vmui: refactor Cardinality panel

* vmui: change width of the search panel

* vmui: code cleanup

* vmui: code cleanup

* vmui: fixed vulnerability (npm audit fix)

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-14 14:40:44 +03:00
Aliaksandr Valialkin
ce953d5e95
docs: update command-line flags' descriptions according to recent changes 2022-06-14 13:28:23 +03:00
Aliaksandr Valialkin
3167fbc21d
lib/storage: improve error message when -search.max* command-line flag values are exceeded 2022-06-14 13:28:21 +03:00
Nikolay
e23af8f05c
lib/httpserver: backport changes from master branch ()
* lib/httpserver: backport changes from master branch
adds basicAuth
adds authKey check for /metrics and /debug/pprof requests
it should improve security for cluster components

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-14 13:02:44 +03:00
Roman Khavronenko
0065b13243
docs: reduce free disk recommendation from 30% to 20% ()
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-14 12:55:41 +03:00
Aliaksandr Valialkin
12618925fc
docs/guides/migrate-from-influx.md: suggest more real-world value for -search.maxStalenessInterval
Suggest `-search.maxStalenessInterval=10s` instead of `-search.maxStalenessInterval=1ms`,
since `1ms` would result in empty graphs in most cases, since the interval between data points
on the graph is usually much higher than 1ms. For example, if the graph shows time range of one hour
and it contains 1000 points, then the interval between points on the graph would equal to
3600s/1000=3.6 seconds.
2022-06-14 12:51:35 +03:00
Roman Khavronenko
9873001539
docs: migrating from influx ()
Mention `-search.maxStalenessInterval` flag and its effect on query engine.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-14 12:51:33 +03:00
Aliaksandr Valialkin
b48ca34e05
docs/Single-server-VictoriaMetrics.md: recommend running all the VictoriaMetrics components behind auth proxy in Security chapter 2022-06-13 10:30:52 +03:00
Aliaksandr Valialkin
e1fc4db109
vendor: make vendor-update 2022-06-13 10:07:19 +03:00
Aliaksandr Valialkin
c846d0db01
docs/CHANGELOG.md: document 99dbe7f9d4 2022-06-13 10:07:19 +03:00
Wataru Manji
0cd750fa5e
Add remote-write headers ()
Co-authored-by: Wataru Manji <wataru.manji@linecorp.com>
2022-06-13 10:07:19 +03:00
Aliaksandr Valialkin
8128c6db6b
app/vmagent: follow-up after 4583ed23a8 2022-06-13 09:56:27 +03:00
Dmytro Kozlov
f0cb3c4999
Added a stub for datadog endpoint ()
* Added a stub for datadog endpoint

* Update app/vmagent/main.go

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-13 09:56:26 +03:00
Roman Khavronenko
81b6efea9e
Readme cleanup ()
* docs: minor styling and wording changes

Changes made after reading https://developers.google.com/tech-writing

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs: set proper types for code blocks

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs: add `copy` wrapper for some commands

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs: sync

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* docs: resolve conflicts

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-06-13 09:49:59 +03:00
Yury Molodov
65107a4585
vmui: enhancements () ()
* feat: make datepicker to be set to last 30 min by default

* fix: correct spinner while loading data

* feat: change legend style

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-06-13 09:49:58 +03:00