Commit Graph

1282 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
9edc12fa99
app/vmagent/remotewrite: fix parallel data sending to remote storage systems at e0d2ba5608 2021-11-04 16:58:53 +02:00
Aliaksandr Valialkin
63c29099ba
app/vmagent: add -remoteWrite.maxRowsPerBlock command-line option, which may be used for improving data ingestion performance under high load 2021-11-04 15:39:55 +02:00
Aliaksandr Valialkin
5f13ba8631
app/vmagent/remotewrite: send data to remote storage systems in parallel
This should improve data ingestion speed many `-remoteWrite.url` options are configured
2021-11-04 15:04:02 +02:00
dependabot[bot]
08f487e7e8
build(deps-dev): bump @typescript-eslint/eslint-plugin (#183)
Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) from 5.2.0 to 5.3.0.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.3.0/packages/eslint-plugin)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-04 12:53:03 +02:00
Aliaksandr Valialkin
be9c7713d7
app/vmselect: make vmui-update 2021-11-04 12:16:55 +02:00
dependabot[bot]
c656e342dd
build(deps-dev): bump @babel/plugin-proposal-nullish-coalescing-operator (#1769)
Bumps [@babel/plugin-proposal-nullish-coalescing-operator](https://github.com/babel/babel/tree/HEAD/packages/babel-plugin-proposal-nullish-coalescing-operator) from 7.14.5 to 7.16.0.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.16.0/packages/babel-plugin-proposal-nullish-coalescing-operator)

---
updated-dependencies:
- dependency-name: "@babel/plugin-proposal-nullish-coalescing-operator"
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-04 12:16:55 +02:00
dependabot[bot]
bb8b426628
build(deps): bump @date-io/dayjs in /app/vmui/packages/vmui (#1770)
Bumps [@date-io/dayjs](https://github.com/dmtrKovalenko/date-io) from 1.3.13 to 2.11.0.
- [Release notes](https://github.com/dmtrKovalenko/date-io/releases)
- [Commits](https://github.com/dmtrKovalenko/date-io/compare/v1.3.13...v2.11.0)

---
updated-dependencies:
- dependency-name: "@date-io/dayjs"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-04 12:16:55 +02:00
Aliaksandr Valialkin
d445d22c0c
lib/promscrape: add -promscrape.maxResponseHeadersSize command-line flag for tuning the maximum http response headers size from Prometheus scrape targets 2021-11-03 22:27:55 +02:00
Aliaksandr Valialkin
a102fca4ac
app/vmselect/promql: add limit_offset(limit, offset, q) function, which can be used for paging over big number of time series
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1778
2021-11-03 16:03:10 +02:00
Aliaksandr Valialkin
43bf0333eb
app/vmselect/promql: add label_graphite_group() function for extracting groups from Graphite metric names 2021-11-03 13:20:34 +02:00
Aliaksandr Valialkin
551a94ba7f
app/{vmagent,vminsert}: add ability to restrict access to /config page with authKey query arg
The authKey can be configured via `-configAuthKey` command-line flag.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764
2021-11-01 16:47:30 +02:00
Aliaksandr Valialkin
94d9c06fc1
docs/vmauth.md: typo fix 2021-10-28 14:06:28 +03:00
Aliaksandr Valialkin
38cbae006a
app/vmbackupmanager: fix links to images
This is a follow-up after bd6b8f7e31
2021-10-27 21:36:25 +03:00
Yury Molodov
a16c88095c
vmui: correct migration material-ui (#1758)
* migration material-ui

* fix: rollback popover

* app/vmselect: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-27 20:43:24 +03:00
Aliaksandr Valialkin
2ebee4e741
app/{vmalert,vmagent}: improve the distribution of scrape offsets among targets / rules
Previously only the lower part of 64-bit hash was used for calculating the offset.
This may give uneven distribution in some cases. So let's use all the available 64 bits from the hash
for calculating the offset.
2021-10-27 20:04:02 +03:00
Aliaksandr Valialkin
99784b21c1
all: fix build issues and tests for Apple M1
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1653
2021-10-27 15:07:19 +03:00
Aliaksandr Valialkin
ad445a06cd
lib/promscrape: properly show proxy_url option value at /config page
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755
2021-10-26 21:24:22 +03:00
Yury Molodov
e7591577f0
vmui: update dependencies (#1754)
* update dependencies

* update codemirror version

* app/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-26 19:32:31 +03:00
Aliaksandr Valialkin
eb783b0ef3
app/vmbackup/README.md: sync with docs/vmbackup.md after e706fb5686 2021-10-26 19:21:26 +03:00
Roman Khavronenko
5321127add
vmalert: allow groups with empty rules for compatibility reasons (#1742)
Prometheus allows to have groups with no rules, so we should support
it in vmalert as well for compatibility reasons.
It is also allowed to hot-reload empty groups by adding or removing rules.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-10-25 14:46:51 +03:00
Aliaksandr Valialkin
d8ce86d685
app/vmselect/promql: reduce the precision from 15 significant digits to 13 significant digits when comparing float64 results in tests
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1738
2021-10-24 13:32:25 +03:00
Aliaksandr Valialkin
559dd996c4
app/vmauth: add ability to specify http headers to send in requests to backends
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1736
2021-10-22 19:11:35 +03:00
Aliaksandr Valialkin
a8bcc3c276
lib/promscrape: reduce memory usage if -promscrape.noStaleMarkers command-line flag is passed
Do not store in memory the response from the last scrape per each target if -promscrape.noStaleMarkers option is enabled.
This should reduce memory usage when the scraped targets return large responses.
2021-10-22 13:22:08 +03:00
Roman Khavronenko
e0f21d6000
vmalert: correctly calculate alert ID including extra labels (#1734)
Previously, ID for alert entity was generated without alertname or groupname.
This led to collision, when multiple alerting rules within the same group
producing same labelsets. E.g. expr: `sum(metric1) by (job) > 0` and
expr: `sum(metric2) by (job) > 0` could result into same labelset `job: "job"`.

The issue affects only UI and Web API parts of vmalert, because alert ID is used
only for displaying and finding active alerts. It does not affect state restore
procedure, since this label was added right before pushing to remote storage.

The change now adds all extra labels right after receiving response from the datasource.
And removes adding extra labels before pushing to remote storage.

Additionally, change introduces a new flag `Restored` which will be displayed in UI
for alerts which have been restored from remote storage on restart.
2021-10-22 12:31:35 +03:00
Yury Molodov
3ccc37a98b
vmui: query history (#1732)
* feat: add query history

* fix: change detect keyUp for nav query history

* feat: set default query history

* app/vmselect/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-22 12:29:27 +03:00
Aliaksandr Valialkin
5705f4b6d1
lib/httpserver: expose command-line flags at /flags page
This should simplify debugging.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695
2021-10-20 00:46:54 +03:00
Roman Khavronenko
ca49853664
vmalert: make group.ID() thread-safe (#1726)
Commit fixes potential race condition when group update
and generating of ID() happens simultaneously.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-10-19 16:45:52 +03:00
Roman Khavronenko
627224d493
vmalert: properly init SIGHUP listener before starting group manager (#1725)
Regression was introduced during code refactoring. It potentially
could lead to situation when SIGHUP signals were ignored while
vmalert was still busy with initing group manager.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-10-19 16:36:08 +03:00
Aliaksandr Valialkin
0a1982b294
app/vmauth: fix metric name prefixes: vmagent -> vmauth 2021-10-19 15:30:16 +03:00
Yury Molodov
744f6a98eb
vmui: features (#1711)
* feat: initial uPlot graph

* feat: add zoom/pan for graph

* fix: add zoom by ctrl/mac

* fix: remove unused code

* feat: add toggle cache for fetch

* feat: add fix y-axis limits

* fix: stop point events while panning

* fix: change getting cursor position when scaling

* feat: add cursor tooltip to graph

* fix: uninstall chart.js

* fix: change link for create an issue

* fix: set default cache value to true

* app/vmalert: follow-up after 0e2486df56

* docs/CHANGELOG.md: document 5416e18007

* app/vmui: `make vmui-update`

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2021-10-18 15:19:01 +03:00
Roman Khavronenko
44a88b7458
vmalert: remove extra / from path in WEB interface (#1717)
The extra `/` may cause issues when additional path prefixes
are configured. Also, removing it makes it consistent
with the rest of declarations.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-10-18 15:19:01 +03:00
Roman Khavronenko
93d4ef1239
vmctl: follow-up after 95d1d38595 (#1718)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-10-18 15:19:01 +03:00
Aliaksandr Valialkin
9273b83cd0
app/vmalert: follow-up after 0e2486df56 2021-10-18 15:08:42 +03:00
Miro Prasil
c0853d4bf8
vmctl influx convert bool to number (#1714)
vmctl: properly convert influx bools into integer representation

When using vmctl influx, the import would fail importing boolean fields
with:

```
failed to convert value "some".0 to float64: unexpected value type true
```

This converts `true` to `1` and `false` to `0`.

Fixes #1709
2021-10-18 14:59:17 +03:00
Alexander Rickardsson
0e1dbcd039
vmalert: add disablePathAppend to remote read (#1712)
* vmalert: add disablePathAppend to remoteRead

* docs: add docs for remoteRead.disablePathAppend
2021-10-18 14:59:17 +03:00
Alexander Rickardsson
63571e1334
vmalert: Redact passwords from error messages (#1713) 2021-10-18 14:59:17 +03:00
Aliaksandr Valialkin
ed994873fd
app/vmselect/promql: randomize the static selection of time series returned from limitk()
Sort series by a hash calculated from the series labels. This should guarantee "random" selection of the returned time series.
Previously the selection could be biased, since time series were sorted alphabetically by label names and label values.
2021-10-16 21:16:34 +03:00
Aliaksandr Valialkin
da0adf446d
app/vmselect/promql: typo fix in comment: didsn't -> didn't 2021-10-16 13:00:10 +03:00
Aliaksandr Valialkin
0452a8d4e8
lib/promscrape: store the full response in stream parsing mode in scrapeWork.lastScrape byte slice
This allows sending staleness marks and properly calculate scrape_series_added metric in stream parsing mode
at the cost of the increased memory usage, since now the potentially big response is kept
in the lastScrape byte slice per each scrapeWork.

In practice the memory usage increase shouldn't be big, since the response size
is usually much smaller than the parsed metrics from this response after the relabeling,
which usually adds a big pile of target-specific labels per each metric.
2021-10-15 15:26:24 +03:00
Aliaksandr Valialkin
25421fa2ae
lib/promscrape: add -promscrape.minResponseSizeForStreamParse command-line option for automatic switching to stream parsing mode when scraping targets with big responses
This should reduce memory usage when vmagent scrapes targets with non-uniform response sizes.
This is common case in Kubernetes monitoring.
2021-10-14 12:30:55 +03:00
Aliaksandr Valialkin
bee130cc78
lib/promscrape: return error if sample_limit or series_limit options are set when stream parsing mode is enabled 2021-10-14 12:30:54 +03:00
Roman Khavronenko
f393145843
Adjust http.Transport.MaxIdleConns setting for vmauth/vmalert services (#1704)
* vmalert: adjust `http.Transport.MaxIdleConns` value accordingly to `http.Transport.MaxIdleConnsPerHost`

`http.Transport.MaxIdleConnsPerHost` setting is controlled by `datasource.maxIdleConnections` flag,
while `http.Transport.MaxIdleConns` is inherited from DefaultTransport and is equal to `100`.
The fix adjusts `http.Transport.MaxIdleConns` value if it is lower than `http.Transport.MaxIdleConnsPerHost`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* vmauth: adjust `http.Transport.MaxIdleConns` value accordingly to `http.Transport.MaxIdleConnsPerHost`

`http.Transport.MaxIdleConnsPerHost` setting is controlled by `maxIdleConnsPerBackend` flag,
while `http.Transport.MaxIdleConns` is inherited from DefaultTransport and is equal to `100`.
The fix adjusts `http.Transport.MaxIdleConns` value if it is lower than `http.Transport.MaxIdleConnsPerHost`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2021-10-13 19:23:52 +03:00
Roman Khavronenko
14995eece6
vmalert: add Source link to alerts UI (#1701)
The source link is controlled by `external.url` and `external.alert.source`
flags, in the same way as for alertmanager notifications.
The source link is added to Alerts list view, and specific Alert view.
2021-10-13 15:26:57 +03:00
Aliaksandr Valialkin
aeedfe2fe2
app/vmagent: expose -promscrape.config contents at /config page as Prometheus does
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695
2021-10-12 16:27:37 +03:00
Aliaksandr Valialkin
48c3668b3d
app/vmselect/promql: add atan2 binary operator, which is going to be added in Prometheus 2.31
See https://github.com/prometheus/prometheus/pull/9248
2021-10-11 21:17:23 +03:00
Aliaksandr Valialkin
b3d28f3872
app/vmselect/promql: add missing trigonometric functions, which are going to be added in Prometheus 2.31
See https://github.com/prometheus/prometheus/issues/9233
2021-10-11 21:17:21 +03:00
Aliaksandr Valialkin
f1e1d20ac4
app/vmselect/promql: consistently return the same set of time series from limitk() function
This is the expected behaviour by most users.
2021-10-08 19:55:29 +03:00
Aliaksandr Valialkin
cf87050d83
app/vmui: follow-up after 7bfb44113e
* Run `vmui-update`
* Document the changes in README.md and CHANGELOG.md
2021-10-08 15:11:09 +03:00
Yury Molodov
41a83a1cc6
vmui: use uPlot as default engine for graph (#1683)
* feat: initial uPlot graph

* feat: add zoom/pan for graph

* fix: add zoom by ctrl/mac

* fix: remove unused code
2021-10-08 15:11:07 +03:00
Aliaksandr Valialkin
4fddcf4c83
app/{vminsert,vmstorage}: follow-up after a171916ef5
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/269
2021-10-08 14:09:51 +03:00