VictoriaMetrics

mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-26 20:30:10 +01:00

Author	SHA1	Message	Date
Aliaksandr Valialkin	5610207549	docs: document -s3ForcePathStyle command-line option This is a follow-up for `b72eed1f5e`	2021-11-17 01:10:24 +02:00
Aliaksandr Valialkin	aa01bb9349	app/vmalert/README.md: sync with docs/vmalert.md This is a follow-up after `d8c70903ec`	2021-11-17 00:56:53 +02:00
Aliaksandr Valialkin	71082c7df8	app/vmauth: add ability to override the `username` label value for `vmauth_user_requests_total` metric by specifying `name` option in `-auth.config` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1805	2021-11-17 00:48:14 +02:00
Aliaksandr Valialkin	1eb756fd11	app/vmselect/promql: arrange function names in the code in alphabetical order This should simplify code maintenance in the future	2021-11-14 13:54:43 +02:00
Aliaksandr Valialkin	bdc813fc33	app/vmui/Dockerfile-web: update Go builder for vmui from v1.17.1 to v1.17.3	2021-11-14 13:54:41 +02:00
Aliaksandr Valialkin	82cf9f26db	app/vmselect/promql: prevent from incorrect calculations for `deriv()` over multiple samples with identical timestamps	2021-11-12 13:53:30 +02:00
Aliaksandr Valialkin	305507930c	app/vmauth: initialize reverse proxy only after flag.Parse() is called This should properly take into accoun the `-maxIdleConnsPerBackend` command-line flag value. Previously it was hardcoded to 100. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1300	2021-11-09 19:23:44 +02:00
Aliaksandr Valialkin	4fb19fe34b	all: consistently return `application/json` content-type without `charset=utf-8` The `application/json` content-type has utf-8 encoding by default. See https://stackoverflow.com/questions/9254891/what-does-content-type-application-json-charset-utf-8-really-mean Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897	2021-11-09 18:07:22 +02:00
Aliaksandr Valialkin	bc357d7132	docs/vmalert.md: improve wording in `Multitenancy` chapter	2021-11-09 14:20:30 +02:00
Aliaksandr Valialkin	31486f2244	app/vmselect/promql: properly return durations smaller than one second from duration_over_time() function Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780	2021-11-09 11:42:43 +02:00
Aliaksandr Valialkin	57aaf914ac	docs: mention that graphs on the official dashboards contain useful hints	2021-11-08 19:54:25 +02:00
Yury Molodov	89e0f66bfc	vmui: migration MUI Core v4 to v5 (#1795 ) * migration MUI Core v4 to v5 * app/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-11-08 15:46:57 +02:00
Aliaksandr Valialkin	3a1ac218ba	app/vmselect/promql: add `duration_over_time(m[d], max_interval)` function This function calculates the actual lifetime of the time series on the given lookbehdind window `d` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780	2021-11-08 13:14:57 +02:00
Yury Molodov	654686fd40	vmui: fix graph reset (#1788 ) * feat: add query history * fix: change detect keyUp for nav query history * feat: set default query history * feat: change graph legend * update dependencies * update codemirror version * fix: correct update period time after zoom/pan * fix: optimize data processing for the graph * fix: eliminate memory leaks related to mouse events * fix: correct display of straight line * Merge branch 'master' into vmui-fix-reset-graph * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-11-05 20:58:49 +02:00
Aliaksandr Valialkin	e58630401e	app/{vmalert,vmbackup}/README.md: sync with docs after the commit `47d1612bf8`	2021-11-05 20:46:07 +02:00
Aliaksandr Valialkin	9aa40fc0c3	docs/vmalert.md: document the addition of -defaultTenant.prometheus and -defaultTenant.graphite command-line options to enterprise version of vmalert	2021-11-05 20:04:53 +02:00
Aliaksandr Valialkin	56a25146cb	app/vmalert/datasource: use plain string literals instead of constants This removes the unneeded level of indirection and improves code readability. The "prometheus" and "graphite" constants aren't going to change in the future, so there is no sense in hiding them behind constants.	2021-11-05 19:58:15 +02:00
Aliaksandr Valialkin	31ac507bee	app/vmalert: remove `rule.type` config, since it doesnt play well with the upcoming default tenants for -clusterMode It is better from the consistency point of view to set up rule types at group level where tenant config is set up.	2021-11-05 19:52:41 +02:00
Aliaksandr Valialkin	e702b7a92f	app/vmagent: allow bigger number of in-memory blocks for big values of `-remoteWrite.queues` This should improve the maximum data ingestion speed for highly-loaded vmagent instances which run on beefy servers with many CPU cores and big amounts of RAM	2021-11-05 15:16:50 +02:00
Aliaksandr Valialkin	847004fa77	app/{vminsert,vmagent}: hide passwords and auth tokens by default at `/config` page Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764	2021-11-05 14:42:13 +02:00
Aliaksandr Valialkin	9edc12fa99	app/vmagent/remotewrite: fix parallel data sending to remote storage systems at `e0d2ba5608`	2021-11-04 16:58:53 +02:00
Aliaksandr Valialkin	63c29099ba	app/vmagent: add `-remoteWrite.maxRowsPerBlock` command-line option, which may be used for improving data ingestion performance under high load	2021-11-04 15:39:55 +02:00
Aliaksandr Valialkin	5f13ba8631	app/vmagent/remotewrite: send data to remote storage systems in parallel This should improve data ingestion speed many `-remoteWrite.url` options are configured	2021-11-04 15:04:02 +02:00
dependabot[bot]	08f487e7e8	build(deps-dev): bump @typescript-eslint/eslint-plugin (#183 ) Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) from 5.2.0 to 5.3.0. - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.3.0/packages/eslint-plugin) --- updated-dependencies: - dependency-name: "@typescript-eslint/eslint-plugin" dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-04 12:53:03 +02:00
Aliaksandr Valialkin	be9c7713d7	app/vmselect: `make vmui-update`	2021-11-04 12:16:55 +02:00
dependabot[bot]	c656e342dd	build(deps-dev): bump @babel/plugin-proposal-nullish-coalescing-operator (#1769 ) Bumps [@babel/plugin-proposal-nullish-coalescing-operator](https://github.com/babel/babel/tree/HEAD/packages/babel-plugin-proposal-nullish-coalescing-operator) from 7.14.5 to 7.16.0. - [Release notes](https://github.com/babel/babel/releases) - [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md) - [Commits](https://github.com/babel/babel/commits/v7.16.0/packages/babel-plugin-proposal-nullish-coalescing-operator) --- updated-dependencies: - dependency-name: "@babel/plugin-proposal-nullish-coalescing-operator" dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-04 12:16:55 +02:00
dependabot[bot]	bb8b426628	build(deps): bump @date-io/dayjs in /app/vmui/packages/vmui (#1770 ) Bumps [@date-io/dayjs](https://github.com/dmtrKovalenko/date-io) from 1.3.13 to 2.11.0. - [Release notes](https://github.com/dmtrKovalenko/date-io/releases) - [Commits](https://github.com/dmtrKovalenko/date-io/compare/v1.3.13...v2.11.0) --- updated-dependencies: - dependency-name: "@date-io/dayjs" dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-04 12:16:55 +02:00
Aliaksandr Valialkin	d445d22c0c	lib/promscrape: add `-promscrape.maxResponseHeadersSize` command-line flag for tuning the maximum http response headers size from Prometheus scrape targets	2021-11-03 22:27:55 +02:00
Aliaksandr Valialkin	a102fca4ac	app/vmselect/promql: add `limit_offset(limit, offset, q)` function, which can be used for paging over big number of time series Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1778	2021-11-03 16:03:10 +02:00
Aliaksandr Valialkin	43bf0333eb	app/vmselect/promql: add `label_graphite_group()` function for extracting groups from Graphite metric names	2021-11-03 13:20:34 +02:00
Aliaksandr Valialkin	551a94ba7f	app/{vmagent,vminsert}: add ability to restrict access to /config page with authKey query arg The authKey can be configured via `-configAuthKey` command-line flag. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764	2021-11-01 16:47:30 +02:00
Aliaksandr Valialkin	94d9c06fc1	docs/vmauth.md: typo fix	2021-10-28 14:06:28 +03:00
Aliaksandr Valialkin	38cbae006a	app/vmbackupmanager: fix links to images This is a follow-up after `bd6b8f7e31`	2021-10-27 21:36:25 +03:00
Yury Molodov	a16c88095c	vmui: correct migration material-ui (#1758 ) * migration material-ui * fix: rollback popover * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-27 20:43:24 +03:00
Aliaksandr Valialkin	2ebee4e741	app/{vmalert,vmagent}: improve the distribution of scrape offsets among targets / rules Previously only the lower part of 64-bit hash was used for calculating the offset. This may give uneven distribution in some cases. So let's use all the available 64 bits from the hash for calculating the offset.	2021-10-27 20:04:02 +03:00
Aliaksandr Valialkin	99784b21c1	all: fix build issues and tests for Apple M1 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1653	2021-10-27 15:07:19 +03:00
Aliaksandr Valialkin	ad445a06cd	lib/promscrape: properly show `proxy_url` option value at `/config` page Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755	2021-10-26 21:24:22 +03:00
Yury Molodov	e7591577f0	vmui: update dependencies (#1754 ) * update dependencies * update codemirror version * app/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-26 19:32:31 +03:00
Aliaksandr Valialkin	eb783b0ef3	app/vmbackup/README.md: sync with docs/vmbackup.md after `e706fb5686`	2021-10-26 19:21:26 +03:00
Roman Khavronenko	5321127add	vmalert: allow groups with empty rules for compatibility reasons (#1742 ) Prometheus allows to have groups with no rules, so we should support it in vmalert as well for compatibility reasons. It is also allowed to hot-reload empty groups by adding or removing rules. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-10-25 14:46:51 +03:00
Aliaksandr Valialkin	d8ce86d685	app/vmselect/promql: reduce the precision from 15 significant digits to 13 significant digits when comparing float64 results in tests See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1738	2021-10-24 13:32:25 +03:00
Aliaksandr Valialkin	559dd996c4	app/vmauth: add ability to specify http headers to send in requests to backends See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1736	2021-10-22 19:11:35 +03:00
Aliaksandr Valialkin	a8bcc3c276	lib/promscrape: reduce memory usage if `-promscrape.noStaleMarkers` command-line flag is passed Do not store in memory the response from the last scrape per each target if -promscrape.noStaleMarkers option is enabled. This should reduce memory usage when the scraped targets return large responses.	2021-10-22 13:22:08 +03:00
Roman Khavronenko	e0f21d6000	vmalert: correctly calculate alert ID including extra labels (#1734 ) Previously, ID for alert entity was generated without alertname or groupname. This led to collision, when multiple alerting rules within the same group producing same labelsets. E.g. expr: `sum(metric1) by (job) > 0` and expr: `sum(metric2) by (job) > 0` could result into same labelset `job: "job"`. The issue affects only UI and Web API parts of vmalert, because alert ID is used only for displaying and finding active alerts. It does not affect state restore procedure, since this label was added right before pushing to remote storage. The change now adds all extra labels right after receiving response from the datasource. And removes adding extra labels before pushing to remote storage. Additionally, change introduces a new flag `Restored` which will be displayed in UI for alerts which have been restored from remote storage on restart.	2021-10-22 12:31:35 +03:00
Yury Molodov	3ccc37a98b	vmui: query history (#1732 ) * feat: add query history * fix: change detect keyUp for nav query history * feat: set default query history * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-22 12:29:27 +03:00
Aliaksandr Valialkin	5705f4b6d1	lib/httpserver: expose command-line flags at `/flags` page This should simplify debugging. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695	2021-10-20 00:46:54 +03:00
Roman Khavronenko	ca49853664	vmalert: make group.ID() thread-safe (#1726 ) Commit fixes potential race condition when group update and generating of ID() happens simultaneously. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-10-19 16:45:52 +03:00
Roman Khavronenko	627224d493	vmalert: properly init SIGHUP listener before starting group manager (#1725 ) Regression was introduced during code refactoring. It potentially could lead to situation when SIGHUP signals were ignored while vmalert was still busy with initing group manager. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-10-19 16:36:08 +03:00
Aliaksandr Valialkin	0a1982b294	app/vmauth: fix metric name prefixes: `vmagent` -> `vmauth`	2021-10-19 15:30:16 +03:00
Yury Molodov	744f6a98eb	vmui: features (#1711 ) * feat: initial uPlot graph * feat: add zoom/pan for graph * fix: add zoom by ctrl/mac * fix: remove unused code * feat: add toggle cache for fetch * feat: add fix y-axis limits * fix: stop point events while panning * fix: change getting cursor position when scaling * feat: add cursor tooltip to graph * fix: uninstall chart.js * fix: change link for create an issue * fix: set default cache value to true * app/vmalert: follow-up after `0e2486df56` * docs/CHANGELOG.md: document `5416e18007` * app/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-18 15:19:01 +03:00
Roman Khavronenko	44a88b7458	vmalert: remove extra `/` from path in WEB interface (#1717 ) The extra `/` may cause issues when additional path prefixes are configured. Also, removing it makes it consistent with the rest of declarations. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-10-18 15:19:01 +03:00
Roman Khavronenko	93d4ef1239	vmctl: follow-up after `95d1d38595` (#1718 ) Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-10-18 15:19:01 +03:00
Aliaksandr Valialkin	9273b83cd0	app/vmalert: follow-up after `0e2486df56`	2021-10-18 15:08:42 +03:00
Miro Prasil	c0853d4bf8	vmctl influx convert bool to number (#1714 ) vmctl: properly convert influx bools into integer representation When using vmctl influx, the import would fail importing boolean fields with: ``` failed to convert value "some".0 to float64: unexpected value type true ``` This converts `true` to `1` and `false` to `0`. Fixes #1709	2021-10-18 14:59:17 +03:00
Alexander Rickardsson	0e1dbcd039	vmalert: add disablePathAppend to remote read (#1712 ) * vmalert: add disablePathAppend to remoteRead * docs: add docs for remoteRead.disablePathAppend	2021-10-18 14:59:17 +03:00
Alexander Rickardsson	63571e1334	vmalert: Redact passwords from error messages (#1713 )	2021-10-18 14:59:17 +03:00
Aliaksandr Valialkin	ed994873fd	app/vmselect/promql: randomize the static selection of time series returned from `limitk()` Sort series by a hash calculated from the series labels. This should guarantee "random" selection of the returned time series. Previously the selection could be biased, since time series were sorted alphabetically by label names and label values.	2021-10-16 21:16:34 +03:00
Aliaksandr Valialkin	da0adf446d	app/vmselect/promql: typo fix in comment: didsn't -> didn't	2021-10-16 13:00:10 +03:00
Aliaksandr Valialkin	0452a8d4e8	lib/promscrape: store the full response in stream parsing mode in scrapeWork.lastScrape byte slice This allows sending staleness marks and properly calculate scrape_series_added metric in stream parsing mode at the cost of the increased memory usage, since now the potentially big response is kept in the lastScrape byte slice per each scrapeWork. In practice the memory usage increase shouldn't be big, since the response size is usually much smaller than the parsed metrics from this response after the relabeling, which usually adds a big pile of target-specific labels per each metric.	2021-10-15 15:26:24 +03:00
Aliaksandr Valialkin	25421fa2ae	lib/promscrape: add `-promscrape.minResponseSizeForStreamParse` command-line option for automatic switching to stream parsing mode when scraping targets with big responses This should reduce memory usage when vmagent scrapes targets with non-uniform response sizes. This is common case in Kubernetes monitoring.	2021-10-14 12:30:55 +03:00
Aliaksandr Valialkin	bee130cc78	lib/promscrape: return error if `sample_limit` or `series_limit` options are set when stream parsing mode is enabled	2021-10-14 12:30:54 +03:00
Roman Khavronenko	f393145843	Adjust `http.Transport.MaxIdleConns` setting for vmauth/vmalert services (#1704 ) * vmalert: adjust `http.Transport.MaxIdleConns` value accordingly to `http.Transport.MaxIdleConnsPerHost` `http.Transport.MaxIdleConnsPerHost` setting is controlled by `datasource.maxIdleConnections` flag, while `http.Transport.MaxIdleConns` is inherited from DefaultTransport and is equal to `100`. The fix adjusts `http.Transport.MaxIdleConns` value if it is lower than `http.Transport.MaxIdleConnsPerHost`. Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmauth: adjust `http.Transport.MaxIdleConns` value accordingly to `http.Transport.MaxIdleConnsPerHost` `http.Transport.MaxIdleConnsPerHost` setting is controlled by `maxIdleConnsPerBackend` flag, while `http.Transport.MaxIdleConns` is inherited from DefaultTransport and is equal to `100`. The fix adjusts `http.Transport.MaxIdleConns` value if it is lower than `http.Transport.MaxIdleConnsPerHost`. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-10-13 19:23:52 +03:00
Roman Khavronenko	14995eece6	vmalert: add Source link to alerts UI (#1701 ) The source link is controlled by `external.url` and `external.alert.source` flags, in the same way as for alertmanager notifications. The source link is added to Alerts list view, and specific Alert view.	2021-10-13 15:26:57 +03:00
Aliaksandr Valialkin	aeedfe2fe2	app/vmagent: expose -promscrape.config contents at /config page as Prometheus does See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695	2021-10-12 16:27:37 +03:00
Aliaksandr Valialkin	48c3668b3d	app/vmselect/promql: add `atan2` binary operator, which is going to be added in Prometheus 2.31 See https://github.com/prometheus/prometheus/pull/9248	2021-10-11 21:17:23 +03:00
Aliaksandr Valialkin	b3d28f3872	app/vmselect/promql: add missing trigonometric functions, which are going to be added in Prometheus 2.31 See https://github.com/prometheus/prometheus/issues/9233	2021-10-11 21:17:21 +03:00
Aliaksandr Valialkin	f1e1d20ac4	app/vmselect/promql: consistently return the same set of time series from `limitk()` function This is the expected behaviour by most users.	2021-10-08 19:55:29 +03:00
Aliaksandr Valialkin	cf87050d83	app/vmui: follow-up after `7bfb44113e` * Run `vmui-update` * Document the changes in README.md and CHANGELOG.md	2021-10-08 15:11:09 +03:00
Yury Molodov	41a83a1cc6	vmui: use uPlot as default engine for graph (#1683 ) * feat: initial uPlot graph * feat: add zoom/pan for graph * fix: add zoom by ctrl/mac * fix: remove unused code	2021-10-08 15:11:07 +03:00
Aliaksandr Valialkin	4fddcf4c83	app/{vminsert,vmstorage}: follow-up after `a171916ef5` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/269	2021-10-08 14:09:51 +03:00
Nikolay	a171916ef5	Adds read-only mode for vmstorage node (#1680 ) * adds read-only mode for vmstorage https://github.com/VictoriaMetrics/VictoriaMetrics/issues/269 * changes order a bit * moves isFreeDiskLimitReached var to storage struct renames functions to be consistent change protoparser api - with optional storage limit check for given openned storage * renames freeSpaceLimit to ReadOnly	2021-10-08 12:52:56 +03:00
Aliaksandr Valialkin	cbf01f7384	app/vmselect/promql: substitute rollupFuncsCannotAdjustWindow with rollupFuncsCanAdjustWindow The list of functions, which can adjust lookbehind window is more limited than the rest of functions, so it is better from maintainability and readability PoV using the allowlist instead of blocklist.	2021-10-07 13:19:22 +03:00
Aliaksandr Valialkin	7e6ec0bbe9	app/vmselect/promql: return back the behaviour for `deriv()` function when the lookbehind window doesnt contain enough points It is expected that the `deriv(m[d])` returns non-empty value if the lookbehind window `d` contains less than 2 samples in the same way as `rate()` does. This is a follow-up after `3e084be06b` .	2021-10-07 12:55:58 +03:00
Roman Khavronenko	0a8804d6aa	app/vmselect: make `predict_linear` and `deriv` compatible with Prometheus (#1681 ) Previously, `predict_linear` returned slightly different results comparing to Prometheus. The change makes linear regression algorithm compatible with Prometheus. `deriv` was excluded from the list of functions which can adjust the time window for the same reasons.	2021-10-07 12:55:56 +03:00
Aliaksandr Valialkin	64b6f3f1c8	app/vminsert: fix uneven distribution of time series among storage nodes Use distinct seed for distribution hash calculations on the second level of vminsert nodes. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1672	2021-10-07 12:22:39 +03:00
Aliaksandr Valialkin	89700433cc	docs/vmagent.md: update docs after 3e9a939a990c8b608414388c96f68eb062364ae7	2021-10-05 10:23:37 +03:00
Aliaksandr Valialkin	a9420317d9	docs/vmagent.md: document how to write data to Kafka	2021-09-30 17:46:23 +03:00
Aliaksandr Valialkin	ec6eb03d65	app/vmselect/promql: add `topk_last` and `bottomk_last` functions	2021-09-30 13:23:27 +03:00
Roman Khavronenko	95ddfda894	app/vmselect: fix binary comparison func (#1667 ) The fix makes the binary comparison func to check for NaNs before executing the actual comparison. This prevents VM to return values for non-existing samples for expressions which contain bool comparisons. Please see added test for example.	2021-09-30 13:04:10 +03:00
Roman Khavronenko	57eb5ef194	app/vmselect: fix `testRowsEqual` func NaN checks (#1666 ) It appeared, that `testRowsEqual` NaN comparison was incorrect. The fix caused some tests to fail. Please see the change and tests updated. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-30 13:04:08 +03:00
Aliaksandr Valialkin	d51118e226	app/vminsert: document that -relabelConfig is reloaded on SIGHUP signal	2021-09-29 21:17:25 +03:00
Aliaksandr Valialkin	8dcf814c48	app/{vmbackup,vmrestore}: switch from `gcs://...` to `gs://...` urls for backups to GCS The `gs://` urls are commonly used, so prefer them instead of `gcs://` urls, while leaving support for `gcs://` urls for backwards compatibility.	2021-09-29 12:12:37 +03:00
Aliaksandr Valialkin	f35f9188bc	docs/vmagent.md: update Telegraf config in the section about Kafka	2021-09-29 12:01:46 +03:00
Aliaksandr Valialkin	16d7ee7b44	docs/vmagent.md: add docs about reading metrics from Kafka	2021-09-29 01:46:40 +03:00
Yury Molodov	a6037a8ce8	vmui: fixed bug with time range (time zone) (#1661 ) * fix: set date in query string in utc format * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-29 01:01:27 +03:00
Nikolay	6ca31acb76	changes vmagent api (#1656 ) * changes vmagent api adds auth.Token to promremotewrite InsertHandlerReader changes remoteWrite client constructor, allows to use multiple remoteWriteUrl schemes, like kafka:// changes url path concatenation for tenant remoteWrite Update app/vmagent/remotewrite/client.go Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com> * Update app/vmagent/remotewrite/remotewrite.go * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-29 00:55:14 +03:00
Roman Khavronenko	444a8882f4	app/vmselect: disable time-window adjustment for `min/max_over_time` funcs (#1658 ) Adjustment results into discrepancy between Prometheus and VM on time windows smaller than scrape interval. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:43:54 +03:00
Roman Khavronenko	f46f847531	app/vmselect: always return zero for `stddev` func if there is only one value (#1659 ) The fix will always return zero if received set of items consists of one element only, which also means no deviation. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:42:19 +03:00
Roman Khavronenko	72468c9d75	app/vmselect: return NaN instead of 0 for empty value sets (#1660 ) The change affects `count/stddev/stdvar_over_time` funcs and makes them to return NaN instead of zero when there is no datapoints in a time window. This is needed for improving compatibility with Prometheus. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:38:24 +03:00
Roman Khavronenko	c8ac8f44b2	app/vmselect: rm `quantile_over_time` fast-path optimisations (#1662 ) The removed fast path optimisations weren't consistent with `quantile` function behavior and results into discrepancy. Specifically, results didn't match in cases when: * 0 < phi > 1; * values contain only one element. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:35:59 +03:00
Aliaksandr Valialkin	4e65bfcc00	app/{vminsert,vmagent}: add ability to ingest data via DataDog "submit metrics" API See https://docs.datadoghq.com/api/latest/metrics/#submit-metrics Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/206	2021-09-29 00:12:26 +03:00
Yury Molodov	80df31b2ee	vmui: use Chart.js as default engine for graph (#1634 ) * feat: add Plotly as default engine for graph * fix: remove unused components * feat: use Chart.js as default engine graph * fix: correct styles for loader * feat: add zoom/pan for chart * feat: add height for chart * fix: remove unused code * fix: remove empty units from duration * fix: change debounce for pan to 500ms * fix: add utility for plugins register globally * fix: optimize render graph * feat: add buffer data for zoom * fix: add limits for zoom in/out * fix: change update data while zooming * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-27 22:27:49 +03:00
Aliaksandr Valialkin	4b3951fd86	app/vmselect/promql: add `rollup_scrape_interval(m[d])` function It calculates the min, max and avg scrape intervals for m over the given lookbehind window d	2021-09-27 19:21:55 +03:00
Aliaksandr Valialkin	eff31c10ec	app/vmselect/promql: follow-up after `526dd93b32` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612	2021-09-27 18:57:03 +03:00
Roman Khavronenko	6061464d80	app/vmselect: `quantile` func compatiblity with Prometheus (#1646 ) * app/vmselect: `quantile` func compatiblity with Prometheus The `quantile` func was previously calculated by https://github.com/valyala/histogram package. The result of such calculation was always the closest real value to requested quantile. While in Prometheus implementation interpolation is used. Such difference may result into discrepancy in output between Prometheus and VictoriaMetrics. This commit adds a Prometheus-like `quantile` function. It also used by other functions which depend on it, such as `quantiles`, `quantile_over_time`, `median` etc. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625 Signed-off-by: hagen1778 <roman@victoriametrics.com> * app/vmselect: `quantile` review fixes * quantile functions were split into multiple to provide different API for already sorted data; * float64sPool is used for reducing allocations. Items in pool may have different sizes, but defining a new pool was complicates due to name collisions; Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-27 18:57:02 +03:00
Aliaksandr Valialkin	a746fe8778	app/vmselect/promql: follow-up after `57b3320478`	2021-09-24 01:25:04 +03:00
Roman Khavronenko	7ac0559d4f	app/vmselect: make sorting for query result similar to Prometheus (#1647 ) * app/vmselect: make sorting for query result similar to Prometheus Updated sorting allows to get the order of series in result similar or equal to what Prometheus returns. The change is needed for compatibility reasons. * Update app/vmselect/promql/exec_test.go Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-24 01:25:03 +03:00
Aliaksandr Valialkin	1115c2f235	app/vmselect/promql: align the behavior of `or`, `and` and `unless` operators with `on (labels)` modifier to Prometheus Previously VictoriaMetrics could return unexpected result of the right-hand side operand had multiple time series with the given set of labels mentioned in `on(labels)`. See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1643	2021-09-24 00:56:35 +03:00
Aliaksandr Valialkin	f93146c81e	docs: `make docs-sync`	2021-09-23 20:55:29 +03:00
Roman Khavronenko	f65ca2ac05	app/vmctl: fix misleading comment about cluster version for native mode (#1648 ) https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1637 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-23 20:47:56 +03:00
Roman Khavronenko	1bdccfc2bf	app/vmalert: remove unnecessary `omitempty` tag for `interval` param (#1649 ) `omitempty` tag resulted into skipping this param on marshaling, which was used as a checksum for groups configuration. Since on config reload checksums are compared before applying changes, any change to `interval` only didn't trigger config reload. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1641 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-23 20:47:56 +03:00
Roman Khavronenko	aa052097cf	app/vmalert: support `http.pathPrefix` flag in UI (#1636 ) The change makes UI to respect `http.pathPrefix` flag for API or navigation items links.	2021-09-21 18:15:08 +03:00
Aliaksandr Valialkin	3f1104ea74	app/vmselect: fix accessing `/graphite/*` endpoints	2021-09-21 14:06:18 +03:00
Aliaksandr Valialkin	e8700b57b9	docs/vmagent.md: typo fix	2021-09-20 16:49:32 +03:00
Aliaksandr Valialkin	233c6a8a82	docs/vmagent.md: typo fixes in `Prometheus staleness markers` docs	2021-09-20 16:44:42 +03:00
Nikolay	dd53abf36d	changes protoparser apis for accepting reading from io.Reader (#1624 ) adds InsertHandlerForReader apis to vmagent	2021-09-20 14:54:20 +03:00
n4mine	a5fcc9e356	fix: typo, dddresses -> addresses (#1630 )	2021-09-20 14:27:34 +03:00
Aliaksandr Valialkin	76c650bbda	app/vmselect/promql: add `quantiles_over_time("phiLabel", phi1, ..., phiN, m[d])` function for calculating multiple quantiles at once	2021-09-17 23:35:04 +03:00
Aliaksandr Valialkin	81d24045c2	app/vmselect/promql: add histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) function This function calculates multiple quantiles over the given buckets at once Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573	2021-09-17 13:39:06 +03:00
Aliaksandr Valialkin	4ed7548ced	app/vmselect/promql: optimize quantiles() calculation Calculate quantiles in one go instead of calculating each quantile individually	2021-09-17 12:38:21 +03:00
Roman Khavronenko	a07fa92ef2	vmalert: add new metric `vmalert_remotewrite_flush_duration_seconds` (#1622 )	2021-09-16 14:01:24 +03:00
f41gh7	348f705d95	follow-up after e2f99214f3e518f79c1c96ff918820a69d350ba1	2021-09-16 13:56:28 +03:00
Roman Khavronenko	05bf6cca82	vmselect: add stub for `graphite/functions` requests The request `graphite/functions` is sent by Grafana datasource and results into error logs in vmselect.	2021-09-16 13:51:37 +03:00
Aliaksandr Valialkin	3f8de4f82f	app/vmselect/promql: add `mad(q)` and `outliers_mad(tolerance, q)` functions to MetricsQL	2021-09-16 13:35:38 +03:00
Aliaksandr Valialkin	a4eaf1a0a1	app/vmselect/prometheus: make more clear log messages for errors during sending data to remote clients	2021-09-16 13:35:37 +03:00
Aliaksandr Valialkin	7ad54041fe	app/{vminsert,vmselect}: automatically add missing port in `-storageNode` lists passed to `vminsert` and `vmselect` This should simplify manual setup of the cluster according to https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#cluster-setup	2021-09-15 18:04:30 +03:00
Aliaksandr Valialkin	794d941125	app/vmui: follow-up after `777ff75874` The commit contains the following changes: - Show vmui when requesting /graph page in order to be compatible with Prometheus datasource in Grafana. - Properly encode query args at vmui url. - Set the number of points on the graph to the number of horizontal pixels divided by 2. Previously it was hardcoded to 30. - Do not save server url to persistent storage at browser, since it should be always obtained from the url. - Run `make vmui-update` for updating vmui embedded into VictoriaMetrics.	2021-09-15 17:41:00 +03:00
Yury Molodov	df87362ef6	vmui: change query params compatible with prometheus (#1619 ) * feat: change url params for compatible prometheus * style: add comment for TimeParams * fix: change get default server for single version * fix: change function for get query string value	2021-09-15 16:36:39 +03:00
Roman Khavronenko	286b79dc48	vmalert: create basic auth config only if args aren't empty (#1618 ) * vmalert: create basic auth config only if args aren't empty follow-up after `68721f6` * vmalert: make lint happy	2021-09-15 09:35:38 +03:00
Aliaksandr Valialkin	63fb5a5ca5	docs/vmalert.md: follow-up after `68721f6e7d` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1608	2021-09-14 14:48:45 +03:00
Roman Khavronenko	eea6317d40	vmalert: support bearer token for datasource, remotewrite and remoteread (#1614 ) https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1608	2021-09-14 14:48:43 +03:00
Aliaksandr Valialkin	5e5ce27df7	app/vmauth: do not log invalid auth tokens by default for security reasons The logging can be enabled by passing `-logInvalidAuthTokens` command-line flag to vmauth	2021-09-14 12:20:17 +03:00
Aliaksandr Valialkin	a8f1b614e5	docs/vmagent.md: mention `out of order sample` errors, which are typically emitted by Thanos, Cortex or Prometheus	2021-09-13 19:36:09 +03:00
Aliaksandr Valialkin	a1b298a842	app/vminsert/netstorage: disable rerouting by default Production clusters work more stable with the disabled rerouting during rolling restarts and/or during spikes in time series churn rate. So it would be better disabling the rerouting by default. The re-routing can be enabled by passing `-disableRerouting=false` command-line flag to `vminsert` nodes. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/791 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1054 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1165	2021-09-13 18:50:47 +03:00
Aliaksandr Valialkin	2932f495f3	docs/CHANGELOG.md: document `5494bc02a6`	2021-09-13 17:14:45 +03:00
Roman Khavronenko	7df731ea91	vmalert: add flag to limit the max value for auto-resovle duration for alerts (#1609 ) * vmalert: add flag to limit the max value for auto-resovle duration for alerts The new flag `rule.maxResolveDuration` suppose to limit max value for alert.End param, which is used by notifiers like Alertmanager for alerts auto resolve. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1586	2021-09-13 17:14:45 +03:00
Aliaksandr Valialkin	184e145570	docs: consistency renaming: Influx -> InfluxDB	2021-09-13 17:14:45 +03:00
Aliaksandr Valialkin	101cdef0a7	app/vmui/Dockerfile-web: update Go builder from 1.16.7 to 1.17.1 and Alpine base image from 3.14.1 to 3.14.2	2021-09-13 17:14:45 +03:00
Aliaksandr Valialkin	b721110902	docs/vmbackup.md: update the outdated link to vmbackupmanager	2021-09-13 14:17:53 +03:00
Roman Khavronenko	9dd121db36	vmalert: display extra filter labels in UI (#1613 )	2021-09-13 14:17:51 +03:00
Aliaksandr Valialkin	19516aedb5	app/vmselect/promql: use Prometheus-compatible label value formatting for `count_values` function	2021-09-13 13:48:18 +03:00
Aliaksandr Valialkin	2b64c1c95e	docs/Single-server-VictoriaMetrics.md: refer to relabeling section for vmagent This removes duplicate docs about additional relabeling actions supported by VictoriaMetrics components	2021-09-12 11:40:23 +03:00
Aliaksandr Valialkin	5aaaa686a4	lib/promscrape: send stale markers for disappeared metrics like Prometheus does	2021-09-11 11:02:56 +03:00
Aliaksandr Valialkin	e1632dc0df	docs/vmalert.md: typo fix in `Multitenancy` chapter	2021-09-10 17:57:43 +03:00
Aliaksandr Valialkin	c2f37f049b	lib/storage: properly search series by multiple tag filters matching empty labels such as foo{bar=~"baz\|",x=~"y\|"} Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1601 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/395	2021-09-09 21:12:53 +03:00
Aliaksandr Valialkin	9b07cb2988	app/vmselect/promql: keep metric name in `rollup_candlestick` results, since they don't change the original series meaning Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1600	2021-09-09 19:24:10 +03:00
Aliaksandr Valialkin	dca6f0f7de	app/vminsert/netstorage: remove the limit on the number of `-storageNode` addresses There is no any reasons to limit the number of `-storageNode` addresses to 255. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1599	2021-09-09 19:11:36 +03:00
Aliaksandr Valialkin	d613a018c8	lib/promscrape: add ability to set `series_limit` and `stream_parse` options via relabeling This allows managing these options on a per-target basis. Typical use case: to manage these options for pods via Kubernetes annotations.	2021-09-09 18:51:23 +03:00
Aliaksandr Valialkin	986807ebd9	docs/vmagent.md: typo fix in `Relabeling` chapter	2021-09-09 16:39:34 +03:00
Aliaksandr Valialkin	75c3514c5c	lib/promrelabel: add `keep_metrics` and `drop_metrics` actions to relabeling rules These actions simlify metrics filtering. For example, - action: keep_metrics regex: 'foo\|bar\|baz' would leave only metrics with `foo`, `bar` and `baz` names, while the rest of metrics will be deleted. The commit also makes possible to split long regexps into multiple lines. For example, the following config is equivalent to the config above: - action: keep_metrics regex: - foo - bar - baz	2021-09-09 16:25:09 +03:00
Aliaksandr Valialkin	d72593cca2	app/vmalert: document GroupAlerts This makes golint happy	2021-09-07 22:50:48 +03:00
Aliaksandr Valialkin	608ed19655	app/vmalert: follow-up after `21f022e5f0`	2021-09-07 22:45:47 +03:00
Roman Khavronenko	bb26aa9b1f	vmalert: add initial UI implementation (#1602 ) New UI pages: / - welcome page with API handlers list; /groups - list of all rules per group; /alerts - list of all active alerts; /groupID/alertID/status - status of the active alert;	2021-09-07 22:45:45 +03:00
Aliaksandr Valialkin	304661c547	docs/vmagent.md: add API path for Prometheus text exposition format	2021-09-07 16:15:28 +03:00
Aliaksandr Valialkin	5c58c8d55e	docs: update `-help` output for victoria-metrics and vmagent after `f77dde837a`	2021-09-01 16:35:18 +03:00
Aliaksandr Valialkin	c4df601f43	lib/promscrape: add the ability to limit the number of unique series per each scrape target The number of series per target can be limited with the following options: * Global limit with `-promscrape.maxSeriesPerTarget` command-line option. * Per-target limit with `max_series: N` option in `scrape_config` section. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1561	2021-09-01 16:08:12 +03:00
Roman Khavronenko	8962a50478	vmctl: update README and flags description (#1588 ) The purpose of update is to make README and flags description more clear to the reader. Especially, show that vm-account-id flag is required for clustered version of VM.	2021-09-01 12:22:58 +03:00
Roman Khavronenko	1cf4f5a715	Vmalert extra params (#1587 ) * vmalert: allow extra GET params in datasource package ExtraParams will be added as GET params to every HTTP request made by datasource. The `roundDigits` param, for example, was substituted by corresponding extra param. * vmalert: add nocache=1 param for replay process The `nocache=1` param is VictoriaMetrics specific parameter which prevents it from caching and boundaries aligning for queries. We set it to avoid cache pollution in `replay` mode and also to avoid unnecessary time range boundaries alignment. * vmalert: mention nocache=1 in replay description * vmalert: fix bug with unused param	2021-09-01 12:20:01 +03:00
Nikolay	210c76b51e	adds external_labels per group for vmalert (#1485 ) * adds external_label per group for vmalert https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1471	2021-09-01 12:19:49 +03:00
Roman Khavronenko	1cb7037fc8	Vmalert metrics update (#1580 ) * vmalert: remove `vmalert_execution_duration_seconds` metric The summary for `vmalert_execution_duration_seconds` metric gives no additional value comparing to `vmalert_iteration_duration_seconds` metric. * vmalert: update config reload success metric properly Previously, if there was unsuccessfull attempt to reload config and then rollback to previous version - the metric remained set to 0. * vmalert: add Grafana dashboard to overview application metrics * docker: include vmalert target into list for scraping * vmalert: extend notifier metrics with addr label The change adds an `addr` label to metrics for alerts_sent and alerts_send_errors to identify which exact address is having issues. The according change was made to vmalert dashboard. * vmalert: update documentation and docker environment for vmalert's dashboard Mention Grafana's dashboard in vmalert's README in a new section #Monitoring. Update docker-compose env to automatically add vmalert's dashboard. Update docker-compose README with additional info about services.	2021-09-01 12:19:34 +03:00
Aliaksandr Valialkin	3788d4f4eb	app/vmselect: show useful endpoints when requested `/select/<accountID>/` page	2021-08-29 12:05:14 +03:00
Aliaksandr Valialkin	39bb6bdd79	app/vmselect/promql: add `quantile("phiLabel", phi1, ..., phiN, q)` aggregate function to MetricsQL See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573	2021-08-27 18:40:25 +03:00
Aliaksandr Valialkin	6e5864b014	docs/{vmgateway,vmbackupmanager}: mention that enterprise binaries are free for download and evaluation	2021-08-27 14:53:48 +03:00
Aliaksandr Valialkin	102ab795f8	docs/vmagent.md: document the ability to load scrape configs from multiple files See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1559	2021-08-26 09:13:54 +03:00
benclive	76816a0193	Remove trailing slash for URLPrefixes with specific path (#1554 )	2021-08-25 13:32:04 +03:00
Aliaksandr Valialkin	335de30083	app/vmselect/promql: `make fmt` after `0078486ea7`	2021-08-23 23:05:34 +03:00
Aliaksandr Valialkin	40b06e84f8	app/vmselect/promql: rename `sign()` function to `sgn()` in order to be consistent with Prometheus See https://github.com/prometheus/prometheus/pull/8457 for details.	2021-08-23 11:46:29 +03:00
Aliaksandr Valialkin	ff4c7c1a3d	docs/vmalert.md: run `make docs-sync` after `9ee3d0378f`	2021-08-21 20:25:26 +03:00
Roman Khavronenko	0c2284b95f	vmalert: add flag `disableAlertgroupLabel` for disabling extra label added to series (#1534 ) The new label added in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/611 may negatively impact deduplication in Alertmanager. The new flag supposed to give an option to disable adding this label. To enable flag just add `-disableAlertgroupLabel` to binary execution command. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1532	2021-08-21 20:23:22 +03:00
Alexander Rickardsson	9e2e9d83a5	vmalert: accept http.StatusOK for remotewrite (#1550 )	2021-08-21 20:23:22 +03:00
Aliaksandr Valialkin	91534057a3	app/vmselect/prometheus: do not extend `[d]` to the detected interval between samples for `first_over_time(m[d])` This is for the sake of consistency with similar change for the last_over_time(m[d]) at `a724229b5d`	2021-08-21 19:56:56 +03:00
Roman Khavronenko	1ccb77904b	vmselect: update `vm_request_duration_seconds` value when request fails (#1537 ) Before, metric `vm_request_duration_seconds` was update only on successful attempts which could be misleading. For example, timeout errors on netstorage request may be not accounted in the metric and won't be visible on dashboards. Using `defer` statement to update the metric after query arguments validation may improve the situation.	2021-08-19 14:07:00 +03:00
Aliaksandr Valialkin	ee1f3414d1	app/vmselect/promql: do not override `[d]` at `last_over_time(m[d])` if `[d]` is smaller than `scrape_interval` Since most users do not expect the overriding of explicitly set `[d]`.	2021-08-19 10:33:10 +03:00
Aliaksandr Valialkin	5d92fafc40	app/vmselect: add `-search.noStaleMarkers` command-line flag for disabling stale markers handling in queries This option allows reducing CPU usage a bit when VictoriaMetrics is used for collecting and processing non-Prometheus data. For example, InfluxDB line protocol, Graphite, OpenTSDB, CSV, etc.	2021-08-18 13:58:06 +03:00
Aliaksandr Valialkin	f21fad53b4	lib/promscrape: add ability to disable sending Prometheus staleness markers with -promscrape.disableStaleMarkers command-line flag This option can be useful when vmagent consumes too much additional memory for staleness markers functionality and when staleness markers aren't needed.	2021-08-18 13:58:05 +03:00
Aliaksandr Valialkin	49886ecbc8	app/vmselect/promql: add bitmap_and(), bitmap_or() and bitmap_xor() functions to MetricsQL Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1541	2021-08-17 13:22:15 +03:00
Aliaksandr Valialkin	38065bec7b	app/vmselect/promql: move common condition to dropStaleNaNs in order to improve code maintainability	2021-08-17 11:00:58 +03:00
Aliaksandr Valialkin	fe8c462044	app/vmalert: mention -remoteWrite.disablePathAppend in the description for -remoteWrite.url	2021-08-16 15:23:39 +03:00
Aliaksandr Valialkin	21974cb571	app/vmalert: follow-up for `2400f85761`	2021-08-16 15:20:35 +03:00
Alexander Rickardsson	d27dc3721b	vmalert: enable configuring explicit path (#1536 ) * vmalert: allow to disable automatically added path to remote write address via disablePathAppend flag * docs: update docs to include remoteWrite.disablePathAppend	2021-08-16 14:58:05 +03:00
Aliaksandr Valialkin	48920bdef8	app/vmagent/remotewrite: expose vmagent_remotewrite_send_duration_seconds_total metric This metric can be used for determining high saturation of every connection to remote storage with an alerting query `rate(vmagent_remotewrite_send_duration_seconds_total) > 0.9s`. This query triggers when a connection is satureated by more than 90%	2021-08-15 13:34:07 +03:00
Aliaksandr Valialkin	5420c3d967	app/vmselect/promql: drop staleness marks before calling rollupConfig.Do This allows dropping staleness marks only once and then calculate multiple rollup functions on the result. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526	2021-08-15 13:22:26 +03:00
Aliaksandr Valialkin	6c4c54eaad	Revert "app/vmselect/promql: properly handle Prometheus staleness marks in removeCounterResets functions" This reverts commit 94dfcb6747a3b29a11d14e71bea21a2312bb6346. It is better to remove staleness marks (decimal.StaleNaN) before calling rollupConfig.Do, e.g. in preFunc	2021-08-15 13:22:24 +03:00
Aliaksandr Valialkin	af4a306d7b	app/vmselect/promql: properly handle Prometheus staleness marks in removeCounterResets functions Prometheus stalenss marks shouldn't be changed in removeCounterResets. Otherwise they will be converted to an ordinary NaN values, which couldn't be removed in dropStaleNaNs() function later. This may result in incorrect calculations for rollup functions. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526	2021-08-14 12:45:31 +03:00
Aliaksandr Valialkin	c1f81f08d4	all: add support for Prometheus staleness markers Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845	2021-08-13 12:13:15 +03:00
Aliaksandr Valialkin	b35ae791f1	app/vmselect: `make vmui-update` after the commit 4ae14df864a7e327955f44941295a286175423b3	2021-08-11 13:42:53 +03:00
Aliaksandr Valialkin	f60ff85dbe	app/vmui: actualize Dockerfiles	2021-08-11 13:42:53 +03:00
Aliaksandr Valialkin	9eb828b2c2	app/vminsert: add vm_rpc_send_duration_seconds_total metric per each `vminsert->vmstorage` link This metric is useful for determining high link saturation with the following alerting rule: rate(vm_rpc_send_duration_seconds_total) > 0.9s	2021-08-11 11:42:33 +03:00
Aliaksandr Valialkin	90efb5831b	lib/envflag: add a link to docs for -envflag.enable	2021-08-11 10:32:40 +03:00
Yury Molodov	aca2cb245e	vmui: fix layout and add server url by default (#1519 ) * fix: change layout for correctly display big query * fix: set default server from url * fix: change get default server url	2021-08-06 12:16:53 +03:00
Roman Khavronenko	d5ba8248cc	vmalert: expose new metrics for tracking number of produced samples during last evaluation (#1518 ) * vmalert: expose new metrics for tracking number of produced samples during last evaluation Two new metrics were added to track the number of samples produced during the last evaluation: * vmalert_recording_rules_last_evaluation_samples * vmalert_alerting_rules_last_evaluation_samples The gauge type is used to remain consistent with Prometheus metric `prometheus_rule_group_last_evaluation_samples` which is on the group level. However, the counter type was considered as well. Two metrics instead of one are used to make it easier to separate recording and alerting rules. It is likely, number of samples produced by recording rules is more important so people will refer to it more frequently. The expected usage of the new metric is the following: ``` - alert: RecordingRuleReturnsEmptyResults expr: sum(vmalert_recording_rules_last_evaluation_samples) by(recording) < 1 annotations: summary: Recording rule {{$labels.recording}} returns empty results. Please verify expression correctness. ``` Addresses https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1494 * vmalert: rename `vmalert_alerts_error` to `vmalert_alerting_rules_error` to remain consistent with recording rules metrics	2021-08-05 10:02:35 +03:00
Aliaksandr Valialkin	13d438d808	app/vmagent: typo fix in the description for `-remoteWrite.queues`	2021-08-05 10:00:58 +03:00
Aliaksandr Valialkin	b877538622	app/vmagent: follow-up after `fe445f753b` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1491	2021-08-05 09:51:00 +03:00
Omar Ghader	fe445f753b	feature: Add multitenant for vmagent (#1505 ) * feature: Add multitenant for vmagent * Minor fix * Fix rcs index out of range * Minor fix * Fix multi Init * Fix multi Init * Fix multi Init * Add default multi * Adjust naming * Add TenantInserted metrics * Add TenantInserted metrics * fix: remove unused metrics for vmagent * fix: remove unused metrics for vmagent Co-authored-by: mghader <marc.ghader@ubisoft.com> Co-authored-by: Sebastian YEPES <syepes@gmail.com>	2021-08-05 09:44:29 +03:00
Qifei Wan	095bb90879	app/vmalert: update config state metrics if config parsed failed (#1507 )	2021-08-03 16:12:48 +03:00
Aliaksandr Valialkin	60cfa5f100	app/vmselect/promql: add `present_over_time(m[d])` function, which will be available starting from Prometheus 2.29.0 See https://github.com/prometheus/prometheus/releases/tag/v2.29.0-rc.0 and https://github.com/prometheus/prometheus/pull/9097	2021-08-03 12:21:53 +03:00
wusphinx	511e5c2e68	Update TimeSelector.tsx (#1515 ) delete garbled code	2021-08-03 11:14:56 +03:00
Nikolay	3f3ad13753	adds /rules and /alerts api for grafana (#1504 ) Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-08-02 17:29:49 +03:00
Aliaksandr Valialkin	99004a6a40	app/vmselect/netstorage: unpack time series data in mostly local big chunks This should improve performance on multi-CPU systems for queries selecting time series with big number of raw samples	2021-07-30 12:26:33 +03:00
Aliaksandr Valialkin	c473d8ffe1	li/storage: re-use the per-day inverted index search code for searching in global index This allows removing a big pile of outdated code for global index search. This may help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1486	2021-07-30 10:28:20 +03:00
Aliaksandr Valialkin	cbb81c2ce9	app/vmselect/netstorage: do not query Go maps with unsafe string keys, since this breaks in Go 1.17	2021-07-30 10:28:19 +03:00
Aliaksandr Valialkin	b709fa387a	app/vmselect: follow-up for `ed95bc9531` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1493	2021-07-29 09:48:47 +03:00
arnoldyahad	ed95bc9531	Add case prometheus/rules for grafana 8 (#1502 )	2021-07-29 06:15:35 +03:00
assassins	6ab0001a1f	Performance optimization (#1481 ) There are redundant steps	2021-07-28 19:29:22 +03:00
Aliaksandr Valialkin	49bf3abf67	app/vmselect: follow-up for `626073bca8` * Rename -search.maxMetricsPointSearch to -search.maxSamplesPerQuery, so it is more consistent with the existing -search.maxSamplesPerSeries * Move the -search.maxSamplesPerQuery from vmstorage to vmselect, so it could effectively limit the number of raw samples obtained from all the vmstorage nodes * Document the -search.maxSamplesPerQuery in docs/CHANGELOG.md	2021-07-28 18:00:04 +03:00
匠心零度	626073bca8	protection vmselect ,avoid metrics point too much let vmselect cup load very, very high (#1478 ) * protection vmselect…… * protection vmselect…… * protection vmselect…… * All checks have failed,fix Co-authored-by: lirenzuo <lirenzuo@shein.com>	2021-07-28 14:39:35 +03:00
Aliaksandr Valialkin	5d255846ac	all: add `go:build` lines for Go1.17 See https://tip.golang.org/doc/go1.17#gofmt for more details	2021-07-26 15:50:46 +03:00
Aliaksandr Valialkin	3921d8afae	app/vmselect: prevent from possible deadlock when f callback blocks inside RunParallel	2021-07-26 15:50:45 +03:00
Aliaksandr Valialkin	c3e6ce1db9	app/vmselect: `make vmui-update` after `a91d41f12a`	2021-07-26 10:32:01 +03:00
Yury Molodov	401de2dca4	Vmui/query editor (#1472 ) * fix: move request button to server input * feat: add switch for query autocomplete * refactor: rename state for popover open * feat: add detect os by userAgent * fix: change hotkey to run query for mac * fix: change detect mac os * fix: change div to span inside Typography Co-authored-by: yury <yurymolodov@victoriametrics.com>	2021-07-23 21:08:58 +03:00
Aliaksandr Valialkin	b047feeb8b	app/vmselect/promql: properly handle `(a op b) default N` if `(a op b)` returns NaN series The result should be a series with `N` values and `a op b` labels. Previously such series has been removed from the result.	2021-07-16 01:44:24 +03:00
Aliaksandr Valialkin	b92702f6d5	app/vmselect/netstorage: use more scalable algorithm for ditributing the work among among multiple channels on systems with big number of CPU cores	2021-07-16 00:35:36 +03:00
Aliaksandr Valialkin	df117f85bd	app/vmselect: do not track queries with less than 1ms execution time at /api/v1/status/top_queries This should improve the readability and usefullness of the /api/v1/status/top_queries when debugging slow queries or queries that take too much cpu time.	2021-07-15 16:53:35 +03:00
Aliaksandr Valialkin	5830ce2706	app/vmselect/netstorage: add `-search.maxSamplesPerSeries` command-line option for limiting the number of samples a query can process per each series This should prevent from out of memory crashes like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1067	2021-07-15 16:53:35 +03:00
Aliaksandr Valialkin	6c42db87a8	app/vmselect/netstorage: improve scalability of series unpacking on multi-CPU systems	2021-07-15 15:40:53 +03:00
Aliaksandr Valialkin	3059e4feec	app/vmui/README.md: typo fix: naviate->navigate	2021-07-15 15:02:56 +03:00
Aliaksandr Valialkin	9add9d86a6	app/vmselect/promql: duration handling improvements in MetricsQL queries - Support durations anywhere in MetricsQL queries. E.g. sum_over_time(m[1h])/1h is equivalent to sum_over_time(m[1h])/3600 - Support durations without suffix. E.g. rate(m[300]) is equivalent to rate(m[5m])	2021-07-12 17:19:32 +03:00
Aliaksandr Valialkin	d98e22fe50	app/vmalert: accept Prometheus-like durations in `interval` config option inside `group` section	2021-07-12 12:36:22 +03:00
Aliaksandr Valialkin	f5fa177141	Revert "app/vmselect: expose vmui at /select/<accountID>/prometheus/vmui additionally to /select/<accountID>/vmui" This reverts commit 885a79def6799f288e14df05b35a12569659ab85. Reason for revert: Grafana doesn't allows accessing /select/<accountID>/prometheus/vmui :(	2021-07-12 09:08:43 +03:00
Aliaksandr Valialkin	ddaa12050d	app/vmselect: expose vmui at /select/<accountID>/prometheus/vmui additionally to /select/<accountID>/vmui The /select/<accountID>/prometheus/vmui is needed for accessing via server-side Prometheus datasource for Grafana. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413	2021-07-10 12:52:25 +03:00
Aliaksandr Valialkin	0b98f6c7ff	app/vmselect: expose vmui at `/vmselect/<accountID>/vmui/` instead of `/vmselect/<accountID>/prometheus/vmui/` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413	2021-07-10 12:32:21 +03:00
Aliaksandr Valialkin	98e049ba6d	app/vmui: move source code from https://github.com/VictoriaMetrics/vmui to app/vmui Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413	2021-07-09 17:13:51 +03:00
Aliaksandr Valialkin	2c5e1cd893	app/vmselect: move web ui from /ui to /select/<accountID>/prometheus/ui This way the UI is available for every tenant (aka accountID) and the UI can automatically determine the needed per-tenant datasource path from page referer. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413	2021-07-08 13:14:50 +03:00
Aliaksandr Valialkin	acb7a95c64	app/vmselect: follow-up after `aa11ef6d3b` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4	2021-07-07 17:45:09 +03:00
tony	aa11ef6d3b	add vmui for vmselect component (#1431 ) Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-07-07 17:04:23 +03:00
Aliaksandr Valialkin	9c19719ad6	app/{vminsert,vmselect}: export vminsert_request_duration_seconds and vmselect_request_duration_seconds histograms	2021-07-07 13:27:23 +03:00
Aliaksandr Valialkin	ceda2b1df4	lib/httpserver: print full requestURI in httpserver.Errorf This should simplify debugging.	2021-07-07 13:11:29 +03:00
Aliaksandr Valialkin	22c6e64bbc	lib/storage: consistency renaming: tagCache -> tagFiltersCache This improves code readability	2021-07-06 11:03:30 +03:00
Aliaksandr Valialkin	44855f0c9b	app/{vmselect,vmstorage}: clarify the description for `-dedup.minScrapeInterval` command-line flag Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1426	2021-07-02 15:06:41 +03:00
Aliaksandr Valialkin	4d8ab5d9fa	docs/vmagent.md: mention about docker_sd_config support	2021-06-25 20:53:09 +03:00
Aliaksandr Valialkin	856aecae05	app/vmselect/promql: return the last timestamp for the max / min value from `tmax_over_time()` and `tmin_over_time()` function as most users expect	2021-06-23 14:18:37 +03:00
Aliaksandr Valialkin	c18017a9c3	app/vminsert/netstorage: sort the `-storageNode` list passed to `vminsert` nodes This should reduce resource usage (CPU, RAM, disk IO) at vmstorage nodes if the addresses of vmstorage nodes are passed in random order to vminsert nodes.	2021-06-23 14:00:08 +03:00
Nikolay	e03a3d3a36	adds http_sd (#1399 ) * adds http_sd * adds X-Prometheus-Refresh-Interval-Seconds header * Update lib/promscrape/discovery/http/api.go Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-06-22 13:42:09 +03:00
Roman Khavronenko	79474baf99	vmctl: add more context to flags description in vm-native mode (#1395 )	2021-06-18 19:20:52 +03:00
Aliaksandr Valialkin	b92d110cad	app/vmselect: log slow requests to all the `/api/v1/*` handlers if their execution time exceeds `-search.logSlowQueryDuration`	2021-06-18 19:07:03 +03:00
Aliaksandr Valialkin	4acc4602b3	app/vmctl: limit JSON line size by 10K samples (#1394 ) This should reduce the maximum memory usage at VictoriaMetrics when importing time series with big number of samples.	2021-06-18 15:41:34 +03:00
Aliaksandr Valialkin	60bc35f550	docs/{vmgateway,vmbackupmanager}: explicitly mention that these components are a part of an enterprise package	2021-06-17 17:19:13 +03:00
Aliaksandr Valialkin	51fc469642	app/vmagent/remotewrite: `go fmt` after `0a796f7c3a`	2021-06-17 13:51:40 +03:00
Zongyang	cf506e300d	Change default value of '-remoteWrite.queues' to cgroup.AvailableCPUs * 2 (#1385 ) * Change default value of '-remoteWrite.queues' to cgroup.AvailableCPUS() * 2 to reduce scrape interval Default value of vmagent option '-remotewrite.queues' is 4 and default size of vmagent ScheudleUnmarshalWorkers is number of CPUs, when available CPUs is much greater than 4, e.g 32, worker are competing push queues which will increase scrape interval and may cause scrape timeout. * Update README and flag description Co-authored-by: xiaozy <xiaozy01@fenbi.com>	2021-06-16 12:37:55 +03:00
Roman Khavronenko	a15c947045	promql: fix `increase_pure` calculation for cases with stale series (#1381 ) Due to staleness handling, increase_pure were using incorrect previous value during calculation in cases where series disappears for period longer than staleness period and then returns back. The fix suppose to account for a real datapoint value before staleness takes place. The fix should remove unexpected spikes while using `increase_pure` for staled series.	2021-06-15 17:37:51 +03:00
Nikolay	e42da47608	adds digital ocean sd (#1376 ) * adds digital ocean sd config * adds digital ocean sd https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1367 * typo fix	2021-06-14 13:19:29 +03:00
Roman Khavronenko	c5f493db8e	Vmalert docs (#1372 ) * vmalert: mention what happens if `for` is set to 0 or omitted * vmalert: add more context to docs	2021-06-14 11:43:01 +03:00
Aliaksandr Valialkin	0672cfffa2	app/vmauth: properly handle http.ErrAbortHandler panic This panic can be raised by the reverseProxy on aborted request to the backend. So handle it (e.g. suppress) at reverseProxy.ServeHTTP call. Do not suppress the panic at lib/httpserver generic HTTP handler, since it may result in an inconsistent state left after the panicking handler. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1353	2021-06-11 12:54:37 +03:00
Roman Khavronenko	f3cb2158a3	vmalert: fix mistake with object reuse while parsing response (#1370 ) * vmalert: fix mistake with object reuse while parsing response During the refactoring, the wrong optimisations was applied in parse function which caused metric fields reset. The change removes optimisation. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1369 * vmalert: add test to cover multiple metrics in one response	2021-06-11 11:30:07 +03:00
John Belmonte	3e79f3994e	spelling fix: synonym (#1363 )	2021-06-11 10:58:48 +03:00
Aliaksandr Valialkin	e8e7f03394	app/vmselect/promql: typo fix in the comment	2021-06-09 18:34:57 +03:00
Aliaksandr Valialkin	247b2a5a08	app/vmauth: improve readability for a config with multiple `src_paths`	2021-06-09 15:38:09 +03:00
Aliaksandr Valialkin	520d62ade2	docs/vmagent.md: mention that vmagent supports scrape targets sharding	2021-06-09 12:30:54 +03:00
Aliaksandr Valialkin	f3749dedba	docs: document rules replay feature for vmalert Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/836 This is a follow-up for `2a259ef5e7`	2021-06-09 12:30:54 +03:00
Roman Khavronenko	5aa7846900	vmalert: support rules backfilling (aka `replay`) (#1358 ) * vmalert: support rules backfilling (aka `replay`) vmalert can `replay` configured rules in the past and backfill results via remote write protocol. It supports MetricsQL/PromQL storage as data source, and can backfill data to remote write compatible storage. Supports recording and alerting rules `replay`. See more details in README. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/836 * vmalert: review fixes * vmalert: readme fixes	2021-06-09 12:30:54 +03:00
Aliaksandr Valialkin	2c6b917749	app/vminsert/netstorage: update storageNode.lastRerouteTime before the rerouting This is needed for reliable detection of storage nodes with recent rerouting	2021-06-08 12:06:32 +03:00
Aliaksandr Valialkin	0d067eb112	app/vminsert/netstorage: tune re-routing algorithm Do not re-route data to unavailable storage node. Send it to the remaining storage nodes instead even if they cannot keep up with the load. This should spread the load more evenly among available storage nodes. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/791 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1054 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1165	2021-06-05 16:23:44 +03:00
Aliaksandr Valialkin	269e35d676	app/{vmagent,vminsert}: follow-up after `2fe045e2a4` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1343	2021-06-04 20:33:22 +03:00
jelmd	d8b46908db	new feature: debug relabeling (#1344 ) * new feature: relabel logging Use scrape_configs[x].relabel_debug = true to log metric names inkl. labels before and after relabeling. After relabeling related metrics get dropped, i.e. not submitted to servers. * vminsert wants relabel logging, too.	2021-06-04 20:33:21 +03:00
Aliaksandr Valialkin	1c09e71f5b	app/vminsert: add `-disableRerouting` command-line flag for disabling re-routing if some vmstorage nodes have lower performance than the others Refactor the rerouting mechanism and make it more resilient to cases when some of vmstorage nodes are temporarily unavailable. Reduce the probability of rerouting storm. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/791 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1054 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1165	2021-06-04 04:33:52 +03:00
Aliaksandr Valialkin	8cdecfc52c	app/vmauth: allow balancing the load among multiple backend nodes by specifying multiple urls in `url_prefix` config	2021-05-29 01:04:22 +03:00
Aliaksandr Valialkin	97de72054e	docs: document `f0c21b6300`	2021-05-27 15:04:13 +03:00
Roman Khavronenko	e183a5c532	vmalert: automatically reload configuration on file change (#1326 ) New flag `-rule.configCheckInterval` defines how often `vmalert` will re-read config file. If it detects any changes, the config will be reloaded. This behaviour is turned off by default. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/512	2021-05-26 12:24:27 +03:00
Aliaksandr Valialkin	a0b001bfec	app/vmselect/netstorage: remove duplicate limiter on concurrent queries It duplicates the `-search.maxConcurrentRequests` limiter.	2021-05-24 19:13:04 +03:00
Aliaksandr Valialkin	890e1bd826	app/vmagent/remotewrite: use WARN level instead of ERROR level for `couldnt send a block with size ... bytes to ...` log message This is really warning, since vmagent re-tries sending the data block until success.	2021-05-24 15:43:32 +03:00
Roman Khavronenko	beee24ecee	vmalert: support `extra_filter_labels` setting per-group (#1319 ) The new setting `extra_filter_labels` may be assigned to group. If it is, then all rules within a group will automatically filter for configured labels. The feature is well-described here https://docs.victoriametrics.com#prometheus-querying-api-enhancements New setting is compatible only with VM datasource.	2021-05-23 14:15:49 +03:00
Aliaksandr Valialkin	71ff7ee18d	lib/promauth: follow-up after `5b8176c68e`	2021-05-22 18:02:03 +03:00
Nikolay	2780d6dbcd	basic OAuth2 support for remoteWrite and scrape targets (#1316 ) * adds OAuth2 support for remoteWrite and scrapping * adds tests changes init	2021-05-22 18:02:01 +03:00
Nikolay	23a6c9c016	changes vmalert query function (#1307 ) * changes vmalert query function for prometheus rules compatibility its better to use labels as map. it simplifies template evaluation and allow to ignore can't evaluate field error because map will return default value. fixes https://github.com/VictoriaMetrics/operator/issues/243	2021-05-21 16:38:20 +03:00
Aliaksandr Valialkin	d77db9d813	all: do not skip SIGHUP signal during service initialization This can lead to stale or incomplete configs like in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240	2021-05-21 16:38:20 +03:00
Aliaksandr Valialkin	6139f6ed6d	app/vmauth: add ability to protect `/-/reload` endpoint with authKey	2021-05-20 18:48:34 +03:00
Aliaksandr Valialkin	69e365cd48	Makefile: update golangci-lint from v1.29.0 to v1.40.1	2021-05-20 18:30:24 +03:00
Aliaksandr Valialkin	da0b32c31a	app/vmagent/remotewrite: expose metrics with the current number of active series per day and per hour These numbers are exposed via the following metrics: - vmagent_hourly_series_limit_current_series - vmagent_daily_series_limit_current_series Expose also the limits via the following metrics: - vmagent_hourly_series_limit_max_series - vmagent_daily_series_limit_max_series	2021-05-20 15:31:57 +03:00
Aliaksandr Valialkin	165a9f9200	app/vmstorage: add ability to limit series cardinality via `-storage.maxHourlySeries` and `-storage.maxDailySeries` command-line flags	2021-05-20 15:31:57 +03:00
Aliaksandr Valialkin	7aad5c3f76	app/vmagent: add ability to limit series cardinality on a per-hour and per-day basis	2021-05-20 15:31:57 +03:00
Roman Khavronenko	12d0c6b6e0	vmctl: explicitly set `::tag` type for labels selector in `influx` mode (#1310 ) The `::tag` type is needed in cases when field and tag names are equal, which results into unexpected results in InfluxQL. Setting the type explicitly helps InfluxDB to understand which exact column we apply filter to. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1299	2021-05-20 12:07:15 +03:00
Aliaksandr Valialkin	180829b8c2	app/vmselect/promql: add `timezone_offset(tz)` function Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1306	2021-05-20 11:54:06 +03:00
Aliaksandr Valialkin	dcac849c1f	app/vmagent/remotewrite: sort labels before sending the series to per-remoteWrite.url queues	2021-05-20 11:54:06 +03:00
Neo He	c5ab00ebee	app/{vmbackup,vmrestore},docs/vmrestore.md: typo fix: vbackup -> vmbackup (#1305 )	2021-05-18 16:38:15 +03:00
Aliaksandr Valialkin	74ef40034c	lib/httpserver: typo fix in `-http.shutdownDelay` command-line flag description: servier -> server	2021-05-18 16:25:27 +03:00
Aliaksandr Valialkin	1668280e67	docs/vmalert.md: document multitenant support https://github.com/VictoriaMetrics/VictoriaMetrics/issues/740	2021-05-18 16:25:21 +03:00
Aliaksandr Valialkin	7fe362deb1	app/vmauth: reload `-auth.config` on the request to `/-/reload` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1194	2021-05-18 02:24:37 +03:00
Aliaksandr Valialkin	25ca108642	docs/vmbackup.md: typo fix: snaphosts -> snapshots Thanks to @jelmd - see `1ab27582a3 (r50884395)`	2021-05-18 01:14:01 +03:00
Aliaksandr Valialkin	6ea191d196	docs: dealay -> delay	2021-05-18 01:07:32 +03:00
Roman Khavronenko	3428df6f15	vmalert: use stringified label keys for duplicates map in recroding rules (#1301 ) duplicates map helps to determine wheter extra labels has overriden labels which make time series unique. It was using a sorted hashed labels sequence as a key. But hashing algorithm could have collisions, so it is more convenient to not use hashing at all. Log message for recording rules duplicates was improved as well. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1293	2021-05-17 01:51:48 +03:00
Aliaksandr Valialkin	a6cb4f10a7	app/{vmalert,vmauth}: explicitly set MaxIdleConnsPerHost in net/http.Client.Transport By default MaxIdleConnsPerHost is set to 2. This limits the possibility to re-use http keep-alive connections. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1300	2021-05-14 18:13:34 +03:00
Aliaksandr Valialkin	23afbd5094	app/vmagent/remotewrite: clarify the comment explaining why vmagent drops blocks if remote storage returns 400 or 409 status code	2021-05-13 16:17:09 +03:00
Aliaksandr Valialkin	2839055513	lib/storage: substitute GetTSDBStatusForDate with GetTSDBStatusWithFiltersForDate with nil tfss	2021-05-13 09:01:05 +03:00
Nikolay	be87be34a4	Adds tsdb match filters (#1282 ) * init work on filters * init propose for status filters * fixes tsdb status adds test * fix bug * removes checks from test	2021-05-12 17:16:58 +03:00
Aliaksandr Valialkin	56b08390f6	app/vmselect/promql: allow to use 2x more memory for query processing in cluster mode compared to single-node mode `vmselect` has no `vmstorage`-related caches. So it can use more memory for query processing compared to single-node VictoriaMetrics.	2021-05-12 14:43:49 +03:00
Aliaksandr Valialkin	cca9670573	docs/CHANGELOG.md: document `-datasource.roundDigits` added at `5c448126dc`	2021-05-10 11:18:58 +03:00
Roman Khavronenko	a7f00101f5	vmalert: add support for `round_digits` param in datasource package (#1278 ) Starting from v1.56.0 VM supports `round_digits` which allows to limit the number of digits after the decimal point in response value. The feature can be used to reduce entropy of produced by recording rules values and significantly improve the compression. See more details in link below. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/525	2021-05-10 11:18:56 +03:00
Roman Khavronenko	35237fe1f5	vmalert: fix error when rule didn't start if restore failed (#1279 ) Previously, `startGroup` could exit on restore errors despite the `remoteRead.ignoreRestoreErrors` flag value. Now vmalert checks the flag value before deciding whether to return error or just log it.	2021-05-10 11:10:32 +03:00
Aliaksandr Valialkin	2dddd68feb	docs/vmagent.md: add `stream parsing mode` chapter	2021-05-08 23:14:47 +03:00
Aliaksandr Valialkin	9c505d27dd	lib/ingestserver: properly close incoming connections during graceful shutdown	2021-05-08 19:53:45 +03:00
Aliaksandr Valialkin	4a5f45c77e	app/vminsert: add support for data ingestion via other vminsert nodes	2021-05-08 19:53:45 +03:00
Aliaksandr Valialkin	07bc021f58	app/vmalert: add missing comment for ErrStateRestore	2021-05-08 19:53:45 +03:00
Aliaksandr Valialkin	e8478e1e97	app/vmbackup: make sure that `-snapshotName` isnt set if `-snapshot.createURL` is set	2021-05-07 08:44:44 +03:00
Roman Khavronenko	bb7e113dd4	vmalert: add flag to control behaviour on startup for state restore errors (#1265 ) Alerting rules now can return specific error type ErrStateRestore to indicate whether restore state procedure failed. Such errors were returned and logged before as well. But now user can specify whether to just log these errors (remoteRead.ignoreRestoreErrors=true) or to stop the process (remoteRead.ignoreRestoreErrors=false). The latter is important when VM isn't ready yet to serve queries from vmalert and it needs to wait. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1252	2021-05-05 12:24:32 +03:00
Aliaksandr Valialkin	0a2e746175	docs/vmalert.md: update docs after `afca7b430c`	2021-04-30 11:49:40 +03:00
Roman Khavronenko	7394967841	vmalert: fix the typo in ApplyParams func (#1259 )	2021-04-30 11:47:11 +03:00
Roman Khavronenko	6fbedd62b8	vmalert: use rule's `evaluationInterval` as `step` param by default (#1258 ) User still can override param by specifying `datasource.queryStep` flag.	2021-04-30 10:03:50 +03:00
Aliaksandr Valialkin	daf2778025	docs/CHANGELOG.md: document the change from `f3a048288e` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1232	2021-04-30 09:56:47 +03:00
Roman Khavronenko	b55677e93d	Vmalert: adjust `time` param for datasource queries according to `evaluationInterval` (#1257 ) * Simplify arguments list for fn `queryDataSource` to improve readbility * vmalert: adjust `time` param according to rule evaluation interval With this change, vmalert will start to use rule's evaluation interval for truncating the `time` param. This is mostly needed to produce consistent time series with timestamps unaffected by vmalert start time. Now, timestamp becomes predictable. Additionally, adjustment is similar to what Grafana does for plotting range graphs. Hence, recording rule series and recording rule expression plotted in grafana suppose to become similar in most of cases.	2021-04-30 09:56:46 +03:00
Aliaksandr Valialkin	8be1cb297b	app/vmagent: list user-visible endpoints at `http://vmagent:8429/` While at it, use common WriteAPIHelp function for the listing in vmagent, vmalert and victoria-metrics	2021-04-30 09:38:23 +03:00
Nikolay	2eb8ef7b2b	changes vmalert Querier with per rule querier (#1249 ) * changes vmalert Querier with per rule querier it allows to changes some parametrs based on rule setting for instance - alert type, tenant for cluster version or event endpoint url.	2021-04-29 11:31:07 +03:00
Roman Khavronenko	0ceb4f7565	vmalert: keep the returned timestamp when persisting recording rule (#1245 ) Previously, vmalert used `lastExecTime` timestamp when writing recording rules to the remote storage. This may be incorrect, if vmalert uses `datasource.lookback` flag, which means rule's expression will be executed at some moment in the past. To avoid such situations, vmalert now will use returned timestamp instead of `lastExecTime`.	2021-04-27 00:16:45 +03:00
Aliaksandr Valialkin	e309b5a83b	app/vmagent/remotewrite: increase the maximum possible number of inmemory blocks for systems with high amounts of RAM This should reduce the probability of using much slower file-based persistent queue when vmagent processes metrics at high rate (millions of metrics per second). Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1235	2021-04-23 22:05:00 +03:00
Aliaksandr Valialkin	f92db26a93	app/vmagent/remotewrite: count maxLabelsPerBlock as 10x of maxRowsPerBlock This should increase block sizes and subsequently increase the maximum possible bandwidth per each connection to remote storage. This, in turn, should reduce the probability of storing the data in local buffers. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1235	2021-04-23 22:05:00 +03:00
Aliaksandr Valialkin	aaee80d158	app/vmbackup: typo fix: snaphsot -> snapshot Follow-up for `9de0fa3649`	2021-04-22 11:18:13 +03:00
Aliaksandr Valialkin	e7c4fde756	app/vmauth: parse `url_prefix` only once during config load	2021-04-21 10:57:17 +03:00
Aliaksandr Valialkin	6dc5d3b357	all: rename https://victoriametrics.github.io to https://docs.victoriametrics.com	2021-04-20 20:20:01 +03:00
Aliaksandr Valialkin	64f1ddefe5	all: consistency renaming `Victoria Metrics` -> `VictoriaMetrics` VMInsert -> vminsert VMSelect -> vmselect VMStorage -> vmstorage	2021-04-20 11:45:02 +03:00
Aliaksandr Valialkin	8d869d112b	app/vmauth: follow-up for `6a81a89b3d`	2021-04-20 10:59:22 +03:00
Nikolay	7d249d787d	adds query params support for vmauth urlPrefix (#1226 ) * adds query params support for vmauth urlPrefix * Update app/vmauth/example_config.yml * Update app/vmauth/example_config.yml Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-04-20 10:59:20 +03:00
Aliaksandr Valialkin	bdcd978d56	app/vmctl: update README.md according to `bfecd0fd55`	2021-04-16 12:10:00 +03:00
Aliaksandr Valialkin	c872ba45b9	docs: update `-help` output after the commit `77be3e3a82`	2021-04-12 12:35:39 +03:00
Artem Navoiev	c3dcfdef8c	improve docs for cli flags (#1202 ) * improve docs for cli flags * improve docs for cli flags.2	2021-04-12 12:28:36 +03:00
Aliaksandr Valialkin	2190fd2148	docs: `make docs-sync`	2021-04-10 19:55:05 +03:00
Aliaksandr Valialkin	0f7ece84f3	app/vmstorage/transport: reduce memory allocations on data ingestion path	2021-04-10 17:36:00 +03:00
Roman Khavronenko	46e6fdb131	Docs update (#1199 ) * docs: drop table of contents for `vmctl` We already have it autogenerated on .github.io, so no need to keep it. * docs: mention OpenTSDB migration feature for vmctl * docs: sync docs for `vmalert`	2021-04-10 15:47:52 +03:00
John Seekins	9e7bf595a0	Improve documentation on OpenTSDB migration tool and fix a bug with hard offsets (#1198 ) * add more documentation on OpenTSDB migration explaining what chunking means * more clarification of OpenTSDB aggregations * break out what a retention string becomes * add more docs around retention strings * add example of running program and fix mistake in how hard offsets are handled * fix formatting	2021-04-10 15:28:56 +03:00
John Seekins	97fafce028	OpenTSDB migration to VictoriaMetrics (#1089 )	2021-04-10 15:28:54 +03:00
Roman Khavronenko	712725b4a5	vmalert: document template functions and mention them in README (#1197 )	2021-04-08 18:20:57 +03:00
Aliaksandr Valialkin	e48fd5776b	app/vmselect/promql: fix tests after `d3fa0ccabd`	2021-04-08 00:17:40 +03:00
Aliaksandr Valialkin	f90bf265f4	app/vmselect/promql: properly detect aggregate `topk` and `bottomk` aggregate functions in order to disable duplicate sorting Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1189	2021-04-08 00:10:33 +03:00
Aliaksandr Valialkin	9ce3b7e1dd	app/vmselect: return `data:null` instead of `data:[]` from `/api/v1/query_exemplars`, since Grafana throws an error otherwise Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1186	2021-04-07 23:34:01 +03:00
Aliaksandr Valialkin	3d5f1f779f	app/vmselect: do not sort series returned from `topk` and `bottomk` functions, since these series are already sorted in user-expected order Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1189	2021-04-07 14:16:02 +03:00
Roman Khavronenko	ff3711eea2	docs: update docs ordering and formatting (#1192 ) The major change is adding `sort` directive to docs. For those docs which are copied from internal packages `sort` is added via makefile command. For the rest it is added manually since they're updated manually as well. The rest of changes is connected with markdown formatting. For example, changing headers in some files (`##` => `#`) makes navigation on .github.io to look better. This especially useful for `changelog` docs. Table of contents for `vmctl` is dropped, since we already have it autogenerated on .github.io. No link changes expected. The corresponding PR to `cluster` branch will be made in follow-up PR.	2021-04-07 13:43:01 +03:00
Aliaksandr Valialkin	7fce4e9fb4	app/vmselect: return dumb response on `/api/v1/query_exemplars` request Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1186	2021-04-05 23:28:21 +03:00
Aliaksandr Valialkin	2585058a5f	Makefile: prepare `arm64` and `amd64` release archives for cluster version on `make release` command	2021-04-05 23:01:45 +03:00
Aliaksandr Valialkin	92d4c80639	docs/vmagent.md: mention that vmagent supports scraping via socks5 proxy	2021-04-04 01:45:46 +03:00
Aliaksandr Valialkin	ab9e1eb41f	lib/promscrape: support for simple HTTP proxies without `CONNECT` method support such as https://github.com/prometheus-community/PushProx See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1179	2021-04-04 00:40:58 +03:00
Aliaksandr Valialkin	4aa59cae7c	app/vmselect/promql: do not delete `dst_label` if `src_label` is empty in `label_copy(q, src_label, dst_label)` and `label_move(q, src_label, dst_label)`	2021-04-03 22:06:50 +03:00
Aliaksandr Valialkin	dfec690548	docs/vmgateway.md: update docs	2021-04-03 00:29:57 +03:00
Aliaksandr Valialkin	b9469de410	app/vmselect/promql: add ability to set label value additionally to label name for the remaining sum of time series returned from `topk_` and `bottomk_` functions in the form: `topk_min(N, m, "label=value")`	2021-04-02 23:56:30 +03:00
Aliaksandr Valialkin	262da1c2e5	docs/{vmauth,vmgateway}.md: small fixes	2021-04-02 23:15:40 +03:00
Aliaksandr Valialkin	5edec0e57e	app/vmgateway: publish docs	2021-04-02 23:10:10 +03:00
Aliaksandr Valialkin	4028d692f5	app: do not process non-GET requests on at `/` handler	2021-04-02 22:56:38 +03:00
Aliaksandr Valialkin	89771d082d	app/vmauth: add support for authorization via `Authorization: Bearer <token>`	2021-04-02 22:15:07 +03:00
Aliaksandr Valialkin	6c6174271a	docs/vmagent.md: mention about proxy_authorization section	2021-04-02 21:24:52 +03:00
Aliaksandr Valialkin	87700f1259	lib/promscrape: add support for `authorization` config in `-promscrape.config` as Prometheus 2.26 does See https://github.com/prometheus/prometheus/pull/8512	2021-04-02 21:20:37 +03:00
Aliaksandr Valialkin	31d3d02d25	app/vmselect/promql: remove the limit on the number of time series that can be sorted, since it may confuse users Always sort time series returned from `/api/v1/query` and `/api/v1/query_range` unless `sort_*` function is used at top level of the query.	2021-04-02 16:50:02 +03:00
Aliaksandr Valialkin	512addc608	app/{vminsert,vmagent}: add `-sortLabels` command-line option for sorting time series labels before ingesting them in the storage This option can be useful when samples for the same time series are ingested with distinct order of labels. For example, metric{k1="v1",k2="v2"} and metric{k2="v2",k1="v1"}.	2021-03-31 23:27:21 +03:00
Aliaksandr Valialkin	ae1c653d55	lib/storage: reduce memory usage when ingesting samples for the same time series with distinct order of labels	2021-03-31 21:22:40 +03:00
Aliaksandr Valialkin	392ba94d1d	app/vmagent/remotewrite: reduce memory usage when `-remoteWrite.queues` is set to a big value See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1167	2021-03-31 16:17:23 +03:00
Aliaksandr Valialkin	b873b965af	app/vmagent/remotewrite: reduce memory usage when samples with big number of labels are sent to remote storage	2021-03-31 00:45:42 +03:00
Aliaksandr Valialkin	d8c769e6af	app/vmselect: add `-search.maxStatusRequestDuration` command-line flag for limiting the duration of requests to `/api/v1/status/*` and `/api/v1/series/count`	2021-03-30 21:39:32 +03:00
Aliaksandr Valialkin	0e29d48628	app/vmselect: remove mentioning for `-search.storageTimeout` from command-line flag descriptions This is a follow-up for `ecfd6fe78d`	2021-03-30 15:07:40 +03:00
Aliaksandr Valialkin	ecfd6fe78d	app/vmselect: remove `-search.storageTimeout` command-line flag, since it has the same meaning as `-search.maxQueryDuration` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711	2021-03-30 14:54:36 +03:00
Aliaksandr Valialkin	7bafaad46d	app/vmselect: prevent from possible incomplete query results after timed out query Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711	2021-03-30 13:34:57 +03:00
Aliaksandr Valialkin	9b4e608199	app/vminsert: return back `type` label to per-tenant metric `vm_tenant_inserted_rows_total The `type` label has been removed in the commit `022fe4efd0` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/932	2021-03-30 00:06:59 +03:00
Aliaksandr Valialkin	0a8f0a4e2f	all: increase minimum supported Go version for building VictoriaMetrics components from v1.14 to v1.15 This is needed after the commit `c0ac740f93`, which uses URL.Redacted() method, which has been added in v1.15. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1147	2021-03-29 23:06:36 +03:00
Aliaksandr Valialkin	3f0534c7f8	app/vminsert/opentsdbhttp: remove rowsInsertedTenant metric, since it is substituted by tenantmetrics.RowsInsertedByTenant in the commit `022fe4efd0`	2021-03-29 12:16:13 +03:00
Nikolay	022fe4efd0	makes tenant metrics names consistent with enterprise version (#1152 )	2021-03-29 11:59:04 +03:00
Aliaksandr Valialkin	8a3939e93e	docs: document that vmagent drops data blocks when remote storage replies with 400 and 409 http status codes This is a follow up for `1b7dc1e5a5`. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1149	2021-03-26 14:43:41 +02:00
Aliaksandr Valialkin	3fedf680f8	app/vmagent/remotewrite: remove superflouos code after `1b7dc1e5a5`	2021-03-26 14:01:37 +02:00
Nikolay	15ff1f3a94	Adds blocks drop (#1151 ) * adds blocks drop at 400 BadRequest status code recieved from remote storage, not expected that remote storage will be able to handle it on retry * removes error logging for dropped blocks, its expected error	2021-03-26 14:01:37 +02:00
Aliaksandr Valialkin	698ccca6ad	app/vmselect/prometheus: explain why partial responses are disallowed during data export See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1148	2021-03-25 18:39:07 +02:00
Aliaksandr Valialkin	b91e18af0e	docs/vmagent.md: add an example for `-remoteWrite.label`	2021-03-25 17:59:30 +02:00
Aliaksandr Valialkin	82047be90b	docs: add a link to the repository from build instruction for all the VictoriaMetrics components	2021-03-25 17:16:55 +02:00
Aliaksandr Valialkin	6de36b4e21	docs/vmagent.md: cosmetic fixes	2021-03-25 17:11:10 +02:00
Aliaksandr Valialkin	1b13d02728	docs/vmagent.md: cosmetic fixes	2021-03-25 16:54:37 +02:00
Aliaksandr Valialkin	155cd90fc8	docs/vmagent.md: typo fix: tupically -> typically	2021-03-25 16:48:27 +02:00
Aliaksandr Valialkin	450e23533d	docs/vmalert.md: remove misleading `-evaluationInterval=3s` from example config args 3s evaluation interval is too small for practical setups. It can result in increased load on datasource. So it is better to remove it from example config args, which are usually copy-pasted by novice users.	2021-03-25 15:31:10 +02:00
Aliaksandr Valialkin	b473c21915	app/vmselect/promql: do not merge time series during requests to `/api/v1/query` See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1141	2021-03-25 13:56:23 +02:00
Aliaksandr Valialkin	6b1f807418	app/vmagent: add `-promscrape.consul.waitTime` command-line flag for configuring Consul service discovery wait time See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1144	2021-03-23 19:34:12 +02:00
Aliaksandr Valialkin	2041008d64	app/vmagent: mention -remoteWrite.maxDiskUsagePerURL in the descriptio of `-remoteWrite.tmpDataPath` flag	2021-03-23 16:38:41 +02:00
Aliaksandr Valialkin	d1e773266f	app/vmselect/graphite: accept and enforce `extra_label` in all the Graphite APIs	2021-03-23 15:30:15 +02:00
Aliaksandr Valialkin	27bb614016	app/vmselect: move getEnforcedTagFiltersFromRequest to searchtuils, since it will be used in Graphite functions soon	2021-03-23 14:57:57 +02:00
Aliaksandr Valialkin	7f9863254d	app/vmselect: improve description for `-search.maxPointsPerTimeseries` command-line flag	2021-03-22 16:44:36 +02:00
Aliaksandr Valialkin	6ebf18ab97	app/{vminsert,vmagent}: use Influx field as metric name if measurement is empty and -influxSkipSingleField command-line is set See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1139	2021-03-22 13:55:11 +02:00
Aliaksandr Valialkin	92531a38c4	app/vmselect/promql: increment key prefix for faster reset for rollup result cache	2021-03-22 11:59:39 +02:00
Aliaksandr Valialkin	799461d8bf	app/vmselect/graphite: follow-up after `529d7be26b`	2021-03-18 16:33:00 +02:00
Nikolay	8807410a00	changes metricsFind api (#1137 ) it should be able mitigate crash if label value contains *,[ or { symbols	2021-03-18 16:32:59 +02:00
Aliaksandr Valialkin	667d129e1e	app/vmselect/promql: typo fix after `9666834045`	2021-03-17 15:11:54 +02:00
Aliaksandr Valialkin	69201806f8	app/vmselect/promql: merge adjancent buckets with the smallest summary number of hits in `buckets_limit()` function This should improve accuracy for the returned buckets	2021-03-17 14:31:29 +02:00
Aliaksandr Valialkin	8ef1184adf	app/vmstorage: add `vm_index_search_duration_seconds` histogram for monitoring the performance of index search	2021-03-17 01:13:15 +02:00
Aliaksandr Valialkin	5e77a939c2	all: make `go vet` happy	2021-03-17 00:48:44 +02:00
Aliaksandr Valialkin	b997f4a418	all: make golangci-lint happy after the commit `6378205415`	2021-03-17 00:24:31 +02:00
Aliaksandr Valialkin	8005ba26b9	lib/netutil: enable IPv6 UDP listening if `-enableTCP6` command-line flag is passed to VictoriaMetrics This is a follow-up for `18cfc4be7b` See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1131	2021-03-17 00:19:30 +02:00
Aliaksandr Valialkin	05e301cfa0	app/vmselect/netstorage: reduce mutex contention when unpacking data on a system with high number of CPU cores	2021-03-16 21:52:05 +02:00
Aliaksandr Valialkin	3caac5edd4	Makefile: prepare vmutils-windows-.zip archive on `make release-vmutils` command The archive contains the following executables for Windows: vmagent * vmalert * vmauth * vmctl Other components - vmbackup, vmrestore, victoria-metrics - aren't supported for Windows yet	2021-03-16 20:54:10 +02:00
Aliaksandr Valialkin	f54ece438d	app/vmselect/promql: do not crash if `histogram_over_time()` function name contains uppercase letters such as `Histogram_over_time()`	2021-03-16 12:23:55 +02:00
Aliaksandr Valialkin	d074326970	app/vmstorage: add `-logNewSeries` command-line flag for determining the source of series churn rate	2021-03-15 22:40:28 +02:00
Aliaksandr Valialkin	e2717d84c0	all: various fixes in command-line flag descriptions	2021-03-15 22:03:49 +02:00
Aliaksandr Valialkin	776b8b32ca	app/{vminsert,vmagent}: a follow-up for `b1aa8c3d8f` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1124	2021-03-15 22:03:49 +02:00
Nikolay	a843dc0219	adds fake response for telegraph queries (#1130 ) https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1124	2021-03-15 22:03:49 +02:00
Aliaksandr Valialkin	30ab0eec27	app/vmselect/promql: reduce overhead on scrape interval estimation It should be enough to use the first 20 datapoints instead of 100 datapoints for scrape interval estimation.	2021-03-15 20:32:35 +02:00
Aliaksandr Valialkin	8844b38745	app/vmselect/promql: fix tests after `2dae0a2c47`	2021-03-15 20:20:27 +02:00
Aliaksandr Valialkin	6b9bba7448	app/vmselect: add `round_digits` query arg to `/api/v1/query` and `/api/v1/query_range` handlers for limiting the number of decimal digits after the point	2021-03-15 12:38:59 +02:00
Aliaksandr Valialkin	afa5b58c2d	app/vminsert/opentsdbhttp: typo fix in `vm_rows_inserted_total{type="opentsdbhttp"}` metric: opentsdb-http -> opentsdbhttp	2021-03-12 15:59:31 +02:00
Aliaksandr Valialkin	b6dcb37fca	app/vmselect/prometheus: do not include datapoints with timestamps matching `t-d` when returning results from `/api/v1/query?query=m[d]&time=t` as Prometheus does	2021-03-12 12:18:37 +02:00
Aliaksandr Valialkin	60e0280a94	lib/promscrape: add ability to configure proxy options via `proxy_tls_config`, `proxy_basic_auth`, `proxy_bearer_token` and `proxy_bearer_token_file` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1116	2021-03-12 03:36:11 +02:00
Ihor Borodin	933de6b9b1	Fixing examples of external.alert.source in documentation (#1120 ) * Fixing examples of external.alert.source in documentation	2021-03-10 12:08:22 +02:00
Aliaksandr Valialkin	942890b1bb	app/vmselect/promql: follow up for `433fff0006`	2021-03-09 12:54:23 +02:00
Nikolay	28e450cd7c	duplicate timeseries fix for prometheus_buckets function (#1119 ) * try fix for prometheus_buckets * merge possible end of the bucket collision	2021-03-09 12:54:21 +02:00
Aliaksandr Valialkin	502fab797a	lib/promscrape: add `scrape_offset` option to `scrape_config` This option can be used for specifying the particular offset per each scrape interval for target scraping	2021-03-08 11:59:32 +02:00
Aliaksandr Valialkin	26cb6f8861	app/vmauth: allow using regexps in `url_map` paths See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1112	2021-03-05 18:53:40 +02:00
Aliaksandr Valialkin	5277507932	app/vmselect/promql: add `histogram_avg()`, `histogram_stddev()` and `histogram_stdvar()` functions to MetricsQL	2021-03-04 14:12:52 +02:00
Aliaksandr Valialkin	133fb9fc00	lib/promscrape: add `-promscrape.cluster.replicationFactor` command-line flag for replicating scrape targets among `vmagent` instances in the cluster	2021-03-04 10:21:27 +02:00
Aliaksandr Valialkin	d109e17f46	all: bump minimum supported Go version from 1.13 to 1.14	2021-03-03 15:58:17 +02:00
Aliaksandr Valialkin	3c11ce9356	docs/vmagent.md: remove outdated suggestion for determining labels that lead to duplicate targets The original labels for duplicate targets is already printed in the error message starting from `71ea4935de`	2021-03-03 12:28:50 +02:00
Roman Khavronenko	219291e084	Vmagent docs upd (#1104 ) * vmagent: port changes from https://github.com/VictoriaMetrics/VictoriaMetrics.github.io/pull/1 Thanks to @dereksfoster99 for this patch! * vmagent: reword to make the meaning clear	2021-03-03 10:53:41 +02:00
Aliaksandr Valialkin	2ecee0515a	app/vmalert/README.md: sync with docs/vmalert.md	2021-03-03 10:42:54 +02:00
Aliaksandr Valialkin	c67a07b469	lib/handshake: log read/write operation duration on connection errors This improve debuggability of network errors	2021-03-02 21:20:20 +02:00
Aliaksandr Valialkin	d9e8af0e8f	docs: actualize `-help` output	2021-03-01 17:02:05 +02:00
Aliaksandr Valialkin	323af49234	app/vmagent/remotewrite: clarify `-remoteWrite.flushInterval` flag description	2021-03-01 11:51:08 +02:00
Aliaksandr Valialkin	b8dd5e8292	app/vmagent/remotewrite: fix rate limiting logic for `-remoteWrite.url`	2021-03-01 00:59:57 +02:00
Aliaksandr Valialkin	d0be1f6f49	app/vmagent: remove data race when applying rate limits to `-remoteWrite.url` with multiple queues	2021-03-01 00:28:20 +02:00
Aliaksandr Valialkin	17eb29206d	docs/vmbackup.md: clarify docs on `vmagent` clustering	2021-02-28 21:59:55 +02:00
Aliaksandr Valialkin	3b1c2f03c3	app/vmselect/querystats: show the number of matching queries in the top by average duration and in the top by summary duration This should help debugging slow queries.	2021-02-28 19:42:29 +02:00
Aliaksandr Valialkin	9a2bf65134	lib/promscrape: add ability to spread scrape targets among multiple vmagent instances Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1084	2021-02-28 18:40:42 +02:00
Nikolay	b52d1e4f19	adds query params for vmalert (#1094 ) remoteWrite.url now accepts query params at provided url https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1087	2021-02-28 14:12:34 +02:00
Aliaksandr Valialkin	3721c5353a	app/vmbackupmanager: add docs; the vmbackupmanager is available as a part of enterprise subscription	2021-02-27 02:18:33 +02:00
Aliaksandr Valialkin	7f1302688f	lib/fs: follow-up after `f3a03c4164`	2021-02-27 01:09:37 +02:00
Nikolay	d88fa5ebe4	Adds windows build (#1040 ) * fixes windows compilation, adds signal impl for windows, adds free space usage for windows, https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1036 NOTE victoria metrics database still CANNOT work under windows system, only vmagent is supported. To completly port victoria metrics, you have to fix issues with separators, parsing and posix file removall * rollback separator * Adds windows setInformation api, it must behave like unix, need to test it. changes procutil * check for invlaid param * Fixes posix delete semantic * refactored a bit * fixes openbsd build * removed windows api call * Fixes code after windows add * Update lib/procutil/signal_windows.go Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-02-27 01:06:22 +02:00
Nikolay	673b10dd7f	adds enforced tag filters into cache key (#1095 )	2021-02-27 00:23:38 +02:00
Aliaksandr Valialkin	e38ca28d99	app/vmselect/promql: increase accuracy for `buckets_limit()` function for small limits by skipping the first and the last buckets during merge The first and the last buckets are usually `[0 ... leMin]` and `(leMax ... +Inf)`. If they are merged with adjancent buckets, then the resulting accuracy can suffer.	2021-02-26 22:56:04 +02:00
Aliaksandr Valialkin	0144b164c7	app/vmselect: add `sign(q)` and `clamp(q, min, max)` functions, which will be added in the upcoming Prometheus release See https://twitter.com/roidelapluie/status/1363428376162295811 The `last_over_time(m[d])` function already exists in MetricsQL.	2021-02-24 17:26:27 +02:00
Aliaksandr Valialkin	1b90a091cf	app/vmselect/netstorage: cleanup after `4805b80977`	2021-02-24 11:43:09 +02:00
weng zhao	4805b80977	vmselect: fix vmselect cache partial result when timeout (#1085 )	2021-02-24 11:39:56 +02:00
Aliaksandr Valialkin	d16effc29e	app/vmselect/promql: properly calculate `histogram_quantile() over zero buckets and only a single non-zero` le="+Inf"` bucket like Prometheus does	2021-02-24 00:41:30 +02:00
Aliaksandr Valialkin	5dbe88a1c6	app/vmselect/promql: add `increase_pure()` function to MetricsQL Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962	2021-02-22 19:15:31 +02:00
Aliaksandr Valialkin	197ecca426	lib/promrelabel: add more optimizations for relabeling for common cases	2021-02-22 16:36:54 +02:00
Aliaksandr Valialkin	72eef964d9	app/vmagent: properly perform graceful shutdown, which was broken in the commit `1d1ba889fe` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065	2021-02-19 00:34:17 +02:00
Aliaksandr Valialkin	41d3ff4f2b	app/vmagent: fix scrape config example for scrape_align_interval option	2021-02-18 23:57:45 +02:00
Aliaksandr Valialkin	502d0e2524	lib/promscrape: add scrape_align_interval config option into scrape config This option allows aligning scrapes to a particular intervals.	2021-02-18 23:53:04 +02:00
Aliaksandr Valialkin	bd1d906eee	app/vmselect/promql: reduce the probability of `duplicate time series` errors when querying Kubernetes metrics	2021-02-18 22:08:13 +02:00
Aliaksandr Valialkin	902a4f6486	app/victoria-metrics: remove accidentally added single-node tests	2021-02-18 18:33:03 +02:00
Aliaksandr Valialkin	901710b9e2	app/vmalert: add missing multiarch Dockerfile	2021-02-18 15:23:57 +02:00
Aliaksandr Valialkin	1dad7ecf54	app/victoria-metrics/testdata: add a test for {__graphite__="foo.*.bar"} selector	2021-02-17 21:53:24 +02:00
Aliaksandr Valialkin	9c81429299	app/vmagent/remotewrite: cleanup after `1d1ba889fe` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065	2021-02-17 21:43:48 +02:00
Nikolay	40973eda1c	adds pushback for fastqueue, (#1075 ) during shutdown currently sending block was lost, now its pushed back to fast queue and will be flushed on disk, it may lead to data duplication. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065	2021-02-17 21:43:46 +02:00
Aliaksandr Valialkin	83da939947	app/vmstorage: export vm_composite_filter_success_conversions_total and vm_composite_filter_missing_conversions_total metrics	2021-02-17 19:13:49 +02:00
Aliaksandr Valialkin	3062ff0fdb	app/vmselect: export per-tenant stats on the number of requests and the cumulative request duration The metrics are: - vm_vmselect_http_requests_total{accountID="...",projectID="..."} - the total number of select requests per each tenant - vm_vmselect_http_duration_ms_total{accountID="...",projectID="..."} - the total duration in milliseconds for per-tenant select requests Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/932	2021-02-16 23:30:29 +02:00
Aliaksandr Valialkin	667a7594b7	docs: rename vmbackuper to vmbackupmanager	2021-02-16 22:01:21 +02:00
Aliaksandr Valialkin	73c9da16b8	app/vmselect/netstorage: reuse timeseriesWork objects in order to reduce memory allocations	2021-02-16 16:09:00 +02:00
Aliaksandr Valialkin	a3a09a3c6e	app/vmselect/netstorage: use unsafe string as a key for a map when the map already contains the given key This should prevent from a memory allocation and a string copy.	2021-02-16 15:46:31 +02:00
Aliaksandr Valialkin	46e98ed490	vendor: update github.com/VictoriaMetrics/metrics from v1.13.1 to v1.14.0 The new version switches from log-linear histograms to log-based histograms, which provide up to 3.6 times better accuracy.	2021-02-15 15:11:15 +02:00
Aliaksandr Valialkin	c769f8321d	deployment/docker: embed tzdata into prod Go app instead of installing it into base docker image While this increases app size by 700Kb, this allows using -loggerTimezone in a scratch base image See https://github.com/golang/go/issues/38017	2021-02-12 04:56:27 +02:00
Aliaksandr Valialkin	ff7850aec0	deployment/docker: use `docker buildx` for creating multiarch builds See https://github.com/docker/buildx/	2021-02-12 04:35:35 +02:00
Aliaksandr Valialkin	ee4288987b	app/vmselect/prometheus: treat `match` query arg in the same way as `match[]` query arg	2021-02-11 15:01:55 +02:00
Aliaksandr Valialkin	9e88ff3075	app/vmauth: add ability to route requests from a single users to multiple targets depending on the requested path Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1064	2021-02-11 12:41:38 +02:00
Aliaksandr Valialkin	54d2f67924	app/vmselect/promql: properly make copies of EvalConfig	2021-02-11 12:41:37 +02:00
Aliaksandr Valialkin	5973fd4067	app/vmselect/promql: make a copy of EvalConfig when executing `q1` and `q2` in parallel for `q1 binary_op q2` This should prevent from data races if the underlying functions modify EvalConfig contents.	2021-02-10 23:06:58 +02:00
Aliaksandr Valialkin	985c3e301d	app/vmselect: parallelize `q1 <binary_op> q2` queries by running `q1` and `q2` in parallel This should reduce query execution times.	2021-02-10 23:00:56 +02:00
Aliaksandr Valialkin	08f21d8761	app/vmstorage: export vm_composite_index_min_timestamp metric	2021-02-10 17:14:00 +02:00
Aliaksandr Valialkin	148422bcba	lib/storage: disable composite index usage when querying old data	2021-02-10 14:57:58 +02:00
Aliaksandr Valialkin	fa0ef143b1	lib/storage: optimize search by label filters matching big number of time series	2021-02-10 00:46:17 +02:00
Aliaksandr Valialkin	e8ee9fa7fe	app/vmstorage: export missing `vm_cache_size_bytes` metrics for indexdb and data caches	2021-02-09 00:49:58 +02:00
Aliaksandr Valialkin	35eb04b7dd	docs/vmctl.md: fix title, so it is properly displayed in the header of https://victoriametrics.github.io/	2021-02-04 20:20:38 +02:00
Roman Khavronenko	2aa37b0450	vmalert: mention `-datasource.appendTypePrefix` in README (#1052 )	2021-02-03 23:48:45 +02:00
Dmitry Shevchuk	007fd6ce9c	Adds ability to query right vmselect endpoint based on the query type (#1050 ) * Adds ability to query right vmselect endpoint based on the query type Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>	2021-02-03 23:48:44 +02:00
Aliaksandr Valialkin	88ee836d0c	docs/vmalert.md: mention that `type` option can be set at group level additionally to rule level	2021-02-03 21:12:39 +02:00
Aliaksandr Valialkin	a3aa9bdc9f	app/vmagent: add `Advanced usage` section with the description for all the command-line flags	2021-02-03 21:03:52 +02:00
Aliaksandr Valialkin	8de3a329ff	app/vmselect: deprecate `-search.treatDotsAsIsInRegexps` in favor to `{__graphite__="foo.*.bar"}` syntax	2021-02-03 20:36:42 +02:00
Aliaksandr Valialkin	8249f13104	app/vmselect,lib/storage: properly parse Graphite selectors with inner wildcards Example: foo{bar{x,yz},a[b-c],*de}	2021-02-03 20:16:28 +02:00
Aliaksandr Valialkin	4b930b9ffe	app/vmselect: add ability to set Graphite-compatible filter via `{__graphite__="foo.*.bar"}` syntax	2021-02-03 01:17:19 +02:00
Aliaksandr Valialkin	03da0b728c	docs: increase heading sizes in vmagent, vmauth, vmbackup and vmrestore docs, so they match the heading sizes in VictoriaMetrics docs	2021-02-01 19:44:58 +02:00
Aliaksandr Valialkin	f4a4665857	docs/vmctl.md: update build instructions after the migration from github.com/VictoriaMetrics/vmctl to github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl	2021-02-01 19:44:57 +02:00
Aliaksandr Valialkin	7d23f3ff3a	app/vmselect: add ability to set an additional label filters via `extra_label` query arg	2021-02-01 17:42:35 +02:00
Aliaksandr Valialkin	6811445b64	docs: document ability to query Graphite datasource from vmalert	2021-02-01 15:28:31 +02:00
Nikolay	b8bc1c2e0f	Graphite vmalert wip (#112 ) * init implementation for graphite alerts * adds graphite support for vmalert * small fix * changes vmalert graphite api with type * updates tests * small fix * fixes graphite parse * Fixes graphite from time	2021-02-01 15:28:30 +02:00
Aliaksandr Valialkin	fdf9de98f8	app/vmagent: add -remoteWrite.roundDigits command-line option for limiting the number of digits after the point for stored values This commit also adds --vm-round-digits command-line option to vmctl tool.	2021-02-01 14:42:15 +02:00
Aliaksandr Valialkin	05474aaa29	app/vmctl: fix `make check-all` warnings	2021-02-01 01:31:45 +02:00
Aliaksandr Valialkin	7f4fb34182	app/vmctl: move vmctl code from github.com/VictoriaMetrics/vmctl It is better developing vmctl tool in VictoriaMetrics repository, so it could be released together with the rest of vmutils tools such as vmalert, vmagent, vmbackup, vmrestore and vmauth.	2021-02-01 01:18:39 +02:00
Aliaksandr Valialkin	5328a102e0	app/vmselect: unconditionally deny partial responses from `/api/v1/export` It is expected that `/api/v1/export` returns full data, so there is no sense in partial responses there.	2021-01-27 14:39:53 +02:00
Aliaksandr Valialkin	4b324da947	all: consistently use timers from timerpool	2021-01-27 00:40:39 +02:00
Aliaksandr Valialkin	29bf531f7d	app/vmagent: add -remoteWrite.rateLimit command-line flag for limiting data rate to remote storage See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1035	2021-01-27 00:40:39 +02:00
weng zhao	2a8a34ea05	vmalert: add option datasource.queryStep to allow user to address the inconsistency between grafana dashboards(query_range with step 15s usually) and ALERTS (#1027 ) Co-authored-by: zhao.weng <zhao.weng@shopee.com>	2021-01-26 16:38:20 +02:00
Aliaksandr Valialkin	44c74f1e79	app/vmselect/promql: improve documentation for `-search.maxPointsPertimeseries` command-line flag This should reduce incorrect usage and assumptions for this flag. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1020	2021-01-22 13:00:35 +02:00
Aliaksandr Valialkin	e55205220b	app/vmselect: add `-search.maxStepForPointsAdjustment` command-line flag, which can be used for disabling adjustment for points returned from `/api/v1/query_range` handler if they have timestamps closer than `-search.latencyOffset` to the current time	2021-01-19 22:57:50 +02:00
Aliaksandr Valialkin	5856611291	app/vmselect/graphite: extract getCanonicalPath() function from loop body inside getCanonicalPaths()	2021-01-18 17:31:27 +02:00
Aliaksandr Valialkin	5640e6cbca	docs/vmagent.md: follow-up for `184a659c5f`	2021-01-13 13:54:28 +02:00
Aliaksandr Valialkin	c5bdab5a4c	app/vmselect/promql: add ability to pass multiple labels to `sort_by_label` and `sort_by_label_desc` functions See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/992	2021-01-13 12:43:47 +02:00
Aliaksandr Valialkin	8cae98aa78	app/vmselect/promql: properly parse escaped multibyte utf8 code sequences in metric names and labels names See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/990	2021-01-13 10:59:32 +02:00
Nikolay	821492bc0b	adds extra_label to all import apis (#1007 ) * adds extra_label to all import apis, changes priority for extra_label - now it has priority over original labels * Update README.md Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com> * Update README.md Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com> * adds extra labels to vmagent import api changes order for adding labels, now its added after user values * adds tests for extra_label * import fix Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-01-13 01:07:24 +02:00
Aliaksandr Valialkin	df6e399f73	app/vmselect/promql: add `tfirst_over_time(m[d])` and `tlast_over_time(m[d])` MetricsQL functions for returning timestamps for the first and the last samples in `m` over `d`	2021-01-12 16:12:47 +02:00
Nikolay	9f0a4fd00e	Fixes error handling for promscrape.streamParse (#1009 ) properly return error if client cannot read data, properly suppress scraper errors	2021-01-12 13:35:09 +02:00
Roman Khavronenko	304512b668	vmalert-989: return non-empty result in template func `query` stub to pass validation (#1002 ) On templates validation stage vmalert does not acutally send queries, so for complex chained expression validation may fail. To avoid this, we add a blank sample in response so validation can pass successfully. Later, during the rule execution, stub will be replaced with real `query` function. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/989	2021-01-11 12:59:33 +02:00
Aliaksandr Valialkin	4ee53c3961	all: use `net.Dial` instead of `fasthttp.Dial`, because `fasthttp.Dial` limits the number of concurrent dials to 1000	2021-01-11 12:52:51 +02:00
Aliaksandr Valialkin	d5a2b120e9	app/vmstorage: disable final merge by default, since it may result in high disk IO and CPU usage without measurable benefits such as increased query performance and reduced disk space usage	2021-01-08 00:12:12 +02:00
Aliaksandr Valialkin	47872ada7e	app/vmselect/promql: do not ajdust `offset` value provided in the query Previously it could be modified in order to improve response cache hit ratio. This is unneeded, since cache hit ratio should remain good because the query time range should be already aligned to multiple of `step` values. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/976	2020-12-27 14:10:15 +02:00
Aliaksandr Valialkin	5bbf200de2	app/vmselect: add per-tenant `/api/v1/status/top_queries` handler	2020-12-27 12:53:50 +02:00
Aliaksandr Valialkin	0e739efc88	app/vmselect/promql: simplify defer call for querystats.RegisterQuery	2020-12-27 12:07:56 +02:00
Aliaksandr Valialkin	44932098b5	app/vmselect/querystats: reduce the default number of last queries to track from 100K to 20K This should reduce memory usage in constrained environments	2020-12-25 17:40:32 +02:00
Aliaksandr Valialkin	e6deb39064	app/vmselect: refactor `/api/v1/stats/top_queries` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907	2020-12-25 17:24:25 +02:00
Nikolay	76d092c091	Adds query stats handler (#945 ) * Adds query stat handler, for query and query_range api, victoriametrics tracks query execution time, stats are expored at /api/v1/status/queries endpoint with topN param https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907 * fixed query stats bugs * improves queryStats tracker * improves query stat * small fix * fix tests * added more tests * fixes 386 tests * naming fixes * adds drop for outdated records	2020-12-25 17:24:24 +02:00
Nikolay	14915071d6	adds escape for CRLF (#984 ) at external.alert.source - \n and \r symbols was url encoded, instead of direct usage. replace it from "\n" to `\n` allows to skip url encoding. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/890	2020-12-25 11:06:47 +02:00
Aliaksandr Valialkin	b480585905	app/vmalert: typo fix in descriptions for notifier.basicAuth.username and notifier.basicAuth.password command-line flags	2020-12-24 12:49:40 +02:00
Nikolay	0b87f02602	fixes panic (#979 ) * fixes panic https://github.com/VictoriaMetrics/helm-charts/issues/89 * add fast-path * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-12-24 09:01:40 +02:00
Aliaksandr Valialkin	d8511b6651	docs: mention that it is possible to set multiple `-notifier.tlsInsecureSkipVerify` command-line flags for vmalert See c3a92968343c2b3619f1ab935702d0e9b3a46733	2020-12-22 22:32:56 +02:00
Nikolay	67e470e598	changes vmalert notifier flag, (#978 ) fixes issue with notifier insecure setting, now its possible to use multiple notifier.tlsInsecureSkipVerify multiple time.	2020-12-22 22:27:03 +02:00
Roman Khavronenko	9ce8b36d2a	vmalert-974: fix order for labels templating (#975 ) The change fixes bug caused by `3adf8c5a6f`. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/974	2020-12-19 14:21:27 +02:00
Aliaksandr Valialkin	262cf81757	app/vmselect: properly parse negative combined offsets such as `-1h2m3s` Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.	2020-12-19 01:25:03 +02:00
Aliaksandr Valialkin	49e800ba55	app/vmagent: add `vmagent_remotewrite_blocks_sent_total` and `vmagent_remotewrite_bytes_sent_total` metrics per each `-remoteWrite.url`	2020-12-15 20:41:08 +02:00
Aliaksandr Valialkin	9ab7ca1133	docs/vmagent.md: typo fix: pearsed->parsed	2020-12-15 19:03:35 +02:00
Aliaksandr Valialkin	11674a9b76	docs/vmagent.md: mention that `sample_limit` option has no sense when stream parsing is enabled	2020-12-15 18:44:19 +02:00
Aliaksandr Valialkin	8d1031c29a	app/vmselect/promql: return expected `increase()` result for the first point on the graph with value not exceeding 100	2020-12-15 14:10:50 +02:00
Nikolay	7064c4eb8e	adds new Array Flags (#965 ) * adds ArrayDuration and ArrayBool flags, makes sendTimeout and tlsInsecure configurable per remoteWrite url * added backward compatibility testcases for ArrayDuration and ArrayBool * fixes bool flag * fixes test cases	2020-12-15 12:59:33 +02:00
Roman Khavronenko	9f578e389c	vmalert: add function "query", "first" and "value" to alert templates functions (#960 ) The commit adds a support for template function `query`, `first` and `value`. The function `query` executes a MetricsQL query for active alerts. In vmalert we update templates on every evaluation for active alerts to keep them up to date. With `query` func it may become a perf issue since it will fire a query on every execution. We should keep it in mind for now. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/539	2020-12-14 20:12:16 +02:00
Aliaksandr Valialkin	a2eb451de4	app/{vmagent,vminsert}: follow-up for `ce8c2dd1f1`: return `/targets` page in HTML when requested via web browser	2020-12-14 14:13:01 +02:00
Nikolay	324e3aa1a5	Changes targets api (#961 ) * changes /targets api adds html response if requester accepts text/html, adds quick template for /targets api, fixes pathPrefix for / requests * changes namings * renamed targets file * Update app/victoria-metrics/main.go Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com> * adds trimspace to qtpl, moves content-type for targets response closer to writer * fixes bug with prefix Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-12-14 14:13:00 +02:00
Aliaksandr Valialkin	fc82c22e50	docs: consistently use links to https://victoriametrics.github.io for documentation references	2020-12-11 21:09:17 +02:00
Aliaksandr Valialkin	d6f9bf2d19	app/vmselect/graphite: properly handle wildcards and charsets inside curly braces For example, `foo{bar,[a-f]ab}` should match `foobar`, `foobar123`, `foofab`, etc. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/952	2020-12-11 17:26:32 +02:00
Aliaksandr Valialkin	9e79fc27c8	app/vminsert/netstorage: properly update `vm_rpc_rerouted_rows_processed_total` metric Previously this metric wasn't updated because of improper defer call. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/955 Thanks to @xemxx for spotting the bug.	2020-12-11 13:07:05 +02:00
Aliaksandr Valialkin	1a237c6903	all: properly handle CPU limits set on the host system/container This can reduce memory usage on systems with enabled CPU limits. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946	2020-12-08 21:07:03 +02:00
Aliaksandr Valialkin	bd8eef2528	app/vmselect/promql: do not reduce lookbehind window for `any_rollup_func(m)` to `-search.maxStalenessInterval`. It should equal to `step` value passed to `/api/v1/query_range` as most users expect	2020-12-08 15:17:05 +02:00
Aliaksandr Valialkin	7bdf07883b	app/{vmalert,vmagent}: skip empty values in `-remoteWrite.label` and `-label` lists	2020-12-08 14:54:02 +02:00
Aliaksandr Valialkin	9660774fd1	app/vmselect/graphite: remove duplicate `name` tag from `/tags/autoComplete/tags` handler Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/942	2020-12-07 01:10:02 +02:00
Aliaksandr Valialkin	d242c2f2bd	app/vmselect/promql: add `count_eq_over_time(m[d], N)` and `count_ne_over_time(m[d], N)` for calculating the number of samples in `m` over `d` that are equal / not equal to `N`	2020-12-05 12:31:01 +02:00
Aliaksandr Valialkin	bdac2171f1	all: do not print usage info for all the flags when incorrect command-line flag is passed This should improve usability for VictoriaMetrics apps that have big number of command-line flags, i.e. all the apps.	2020-12-03 21:46:19 +02:00
Aliaksandr Valialkin	8cf76d8747	app/vmselect/promql: add `label_uppercase(q, label1, ... labelN)` and `label_lowercase(q, label1, ... labelN)` functions Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/936	2020-12-03 21:46:18 +02:00
Aliaksandr Valialkin	11bbb3552d	app/vmselect/promql: `make fmt`	2020-12-02 21:34:15 +02:00
Aliaksandr Valialkin	9e98a8f3d3	app/vmselect/promql: return `nan` from `minute(m)` when `m` equals to `nan` This aligns VictoriaMetrics behaviour with Prometheus behaviour. The issue has been spotted in https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/	2020-12-02 20:16:40 +02:00
Aliaksandr Valialkin	def513355e	app/vmselect/promql: do not return `0` value from `sum_over_time(m[d])` when there are no samples on the given `d` window. This aligns the behaviour of `sum_over_time()` with other `_over_time()` functions and with Prometheus behavior.	2020-12-02 13:12:33 +02:00
Aliaksandr Valialkin	490c70a958	app/vmselect: return `metric` values from `time() cmp_op metric` query when `cmp_op` comparison is true This aligns MetricsQL behavior to Prometheus' one. The issue has been identified at https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/	2020-12-02 12:09:40 +02:00
Aliaksandr Valialkin	4ef7158e89	app/vmselect/promql: return `nan` from `a >bool b` if `a` is `nan` in the same way as Prometheus does	2020-12-02 00:28:56 +02:00
Aliaksandr Valialkin	adf45b730c	app/vmselect/searchutils: return elapsed time in Deadline.String() output This should improve debuggability for error messages containing Deadline.String() output	2020-12-01 00:14:36 +02:00
Aliaksandr Valialkin	1dce37b2fa	app/vmbackup/snapshot: add missing status code check for the returned response when working with snapshot API Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/929	2020-11-30 14:49:29 +02:00
Aliaksandr Valialkin	8b5a38376d	app/vmbackup/snapshot: log url and response body on failed JSON response parsing Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/929	2020-11-29 12:16:08 +02:00
Nikolay	e4e33cb757	fixes checksum calculation (#928 ) * fixes checksum calculation, 'for' rule param wasnt marshal properly during checksum calculation * fixes error	2020-11-29 09:50:57 +02:00
Aliaksandr Valialkin	3f52e59efe	app/{vmagent,victoria-metrics}: add `-dryRun` option and make more clear handling for `-promscrape.config.dryRun`	2020-11-25 23:01:39 +02:00
Aliaksandr Valialkin	ed06990609	app/vmagent: do not enable -promscrape.config.strictParse when -dryRun command-line flag is set Users can specify -promscrape.config.strictParse if -promscrape.config shouldn't contain unknown config entries	2020-11-25 22:27:41 +02:00
BigFish	3159b41689	Update main.go (#922 ) fix spelling mistake	2020-11-24 12:36:47 +02:00
Aliaksandr Valialkin	2cc288c023	app/vmbackup: cosmetic fixes	2020-11-23 17:10:13 +02:00
Aliaksandr Valialkin	e1297c0b78	app/vmselect: add `/tags/delSeries` handler from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#removing-series-from-the-tagdb	2020-11-23 15:32:14 +02:00
Aliaksandr Valialkin	3d2ce31cad	app/vmselect/netstorage: code readability improvement: rename RequestErrors to Errors	2020-11-23 15:00:15 +02:00
Aliaksandr Valialkin	433ae806ac	app/vmselect: implement `/tags/tagSeries` and /tags/tagMultiSeries` in order to be consistent with single-node VictoriaMetrics	2020-11-23 14:57:08 +02:00
Aliaksandr Valialkin	7987129baa	app/vmselect/netstorage: move common code for requests execution on all the storage nodes to startStorageNodesRequest func	2020-11-23 10:51:48 +02:00
Aliaksandr Valialkin	25a57ced6c	app/vmselect/netstorage: prevent from data races in ProcessSearchQuery and in Export funcs when -replicationFactor > 1 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711	2020-11-23 10:25:51 +02:00
Aliaksandr Valialkin	f4fd917e4f	lib/fs: replace fs.OpenReaderAt with fs.MustOpenReaderAt All the callers for fs.OpenReaderAt expect that the file will be opened. So it is better to log fatal error inside fs.MustOpenReaderAt instead of leaving this to the caller.	2020-11-23 09:57:30 +02:00
Aliaksandr Valialkin	1dcb438c3b	app/vmselect/netstorage: typo fix after `990eb29a9b`	2020-11-23 01:09:43 +02:00
Aliaksandr Valialkin	85eecf5801	app/vmselect/netstorage: add `-replicationFactor` command-line flag for reducing query duration when a part of vmstorage nodes are temporarily slow and/or temporarily unavailable Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711	2020-11-23 00:39:53 +02:00
Aliaksandr Valialkin	990eb29a9b	app/vmselect/netstorage: move common code for collecting query results from vmstorage nodes to collectResults function	2020-11-23 00:16:02 +02:00
Nikolay	bb2bcb9725	Adds eureka service discovery (#913 ) * Adds eureka service discovery https://github.com/VictoriaMetrics/VictoriaMetrics/issues/851 Netflix service discovery for AWS * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-11-20 14:02:13 +02:00
Aliaksandr Valialkin	e72ccc9239	app/vmselect: add remoteAddr to slow query log in order to improve debuggability This will simplify identifying the client that sends slow queries to VictoriaMetrics.	2020-11-18 20:40:02 +02:00
Aliaksandr Valialkin	ea4afb201b	app/vmselect/netstorage: typo fix in a comment inside SearchMetricNames func	2020-11-18 01:35:37 +02:00
Aliaksandr Valialkin	c6adcafedb	app/vminsert: export `vm_rpc_vmstorage_is_reachable` metric, which can be used for monitoring reachability of vmstorage nodes from vminsert nodes	2020-11-17 22:13:26 +02:00
Aliaksandr Valialkin	7d76fdedcc	app/vmselect: use storage.NewSearchQuery() instead of constructing storage.SearchQuery in-place This should prevent from bugs when AccountID and ProjectID aren't set in storage.SearchQuery.	2020-11-16 18:04:33 +02:00
Aliaksandr Valialkin	911c6d3bcd	app/vmselect: add missing graphite prefix to /tags/autoComplete/{tags,values}	2020-11-16 18:04:24 +02:00
Aliaksandr Valialkin	f7f866d83b	app/vmselect/netstorage: typo fix	2020-11-16 15:54:45 +02:00
Aliaksandr Valialkin	59fb75717e	app/vmselect/netstorage: apply Graphite filter after substituting `__name__` with `name`	2020-11-16 15:50:53 +02:00
Aliaksandr Valialkin	eb763bcb9d	app/vmselect/graphite: add `/tags/autoComplete/values` handler from Graphite Tags API	2020-11-16 15:29:29 +02:00
Aliaksandr Valialkin	f2f16d8e79	app/vmselect/graphite: add `/tags/autoComplete/tags` handler from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#auto-complete-support	2020-11-16 14:58:10 +02:00
Aliaksandr Valialkin	2f4421b86c	app/vmselect/prometheus: return `__name__` label if `match[]` query to `/api/v1/labels` matches at least a single time series	2020-11-16 13:54:50 +02:00
Aliaksandr Valialkin	852aed62f7	app/vmselect/prometheus: improve performance for `/api/v1/labels` and `/api/v1/label/<labelName>/values` on time ranges exceeding one day when `match[]` query arg is set	2020-11-16 13:46:51 +02:00
Aliaksandr Valialkin	e969346e3e	app/vmselect/prometheus: fix deadlock in `/api/v1/series` on a time range exceeding one day	2020-11-16 13:30:57 +02:00
Aliaksandr Valialkin	eea1be0d5c	app/vmselect/graphite: add /tags/findSeries handler from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#exploring-tags	2020-11-16 12:52:23 +02:00
Aliaksandr Valialkin	97100b1d42	app/vmselect/graphite: apply filter then limit	2020-11-16 12:52:18 +02:00
Aliaksandr Valialkin	5889273920	app/vmselect/graphite: add `/tags/<tag_name>` handler for Graphite Tags API	2020-11-16 03:41:41 +02:00
Aliaksandr Valialkin	99cb1a70cf	app/vmselect/graphite: add `/tags` handler from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#exploring-tags	2020-11-16 02:57:20 +02:00
Aliaksandr Valialkin	2ac5f00d98	app/vmselect: propagate errors from `vmstorage` to response to the client if `-search.denyPartialResponse` command-line flag is set Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/891 This commit also adds `"isPartial":{true\|false}` field to `/api/v1/*` responses. `"isPartial":true` is set when the response is based on a partial data because some of vmstorage nodes weren't available during query processing.	2020-11-14 13:20:10 +02:00
Aliaksandr Valialkin	882e2e2099	app/vminsert/netstorage: return 503 status code to client when all the vmstorage nodes are unavailable Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/896	2020-11-14 00:44:41 +02:00
Aliaksandr Valialkin	8f42e59e05	app/vmselect/promql: remove spikes from `increase()` and `delta()` results on time series with spare irregular data points Do not take into account spare data point value if the next point will is located too far from the current point. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894	2020-11-13 15:23:37 +02:00
Aliaksandr Valialkin	da6d82a8dd	app/vmselect/promql: assume that time series value doesnt change during gaps when calculating increase() and delta() This should remove unexpected spikes at the end of gaps. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894	2020-11-13 14:59:32 +02:00
Aliaksandr Valialkin	7ceaf4ba8f	all: consistently return text-based HTTP responses with charset=utf-8 This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897	2020-11-13 10:30:21 +02:00
faceair	64e99744f1	add charset on targets response (#897 )	2020-11-13 10:18:13 +02:00
Aliaksandr Valialkin	f7a6ae3d11	docs/vmagent.md: added a link to https://valyala.medium.com/how-to-use-relabeling-in-prometheus-and-victoriametrics-8b90fc22c4b2 into `Relabeling` section	2020-11-12 12:27:13 +02:00
Aliaksandr Valialkin	069979c367	docs/vmagent.md: typo fix	2020-11-11 16:05:04 +02:00
Aliaksandr Valialkin	7a0094adae	docs/vmagent.md: add `Configuration update` section	2020-11-11 16:01:21 +02:00
immerrr again	1ec1a9f27f	app/vmstorage: add "/internal/force_flush" endpoint (#893 )	2020-11-11 14:46:37 +02:00
Aliaksandr Valialkin	4f2c5877db	app/vmselect: add `-search.treatDotsAsIsInRegexps` command-line flag for automatic escaping of dots in regexp label filters	2020-11-11 12:40:28 +02:00
Aliaksandr Valialkin	a78bf34ff3	app/vmselect: do not return `isPartialResponse=true` when all the storageNodes return errors	2020-11-10 18:48:57 +02:00

... 9 10 11 12 13 ...

1802 Commits