VictoriaMetrics

mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-21 07:56:26 +01:00

Author	SHA1	Message	Date
dependabot[bot]	079ede79a3	build(deps): bump typescript in /app/vmui/packages/vmui (#1884 ) Bumps [typescript](https://github.com/Microsoft/TypeScript) from 4.4.4 to 4.5.2. - [Release notes](https://github.com/Microsoft/TypeScript/releases) - [Commits](https://github.com/Microsoft/TypeScript/compare/v4.4.4...v4.5.2) --- updated-dependencies: - dependency-name: typescript dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 22:00:32 +03:00
dependabot[bot]	f977aaee41	build(deps): bump @testing-library/jest-dom in /app/vmui/packages/vmui (#1883 ) Bumps [@testing-library/jest-dom](https://github.com/testing-library/jest-dom) from 5.15.0 to 5.15.1. - [Release notes](https://github.com/testing-library/jest-dom/releases) - [Changelog](https://github.com/testing-library/jest-dom/blob/main/CHANGELOG.md) - [Commits](https://github.com/testing-library/jest-dom/compare/v5.15.0...v5.15.1) --- updated-dependencies: - dependency-name: "@testing-library/jest-dom" dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 22:00:19 +03:00
dependabot[bot]	f56456a45c	build(deps-dev): bump eslint-plugin-react in /app/vmui/packages/vmui (#1882 ) Bumps [eslint-plugin-react](https://github.com/yannickcr/eslint-plugin-react) from 7.26.1 to 7.27.1. - [Release notes](https://github.com/yannickcr/eslint-plugin-react/releases) - [Changelog](https://github.com/yannickcr/eslint-plugin-react/blob/master/CHANGELOG.md) - [Commits](https://github.com/yannickcr/eslint-plugin-react/compare/v7.26.1...v7.27.1) --- updated-dependencies: - dependency-name: eslint-plugin-react dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:58:30 +03:00
dependabot[bot]	4da6e28802	build(deps): bump @codemirror/state in /app/vmui/packages/vmui (#1881 ) Bumps [@codemirror/state](https://github.com/codemirror/state) from 0.19.4 to 0.19.6. - [Release notes](https://github.com/codemirror/state/releases) - [Changelog](https://github.com/codemirror/state/blob/main/CHANGELOG.md) - [Commits](https://github.com/codemirror/state/compare/0.19.4...0.19.6) --- updated-dependencies: - dependency-name: "@codemirror/state" dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:57:56 +03:00
dependabot[bot]	f85480bb3c	build(deps): bump @mui/icons-material in /app/vmui/packages/vmui (#1880 ) Bumps [@mui/icons-material](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-icons-material) from 5.0.5 to 5.2.0. - [Release notes](https://github.com/mui-org/material-ui/releases) - [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md) - [Commits](https://github.com/mui-org/material-ui/commits/v5.2.0/packages/mui-icons-material) --- updated-dependencies: - dependency-name: "@mui/icons-material" dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:57:41 +03:00
dependabot[bot]	bfea7271d5	build(deps): bump @types/react in /app/vmui/packages/vmui (#1879 ) Bumps [@types/react](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/react) from 17.0.34 to 17.0.37. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/react) --- updated-dependencies: - dependency-name: "@types/react" dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:56:35 +03:00
dependabot[bot]	50dac0bd8f	build(deps): bump @codemirror/view in /app/vmui/packages/vmui (#1878 ) Bumps [@codemirror/view](https://github.com/codemirror/view) from 0.19.20 to 0.19.21. - [Release notes](https://github.com/codemirror/view/releases) - [Changelog](https://github.com/codemirror/view/blob/main/CHANGELOG.md) - [Commits](https://github.com/codemirror/view/compare/0.19.20...0.19.21) --- updated-dependencies: - dependency-name: "@codemirror/view" dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:56:18 +03:00
dependabot[bot]	cb508e9678	build(deps): bump @emotion/react in /app/vmui/packages/vmui (#1877 ) Bumps [@emotion/react](https://github.com/emotion-js/emotion) from 11.6.0 to 11.7.0. - [Release notes](https://github.com/emotion-js/emotion/releases) - [Changelog](https://github.com/emotion-js/emotion/blob/main/CHANGELOG.md) - [Commits](https://github.com/emotion-js/emotion/compare/@emotion/react@11.6.0...@emotion/react@11.7.0) --- updated-dependencies: - dependency-name: "@emotion/react" dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:55:48 +03:00
dependabot[bot]	b78fe28f0b	build(deps): bump @mui/lab in /app/vmui/packages/vmui (#1876 ) Bumps [@mui/lab](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-lab) from 5.0.0-alpha.55 to 5.0.0-alpha.58. - [Release notes](https://github.com/mui-org/material-ui/releases) - [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md) - [Commits](https://github.com/mui-org/material-ui/commits/HEAD/packages/mui-lab) --- updated-dependencies: - dependency-name: "@mui/lab" dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:55:09 +03:00
dependabot[bot]	26777abd02	build(deps): bump @emotion/styled in /app/vmui/packages/vmui (#1814 ) Bumps [@emotion/styled](https://github.com/emotion-js/emotion) from 11.3.0 to 11.6.0. - [Release notes](https://github.com/emotion-js/emotion/releases) - [Changelog](https://github.com/emotion-js/emotion/blob/main/CHANGELOG.md) - [Commits](https://github.com/emotion-js/emotion/compare/@emotion/styled@11.3.0...@emotion/styled@11.6.0) --- updated-dependencies: - dependency-name: "@emotion/styled" dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:53:31 +03:00
dependabot[bot]	fc67ca5cfa	build(deps): bump uplot from 1.6.16 to 1.6.17 in /app/vmui/packages/vmui (#1848 ) Bumps [uplot](https://github.com/leeoniya/uPlot) from 1.6.16 to 1.6.17. - [Release notes](https://github.com/leeoniya/uPlot/releases) - [Commits](https://github.com/leeoniya/uPlot/compare/1.6.16...1.6.17) --- updated-dependencies: - dependency-name: uplot dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:53:16 +03:00
dependabot[bot]	6e9f753057	build(deps): bump @mui/styles in /app/vmui/packages/vmui (#1875 ) Bumps [@mui/styles](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-styles) from 5.0.2 to 5.2.2. - [Release notes](https://github.com/mui-org/material-ui/releases) - [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md) - [Commits](https://github.com/mui-org/material-ui/commits/v5.2.2/packages/mui-styles) --- updated-dependencies: - dependency-name: "@mui/styles" dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:52:36 +03:00
dependabot[bot]	ca3106f3bd	build(deps): bump @codemirror/autocomplete in /app/vmui/packages/vmui (#1874 ) Bumps [@codemirror/autocomplete](https://github.com/codemirror/autocomplete) from 0.19.4 to 0.19.9. - [Release notes](https://github.com/codemirror/autocomplete/releases) - [Changelog](https://github.com/codemirror/autocomplete/blob/main/CHANGELOG.md) - [Commits](https://github.com/codemirror/autocomplete/compare/0.19.4...0.19.9) --- updated-dependencies: - dependency-name: "@codemirror/autocomplete" dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:51:56 +03:00
dependabot[bot]	b36fe59dd6	build(deps): bump @types/node in /app/vmui/packages/vmui (#1868 ) Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 16.11.6 to 16.11.10. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) --- updated-dependencies: - dependency-name: "@types/node" dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:51:21 +03:00
dependabot[bot]	5a86354aaa	build(deps): bump @mui/material in /app/vmui/packages/vmui (#1866 ) Bumps [@mui/material](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-material) from 5.0.6 to 5.2.1. - [Release notes](https://github.com/mui-org/material-ui/releases) - [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md) - [Commits](https://github.com/mui-org/material-ui/commits/v5.2.1/packages/mui-material) --- updated-dependencies: - dependency-name: "@mui/material" dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-29 21:50:52 +03:00
Aliaksandr Valialkin	ce31e837eb	app/vmselect/vmui: `make vmui-update`	2021-11-29 12:22:59 +02:00
dependabot[bot]	083044c3e2	build(deps-dev): bump @typescript-eslint/parser (#1843 ) Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) from 5.3.0 to 5.4.0. - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.4.0/packages/parser) --- updated-dependencies: - dependency-name: "@typescript-eslint/parser" dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-22 16:43:37 +03:00
dependabot[bot]	96e6f9ecb6	build(deps): bump @emotion/react in /app/vmui/packages/vmui (#1817 ) Bumps [@emotion/react](https://github.com/emotion-js/emotion) from 11.5.0 to 11.6.0. - [Release notes](https://github.com/emotion-js/emotion/releases) - [Changelog](https://github.com/emotion-js/emotion/blob/main/CHANGELOG.md) - [Commits](https://github.com/emotion-js/emotion/compare/@emotion/react@11.5.0...@emotion/react@11.6.0) --- updated-dependencies: - dependency-name: "@emotion/react" dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-22 16:42:21 +03:00
dependabot[bot]	ad7e225193	build(deps): bump @codemirror/view in /app/vmui/packages/vmui (#1841 ) Bumps [@codemirror/view](https://github.com/codemirror/view) from 0.19.14 to 0.19.20. - [Release notes](https://github.com/codemirror/view/releases) - [Changelog](https://github.com/codemirror/view/blob/main/CHANGELOG.md) - [Commits](https://github.com/codemirror/view/compare/0.19.14...0.19.20) --- updated-dependencies: - dependency-name: "@codemirror/view" dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-22 16:41:50 +03:00
dependabot[bot]	ee2405b042	build(deps): bump @mui/lab in /app/vmui/packages/vmui (#1842 ) Bumps [@mui/lab](https://github.com/mui-org/material-ui/tree/HEAD/packages/mui-lab) from 5.0.0-alpha.53 to 5.0.0-alpha.55. - [Release notes](https://github.com/mui-org/material-ui/releases) - [Changelog](https://github.com/mui-org/material-ui/blob/master/CHANGELOG.md) - [Commits](https://github.com/mui-org/material-ui/commits/HEAD/packages/mui-lab) --- updated-dependencies: - dependency-name: "@mui/lab" dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-22 16:41:16 +03:00
Michael Fuller	cf8c171f85	vmselect: in promql evaluation, return bytes requested when rollup memory limiter is unable to satisfy the request (#1838 ) Co-authored-by: Michael Fuller <mfuller@digitalocean.com>	2021-11-22 13:20:42 +03:00
John Seekins	695cb617b2	Simplify queries to OpenTSDB for migration (#1809 ) * Simplify queries to OpenTSDB (and make them properly appear in OpenTSDB query stats) and also tweak defaults a bit Signed-off-by: John Seekins <jseekins@datto.com> * remove extraneous printlns Signed-off-by: John Seekins <jseekins@datto.com> * remove empty line Signed-off-by: John Seekins <jseekins@datto.com> * fix bug in offset calcuation and closer to working with simpler queries Signed-off-by: John Seekins <jseekins@datto.com> * fix boolean eval Signed-off-by: John Seekins <jseekins@datto.com> * fix casting and check for multiple series Signed-off-by: John Seekins <jseekins@datto.com>	2021-11-18 20:18:15 +03:00
Aliaksandr Valialkin	9bee043ff2	app/vmselect/promql: consistently return zero from `deriv(const)`	2021-11-17 18:02:05 +02:00
Aliaksandr Valialkin	b900560b83	app/vmselect/promql: add `now()` function, which returns the current timestamp as a floating-point value in seconds	2021-11-17 16:35:30 +02:00
Aliaksandr Valialkin	284fec8fcd	app/vmauth: accept requests with Basic Auth username which is equal to `bearer_token` value from the `-auth.config`	2021-11-17 13:31:19 +02:00
Aliaksandr Valialkin	52e19a0577	docs: document -s3ForcePathStyle command-line option This is a follow-up for `b72eed1f5e`	2021-11-17 01:09:32 +02:00
Aliaksandr Valialkin	fc534a1e7f	app/vmalert/README.md: sync with docs/vmalert.md This is a follow-up after `d8c70903ec`	2021-11-17 00:56:04 +02:00
Aliaksandr Valialkin	f3ac945d74	app/vmauth: add ability to override the `username` label value for `vmauth_user_requests_total` metric by specifying `name` option in `-auth.config` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1805	2021-11-17 00:47:34 +02:00
Aliaksandr Valialkin	f43586c63c	app/vmselect/promql: arrange function names in the code in alphabetical order This should simplify code maintenance in the future	2021-11-14 13:55:06 +02:00
Aliaksandr Valialkin	b585a550ba	app/vmui/Dockerfile-web: update Go builder for vmui from v1.17.1 to v1.17.3	2021-11-14 13:55:05 +02:00
Aliaksandr Valialkin	9fa098d8e3	app/vmselect/promql: prevent from incorrect calculations for `deriv()` over multiple samples with identical timestamps	2021-11-12 13:50:43 +02:00
Aliaksandr Valialkin	e5d4c7f4a7	app/vmauth: initialize reverse proxy only after flag.Parse() is called This should properly take into accoun the `-maxIdleConnsPerBackend` command-line flag value. Previously it was hardcoded to 100. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1300	2021-11-09 19:22:34 +02:00
Aliaksandr Valialkin	e5ac9d8e57	all: consistently return `application/json` content-type without `charset=utf-8` The `application/json` content-type has utf-8 encoding by default. See https://stackoverflow.com/questions/9254891/what-does-content-type-application-json-charset-utf-8-really-mean Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897	2021-11-09 18:04:44 +02:00
Aliaksandr Valialkin	5046efb94b	docs/vmalert.md: improve wording in `Multitenancy` chapter	2021-11-09 14:19:52 +02:00
Aliaksandr Valialkin	840ac283ef	app/vmselect/promql: properly return durations smaller than one second from duration_over_time() function Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780	2021-11-09 11:41:56 +02:00
Aliaksandr Valialkin	a67518fc6d	docs: mention that graphs on the official dashboards contain useful hints	2021-11-08 19:54:10 +02:00
Yury Molodov	b78ab88a1c	vmui: migration MUI Core v4 to v5 (#1795 ) * migration MUI Core v4 to v5 * app/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-11-08 15:45:59 +02:00
Aliaksandr Valialkin	3419ac1d36	app/vmselect/promql: add `duration_over_time(m[d], max_interval)` function This function calculates the actual lifetime of the time series on the given lookbehdind window `d` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780	2021-11-08 13:14:09 +02:00
Yury Molodov	6d1d558c4f	vmui: fix graph reset (#1788 ) * feat: add query history * fix: change detect keyUp for nav query history * feat: set default query history * feat: change graph legend * update dependencies * update codemirror version * fix: correct update period time after zoom/pan * fix: optimize data processing for the graph * fix: eliminate memory leaks related to mouse events * fix: correct display of straight line * Merge branch 'master' into vmui-fix-reset-graph * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-11-05 20:56:57 +02:00
Aliaksandr Valialkin	34b5414ba8	app/{vmalert,vmbackup}/README.md: sync with docs after the commit `47d1612bf8`	2021-11-05 20:45:38 +02:00
Aliaksandr Valialkin	237885e0d2	docs/vmalert.md: document the addition of -defaultTenant.prometheus and -defaultTenant.graphite command-line options to enterprise version of vmalert	2021-11-05 20:04:09 +02:00
Aliaksandr Valialkin	24dce03aaa	app/vmalert/datasource: use plain string literals instead of constants This removes the unneeded level of indirection and improves code readability. The "prometheus" and "graphite" constants aren't going to change in the future, so there is no sense in hiding them behind constants.	2021-11-05 19:57:47 +02:00
Aliaksandr Valialkin	bf814320b0	app/vmalert: remove `rule.type` config, since it doesnt play well with the upcoming default tenants for -clusterMode It is better from the consistency point of view to set up rule types at group level where tenant config is set up.	2021-11-05 19:52:32 +02:00
Aliaksandr Valialkin	c43bcdb5fb	app/vmagent: allow bigger number of in-memory blocks for big values of `-remoteWrite.queues` This should improve the maximum data ingestion speed for highly-loaded vmagent instances which run on beefy servers with many CPU cores and big amounts of RAM	2021-11-05 15:16:05 +02:00
Aliaksandr Valialkin	cbfc7b7c92	app/{vminsert,vmagent}: hide passwords and auth tokens by default at `/config` page Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764	2021-11-05 14:41:16 +02:00
Aliaksandr Valialkin	4839d07f34	app/vmagent/remotewrite: fix parallel data sending to remote storage systems at `e0d2ba5608`	2021-11-04 16:58:28 +02:00
Aliaksandr Valialkin	a69264e885	app/vmagent: add `-remoteWrite.maxRowsPerBlock` command-line option, which may be used for improving data ingestion performance under high load	2021-11-04 15:39:14 +02:00
Aliaksandr Valialkin	e0d2ba5608	app/vmagent/remotewrite: send data to remote storage systems in parallel This should improve data ingestion speed when many `-remoteWrite.url` command-line flags are configured	2021-11-04 15:04:16 +02:00
dependabot[bot]	558f77c259	build(deps-dev): bump @typescript-eslint/eslint-plugin (#183 ) Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) from 5.2.0 to 5.3.0. - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v5.3.0/packages/eslint-plugin) --- updated-dependencies: - dependency-name: "@typescript-eslint/eslint-plugin" dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-04 12:53:20 +02:00
Aliaksandr Valialkin	2178335618	app/vmselect: `make vmui-update`	2021-11-04 12:13:12 +02:00
dependabot[bot]	ebaa4e7256	build(deps-dev): bump @babel/plugin-proposal-nullish-coalescing-operator (#1769 ) Bumps [@babel/plugin-proposal-nullish-coalescing-operator](https://github.com/babel/babel/tree/HEAD/packages/babel-plugin-proposal-nullish-coalescing-operator) from 7.14.5 to 7.16.0. - [Release notes](https://github.com/babel/babel/releases) - [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md) - [Commits](https://github.com/babel/babel/commits/v7.16.0/packages/babel-plugin-proposal-nullish-coalescing-operator) --- updated-dependencies: - dependency-name: "@babel/plugin-proposal-nullish-coalescing-operator" dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-04 12:09:34 +02:00
dependabot[bot]	9ed7ead84f	build(deps): bump @date-io/dayjs in /app/vmui/packages/vmui (#1770 ) Bumps [@date-io/dayjs](https://github.com/dmtrKovalenko/date-io) from 1.3.13 to 2.11.0. - [Release notes](https://github.com/dmtrKovalenko/date-io/releases) - [Commits](https://github.com/dmtrKovalenko/date-io/compare/v1.3.13...v2.11.0) --- updated-dependencies: - dependency-name: "@date-io/dayjs" dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2021-11-04 12:09:11 +02:00
Aliaksandr Valialkin	aa534c2582	lib/promscrape: add `-promscrape.maxResponseHeadersSize` command-line flag for tuning the maximum http response headers size from Prometheus scrape targets	2021-11-03 22:26:56 +02:00
Aliaksandr Valialkin	27044b84d2	app/vmselect/promql: add `limit_offset(limit, offset, q)` function, which can be used for paging over big number of time series Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1778	2021-11-03 16:02:27 +02:00
Aliaksandr Valialkin	43a58bd618	app/vmselect/promql: add `label_graphite_group()` function for extracting groups from Graphite metric names	2021-11-03 13:19:08 +02:00
Aliaksandr Valialkin	d1eb87c831	app/{vmagent,vminsert}: add ability to restrict access to /config page with authKey query arg The authKey can be configured via `-configAuthKey` command-line flag. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764	2021-11-01 16:44:54 +02:00
Aliaksandr Valialkin	1a16dab9e1	docs/vmauth.md: typo fix	2021-10-28 14:06:00 +03:00
Aliaksandr Valialkin	acfda6d8fd	app/vmbackupmanager: fix links to images This is a follow-up after `bd6b8f7e31`	2021-10-27 21:35:52 +03:00
Yury Molodov	47ee3744f2	vmui: correct migration material-ui (#1758 ) * migration material-ui * fix: rollback popover * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-27 20:41:26 +03:00
Aliaksandr Valialkin	6608705652	app/{vmalert,vmagent}: improve the distribution of scrape offsets among targets / rules Previously only the lower part of 64-bit hash was used for calculating the offset. This may give uneven distribution in some cases. So let's use all the available 64 bits from the hash for calculating the offset.	2021-10-27 19:59:16 +03:00
Aliaksandr Valialkin	1952ab99aa	all: fix build issues and tests for Apple M1 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1653	2021-10-27 15:06:34 +03:00
Aliaksandr Valialkin	4821adfd95	lib/promscrape: properly show `proxy_url` option value at `/config` page Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755	2021-10-26 21:23:54 +03:00
Yury Molodov	956cf83e7b	vmui: update dependencies (#1754 ) * update dependencies * update codemirror version * app/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-26 19:31:20 +03:00
Aliaksandr Valialkin	88d42c3ac1	app/vmbackup/README.md: sync with docs/vmbackup.md after `e706fb5686`	2021-10-26 19:20:47 +03:00
Roman Khavronenko	3dbdf1632e	vmalert: allow groups with empty rules for compatibility reasons (#1742 ) Prometheus allows to have groups with no rules, so we should support it in vmalert as well for compatibility reasons. It is also allowed to hot-reload empty groups by adding or removing rules. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-10-25 12:15:02 +03:00
Aliaksandr Valialkin	df8f967040	app/vmselect/promql: reduce the precision from 15 significant digits to 13 significant digits when comparing float64 results in tests See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1738	2021-10-24 13:31:14 +03:00
Aliaksandr Valialkin	013d626889	app/vmauth: add ability to specify http headers to send in requests to backends See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1736	2021-10-22 19:10:29 +03:00
Aliaksandr Valialkin	2876137c92	lib/promscrape: reduce memory usage if `-promscrape.noStaleMarkers` command-line flag is passed Do not store in memory the response from the last scrape per each target if -promscrape.noStaleMarkers option is enabled. This should reduce memory usage when the scraped targets return large responses.	2021-10-22 13:10:29 +03:00
Roman Khavronenko	43a7984cd8	vmalert: correctly calculate alert ID including extra labels (#1734 ) Previously, ID for alert entity was generated without alertname or groupname. This led to collision, when multiple alerting rules within the same group producing same labelsets. E.g. expr: `sum(metric1) by (job) > 0` and expr: `sum(metric2) by (job) > 0` could result into same labelset `job: "job"`. The issue affects only UI and Web API parts of vmalert, because alert ID is used only for displaying and finding active alerts. It does not affect state restore procedure, since this label was added right before pushing to remote storage. The change now adds all extra labels right after receiving response from the datasource. And removes adding extra labels before pushing to remote storage. Additionally, change introduces a new flag `Restored` which will be displayed in UI for alerts which have been restored from remote storage on restart.	2021-10-22 12:30:38 +03:00
Yury Molodov	2b266cb87e	vmui: query history (#1732 ) * feat: add query history * fix: change detect keyUp for nav query history * feat: set default query history * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-22 12:21:22 +03:00
Aliaksandr Valialkin	8ad95f0db7	lib/httpserver: expose command-line flags at `/flags` page This should simplify debugging. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695	2021-10-20 00:45:09 +03:00
Roman Khavronenko	bdfac4ff53	vmalert: make group.ID() thread-safe (#1726 ) Commit fixes potential race condition when group update and generating of ID() happens simultaneously. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-10-19 16:44:13 +03:00
Roman Khavronenko	dcd881bb7a	vmalert: properly init SIGHUP listener before starting group manager (#1725 ) Regression was introduced during code refactoring. It potentially could lead to situation when SIGHUP signals were ignored while vmalert was still busy with initing group manager. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-10-19 16:35:27 +03:00
Aliaksandr Valialkin	b8123b862a	app/vmauth: fix metric name prefixes: `vmagent` -> `vmauth`	2021-10-19 15:29:07 +03:00
Yury Molodov	a3e09a57c2	vmui: features (#1711 ) * feat: initial uPlot graph * feat: add zoom/pan for graph * fix: add zoom by ctrl/mac * fix: remove unused code * feat: add toggle cache for fetch * feat: add fix y-axis limits * fix: stop point events while panning * fix: change getting cursor position when scaling * feat: add cursor tooltip to graph * fix: uninstall chart.js * fix: change link for create an issue * fix: set default cache value to true * app/vmalert: follow-up after `0e2486df56` * docs/CHANGELOG.md: document `5416e18007` * app/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-18 15:16:57 +03:00
Roman Khavronenko	146a5b504c	vmalert: remove extra `/` from path in WEB interface (#1717 ) The extra `/` may cause issues when additional path prefixes are configured. Also, removing it makes it consistent with the rest of declarations. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-10-18 15:12:47 +03:00
Roman Khavronenko	478854d36d	vmctl: follow-up after `95d1d38595` (#1718 ) Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-10-18 15:10:44 +03:00
Miro Prasil	5416e18007	vmctl influx convert bool to number (#1714 ) vmctl: properly convert influx bools into integer representation When using vmctl influx, the import would fail importing boolean fields with: ``` failed to convert value "some".0 to float64: unexpected value type true ``` This converts `true` to `1` and `false` to `0`. Fixes #1709	2021-10-18 10:29:34 +03:00
Alexander Rickardsson	0e2486df56	vmalert: add disablePathAppend to remote read (#1712 ) * vmalert: add disablePathAppend to remoteRead * docs: add docs for remoteRead.disablePathAppend	2021-10-18 10:24:52 +03:00
Alexander Rickardsson	c0e58ade45	vmalert: Redact passwords from error messages (#1713 )	2021-10-18 10:20:26 +03:00
Aliaksandr Valialkin	da97e58979	app/vmselect/promql: randomize the static selection of time series returned from `limitk()` Sort series by a hash calculated from the series labels. This should guarantee "random" selection of the returned time series. Previously the selection could be biased, since time series were sorted alphabetically by label names and label values.	2021-10-16 21:16:49 +03:00
Aliaksandr Valialkin	cae174b11c	app/vmselect/promql: typo fix in comment: didsn't -> didn't	2021-10-16 13:00:34 +03:00
Aliaksandr Valialkin	9866dd95c1	lib/promscrape: store the full response in stream parsing mode in scrapeWork.lastScrape byte slice This allows sending staleness marks and properly calculate scrape_series_added metric in stream parsing mode at the cost of the increased memory usage, since now the potentially big response is kept in the lastScrape byte slice per each scrapeWork. In practice the memory usage increase shouldn't be big, since the response size is usually much smaller than the parsed metrics from this response after the relabeling, which usually adds a big pile of target-specific labels per each metric.	2021-10-15 15:39:23 +03:00
Aliaksandr Valialkin	bbd34fa15e	lib/promscrape: add `-promscrape.minResponseSizeForStreamParse` command-line option for automatic switching to stream parsing mode when scraping targets with big responses This should reduce memory usage when vmagent scrapes targets with non-uniform response sizes. This is common case in Kubernetes monitoring.	2021-10-14 12:29:35 +03:00
Aliaksandr Valialkin	1a7287c408	lib/promscrape: return error if `sample_limit` or `series_limit` options are set when stream parsing mode is enabled	2021-10-14 12:11:23 +03:00
Roman Khavronenko	7fcbd3fa4b	Adjust `http.Transport.MaxIdleConns` setting for vmauth/vmalert services (#1704 ) * vmalert: adjust `http.Transport.MaxIdleConns` value accordingly to `http.Transport.MaxIdleConnsPerHost` `http.Transport.MaxIdleConnsPerHost` setting is controlled by `datasource.maxIdleConnections` flag, while `http.Transport.MaxIdleConns` is inherited from DefaultTransport and is equal to `100`. The fix adjusts `http.Transport.MaxIdleConns` value if it is lower than `http.Transport.MaxIdleConnsPerHost`. Signed-off-by: hagen1778 <roman@victoriametrics.com> * vmauth: adjust `http.Transport.MaxIdleConns` value accordingly to `http.Transport.MaxIdleConnsPerHost` `http.Transport.MaxIdleConnsPerHost` setting is controlled by `maxIdleConnsPerBackend` flag, while `http.Transport.MaxIdleConns` is inherited from DefaultTransport and is equal to `100`. The fix adjusts `http.Transport.MaxIdleConns` value if it is lower than `http.Transport.MaxIdleConnsPerHost`. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-10-13 17:29:28 +03:00
Roman Khavronenko	8df3c569c7	vmalert: add Source link to alerts UI (#1701 ) The source link is controlled by `external.url` and `external.alert.source` flags, in the same way as for alertmanager notifications. The source link is added to Alerts list view, and specific Alert view.	2021-10-13 15:25:11 +03:00
Aliaksandr Valialkin	5a58c041c2	app/vmagent: expose -promscrape.config contents at /config page as Prometheus does See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695	2021-10-12 16:25:37 +03:00
Aliaksandr Valialkin	a5001b9c20	app/vmselect/promql: add `atan2` binary operator, which is going to be added in Prometheus 2.31 See https://github.com/prometheus/prometheus/pull/9248	2021-10-11 21:15:53 +03:00
Aliaksandr Valialkin	81c6720392	app/vmselect/promql: add missing trigonometric functions, which are going to be added in Prometheus 2.31 See https://github.com/prometheus/prometheus/issues/9233	2021-10-11 21:01:33 +03:00
Aliaksandr Valialkin	92b92d4d2c	app/vmselect/promql: consistently return the same set of time series from `limitk()` function This is the expected behaviour by most users.	2021-10-08 19:53:52 +03:00
Aliaksandr Valialkin	0ff8fcac6a	app/vmui: follow-up after `7bfb44113e` * Run `vmui-update` * Document the changes in README.md and CHANGELOG.md	2021-10-08 15:09:29 +03:00
Yury Molodov	7bfb44113e	vmui: use uPlot as default engine for graph (#1683 ) * feat: initial uPlot graph * feat: add zoom/pan for graph * fix: add zoom by ctrl/mac * fix: remove unused code	2021-10-08 15:07:35 +03:00
Aliaksandr Valialkin	cf5cbd1c70	app/{vminsert,vmstorage}: follow-up after `a171916ef5` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/269	2021-10-08 14:35:49 +03:00
Nikolay	4290b46e8c	Adds read-only mode for vmstorage node (#1680 ) * adds read-only mode for vmstorage https://github.com/VictoriaMetrics/VictoriaMetrics/issues/269 * changes order a bit * moves isFreeDiskLimitReached var to storage struct renames functions to be consistent change protoparser api - with optional storage limit check for given openned storage * renames freeSpaceLimit to ReadOnly	2021-10-08 14:35:48 +03:00
Aliaksandr Valialkin	2748255c8b	app/vmselect/promql: substitute rollupFuncsCannotAdjustWindow with rollupFuncsCanAdjustWindow The list of functions, which can adjust lookbehind window is more limited than the rest of functions, so it is better from maintainability and readability PoV using the allowlist instead of blocklist.	2021-10-07 13:18:42 +03:00
Aliaksandr Valialkin	c45210a6f9	app/vmselect/promql: return back the behaviour for `deriv()` function when the lookbehind window doesnt contain enough points It is expected that the `deriv(m[d])` returns non-empty value if the lookbehind window `d` contains less than 2 samples in the same way as `rate()` does. This is a follow-up after `3e084be06b` .	2021-10-07 12:52:27 +03:00
Roman Khavronenko	3e084be06b	app/vmselect: make `predict_linear` and `deriv` compatible with Prometheus (#1681 ) Previously, `predict_linear` returned slightly different results comparing to Prometheus. The change makes linear regression algorithm compatible with Prometheus. `deriv` was excluded from the list of functions which can adjust the time window for the same reasons.	2021-10-07 12:50:49 +03:00
Aliaksandr Valialkin	c7c966d0e9	docs/vmagent.md: update docs after 3e9a939a990c8b608414388c96f68eb062364ae7	2021-10-05 10:23:33 +03:00
Aliaksandr Valialkin	9515e58e28	docs/vmagent.md: document how to write data to Kafka	2021-09-30 17:45:53 +03:00
Aliaksandr Valialkin	0e3de5a0cc	app/vmselect/promql: add `topk_last` and `bottomk_last` functions	2021-09-30 13:22:52 +03:00
Roman Khavronenko	a31407006c	app/vmselect: fix binary comparison func (#1667 ) The fix makes the binary comparison func to check for NaNs before executing the actual comparison. This prevents VM to return values for non-existing samples for expressions which contain bool comparisons. Please see added test for example.	2021-09-30 12:24:17 +03:00
Roman Khavronenko	344490d89b	app/vmselect: fix `testRowsEqual` func NaN checks (#1666 ) It appeared, that `testRowsEqual` NaN comparison was incorrect. The fix caused some tests to fail. Please see the change and tests updated. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-30 12:08:47 +03:00
Aliaksandr Valialkin	6d9f1d4227	app/vminsert: document that -relabelConfig is reloaded on SIGHUP signal	2021-09-29 21:18:58 +03:00
Aliaksandr Valialkin	d80d72efec	app/{vmbackup,vmrestore}: switch from `gcs://...` to `gs://...` urls for backups to GCS The `gs://` urls are commonly used, so prefer them instead of `gcs://` urls, while leaving support for `gcs://` urls for backwards compatibility.	2021-09-29 12:10:29 +03:00
Aliaksandr Valialkin	396e233ac1	docs/vmagent.md: update Telegraf config in the section about Kafka	2021-09-29 11:21:15 +03:00
Aliaksandr Valialkin	0e5ab52908	docs/vmagent.md: add docs about reading metrics from Kafka	2021-09-29 01:46:12 +03:00
Yury Molodov	893af0a92c	vmui: fixed bug with time range (time zone) (#1661 ) * fix: set date in query string in utc format * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-29 01:00:44 +03:00
Nikolay	cc72f9428d	changes vmagent api (#1656 ) * changes vmagent api adds auth.Token to promremotewrite InsertHandlerReader changes remoteWrite client constructor, allows to use multiple remoteWriteUrl schemes, like kafka:// changes url path concatenation for tenant remoteWrite Update app/vmagent/remotewrite/client.go Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com> * Update app/vmagent/remotewrite/remotewrite.go * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-29 00:52:07 +03:00
Roman Khavronenko	5dc84bf210	app/vmselect: disable time-window adjustment for `min/max_over_time` funcs (#1658 ) Adjustment results into discrepancy between Prometheus and VM on time windows smaller than scrape interval. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:43:21 +03:00
Roman Khavronenko	de810031bf	app/vmselect: always return zero for `stddev` func if there is only one value (#1659 ) The fix will always return zero if received set of items consists of one element only, which also means no deviation. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:38:55 +03:00
Roman Khavronenko	dd536b475c	app/vmselect: return NaN instead of 0 for empty value sets (#1660 ) The change affects `count/stddev/stdvar_over_time` funcs and makes them to return NaN instead of zero when there is no datapoints in a time window. This is needed for improving compatibility with Prometheus. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:37:04 +03:00
Roman Khavronenko	03cd93bf1a	app/vmselect: rm `quantile_over_time` fast-path optimisations (#1662 ) The removed fast path optimisations weren't consistent with `quantile` function behavior and results into discrepancy. Specifically, results didn't match in cases when: * 0 < phi > 1; * values contain only one element. Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-29 00:35:14 +03:00
Aliaksandr Valialkin	91b3c601bc	app/{vminsert,vmagent}: add ability to ingest data via DataDog "submit metrics" API See https://docs.datadoghq.com/api/latest/metrics/#submit-metrics Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/206	2021-09-29 00:13:08 +03:00
Yury Molodov	a64155d91e	vmui: use Chart.js as default engine for graph (#1634 ) * feat: add Plotly as default engine for graph * fix: remove unused components * feat: use Chart.js as default engine graph * fix: correct styles for loader * feat: add zoom/pan for chart * feat: add height for chart * fix: remove unused code * fix: remove empty units from duration * fix: change debounce for pan to 500ms * fix: add utility for plugins register globally * fix: optimize render graph * feat: add buffer data for zoom * fix: add limits for zoom in/out * fix: change update data while zooming * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-27 22:26:14 +03:00
Aliaksandr Valialkin	8c0283381d	app/victoria-metrics/testdata/graphite/max_lookback_unset.json: fix the test after `c4c77aa2dd` The commit `c4c77aa2dd` slightly changed how scrape_interval is detected per-time series, so the max_lookback_unset test should be updated accordingly.	2021-09-27 21:41:14 +03:00
Aliaksandr Valialkin	2efe0acfc9	app/vmselect/promql: add `rollup_scrape_interval(m[d])` function It calculates the min, max and avg scrape intervals for m over the given lookbehind window d	2021-09-27 19:21:24 +03:00
Aliaksandr Valialkin	c4c77aa2dd	app/vmselect/promql: follow-up after `526dd93b32` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612	2021-09-27 18:55:39 +03:00
Roman Khavronenko	526dd93b32	app/vmselect: `quantile` func compatiblity with Prometheus (#1646 ) * app/vmselect: `quantile` func compatiblity with Prometheus The `quantile` func was previously calculated by https://github.com/valyala/histogram package. The result of such calculation was always the closest real value to requested quantile. While in Prometheus implementation interpolation is used. Such difference may result into discrepancy in output between Prometheus and VictoriaMetrics. This commit adds a Prometheus-like `quantile` function. It also used by other functions which depend on it, such as `quantiles`, `quantile_over_time`, `median` etc. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625 Signed-off-by: hagen1778 <roman@victoriametrics.com> * app/vmselect: `quantile` review fixes * quantile functions were split into multiple to provide different API for already sorted data; * float64sPool is used for reducing allocations. Items in pool may have different sizes, but defining a new pool was complicates due to name collisions; Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-27 18:02:41 +03:00
Aliaksandr Valialkin	8ed95e82c6	app/vmselect/promql: follow-up after `57b3320478`	2021-09-24 01:24:18 +03:00
Roman Khavronenko	57b3320478	app/vmselect: make sorting for query result similar to Prometheus (#1647 ) * app/vmselect: make sorting for query result similar to Prometheus Updated sorting allows to get the order of series in result similar or equal to what Prometheus returns. The change is needed for compatibility reasons. * Update app/vmselect/promql/exec_test.go Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-24 01:03:12 +03:00
Aliaksandr Valialkin	e564411a62	app/vmselect/promql: align the behavior of `or`, `and` and `unless` operators with `on (labels)` modifier to Prometheus Previously VictoriaMetrics could return unexpected result of the right-hand side operand had multiple time series with the given set of labels mentioned in `on(labels)`. See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1643	2021-09-24 00:46:25 +03:00
Aliaksandr Valialkin	c5bb95a417	docs: `make docs-sync`	2021-09-23 20:51:35 +03:00
Roman Khavronenko	9dc4d16664	app/vmctl: fix misleading comment about cluster version for native mode (#1648 ) https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1637 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-23 17:57:25 +03:00
Roman Khavronenko	0e35fc9538	app/vmalert: remove unnecessary `omitempty` tag for `interval` param (#1649 ) `omitempty` tag resulted into skipping this param on marshaling, which was used as a checksum for groups configuration. Since on config reload checksums are compared before applying changes, any change to `interval` only didn't trigger config reload. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1641 Signed-off-by: hagen1778 <roman@victoriametrics.com>	2021-09-23 17:55:59 +03:00
Roman Khavronenko	ac1abe2faf	app/vmalert: support `http.pathPrefix` flag in UI (#1636 ) The change makes UI to respect `http.pathPrefix` flag for API or navigation items links.	2021-09-21 14:41:01 +03:00
Aliaksandr Valialkin	a22aa0608b	app/vmselect: fix accessing `/graphite/*` endpoints	2021-09-21 13:56:35 +03:00
Aliaksandr Valialkin	94148d5ad7	docs/vmagent.md: typo fix	2021-09-20 16:49:20 +03:00
Aliaksandr Valialkin	51657b1e04	docs/vmagent.md: typo fixes in `Prometheus staleness markers` docs	2021-09-20 16:44:09 +03:00
Nikolay	ad08d9dfc0	changes protoparser apis for accepting reading from io.Reader (#1624 ) adds InsertHandlerForReader apis to vmagent	2021-09-20 14:49:28 +03:00
Aliaksandr Valialkin	9a3d0c43b5	app/vmselect/promql: add `quantiles_over_time("phiLabel", phi1, ..., phiN, m[d])` function for calculating multiple quantiles at once	2021-09-17 23:35:10 +03:00
Aliaksandr Valialkin	2951dd0a57	app/vmselect/promql: add histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) function This function calculates multiple quantiles over the given buckets at once Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573	2021-09-17 13:32:39 +03:00
Aliaksandr Valialkin	5a44be0e52	app/vmselect/promql: optimize quantiles() calculation Calculate quantiles in one go instead of calculating each quantile individually	2021-09-17 12:33:42 +03:00
Roman Khavronenko	b75455c650	vmalert: add new metric `vmalert_remotewrite_flush_duration_seconds` (#1622 )	2021-09-16 14:00:16 +03:00
f41gh7	9375b60c5f	adds stub for functions api	2021-09-16 13:49:52 +03:00
Aliaksandr Valialkin	e60dfc96ff	app/vmselect/promql: add `mad(q)` and `outliers_mad(tolerance, q)` functions to MetricsQL	2021-09-16 13:33:53 +03:00
Aliaksandr Valialkin	eca75cc650	app/vmselect/prometheus: make more clear log messages for errors during sending data to remote clients	2021-09-16 12:56:58 +03:00
Aliaksandr Valialkin	2f86d4cf38	app/vmui: follow-up after `777ff75874` The commit contains the following changes: - Show vmui when requesting /graph page in order to be compatible with Prometheus datasource in Grafana. - Properly encode query args at vmui url. - Set the number of points on the graph to the number of horizontal pixels divided by 2. Previously it was hardcoded to 30. - Do not save server url to persistent storage at browser, since it should be always obtained from the url. - Run `make vmui-update` for updating vmui embedded into VictoriaMetrics.	2021-09-15 17:40:48 +03:00
Yury Molodov	777ff75874	vmui: change query params compatible with prometheus (#1619 ) * feat: change url params for compatible prometheus * style: add comment for TimeParams * fix: change get default server for single version * fix: change function for get query string value	2021-09-15 09:42:49 +03:00
Roman Khavronenko	ecd3069b6c	vmalert: create basic auth config only if args aren't empty (#1618 ) * vmalert: create basic auth config only if args aren't empty follow-up after `68721f6` * vmalert: make lint happy	2021-09-15 01:53:31 +03:00
Aliaksandr Valialkin	3e1683756b	docs/vmalert.md: follow-up after `68721f6e7d` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1608	2021-09-14 14:47:47 +03:00
Roman Khavronenko	68721f6e7d	vmalert: support bearer token for datasource, remotewrite and remoteread (#1614 ) https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1608	2021-09-14 14:32:06 +03:00
Aliaksandr Valialkin	adedc83b3b	app/vmauth: do not log invalid auth tokens by default for security reasons The logging can be enabled by passing `-logInvalidAuthTokens` command-line flag to vmauth	2021-09-14 12:20:03 +03:00
Aliaksandr Valialkin	07b9c7994f	docs/vmagent.md: mention `out of order sample` errors, which are typically emitted by Thanos, Cortex or Prometheus	2021-09-13 19:36:31 +03:00
Aliaksandr Valialkin	c4f11a49f8	docs/CHANGELOG.md: document `5494bc02a6`	2021-09-13 17:11:23 +03:00
Aliaksandr Valialkin	7f0a8d4bdb	docs: consistency renaming: Influx -> InfluxDB	2021-09-13 17:05:16 +03:00
Aliaksandr Valialkin	143a3b34ee	app/vmui/Dockerfile-web: update Go builder from 1.16.7 to 1.17.1 and Alpine base image from 3.14.1 to 3.14.2	2021-09-13 17:05:16 +03:00
Roman Khavronenko	5494bc02a6	vmalert: add flag to limit the max value for auto-resovle duration for alerts (#1609 ) * vmalert: add flag to limit the max value for auto-resovle duration for alerts The new flag `rule.maxResolveDuration` suppose to limit max value for alert.End param, which is used by notifiers like Alertmanager for alerts auto resolve. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1586	2021-09-13 15:48:18 +03:00
Aliaksandr Valialkin	b9727a36dc	docs/vmbackup.md: update the outdated link to vmbackupmanager	2021-09-13 14:16:53 +03:00
Roman Khavronenko	75f35c3b11	vmalert: display extra filter labels in UI (#1613 )	2021-09-13 14:11:38 +03:00
Aliaksandr Valialkin	d1a16e0891	app/vmselect/promql: use Prometheus-compatible label value formatting for `count_values` function	2021-09-13 13:48:06 +03:00
Aliaksandr Valialkin	674a6eee6c	docs/Single-server-VictoriaMetrics.md: refer to relabeling section for vmagent This removes duplicate docs about additional relabeling actions supported by VictoriaMetrics components	2021-09-12 11:39:19 +03:00
Aliaksandr Valialkin	9286107e82	lib/promscrape: send stale markers for disappeared metrics like Prometheus does	2021-09-11 10:51:04 +03:00
Aliaksandr Valialkin	cfed015bb6	docs/vmalert.md: typo fix in `Multitenancy` chapter	2021-09-10 17:57:14 +03:00
Aliaksandr Valialkin	f4dead529f	lib/storage: properly search series by multiple tag filters matching empty labels such as foo{bar=~"baz\|",x=~"y\|"} Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1601 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/395	2021-09-09 21:09:21 +03:00
Aliaksandr Valialkin	ea943911bc	app/vmselect/promql: keep metric name in `rollup_candlestick` results, since they don't change the original series meaning Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1600	2021-09-09 19:21:18 +03:00
Aliaksandr Valialkin	4aeb8db83f	lib/promscrape: add ability to set `series_limit` and `stream_parse` options via relabeling This allows managing these options on a per-target basis. Typical use case: to manage these options for pods via Kubernetes annotations.	2021-09-09 18:49:39 +03:00
Aliaksandr Valialkin	d2708a1fb7	docs/vmagent.md: typo fix in `Relabeling` chapter	2021-09-09 16:39:40 +03:00
Aliaksandr Valialkin	d6bd956930	lib/promrelabel: add `keep_metrics` and `drop_metrics` actions to relabeling rules These actions simlify metrics filtering. For example, - action: keep_metrics regex: 'foo\|bar\|baz' would leave only metrics with `foo`, `bar` and `baz` names, while the rest of metrics will be deleted. The commit also makes possible to split long regexps into multiple lines. For example, the following config is equivalent to the config above: - action: keep_metrics regex: - foo - bar - baz	2021-09-09 16:18:21 +03:00
Aliaksandr Valialkin	e84fa9eb38	app/vmalert: document GroupAlerts This makes golint happy	2021-09-07 22:50:08 +03:00
Aliaksandr Valialkin	e6c9869d86	app/vmalert: follow-up after `21f022e5f0`	2021-09-07 22:43:37 +03:00
Roman Khavronenko	21f022e5f0	vmalert: add initial UI implementation (#1602 ) New UI pages: / - welcome page with API handlers list; /groups - list of all rules per group; /alerts - list of all active alerts; /groupID/alertID/status - status of the active alert;	2021-09-07 22:39:22 +03:00
Aliaksandr Valialkin	e34f7081d0	docs/vmagent.md: add API path for Prometheus text exposition format	2021-09-07 16:14:51 +03:00
Aliaksandr Valialkin	ed818fceef	docs: update `-help` output for victoria-metrics and vmagent after `f77dde837a`	2021-09-01 16:34:32 +03:00
Aliaksandr Valialkin	f77dde837a	lib/promscrape: add the ability to limit the number of unique series per each scrape target The number of series per target can be limited with the following options: * Global limit with `-promscrape.maxSeriesPerTarget` command-line option. * Per-target limit with `max_series: N` option in `scrape_config` section. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1561	2021-09-01 16:03:59 +03:00
Roman Khavronenko	de26b1d4a2	vmctl: update README and flags description (#1588 ) The purpose of update is to make README and flags description more clear to the reader. Especially, show that vm-account-id flag is required for clustered version of VM.	2021-09-01 09:31:44 +03:00
Roman Khavronenko	cfb6436be5	Vmalert extra params (#1587 ) * vmalert: allow extra GET params in datasource package ExtraParams will be added as GET params to every HTTP request made by datasource. The `roundDigits` param, for example, was substituted by corresponding extra param. * vmalert: add nocache=1 param for replay process The `nocache=1` param is VictoriaMetrics specific parameter which prevents it from caching and boundaries aligning for queries. We set it to avoid cache pollution in `replay` mode and also to avoid unnecessary time range boundaries alignment. * vmalert: mention nocache=1 in replay description * vmalert: fix bug with unused param	2021-08-31 14:57:47 +03:00
Nikolay	7c70dcbe3b	adds external_labels per group for vmalert (#1485 ) * adds external_label per group for vmalert https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1471	2021-08-31 14:52:34 +03:00
Roman Khavronenko	eff940aa76	Vmalert metrics update (#1580 ) * vmalert: remove `vmalert_execution_duration_seconds` metric The summary for `vmalert_execution_duration_seconds` metric gives no additional value comparing to `vmalert_iteration_duration_seconds` metric. * vmalert: update config reload success metric properly Previously, if there was unsuccessfull attempt to reload config and then rollback to previous version - the metric remained set to 0. * vmalert: add Grafana dashboard to overview application metrics * docker: include vmalert target into list for scraping * vmalert: extend notifier metrics with addr label The change adds an `addr` label to metrics for alerts_sent and alerts_send_errors to identify which exact address is having issues. The according change was made to vmalert dashboard. * vmalert: update documentation and docker environment for vmalert's dashboard Mention Grafana's dashboard in vmalert's README in a new section #Monitoring. Update docker-compose env to automatically add vmalert's dashboard. Update docker-compose README with additional info about services.	2021-08-31 12:28:02 +03:00
Aliaksandr Valialkin	5ea689d61b	app/vmselect/promql: add `quantile("phiLabel", phi1, ..., phiN, q)` aggregate function to MetricsQL See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573	2021-08-27 18:37:20 +03:00
Aliaksandr Valialkin	bec18e4fe9	app/vmselect: add `-search.disableAutoCacheReset` command-line option for disabling automatic cache reset when a sample with old timestamp outside -search.cacheTimestampOffset is inserted See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1570	2021-08-27 17:15:31 +03:00
Aliaksandr Valialkin	67189be1cb	docs/{vmgateway,vmbackupmanager}: mention that enterprise binaries are free for download and evaluation	2021-08-27 14:54:09 +03:00
Aliaksandr Valialkin	2bef940add	docs/vmagent.md: document the ability to load scrape configs from multiple files See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1559	2021-08-26 09:13:14 +03:00
benclive	48f33d098b	Remove trailing slash for URLPrefixes with specific path (#1554 )	2021-08-25 13:28:50 +03:00
Aliaksandr Valialkin	6c5760db9c	app/vmselect/promql: `make fmt` after `0078486ea7`	2021-08-23 23:06:00 +03:00
Aliaksandr Valialkin	0078486ea7	app/vmselect/promql: rename `sign()` function to `sgn()` in order to be consistent with Prometheus See https://github.com/prometheus/prometheus/pull/8457 for details.	2021-08-23 11:45:51 +03:00
Aliaksandr Valialkin	2288e75f03	docs/vmalert.md: run `make docs-sync` after `9ee3d0378f`	2021-08-21 20:24:56 +03:00
Roman Khavronenko	9ee3d0378f	vmalert: add flag `disableAlertgroupLabel` for disabling extra label added to series (#1534 ) The new label added in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/611 may negatively impact deduplication in Alertmanager. The new flag supposed to give an option to disable adding this label. To enable flag just add `-disableAlertgroupLabel` to binary execution command. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1532	2021-08-21 20:08:55 +03:00
Aliaksandr Valialkin	4f3a5742eb	app/vmselect/prometheus: do not extend `[d]` to the detected interval between samples for `first_over_time(m[d])` This is for the sake of consistency with similar change for the last_over_time(m[d]) at `a724229b5d`	2021-08-21 19:56:14 +03:00
Alexander Rickardsson	f4cecaf296	vmalert: accept http.StatusOK for remotewrite (#1550 )	2021-08-20 11:58:32 +03:00
Roman Khavronenko	b5d6a0e499	vmselect: update `vm_request_duration_seconds` value when request fails (#1537 ) Before, metric `vm_request_duration_seconds` was update only on successful attempts which could be misleading. For example, timeout errors on netstorage request may be not accounted in the metric and won't be visible on dashboards. Using `defer` statement to update the metric after query arguments validation may improve the situation.	2021-08-19 13:58:54 +03:00
Aliaksandr Valialkin	a724229b5d	app/vmselect/promql: do not override `[d]` at `last_over_time(m[d])` if `[d]` is smaller than `scrape_interval` Since most users do not expect the overriding of explicitly set `[d]`.	2021-08-19 10:31:48 +03:00
Aliaksandr Valialkin	cdc372bb98	app/vmselect: add `-search.noStaleMarkers` command-line flag for disabling stale markers handling in queries This option allows reducing CPU usage a bit when VictoriaMetrics is used for collecting and processing non-Prometheus data. For example, InfluxDB line protocol, Graphite, OpenTSDB, CSV, etc.	2021-08-18 13:59:02 +03:00
Aliaksandr Valialkin	226143f31b	lib/promscrape: add ability to disable sending Prometheus staleness markers with -promscrape.disableStaleMarkers command-line flag This option can be useful when vmagent consumes too much additional memory for staleness markers functionality and when staleness markers aren't needed.	2021-08-18 13:43:21 +03:00
Aliaksandr Valialkin	db1e62495b	app/vmselect/promql: add bitmap_and(), bitmap_or() and bitmap_xor() functions to MetricsQL Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1541	2021-08-17 13:21:21 +03:00
Aliaksandr Valialkin	bd14b0887e	app/vmselect/promql: move common condition to dropStaleNaNs in order to improve code maintainability	2021-08-17 11:01:16 +03:00
Aliaksandr Valialkin	90434ba25b	app/vmalert: mention -remoteWrite.disablePathAppend in the description for -remoteWrite.url	2021-08-16 15:22:47 +03:00
Aliaksandr Valialkin	f37b963619	app/vmalert: follow-up for `2400f85761`	2021-08-16 15:20:22 +03:00
Alexander Rickardsson	2400f85761	vmalert: enable configuring explicit path (#1536 ) * vmalert: allow to disable automatically added path to remote write address via disablePathAppend flag * docs: update docs to include remoteWrite.disablePathAppend	2021-08-16 14:20:57 +03:00
Aliaksandr Valialkin	aefba16d5e	app/vmagent/remotewrite: expose vmagent_remotewrite_send_duration_seconds_total metric This metric can be used for determining high saturation of every connection to remote storage with an alerting query `rate(vmagent_remotewrite_send_duration_seconds_total) > 0.9s`. This query triggers when a connection is satureated by more than 90%	2021-08-15 13:34:12 +03:00
Aliaksandr Valialkin	113f0a8a07	app/vmselect/promql: drop staleness marks before calling rollupConfig.Do This allows dropping staleness marks only once and then calculate multiple rollup functions on the result. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526	2021-08-15 13:21:10 +03:00
Aliaksandr Valialkin	25997a70f1	Revert "app/vmselect/promql: properly handle Prometheus staleness marks in removeCounterResets functions" This reverts commit 94dfcb6747a3b29a11d14e71bea21a2312bb6346. It is better to remove staleness marks (decimal.StaleNaN) before calling rollupConfig.Do, e.g. in preFunc	2021-08-15 13:19:16 +03:00
Aliaksandr Valialkin	73d7b568da	app/vmselect/promql: properly handle Prometheus staleness marks in removeCounterResets functions Prometheus stalenss marks shouldn't be changed in removeCounterResets. Otherwise they will be converted to an ordinary NaN values, which couldn't be removed in dropStaleNaNs() function later. This may result in incorrect calculations for rollup functions. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526	2021-08-14 12:45:57 +03:00
Aliaksandr Valialkin	2ae2c1dd09	app/victoria-metrics/testdata: fix tests after `4401464c22`	2021-08-13 12:21:54 +03:00
Aliaksandr Valialkin	4401464c22	all: add support for Prometheus staleness markers Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845	2021-08-13 12:10:17 +03:00
Aliaksandr Valialkin	95dd5a48bb	app/vmselect: `make vmui-update` after the commit 4ae14df864a7e327955f44941295a286175423b3	2021-08-11 13:41:41 +03:00
Aliaksandr Valialkin	860b272a95	app/vmui: actualize Dockerfiles	2021-08-11 13:41:41 +03:00
Aliaksandr Valialkin	d375d9b878	lib/envflag: add a link to docs for -envflag.enable	2021-08-11 10:29:33 +03:00
Yury Molodov	236fc7d739	vmui: fix layout and add server url by default (#1519 ) * fix: change layout for correctly display big query * fix: set default server from url * fix: change get default server url	2021-08-06 12:06:08 +03:00
Aliaksandr Valialkin	c1185363ca	app/vmagent: typo fix in the description for `-remoteWrite.queues`	2021-08-05 10:01:35 +03:00
Roman Khavronenko	7416fdaa8b	vmalert: expose new metrics for tracking number of produced samples during last evaluation (#1518 ) * vmalert: expose new metrics for tracking number of produced samples during last evaluation Two new metrics were added to track the number of samples produced during the last evaluation: * vmalert_recording_rules_last_evaluation_samples * vmalert_alerting_rules_last_evaluation_samples The gauge type is used to remain consistent with Prometheus metric `prometheus_rule_group_last_evaluation_samples` which is on the group level. However, the counter type was considered as well. Two metrics instead of one are used to make it easier to separate recording and alerting rules. It is likely, number of samples produced by recording rules is more important so people will refer to it more frequently. The expected usage of the new metric is the following: ``` - alert: RecordingRuleReturnsEmptyResults expr: sum(vmalert_recording_rules_last_evaluation_samples) by(recording) < 1 annotations: summary: Recording rule {{$labels.recording}} returns empty results. Please verify expression correctness. ``` Addresses https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1494 * vmalert: rename `vmalert_alerts_error` to `vmalert_alerting_rules_error` to remain consistent with recording rules metrics	2021-08-05 09:59:46 +03:00
Aliaksandr Valialkin	d826352688	app/vmagent: follow-up after `fe445f753b` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1491	2021-08-05 09:52:32 +03:00
Omar Ghader	46e27d60a6	feature: Add multitenant for vmagent (#1505 ) * feature: Add multitenant for vmagent * Minor fix * Fix rcs index out of range * Minor fix * Fix multi Init * Fix multi Init * Fix multi Init * Add default multi * Adjust naming * Add TenantInserted metrics * Add TenantInserted metrics * fix: remove unused metrics for vmagent * fix: remove unused metrics for vmagent Co-authored-by: mghader <marc.ghader@ubisoft.com> Co-authored-by: Sebastian YEPES <syepes@gmail.com>	2021-08-05 09:52:31 +03:00
Aliaksandr Valialkin	e92fde7945	app/vmselect/promql: add `present_over_time(m[d])` function, which will be available starting from Prometheus 2.29.0 See https://github.com/prometheus/prometheus/releases/tag/v2.29.0-rc.0 and https://github.com/prometheus/prometheus/pull/9097	2021-08-03 16:11:49 +03:00
Qifei Wan	fa9c5c5940	app/vmalert: update config state metrics if config parsed failed (#1507 )	2021-08-03 12:55:29 +03:00
wusphinx	c1ed7b77aa	Update TimeSelector.tsx (#1515 ) delete garbled code	2021-08-03 10:01:01 +03:00
Nikolay	7bbff7fb86	adds /rules and /alerts api for grafana (#1504 ) Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-08-02 17:28:09 +03:00
Aliaksandr Valialkin	a1911e1330	app/vmselect/netstorage: unpack time series data in mostly local big chunks This should improve performance on multi-CPU systems for queries selecting time series with big number of raw samples	2021-07-30 12:03:17 +03:00
Aliaksandr Valialkin	d05cac6c98	li/storage: re-use the per-day inverted index search code for searching in global index This allows removing a big pile of outdated code for global index search. This may help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1486	2021-07-30 10:31:37 +03:00
Aliaksandr Valialkin	74ffaa45d9	app/vmselect/netstorage: do not query Go maps with unsafe string keys, since this breaks in Go 1.17	2021-07-30 09:57:53 +03:00
Aliaksandr Valialkin	192dfbfd90	app/vmselect: follow-up for `ed95bc9531` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1493	2021-07-29 09:53:28 +03:00
arnoldyahad	00af4ff5a4	Add case prometheus/rules for grafana 8 (#1502 )	2021-07-29 09:53:27 +03:00
assassins	a483044557	Performance optimization (#1481 ) There are redundant steps	2021-07-28 19:26:20 +03:00
Aliaksandr Valialkin	8ee8660ac4	app/vmselect: follow-up for `626073bca8` * Rename -search.maxMetricsPointSearch to -search.maxSamplesPerQuery, so it is more consistent with the existing -search.maxSamplesPerSeries * Move the -search.maxSamplesPerQuery from vmstorage to vmselect, so it could effectively limit the number of raw samples obtained from all the vmstorage nodes * Document the -search.maxSamplesPerQuery in docs/CHANGELOG.md	2021-07-28 18:00:23 +03:00
Aliaksandr Valialkin	8b7917cd81	all: add `go:build` lines for Go1.17 See https://tip.golang.org/doc/go1.17#gofmt for more details	2021-07-26 15:48:21 +03:00
Aliaksandr Valialkin	95aff47330	app/vmselect: prevent from possible deadlock when f callback blocks inside RunParallel	2021-07-26 15:47:30 +03:00
Aliaksandr Valialkin	fcaf152480	app/vmselect: `make vmui-update` after `a91d41f12a`	2021-07-26 10:31:11 +03:00
Yury Molodov	a91d41f12a	Vmui/query editor (#1472 ) * fix: move request button to server input * feat: add switch for query autocomplete * refactor: rename state for popover open * feat: add detect os by userAgent * fix: change hotkey to run query for mac * fix: change detect mac os * fix: change div to span inside Typography Co-authored-by: yury <yurymolodov@victoriametrics.com>	2021-07-23 12:00:44 +03:00
Aliaksandr Valialkin	05672ffc32	app/vmselect/promql: properly handle `(a op b) default N` if `(a op b)` returns NaN series The result should be a series with `N` values and `a op b` labels. Previously such series has been removed from the result.	2021-07-16 01:44:58 +03:00
Aliaksandr Valialkin	ed10141ff8	app/vmselect/netstorage: use more scalable algorithm for ditributing the work among among multiple channels on systems with big number of CPU cores	2021-07-16 00:35:23 +03:00
Aliaksandr Valialkin	ca75432e66	app/vmselect: do not track queries with less than 1ms execution time at /api/v1/status/top_queries This should improve the readability and usefullness of the /api/v1/status/top_queries when debugging slow queries or queries that take too much cpu time.	2021-07-15 16:44:28 +03:00
Aliaksandr Valialkin	f4e81aef7e	app/vmselect/netstorage: add `-search.maxSamplesPerSeries` command-line option for limiting the number of samples a query can process per each series This should prevent from out of memory crashes like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1067	2021-07-15 16:03:28 +03:00
Aliaksandr Valialkin	e6ef97a5ee	app/vmselect/netstorage: improve scalability of series unpacking on multi-CPU systems	2021-07-15 15:41:58 +03:00
Aliaksandr Valialkin	f81d972581	app/vmui/README.md: typo fix: naviate->navigate	2021-07-15 15:02:04 +03:00
Aliaksandr Valialkin	a925d5a3e1	app/vmselect/promql: duration handling improvements in MetricsQL queries - Support durations anywhere in MetricsQL queries. E.g. sum_over_time(m[1h])/1h is equivalent to sum_over_time(m[1h])/3600 - Support durations without suffix. E.g. rate(m[300]) is equivalent to rate(m[5m])	2021-07-12 17:16:41 +03:00
Aliaksandr Valialkin	bfba4c28a4	app/vmalert: accept Prometheus-like durations in `interval` config option inside `group` section	2021-07-12 12:35:17 +03:00
Aliaksandr Valialkin	8c764e88f0	app/vmui: move source code from https://github.com/VictoriaMetrics/vmui to app/vmui Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413	2021-07-09 17:15:23 +03:00
Aliaksandr Valialkin	c5f0b454f0	app/vmselect: follow-up after `aa11ef6d3b` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413	2021-07-07 17:43:35 +03:00
tony	e9e35a7d6a	add vmui for vmselect component (#1431 ) Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-07-07 17:33:02 +03:00
Aliaksandr Valialkin	ed944313b0	app/{vminsert,vmselect}: export vminsert_request_duration_seconds and vmselect_request_duration_seconds histograms	2021-07-07 13:25:21 +03:00
Aliaksandr Valialkin	766edbc421	lib/httpserver: print full requestURI in httpserver.Errorf This should simplify debugging.	2021-07-07 13:09:40 +03:00
Aliaksandr Valialkin	f3acf065c9	lib/storage: consistency renaming: tagCache -> tagFiltersCache This improves code readability	2021-07-06 11:03:51 +03:00
Aliaksandr Valialkin	f6bb130898	app/{vmselect,vmstorage}: clarify the description for `-dedup.minScrapeInterval` command-line flag Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1426	2021-07-02 15:05:57 +03:00
Aliaksandr Valialkin	3af2162085	docs/vmagent.md: mention about docker_sd_config support	2021-06-25 20:52:15 +03:00
Aliaksandr Valialkin	cdfae0117a	app/vmselect/promql: return the last timestamp for the max / min value from `tmax_over_time()` and `tmin_over_time()` function as most users expect	2021-06-23 14:19:00 +03:00
Nikolay	e307bbb29a	adds http_sd (#1399 ) * adds http_sd * adds X-Prometheus-Refresh-Interval-Seconds header * Update lib/promscrape/discovery/http/api.go Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-06-22 13:33:37 +03:00
Roman Khavronenko	da8c901fab	vmctl: add more context to flags description in vm-native mode (#1395 )	2021-06-18 19:20:01 +03:00
Aliaksandr Valialkin	83a4db813e	app/vmselect: log slow requests to all the `/api/v1/*` handlers if their execution time exceeds `-search.logSlowQueryDuration`	2021-06-18 19:04:42 +03:00
Aliaksandr Valialkin	570f36b344	app/vmctl: limit JSON line size by 10K samples (#1394 ) This should reduce the maximum memory usage at VictoriaMetrics when importing time series with big number of samples.	2021-06-18 15:26:47 +03:00
Aliaksandr Valialkin	d8ab409418	docs/{vmgateway,vmbackupmanager}: explicitly mention that these components are a part of an enterprise package	2021-06-17 17:19:49 +03:00
Aliaksandr Valialkin	12a83d25bf	app/vmagent/remotewrite: `go fmt` after `0a796f7c3a`	2021-06-17 13:52:06 +03:00
Zongyang	0a796f7c3a	Change default value of '-remoteWrite.queues' to cgroup.AvailableCPUs * 2 (#1385 ) * Change default value of '-remoteWrite.queues' to cgroup.AvailableCPUS() * 2 to reduce scrape interval Default value of vmagent option '-remotewrite.queues' is 4 and default size of vmagent ScheudleUnmarshalWorkers is number of CPUs, when available CPUs is much greater than 4, e.g 32, worker are competing push queues which will increase scrape interval and may cause scrape timeout. * Update README and flag description Co-authored-by: xiaozy <xiaozy01@fenbi.com>	2021-06-16 12:16:44 +03:00
Roman Khavronenko	fb4f758715	promql: fix `increase_pure` calculation for cases with stale series (#1381 ) Due to staleness handling, increase_pure were using incorrect previous value during calculation in cases where series disappears for period longer than staleness period and then returns back. The fix suppose to account for a real datapoint value before staleness takes place. The fix should remove unexpected spikes while using `increase_pure` for staled series.	2021-06-15 17:37:19 +03:00
Nikolay	729c4eeb9c	adds digital ocean sd (#1376 ) * adds digital ocean sd config * adds digital ocean sd https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1367 * typo fix	2021-06-14 13:15:04 +03:00
Roman Khavronenko	6d5a8c28cd	Vmalert docs (#1372 ) * vmalert: mention what happens if `for` is set to 0 or omitted * vmalert: add more context to docs	2021-06-11 13:25:53 +03:00
Aliaksandr Valialkin	ed83558646	app/vmauth: properly handle http.ErrAbortHandler panic This panic can be raised by the reverseProxy on aborted request to the backend. So handle it (e.g. suppress) at reverseProxy.ServeHTTP call. Do not suppress the panic at lib/httpserver generic HTTP handler, since it may result in an inconsistent state left after the panicking handler. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1353	2021-06-11 12:50:25 +03:00
Roman Khavronenko	7adfe878e1	vmalert: fix mistake with object reuse while parsing response (#1370 ) * vmalert: fix mistake with object reuse while parsing response During the refactoring, the wrong optimisations was applied in parse function which caused metric fields reset. The change removes optimisation. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1369 * vmalert: add test to cover multiple metrics in one response	2021-06-11 11:22:05 +03:00
John Belmonte	67b17cdd68	spelling fix: synonym (#1363 )	2021-06-10 08:32:52 +03:00
Aliaksandr Valialkin	d0dca62026	app/vmselect/promql: typo fix in the comment	2021-06-09 18:34:31 +03:00
Aliaksandr Valialkin	2422b5091f	app/vmauth: improve readability for a config with multiple `src_paths`	2021-06-09 15:39:32 +03:00
Aliaksandr Valialkin	f93a31b490	docs/vmagent.md: mention that vmagent supports scrape targets sharding	2021-06-09 12:29:34 +03:00
Aliaksandr Valialkin	ab15bf8c90	docs: document rules replay feature for vmalert Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/836 This is a follow-up for `2a259ef5e7`	2021-06-09 12:27:34 +03:00
Roman Khavronenko	2a259ef5e7	vmalert: support rules backfilling (aka `replay`) (#1358 ) * vmalert: support rules backfilling (aka `replay`) vmalert can `replay` configured rules in the past and backfill results via remote write protocol. It supports MetricsQL/PromQL storage as data source, and can backfill data to remote write compatible storage. Supports recording and alerting rules `replay`. See more details in README. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/836 * vmalert: review fixes * vmalert: readme fixes	2021-06-09 12:20:38 +03:00
Aliaksandr Valialkin	78f83dc5ad	app/{vmagent,vminsert}: follow-up after `2fe045e2a4` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1343	2021-06-04 20:27:58 +03:00
jelmd	2fe045e2a4	new feature: debug relabeling (#1344 ) * new feature: relabel logging Use scrape_configs[x].relabel_debug = true to log metric names inkl. labels before and after relabeling. After relabeling related metrics get dropped, i.e. not submitted to servers. * vminsert wants relabel logging, too.	2021-06-04 17:50:23 +03:00
Aliaksandr Valialkin	b22e380a34	app/vmauth: allow balancing the load among multiple backend nodes by specifying multiple urls in `url_prefix` config	2021-05-29 01:03:37 +03:00
Aliaksandr Valialkin	60341722d5	docs: document `f0c21b6300`	2021-05-27 15:03:30 +03:00
Roman Khavronenko	d210958fd0	vmalert: automatically reload configuration on file change (#1326 ) New flag `-rule.configCheckInterval` defines how often `vmalert` will re-read config file. If it detects any changes, the config will be reloaded. This behaviour is turned off by default. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/512	2021-05-25 14:27:22 +01:00
Aliaksandr Valialkin	8e7d1f8824	app/vmagent/remotewrite: use WARN level instead of ERROR level for `couldnt send a block with size ... bytes to ...` log message This is really warning, since vmagent re-tries sending the data block until success.	2021-05-24 15:43:59 +03:00
Roman Khavronenko	84cc0513e1	vmalert: support `extra_filter_labels` setting per-group (#1319 ) The new setting `extra_filter_labels` may be assigned to group. If it is, then all rules within a group will automatically filter for configured labels. The feature is well-described here https://docs.victoriametrics.com#prometheus-querying-api-enhancements New setting is compatible only with VM datasource.	2021-05-23 00:26:01 +03:00
Aliaksandr Valialkin	78dddfb98f	lib/promauth: follow-up after `5b8176c68e`	2021-05-22 18:01:11 +03:00
Nikolay	5b8176c68e	basic OAuth2 support for remoteWrite and scrape targets (#1316 ) * adds OAuth2 support for remoteWrite and scrapping * adds tests changes init	2021-05-22 16:20:18 +03:00
Aliaksandr Valialkin	c54bb73867	all: do not skip SIGHUP signal during service initialization This can lead to stale or incomplete configs like in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240	2021-05-21 16:34:06 +03:00
Nikolay	d626c5c2a9	changes vmalert query function (#1307 ) * changes vmalert query function for prometheus rules compatibility its better to use labels as map. it simplifies template evaluation and allow to ignore can't evaluate field error because map will return default value. fixes https://github.com/VictoriaMetrics/operator/issues/243	2021-05-21 13:55:43 +03:00
Aliaksandr Valialkin	49ecbc765d	app/vmauth: add ability to protect `/-/reload` endpoint with authKey	2021-05-20 18:47:01 +03:00
Aliaksandr Valialkin	4c7bb75fa2	Makefile: update golangci-lint from v1.29.0 to v1.40.1	2021-05-20 18:27:10 +03:00
Aliaksandr Valialkin	e394ff6466	app/vmagent/remotewrite: expose metrics with the current number of active series per day and per hour These numbers are exposed via the following metrics: - vmagent_hourly_series_limit_current_series - vmagent_daily_series_limit_current_series Expose also the limits via the following metrics: - vmagent_hourly_series_limit_max_series - vmagent_daily_series_limit_max_series	2021-05-20 15:28:09 +03:00
Aliaksandr Valialkin	ad73f226ff	app/vmstorage: add ability to limit series cardinality via `-storage.maxHourlySeries` and `-storage.maxDailySeries` command-line flags	2021-05-20 14:15:19 +03:00
Aliaksandr Valialkin	7e526effaa	app/vmagent: add ability to limit series cardinality on a per-hour and per-day basis	2021-05-20 13:13:40 +03:00
Roman Khavronenko	dcf8803bbd	vmctl: explicitly set `::tag` type for labels selector in `influx` mode (#1310 ) The `::tag` type is needed in cases when field and tag names are equal, which results into unexpected results in InfluxQL. Setting the type explicitly helps InfluxDB to understand which exact column we apply filter to. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1299	2021-05-20 12:03:16 +03:00
Aliaksandr Valialkin	0842bb9294	app/vmselect/promql: add `timezone_offset(tz)` function Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1306	2021-05-20 11:53:09 +03:00
Aliaksandr Valialkin	f7e73fbe8b	app/vmagent/remotewrite: sort labels before sending the series to per-remoteWrite.url queues	2021-05-20 02:12:36 +03:00
Neo He	e8a6c6927d	app/{vmbackup,vmrestore},docs/vmrestore.md: typo fix: vbackup -> vmbackup (#1305 )	2021-05-18 16:37:28 +03:00
Aliaksandr Valialkin	f4719889da	lib/httpserver: typo fix in `-http.shutdownDelay` command-line flag description: servier -> server	2021-05-18 16:26:16 +03:00
Aliaksandr Valialkin	b30925738b	docs/vmalert.md: document multitenant support https://github.com/VictoriaMetrics/VictoriaMetrics/issues/740	2021-05-18 16:26:14 +03:00
Aliaksandr Valialkin	66aba00549	app/vmauth: reload `-auth.config` on the request to `/-/reload` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1194	2021-05-18 02:23:55 +03:00
Aliaksandr Valialkin	3339ea41e7	docs/vmbackup.md: typo fix: snaphosts -> snapshots Thanks to @jelmd - see `1ab27582a3 (r50884395)`	2021-05-18 01:12:36 +03:00
Aliaksandr Valialkin	6c944b86d8	docs: dealay -> delay Thanks to @jelmd . See `0b7e3510c8 (r50884991)`	2021-05-18 01:07:52 +03:00
Roman Khavronenko	b38edec7ee	vmalert: use stringified label keys for duplicates map in recroding rules (#1301 ) duplicates map helps to determine wheter extra labels has overriden labels which make time series unique. It was using a sorted hashed labels sequence as a key. But hashing algorithm could have collisions, so it is more convenient to not use hashing at all. Log message for recording rules duplicates was improved as well. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1293	2021-05-15 13:25:57 +03:00
Aliaksandr Valialkin	10a47af631	app/{vmalert,vmauth}: explicitly set MaxIdleConnsPerHost in net/http.Client.Transport By default MaxIdleConnsPerHost is set to 2. This limits the possibility to re-use http keep-alive connections. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1300	2021-05-14 18:12:24 +03:00
Aliaksandr Valialkin	a422165dc6	app/vmagent/remotewrite: clarify the comment explaining why vmagent drops blocks if remote storage returns 400 or 409 status code	2021-05-13 16:16:16 +03:00
Aliaksandr Valialkin	d7be2753c0	lib/storage: substitute GetTSDBStatusForDate with GetTSDBStatusWithFiltersForDate with nil tfss	2021-05-13 09:02:33 +03:00
Aliaksandr Valialkin	832651c6c2	app/vmselect: follow up after `8a0678678b` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1168	2021-05-12 17:18:30 +03:00
Nikolay	8a0678678b	Adds tsdb match filters (#1282 ) * init work on filters * init propose for status filters * fixes tsdb status adds test * fix bug * removes checks from test	2021-05-12 15:18:45 +03:00
Aliaksandr Valialkin	229d9d6dd7	docs/CHANGELOG.md: document `-datasource.roundDigits` added at `5c448126dc`	2021-05-10 11:18:26 +03:00
Roman Khavronenko	5c448126dc	vmalert: add support for `round_digits` param in datasource package (#1278 ) Starting from v1.56.0 VM supports `round_digits` which allows to limit the number of digits after the decimal point in response value. The feature can be used to reduce entropy of produced by recording rules values and significantly improve the compression. See more details in link below. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/525	2021-05-10 11:11:45 +03:00
Roman Khavronenko	4247168a2d	vmalert: fix error when rule didn't start if restore failed (#1279 ) Previously, `startGroup` could exit on restore errors despite the `remoteRead.ignoreRestoreErrors` flag value. Now vmalert checks the flag value before deciding whether to return error or just log it.	2021-05-10 11:06:31 +03:00
Aliaksandr Valialkin	6ff19096be	docs/vmagent.md: add `stream parsing mode` chapter	2021-05-08 23:14:07 +03:00
Aliaksandr Valialkin	12d733dd5d	app/vminsert: add support for data ingestion via other vminsert nodes	2021-05-08 19:52:57 +03:00
Aliaksandr Valialkin	237e9f9fd7	app/vmalert: add missing comment for ErrStateRestore	2021-05-08 15:59:35 +03:00
Aliaksandr Valialkin	79ec35cef9	app/vmbackup: make sure that `-snapshotName` isnt set if `-snapshot.createURL` is set	2021-05-07 08:44:25 +03:00
Roman Khavronenko	9cdd4696fe	vmalert: add flag to control behaviour on startup for state restore errors (#1265 ) Alerting rules now can return specific error type ErrStateRestore to indicate whether restore state procedure failed. Such errors were returned and logged before as well. But now user can specify whether to just log these errors (remoteRead.ignoreRestoreErrors=true) or to stop the process (remoteRead.ignoreRestoreErrors=false). The latter is important when VM isn't ready yet to serve queries from vmalert and it needs to wait. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1252	2021-05-05 10:07:19 +03:00
Aliaksandr Valialkin	508f500477	docs/vmalert.md: update docs after `afca7b430c`	2021-04-30 11:49:01 +03:00
Roman Khavronenko	0f988e5a31	vmalert: fix the typo in ApplyParams func (#1259 )	2021-04-30 10:31:45 +03:00
Roman Khavronenko	afca7b430c	vmalert: use rule's `evaluationInterval` as `step` param by default (#1258 ) User still can override param by specifying `datasource.queryStep` flag.	2021-04-30 10:01:05 +03:00
Aliaksandr Valialkin	4394dc6cbb	docs/CHANGELOG.md: document the change from `f3a048288e` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1232	2021-04-30 09:54:41 +03:00
Roman Khavronenko	f3a048288e	Vmalert: adjust `time` param for datasource queries according to `evaluationInterval` (#1257 ) * Simplify arguments list for fn `queryDataSource` to improve readbility * vmalert: adjust `time` param according to rule evaluation interval With this change, vmalert will start to use rule's evaluation interval for truncating the `time` param. This is mostly needed to produce consistent time series with timestamps unaffected by vmalert start time. Now, timestamp becomes predictable. Additionally, adjustment is similar to what Grafana does for plotting range graphs. Hence, recording rule series and recording rule expression plotted in grafana suppose to become similar in most of cases.	2021-04-30 09:46:03 +03:00
Aliaksandr Valialkin	6fa5981e68	app/vmagent: list user-visible endpoints at `http://vmagent:8429/` While at it, use common WriteAPIHelp function for the listing in vmagent, vmalert and victoria-metrics	2021-04-30 09:36:43 +03:00
Nikolay	15609ee447	changes vmalert Querier with per rule querier (#1249 ) * changes vmalert Querier with per rule querier it allows to changes some parametrs based on rule setting for instance - alert type, tenant for cluster version or event endpoint url.	2021-04-28 21:41:15 +01:00
Roman Khavronenko	87018650dd	vmalert: keep the returned timestamp when persisting recording rule (#1245 ) Previously, vmalert used `lastExecTime` timestamp when writing recording rules to the remote storage. This may be incorrect, if vmalert uses `datasource.lookback` flag, which means rule's expression will be executed at some moment in the past. To avoid such situations, vmalert now will use returned timestamp instead of `lastExecTime`.	2021-04-26 01:03:22 +03:00
Aliaksandr Valialkin	52b4b1605e	app/vmagent/remotewrite: increase the maximum possible number of inmemory blocks for systems with high amounts of RAM This should reduce the probability of using much slower file-based persistent queue when vmagent processes metrics at high rate (millions of metrics per second). Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1235	2021-04-23 22:03:51 +03:00
Aliaksandr Valialkin	fb37b853e9	app/vmagent/remotewrite: count maxLabelsPerBlock as 10x of maxRowsPerBlock This should increase block sizes and subsequently increase the maximum possible bandwidth per each connection to remote storage. This, in turn, should reduce the probability of storing the data in local buffers. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1235	2021-04-23 21:55:47 +03:00
Aliaksandr Valialkin	1fbc04facd	app/vmbackup: typo fix: snaphsot -> snapshot Follow-up for `9de0fa3649`	2021-04-22 11:16:56 +03:00
Aliaksandr Valialkin	7c4e460513	app/vmauth: parse `url_prefix` only once during config load	2021-04-21 10:55:29 +03:00
Aliaksandr Valialkin	6bc52fe41a	all: rename https://victoriametrics.github.io to https://docs.victoriametrics.com	2021-04-20 20:16:17 +03:00
Aliaksandr Valialkin	5720b283fa	all: consistency renaming `Victoria Metrics` -> `VictoriaMetrics` VMInsert -> vminsert VMSelect -> vmselect VMStorage -> vmstorage	2021-04-20 11:45:48 +03:00
Aliaksandr Valialkin	187e3ec909	app/vmauth: follow-up for `6a81a89b3d`	2021-04-20 10:58:29 +03:00
Nikolay	6a81a89b3d	adds query params support for vmauth urlPrefix (#1226 ) * adds query params support for vmauth urlPrefix * Update app/vmauth/example_config.yml * Update app/vmauth/example_config.yml Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-04-20 10:51:03 +03:00
Aliaksandr Valialkin	b3d88610fd	app/vmctl: update README.md according to `bfecd0fd55`	2021-04-16 12:09:45 +03:00
Aliaksandr Valialkin	663a91bb82	docs: update `-help` output after the commit `77be3e3a82`	2021-04-12 12:34:59 +03:00
Artem Navoiev	77be3e3a82	improve docs for cli flags (#1202 ) * improve docs for cli flags * improve docs for cli flags.2	2021-04-12 12:28:04 +03:00
Aliaksandr Valialkin	0b7e3510c8	docs: `make docs-sync`	2021-04-10 19:54:18 +03:00
Roman Khavronenko	7644efae01	Docs update (#1199 ) * docs: drop table of contents for `vmctl` We already have it autogenerated on .github.io, so no need to keep it. * docs: mention OpenTSDB migration feature for vmctl * docs: sync docs for `vmalert`	2021-04-10 15:46:08 +03:00
John Seekins	a9d76b06a7	Improve documentation on OpenTSDB migration tool and fix a bug with hard offsets (#1198 ) * add more documentation on OpenTSDB migration explaining what chunking means * more clarification of OpenTSDB aggregations * break out what a retention string becomes * add more docs around retention strings * add example of running program and fix mistake in how hard offsets are handled * fix formatting	2021-04-10 13:24:18 +01:00
John Seekins	cd9786c2a7	OpenTSDB migration to VictoriaMetrics (#1089 )	2021-04-08 20:58:06 +01:00
Roman Khavronenko	4ed8de62ac	vmalert: document template functions and mention them in README (#1197 )	2021-04-08 18:19:08 +03:00
Aliaksandr Valialkin	544821b719	app/vmselect/promql: fix tests after `d3fa0ccabd`	2021-04-08 00:18:01 +03:00
Aliaksandr Valialkin	d3fa0ccabd	app/vmselect/promql: properly detect aggregate `topk` and `bottomk` aggregate functions in order to disable duplicate sorting Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1189	2021-04-08 00:09:40 +03:00
Aliaksandr Valialkin	cb12a8f0a8	app/vmselect: return `data:null` instead of `data:[]` from `/api/v1/query_exemplars`, since Grafana throws an error otherwise Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1186	2021-04-07 23:34:06 +03:00
Aliaksandr Valialkin	1177dca3da	app/vmselect: do not sort series returned from `topk` and `bottomk` functions, since these series are already sorted in user-expected order Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1189	2021-04-07 14:16:08 +03:00
Roman Khavronenko	c6a8ebb11f	docs: update docs ordering and formatting (#1192 ) The major change is adding `sort` directive to docs. For those docs which are copied from internal packages `sort` is added via makefile command. For the rest it is added manually since they're updated manually as well. The rest of changes is connected with markdown formatting. For example, changing headers in some files (`##` => `#`) makes navigation on .github.io to look better. This especially useful for `changelog` docs. Table of contents for `vmctl` is dropped, since we already have it autogenerated on .github.io. No link changes expected. The corresponding PR to `cluster` branch will be made in follow-up PR.	2021-04-07 13:39:16 +03:00
Aliaksandr Valialkin	7d23598b33	app/vmselect: return dumb response on `/api/v1/query_exemplars` request Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1186	2021-04-05 23:25:08 +03:00
Aliaksandr Valialkin	9a97941e2a	docs/vmagent.md: mention that vmagent supports scraping via socks5 proxy	2021-04-04 01:45:52 +03:00
Aliaksandr Valialkin	5153410ced	lib/promscrape: support for simple HTTP proxies without `CONNECT` method support such as https://github.com/prometheus-community/PushProx See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1179	2021-04-04 00:40:40 +03:00
Aliaksandr Valialkin	7a0b964e8d	app/vmselect/promql: do not delete `dst_label` if `src_label` is empty in `label_copy(q, src_label, dst_label)` and `label_move(q, src_label, dst_label)`	2021-04-03 22:05:06 +03:00
Aliaksandr Valialkin	2256b79a89	docs/vmgateway.md: update docs	2021-04-03 00:29:19 +03:00
Aliaksandr Valialkin	3055ab0115	app/vmselect/promql: add ability to set label value additionally to label name for the remaining sum of time series returned from `topk_` and `bottomk_` functions in the form: `topk_min(N, m, "label=value")`	2021-04-02 23:55:54 +03:00
Aliaksandr Valialkin	25e19c75c7	docs/{vmauth,vmgateway}.md: small fixes	2021-04-02 23:15:00 +03:00
Aliaksandr Valialkin	3c1b39c978	app/vmgateway: publish docs	2021-04-02 23:08:41 +03:00
Aliaksandr Valialkin	0db901617d	app: do not process non-GET requests on at `/` handler	2021-04-02 22:54:06 +03:00
Aliaksandr Valialkin	b1d0028e79	app/vmauth: add support for authorization via `Authorization: Bearer <token>`	2021-04-02 22:14:53 +03:00
Aliaksandr Valialkin	b88feb631e	docs/vmagent.md: mention about proxy_authorization section	2021-04-02 21:24:11 +03:00
Aliaksandr Valialkin	df148f48b7	lib/promscrape: add support for `authorization` config in `-promscrape.config` as Prometheus 2.26 does See https://github.com/prometheus/prometheus/pull/8512	2021-04-02 21:17:45 +03:00
Aliaksandr Valialkin	c79e4a2f90	app/vmselect/promql: remove the limit on the number of time series that can be sorted, since it may confuse users Always sort time series returned from `/api/v1/query` and `/api/v1/query_range` unless `sort_*` function is used at top level of the query.	2021-04-02 15:02:08 +03:00
Aliaksandr Valialkin	dc9eafcd02	app/{vminsert,vmagent}: add `-sortLabels` command-line option for sorting time series labels before ingesting them in the storage This option can be useful when samples for the same time series are ingested with distinct order of labels. For example, metric{k1="v1",k2="v2"} and metric{k2="v2",k1="v1"}.	2021-03-31 23:27:58 +03:00
Aliaksandr Valialkin	e1f699bb6c	lib/storage: reduce memory usage when ingesting samples for the same time series with distinct order of labels	2021-03-31 21:24:46 +03:00
Aliaksandr Valialkin	48275d8c12	app/vmagent/remotewrite: reduce memory usage when `-remoteWrite.queues` is set to a big value See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1167	2021-03-31 16:16:33 +03:00
Aliaksandr Valialkin	33622c409c	app/vmagent/remotewrite: reduce memory usage when samples with big number of labels are sent to remote storage	2021-03-31 00:44:54 +03:00
Aliaksandr Valialkin	e7fdea5953	app/vmselect: add `-search.maxStatusRequestDuration` command-line flag for limiting the duration of requests to `/api/v1/status/*` and `/api/v1/series/count`	2021-03-30 21:41:35 +03:00
Aliaksandr Valialkin	7962cf1af8	app/vmselect: prevent from possible incomplete query results after timed out query Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711	2021-03-30 13:35:45 +03:00
Aliaksandr Valialkin	1db1a29ffa	all: increase minimum supported Go version for building VictoriaMetrics components from v1.14 to v1.15 This is needed after the commit `c0ac740f93`, which uses URL.Redacted() method, which has been added in v1.15. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1147	2021-03-29 23:04:53 +03:00
Aliaksandr Valialkin	78188decf9	docs: document that vmagent drops data blocks when remote storage replies with 400 and 409 http status codes This is a follow up for `1b7dc1e5a5`. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1149	2021-03-26 14:44:06 +02:00
Aliaksandr Valialkin	c54cb3e63c	app/vmagent/remotewrite: remove superflouos code after `1b7dc1e5a5`	2021-03-26 13:59:46 +02:00
Nikolay	1b7dc1e5a5	Adds blocks drop (#1151 ) * adds blocks drop at 400 BadRequest status code recieved from remote storage, not expected that remote storage will be able to handle it on retry * removes error logging for dropped blocks, its expected error	2021-03-26 14:17:59 +03:00
Aliaksandr Valialkin	aa81039b42	app/vmselect: log the metric which trigger rollup result cache reset This should help finding the source of stale metrics	2021-03-25 21:31:39 +02:00
Aliaksandr Valialkin	5ec9e49103	docs/vmagent.md: add an example for `-remoteWrite.label`	2021-03-25 17:54:55 +02:00
Aliaksandr Valialkin	f6529f932a	docs: add a link to the repository from build instruction for all the VictoriaMetrics components	2021-03-25 17:14:42 +02:00
Aliaksandr Valialkin	2065d11300	docs/vmagent.md: cosmetic fixes	2021-03-25 17:11:19 +02:00
Aliaksandr Valialkin	35fb9bdee1	docs/vmagent.md: cosmetic fixes	2021-03-25 16:54:03 +02:00
Aliaksandr Valialkin	a647144616	docs/vmagent.md: typo fix: tupically -> typically	2021-03-25 16:48:45 +02:00
Aliaksandr Valialkin	c3c3e51f17	docs/vmalert.md: remove misleading `-evaluationInterval=3s` from example config args 3s evaluation interval is too small for practical setups. It can result in increased load on datasource. So it is better to remove it from example config args, which are usually copy-pasted by novice users.	2021-03-25 15:29:06 +02:00
Aliaksandr Valialkin	0b2a66db30	app/vmselect/promql: do not merge time series during requests to `/api/v1/query` See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1141	2021-03-25 13:56:07 +02:00
Aliaksandr Valialkin	d4aadba9fa	app/vmagent: add `-promscrape.consul.waitTime` command-line flag for configuring Consul service discovery wait time See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1144	2021-03-23 19:33:25 +02:00
Aliaksandr Valialkin	9c566f7db9	app/vmagent: mention -remoteWrite.maxDiskUsagePerURL in the descriptio of `-remoteWrite.tmpDataPath` flag	2021-03-23 16:38:48 +02:00
Aliaksandr Valialkin	331a6a2015	app/vmselect/graphite: accept and enforce `extra_label` in all the Graphite APIs	2021-03-23 15:29:16 +02:00
Aliaksandr Valialkin	b521d1d4f2	app/vmselect: move getEnforcedTagFiltersFromRequest to searchtuils, since it will be used in Graphite functions soon	2021-03-23 14:16:29 +02:00
Aliaksandr Valialkin	71b72304ae	app/vmselect: improve description for `-search.maxPointsPerTimeseries` command-line flag	2021-03-22 16:45:34 +02:00
Aliaksandr Valialkin	44a6cc5eca	app/{vminsert,vmagent}: use Influx field as metric name if measurement is empty and -influxSkipSingleField command-line is set See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1139	2021-03-22 13:53:53 +02:00
Aliaksandr Valialkin	cef010d5f7	app/vmselect/promql: increment key prefix for faster reset for rollup result cache	2021-03-22 11:59:07 +02:00
Aliaksandr Valialkin	fb83e97170	app/victoria-metrics: use flag.Parse instead of envflag.Parse for avoiding possible side effects of envflag	2021-03-18 18:20:22 +02:00
Aliaksandr Valialkin	b0c956a178	app/vmselect/graphite: follow-up after `529d7be26b`	2021-03-18 16:30:20 +02:00
Nikolay	529d7be26b	changes metricsFind api (#1137 ) it should be able mitigate crash if label value contains *,[ or { symbols	2021-03-18 16:12:02 +02:00
Aliaksandr Valialkin	b1713e3fcd	app/vmselect/promql: typo fix after `9666834045`	2021-03-17 15:12:11 +02:00
Aliaksandr Valialkin	9666834045	app/vmselect/promql: merge adjancent buckets with the smallest summary number of hits in `buckets_limit()` function This should improve accuracy for the returned buckets	2021-03-17 14:31:41 +02:00
Aliaksandr Valialkin	7bc728bf53	app/vmselect: add `vm_index_search_duration_seconds` histogram for monitoring the performance of index search	2021-03-17 01:17:41 +02:00
Aliaksandr Valialkin	828669e4e1	all: make `golint` happy	2021-03-17 00:49:28 +02:00
Aliaksandr Valialkin	f104f3eb2a	all: make golangci-lint happy after the commit `6378205415`	2021-03-17 00:24:40 +02:00
Aliaksandr Valialkin	6378205415	lib/netutil: enable IPv6 UDP listening if `-enableTCP6` command-line flag is passed to VictoriaMetrics This is a follow-up for `18cfc4be7b` See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1131	2021-03-17 00:16:17 +02:00
Aliaksandr Valialkin	0ce557951f	app/vmselect/netstorage: reduce mutex contention when unpacking data on a system with high number of CPU cores	2021-03-16 21:51:31 +02:00
Aliaksandr Valialkin	aba955fa16	Makefile: prepare vmutils-windows-.zip archive on `make release-vmutils` command The archive contains the following executables for Windows: vmagent * vmalert * vmauth * vmctl Other components - vmbackup, vmrestore, victoria-metrics - aren't supported for Windows yet	2021-03-16 20:52:41 +02:00
Aliaksandr Valialkin	f0a4157f89	app/vmselect/promql: do not crash if `histogram_over_time()` function name contains uppercase letters such as `Histogram_over_time()`	2021-03-16 12:24:21 +02:00
Aliaksandr Valialkin	dd7e82c34f	app/vmstorage: add `-logNewSeries` command-line flag for determining the source of series churn rate	2021-03-15 22:38:50 +02:00
Aliaksandr Valialkin	85a95bf60c	all: various fixes in command-line flag descriptions	2021-03-15 21:59:25 +02:00
Aliaksandr Valialkin	7c37e9aea9	app/{vminsert,vmagent}: a follow-up for `b1aa8c3d8f` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1124	2021-03-15 21:37:55 +02:00
Nikolay	b1aa8c3d8f	adds fake response for telegraph queries (#1130 ) https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1124	2021-03-15 21:10:47 +02:00
Aliaksandr Valialkin	923cdb0552	app/vmselect/promql: reduce overhead on scrape interval estimation It should be enough to use the first 20 datapoints instead of 100 datapoints for scrape interval estimation.	2021-03-15 20:31:33 +02:00
Aliaksandr Valialkin	82aab87446	app/vmselect/promql: fix tests after `2dae0a2c47`	2021-03-15 20:18:59 +02:00
Aliaksandr Valialkin	2dae0a2c47	app/vmselect: add `round_digits` query arg to `/api/v1/query` and `/api/v1/query_range` handlers for limiting the number of decimal digits after the point	2021-03-15 12:36:33 +02:00
Aliaksandr Valialkin	2096c6e464	app/vmselect/prometheus: typo fix after `7c002023d7`	2021-03-12 12:19:36 +02:00
Aliaksandr Valialkin	7c002023d7	app/vmselect/prometheus: do not include datapoints with timestamps matching `t-d` when returning results from `/api/v1/query?query=m[d]&time=t` as Prometheus does	2021-03-12 12:16:50 +02:00
Aliaksandr Valialkin	a6a71ef861	lib/promscrape: add ability to configure proxy options via `proxy_tls_config`, `proxy_basic_auth`, `proxy_bearer_token` and `proxy_bearer_token_file` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1116	2021-03-12 03:36:19 +02:00
Ihor Borodin	d212f35ae8	Fixing examples of external.alert.source in documentation (#1120 ) * Fixing examples of external.alert.source in documentation	2021-03-09 20:49:50 +00:00
Aliaksandr Valialkin	fe8b12fbad	app/vmselect/promql: follow up for `433fff0006`	2021-03-09 12:48:44 +02:00
Nikolay	433fff0006	duplicate timeseries fix for prometheus_buckets function (#1119 ) * try fix for prometheus_buckets * merge possible end of the bucket collision	2021-03-09 12:26:23 +02:00
Aliaksandr Valialkin	14a399dd06	lib/promscrape: add `scrape_offset` option to `scrape_config` This option can be used for specifying the particular offset per each scrape interval for target scraping	2021-03-08 12:03:33 +02:00
Aliaksandr Valialkin	47ac2051bb	app/vmauth: allow using regexps in `url_map` paths See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1112	2021-03-05 18:21:36 +02:00
Aliaksandr Valialkin	a14053ffa0	app/vmselect/promql: add `histogram_avg()`, `histogram_stddev()` and `histogram_stdvar()` functions to MetricsQL	2021-03-04 14:12:07 +02:00
Aliaksandr Valialkin	423cd981fb	lib/promscrape: add `-promscrape.cluster.replicationFactor` command-line flag for replicating scrape targets among `vmagent` instances in the cluster	2021-03-04 10:20:15 +02:00
Aliaksandr Valialkin	201b685b13	all: bump minimum supported Go version from 1.13 to 1.14	2021-03-03 15:57:13 +02:00
Aliaksandr Valialkin	f391e5a3a0	docs/vmagent.md: remove outdated suggestion for determining labels that lead to duplicate targets The original labels for duplicate targets is already printed in the error message starting from `71ea4935de`	2021-03-03 12:26:11 +02:00
Roman Khavronenko	621bf03745	Vmagent docs upd (#1104 ) * vmagent: port changes from https://github.com/VictoriaMetrics/VictoriaMetrics.github.io/pull/1 Thanks to @dereksfoster99 for this patch! * vmagent: reword to make the meaning clear	2021-03-03 10:51:51 +02:00
Aliaksandr Valialkin	e09a245b2b	app/vmalert/README.md: sync with docs/vmalert.md	2021-03-03 10:43:59 +02:00
Aliaksandr Valialkin	baefe5a8ad	docs: actualize `-help` output	2021-03-01 17:01:27 +02:00
Aliaksandr Valialkin	4e3cfe8461	app/vmagent/remotewrite: clarify `-remoteWrite.flushInterval` flag description	2021-03-01 11:50:54 +02:00
Aliaksandr Valialkin	369f01c738	app/vmagent/remotewrite: fix rate limiting logic for `-remoteWrite.url`	2021-03-01 00:58:34 +02:00
Aliaksandr Valialkin	cb943f35c7	app/vmagent: remove data race when applying rate limits to `-remoteWrite.url` with multiple queues	2021-03-01 00:29:07 +02:00
Aliaksandr Valialkin	8d021b73b5	docs/vmbackup.md: clarify docs on `vmagent` clustering	2021-02-28 22:00:15 +02:00
Aliaksandr Valialkin	2b53add6b2	app/vmselect/querystats: show the number of matching queries in the top by average duration and in the top by summary duration This should help debugging slow queries.	2021-02-28 19:40:19 +02:00
Aliaksandr Valialkin	e5ca8ac0db	lib/promscrape: add ability to spread scrape targets among multiple vmagent instances Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1084	2021-02-28 18:41:08 +02:00
Nikolay	317b0cbed2	adds query params for vmalert (#1094 ) remoteWrite.url now accepts query params at provided url https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1087	2021-02-27 10:04:58 +00:00
Aliaksandr Valialkin	4f8a72806a	app/vmbackupmanager: add docs; the vmbackupmanager is available as a part of enterprise subscription	2021-02-27 02:17:55 +02:00
Aliaksandr Valialkin	7cc3d96a41	lib/fs: follow-up after `f3a03c4164`	2021-02-27 01:01:47 +02:00
Nikolay	f3a03c4164	Adds windows build (#1040 ) * fixes windows compilation, adds signal impl for windows, adds free space usage for windows, https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70 https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1036 NOTE victoria metrics database still CANNOT work under windows system, only vmagent is supported. To completly port victoria metrics, you have to fix issues with separators, parsing and posix file removall * rollback separator * Adds windows setInformation api, it must behave like unix, need to test it. changes procutil * check for invlaid param * Fixes posix delete semantic * refactored a bit * fixes openbsd build * removed windows api call * Fixes code after windows add * Update lib/procutil/signal_windows.go Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-02-27 00:37:07 +02:00
Nikolay	186c078fac	adds enforced tag filters into cache key (#1095 )	2021-02-27 00:15:53 +02:00
Aliaksandr Valialkin	d86e9b49c4	app/vmselect/promql: increase accuracy for `buckets_limit()` function for small limits by skipping the first and the last buckets during merge The first and the last buckets are usually `[0 ... leMin]` and `(leMax ... +Inf)`. If they are merged with adjancent buckets, then the resulting accuracy can suffer.	2021-02-26 22:56:36 +02:00
Aliaksandr Valialkin	98854e5f2b	app/vmselect: add `sign(q)` and `clamp(q, min, max)` functions, which will be added in the upcoming Prometheus release See https://twitter.com/roidelapluie/status/1363428376162295811 The `last_over_time(m[d])` function already exists in MetricsQL.	2021-02-24 17:24:56 +02:00
Aliaksandr Valialkin	f4135b0d14	app/vmselect/promql: properly calculate `histogram_quantile() over zero buckets and only a single non-zero` le="+Inf"` bucket like Prometheus does	2021-02-24 00:42:22 +02:00
Aliaksandr Valialkin	fa03e0d210	app/vmselect/promql: add `increase_pure()` function to MetricsQL Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962	2021-02-22 19:14:15 +02:00
Aliaksandr Valialkin	d136081040	lib/promrelabel: add more optimizations for relabeling for common cases	2021-02-22 16:33:55 +02:00
Aliaksandr Valialkin	c2678754e4	app/vmagent: properly perform graceful shutdown, which was broken in the commit `1d1ba889fe` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065	2021-02-19 00:31:34 +02:00
Aliaksandr Valialkin	49e36e8d9d	app/vmagent: fix scrape config example for scrape_align_interval option	2021-02-18 23:57:23 +02:00
Aliaksandr Valialkin	f26162ec99	lib/promscrape: add scrape_align_interval config option into scrape config This option allows aligning scrapes to a particular intervals.	2021-02-18 23:53:44 +02:00
Aliaksandr Valialkin	9c70c1f21f	app/vmselect/promql: reduce the probability of `duplicate time series` errors when querying Kubernetes metrics	2021-02-18 22:07:29 +02:00
Aliaksandr Valialkin	a537c4f602	lib/storage: properly handle queries containing a filter on metric name plus any number of negative filters and zero non-negative filters Example: `node_cpu_seconds_total{mode!="idle"}`	2021-02-18 18:46:36 +02:00
Aliaksandr Valialkin	03ebc028f7	app/vmalert: add missing multiarch Dockerfile	2021-02-18 15:23:17 +02:00
Aliaksandr Valialkin	d7de4807e1	app/victoria-metrics/testdata: add a test for {__graphite__="foo.*.bar"} selector	2021-02-17 21:52:43 +02:00
Aliaksandr Valialkin	edcdc39eb3	app/vmagent/remotewrite: cleanup after `1d1ba889fe` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065	2021-02-17 21:42:55 +02:00
Nikolay	1d1ba889fe	adds pushback for fastqueue, (#1075 ) during shutdown currently sending block was lost, now its pushed back to fast queue and will be flushed on disk, it may lead to data duplication. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065	2021-02-17 21:23:38 +02:00
Aliaksandr Valialkin	d4849561ef	app/vmstorage: export vm_composite_filter_success_conversions_total and vm_composite_filter_missing_conversions_total metrics	2021-02-17 19:13:38 +02:00
Aliaksandr Valialkin	7f2a6c7b54	docs: rename vmbackuper to vmbackupmanager	2021-02-16 22:00:39 +02:00
Aliaksandr Valialkin	c4756f94da	app/vmselect/netstorage: reuse timeseriesWork objects in order to reduce memory allocations	2021-02-16 16:08:53 +02:00
Aliaksandr Valialkin	5a401225c7	app/vmselect/netstorage: use unsafe string as a key for a map when the map already contains the given key This should prevent from a memory allocation and a string copy.	2021-02-16 15:43:10 +02:00
Aliaksandr Valialkin	4e39bf148c	vendor: update github.com/VictoriaMetrics/metrics from v1.13.1 to v1.14.0 The new version switches from log-linear histograms to log-based histograms, which provide up to 3.6 times better accuracy.	2021-02-15 15:12:29 +02:00
Aliaksandr Valialkin	802fabf0d7	deployment/docker: embed tzdata into prod Go app instead of installing it into base docker image While this increases app size by 700Kb, this allows using -loggerTimezone in a scratch base image See https://github.com/golang/go/issues/38017	2021-02-12 04:54:27 +02:00
Aliaksandr Valialkin	f9902b3372	deployment/docker: use `docker buildx` for creating multiarch builds See https://github.com/docker/buildx/	2021-02-12 04:31:22 +02:00
Aliaksandr Valialkin	acdb401585	app/vmselect/prometheus: treat `match` query arg in the same way as `match[]` query arg	2021-02-11 15:02:21 +02:00
Aliaksandr Valialkin	1e38ad6d20	app/vmauth: add ability to route requests from a single users to multiple targets depending on the requested path Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1064	2021-02-11 12:41:16 +02:00
Aliaksandr Valialkin	2d33230793	app/vmselect/promql: properly make copies of EvalConfig	2021-02-11 12:41:15 +02:00
Aliaksandr Valialkin	7a3a9421f3	app/vmselect/promql: make a copy of EvalConfig when executing `q1` and `q2` in parallel for `q1 binary_op q2` This should prevent from data races if the underlying functions modify EvalConfig contents.	2021-02-10 23:05:55 +02:00
Aliaksandr Valialkin	04faea8b45	app/vmselect: parallelize `q1 <binary_op> q2` queries by running `q1` and `q2` in parallel This should reduce query execution times.	2021-02-10 22:59:39 +02:00
Aliaksandr Valialkin	cdecf83ce5	app/vmstorage: export vm_composite_index_min_timestamp metric	2021-02-10 17:14:08 +02:00
Aliaksandr Valialkin	553016ea99	lib/storage: disable composite index usage when querying old data	2021-02-10 14:57:50 +02:00
Aliaksandr Valialkin	c7dccebaef	lib/storage: optimize search by label filters matching big number of time series	2021-02-10 00:44:54 +02:00
Aliaksandr Valialkin	cb96a1865b	app/vmstorage: export missing `vm_cache_size_bytes` metrics for indexdb and data caches	2021-02-09 00:47:00 +02:00
Aliaksandr Valialkin	9b9cb04511	docs/vmctl.md: fix title, so it is properly displayed in the header of https://victoriametrics.github.io/	2021-02-04 20:20:08 +02:00
Aliaksandr Valialkin	803a00102a	app/{vminsert,vmselect}: accept requests to paths with `/graphite` and `/prometheus` prefixes This should improve compatibility with path prefixes from VictoriaMetrics cluster. See https://victoriametrics.github.io/Cluster-VictoriaMetrics.html#url-format	2021-02-04 20:01:18 +02:00
Aliaksandr Valialkin	4d43ab0875	app/vmselect: typo fix when stripping url path prefixes	2021-02-04 19:29:25 +02:00
Roman Khavronenko	2ff038d841	vmalert: mention `-datasource.appendTypePrefix` in README (#1052 )	2021-02-03 23:44:37 +02:00
Dmitry Shevchuk	2baf98082b	Adds ability to query right vmselect endpoint based on the query type (#1050 ) * Adds ability to query right vmselect endpoint based on the query type Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>	2021-02-03 21:26:30 +00:00
Aliaksandr Valialkin	a2344ef4b7	docs/vmalert.md: mention that `type` option can be set at group level additionally to rule level	2021-02-03 21:13:13 +02:00
Aliaksandr Valialkin	8dc6095749	app/vmagent: add `Advanced usage` section with the description for all the command-line flags	2021-02-03 21:02:59 +02:00
Aliaksandr Valialkin	8629fd8a72	app/vmselect: deprecate `-search.treatDotsAsIsInRegexps` in favor to `{__graphite__="foo.*.bar"}` syntax	2021-02-03 20:36:01 +02:00
Aliaksandr Valialkin	d16f22f3a1	app/vmselect,lib/storage: properly parse Graphite selectors with inner wildcards Example: foo{bar{x,yz},a[b-c],*de}	2021-02-03 20:14:22 +02:00
Aliaksandr Valialkin	157c02622b	app/vmselect: add ability to set Graphite-compatible filter via `{__graphite__="foo.*.bar"}` syntax	2021-02-03 01:21:54 +02:00
Aliaksandr Valialkin	b577cdd855	docs: increase heading sizes in vmagent, vmauth, vmbackup and vmrestore docs, so they match the heading sizes in VictoriaMetrics docs	2021-02-01 19:44:00 +02:00
Aliaksandr Valialkin	8164cd8932	docs/vmctl.md: update build instructions after the migration from github.com/VictoriaMetrics/vmctl to github.com/VictoriaMetrics/VictoriaMetrics/app/vmctl	2021-02-01 19:39:08 +02:00
Aliaksandr Valialkin	b43b498fd8	app/vmselect: add ability to pass `extra_label=<label>=<value>` query arg to Prometheus Querying API This enforced `{label="value"}` label filter to the query. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1021	2021-02-01 18:04:17 +02:00
Aliaksandr Valialkin	5d87dbfd65	docs: document ability to query Graphite datasource from vmalert	2021-02-01 15:26:33 +02:00
Nikolay	195341a7cf	Graphite vmalert wip (#112 ) * init implementation for graphite alerts * adds graphite support for vmalert * small fix * changes vmalert graphite api with type * updates tests * small fix * fixes graphite parse * Fixes graphite from time	2021-02-01 15:05:32 +02:00
Aliaksandr Valialkin	a4ae945a79	app/victoria-metrics: fix tests after `8749c2dd92`	2021-02-01 14:34:11 +02:00
Aliaksandr Valialkin	b2aa80e74b	app/vmagent: add -remoteWrite.roundDigits command-line option for limiting the number of digits after the point for stored values This commit also adds --vm-round-digits command-line option to vmctl tool.	2021-02-01 14:27:09 +02:00
Aliaksandr Valialkin	29a7067827	app/vmctl: fix `make check-all` warnings	2021-02-01 01:31:25 +02:00
Aliaksandr Valialkin	d5c180e680	app/vmctl: move vmctl code from github.com/VictoriaMetrics/vmctl It is better developing vmctl tool in VictoriaMetrics repository, so it could be released together with the rest of vmutils tools such as vmalert, vmagent, vmbackup, vmrestore and vmauth.	2021-02-01 01:10:20 +02:00
Aliaksandr Valialkin	de3c662e8a	all: consistently use timers from timerpool	2021-01-27 00:39:26 +02:00
Aliaksandr Valialkin	419ad74269	app/vmagent: add -remoteWrite.rateLimit command-line flag for limiting data rate to remote storage See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1035	2021-01-27 00:39:26 +02:00
weng zhao	cc3e69e963	vmalert: add option datasource.queryStep to allow user to address the inconsistency between grafana dashboards(query_range with step 15s usually) and ALERTS (#1027 ) Co-authored-by: zhao.weng <zhao.weng@shopee.com>	2021-01-26 08:12:04 +00:00
Aliaksandr Valialkin	c164a8d231	app/vmselect/promql: improve documentation for `-search.maxPointsPertimeseries` command-line flag This should reduce incorrect usage and assumptions for this flag. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1020	2021-01-22 13:00:10 +02:00
Aliaksandr Valialkin	8749c2dd92	app/vmselect: add `-search.maxStepForPointsAdjustment` command-line flag, which can be used for disabling adjustment for points returned from `/api/v1/query_range` handler if they have timestamps closer than `-search.latencyOffset` to the current time	2021-01-19 22:56:32 +02:00
Aliaksandr Valialkin	011c5da785	app/vmselect/graphite: extract getCanonicalPath() function from loop body inside getCanonicalPaths()	2021-01-18 17:30:26 +02:00
Aliaksandr Valialkin	75f7c51cab	docs/vmagent.md: follow-up for `184a659c5f`	2021-01-13 13:53:14 +02:00
Aliaksandr Valialkin	1051d8aa2d	app/vmselect/promql: add ability to pass multiple labels to `sort_by_label` and `sort_by_label_desc` functions See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/992	2021-01-13 12:44:51 +02:00
Aliaksandr Valialkin	acf1a2c72b	app/vmselect/promql: properly parse escaped multibyte utf8 code sequences in metric names and labels names See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/990	2021-01-13 10:59:42 +02:00
Nikolay	dc9d7aedd5	adds extra_label to all import apis (#1007 ) * adds extra_label to all import apis, changes priority for extra_label - now it has priority over original labels * Update README.md Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com> * Update README.md Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com> * adds extra labels to vmagent import api changes order for adding labels, now its added after user values * adds tests for extra_label * import fix Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2021-01-13 00:52:50 +02:00
Aliaksandr Valialkin	1837f2f7d3	app/vmselect/promql: add `tfirst_over_time(m[d])` and `tlast_over_time(m[d])` MetricsQL functions for returning timestamps for the first and the last samples in `m` over `d`	2021-01-12 16:12:12 +02:00
Nikolay	7976c22797	Fixes error handling for promscrape.streamParse (#1009 ) properly return error if client cannot read data, properly suppress scraper errors	2021-01-12 13:31:47 +02:00
Aliaksandr Valialkin	24ffad74c1	all: use `net.Dial` instead of `fasthttp.Dial`, because `fasthttp.Dial` limits the number of concurrent dials to 1000	2021-01-11 12:53:30 +02:00
Roman Khavronenko	2e2e4f7e21	vmalert-989: return non-empty result in template func `query` stub to pass validation (#1002 ) On templates validation stage vmalert does not acutally send queries, so for complex chained expression validation may fail. To avoid this, we add a blank sample in response so validation can pass successfully. Later, during the rule execution, stub will be replaced with real `query` function. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/989	2021-01-10 02:56:11 +03:00
Aliaksandr Valialkin	9dcb18e03d	app/vmstorage: disable final merge by default, since it may result in high disk IO and CPU usage without measurable benefits such as increased query performance and reduced disk space usage	2021-01-08 00:16:05 +02:00
Aliaksandr Valialkin	c86286ec1d	app/vmselect/promql: do not ajdust `offset` value provided in the query Previously it could be modified in order to improve response cache hit ratio. This is unneeded, since cache hit ratio should remain good because the query time range should be already aligned to multiple of `step` values. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/976	2020-12-27 14:09:25 +02:00
Aliaksandr Valialkin	4b7105a65b	app/vmselect: sync query stats handling with cluster version	2020-12-27 13:00:29 +02:00
Aliaksandr Valialkin	df0309eae0	app/vmselect/promql: simplify defer call for querystats.RegisterQuery	2020-12-27 12:06:04 +02:00
Aliaksandr Valialkin	ad4e6a9283	app/vmselect/querystats: reduce the default number of last queries to track from 100K to 20K This should reduce memory usage in constrained environments	2020-12-25 17:40:47 +02:00
Aliaksandr Valialkin	59183f66d0	app/vmselect: refactor `/api/v1/stats/top_queries` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907	2020-12-25 16:44:29 +02:00
Aliaksandr Valialkin	fb338c50a3	app/victoria-metrics: show usage info when incorrect command-line flag is passed to executable	2020-12-25 16:42:21 +02:00
Nikolay	86630350bf	Adds query stats handler (#945 ) * Adds query stat handler, for query and query_range api, victoriametrics tracks query execution time, stats are expored at /api/v1/status/queries endpoint with topN param https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907 * fixed query stats bugs * improves queryStats tracker * improves query stat * small fix * fix tests * added more tests * fixes 386 tests * naming fixes * adds drop for outdated records	2020-12-25 16:42:05 +02:00
Nikolay	1de15ad490	adds escape for CRLF (#984 ) at external.alert.source - \n and \r symbols was url encoded, instead of direct usage. replace it from "\n" to `\n` allows to skip url encoding. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/890	2020-12-25 11:03:13 +02:00
Aliaksandr Valialkin	0326638c90	app/vmalert: typo fix in descriptions for notifier.basicAuth.username and notifier.basicAuth.password command-line flags	2020-12-24 12:48:59 +02:00
Aliaksandr Valialkin	9df60518bb	docs: mention that it is possible to set multiple `-notifier.tlsInsecureSkipVerify` command-line flags for vmalert See c3a92968343c2b3619f1ab935702d0e9b3a46733	2020-12-22 22:32:13 +02:00
Nikolay	c270f8f3e6	changes vmalert notifier flag, (#978 ) fixes issue with notifier insecure setting, now its possible to use multiple notifier.tlsInsecureSkipVerify multiple time.	2020-12-22 23:23:04 +03:00
Roman Khavronenko	404cbd1522	vmalert-974: fix order for labels templating (#975 ) The change fixes bug caused by `3adf8c5a6f`. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/974	2020-12-19 14:10:59 +02:00
Aliaksandr Valialkin	88ac4dfc07	app/vmselect: properly parse negative combined offsets such as `-1h2m3s` Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.	2020-12-19 01:23:46 +02:00
Aliaksandr Valialkin	aa56b9217e	app/vmagent: add `vmagent_remotewrite_blocks_sent_total` and `vmagent_remotewrite_bytes_sent_total` metrics per each `-remoteWrite.url`	2020-12-15 20:39:57 +02:00
Aliaksandr Valialkin	b10ad44692	docs/vmagent.md: typo fix: pearsed->parsed	2020-12-15 19:03:01 +02:00
Aliaksandr Valialkin	1eabbc0e27	docs/vmagent.md: mention that `sample_limit` option has no sense when stream parsing is enabled	2020-12-15 18:43:41 +02:00
Aliaksandr Valialkin	11fa458e39	app/vmselect/promql: return expected `increase()` result for the first point on the graph with value not exceeding 100	2020-12-15 13:40:46 +02:00
Nikolay	95c9b630cc	adds new Array Flags (#965 ) * adds ArrayDuration and ArrayBool flags, makes sendTimeout and tlsInsecure configurable per remoteWrite url * added backward compatibility testcases for ArrayDuration and ArrayBool * fixes bool flag * fixes test cases	2020-12-15 12:51:12 +02:00
Roman Khavronenko	6247884057	vmalert: add function "query", "first" and "value" to alert templates functions (#960 ) The commit adds a support for template function `query`, `first` and `value`. The function `query` executes a MetricsQL query for active alerts. In vmalert we update templates on every evaluation for active alerts to keep them up to date. With `query` func it may become a perf issue since it will fire a query on every execution. We should keep it in mind for now. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/539	2020-12-14 20:11:45 +02:00
Aliaksandr Valialkin	f8e7f433cf	app/victoria-metrics: prettify `/` page output	2020-12-14 14:07:58 +02:00
Aliaksandr Valialkin	069c9ade52	app/{vmagent,vminsert}: follow-up for `ce8c2dd1f1`: return `/targets` page in HTML when requested via web browser	2020-12-14 14:06:00 +02:00
Nikolay	ce8c2dd1f1	Changes targets api (#961 ) * changes /targets api adds html response if requester accepts text/html, adds quick template for /targets api, fixes pathPrefix for / requests * changes namings * renamed targets file * Update app/victoria-metrics/main.go Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com> * adds trimspace to qtpl, moves content-type for targets response closer to writer * fixes bug with prefix Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-12-14 13:36:48 +02:00
Aliaksandr Valialkin	5ebfc275e6	app/victoria-metrics: automatically reset response cache when samples with too timestamps older than `now - search.cacheTimestampOffset` are ingested	2020-12-14 13:08:28 +02:00
Aliaksandr Valialkin	9d42546a27	docs: consistently use links to https://victoriametrics.github.io for documentation references	2020-12-11 21:08:18 +02:00
Aliaksandr Valialkin	ba8ac08739	app/vmselect/graphite: properly handle wildcards and charsets inside curly braces For example, `foo{bar,[a-f]ab}` should match `foobar`, `foobar123`, `foofab`, etc. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/952	2020-12-11 17:24:12 +02:00
Aliaksandr Valialkin	4146fc4668	all: properly handle CPU limits set on the host system/container This can reduce memory usage on systems with enabled CPU limits. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946	2020-12-08 21:07:29 +02:00
Aliaksandr Valialkin	1906f841c9	app/vmselect/promql: do not reduce lookbehind window for `any_rollup_func(m)` to `-search.maxStalenessInterval`. It should equal to `step` value passed to `/api/v1/query_range` as most users expect	2020-12-08 15:16:17 +02:00
Aliaksandr Valialkin	b6b1b06d70	app/{vmalert,vmagent}: skip empty values in `-remoteWrite.label` and `-label` lists	2020-12-08 14:55:13 +02:00
Aliaksandr Valialkin	007dbf273d	app/vmselect/graphite: remove duplicate `name` tag from `/tags/autoComplete/tags` handler Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/942	2020-12-07 01:08:52 +02:00
Aliaksandr Valialkin	e2e8ef86d9	app/vmselect/promql: add `count_eq_over_time(m[d], N)` and `count_ne_over_time(m[d], N)` for calculating the number of samples in `m` over `d` that are equal / not equal to `N`	2020-12-05 12:30:46 +02:00
Aliaksandr Valialkin	9d787f9edd	all: do not print usage info for all the flags when incorrect command-line flag is passed This should improve usability for VictoriaMetrics apps that have big number of command-line flags, i.e. all the apps.	2020-12-03 21:47:37 +02:00
Aliaksandr Valialkin	66379cc69f	app/vmselect/promql: add `label_uppercase(q, label1, ... labelN)` and `label_lowercase(q, label1, ... labelN)` functions Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/936	2020-12-03 21:47:36 +02:00
Aliaksandr Valialkin	44a34a0f5f	app/vmselect/promql: `make fmt`	2020-12-02 21:33:35 +02:00
Aliaksandr Valialkin	1982505c2b	app/vmselect/promql: return `nan` from `minute(m)` when `m` equals to `nan` This aligns VictoriaMetrics behaviour with Prometheus behaviour. The issue has been spotted in https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/	2020-12-02 20:16:58 +02:00
Aliaksandr Valialkin	9d87496b50	app/vmselect/promql: do not return `0` value from `sum_over_time(m[d])` when there are no samples on the given `d` window. This aligns the behaviour of `sum_over_time()` with other `_over_time()` functions and with Prometheus behavior.	2020-12-02 13:12:50 +02:00
Aliaksandr Valialkin	91a4c279cc	app/vmselect: return `metric` values from `time() cmp_op metric` query when `cmp_op` comparison is true This aligns MetricsQL behavior to Prometheus' one. The issue has been identified at https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/	2020-12-02 12:09:34 +02:00
Aliaksandr Valialkin	700bda8e2e	app/vmselect/promql: return `nan` from `a >bool b` if `a` is `nan` in the same way as Prometheus does	2020-12-02 00:28:26 +02:00
Aliaksandr Valialkin	efdc3c71af	app/vmselect/searchutils: return elapsed time in Deadline.String() output This should improve debuggability for error messages containing Deadline.String() output	2020-12-01 00:15:18 +02:00
Aliaksandr Valialkin	ca091bade3	app/vmbackup/snapshot: add missing status code check for the returned response when working with snapshot API Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/929	2020-11-30 14:49:39 +02:00
Aliaksandr Valialkin	b35b3dc043	app/vmbackup/snapshot: log url and response body on failed JSON response parsing Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/929	2020-11-29 12:16:16 +02:00
Nikolay	0463cb5550	fixes checksum calculation (#928 ) * fixes checksum calculation, 'for' rule param wasnt marshal properly during checksum calculation * fixes error	2020-11-29 09:48:42 +02:00
Aliaksandr Valialkin	b7fcdb528d	app/{vmagent,victoria-metrics}: add `-dryRun` option and make more clear handling for `-promscrape.config.dryRun`	2020-11-25 22:59:13 +02:00
Aliaksandr Valialkin	dabbf930d8	app/vmagent: do not enable -promscrape.config.strictParse when -dryRun command-line flag is set Users can specify -promscrape.config.strictParse if -promscrape.config shouldn't contain unknown config entries	2020-11-25 22:26:25 +02:00
BigFish	0acdab3ab9	Update main.go (#922 ) fix spelling mistake	2020-11-23 17:33:17 +02:00
Aliaksandr Valialkin	7e8dcf9ddc	app/vmbackup: cosmetic fixes	2020-11-23 17:10:04 +02:00
Aliaksandr Valialkin	a724dde90a	app/vmselect: protect `/tags/delSeries` with `-deleteAuthKey` in the same way as `/api/v1/admin/tsdb/delete_series`	2020-11-23 15:35:59 +02:00
Aliaksandr Valialkin	f0c207fae2	app/vmselect: add `/tags/delSeries` handler from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#removing-series-from-the-tagdb	2020-11-23 15:27:21 +02:00
Aliaksandr Valialkin	d3794eb994	app/{vminsert,vmselect}: move `/tags/tagSeries` and `/tags/tagMultiSeries` api from vminsert to vmselect This is needed for consistency, since all the `/tags*` api handlers are located in vmselect.	2020-11-23 12:33:19 +02:00
Aliaksandr Valialkin	f765985947	lib/fs: replace fs.OpenReaderAt with fs.MustOpenReaderAt All the callers for fs.OpenReaderAt expect that the file will be opened. So it is better to log fatal error inside fs.MustOpenReaderAt instead of leaving this to the caller.	2020-11-23 09:57:21 +02:00
Nikolay	eec76718e9	Adds eureka service discovery (#913 ) * Adds eureka service discovery https://github.com/VictoriaMetrics/VictoriaMetrics/issues/851 Netflix service discovery for AWS * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-11-20 13:38:12 +02:00
Aliaksandr Valialkin	2859a452d4	app/vmselect: add remoteAddr to slow query log in order to improve debuggability This will simplify identifying the client that sends slow queries to VictoriaMetrics.	2020-11-18 20:38:32 +02:00
Nikolay	f818ab497b	Fixes snap script (#909 )	2020-11-18 17:46:31 +03:00
Nikolay	c2afa3fdd7	adds snap package for victoria-metrics (#904 ) * adds snap package for victoria-metrics * Update README.md Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-11-18 02:00:06 +02:00
Aliaksandr Valialkin	266788be14	app/vmselect: use storage.NewSearchQuery() instead of constructing storage.SearchQuery in-place This should prevent from bugs when AccountID and ProjectID aren't set in storage.SearchQuery.	2020-11-16 18:24:00 +02:00
Aliaksandr Valialkin	cf18df367d	app/vmselect/netstorage: apply Graphite filter after substituting `__name__` with `name`	2020-11-16 15:52:16 +02:00
Aliaksandr Valialkin	2c67232565	app/vmselect/graphite: add `/tags/autoComplete/values` handler from Graphite Tags API	2020-11-16 15:29:35 +02:00
Aliaksandr Valialkin	86f99c6b55	app/vmselect/graphite: add `/tags/autoComplete/tags` handler from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#auto-complete-support	2020-11-16 14:50:05 +02:00
Aliaksandr Valialkin	3c1434118e	app/vmselect/prometheus: return `__name__` label if `match[]` query to `/api/v1/labels` matches at least a single time series	2020-11-16 13:54:34 +02:00
Aliaksandr Valialkin	27a417bcd3	app/vmselect/prometheus: improve performance for `/api/v1/labels` and `/api/v1/label/<labelName>/values` on time ranges exceeding one day when `match[]` query arg is set	2020-11-16 13:51:59 +02:00
Aliaksandr Valialkin	6fa806f1ca	app/vmselect/prometheus: fix deadlock in `/api/v1/series` on a time range exceeding one day	2020-11-16 13:30:47 +02:00
Aliaksandr Valialkin	465923b181	app/vmselect/graphite: add /tags/findSeries handler from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#exploring-tags	2020-11-16 12:53:13 +02:00
Aliaksandr Valialkin	414cd39659	app/vmselect/graphite: apply filter then limit	2020-11-16 04:09:14 +02:00
Aliaksandr Valialkin	d100341394	app/vmselect/graphite: add `/tags/<tag_name>` handler for Graphite Tags API	2020-11-16 03:42:25 +02:00
Aliaksandr Valialkin	6251762787	app/vmselect/graphite: add `/tags` handler from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#exploring-tags	2020-11-16 03:00:01 +02:00
Aliaksandr Valialkin	48d033a198	app/vminsert: add `/tags/tagSeries` and `/tags/tagMultiSeries` handlers from Graphite Tags API See https://graphite.readthedocs.io/en/stable/tags.html#adding-series-to-the-tagdb	2020-11-16 02:39:58 +02:00
Aliaksandr Valialkin	d9d01f976b	app/vmselect/promql: remove spikes from `increase()` and `delta()` results on time series with spare irregular data points Do not take into account spare data point value if the next point will is located too far from the current point. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894	2020-11-13 15:23:44 +02:00
Aliaksandr Valialkin	1f19c167a4	app/vmselect/promql: assume that time series value doesnt change during gaps when calculating increase() and delta() This should remove unexpected spikes at the end of gaps. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894	2020-11-13 14:59:24 +02:00
Aliaksandr Valialkin	47a038401b	all: consistently return text-based HTTP responses with charset=utf-8 This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897	2020-11-13 10:35:41 +02:00
faceair	077f8cbe1c	add charset on targets response (#897 )	2020-11-13 10:17:37 +02:00
Aliaksandr Valialkin	4057305148	docs/vmagent.md: added a link to https://valyala.medium.com/how-to-use-relabeling-in-prometheus-and-victoriametrics-8b90fc22c4b2 into `Relabeling` section	2020-11-12 12:27:06 +02:00
Aliaksandr Valialkin	bb06b98202	docs/vmagent.md: typo fix	2020-11-11 16:04:46 +02:00
Aliaksandr Valialkin	4adb96161a	docs/vmagent.md: add `Configuration update` section	2020-11-11 16:01:15 +02:00
immerrr again	51c529a2b6	app/vmstorage: add "/internal/force_flush" endpoint (#893 )	2020-11-11 14:40:27 +02:00
Aliaksandr Valialkin	348edd92fe	app/vmselect: add `-search.treatDotsAsIsInRegexps` command-line flag for automatic escaping of dots in regexp label filters	2020-11-11 12:39:07 +02:00
Aliaksandr Valialkin	47390d8947	app/vmselect/promql: do not return data points in the end of the selected time range for time series ending in the middle of the selected time range Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/887 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845	2020-11-10 14:51:44 +02:00
Aliaksandr Valialkin	ba4a2c8bca	app/vmselect: typo fix in a description for `-search.minStalenessInterval`: mimimum->minimum	2020-11-10 01:18:08 +02:00
Roman Khavronenko	3adf8c5a6f	vmalert: explicitly set extra labels to alert entities (#886 ) The previous implementation treated extra labels (global and rule labels) as separate label set to returned time series labels. Hence, time series always contained only original labels and alert ID was generated from sorted labels key-values. Extra labels didn't affect the generated ID and were applied on the following actions: - templating for Summary and Annotations; - persisting state via remote write; - restoring state via remote read. Such behaviour caused difficulties on restore procedure because extra labels had to be dropped before checking the alert ID, but that not always worked. Consider the case when expression returns the following time series `up{job="foo"}` and rule has extra label `job=bar`. This would mean that restored alert ID will be always different to the real time series because of collision. To solve the situation extra labels are now always applied beforehand and `vmalert` doesn't store original labels anymore. However, this could result into a new error situation. Consider the case when expression returns two time series `up{job="foo"}` and `up{job="baz"}`, while rule has extra label `job=bar`. In such case, applying extra labels will result into two identical time series and `vmalert` will return error: `result contains metrics with the same labelset after applying rule labels` https://github.com/VictoriaMetrics/VictoriaMetrics/issues/870	2020-11-10 00:27:32 +02:00
Aliaksandr Valialkin	bcd139362b	lib/promscrape: add `-promscrape.dropOriginalLabels` command-line flag for reducing memory usage when discovering big number of scrape targets Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825	2020-11-10 00:19:57 +02:00
Aliaksandr Valialkin	83df20b5b5	lib/promscrape: clean references to label name and label value strings after applying per-target relabeling This should reduce memory usage when per-target relabeling creates big number of temporary labels with long names and/or values. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825	2020-11-07 16:19:59 +02:00
Aliaksandr Valialkin	dd6bfa50e9	app/vmselect/promql: code cleanup after `43823addea`	2020-11-06 01:30:50 +02:00
n4mine	43823addea	app/vmselect/promql: fix when the parameter of `maxValue()`, `minValue()` leading by `NaN`. it will cause `{top,bottom}k_{max,min}` return inappropriate result (#883 )	2020-11-06 01:29:24 +02:00
Aliaksandr Valialkin	368b69b4c4	app/vmselect: properly handle errors in GetLabelsOnTimeRange and GetLabelValuesOnTimeRange	2020-11-05 01:38:38 +02:00
Aliaksandr Valialkin	b378cd6ed8	app/vmselect: optimize querying for `/api/v1/labels` and `/api/v1/label/<name>/values` when `start` and `end` args are set	2020-11-05 01:01:33 +02:00
Aliaksandr Valialkin	381ad564a2	docs/vmagent.md: update after `4c808d58bf`	2020-11-04 20:31:43 +02:00
Nikolay	4c808d58bf	Adds ready probe (#874 ) * adds leading forward slash check for scrapeURL path https://github.com/VictoriaMetrics/VictoriaMetrics/issues/835 * adds ready probe for scrape config initialization, it should prevent metrics loss during vmagent rolling update, /ready api will return 425 http code, if some scrape config still waits for initialization. * updates docs * Update app/vmagent/README.md * renames var * Update app/vmagent/README.md Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-11-04 20:29:18 +02:00
Aliaksandr Valialkin	caeb74f068	app/vmselect: reduce memory usage when query touches big number of time series	2020-11-04 17:04:04 +02:00
Nikolay	e4182dd896	reduces memory usage for vmagent, (#880 ) * reduces memory usage for vmagent, limits count of droppedTarget, that can be stored for /api/v1/targets page up to 999 items, https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878 * Update app/vmagent/README.md * Update app/vmagent/README.md Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-11-04 17:03:43 +02:00
Aliaksandr Valialkin	600f225cff	Revert "docs/vmagent.md: mention about `-promscrape.dropOriginalLabels`" This reverts commit `bd81f926a4`.	2020-11-04 11:44:57 +02:00
Aliaksandr Valialkin	bd81f926a4	docs/vmagent.md: mention about `-promscrape.dropOriginalLabels`	2020-11-04 11:16:33 +02:00
Aliaksandr Valialkin	e0a91ef163	app/vmagent/remotewrite: drop packets only on 409 status code, since there are other valid 4xx status codes, which shouldnt result in packet drop	2020-11-03 14:25:10 +02:00
Aliaksandr Valialkin	5e75c389e6	app/vmselect/promql: allow dropping trailing sample only for `default_rollup` function Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/850	2020-11-02 02:10:59 +02:00
Aliaksandr Valialkin	0b35da159c	app/vmagent/remotewrite: drop packets if remote storage returns 4xx status code This makes consistent the behaviour with Prometheus. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/873	2020-11-02 00:45:09 +02:00
Aliaksandr Valialkin	cb71af216a	app/vmselect/promql: go fmt	2020-11-02 00:15:29 +02:00
Aliaksandr Valialkin	daacbc7e34	app/vmselect/promql: do not drop trailing datapoints for instant queries Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748	2020-11-02 00:12:37 +02:00
Aliaksandr Valialkin	e277c3d07b	lib/promscrape: add `stream parse` mode for efficient scraping of targets that expose millions of metrics	2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin	b7638f04a7	app/vmagent: expose `/api/v1/targets` page according to https://prometheus.io/docs/prometheus/latest/querying/api/#targets This page is exposed by vmagent and by a single-node VictoriaMetrics Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/643	2020-11-01 23:35:06 +02:00
Aliaksandr Valialkin	c539494b36	app/vmselect/promql: allow passing optional third argument to `topk_` and `bottomk_` functions in order to obtain sum of time series outside top/bottom K	2020-11-01 23:35:06 +02:00
Roman Khavronenko	f0bdc5716e	vmalert: skip automatically added labels on alerts restore (#871 ) Label `alertgroup` was introduced in #611 and automatically added to generated time series. By mistake, this new label wasn't correctly purged on restore event and affected alert's ID uniqueness. This commit removes `alertgroup` label in restore function. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/870	2020-10-30 08:18:20 +00:00
kreedom	a51a7b2a20	vmbackup fix panic when no origin fs given (#859 ) * use fsnil when no origin fs	2020-10-28 20:09:10 +02:00
Aliaksandr Valialkin	5bfd4e6218	app/vmstorage: support for `-retentionPeriod` smaller than one month Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/173 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/17	2020-10-20 14:31:44 +03:00
kreedom	ef77120170	vmalert - add dryRun (#842 ) vmalert: add `dryRun` flag for rules validation without running the service	2020-10-20 08:15:21 +01:00
Seva Poliakov	b3f3c078e5	Fix typo in vnrestore readme	2020-10-18 15:41:39 +03:00
Aliaksandr Valialkin	28353e48ca	app/vmselect/promql: an attempt to improve heuristics for dropping trailing data points in time series Now trailing data points are additionally dropped for time series with a single raw sample Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748	2020-10-17 10:44:34 +03:00
Roman Khavronenko	bc42b5598f	vmalert: update docs to highlight the state restore requirements; (#833 ) Address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/830	2020-10-13 18:32:43 +03:00
Aliaksandr Valialkin	a2b9476897	app/vmselect/promql: return a single time series at max from `absent()` function like Prometheus does	2020-10-13 15:56:04 +03:00
Aliaksandr Valialkin	9aa3b65766	app/vmselect/promql: improve time series staleness detection This should prevent from double counting for time series at the time when it changes label. The most common case is in K8S, which changes pod uid label with each new deployment. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748	2020-10-13 12:19:57 +03:00
Aliaksandr Valialkin	d8af290947	app/vmselect/promql: fix `mode_over_time` calculations Previously `mode_over_time` could return garbage due to improper shuffling of input data points.	2020-10-13 11:58:25 +03:00
Aliaksandr Valialkin	1e27420243	app/vmselect/prometheus: fix golangci-lint warning	2020-10-13 09:36:11 +03:00
Aliaksandr Valialkin	4f16a964e3	app/vmselect: add ability to export data in CSV format via `/api/v1/export/csv`	2020-10-12 20:08:17 +03:00
Aliaksandr Valialkin	63c4999e06	lib/promscrape: code prettifying after `9bd9f67718`	2020-10-12 16:12:36 +03:00
Nikolay Khramchikhin	9bd9f67718	Adds dockerswarm sd (#818 ) * adds dockerswarm service discovery https://github.com/VictoriaMetrics/VictoriaMetrics/issues/656 Following roles supported: services, tasks and nodes. Basic, token and tls auth supported. Added tests for labels generation. * added unix socket support to discovery utils Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-10-12 13:38:21 +03:00
Aliaksandr Valialkin	762c967855	app/vmselect/promql: keep metric name after applying more functions, which dont change time series meaning Functions are: * keep_last_value * keep_next_value * interpolate * running_min * running_max * running_avg * range_min * range_max * range_avg * range_first * range_last * range_quantile * smooth_exponential Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674	2020-10-12 11:47:06 +03:00
Aliaksandr Valialkin	45f7cdc532	Revert "app/vmselect/promql: remove metric name after applying `ceil`, `floor` and `round` functions in order to be more consistent with Prometheus" This reverts commit `ac45082216`. Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users - functions, which don't change the meaning of the time series shouldn't drop metric name. Now the following functions do not drop metric names: * ceil * floor * round Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674	2020-10-12 11:40:34 +03:00
Aliaksandr Valialkin	a94825b169	Revert "app/vmselect/promql: remove metric name after applying `clamp_min` and `clamp_max` functions in order to be consistent with Prometheus" This reverts commit `bb61a4769b`. Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users - functions, which don't change the meaning of the time series shouldn't drop metric name. Now the following functions do not drop metric name: * clamp_min * clamp_max Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674	2020-10-12 11:38:27 +03:00
Aliaksandr Valialkin	f7d28bddbf	Revert "app/vmselect/promql: remove metric name from results of certain rollup functions in order to be consistent with Prometheus" This reverts commit `e5202a4eae`. Reason for revert: the previous behavior for VictoriaMetrics is easier to understand and use by users - functions, which don't change the meaning of the time series shouldn't drop metric name. Now the following functions do not drop metric name: * max_over_time * min_over_time * avg_over_time * quantile_over_time * geomean_over_time * mode_over_time * holt_winters * predict_linear Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/674	2020-10-12 11:35:18 +03:00
Aliaksandr Valialkin	b449607181	lib/backup: add MustStop() method for all remote filesystems	2020-10-09 15:32:19 +03:00
Aliaksandr Valialkin	cf5f2874cd	lib/backup/fslocal: add FS.MustStop() method for stopping bandwidth limiter	2020-10-09 15:12:03 +03:00
Aliaksandr Valialkin	68f0e00761	app/vmstorage: add `vm_rows_added_to_storage_total` metric, which shows the total number of rows added to storage since app start	2020-10-09 13:35:48 +03:00
Aliaksandr Valialkin	84227ea2fc	app/{vminsert,vmagent}: take into account all the inserted rows before relabeling in `vm_rows_inserted_total` and `vmagent_rows_inserted_total` metrics	2020-10-09 13:29:51 +03:00
Aliaksandr Valialkin	f4e8687c88	app/vmalert: accept days, weeks and years in `for:` part of config like Prometheus does Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/817	2020-10-08 20:13:15 +03:00
Aliaksandr Valialkin	6105d61d11	docs/vmagent.md: clarify `-promscrape.suppressDuplicateScrapeTargetErrors` command-line flag usage	2020-10-08 19:24:31 +03:00
Aliaksandr Valialkin	71ea4935de	lib/promscrape: add `-promscrape.suppressDuplicateScrapeTargetErrors` command-line flag in order to suppress `duplicate scrape target` errors Show also original labels for duplicate targets in error message in order to simplify debugging the issue. Now `/targets` endpoint accepts optional `show_original_labels=1` query arg, which shows original labels for each target. This may simplify debugging for target relabeling. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/651	2020-10-08 18:58:30 +03:00
Aliaksandr Valialkin	d423d73251	app/vmalert: do not pring description for all the flags on config errors The description is too big to consume by human and it just distracts humans.	2020-10-08 13:35:57 +03:00
Aliaksandr Valialkin	e9f2e2cbc9	app/vmselect/promql: add missing label filters to binary operands before query execution This implements the optimization described at https://utcc.utoronto.ca/~cks/space/blog/sysadmin/PrometheusLabelNonOptimization See also https://github.com/cortexproject/cortex/issues/3253	2020-10-07 21:15:09 +03:00
Dmitry Shihovtsev	92e5d89fc9	Fix typos in the vmalert datasource (#814 ) * Fix typos in the vmalert datasource * Fix typo in the vmalert datasource test	2020-10-07 17:59:50 +03:00
Aliaksandr Valialkin	af90b3121c	app/vmstorage: add `-finalMergeDelay` command-line flag for configuring the delay before final merge for per-month partitions after no new data is ingested to it	2020-10-07 17:35:44 +03:00
Aliaksandr Valialkin	d16dbfd639	app/vmagent: add a link to https://victoriametrics.github.io/vmagent.html from main page	2020-10-06 15:29:49 +03:00
Aliaksandr Valialkin	cfd720e772	app/victoria-metrics: add a link to https://victoriametrics.github.io/ from main page	2020-10-06 15:29:49 +03:00
Nikolay Khramchikhin	cbe3cf683b	Adds openstack sd (#811 ) * adds openstack service discovery https://github.com/VictoriaMetrics/VictoriaMetrics/issues/728 implemented hypervisors and instance discovery with openstack v3 api. Added tests for labeling and data parsing. Added token refresh. * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-10-05 16:45:33 +03:00
Aliaksandr Valialkin	59c26feefa	app/vmselect/promql: fill gaps on graphs for `range_` and `running_` functions Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/806	2020-10-02 13:59:45 +03:00
Aliaksandr Valialkin	764dc2499f	lib/storage: code cleanup after `10f2eedee0` Remove the code that uses metricIDs caches for the current and the previous hour during metricIDs search, since this code became unused after implementing per-day inverted index almost a year ago. While at it, fix a bug, which could prevent from finding time series with names containing dots (aka Graphite-like names such as `foo.bar.baz`).	2020-10-01 19:06:23 +03:00
Roman Khavronenko	daa2d1c065	vmalert: make maxIdleConnections configurable for datasource HTTP client (#797 ) Address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/795	2020-09-30 09:49:45 +03:00
Aliaksandr Valialkin	44bcda81ab	app/vmstorage: rename `vm_{big\|small}_merge_need_free_disk_space` to `vm_merge_need_free_disk_space` This simplifies alerting.	2020-09-29 22:44:19 +03:00
Aliaksandr Valialkin	a9db81c4ab	app/vmstorage: add metrics for determining whether background merges need additional disk space to complete These metrics are: * vm_small_merge_need_free_disk_space * vm_big_merge_need_free_disk_space Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686	2020-09-29 21:48:33 +03:00
Aliaksandr Valialkin	127537d631	app/vmagent/remotewrite: do not show `-remoteWrite.url` in logs if `-remoteWrite.showURL` isn't set Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/773	2020-09-29 19:49:12 +03:00
Aliaksandr Valialkin	f7636b0342	app/vmselect/graphite: do not substitute `path` and `path.` with `path..` in `/metrics/find/?format=completer` output	2020-09-29 18:03:26 +03:00
Aliaksandr Valialkin	7dc67cd883	docs/{vmbackup,vmrestore}: formatting fixes	2020-09-29 13:19:07 +03:00
Aliaksandr Valialkin	efdefbc1cb	docs/vmbackup.md: make docs about minio config more prominent	2020-09-29 13:16:04 +03:00
Aliaksandr Valialkin	9945b8c98d	docs: improve readability a bit	2020-09-29 13:03:38 +03:00
Nikolay Khramchikhin	1e679f3e0d	update vmbackup/vmrestore README usage (#794 ) * update vmbackup/vmrestore README usage https://github.com/VictoriaMetrics/VictoriaMetrics/issues/381 with minio and configuration file examples. * Apply suggestions from code review Co-authored-by: Roman Khavronenko <hagen1778@gmail.com> * added backup/restore docs changes * added example for relabelConfig flag * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com> Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>	2020-09-29 12:53:10 +03:00
Aliaksandr Valialkin	96ee276e6e	app/vmselect/prometheus: check for errors returned from bufferedwriter.Write This makes `make errcheck` happy	2020-09-29 11:37:01 +03:00
Aliaksandr Valialkin	6fdfc67620	app/vmselect/graphite: properly handle case when `/metrics/find` finds both leaf and node for the given `query=prefix.*` In this case only node must be returned with stripped dot in the end of id as carbonapi does	2020-09-29 11:01:59 +03:00
Aliaksandr Valialkin	124f78857b	app/{vminsert,vmagent}: improve data ingestion speed over a single connection Process data obtianed from a single connection on all the available CPU cores.	2020-09-28 04:13:08 +03:00
Aliaksandr Valialkin	1b3efccb24	app/vmselect: stop `/api/v1/export/*` execution if client disconnects	2020-09-27 23:53:13 +03:00
Aliaksandr Valialkin	95688cbfc5	all: add native format for data export and import The data can be exported via [/api/v1/export/native](https://victoriametrics.github.io/#how-to-export-data-in-native-format) handler and imported via [/api/v1/import/native](https://victoriametrics.github.io/#how-to-import-data-in-native-format) handler.	2020-09-27 19:54:07 +03:00
Aliaksandr Valialkin	c00627c103	app/vminsert: code prettifying	2020-09-26 04:13:18 +03:00
Aliaksandr Valialkin	b6a976b98d	app/vmagent: reduce memory usage when importing data via /api/v1/import Previously vmagent could use big amounts of RAM when each ingested JSON line contained many samples.	2020-09-26 04:10:24 +03:00
Aliaksandr Valialkin	82973f8ae7	Revert "lib/storage: remove unused `fetchData` arg from BlockRef.MustReadBlock" This reverts commit `bab6a15ae0`. Reason for revert: the `fetchData` arg is used in cluster branch. Leaving this arg in master branch makes smaller the diff with cluster branch.	2020-09-24 22:44:23 +03:00
Aliaksandr Valialkin	bab6a15ae0	lib/storage: remove unused `fetchData` arg from BlockRef.MustReadBlock This arg became unused after `23bdc1f107`	2020-09-24 20:48:40 +03:00
Aliaksandr Valialkin	23bdc1f107	app/vmselect/netstorage: do not spend CPU time on unpacking empty blocks during `/api/v1/series` calls	2020-09-24 20:18:10 +03:00
Aliaksandr Valialkin	c584aece38	app/vmselect/promql: properly limit implicitly set rollup window to `-search.maxStalenessInterval` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/784	2020-09-23 23:23:59 +03:00
Aliaksandr Valialkin	2985077c35	all: consistently use "%w" formatting in fmt.Errorf for wrapped errors	2020-09-23 22:46:34 +03:00
Aliaksandr Valialkin	27500d7d4c	app/vmselect/prometheus: code cleanup after `3ba507000c`	2020-09-23 13:04:17 +03:00
Aliaksandr Valialkin	3ba507000c	app/vmselect/prometheus: return timestamps from `/api/v1/query`, which match the `time` query arg Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720	2020-09-23 12:58:48 +03:00
Aliaksandr Valialkin	bed25e3c24	app/vmselect/netstorage: properly pre-allocate space for sbs	2020-09-22 23:49:55 +03:00
Aliaksandr Valialkin	09b0f7c202	app/vmselect/netstorage: release search resources on timeout errors Previously these resources weren't released, which could lead to resource leaks.	2020-09-22 22:57:38 +03:00
Aliaksandr Valialkin	3b1e3a03e0	app/vmselect: make sure the request doesnt wait in pending queue more than the configured timeout Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711	2020-09-22 01:23:19 +03:00
Aliaksandr Valialkin	94f7d00537	docs/vmagent.md: typo fix	2020-09-21 21:49:22 +03:00
Aliaksandr Valialkin	00b5145c69	app/vmselect/searchutils: fixed tests after `2eb72e09ab`	2020-09-21 21:31:38 +03:00
Aliaksandr Valialkin	2eb72e09ab	app/vmselect: use `time` value rounded to seconds if it isnt passed to `/api/v1/query` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/720	2020-09-21 21:24:40 +03:00
Nikolay Khramchikhin	312fead9a2	Add improvements to ec2_sd_discovery (#775 ) * Add improvements to ec2 discovery https://github.com/VictoriaMetrics/VictoriaMetrics/issues/771 role_arn support with aws sts instance iam_role support refreshing temporary tokens * Apply suggestions from code review Co-authored-by: Roman Khavronenko <hagen1778@gmail.com> * changed implementation, removed tests, clean up code * moved endpoint builder into getEC2APIResponse Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>	2020-09-21 16:04:15 +03:00
Aliaksandr Valialkin	1e1a27d803	app/vmalert: remove unneeded UTC() call UTC() doesn't change the underlying timestamp, so the call isn't needed here	2020-09-21 15:55:59 +03:00
Roman Khavronenko	5dffc7a553	vmalert: add support for `datasource.lookback` flag (#779 ) New datasource flag `datasource.lookback` defines how far to look into past when evaluating queries. Address https://github.com/VictoriaMetrics/VictoriaMetrics/issues/668	2020-09-21 15:53:49 +03:00
Roman Khavronenko	82c3bbce34	vmalert: fix the typo in error message (#782 ) The error will be always nil so no sense in printing it.	2020-09-21 11:34:23 +03:00
Aliaksandr Valialkin	d50165ad59	app/vmagent: increase default value for `-remoteWrite.queues` from 1 to 4, since it has been appeared that many users hit this limit	2020-09-18 14:21:54 +03:00
Aliaksandr Valialkin	98d1cd0971	app/vmselect/graphite: return proper results `/metrics/find?query=foo.*.bar` according to Graphite Metrics API	2020-09-18 11:00:00 +03:00
Aliaksandr Valialkin	7a134b0fd7	app/vmstorage: added `-forceMergeAuthKey` command-line flag for protecting `/internal/force_merge` endpoint	2020-09-17 14:21:53 +03:00
Aliaksandr Valialkin	1f33dd717f	lib/storage: add `/internal/force_merge` handler for running forced compactions on historical per-month partitions This may be useful for freeing up storage space after time series deletion. See https://victoriametrics.github.io/#force-merge for more details. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/686	2020-09-17 12:20:40 +03:00
Aliaksandr Valialkin	ab53cb6f7b	app/vmagent: substitute `-remoteWrite.url` with `secret-url` value in logs, since it may contain sensitive info such as passwords or auth tokens Pass `-remoteWrite.showURL` command-line flag in order to see real `-remoteWrite.url` values in logs and at `/metrics` page. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/773	2020-09-16 22:36:25 +03:00
Aliaksandr Valialkin	9f79bcf64a	app/vmselect: improve description for -search.maxQueryDuration	2020-09-16 21:15:41 +03:00
Aliaksandr Valialkin	1fec47a289	app/vmselect/netstorage: reduce memory usage when the time range from query touches big number of samples per each time series	2020-09-15 21:08:28 +03:00
Aliaksandr Valialkin	8c3d7c1a59	app/vmselect: typo fix in -search.maxStalenessInterval description	2020-09-15 14:24:27 +03:00
Aliaksandr Valialkin	0e533d1a9c	app/vmselect/promql: support composite durations like Prometheus 2.21 does The following durations are supported now: `1h5m35s` or `1s543ms` See https://github.com/prometheus/prometheus/releases/tag/v2.21.0 and https://github.com/prometheus/prometheus/pull/7713	2020-09-11 23:39:13 +03:00
Roman Khavronenko	6ad6480400	vmalert: add Group name as label to generated alerts and timeseries (#761 ) Solves #611	2020-09-11 20:52:56 +01:00
Roman Khavronenko	4cdffb04a4	vmalert: update groups on config reload only if changes detected (#759 ) On config reload event `vmalert` reloads configuration for every group. While it works for simple configurations, the more complex and heavy installations may suffer from frequent config reloads. The change introduces the `checksum` field for every group and is set to md5 hash of yaml configuration. The checksum will change if on any change to group definition like rules order or annotation change. Comparing the `checksum` field on config reload event helps to detect if group should be updated. The groups update is now done concurrently, so reload duration will be limited by the slowest group now. Partially solves #691 by improving config reload speed.	2020-09-11 20:14:30 +01:00
Aliaksandr Valialkin	ca856284e4	app/vmagent: allow setting multiple identical `-remoteWrite.url` values This may be useful when each url is authenticated via different `-remoteWrite.basicAuth.username`. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/755	2020-09-11 15:17:22 +03:00
Aliaksandr Valialkin	a2f647d142	app/vmselect/prometheus: typo fix in the description for `-search.latencyOffset` command-line flag	2020-09-11 14:16:46 +03:00
Aliaksandr Valialkin	2380e9b017	app/{vminsert,vmagent}: allow passing timestamp via `timestamp` query arg when ingesting data to `/api/v1/import/prometheus` See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/750	2020-09-11 13:27:14 +03:00
Aliaksandr Valialkin	f0005c3007	app/vmselect: move Deadline from netstorage to searchutils This removes dependency on netstorage from searchutils.	2020-09-11 13:27:13 +03:00
Aliaksandr Valialkin	2114179e19	app/vmselect: substitute inf values at smooth_exponential with the previous values Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/757	2020-09-11 12:24:14 +03:00
Aliaksandr Valialkin	204ec415b4	app/vmselect: skip infinite values when calculating smooth_exponential Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/757	2020-09-11 11:29:58 +03:00
Aliaksandr Valialkin	8a8b5a73d3	app/vmselect/graphite: typo fix in label name for vm_request_duration_seconds metric	2020-09-11 01:58:28 +03:00
Aliaksandr Valialkin	f6bc608e86	app/vmselect: initial implementation of Graphite Metrics API See https://graphite-api.readthedocs.io/en/latest/api.html#the-metrics-api	2020-09-11 00:30:01 +03:00
Aliaksandr Valialkin	9d8fdff6c5	lib/storage: reuse timestamp blocks for adjancent metric blocks with identical timestamps This should reduce disk space usage when scraping targets containing metrics with identical names such as `node_cpu_seconds_total`, histograms, quantiles, etc. Expose `vm_timestamps_blocks_merged_total` and `vm_timestamps_bytes_saved_total` metrics for monitoring the effectiveness of timestamp blocks merging.	2020-09-09 23:59:32 +03:00
Aliaksandr Valialkin	d7c04db1fc	docs: sync docs for vmalert, vmauth, vmbackup and vmrestore	2020-09-09 21:10:34 +03:00
Aliaksandr Valialkin	11eaa37111	docs/vmagent.md: clarified the case when `-remoteWrite.queues` must be tuned Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/745	2020-09-08 20:15:27 +03:00
Aliaksandr Valialkin	62919eaf7e	app/vmselect/promql: go fmt	2020-09-08 15:19:59 +03:00
Aliaksandr Valialkin	e6da63dffe	app/vmselect/promql: adjust `integrate()` calculations to be more similar to calculations from InfluxDB: attempt #2 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/701	2020-09-08 14:35:50 +03:00
Aliaksandr Valialkin	8e85b56737	app/vmselect/promql: adjust `integrate()` calculations to be more similar to calculations from InfluxDB Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/701	2020-09-08 14:23:39 +03:00
Aliaksandr Valialkin	c0343a661b	app/vmselect/promql: increase floating point calculations accuracy by dividing by `1e3` instead of multiplying by `1e-3`	2020-09-08 14:00:47 +03:00
Aliaksandr Valialkin	f4e7e5fb90	app/vmselect/promql: add `count_le_over_time(m[d], le)` and `count_gt_over_time(m[d], gt)` functions These functions returns the number of raw samples that don't exceed `le` or are bigger than `gt`. These functions are complement to already existing `share_le_over_time(m[d], le)` and `share_gt_over_time(m[d], gt)`.	2020-09-03 15:29:10 +03:00
Aliaksandr Valialkin	e706e59d49	app/vmselect: unconditionally align time range boundaries to step for subqueries as Prometheus does	2020-09-03 13:29:50 +03:00
Aliaksandr Valialkin	ddabc13796	app/vmagent: properly flush big blocks of data Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/741 Thanks to @IceRain00 for the investigation and initial attempt to fix the issue at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/742	2020-09-03 12:12:39 +03:00
Aliaksandr Valialkin	7a839b461f	app/vmagent: fix data race when accessing writeRequest.lastFlushTime	2020-09-03 12:12:37 +03:00
Nikolay Khramchikhin	764b3d4fda	changed vmalert behaviour (#738 ) * VMAlert start with empty rules dir There are some applications (operator for instance), that generates alerts configuration at runtime and vmalert must start correctly without rules to support this behaviour. Later application will add rules files and send SIGHUP to vmalert, which will trigger reading rules files and start rules exectuion. Removing rules files with SIGHUP signal must stop rules execution and vmalert will wait for new rules. * imports sorted * added test cases for empty rules, removed blank line * fixed imports conflict * updated tests	2020-09-03 11:04:42 +03:00
Aliaksandr Valialkin	5f16ceb294	app/vmalert: imrovements over `3f932c2db1`	2020-09-03 01:00:55 +03:00
DexterZhang	3f932c2db1	feat: spread load of rule evaluation by group when starting new groups (#724 ) * feat: spread load of rule evaluation by group when starting new groups * review: reduce the resulting diff. * Update app/vmalert/group.go Co-authored-by: Roman Khavronenko <hagen1778@gmail.com> Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com> Co-authored-by: Roman Khavronenko <hagen1778@gmail.com>	2020-09-03 00:58:54 +03:00
Aliaksandr Valialkin	f41b36bb9a	app/{vminsert,vmagent}: allow adding extra labels when importing data via Prometheus, CSV and JSON line formats Extra labels may be added to the imported data by passing `extra_label=name=value` query args. Multiple query args may be passed in order to add multiple extra labels. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/719	2020-09-02 19:43:21 +03:00
Aliaksandr Valialkin	5af777469a	app/vmagent: log unsuccessful attempt number when sending data to -remoteWrite.url	2020-08-30 21:40:22 +03:00
Aliaksandr Valialkin	2149733bd2	app/vmagent: apply sane limits to `-remoteWrite.queues` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/707	2020-08-30 21:25:37 +03:00
Roman Khavronenko	10601bc652	vmalert: update `-rule` flag description to enforce quotes using (#709 ) Description for `-rule` flag uses as example specific chars like asterisks which could be interpreted wrong by different shells. To avoid this, description now contains quoted flag values. See also #708	2020-08-20 22:36:38 +01:00
Roman Khavronenko	f5f59896ec	lib/decimal: rename `significant decimal digits` to `significant figures` (#698 ) The previous notion was inconsistent with what `decimal.Round` does. According to [wiki](https://en.wikipedia.org/wiki/Significant_figures) rounding applied to all significant figures, not just decimal ones.	2020-08-16 17:21:35 +03:00
Aliaksandr Valialkin	147c35ebd4	all: allow using `KB`, `MB`, `GB`, `KiB`, `MiB` and `GiB` suffixes in command-line flag values related to byte sizes or byte rates	2020-08-16 17:05:52 +03:00
Aliaksandr Valialkin	7d89fafe1a	app/vmselect/promql: allow passing multiple args to aggregate functions such as `avg(q1, q2, q3)`	2020-08-15 01:15:09 +03:00
Aliaksandr Valialkin	cd96248480	docs/vmagent.md: mention that gaps in remote storage may appear if vmagent cannot keep up with data ingestion	2020-08-14 20:47:57 +03:00
Aliaksandr Valialkin	184670fb9b	docs: update docs	2020-08-14 19:13:42 +03:00
Aliaksandr Valialkin	576da0fe46	app/{vminsert,vmagent}: improve documentation for `-influxListenAddr` command-line flag	2020-08-14 18:04:44 +03:00
Aliaksandr Valialkin	c402265e88	all: support `%{ENV_VAR}` placeholders in yaml configs in all the vm* components Such placeholders are substituted by the corresponding environment variable values. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/583	2020-08-13 17:15:25 +03:00
Aliaksandr Valialkin	ef7e2af8f5	app: respect CPU limits set via cgroups Update GOMAXPROCS to limits set via cgroups. This should reduce CPU trashing and reduce memory usage for cases when VictoriaMetrics components run in containers with CPU limits. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/685	2020-08-11 22:59:19 +03:00
Aliaksandr Valialkin	e969ef2639	app/vmselect: reduce memory usage when exporting time series with big number of samples via `/api/v1/export` if `max_rows_per_line` is set to non-zero value Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/685	2020-08-10 20:57:36 +03:00
Aliaksandr Valialkin	8adba82c02	app/vmselect/netstorage: vary batch size for data unpacking depending on the available CPU cores This should reduce contention on the channel with unpack work for systems with high number of CPU cores	2020-08-10 15:16:42 +03:00
Aliaksandr Valialkin	f3d33e23c9	app/vmstorage: improve error logging when the request times out	2020-08-10 13:23:26 +03:00
Roman Khavronenko	0be5b09fb4	app/vmalert: extend metrics set exported by `vmalert` #573 (#654 ) * app/vmalert: extend metrics set exported by `vmalert` #573 New metrics were added to improve observability: + vmalert_alerts_pending{alertname, group} - number of pending alerts per group per alert; + vmalert_alerts_acitve{alertname, group} - number of active alerts per group per alert; + vmalert_alerts_error{alertname, group} - is 1 if alertname ended up with error during prev execution, is 0 if no errors happened; + vmalert_recording_rules_error{recording, group} - is 1 if recording rule ended up with error during prev execution, is 0 if no errors happened; * vmalert_iteration_total{group, file} - now contains group and file name labels. This should improve control over specific groups; * vmalert_iteration_duration_seconds{group, file} - now contains group and file name labels. This should improve control over specific groups; Some collisions for alerts and recording rules are possible, because neither group name nor alert/recording rule name are unique for compatibility reasons. Commit contains list of TODOs for Unregistering metrics since groups and rules are ephemeral and could be removed without application restart. In order to unlock Unregistering feature corresponding PR was filed - https://github.com/VictoriaMetrics/metrics/pull/13 * app/vmalert: extend metrics set exported by `vmalert` #573 The changes are following: * add an ID label to rules metrics, since `name` collisions within one group is a common case - see the k8s example alerts; * supports metrics unregistering on rule updates. Consider the case when one rule was added or removed from the group, or the whole group was added or removed. The change depends on https://github.com/VictoriaMetrics/metrics/pull/16 where race condition for Unregister method was fixed.	2020-08-09 09:41:29 +03:00
ofen	81746d14b9	401 Unauthorize HTTP error added (#681 ) 401 Unauthorize HTTP error added to trigger browser credentials pop-up promt [RFC 7235 https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication]	2020-08-09 09:38:41 +03:00
Aliaksandr Valialkin	716754fae6	app/vmselect/promql: properly handle `-n^m` like Prometheus does `-n^m` must be handled as `-(n^m)` instead of `(-n)^m`. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/675	2020-08-07 07:42:18 +03:00
Aliaksandr Valialkin	bb61a4769b	app/vmselect/promql: remove metric name after applying `clamp_min` and `clamp_max` functions in order to be consistent with Prometheus This improves VictoriaMetrics score at https://promlabs.com/promql-compliance-test-results-victoriametrics/	2020-08-06 23:42:37 +03:00
Aliaksandr Valialkin	ac45082216	app/vmselect/promql: remove metric name after applying `ceil`, `floor` and `round` functions in order to be more consistent with Prometheus This improves VictoriaMetrics score at https://promlabs.com/promql-compliance-test-results-victoriametrics/	2020-08-06 23:34:37 +03:00
Aliaksandr Valialkin	e5202a4eae	app/vmselect/promql: remove metric name from results of certain rollup functions in order to be consistent with Prometheus Rollup functions: - avg_over_time - min_over_time - max_over_time - quantile_over_time This improves VictoriaMetrics results at https://promlabs.com/promql-compliance-test-results-victoriametrics/	2020-08-06 23:29:13 +03:00
Aliaksandr Valialkin	68e4f40a72	app/vmselect: properly handle PromQL queries like `scalar1 < metric < scalar2` like Prometheus does This fixes some cases from https://promlabs.com/promql-compliance-test-results-victoriametrics/	2020-08-06 23:21:03 +03:00
Aliaksandr Valialkin	bc8381613d	app/vmselect: reduce memory allocations by pre-allocatin memory for time series map and for a list of time series names	2020-08-06 19:17:58 +03:00
Aliaksandr Valialkin	3f85c06b65	app/vmselect/netstorage: reduce CPU contention when upacking time series blocks by unpacking batches of such blocks instead of a single block This should improve query performance on systems with big number of CPU cores (16 and more)	2020-08-06 17:50:17 +03:00
Aliaksandr Valialkin	d20c2156e4	app/vmselect/netstorage: reduce contention on unpackworkCh and timeseriesWorkCh for multi-CPU system by providing more capacity for these chans	2020-08-06 17:22:48 +03:00
Aliaksandr Valialkin	dbbdfbe7ee	app/vmstorage: rename `vm_cache_size_entries{type="storage/prefetchedMetricIDs"}` to `vm_cache_entries{type="storage/prefetchedMetricIDs"}` to be consistent with other `vm_cache_entries` metrics	2020-08-06 16:34:24 +03:00
Aliaksandr Valialkin	8f16388428	lib/storage: limit the number of concurrent calls to storage.searchTSIDs to GOMAXPROCS*2 This should limit the maximum memory usage and reduce CPU trashing on vmstorage when multiple heavy queries are executed. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648	2020-08-05 18:30:07 +03:00
Aliaksandr Valialkin	c25b0c2cd5	app/vmagent: tune http client for sending data to remote storage in order to disable closing keep-alive connections Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/663	2020-08-04 21:00:29 +03:00
Aliaksandr Valialkin	5d0c37bec0	app/vmselect: use warning level instead of info level for logging slow queries that take longer than `-search.logSlowQueryDuration`	2020-08-04 20:25:35 +03:00
Aliaksandr Valialkin	e6eee2bebf	app/vmselect/promql: add zscore-related functions: `zscore_over_time(m[d])` and `zscore(q) by (...)`	2020-08-03 21:52:18 +03:00
Aliaksandr Valialkin	509d12643b	app/vmselect: show `X-Forwarded-For` contents on `/api/v1/status/active_queries` page Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/659	2020-07-31 20:05:18 +03:00
Aliaksandr Valialkin	d01f3c1943	all: add mssing APP_NAME to vm*-GOARCH builds	2020-07-31 13:42:18 +03:00
Aliaksandr Valialkin	3f498cf2dc	docs/{vmagent,vmalert}: add instruction on how to build for ARM	2020-07-31 09:25:22 +03:00
Aliaksandr Valialkin	f0c678c41b	app/vmselect: do not adjust `start` and `end` query args passed to `/api/v1/query_range` when `-search.disableCache` command-line flag is set Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/563	2020-07-30 23:14:37 +03:00
Aliaksandr Valialkin	e7959094f6	lib/storage: remove prioritizing of merging small parts over merging big parts, since it doesn't work as expected The prioritizing could lead to big merge starvation, which could end up in too big number of parts that must be merged into big parts. Multiple big merges may be initiated after the migration from v1.39.0 or v1.39.1. It is OK - these merges should be finished soon, which should return CPU and disk IO usage to normal levels. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/618	2020-07-30 19:57:27 +03:00
Aliaksandr Valialkin	328b52e5ff	app/vmselect/promql: return non-empty value from `rate_over_sum(m[d])` even if a single data point is located in the given `[d]` window Just divide the data point value by the window duration in this case.	2020-07-29 12:37:58 +03:00
Aliaksandr Valialkin	700737c181	app/vmselect/promql: remove rollupFuncArg.realPrevValue handling, since the corner case in `increase()` is handled in another way now See `e00cfc854d` for the approach used now.	2020-07-29 12:37:58 +03:00
Aliaksandr Valialkin	2f735f112d	app/vmselect/promql: fill gaps with 0 in `rate_over_sum` response when the last value before the selected time window isnt empty	2020-07-29 12:37:58 +03:00
Aliaksandr Valialkin	0f63da3698	app/{vmagent,vminsert}: properly preserve `db` tag from query string passed to Influx line protocol query Previously `db` tag from the query string wasn't added to metrics after encountering `db` tag in the Influx line Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/653	2020-07-28 21:25:19 +03:00
Aliaksandr Valialkin	62ed38c6f0	app/vmagent/remotewrite: add missing `resp.Body.Close()` after pushing data to remote storage Missing body close could disable HTTP keep-alive connections. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/653	2020-07-28 21:00:15 +03:00
Aliaksandr Valialkin	79c30cf4cb	app/vmselect: show query origin (aka remote_addr or client address) on the `/api/v1/status/active_queries` page for every query	2020-07-28 15:13:08 +03:00
Roman Khavronenko	2f1e7298ce	app/vmalert: support `external.label` to specify global labelset for all rules #622 (#652 ) `external.label` flag supposed to help to distinguish alert or recording rules source in situations when more than one `vmalert` runs for the same datasource or AlertManager.	2020-07-28 14:20:31 +03:00
Aliaksandr Valialkin	0da202023b	app/vmselect/promql: return empty values from `group()` if all the time series have no values at the given timestamp This aligns `group()` behaviour to Prometheus	2020-07-28 13:40:11 +03:00
Aliaksandr Valialkin	cb8c6908dc	app/vmagent/remotewrite: create new request on failure to send a block of data to remote storage Previously the request body was already consumed before the retry, so this led to the following error: http: ContentLength=... with Body length 0	2020-07-27 17:32:46 +03:00
Aliaksandr Valialkin	894dcb7c1c	app/vmselect/promql: improve further the accuracy of buckets_limit() function The accuracy is increased by mergin the smallest bucket with the smallest adjacent bucket.	2020-07-26 12:10:13 +03:00
Aliaksandr Valialkin	215eba0b82	app/vminsert: flush bufs if needed after the current row is added Previously the data for the added row could be overwritten by the flush before the row addition is complete.	2020-07-26 12:10:11 +03:00
Aliaksandr Valialkin	edb1eca6f1	app/vmselect/promql: avoid dropping `inf` bucket in `buckets_limit` The `le="inf"` bucket must be preserved in order to maintain the maximum level of accuracy.	2020-07-25 17:00:36 +03:00
Aliaksandr Valialkin	97b6f5d223	app/vmselect/promql: optimize `buckets_limit(k, buckets)` for big number of buckets	2020-07-25 13:24:03 +03:00
Aliaksandr Valialkin	a090627059	app/vminsert: limit memory usage when ingesting data in big packets	2020-07-24 23:32:40 +03:00
Aliaksandr Valialkin	bb161497cf	app/vmselect/promql: improve the accuracy of `buckets_limit(k, buckets)` function Now it properly merges the bucket with the previous bucket after deletion.	2020-07-24 17:07:49 +03:00
Aliaksandr Valialkin	994fa2f3bf	app/vmselect/promql: add `buckets_limit(k, buckets)` function, which limits the number of buckets per time series to `k` This function works with both Prometheus-style and VictoriaMetrics-style buckets. The function removes buckets with the lowest values in order to reserve the highest precision. The function is useful for building heatmaps in Grafana from too big number of buckets.	2020-07-24 16:13:53 +03:00
Aliaksandr Valialkin	e151c5c644	app/vmselect: fix tests for rate_over_sum	2020-07-24 02:35:28 +03:00
Aliaksandr Valialkin	3107c633e3	app/vmselect/promql: typo fix after `3e557c9861`	2020-07-24 02:15:58 +03:00
Aliaksandr Valialkin	3e557c9861	app/vmselect/promql: add `rate_over_sum(m[d])` function to MetricsQL, which returns rate over sum of `m` values over `d` duration Something like `sum_over_time(m[d]) / d`, but more accurate.	2020-07-24 01:17:42 +03:00
Aliaksandr Valialkin	b1f6843bd0	app/vmselect/promql: allow setting `[d]` window smaller than the interval between raw points for `avg_over_time` This makes `avg_over_time` behavior consistent with `sum_over_time` and `count_over_time` behaviors. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/636	2020-07-23 22:25:43 +03:00
Aliaksandr Valialkin	039c9d2441	lib/storage: respect `-search.maxQueryDuration` when searching for time series in inverted index Previously the time spent on inverted index search could exceed the configured `-search.maxQueryDuration`. This commit stops searching in inverted index on query timeout.	2020-07-23 21:21:42 +03:00
Aliaksandr Valialkin	2a45871823	lib/storage: add more fine-grained pace limiting for search	2020-07-23 19:26:08 +03:00
Aliaksandr Valialkin	461481fbdf	app/vmselect/netstorage: protect from too smart compiler, which may break memory usage optimization in ProcessSearchQuery	2020-07-23 17:54:01 +03:00
Aliaksandr Valialkin	4c8b49b193	app/vminsert: export `vm_relabel_metrics_dropped_total` metric that shows the number of metrics dropped due to relabeling	2020-07-23 14:57:53 +03:00
Aliaksandr Valialkin	e79de9774b	app/vmselect: typo fix after `34563916f7`	2020-07-23 14:12:28 +03:00
Aliaksandr Valialkin	34563916f7	app/vmselect: reduce memory usage when querying big number of time series with long labels Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/646	2020-07-23 13:53:52 +03:00
Aliaksandr Valialkin	9257eee982	app/vminsert: do not call ApplyRelabeling function if relabeling is disabled This should reduce CPU usage a bit when `-relabelConfig` isn't set	2020-07-23 13:39:44 +03:00
Aliaksandr Valialkin	6f05c4d351	lib/storage: improve prioritizing of data ingestion over querying Prioritize also small merges over big merges. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/291 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/648	2020-07-23 13:23:36 +03:00
Aliaksandr Valialkin	2f612e0c67	app/vminsert: fix relabeling for metrics ingested via Influx line protocol Previously the enabled relabeling with `-relabelConfig` command-line flag could result in missing labels if a single Influx line protocol message contains multiple field values. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/638	2020-07-23 13:23:14 +03:00
Aliaksandr Valialkin	4f526cc816	app/vmselect/prometheus: support `d`, `w` and `y` suffixes for durations passed to `step` in `/api/v1/query_range` like Prometheus does Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/641	2020-07-22 16:26:18 +03:00
Aliaksandr Valialkin	dfb113f175	app/vmselect/netstorage: reduce memory allocations when unpacking time series data by using a pool for unpackWork entries This should slightly reduce load on GC when processing queries that touch big number of time series. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/646 according to the provided memory profile there.	2020-07-22 15:03:57 +03:00
Aliaksandr Valialkin	31ae5911a8	app/vmagent: add `-remoteWrite.decimalPlaces` command-line flag, which may be used for reducing disk space usage on the remote storage	2020-07-21 21:55:32 +03:00
Aliaksandr Valialkin	caa2952aa6	app/vmselect: take into account the time spent in wait queue before query execution as time spent on the query	2020-07-21 19:00:09 +03:00
Aliaksandr Valialkin	e00cfc854d	app/vmselect/promql: skip the first value in time series passed to `increase()` if it exceeds by more than 10x the delta between the next value and the first value This should prvent from inflated `increase()` results for time series that start from big initial values. Such cases may occur when a label value changes in a metric without counter reset.	2020-07-21 17:24:10 +03:00
Aliaksandr Valialkin	b9c8f6bf34	app/vmselect: log the total available memory for concurrent requests on `not enough memory` errors This should simplify root cause analysis	2020-07-20 19:51:40 +03:00
Aliaksandr Valialkin	ad6290953c	app/vmagent: add `-remoteWrite.proxyURL` command-line option This option allows writing data to `-remoteWrite.url` via http, https or socks5 proxy. This is similar to `proxy_url` option in `remote_write` section of Prometheus. See https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write	2020-07-20 19:28:49 +03:00
Aliaksandr Valialkin	efcbb51968	docs/vmagent.md: sync with app/vmagent/README.md	2020-07-20 17:08:34 +03:00
Roman Khavronenko	ed0df37ee7	app/vmagent: mention grafana dashboard in README (#639 )	2020-07-20 17:07:27 +03:00
Aliaksandr Valialkin	11be704109	app/vmagent/remotewrite: allow passing empty `-remoteWrite.urlRelabelConfig` entries	2020-07-20 15:49:27 +03:00
Aliaksandr Valialkin	5a4675c528	app/vmselect/prometheus: do not return time series with empty list of datapoints from /api/v1/query_range This matches Prometheus behaviour. This should fix https://github.com/jacksontj/promxy/issues/329	2020-07-20 15:31:21 +03:00
Aliaksandr Valialkin	ecb1b2564a	app/vmselect/promql: add `mode()` aggregate function	2020-07-20 15:31:20 +03:00
Aliaksandr Valialkin	b35cb293f5	lib/httpserver: log remote address in error message from `httpserver.Errorf` This should improve detection of the root cause of errors. Thanks to Anant for the idea.	2020-07-20 14:11:22 +03:00
Aliaksandr Valialkin	1c641037e8	app/vmselect/promql: add `mode_over_time(m[d])` function See https://en.wikipedia.org/wiki/Mode_(statistics) and https://stackoverflow.com/questions/61134078/promql-query-to-return-the-value-from-a-range-vector-which-occurs-maximum-no-of	2020-07-17 18:28:45 +03:00
Aliaksandr Valialkin	6b5ad535ae	app/vmselect/promql: optimize `group(rollup(m))` calculations	2020-07-17 16:47:16 +03:00
Aliaksandr Valialkin	8949d65ad1	app/vmselect/promql: check that `any()` doesn't touch metric name	2020-07-17 16:23:21 +03:00
Aliaksandr Valialkin	aa5d88055d	app/vmselect/promql: add `group()` aggregate function to MetricsQL This function has been added in Prometheus 2.20. See https://github.com/prometheus/prometheus/pull/7480	2020-07-17 15:17:55 +03:00
Aliaksandr Valialkin	df01836818	app/vmselect/promql: keep all labels for time series from `any()` call	2020-07-17 15:17:54 +03:00
Aliaksandr Valialkin	8c14ca93fa	app/vminsert/influx: properly handle the case when certain labels with empty values are removed by ApplyRelabeling() call Previously this could lead to `out of range` panic	2020-07-17 00:07:06 +03:00
Aliaksandr Valialkin	e4e1cd1de2	app/vmselect: fix `nil pointer dereference` panic when unsuccessfully querying `vmstorage`	2020-07-16 19:15:43 +03:00
Aliaksandr Valialkin	ed7580ad22	app/vmalert: consistently use "%w" instead of "%s" in `fmt.Errorf` when wrapping errors	2020-07-15 13:56:47 +03:00
Aliaksandr Valialkin	328814ee60	docs/vmagent.md: make filtering rules for init container pods less confusing	2020-07-14 20:32:47 +03:00
Aliaksandr Valialkin	b442a42d8e	app/vmagent/remotewrite: return proper value from `tssRelabelPool.New` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/599	2020-07-14 14:29:20 +03:00
Aliaksandr Valialkin	4081e2295e	app/{vminsert,vmagent}: add `-influxSkipMeasurement` command-line flag for using field name as metric name See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/626	2020-07-14 14:17:24 +03:00
Aliaksandr Valialkin	25f80d320b	app/vmselect/prometheus: do not adjust last points in time series with timestamps exceeding the current time Such timestamps usually mean that the query contains `offset`. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/625	2020-07-14 12:52:16 +03:00
Aliaksandr Valialkin	55d83e777d	app/vmselect/prometheus: minimize the diff for the change `1033dc7e2a` over `619b0a25c9`	2020-07-13 21:40:38 +03:00
faceair	1033dc7e2a	fix empty response template (#617 )	2020-07-13 21:31:19 +03:00
Aliaksandr Valialkin	619b0a25c9	docs/vmagent.md: sync with app/vmagent/README.md	2020-07-13 21:25:11 +03:00
ofen	666c795b98	Update README.md (#621 ) Troubleshooting section updated to help out with duplicate targets detection	2020-07-13 21:18:54 +03:00
Aliaksandr Valialkin	a730b3f6a1	app/vmagent: fix data race when multiple `-remoteWrite.urlRelabelConfig` options are set Previously multiple goroutines could access remoteWriteCtx.tss concurrently, which could lead to data race and improper relabeling. Now each goroutine has its own copy of tss during relabeling. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/599	2020-07-10 15:16:59 +03:00
Aliaksandr Valialkin	508ad46e0e	app/vmagent/remotewrite: typo fix in `-remoteWrite.showURL` help message	2020-07-10 14:07:08 +03:00
Aliaksandr Valialkin	cba820e390	app/{vminsert,vmagent}: add ability to import data in Prometheus exposition format via `/api/v1/import/prometheus`	2020-07-10 12:14:07 +03:00
Aliaksandr Valialkin	9c350bc20d	app/vmselect/promql: add missing tests for `ifnot` binary operation	2020-07-09 13:24:06 +03:00
Aliaksandr Valialkin	256fd9a87e	app/vmselect/promql: refactor implementations for `and` and `unless` binary operations, so they are closer to `or` implementation	2020-07-09 13:05:55 +03:00
Aliaksandr Valialkin	2d9b3ad5b3	app/vmselect/promql/active_queries.go: simplify code a bit by inlining getNextActiveQueryID function	2020-07-09 11:18:30 +03:00
Aliaksandr Valialkin	742da690f4	app/vmselect: add `/api/v1/status/active_queries` page with the list of currently running queries This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/598 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/575	2020-07-08 18:55:38 +03:00
DexterZhang	99f54e44ff	feat(vmselect): add current running query list, add ability for getting the running query info and killing running query for master branch (#598 )	2020-07-08 18:52:55 +03:00
Aliaksandr Valialkin	6daa5f7500	lib/storage: prioritize data ingestion over heavy queries Heavy queries could result in the lack of CPU resources for processing the current data ingestion stream. Prevent this by delaying queries' execution until free resources are available for data ingestion. Expose `vm_search_delays_total` metric, which may be used in for alerting when there is no enough CPU resources for data ingestion and/or for executing heavy queries. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/291	2020-07-05 19:42:05 +03:00
Roman Khavronenko	703def4b2e	app/vmalert: add retries to remotewrite (#605 ) * app/vmalert: add retries to remotewrite Remotewrite pkg now does limited number of retries if write request failed. This suppose to make vmalert state persisting more reliable. New metrics were added to remotewrite in order to track rows/bytes sent/dropped. defaultFlushInterval was increased from 1s to 5s for sanity reasons. * fix * wip * wip * wip * fix bits alignment bug for 32-bit systems * fix mistakenly dropped field	2020-07-05 18:46:52 +03:00
Aliaksandr Valialkin	de137aef98	app/victoria-metrics: fix tests after the commit `acf828a759`	2020-07-05 18:24:41 +03:00
Aliaksandr Valialkin	acf828a759	app/vmselect/prometheus: small fixes on top of `8bb762124a`	2020-07-05 18:17:06 +03:00
faceair	8bb762124a	fix adjust last points avoid influence earlier value (#606 )	2020-07-05 17:56:54 +03:00
Ween	d59cdbe90c	[VMAlert] Fix error log when remoteWrite queue size is full (#602 ) * Fix Auto metrics relabeled errors * Finalize auto-genenated Labels * Fix Test Errors * fix error logs when queue is full Co-authored-by: xinyulong <xinyulong@kuaishou.com>	2020-07-03 16:49:37 +03:00
Aliaksandr Valialkin	0b2086b7a5	app/vminsert: prevent from adding and/or selecting labels with empty values Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/600	2020-07-02 23:14:11 +03:00
Aliaksandr Valialkin	8f628cd805	app/victoria-metrics: removed debug log message when `-selfScrapeInterval` is set	2020-07-02 20:39:41 +03:00
Aliaksandr Valialkin	91b3482894	app/vminsert: add ability to apply relabeling to all the incoming metrics if `-relabelConfig` command-line arg points to a file with a list of `relabel_config` entries See https://victoriametrics.github.io/#relabeling	2020-07-02 20:39:28 +03:00
Aliaksandr Valialkin	e5500bfcf2	all: typo fix: exptected -> expected	2020-07-02 18:05:52 +03:00
Aliaksandr Valialkin	5d3db3ff7c	app/vmselect: add `interpolate` function for filling gaps with linearly interpolated values See https://stackoverflow.com/q/62565021/274937 for details	2020-07-02 14:54:21 +03:00
Aliaksandr Valialkin	4dd3de9286	lib/promscrape: add ability to set `disable_compression` and `disable_keepalive` options in `scrape_config` section of the config passed to `-promscrape.config` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/580	2020-07-02 14:19:14 +03:00
BigFish	9d5f5b6878	fix: spelling mistakes (#594 ) Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-07-01 01:35:26 +03:00
Aliaksandr Valialkin	84a37098ed	app/vmstorage: add `-denyQueriesOutsideRetention` command-line flag for denying queries outside the configured retention VictoriaMetrics returns `503 Service Unavailable` http error for requests with time ranges outside the configured retention if `-denyQueriesOutsideRetention` command-line flag is set.	2020-07-01 00:21:44 +03:00
Aliaksandr Valialkin	7c2c8b2981	all: use `errors.As` for inspecting errors that implement httpserver.ErrorWithStatusCode	2020-07-01 00:04:34 +03:00
Aliaksandr Valialkin	d5dddb0953	all: use %w instead of %s for wrapping errors in `fmt.Errorf` This will simplify examining the returned errors such as httpserver.ErrorWithStatusCode . See https://blog.golang.org/go1.13-errors for details.	2020-06-30 23:05:11 +03:00
Roman Khavronenko	88538df267	app/vmalert: support multiple notifier urls (#584 ) (#590 ) * app/vmalert: support multiple notifier urls (#584) User now can set multiple notifier URLs in the same fashion as for other vmutils (e.g. vmagent). The same is correct for TLS setting for every configured URL. Alerts sending is done in sequential way for respecting the specified URLs order. * app/vmalert: add basicAuth support for notifier client (#585) The change adds possibility to set basicAuth creds for notifier client in the same fasion as for remote write/read and datasource.	2020-06-29 22:21:03 +03:00
Roman Khavronenko	82ecfa3b32	app/vmalert: move flags description and initialization into subpackages The change adds no new functionality and aims to move flags definitions to subpackages that are using them. This should improve readability of the main function.	2020-06-28 12:26:22 +01:00
kreedom	dc4e3f0e0b	app/vmalert: properly set transport for HTTP clients Fixes issue #586	2020-06-27 08:31:54 +01:00
Aliaksandr Valialkin	8f2e88234f	docs: update the info that docker images are built on top of `alpine` image now A follow-up after the commit `ff624c9125` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/522	2020-06-26 13:54:10 +03:00
Aliaksandr Valialkin	05d754d7bb	app/vmselect/netstorage: reset big result values every 10 seconds instead of after processing every time series This should reduce GC pressure when processing time series with big number of rows	2020-06-24 19:38:39 +03:00
nicbaz	774f7ca1c1	vmselect: fix label_replace when mismatch (#579 ) As per documentation on `label_replace` function: "If the regular expression doesn't match then the timeseries is returned unchanged". Currently this behavior is not enforced, if a regexp on an existing tag doesn't match then the tag value is copied as-is in the destination tag. This fix first checks that the regular expression matches the source tag before applying anything. Given the current implementation, this fix also changes the behavior of the MetricsQL `label_transform` function which does not document this behavior at the moment.	2020-06-23 23:50:33 +03:00
Aliaksandr Valialkin	7209d58fbd	app/vmselect/netstorage: increase concurrency when processing small number of time series with big number of data points per each time series Previously VictoriaMetrics was processing up to 32 time series in a single goroutine. This could be slow if each time series contains big number of data points (10M+ or more), since only a single CPU core could be loaded with work, while other CPU cores were idle. Fix this by launching GOMAXPROCS workers for time series processing. This should help with https://github.com/VictoriaMetrics/VictoriaMetrics/issues/572	2020-06-23 22:46:15 +03:00
nicbaz	72c90bfd8b	vmalert: add support for TLS configuration (#578 ) app/vmalert: add support for TLS configuration Add support for TLS optional configuration in a similar fashion to what is currently supported in other vmutils such as vmagent. TLS configuration options are distinct for datasource, remoteRead, remoteWrite as well as notifier.	2020-06-23 20:45:45 +01:00
Aliaksandr Valialkin	2a39ba639d	lib/promrelabel: add support for `keep_if_equal` and `drop_if_equal` actions to relabel configs These actions may be useful for filtering out unneeded targets and/or metrics if they contain equal label values. For example, the following rule would leave the target only if __meta_kubernetes_annotation_prometheus_io_port equals __meta_kubernetes_pod_container_port_number: - action: keep_if_equal source_labels: [__meta_kubernetes_annotation_prometheus_io_port, __meta_kubernetes_pod_container_port_number]	2020-06-23 17:29:03 +03:00
Aliaksandr Valialkin	7532dbcdf5	app/vmselect/promql: properly override label values from `group_left` and `group_right` lists like Prometheus does Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/577	2020-06-21 16:33:01 +03:00
kreedom	7ec6711f06	Support of custom URL path for alert (#560 ) app/vmalert: Support custom URL for alerts source Add flag `external.alert.source` for configuring custom URL for alert's source. This may be handy to re-point default source URL to other systems like Grafana. Updates #517	2020-06-21 11:32:46 +01:00
Aliaksandr Valialkin	9f3e3a4d7a	docs/vmauth.md: mention that we can provide custom integration with SAML	2020-06-19 13:14:07 +03:00
Aliaksandr Valialkin	b542e50680	app/vminsert: export metrics for determining ingested rows with dropped or truncated labels Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/565	2020-06-19 01:10:37 +03:00
Aliaksandr Valialkin	88e1b7d144	app/vmselect/promql: fill gaps on right side with values from left side of `or` operator in the same way as Prometheus does Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/552	2020-06-18 23:05:16 +03:00
Aliaksandr Valialkin	a12364ad37	docs/vmbackup.md: mention that backups from single-node and cluster versions are incompatible	2020-06-18 18:52:43 +03:00
Roman Khavronenko	e91d758831	vmalert-537: allow name duplication for rules within one group. (#559 ) Uniqueness of rule is now defined by combination of its name, expression and labels. The hash of the combination is now used as rule ID and identifies rule within the group. Set of rules from coreos/kube-prometheus was added for testing purposes to verify compatibility. The check also showed that `vmalert` doesn't support `query` template function that was mentioned as limitation in README.	2020-06-15 20:15:47 +01:00
Clémence Saussez	e426434770	app/vmalert: fix link to testdata (#547 ) Fix broken link to vmalert test data Signed-off-by: Clemence Saussez <clemence@zen.ly>	2020-06-10 17:26:27 +01:00
Roman Khavronenko	3e277020a5	vmalert-491: allow to configure concurrent rules execution per group. (#542 ) The feature allows to speed up group rules execution by executing them concurrently. Change also contains README changes to reflect configuration details.	2020-06-09 15:21:20 +03:00

... 14 15 16 17 18 ...

2026 Commits