1737 Commits

Author	SHA1	Message	Date
Aliaksandr Valialkin	578a37aa14	docs/CHANGELOG.md: document c90c1c4d54	2022-02-14 13:09:12 +02:00
Aliaksandr Valialkin	f10c38b827	lib/promscrape: add expand all and collapse all buttons to /targets page	2022-02-12 18:41:29 +02:00
Aliaksandr Valialkin	b1f94f7f0e	app/vmselect/promql: return at most one time series from absent_over_time() in the same way as Prometheus does ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2130	2022-02-12 15:45:09 +02:00
Aliaksandr Valialkin	d8ffbf55a2	docs/CHANGELOG.md: document ea153e5f90	2022-02-12 00:48:06 +02:00
Roman Khavronenko	cf1a8bce6b	lib/index: reduce read/write load after indexDB rotation (#2177) ... * lib/index: reduce read/write load after indexDB rotation IndexDB in VM is responsible for storing TSID - ID's used for identifying time series. The index is stored on disk and used by both ingestion and read path. IndexDB is stored separately to data parts and is global for all stored data. It can't be deleted partially as VM deletes data parts. Instead, indexDB is rotated once in `retention` interval. The rotation procedure means that `current` indexDB becomes `previous`, and new freshly created indexDB struct becomes `current`. So in any time, VM holds indexDB for current and previous retention periods. When time series is ingested or queried, VM checks if its TSID is present in `current` indexDB. If it is missing, it checks the `previous` indexDB. If TSID was found, it gets copied to the `current` indexDB. In this way `current` indexDB stores only series which were active during the retention period. To improve indexDB lookups, VM uses a cache layer called `tsidCache`. Both write and read path consult `tsidCache` and on miss the relad lookup happens. When rotation happens, VM resets the `tsidCache`. This is needed for ingestion path to trigger `current` indexDB re-population. Since index re-population requires additional resources, every index rotation event may cause some extra load on CPU and disk. While it may be unnoticeable for most of the cases, for systems with very high number of unique series each rotation may lead to performance degradation for some period of time. This PR makes an attempt to smooth out resource usage after the rotation. The changes are following: 1. `tsidCache` is no longer reset after the rotation; 2. Instead, each entry in `tsidCache` gains a notion of indexDB to which they belong; 3. On ingestion path after the rotation we check if requested TSID was found in `tsidCache`. Then we have 3 branches: 3.1 Fast path. It was found, and belongs to the `current` indexDB. Return TSID. 3.2 Slow path. It wasn't found, so we generate it from scratch, add to `current` indexDB, add it to `tsidCache`. 3.3 Smooth path. It was found but does not belong to the `current` indexDB. In this case, we add it to the `current` indexDB with some probability. The probability is based on time passed since the last rotation with some threshold. The more time has passed since rotation the higher is chance to re-populate `current` indexDB. The default re-population interval in this PR is set to `1h`, during which entries from `previous` index supposed to slowly re-populate `current` index. The new metric `vm_timeseries_repopulated_total` was added to identify how many TSIDs were moved from `previous` indexDB to the `current` indexDB. This metric supposed to grow only during the first `1h` after the last rotation. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1401 Signed-off-by: hagen1778 <roman@victoriametrics.com> * wip * wip Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-12 00:30:08 +02:00
Roman Khavronenko	e3adcbec6e	lib/promscrape: support prometheus-like duration in scrape configs (#2169) ... * lib/promscrape: support prometheus-like duration in scrape configs The change allows to specify duration values like `1d`, `1w` for fields `scrape_interval`, `scrape_timeout`, etc. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/817#issuecomment-1033384766 Signed-off-by: hagen1778 <roman@victoriametrics.com> * lib/blockcache: make linter happy Signed-off-by: hagen1778 <roman@victoriametrics.com> * lib/promscrape: support prometheus-like duration in scrape configs * add support for extra fields `scrape_align_interval` and `scrape_offset`; * support Prometheus duration parsing for `__scrape_interval__` and `__scrape_duration__` labels; Signed-off-by: hagen1778 <roman@victoriametrics.com> * wip * wip * docs/CHANGELOG.md: document the feature Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-11 16:17:00 +02:00
Aliaksandr Valialkin	3cb72ccc2a	lib/promscrape/discovery/kubernetes: add __meta_kubernetes_endpointslice_{label,annotation}* labels to be consistent with other role values for Kubernetes service discovery	2022-02-11 14:54:47 +02:00
Nikolay	4e7f7f3302	fixes service discovery for kubernetes (#2173) ... * fixes service discovery for kubernetes now it must take in account all pods that belong to the discovered endpoint and endpointslice adds simple test for endpoints https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2134 * wip * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-11 13:34:22 +02:00
Aliaksandr Valialkin	8f2d03fdc7	docs/CHANGELOG.md: document 4e722c459b ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2167	2022-02-10 12:20:12 +02:00
Aliaksandr Valialkin	0028b2c6d1	docs/CHANGELOG.md: add instructions on how to build VictoriaMetrics components from source code in order to test tip changes	2022-02-08 16:44:08 +02:00
Nikolay	a8acad7453	adds CGO build for arm64 (#2102) ... * adds CGO build for arm64 it must improve performance for arm64 based deployments of vmstorage and vmsingle for 15-20% it depends on gozstd package update for correct musl gozstd vendoring * typo fixes * docs/CHANGELOG.md: document the change Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-02-08 16:25:59 +02:00
Aliaksandr Valialkin	9bb60ab00f	lib/promscrape: set -promscrape.config.strictParse to true by default ... This allows detecting long-living silent errors in -promscrape.config	2022-02-08 15:41:43 +02:00
Aliaksandr Valialkin	8b36044c93	docs/CHANGELOG.md: add links to issues, which could benefit from improved re-routing algorithm	2022-02-07 16:55:28 +02:00
Aliaksandr Valialkin	865f09ecbb	docs: sync with cluster branch	2022-02-07 14:54:16 +02:00
Aliaksandr Valialkin	b5b3c585b3	lib/promscrape: show the total number of scrapes and the total number of scrape errors per target at /targets page ... This information may be useful when debugging unreliable scrape targets	2022-02-03 20:22:41 +02:00
Aliaksandr Valialkin	2968779f16	lib/promscrape: provide the ability to fetch target responses on behalf of vmagent or single-node VictoriaMetrics ... This feature may be useful when debugging metrics for the given target located in isolated environment	2022-02-03 19:00:55 +02:00
Aliaksandr Valialkin	4b850c2a59	app/vmselect/promql: do not push down filters, which enumerate more than 10k unique values ... Such filters may slow down time series search, so just skip them. This is a follow-up for e7f1ceeb84 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827	2022-02-02 23:40:02 +02:00
Aliaksandr Valialkin	4ef32df4fa	docs/CHANGELOG.md: document 55e3bbd4cc ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1567	2022-02-02 23:32:35 +02:00
Aliaksandr Valialkin	6530bcedec	docs: updates after 5da71eb685 ... * Mention about the ability to configure vmalert notifiers via files in docs/CHANGELOG.md * Mention about the ability to use Consul service discovery for vmalert notifiers in docs/CHANGELOG.md * Run `make docs-sync` in order to sync app/vmalert/README.md to docs/vmalert.md	2022-02-02 23:26:18 +02:00
Aliaksandr Valialkin	ead66155ef	lib/cgroup: expose process_cpu_cores_available metric ... This metric shows the number of CPU cores available to the process. This allows creating alerting rules on CPU saturation with the following query: rate(process_cpu_seconds_total[5m]) / process_cpu_cores_available > 0.9 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2107	2022-01-31 20:24:41 +02:00
Aliaksandr Valialkin	e7f1ceeb84	app/vmselect/promql: optimize queries, which join on _info metrics. ... Automatically add common filters from one side of binary operation to the other side before sending the query to storage subsystem. See https://grafana.com/blog/2021/08/04/how-to-use-promql-joins-for-more-effective-queries-of-prometheus-metrics-at-scale/ and https://www.robustperception.io/exposing-the-software-version-to-prometheus	2022-01-31 19:32:36 +02:00
Aliaksandr Valialkin	3d8a4bf023	docs/CHANGELOG.md: document 6a519896db	2022-01-31 12:41:57 +02:00
Aliaksandr Valialkin	e02e0508da	vendor: update github.com/VictoriaMetrics/metricsql from v0.37.0 to v0.38.0 ... This adds more optimization cases for https://utcc.utoronto.ca/~cks/space/blog/sysadmin/PrometheusLabelNonOptimization For example: * Multi-level transform functions. For example, abs(round(foo{a="b"})) + bar{x="y"} is now optimized to abs(round(foo{a="b",x="y"})) + bar{a="b",x="y"} * Binary operations with `on()`, `without()`, `group_left()` and `group_right()` modifiers. For example, foo{a="b"} on (a) + bar is now optimized to foo{a="b"} on (a) + bar{a="b"} * Multi-level binary operations. For example, foo{a="b"} + bar{x="y"} + baz{z="q"} is now optimized to foo{a="b",x="y",z="q"} + bar{a="b",x="y",z="q"} + baz{a="b",x="y",z="q"} * Aggregate functions. For example, sum(foo{a="b"}) by (c) + bar{c="d"} is now optimized to sum(foo{a="b",c="d"}) by (c) + bar{c="d"}	2022-01-27 19:03:54 +02:00
Aliaksandr Valialkin	746ee191e8	lib/logger/throttler.go: show the original location of the error and warning message ... Previously the location inside LogThrottler implementation was shown. This could complicate debugging.	2022-01-23 13:55:00 +02:00
Yury Molodov	ad5059f2d3	vmui: fixed display type switching (#2088) ... * fix: correct switch display type * docs/CHANGELOG.md: document the bugfix Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-21 16:56:22 +02:00
Yury Molodov	adbb821eac	vmui: fix time range selector (#2085) ... * fix: add date validate for time range * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document the bugfix Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-21 12:02:38 +02:00
Aliaksandr Valialkin	145337792d	lib/{mergeset,storage}: properly limit cache sizes for indexdb ... Previously these caches could exceed limits set via `-memory.allowedPercent` and/or `-memory.allowedBytes`, since limits were set independently per each data part. If the number of data parts was big, then limits could be exceeded, which could result to out of memory errors. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2007	2022-01-20 18:37:17 +02:00
Aliaksandr Valialkin	84f6b3014c	docs/CHANGELOG.md: document the bugfix for highestMax() function is Graphite render API	2022-01-20 12:16:18 +02:00
Aliaksandr Valialkin	109363de49	docs/CHANGELOG.md: add missing parens in example for @ modifier	2022-01-19 13:04:51 +02:00
Aliaksandr Valialkin	98edeac7b7	docs/CHANGELOG.md: fix incorrect link to the issue ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1911	2022-01-19 00:06:44 +02:00
Aliaksandr Valialkin	8f5902dfcf	docs/CHANGELOG.md: cut v1.72.0	2022-01-18 22:43:18 +02:00
Yury Molodov	8bdc45ba00	fix: remove buffer period (#2078) ... * fix: remove buffer period * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document the implemented feature Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2064 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-18 21:42:56 +02:00
Yury Molodov	70737ea4ac	vmui: correct url encoding (#2067) ... * fix: correct encode multi-line queries * fix: change autocomplete for correct arrows work * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document the bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2039 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2022-01-18 21:31:46 +02:00
Aliaksandr Valialkin	dcadec65b6	docs/CHANGELOG.md: document 8e3f9c1fbb	2022-01-18 21:24:49 +02:00
Aliaksandr Valialkin	e933e3150d	docs/CHANGELOG.md: document fcd33fc409	2022-01-18 12:46:33 +02:00
Aliaksandr Valialkin	c2a3911bb5	docs/CHANGELOG.md: document that vmgateway now supports extra_filters option ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863	2022-01-18 12:39:25 +02:00
Aliaksandr Valialkin	1bdc71d917	app/vmselect/promql: implement keep_metric_names modifier for transform and rollup functions ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949	2022-01-14 04:14:59 +02:00
Aliaksandr Valialkin	f41846d002	app/vmselect/promql: add stale_samples_over_time() function	2022-01-14 01:48:04 +02:00
Aliaksandr Valialkin	96707223db	docs/CHANGELOG.md: yet another attempt to fix formatting for yaml snippet	2022-01-14 01:14:55 +02:00
Aliaksandr Valialkin	d7d83d6d93	docs/CHANGELOG.md: fix formatting for scrape_configs example	2022-01-14 01:02:32 +02:00
Aliaksandr Valialkin	1d05444b33	lib/promscrape: expose promscrape_stale_samples_created_total metric for monitoring the number of created stale samples	2022-01-14 01:00:46 +02:00
Aliaksandr Valialkin	831b93a755	app/vmalert: add parseDuration function in the same way as Prometheus does ... See https://github.com/prometheus/prometheus/pull/8817	2022-01-13 23:30:41 +02:00
Aliaksandr Valialkin	80f03177c4	lib/promscrape/discovery/kubernetes: add __meta_kubernetes_node_provider_id label for discovered Kubernetes nodes in the same way as Prometheus does ... See https://github.com/prometheus/prometheus/pull/9603	2022-01-13 23:16:02 +02:00
Aliaksandr Valialkin	80f966b80c	app/vmalert: add stripPort template function in the same way as Prometheus does ... See https://github.com/prometheus/prometheus/pull/10002	2022-01-13 22:53:42 +02:00
Aliaksandr Valialkin	355a63733d	lib/promscrape/discovery/kubernetes: add the ability to limit service discovery to the current namespace ... See https://github.com/prometheus/prometheus/issues/9782 and https://github.com/prometheus/prometheus/pull/9881	2022-01-13 22:44:35 +02:00
Aliaksandr Valialkin	c883c15878	app/vmselect/promql: add support for @ modifier ... Add support for `@` modifier in MetricsQL according to https://prometheus.io/docs/prometheus/latest/querying/basics/#modifier Extend the support with the following features: * Allow using `@` modifier everywhere in the query. For example, `sum(foo) @ end()` * Allow using arbitrary expression as `@` modifier. For example, `foo @ (end() - 1h)` returns `foo` value at `end - 1 hour` timestamp on the selected time range `[start ... end]` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1348	2022-01-13 22:12:06 +02:00
Aliaksandr Valialkin	17eb86a689	lib/promscrape/discovery/dockerswarm: follow up after 68a117a25a ... - Document the bugfix at docs/CHANGELOG.md - Set __address__ field after copying commonLabels to the resulting map of discovered labels. This makes sure that the correct __address__ label is used.	2022-01-11 09:20:10 +02:00
Aliaksandr Valialkin	7a2c46d951	docs/CHANGELOG.md: document 77bfa8181d	2022-01-11 08:53:49 +02:00
Aliaksandr Valialkin	bdba50432b	docs/CHANGELOG.md: add release dates for every release	2022-01-07 12:27:32 +02:00
Aliaksandr Valialkin	e4e36383e2	lib/promscrape: do not send staleness markers on graceful shutdown ... This follows Prometheus behavior. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2013#issuecomment-1006994079	2022-01-07 01:17:57 +02:00
Denys Holius	ae89b4e818	Old links replaced for newest (#2033) ... * replaced old links to the website * fixed deletion main README.md file * fix: added docs files after docs-sync	2022-01-05 16:30:13 +02:00
Aliaksandr Valialkin	dbe592597f	docs/CHANGELOG.md: clarify the issue description, which is fixed by 38bf5fc136	2022-01-05 16:19:06 +02:00
Aliaksandr Valialkin	178dd87e26	lib/storage: follow-up for 38bf5fc136	2022-01-05 16:00:11 +02:00
Aliaksandr Valialkin	e1d7cbfc77	docs/CHANGELOG.md: document 60266078ca	2022-01-04 11:44:57 +02:00
Aliaksandr Valialkin	5ce94e1dd3	docs/CHANGELOG.md: document ac47733044	2022-01-03 21:15:44 +02:00
Aliaksandr Valialkin	b5ce35dfc8	docs/CHANGELOG.md: document 543bd0ea0c ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1999	2021-12-23 12:08:06 +02:00
Aliaksandr Valialkin	cbaa2af280	lib/promscrape: scrape replicated targets at different offsets in vmagent replicated clustering mode ... This guarantees that the deduplication consistently leaves samples from the same vmagent replica. See https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets	2021-12-23 00:20:39 +02:00
Nikolay	8ff7da7202	adds restore.lock (#1988) ... * adds restore.lock it must prevent from running storage after incomplete restore process https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1958 * return back flock file deletion * Apply suggestions from code review * wip * docs/CHANGELOG.md: document https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1958 Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-22 13:10:15 +02:00
Aliaksandr Valialkin	3cfb90b227	docs/CHANGELOG.md: document 34fdc8881b ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1911	2021-12-21 16:40:39 +02:00
Aliaksandr Valialkin	4701c108ff	docs/CHANGELOG.md: cut v1.71.0	2021-12-20 19:10:24 +02:00
Aliaksandr Valialkin	974d9c0eee	app/vmselect/promql: follow-up after 177e345d8a ... * Document changes_prometheus(), increase_prometheus() and delta_prometheus() functions. * Simplify their implementation * Mention these functions in docs/CHANGELOG.md	2021-12-20 13:19:44 +02:00
Aliaksandr Valialkin	16636a458f	docs/CHANGELOG.md: document 6814cc6809 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1943	2021-12-17 20:16:22 +02:00
Aliaksandr Valialkin	193331d522	app/vmselect: de-duplicate data exported via /api/v1/export/csv by default ... Previously the exported data wasn't de-duplicated. Now it is possible to export the raw data without deduplication by passing reduce_mem_usage=1 query arg to /api/v1/export/csv See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1837	2021-12-17 10:57:39 +02:00
Aliaksandr Valialkin	a3adf24527	lib/promscrape: allow up to 5 redirects when scraping a target by default ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1945	2021-12-16 00:14:14 +02:00
Aliaksandr Valialkin	5efe377a26	app/vmselect/promql: add timestamp_with_name(m[d]) function ... This function works the same as `timestamp()`, but doesn't remove source time series names. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/949#issuecomment-995222388	2021-12-15 23:37:07 +02:00
Aliaksandr Valialkin	8e1a87491a	docs: document the added dowsnampling support in VictoriaMetrics enterprise ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/36	2021-12-15 16:25:51 +02:00
Aliaksandr Valialkin	4ff647137a	lib/storage: deduplicate samples more thoroughly ... Previously some duplicate samples may be left on disk for time series with high churn rate. This may result in higher disk space usage.	2021-12-15 15:59:58 +02:00
Aliaksandr Valialkin	e1a715b0f5	lib/storage: convert alternate regexps into Graphite wildcards inside __graphite__ pseudo-label ... For example, `{__graphite__=~"foo.(bar|baz)"}` is automatically converted to `{__graphite__=~"foo.{bar,baz}"}` before execution. This allows using multi-value Grafana template variables such as `{__graphite__=~"foo.($app)"}`.	2021-12-14 19:51:49 +02:00
Aliaksandr Valialkin	d456af7499	docs/CHANGELOG.md: link to the issue about unaligned 64-bit atomic opertion panic on 32-bit architectures	2021-12-14 15:00:10 +02:00
Aliaksandr Valialkin	80996d916b	docs/CHANGELOG.md: an attempt to properly show $labels.alertname at https://docs.victoriametrics.com/CHANGELOG.html	2021-12-14 14:56:57 +02:00
Yury Molodov	49e6a921df	vmui: custom step (#1942) ... * feat: add a label for the Query field * fix: change zoom position * fix: add description and error code to alerts * fix: correct logic query history * fix: correct update query history * feat: add custom step * update package-lock.json * docs: document that VMUI now supports overriding of `step` query arg, which is passed to `/api/v1/query_range` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-14 14:51:45 +02:00
Yury Molodov	b5b701d590	vmui: minor fixes (#1936) ... * feat: add a label for the Query field * fix: change zoom position * fix: add description and error code to alerts * fix: correct logic query history * fix: correct update query history * app/vmselect/vmui: `make vmui-update` * docs/CHANGELOG.md: document bugfixes Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-13 13:42:37 +02:00
Aliaksandr Valialkin	c82cc9cd11	docs/CHANGELOG.md: document 7c3b6365f0	2021-12-12 19:09:20 +02:00
Aliaksandr Valialkin	d1f8915ed1	app/vmselect/promql: preserve the order of time series passed to limit_offset() function ... Previously time series passed to `limit_offset()` were shuffled according to hash for their labels. This was unexpected behaviour for most users. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1920 and https://github.com/VictoriaMetrics/VictoriaMetrics/issues/951	2021-12-12 18:04:58 +02:00
Aliaksandr Valialkin	3d4349343d	docs/CHANGELOG.md: document 2851709745 ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1921	2021-12-10 12:13:34 +02:00
Aliaksandr Valialkin	52eb9c99e2	docs: document the ability to investigate correlation between two queries at /vmui ... This is a follow-up for c1fd93e8a0	2021-12-08 17:24:01 +02:00
Aliaksandr Valialkin	ff15a752c1	app/vmselect: accept optional extra_filters[] query args for all the supported Prometheus querying APIs ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1863	2021-12-06 17:07:09 +02:00
Aliaksandr Valialkin	45d082bbe2	app/vminsert: add -maxLabelValueLen command-line flag ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1908	2021-12-06 11:40:34 +02:00
Aliaksandr Valialkin	da402fbdfa	lib/workingsetcache: fix unaligned 64-bit atomic operation panic on 32-bit architectures ... The panic has been introduced in 7275ebf91a	2021-12-03 01:21:51 +02:00
Aliaksandr Valialkin	06642d97f5	app: allow specifying http and https urls in the following command-line flags ... * -promscrape.config * -relabelConfig * -remoteWrite.relabelConfig * -remoteWrite.urlRelabelConfig	2021-12-03 00:10:02 +02:00
Aliaksandr Valialkin	62b4efb3e7	app/vmauth: follow-up for 13368bed18 ... * Document the ability to specify http or https urls in `-auth.config` at docs/CHANGELOG.md * Move the ReadFileOrHTTP to lib/fs, so it can be re-used in other places where a file should be read from the given path. For example, in `-promscrape.config` at `vmagent`.	2021-12-02 23:32:05 +02:00
Aliaksandr Valialkin	7f2f26b25f	docs/CHANGELOG.md: cut v1.70.0	2021-12-02 15:01:25 +02:00
Aliaksandr Valialkin	ed9ef7733b	docs/CHANGELOG.md: document 0afd14a14a	2021-12-02 14:47:20 +02:00
Aliaksandr Valialkin	394a345ae0	lib/httpserver: expose /-/healthy and /-/ready endpoints as Prometheus does ... This improves integration with third-party solutions, which rely on these endpoints. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1833	2021-12-02 14:36:58 +02:00
Aliaksandr Valialkin	03f5ad3060	lib/protoparser/graphite: allow multiple separators between metric name, value and timestamp	2021-12-02 13:43:49 +02:00
Aliaksandr Valialkin	49a18b8660	lib/protoparser/graphite: properly parse Graphite line with whitespace after the timestamp ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1865	2021-12-02 13:33:26 +02:00
Aliaksandr Valialkin	c0cbf0de2a	app/{vmbackup,vmrestore}: export internal metrics at /metrics http handler	2021-12-02 11:55:58 +02:00
Aliaksandr Valialkin	7275ebf91a	app/vmstorage: export vm_cache_size_max_bytes metrics for determining capacity of various caches ... The vm_cache_size_max_bytes metric can be used for determining caches which reach their capacity via the following query: vm_cache_size_bytes / vm_cache_size_max_bytes > 0.9	2021-12-02 10:30:43 +02:00
Aliaksandr Valialkin	2f63dec2e3	lib/fs: add vm_filestream_read_duration_seconds_total and vm_filestream_write_duration_seconds_total metrics ... These metrics help determining persistent disk saturation with `rate(vm_filestream_read_duration_seconds_total) > 0.9`	2021-12-02 10:30:42 +02:00
Aliaksandr Valialkin	ed54b27b85	docs/CHANGELOG.md: document 06eff5a72c ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/487	2021-12-01 12:28:06 +02:00
Aliaksandr Valialkin	10bd8b1d86	docs/CHANGELOG.md: document 852a895b70	2021-11-30 01:22:37 +02:00
Aliaksandr Valialkin	ca6fc0265e	docs/CHANGELOG.md: document f05cddd2fc ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1830	2021-11-30 01:15:41 +02:00
Aliaksandr Valialkin	841647643a	docs/CHANGELOG.md: document 9dd650f67f	2021-11-30 01:09:10 +02:00
Aliaksandr Valialkin	7e79fc6e3c	docs/CHANGELOG.md: document 624ad73705	2021-11-30 01:05:05 +02:00
Aliaksandr Valialkin	98d244b288	docs/CHANGELOG.md: document c6d5927281	2021-11-30 01:00:36 +02:00
Aliaksandr Valialkin	d44c585ca4	lib/protoparser: do not log connection reset by peer error when reading the data via InfluxDB, Graphite and OpenTSDB protocols over plain TCP connections ... This error is expected, so there is no need in spamming the log with this error.	2021-11-29 21:47:56 +02:00
Aliaksandr Valialkin	03509025bc	docs/CHANGELOG.md: document 695cb617b2	2021-11-29 12:13:11 +02:00
Aliaksandr Valialkin	9bee043ff2	app/vmselect/promql: consistently return zero from deriv(const)	2021-11-17 18:02:05 +02:00
Aliaksandr Valialkin	b688960db0	lib/persistentqueue: add vm_persistentqueue_read_duration_seconds_total and vm_persistentqueue_write_duration_seconds_total metrics for determining disk usage saturation at vmagent	2021-11-17 16:41:35 +02:00
Aliaksandr Valialkin	b900560b83	app/vmselect/promql: add now() function, which returns the current timestamp as a floating-point value in seconds	2021-11-17 16:35:30 +02:00
Aliaksandr Valialkin	52e19a0577	docs: document -s3ForcePathStyle command-line option ... This is a follow-up for b72eed1f5e	2021-11-17 01:09:32 +02:00
Aliaksandr Valialkin	f3ac945d74	app/vmauth: add ability to override the username label value for vmauth_user_requests_total metric by specifying name option in -auth.config ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1805	2021-11-17 00:47:34 +02:00
Aliaksandr Valialkin	5a180c6659	docs/CHANGELOG.md: document the addition of vm_tenant_used_tenant_bytes metric, which shows the per-tenant disk usage ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1605	2021-11-16 23:36:48 +02:00
Aliaksandr Valialkin	8b6c89423d	docs/CHANGELOG.md: document bugfixes in enteprise versions of vmagent and vmalert	2021-11-12 13:24:07 +02:00
Aliaksandr Valialkin	e5d4c7f4a7	app/vmauth: initialize reverse proxy only after flag.Parse() is called ... This should properly take into accoun the `-maxIdleConnsPerBackend` command-line flag value. Previously it was hardcoded to 100. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1300	2021-11-09 19:22:34 +02:00
Aliaksandr Valialkin	69e655ba7f	docs/CHANGELOG.md: cut v1.69.0	2021-11-08 15:47:36 +02:00
Aliaksandr Valialkin	3419ac1d36	app/vmselect/promql: add duration_over_time(m[d], max_interval) function ... This function calculates the actual lifetime of the time series on the given lookbehdind window `d` Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1780	2021-11-08 13:14:09 +02:00
Aliaksandr Valialkin	5edf695bc9	docs/CHANGELOG.md: document b9cdbcb5046315db96e1e7ca9923d09d0f30dc25	2021-11-08 12:11:30 +02:00
Aliaksandr Valialkin	237885e0d2	docs/vmalert.md: document the addition of -defaultTenant.prometheus and -defaultTenant.graphite command-line options to enterprise version of vmalert	2021-11-05 20:04:09 +02:00
Aliaksandr Valialkin	cbfc7b7c92	app/{vminsert,vmagent}: hide passwords and auth tokens by default at /config page ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764	2021-11-05 14:41:16 +02:00
Aliaksandr Valialkin	a69264e885	app/vmagent: add -remoteWrite.maxRowsPerBlock command-line option, which may be used for improving data ingestion performance under high load	2021-11-04 15:39:14 +02:00
Aliaksandr Valialkin	e0d2ba5608	app/vmagent/remotewrite: send data to remote storage systems in parallel ... This should improve data ingestion speed when many `-remoteWrite.url` command-line flags are configured	2021-11-04 15:04:16 +02:00
Aliaksandr Valialkin	aa534c2582	lib/promscrape: add -promscrape.maxResponseHeadersSize command-line flag for tuning the maximum http response headers size from Prometheus scrape targets	2021-11-03 22:26:56 +02:00
Aliaksandr Valialkin	27044b84d2	app/vmselect/promql: add limit_offset(limit, offset, q) function, which can be used for paging over big number of time series ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1778	2021-11-03 16:02:27 +02:00
Aliaksandr Valialkin	43a58bd618	app/vmselect/promql: add label_graphite_group() function for extracting groups from Graphite metric names	2021-11-03 13:19:08 +02:00
Aliaksandr Valialkin	da2e0e29a4	docs/CHANGELOG.md: document e3a91b186a	2021-11-02 18:39:14 +02:00
Aliaksandr Valialkin	d1eb87c831	app/{vmagent,vminsert}: add ability to restrict access to /config page with authKey query arg ... The authKey can be configured via `-configAuthKey` command-line flag. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1764	2021-11-01 16:44:54 +02:00
Aliaksandr Valialkin	bb87949d5c	lib/protoparser/influx: automatically detect timestamp precision depending on the number of decimal digits in the timestamp	2021-10-28 12:47:22 +03:00
Aliaksandr Valialkin	74b8af9891	lib/promscrape: add collapse and expand buttons per each group of targets from the same scrape job	2021-10-27 20:03:24 +03:00
Aliaksandr Valialkin	1952ab99aa	all: fix build issues and tests for Apple M1 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1653	2021-10-27 15:06:34 +03:00
Aliaksandr Valialkin	9ec0175e83	docs/CHANGELOG.md: mention the issue about missing proxy_url config option at /config page ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755	2021-10-26 22:06:35 +03:00
Aliaksandr Valialkin	4821adfd95	lib/promscrape: properly show proxy_url option value at /config page ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1755	2021-10-26 21:23:54 +03:00
Aliaksandr Valialkin	a7e3cbd6ad	docs/CHANGELOG.md: document 3dbdf1632e	2021-10-25 12:16:37 +03:00
Aliaksandr Valialkin	8d0fafc377	docs/CHANGELOG.md: typo fix	2021-10-22 21:11:54 +03:00
Aliaksandr Valialkin	7f7cac20c1	docs/CHANGELOG.md: cut v1.68.0	2021-10-22 19:37:48 +03:00
Aliaksandr Valialkin	013d626889	app/vmauth: add ability to specify http headers to send in requests to backends ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1736	2021-10-22 19:10:29 +03:00
Aliaksandr Valialkin	7e88713ca3	docs/CHANGELOG.md: document 43a7984cd8	2021-10-22 14:00:20 +03:00
Aliaksandr Valialkin	8568003bb1	docs/CHANGELOG.md: document a3684fe3de	2021-10-22 12:28:01 +03:00
Yury Molodov	2b266cb87e	vmui: query history (#1732) ... * feat: add query history * fix: change detect keyUp for nav query history * feat: set default query history * app/vmselect/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-22 12:21:22 +03:00
Aliaksandr Valialkin	8ad95f0db7	lib/httpserver: expose command-line flags at /flags page ... This should simplify debugging. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695	2021-10-20 00:45:09 +03:00
Aliaksandr Valialkin	0d48b89afe	docs/CHANGELOG.md: document 146a5b504c	2021-10-19 11:25:02 +03:00
Aliaksandr Valialkin	c64a134146	docs/CHANGELOG.md: document cbcc622786	2021-10-19 08:56:23 +03:00
Yury Molodov	a3e09a57c2	vmui: features (#1711) ... * feat: initial uPlot graph * feat: add zoom/pan for graph * fix: add zoom by ctrl/mac * fix: remove unused code * feat: add toggle cache for fetch * feat: add fix y-axis limits * fix: stop point events while panning * fix: change getting cursor position when scaling * feat: add cursor tooltip to graph * fix: uninstall chart.js * fix: change link for create an issue * fix: set default cache value to true * app/vmalert: follow-up after 0e2486df56 * docs/CHANGELOG.md: document 5416e18007 * app/vmui: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-10-18 15:16:57 +03:00
Aliaksandr Valialkin	c37f285466	lib/promscrape: set honor_timestamps: true by default if this option isnt set explicitly in scrape configs ... This aligns the behavior to Prometheus - see https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config	2021-10-16 20:49:08 +03:00
Aliaksandr Valialkin	c055bc478c	lib/promscrape: expose promscrape_series_limit_max_series and promscrape_series_limit_current_series metrics per each scrape target with the enabled unique series limiter	2021-10-16 18:47:13 +03:00
Aliaksandr Valialkin	9866dd95c1	lib/promscrape: store the full response in stream parsing mode in scrapeWork.lastScrape byte slice ... This allows sending staleness marks and properly calculate scrape_series_added metric in stream parsing mode at the cost of the increased memory usage, since now the potentially big response is kept in the lastScrape byte slice per each scrapeWork. In practice the memory usage increase shouldn't be big, since the response size is usually much smaller than the parsed metrics from this response after the relabeling, which usually adds a big pile of target-specific labels per each metric.	2021-10-15 15:39:23 +03:00
Aliaksandr Valialkin	78425561ce	docs/CHANGELOG.md: document the change at 7fcbd3fa4b	2021-10-14 14:37:44 +03:00
Aliaksandr Valialkin	bbd34fa15e	lib/promscrape: add -promscrape.minResponseSizeForStreamParse command-line option for automatic switching to stream parsing mode when scraping targets with big responses ... This should reduce memory usage when vmagent scrapes targets with non-uniform response sizes. This is common case in Kubernetes monitoring.	2021-10-14 12:29:35 +03:00
Aliaksandr Valialkin	1a7287c408	lib/promscrape: return error if sample_limit or series_limit options are set when stream parsing mode is enabled	2021-10-14 12:11:23 +03:00
Aliaksandr Valialkin	1c17fe70e0	docs/CHANGELOG.md: document e3c8304deb	2021-10-13 16:00:50 +03:00
Aliaksandr Valialkin	e3c8304deb	lib/promscrape: add ability to show the original labels for discovered targets at /targets page ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1698	2021-10-13 15:59:58 +03:00
Aliaksandr Valialkin	9882cda8b9	lib/promscrape: shard targets among cluster nodes after relabeling is applied ... This guarantees that targets with the same set of labels go to the same vmagent node. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1687#issuecomment-940629495	2021-10-12 17:06:00 +03:00
Aliaksandr Valialkin	5a58c041c2	app/vmagent: expose -promscrape.config contents at /config page as Prometheus does ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1695	2021-10-12 16:25:37 +03:00
Aliaksandr Valialkin	a5001b9c20	app/vmselect/promql: add atan2 binary operator, which is going to be added in Prometheus 2.31 ... See https://github.com/prometheus/prometheus/pull/9248	2021-10-11 21:15:53 +03:00
Aliaksandr Valialkin	81c6720392	app/vmselect/promql: add missing trigonometric functions, which are going to be added in Prometheus 2.31 ... See https://github.com/prometheus/prometheus/issues/9233	2021-10-11 21:01:33 +03:00
Aliaksandr Valialkin	92b92d4d2c	app/vmselect/promql: consistently return the same set of time series from limitk() function ... This is the expected behaviour by most users.	2021-10-08 19:53:52 +03:00
Aliaksandr Valialkin	001750c239	lib/storage: fix unaligned access on 32-bit architectures. ... The bug has been introduced at a171916ef5	2021-10-08 19:43:03 +03:00
Aliaksandr Valialkin	0a3a774202	docs/CHANGELOG.md: cut v1.67.0	2021-10-08 16:00:33 +03:00
Aliaksandr Valialkin	0ff8fcac6a	app/vmui: follow-up after 7bfb44113e ... * Run `vmui-update` * Document the changes in README.md and CHANGELOG.md	2021-10-08 15:09:29 +03:00
Aliaksandr Valialkin	cf5cbd1c70	app/{vminsert,vmstorage}: follow-up after a171916ef5 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/269	2021-10-08 14:35:49 +03:00
Aliaksandr Valialkin	a19e7c7ce8	app/vminsert: fix uneven distribution of time series among storage nodes ... Use distinct seed for distribution hash calculations on the second level of vminsert nodes. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1672	2021-10-07 12:23:39 +03:00
Aliaksandr Valialkin	9515e58e28	docs/vmagent.md: document how to write data to Kafka	2021-09-30 17:45:53 +03:00
Aliaksandr Valialkin	0e3de5a0cc	app/vmselect/promql: add topk_last and bottomk_last functions	2021-09-30 13:22:52 +03:00
Aliaksandr Valialkin	2b623ae302	docs/CHANGELOG.md: link to Kafka integration docs	2021-09-29 12:31:23 +03:00
Aliaksandr Valialkin	ead59bdebf	docs/CHANGELOG.md: document the bugfix from de810031bf	2021-09-29 00:41:35 +03:00
Aliaksandr Valialkin	50ec259750	docs/CHANGELOG.md: document 3d17112a7e	2021-09-29 00:33:08 +03:00
Aliaksandr Valialkin	91b3c601bc	app/{vminsert,vmagent}: add ability to ingest data via DataDog "submit metrics" API ... See https://docs.datadoghq.com/api/latest/metrics/#submit-metrics Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/206	2021-09-29 00:13:08 +03:00
Yury Molodov	a64155d91e	vmui: use Chart.js as default engine for graph (#1634) ... * feat: add Plotly as default engine for graph * fix: remove unused components * feat: use Chart.js as default engine graph * fix: correct styles for loader * feat: add zoom/pan for chart * feat: add height for chart * fix: remove unused code * fix: remove empty units from duration * fix: change debounce for pan to 500ms * fix: add utility for plugins register globally * fix: optimize render graph * feat: add buffer data for zoom * fix: add limits for zoom in/out * fix: change update data while zooming * app/vmselect: `make vmui-update` Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-09-27 22:26:14 +03:00
Aliaksandr Valialkin	2efe0acfc9	app/vmselect/promql: add rollup_scrape_interval(m[d]) function ... It calculates the min, max and avg scrape intervals for m over the given lookbehind window d	2021-09-27 19:21:24 +03:00
Aliaksandr Valialkin	c4c77aa2dd	app/vmselect/promql: follow-up after 526dd93b32 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1625 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612	2021-09-27 18:55:39 +03:00
Aliaksandr Valialkin	e564411a62	app/vmselect/promql: align the behavior of or, and and unless operators with on (labels) modifier to Prometheus ... Previously VictoriaMetrics could return unexpected result of the right-hand side operand had multiple time series with the given set of labels mentioned in `on(labels)`. See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1643	2021-09-24 00:46:25 +03:00
Aliaksandr Valialkin	9f19649672	docs/CHANGELOG.md: cut v1.66.2	2021-09-23 22:53:36 +03:00
Aliaksandr Valialkin	718eca33ab	lib/storage: properly handle {__name__=~"prefix(suffix1|suffix2)",other_label="..."} queries ... They were broken in the commit 00cbb099b6 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1644	2021-09-23 21:48:51 +03:00
Aliaksandr Valialkin	f5896b7420	docs/CHANGELOG.md: document 0e35fc9538 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1641	2021-09-23 20:46:23 +03:00
Aliaksandr Valialkin	a0313c046b	lib/promscrape: add vm_promscrape_max_scrape_size_exceeded_errors_total metric for counting of the failed scrapes due to the exceeded response size ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1639	2021-09-23 14:47:54 +03:00
Aliaksandr Valialkin	9b3d1a1996	docs/CHANGELOG.md: cut v1.66.1	2021-09-22 01:47:05 +03:00
Aliaksandr Valialkin	a13c3de36f	docs/CHANGELOG.md: document 9ca1cbced1 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1635	2021-09-21 23:17:08 +03:00
Aliaksandr Valialkin	207c5760ce	lib/promrelabel: fix parsing regex: true in relabeling rules	2021-09-21 23:00:53 +03:00
Aliaksandr Valialkin	a22aa0608b	app/vmselect: fix accessing /graphite/* endpoints	2021-09-21 13:56:35 +03:00
Aliaksandr Valialkin	76811c2f60	docs/CHANGELOG.md: cut v1.66.0	2021-09-20 15:20:25 +03:00
Aliaksandr Valialkin	9a3d0c43b5	app/vmselect/promql: add quantiles_over_time("phiLabel", phi1, ..., phiN, m[d]) function for calculating multiple quantiles at once	2021-09-17 23:35:10 +03:00
Aliaksandr Valialkin	e1e5a20b36	docs/CHANGELOG.md: document 0e09fdb8b0	2021-09-17 18:47:06 +03:00
Aliaksandr Valialkin	2951dd0a57	app/vmselect/promql: add histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) function ... This function calculates multiple quantiles over the given buckets at once Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573	2021-09-17 13:32:39 +03:00
Aliaksandr Valialkin	8c504d6efa	docs/CHANGELOG.md: document the change in enterprise apps, which allows passing -version without -eula flag ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1621	2021-09-17 12:37:45 +03:00
Aliaksandr Valialkin	e60dfc96ff	app/vmselect/promql: add mad(q) and outliers_mad(tolerance, q) functions to MetricsQL	2021-09-16 13:33:53 +03:00
Aliaksandr Valialkin	44b01fff13	app/{vminsert,vmselect}: automatically add missing port in -storageNode lists passed to vminsert and vmselect ... This should simplify manual setup of the cluster according to https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#cluster-setup	2021-09-15 18:08:30 +03:00
Aliaksandr Valialkin	06ed694ad9	docs/CHANGELOG.md: document 777ff75874	2021-09-15 17:45:08 +03:00
Aliaksandr Valialkin	68ea3d18f7	vendor: update github.com/valyala/histogram from v1.1.2 to v1.2.0 ... This fixes the non-repeatable quantile_over_time() results when the number of input samples exceeds 1000. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1612	2021-09-15 09:34:07 +03:00
Aliaksandr Valialkin	3e1683756b	docs/vmalert.md: follow-up after 68721f6e7d ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1608	2021-09-14 14:47:47 +03:00
Aliaksandr Valialkin	8f685d81c6	lib/storage: follow up after 00cbb099b6	2021-09-14 14:16:25 +03:00
Aliaksandr Valialkin	adedc83b3b	app/vmauth: do not log invalid auth tokens by default for security reasons ... The logging can be enabled by passing `-logInvalidAuthTokens` command-line flag to vmauth	2021-09-14 12:20:03 +03:00
Aliaksandr Valialkin	8a6a36429a	app/vminsert/netstorage: disable rerouting by default ... Production clusters work more stable with the disabled rerouting during rolling restarts and/or during spikes in time series churn rate. So it would be better disabling the rerouting by default. The re-routing can be enabled by passing `-disableRerouting=false` command-line flag to `vminsert` nodes. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/791 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1054 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1165	2021-09-13 18:51:56 +03:00
Aliaksandr Valialkin	c4f11a49f8	docs/CHANGELOG.md: document 5494bc02a6	2021-09-13 17:11:23 +03:00
Aliaksandr Valialkin	7f0a8d4bdb	docs: consistency renaming: Influx -> InfluxDB	2021-09-13 17:05:16 +03:00
Aliaksandr Valialkin	d1a16e0891	app/vmselect/promql: use Prometheus-compatible label value formatting for count_values function	2021-09-13 13:48:06 +03:00
Aliaksandr Valialkin	fb6ed0ce19	lib/promscrape/discovery/docker: support host networking mode ... See https://github.com/prometheus/prometheus/issues/9116	2021-09-13 13:30:16 +03:00
Aliaksandr Valialkin	6295861acd	lib/promscrape/discovery/kubernetes: properly use https scheme for wildcard TLS certificates in ingress target discovery	2021-09-13 13:03:42 +03:00
Aliaksandr Valialkin	728c4c3841	lib/promscrape: generate scrape_timeout_seconds metric per each scrape target in the same way as Prometheus 2.30 does ... See https://github.com/prometheus/prometheus/pull/9247	2021-09-12 15:20:44 +03:00
Aliaksandr Valialkin	48e3e6c8df	lib/promscrape: add ability to configure scrape_timeout and scrape_interval via relabeling ... See https://github.com/prometheus/prometheus/pull/8911	2021-09-12 13:33:41 +03:00
Aliaksandr Valialkin	f3e89754a9	lib/promscrape: reduce CPU usage for common case when calculating scrape_series_added metric ... Also reduce CPU usage when applying `series_limit` to scrape targets with constant set of metrics. The main idea is to perform the calculations on scrape_series_added and series_limit only if the set of metrics exposed by the target has been changed. Scrape targets rarely change the set of exposed metrics, so this optimization should reduce CPU usage in general case.	2021-09-12 12:53:14 +03:00
Aliaksandr Valialkin	9286107e82	lib/promscrape: send stale markers for disappeared metrics like Prometheus does	2021-09-11 10:51:04 +03:00
Aliaksandr Valialkin	f4dead529f	lib/storage: properly search series by multiple tag filters matching empty labels such as foo{bar=~"baz|",x=~"y|"} ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1601 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/395	2021-09-09 21:09:21 +03:00
Aliaksandr Valialkin	ea943911bc	app/vmselect/promql: keep metric name in rollup_candlestick results, since they don't change the original series meaning ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1600	2021-09-09 19:21:18 +03:00
Aliaksandr Valialkin	f27980dcb3	docs/CHANGELOG.md: typo fix	2021-09-09 18:57:30 +03:00
Aliaksandr Valialkin	4aeb8db83f	lib/promscrape: add ability to set series_limit and stream_parse options via relabeling ... This allows managing these options on a per-target basis. Typical use case: to manage these options for pods via Kubernetes annotations.	2021-09-09 18:49:39 +03:00
Aliaksandr Valialkin	d6bd956930	lib/promrelabel: add keep_metrics and drop_metrics actions to relabeling rules ... These actions simlify metrics filtering. For example, - action: keep_metrics regex: 'foo|bar|baz' would leave only metrics with `foo`, `bar` and `baz` names, while the rest of metrics will be deleted. The commit also makes possible to split long regexps into multiple lines. For example, the following config is equivalent to the config above: - action: keep_metrics regex: - foo - bar - baz	2021-09-09 16:18:21 +03:00
Aliaksandr Valialkin	e6c9869d86	app/vmalert: follow-up after 21f022e5f0	2021-09-07 22:43:37 +03:00
Aliaksandr Valialkin	6fbaf8f978	docs/CHANGELOG.md: document 42e07cfaea	2021-09-07 22:34:39 +03:00
Aliaksandr Valialkin	f529058d3a	docs/CHANGELOG.md: cut v1.65.0	2021-09-01 17:12:32 +03:00
Aliaksandr Valialkin	f77dde837a	lib/promscrape: add the ability to limit the number of unique series per each scrape target ... The number of series per target can be limited with the following options: * Global limit with `-promscrape.maxSeriesPerTarget` command-line option. * Per-target limit with `max_series: N` option in `scrape_config` section. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1561	2021-09-01 16:03:59 +03:00
Aliaksandr Valialkin	9e41b05401	docs/CHANGELOG.md: document eff940aa76	2021-09-01 12:00:02 +03:00
Aliaksandr Valialkin	fce87bfe8d	docs/CHANGELOG.md: document 7c70dcbe3b ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1471	2021-09-01 11:56:23 +03:00
Aliaksandr Valialkin	8b228a5873	docs/CHANGELOG.md: add a link to Prometheus staleness tracking	2021-08-31 11:48:32 +03:00
Aliaksandr Valialkin	5c63d69454	lib/promscrape/discovery/kubernetes: return back support role: endpointslices, since it is used by VictoriaMetrics operator ... This is a follow up commit after 31b42b30b6	2021-08-29 12:37:03 +03:00
Aliaksandr Valialkin	db330232ac	lib/protoparser/opentsdb: follow-up after 8ee75ca45a	2021-08-29 11:49:21 +03:00
Aliaksandr Valialkin	31b42b30b6	lib/promscrape/discovery/kubernetes: rename role: endpointslices to role: endpointslice to be consistent with Prometheus ... See 2ec6c7dbb8/discovery/kubernetes/kubernetes.go (L99)	2021-08-29 11:23:08 +03:00
Aliaksandr Valialkin	2e001db4de	lib/promscrape/discovery/kubernetes: use v1 API instead of v1beta1 API for role: ingress and role: endpointslices ... This should fix service discovery for these roles in Kubernetes v1.22 and newer versions. See https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122 The corresponding change in Prometheus - https://github.com/prometheus/prometheus/pull/9205	2021-08-29 11:16:59 +03:00
Aliaksandr Valialkin	5ea689d61b	app/vmselect/promql: add quantile("phiLabel", phi1, ..., phiN, q) aggregate function to MetricsQL ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1573	2021-08-27 18:37:20 +03:00
Aliaksandr Valialkin	bec18e4fe9	app/vmselect: add -search.disableAutoCacheReset command-line option for disabling automatic cache reset when a sample with old timestamp outside -search.cacheTimestampOffset is inserted ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1570	2021-08-27 17:15:31 +03:00
Aliaksandr Valialkin	c8c153fb91	docs/CHANGELOG.md: document the bugfix for possible timeout error in vmbackupmanager when making snapshots ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1571	2021-08-27 13:04:06 +03:00
Aliaksandr Valialkin	10f960fa0c	lib/promscrape: add ability to load scrape configs from multiple files ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1559	2021-08-26 08:51:16 +03:00
Aliaksandr Valialkin	e24203cde8	docs/CHANGELOG.md: document 48f33d098b	2021-08-25 13:31:35 +03:00
Aliaksandr Valialkin	c27ee35c5c	lib/promscrape: expose promscrape_discovery_http_errors_total metric for tracking errors per each http_sd config	2021-08-25 13:05:49 +03:00
Aliaksandr Valialkin	ffc0ab1774	lib/{mergeset,storage}: improve the detection of the needed free space for background merge ... This should prevent from possible out of disk space crashes during big merges. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1560	2021-08-25 09:35:44 +03:00
Aliaksandr Valialkin	0078486ea7	app/vmselect/promql: rename sign() function to sgn() in order to be consistent with Prometheus ... See https://github.com/prometheus/prometheus/pull/8457 for details.	2021-08-23 11:45:51 +03:00
Aliaksandr Valialkin	69c291353b	deployment/docker: update Go builder from Go1.16.0 to Go1.17.0 ... This improves data ingestion and query performance by up to 5% according to benchmarks. See https://go.dev/blog/go1.17	2021-08-21 22:20:49 +03:00
Aliaksandr Valialkin	d5622b32e2	lib/promscrape: reduce memory and CPU usage when Prometheus staleness tracking is enabled for metrics from deleted / disappeared scrape targets ... Store the scraped response body instead of storing the parsed and relabeld metrics. This should reduce memory usage, since the response body takes less memory than the parsed and relabeled metrics. This is especially true for Kubernetes service discovery, which adds many long labels for all the scraped metrics. This should also reduce CPU usage, since the marshaling of the parsed and relabeld metrics has been substituted by response body copying. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526	2021-08-21 21:17:26 +03:00
Aliaksandr Valialkin	6ffbb46aef	docs/CHANGELOG.md: document 9ee3d0378f	2021-08-21 20:20:08 +03:00
Aliaksandr Valialkin	c14edc860b	docs/CHANGELOG.md: document b5d6a0e499	2021-08-19 14:03:20 +03:00
Aliaksandr Valialkin	80ddade4ed	docs/CHANGELOG.md: clarify the change, which adds -search.noStaleMarkers command-line flag	2021-08-19 13:56:04 +03:00
Aliaksandr Valialkin	a724229b5d	app/vmselect/promql: do not override [d] at last_over_time(m[d]) if [d] is smaller than scrape_interval ... Since most users do not expect the overriding of explicitly set `[d]`.	2021-08-19 10:31:48 +03:00
Aliaksandr Valialkin	ce0c270e75	docs/CHANGELOG.md: cut v1.64.1 ... This is mostly bugfix release, which includes fixes for staleness handling and a security update for Alpine base image	2021-08-18 22:06:05 +03:00
Aliaksandr Valialkin	c09446a9aa	lib/promscrape: send stale markers for the previously scraped metrics on failed scrapes like Prometheus does	2021-08-18 21:59:03 +03:00
Aliaksandr Valialkin	04c3e9916d	docs/CHANGELOG.md: document 06bf21c21b	2021-08-18 14:01:04 +03:00
Aliaksandr Valialkin	cdc372bb98	app/vmselect: add -search.noStaleMarkers command-line flag for disabling stale markers handling in queries ... This option allows reducing CPU usage a bit when VictoriaMetrics is used for collecting and processing non-Prometheus data. For example, InfluxDB line protocol, Graphite, OpenTSDB, CSV, etc.	2021-08-18 13:59:02 +03:00
Aliaksandr Valialkin	226143f31b	lib/promscrape: add ability to disable sending Prometheus staleness markers with -promscrape.disableStaleMarkers command-line flag ... This option can be useful when vmagent consumes too much additional memory for staleness markers functionality and when staleness markers aren't needed.	2021-08-18 13:43:21 +03:00
Aliaksandr Valialkin	db1e62495b	app/vmselect/promql: add bitmap_and(), bitmap_or() and bitmap_xor() functions to MetricsQL ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1541	2021-08-17 13:21:21 +03:00
Aliaksandr Valialkin	03c959f1df	lib/promscrape: stop scrapers for the removed targets before starting scrapers for the added targets ... This should prevent from possible time series overlap when old target is substituted by new target (for example, during Kubernetes deployments). Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509	2021-08-17 00:55:51 +03:00
Aliaksandr Valialkin	f37b963619	app/vmalert: follow-up for 2400f85761	2021-08-16 15:20:22 +03:00
Aliaksandr Valialkin	4547d4f692	docs/CHANGELOG.md: update urls to Prometheus 2.29 release ... Previously these urls were pointing to rc0 release	2021-08-16 14:53:38 +03:00
Aliaksandr Valialkin	ae9f923449	docs/CHANGELOG.md: typo fix: satureated -> saturated	2021-08-16 14:53:38 +03:00
Aliaksandr Valialkin	c518858145	docs/CHANGELOG.md: cut v1.64.0	2021-08-15 23:52:03 +03:00
Aliaksandr Valialkin	a0e18f06eb	lib/promscrape: restore red highlighting for DOWN targets at /targets page ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1461	2021-08-15 16:03:57 +03:00
Aliaksandr Valialkin	40a859a760	docs/CHANGELOG.md: mention the bugfix when more than 27 time series are selected at /vmui	2021-08-15 15:10:41 +03:00
Aliaksandr Valialkin	386ee5b82c	docs/CHANGELOG.md: mention that VMUI automatically fills Server URL field ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1506	2021-08-15 14:45:09 +03:00
Aliaksandr Valialkin	aefba16d5e	app/vmagent/remotewrite: expose vmagent_remotewrite_send_duration_seconds_total metric ... This metric can be used for determining high saturation of every connection to remote storage with an alerting query `rate(vmagent_remotewrite_send_duration_seconds_total) > 0.9s`. This query triggers when a connection is satureated by more than 90%	2021-08-15 13:34:12 +03:00
Aliaksandr Valialkin	4401464c22	all: add support for Prometheus staleness markers ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/748 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1509 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1530 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845	2021-08-13 12:10:17 +03:00
Aliaksandr Valialkin	755f65f4bc	app/vminsert: add vm_rpc_send_duration_seconds_total metric per each vminsert->vmstorage link ... This metric is useful for determining high link saturation with the following alerting rule: rate(vm_rpc_send_duration_seconds_total) > 0.9s	2021-08-11 11:44:40 +03:00
Aliaksandr Valialkin	5ce531027f	docs/CHANGELOG.md: document new metrics added to vmalert at 7416fdaa8b	2021-08-05 10:13:08 +03:00
Aliaksandr Valialkin	d826352688	app/vmagent: follow-up after fe445f753b ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1491	2021-08-05 09:52:32 +03:00
Aliaksandr Valialkin	50663ba41f	lib/promscrape/discovery/gce: add __meta_gce_interface_ipv4_<name> labels as in Prometheus 2.29 ... See https://github.com/prometheus/prometheus/pull/8978	2021-08-03 16:11:49 +03:00
Aliaksandr Valialkin	3cad8b4564	lib/promscrape/discovery/ec2: add __meta_ec2_availability_zone_id label as Prometheus 2.29 does	2021-08-03 16:11:49 +03:00
Aliaksandr Valialkin	e92fde7945	app/vmselect/promql: add present_over_time(m[d]) function, which will be available starting from Prometheus 2.29.0 ... See https://github.com/prometheus/prometheus/releases/tag/v2.29.0-rc.0 and https://github.com/prometheus/prometheus/pull/9097	2021-08-03 16:11:49 +03:00
Aliaksandr Valialkin	8a3c13fd53	docs/CHANGELOG.md: typo fix	2021-07-30 12:35:57 +03:00
Aliaksandr Valialkin	a3b4fc0474	docs/CHANGELOG.md: document d05cac6c98 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1486	2021-07-30 12:19:53 +03:00
Aliaksandr Valialkin	a1911e1330	app/vmselect/netstorage: unpack time series data in mostly local big chunks ... This should improve performance on multi-CPU systems for queries selecting time series with big number of raw samples	2021-07-30 12:03:17 +03:00
Aliaksandr Valialkin	192dfbfd90	app/vmselect: follow-up for ed95bc9531 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1493	2021-07-29 09:53:28 +03:00
Aliaksandr Valialkin	8ee8660ac4	app/vmselect: follow-up for 626073bca8 ... * Rename -search.maxMetricsPointSearch to -search.maxSamplesPerQuery, so it is more consistent with the existing -search.maxSamplesPerSeries * Move the -search.maxSamplesPerQuery from vmstorage to vmselect, so it could effectively limit the number of raw samples obtained from all the vmstorage nodes * Document the -search.maxSamplesPerQuery in docs/CHANGELOG.md	2021-07-28 18:00:23 +03:00
Aliaksandr Valialkin	8481f4f004	docs/CHANGELOG.md: document 9d45b46f4c	2021-07-27 12:38:31 +03:00
Aliaksandr Valialkin	95aff47330	app/vmselect: prevent from possible deadlock when f callback blocks inside RunParallel	2021-07-26 15:47:30 +03:00
Aliaksandr Valialkin	05672ffc32	app/vmselect/promql: properly handle (a op b) default N if (a op b) returns NaN series ... The result should be a series with `N` values and `a op b` labels. Previously such series has been removed from the result.	2021-07-16 01:44:58 +03:00
Aliaksandr Valialkin	f4e81aef7e	app/vmselect/netstorage: add -search.maxSamplesPerSeries command-line option for limiting the number of samples a query can process per each series ... This should prevent from out of memory crashes like in https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1067	2021-07-15 16:03:28 +03:00
Aliaksandr Valialkin	171d44acd8	docs/CHANGELOG.md: typo fix: suffxies->suffixes	2021-07-15 15:02:20 +03:00
Aliaksandr Valialkin	61cc13c16f	docs/CHANGELOG.md: cut v1.63.0	2021-07-15 14:02:13 +03:00
Aliaksandr Valialkin	682662b2ae	lib/storage: remove cache directory if it contains reset_cache_on_startup file ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1447	2021-07-13 17:58:51 +03:00
Aliaksandr Valialkin	2df66dad7b	lib/httpserver: add is_set label to flag metrics ... This label allows determining the set flags with the query `flag{is_set="true"}`	2021-07-13 15:10:13 +03:00
Aliaksandr Valialkin	a925d5a3e1	app/vmselect/promql: duration handling improvements in MetricsQL queries ... - Support durations anywhere in MetricsQL queries. E.g. sum_over_time(m[1h])/1h is equivalent to sum_over_time(m[1h])/3600 - Support durations without suffix. E.g. rate(m[300]) is equivalent to rate(m[5m])	2021-07-12 17:16:41 +03:00
Aliaksandr Valialkin	f3a5465ece	docs/CHANGELOG.md: document the change from bfba4c28a4 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1444	2021-07-12 12:44:06 +03:00
Aliaksandr Valialkin	f539772ca6	docs: sync with the cluster branch	2021-07-10 12:45:38 +03:00
Aliaksandr Valialkin	8c764e88f0	app/vmui: move source code from https://github.com/VictoriaMetrics/vmui to app/vmui ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413	2021-07-09 17:15:23 +03:00
Aliaksandr Valialkin	c5f0b454f0	app/vmselect: follow-up after aa11ef6d3b ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1413	2021-07-07 17:43:35 +03:00
Aliaksandr Valialkin	6e0553c92e	lib/mergeset: cache indexBlock items only on the second request ... This should reduce the indexdb/indexBlocks cache size, since it won't contain one-time-wonders items.	2021-07-07 15:23:06 +03:00
Aliaksandr Valialkin	ed944313b0	app/{vminsert,vmselect}: export vminsert_request_duration_seconds and vmselect_request_duration_seconds histograms	2021-07-07 13:25:21 +03:00
Aliaksandr Valialkin	766edbc421	lib/httpserver: print full requestURI in httpserver.Errorf ... This should simplify debugging.	2021-07-07 13:09:40 +03:00
Aliaksandr Valialkin	8f973e34fb	lib/workingsetcache: properly switch to whole mode ... Previously the switch from `split` to `whole` mode had been performed too early, e.g. when the current cache size became bigger than 1/4 of the allowed cache size. Now it is performed when the current cache size becomes bigger than 1/2 of the allowed cache size. This change can reduce memory usage for data ingestion path when big number of active time series are ingested.	2021-07-05 15:16:11 +03:00
Aliaksandr Valialkin	888d62e40c	docs/CHANGELOG.md: document the bugfix for vm_merge_need_free_disk_space metric at 9a83e9018d	2021-07-05 12:01:08 +03:00
Aliaksandr Valialkin	158c50c0ee	lib/promauth: reload TLS certificates from disk on every mTLS connection as Prometheus does ... This allows updating client certificates without the need to restart vmagent and/or single-node VictoriaMetrics. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1420 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/470	2021-07-01 15:27:47 +03:00
Aliaksandr Valialkin	02ed4b8b46	docs/CHANGELOG.md: document ae485c2bfd	2021-07-01 11:51:19 +03:00
Aliaksandr Valialkin	c25b839078	lib/workingsetcache: reset the cache mode when the cache is reset ... This should reduce memory usage if the working set is reduced after the cache reset.	2021-07-01 11:50:11 +03:00
Aliaksandr Valialkin	1b0501a09e	lib/promscrape: typo fix in /targets output ... The typo has been introduced in fb72a2133f Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1408	2021-06-28 21:26:37 +03:00
Aliaksandr Valialkin	008033a374	docs/CHANGELOG.md: cut v1.62.0	2021-06-25 13:29:38 +03:00
Aliaksandr Valialkin	cb5453953f	lib/promscrape: split docker and dockerswarm service discovery code bases, since they have very little in common ... This is a follow up after c85a5b7fcb	2021-06-25 13:20:20 +03:00
Aliaksandr Valialkin	ca42410afd	docs/CHANGELOG.md: document the bugfix in increase_pure() function from the commit fb4f758715	2021-06-24 12:05:39 +03:00
Aliaksandr Valialkin	5d64ed73c5	lib/protoparser/clusternative: do not pool unmarshalWork structs, since they can occupy big amounts of memory (more than 100MB per each struct) ... This should reduce memory usage for vmstorage under high ingestion rate when the vmstorage runs on a system with big number of CPU cores	2021-06-23 15:46:50 +03:00
Aliaksandr Valialkin	cdfae0117a	app/vmselect/promql: return the last timestamp for the max / min value from tmax_over_time() and tmin_over_time() function as most users expect	2021-06-23 14:19:00 +03:00
Aliaksandr Valialkin	70e2852376	docs/CHANGELOG.md: document the bugfix for incorrect stats collection for concurrently executed tag filter ... Follow up for c22114c6f0	2021-06-23 14:05:28 +03:00
Aliaksandr Valialkin	94f3e40ab3	app/vminsert/netstorage: sort the -storageNode list passed to vminsert nodes ... This should reduce resource usage (CPU, RAM, disk IO) at vmstorage nodes if the addresses of vmstorage nodes are passed in random order to vminsert nodes.	2021-06-23 14:01:41 +03:00
Aliaksandr Valialkin	ac54f34f9e	lib/promscrape/discovery/http: follow up after e307bbb29a	2021-06-22 13:40:33 +03:00
Aliaksandr Valialkin	755040a171	lib/promscrape/discovery: support generic auth configs in Consul service discovery in the same way as Prometheus 2.28 does	2021-06-22 13:34:02 +03:00
Aliaksandr Valialkin	59e7755df9	docs/CHANGELOG.md: document the support for Consul namepsace ... See 58a2989fe7	2021-06-22 13:34:02 +03:00
Aliaksandr Valialkin	a3262daac0	docs/CHANGELOG.md: typo fixes	2021-06-18 19:14:53 +03:00
Aliaksandr Valialkin	83a4db813e	app/vmselect: log slow requests to all the /api/v1/* handlers if their execution time exceeds -search.logSlowQueryDuration	2021-06-18 19:04:42 +03:00
Aliaksandr Valialkin	570f36b344	app/vmctl: limit JSON line size by 10K samples (#1394) ... This should reduce the maximum memory usage at VictoriaMetrics when importing time series with big number of samples.	2021-06-18 15:26:47 +03:00
Aliaksandr Valialkin	cbd9159a22	docs/CHANGELOG.md: document the reduced disk write IO usage ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1338	2021-06-18 14:02:59 +03:00
Aliaksandr Valialkin	fb72a2133f	lib/promscrape: show jobs with empty scrape targets on /targets page	2021-06-18 10:53:52 +03:00
Aliaksandr Valialkin	6d17a4e12d	docs/CHANGELOG.md: document the changed -remoteWrite.queues value ... This is a follow-up for 0a796f7c3a See https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1385	2021-06-16 12:35:46 +03:00
Aliaksandr Valialkin	84fb59b0ba	lib/storage: move deletedMetricIDs set from indexDB to Storage ... This makes consitent the list of deleted metricIDs when it is used from both the current indexDB and the previous indexDB (aka extDB). This should fix the issue, which could lead to storing new samples under deleted metricIDs after indexDB rotation. See more details at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347#issuecomment-861232136 . Thanks to @tangqipengleoo for the initial analysis and the pull request - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 . This commit resolves the issue in more generic way compared to https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1383 . The downside of the commit is the deletedMetricIDs set isn't cleaned from the metricIDs outside the retention. It needs app restart. This should be OK in most cases.	2021-06-15 15:04:30 +03:00
Aliaksandr Valialkin	36d55bff66	lib/promscrape: show the number of samples collected during the last scrape at /targets and /api/v1/targets pages ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1377	2021-06-14 14:04:00 +03:00
Aliaksandr Valialkin	05bc9667c1	docs/CHANGELOG.md: document the addition of DigitalOcean service discovery ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1367	2021-06-14 13:18:19 +03:00
Aliaksandr Valialkin	2ef2e3d6f8	docs/CHANGELOG.md: cut v1.61.1	2021-06-11 13:01:31 +03:00
Aliaksandr Valialkin	c4f3fbfa5d	lib/storage: reset cache on disk during series deletion and during indexdb rotation ... This should prevent from inconsistent behavior (aka partially missing data for some time series) after unclean shutdown. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1347	2021-06-11 12:42:28 +03:00
Aliaksandr Valialkin	d979e14da2	docs/CHANGELOG.md: document the bugfix from 7adfe878e1	2021-06-11 11:28:10 +03:00
Aliaksandr Valialkin	b83a51366e	docs/CHANGELOG.md: cut v1.61.0	2021-06-09 19:04:44 +03:00
Aliaksandr Valialkin	329b6cd146	docs/CHANGELOG.md: document the enterprise bugfix for the target property in Graphite Render API	2021-06-09 13:50:52 +03:00
Aliaksandr Valialkin	db1c548cb4	docs/CHANGELOG.md: document improvements in re-routing handling in vminsert ... See the following commits: * 1c09e71f5b * 0d067eb112 * 2c6b917749	2021-06-09 13:40:37 +03:00
Aliaksandr Valialkin	ab15bf8c90	docs: document rules replay feature for vmalert ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/836 This is a follow-up for 2a259ef5e7	2021-06-09 12:27:34 +03:00
Aliaksandr Valialkin	c16edf8287	docs/CHANGELOG.md: document the bugfix, which prevents panics for aborted http requests in vmauth ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1353 This is a follow-up for 6b29b955c0	2021-06-09 12:11:53 +03:00
Aliaksandr Valialkin	78f83dc5ad	app/{vmagent,vminsert}: follow-up after 2fe045e2a4 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1343	2021-06-04 20:27:58 +03:00
Aliaksandr Valialkin	5ac25d2585	docs/CHANGELOG.md: document the bugfix from 6f19bb23a1	2021-06-04 11:53:00 +03:00
Aliaksandr Valialkin	f963f04d3d	app/vminsert: add -disableRerouting command-line flag for disabling re-routing if some vmstorage nodes have lower performance than the others	2021-06-04 04:42:01 +03:00
Aliaksandr Valialkin	d2d746c4fc	docs/CHANGELOG.md: document that it is possible to build VictoriaMetrics components for Solaris ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1322 This is a follow-up for ddc8022702	2021-05-31 09:33:29 +03:00
Aliaksandr Valialkin	b22e380a34	app/vmauth: allow balancing the load among multiple backend nodes by specifying multiple urls in url_prefix config	2021-05-29 01:03:37 +03:00
Aliaksandr Valialkin	60341722d5	docs: document f0c21b6300	2021-05-27 15:03:30 +03:00
Aliaksandr Valialkin	7f531e3a60	docs/CHANGELOG.md: document changes from 2233d6ed8a and d210958fd0	2021-05-26 12:23:22 +03:00
Aliaksandr Valialkin	08234aa7a0	docs/CHANGELOG.md: cut v1.60.0	2021-05-24 15:55:08 +03:00
Aliaksandr Valialkin	f54133b200	lib/storage: do not populate MetricID->MetricName cache during data ingestion ... This cache isn't needed during data ingestion, so there is no need in spending RAM on it. This reduces RAM usage on data ingestion path by 30%	2021-05-24 03:02:46 +03:00
Aliaksandr Valialkin	24858820b5	docs/CHANGELOG.md: small typo fix	2021-05-23 14:15:01 +03:00
Aliaksandr Valialkin	04eb37a590	docs/CHANGELOG.md: document the addition of extra_filter_labels at 84cc0513e1	2021-05-23 14:10:33 +03:00
Aliaksandr Valialkin	78dddfb98f	lib/promauth: follow-up after 5b8176c68e	2021-05-22 18:01:11 +03:00
Aliaksandr Valialkin	8e2985b53d	lib/fs: wait for a while before giving up on NFS file removal if the removal queue is full ... This should reduce the probability of the panic on a highly loaded VictoriaMetrics accepting millions of samples per second. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1313	2021-05-21 17:21:00 +03:00
Aliaksandr Valialkin	14ec2b9f26	docs/CHANGELOG.md: mention the bugfix from d626c5c2a9 ... Updates https://github.com/VictoriaMetrics/operator/issues/243	2021-05-21 16:37:14 +03:00
Aliaksandr Valialkin	c54bb73867	all: do not skip SIGHUP signal during service initialization ... This can lead to stale or incomplete configs like in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240	2021-05-21 16:34:06 +03:00
Aliaksandr Valialkin	0d3e78b9ee	docs/CHANGELOG.md: move tip to proper place	2021-05-20 17:56:03 +03:00
Aliaksandr Valialkin	ad73f226ff	app/vmstorage: add ability to limit series cardinality via -storage.maxHourlySeries and -storage.maxDailySeries command-line flags	2021-05-20 14:15:19 +03:00
Aliaksandr Valialkin	7e526effaa	app/vmagent: add ability to limit series cardinality on a per-hour and per-day basis	2021-05-20 13:13:40 +03:00
Aliaksandr Valialkin	3cd8606abd	docs/CHANGELOG.md: document the bugfix in vmctl import for InfluxDB lines with identical names for field and tag ... See dcf8803bbd Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1299	2021-05-20 12:06:16 +03:00
Aliaksandr Valialkin	98e425ee09	docs/CHANGELOG.md: refer to the issue related to timezone_offset() function	2021-05-20 12:03:39 +03:00
Aliaksandr Valialkin	0842bb9294	app/vmselect/promql: add timezone_offset(tz) function ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1306	2021-05-20 11:53:09 +03:00
Aliaksandr Valialkin	b30925738b	docs/vmalert.md: document multitenant support ... https://github.com/VictoriaMetrics/VictoriaMetrics/issues/740	2021-05-18 16:26:14 +03:00
Aliaksandr Valialkin	66aba00549	app/vmauth: reload -auth.config on the request to /-/reload ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1194	2021-05-18 02:23:55 +03:00
Aliaksandr Valialkin	ede2ba5a45	docs/CHANGELOG.md: document b38edec7ee ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1293	2021-05-17 01:57:34 +03:00
Aliaksandr Valialkin	733706e6c6	lib/promscrape: reload auth tokens from files every second ... Previously auth tokens were loaded at startup and couldn't be updated without vmagent restart. Now there is no need in vmagent restart. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1297	2021-05-14 20:00:08 +03:00
Aliaksandr Valialkin	10a47af631	app/{vmalert,vmauth}: explicitly set MaxIdleConnsPerHost in net/http.Client.Transport ... By default MaxIdleConnsPerHost is set to 2. This limits the possibility to re-use http keep-alive connections. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1300	2021-05-14 18:12:24 +03:00
Aliaksandr Valialkin	fc3519fa26	lib/promscrape: limit scrape_timeout by scrape_interval like Prometheus does	2021-05-13 16:09:45 +03:00
Aliaksandr Valialkin	1f75ae6006	docs/CHANGELOG.md: document the bugfix from b4f5be8bd8	2021-05-13 11:18:39 +03:00
Aliaksandr Valialkin	e6fda03e8f	vendor: update github.com/VictoriaMetrics/fasthttp from v1.0.14 to v1.0.15 ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1289	2021-05-13 10:44:14 +03:00
Aliaksandr Valialkin	c0ec541559	lib/cgroup: document the ability to detect cgroup v2 memory and cpu limits. This is follow-up for b50024812e	2021-05-13 09:26:20 +03:00
Aliaksandr Valialkin	832651c6c2	app/vmselect: follow up after 8a0678678b ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1168	2021-05-12 17:18:30 +03:00
Aliaksandr Valialkin	f2d5c4e2d0	lib/httpserver: return X-Server-Hostname http header in all the responses for better debuggability	2021-05-11 22:03:48 +03:00
Aliaksandr Valialkin	229d9d6dd7	docs/CHANGELOG.md: document -datasource.roundDigits added at 5c448126dc	2021-05-10 11:18:26 +03:00
Aliaksandr Valialkin	3b0966c00c	docs/CHANGELOG.md: document vmalert fix for state restoration on startup	2021-05-10 11:10:04 +03:00
Aliaksandr Valialkin	cbd0569ce2	docs/CHANGELOG.md: mention the comment, which gives an example of multi-level vminsert setup	2021-05-08 22:57:04 +03:00
Aliaksandr Valialkin	7aea5f58c4	lib/ingestserver: properly close incoming connections during graceful shutdown	2021-05-08 19:52:58 +03:00
Aliaksandr Valialkin	12d733dd5d	app/vminsert: add support for data ingestion via other vminsert nodes	2021-05-08 19:52:57 +03:00
Aliaksandr Valialkin	d9e3872b1c	docs/CHANGELOG.md: document 904bbffc7f ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240	2021-05-05 20:33:05 +03:00
Aliaksandr Valialkin	4bea1afc6d	docs/CHANGELOG.md: document 9cdd4696fe ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1252	2021-05-05 20:28:58 +03:00
Aliaksandr Valialkin	ce9e163e94	lib/httpserver: stop the process on panics in request handlers ... Panics may leave the process in inconsistent state. That's why it is better to stop the process after the panic instead of recovering from the panic. Unfortunately, the standard net/http.Server recovers panics in request handlers. See https://github.com/golang/go/issues/16542 . That's lib/httpserver must stop the process on itself after the panic.	2021-05-03 11:59:40 +03:00
Aliaksandr Valialkin	988c3b386f	docs/CHANGELOG.md: document the bugfix for proper removal of stale parts (477369b62f)	2021-05-03 11:38:15 +03:00
Aliaksandr Valialkin	fb097ff774	docs/CHANGELOG.md: cut v1.59.0	2021-05-01 09:39:48 +03:00
Aliaksandr Valialkin	58d1e6eeea	lib/storage: log dropped labels if the number of labels in a metric exceeds -maxLabelsPerTimeseries command-line flag value ... This should improve debuggability for this case.	2021-05-01 09:27:55 +03:00
Aliaksandr Valialkin	4394dc6cbb	docs/CHANGELOG.md: document the change from f3a048288e ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1232	2021-04-30 09:54:41 +03:00
Aliaksandr Valialkin	6fa5981e68	app/vmagent: list user-visible endpoints at http://vmagent:8429/ ... While at it, use common WriteAPIHelp function for the listing in vmagent, vmalert and victoria-metrics	2021-04-30 09:36:43 +03:00
Aliaksandr Valialkin	56b6b893ce	lib/mergeset: split rows ingestion among multiple shards ... This improves rows ingestion on systems with many CPU cores by reducing lock contention. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1244 Thanks to @waldoweng for the original idea and draft implementation at https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1243	2021-04-27 15:36:34 +03:00
Aliaksandr Valialkin	60947fb2d5	lib/persistentqueue: eliminate possible data race when obtaining vm_persistentqueue_bytes_pending metric value	2021-04-27 00:25:52 +03:00
Aliaksandr Valialkin	20b77abc17	docs: update docs order	2021-04-24 01:27:13 +03:00
Aliaksandr Valialkin	908e35affd	lib/promscrape: apply scrape_timeout on receiving the first response byte for stream_parse: true scrape targets ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1017#issuecomment-767235047	2021-04-23 21:53:35 +03:00
Aliaksandr Valialkin	ae37cfd528	lib/promscrape/discovery/kubernetes: refresh endpoints and endpointslices targets on service object update like Prometheus does ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1240	2021-04-23 20:11:40 +03:00
Aliaksandr Valialkin	bbebdf9ba1	lib/{storage,mergeset}: remove empty directories on startup. Such directories can be left after unclean shutdown on NFS storage ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1142	2021-04-22 13:02:44 +03:00
Aliaksandr Valialkin	6bc52fe41a	all: rename https://victoriametrics.github.io to https://docs.victoriametrics.com	2021-04-20 20:16:17 +03:00
Aliaksandr Valialkin	187e3ec909	app/vmauth: follow-up for 6a81a89b3d	2021-04-20 10:58:29 +03:00
f41gh7	e5db518c0f	updates per-tenant stats docs ... changes docs order changes per-tenant-stats pic	2021-04-14 12:14:51 +03:00
Roman Khavronenko	7644efae01	Docs update (#1199) ... * docs: drop table of contents for `vmctl` We already have it autogenerated on .github.io, so no need to keep it. * docs: mention OpenTSDB migration feature for vmctl * docs: sync docs for `vmalert`	2021-04-10 15:46:08 +03:00
Aliaksandr Valialkin	f1a22b097a	docs/CHANGELOG.md: cut v1.58.0	2021-04-08 00:48:16 +03:00
Aliaksandr Valialkin	1177dca3da	app/vmselect: do not sort series returned from topk* and bottomk* functions, since these series are already sorted in user-expected order ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1189	2021-04-07 14:16:08 +03:00
Roman Khavronenko	c6a8ebb11f	docs: update docs ordering and formatting (#1192) ... The major change is adding `sort` directive to docs. For those docs which are copied from internal packages `sort` is added via makefile command. For the rest it is added manually since they're updated manually as well. The rest of changes is connected with markdown formatting. For example, changing headers in some files (`##` => `#`) makes navigation on .github.io to look better. This especially useful for `changelog` docs. Table of contents for `vmctl` is dropped, since we already have it autogenerated on .github.io. No link changes expected. The corresponding PR to `cluster` branch will be made in follow-up PR.	2021-04-07 13:39:16 +03:00
Aliaksandr Valialkin	df32d2836c	lib/storage: properly handle big time ranges passed to /api/v1/labels and /api/v1/label/<labelName>/values ... It should be faster querying all the labels and/or all the values instead of querying per-day labels/values on time ranges exceeding maxDaysForPerDaySearch	2021-04-07 13:33:46 +03:00
Aliaksandr Valialkin	78d35d4f46	Makefile: prepare arm64 and amd64 release archives for cluster version on make release command	2021-04-05 23:03:19 +03:00
Aliaksandr Valialkin	95dbebf512	lib/persistentqueue: delete corrupted persistent queue instead of throwing a fatal error ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1030	2021-04-05 19:26:11 +03:00
Aliaksandr Valialkin	f010d773d6	lib/promscrape/discovery/kubernetes: synchronously load Kubernetes objects on first access ... Remove async registration of apiWatchers, since it breaks discovering `role: endpoints` and `role: endpointslices` targets, which depend on pod and service objects. There is no need in reloading `endpoints` and `endpointslices` targets if the referenced `pod` or `service` objects change, since in this case the corresponding `endpoints` and `endpointslices` objects should also change because they contain ResourceVersion of the referenced `pod` or `service` objects, which is modified on object update. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1182	2021-04-05 14:20:12 +03:00
Aliaksandr Valialkin	a5c5b54c22	lib/proxy: add support for socks5 over tls proxy	2021-04-05 13:00:05 +03:00
Aliaksandr Valialkin	6742839fd6	lib/promscrape: pass X-Prometheus-Scrape-Timeout-Seconds header to scrape targets as Prometheus does	2021-04-05 12:15:24 +03:00
Aliaksandr Valialkin	9ff3ecb991	docs/CHANGELOG.md: explain why -sortLabels is set to false by default	2021-04-04 01:59:25 +03:00
Aliaksandr Valialkin	fc2240fb22	docs/CHANGELOG.md: document the ability to use socks5 proxy ... Follow-up for a4c6a3b3e1 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1177	2021-04-04 01:42:00 +03:00
Aliaksandr Valialkin	5153410ced	lib/promscrape: support for simple HTTP proxies without CONNECT method support such as https://github.com/prometheus-community/PushProx ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1179	2021-04-04 00:40:40 +03:00
Aliaksandr Valialkin	7a0b964e8d	app/vmselect/promql: do not delete dst_label if src_label is empty in label_copy(q, src_label, dst_label) and label_move(q, src_label, dst_label)	2021-04-03 22:05:06 +03:00
Aliaksandr Valialkin	6f3080f9fb	docs/CHANGELOG.md: document the change from 3055ab0115 (add ability to pass "label=value" to the third argument to topk_* and bottomk_* functions	2021-04-03 21:42:08 +03:00
Aliaksandr Valialkin	b8d9d6c326	docs/CHANGELOG.md: yet another typo fix	2021-04-03 00:25:39 +03:00
Aliaksandr Valialkin	22949911e9	docs/CHANGELOG.md: typo fix	2021-04-03 00:23:50 +03:00
Aliaksandr Valialkin	b1d0028e79	app/vmauth: add support for authorization via Authorization: Bearer <token>	2021-04-02 22:14:53 +03:00
Aliaksandr Valialkin	df148f48b7	lib/promscrape: add support for authorization config in -promscrape.config as Prometheus 2.26 does ... See https://github.com/prometheus/prometheus/pull/8512	2021-04-02 21:17:45 +03:00
Aliaksandr Valialkin	7f9c68cdcb	lib/promscrape: add follow_redirect option to scrape_configs section like Prometheus does ... See https://github.com/prometheus/prometheus/pull/8546	2021-04-02 19:56:40 +03:00
Aliaksandr Valialkin	12e4785fe8	lib/promscrape/discovery/kubernetes: properly discover targets in multiple namespaces ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1170	2021-04-02 14:28:30 +03:00
Aliaksandr Valialkin	3967bd705a	docs/CHANGELOG.md: mention about AWS IAM roles for tasks support for EC2 service discovery ... Follow-up for fdb8995642	2021-04-02 13:10:03 +03:00
Aliaksandr Valialkin	dc9eafcd02	app/{vminsert,vmagent}: add -sortLabels command-line option for sorting time series labels before ingesting them in the storage ... This option can be useful when samples for the same time series are ingested with distinct order of labels. For example, metric{k1="v1",k2="v2"} and metric{k2="v2",k1="v1"}.	2021-03-31 23:27:58 +03:00
Aliaksandr Valialkin	e1f699bb6c	lib/storage: reduce memory usage when ingesting samples for the same time series with distinct order of labels	2021-03-31 21:24:46 +03:00
Aliaksandr Valialkin	48275d8c12	app/vmagent/remotewrite: reduce memory usage when -remoteWrite.queues is set to a big value ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1167	2021-03-31 16:16:33 +03:00
Aliaksandr Valialkin	2d3082bb55	docs/CHANGELOG.md: cut v1.57.1	2021-03-30 15:40:06 +03:00
Aliaksandr Valialkin	0fe8f11090	docs/CHANGELOG.md: mention about returned back type label for vm_tenant_inserted_rows_total metric ... See 9b4e608199	2021-03-30 15:16:57 +03:00
Aliaksandr Valialkin	d58d5562f1	app/vmselect: remove -search.storageTimeout command-line flag, since it has the same meaning as -search.maxQueryDuration ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711	2021-03-30 15:04:54 +03:00
Aliaksandr Valialkin	7962cf1af8	app/vmselect: prevent from possible incomplete query results after timed out query ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/711	2021-03-30 13:35:45 +03:00
Aliaksandr Valialkin	75991277fa	Makefile: build vmutils for arm on make release-vmutils ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1147	2021-03-29 23:15:29 +03:00
Aliaksandr Valialkin	947b37ba8e	docs/CHANGELOG.md: cut v1.57.0	2021-03-29 19:14:49 +03:00
Aliaksandr Valialkin	e6fd1f7875	docs/CHANGELOG.md: typo fixes	2021-03-29 15:46:35 +03:00
Aliaksandr Valialkin	5c7c0b2bda	docs/CHANGELOG.md: mention about logging of metrics with too old timestamps in a single-node VictoriaMetrics ... This is a follow up for aa81039b42	2021-03-29 15:42:46 +03:00
Aliaksandr Valialkin	5f4a9f782f	docs/CHANGELOG.md: mention Graphite Render API fixes	2021-03-29 14:27:42 +03:00
Aliaksandr Valialkin	602a3d99ae	docs/CHANGELOG.md: mention optimized query performance on systems with many CPU cores	2021-03-29 13:55:10 +03:00
Aliaksandr Valialkin	78188decf9	docs: document that vmagent drops data blocks when remote storage replies with 400 and 409 http status codes ... This is a follow up for 1b7dc1e5a5. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1149	2021-03-26 14:44:06 +02:00
Aliaksandr Valialkin	9761ffd161	lib/promscrape/discovery/kubernetes: properly handle too old resource version error message from Kubernetes watch API	2021-03-26 12:28:10 +02:00
Aliaksandr Valialkin	0b2a66db30	app/vmselect/promql: do not merge time series during requests to /api/v1/query ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1141	2021-03-25 13:56:07 +02:00
Aliaksandr Valialkin	d4aadba9fa	app/vmagent: add -promscrape.consul.waitTime command-line flag for configuring Consul service discovery wait time ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1144	2021-03-23 19:33:25 +02:00
Aliaksandr Valialkin	edc0a94a3c	docs/CHANGELOG.md: mention the feature from 44a6cc5eca	2021-03-23 19:00:18 +02:00
Aliaksandr Valialkin	331a6a2015	app/vmselect/graphite: accept and enforce extra_label in all the Graphite APIs	2021-03-23 15:29:16 +02:00
Aliaksandr Valialkin	b0c956a178	app/vmselect/graphite: follow-up after 529d7be26b	2021-03-18 16:30:20 +02:00
Aliaksandr Valialkin	6cee5338b2	lib/storage: prevent from infinite loop if {__graphite__="..."} filter matches a metric name with *, [ or { chars ... The idea has been borrowed from https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1137	2021-03-18 14:53:47 +02:00
Aliaksandr Valialkin	17d4a6e900	vendor: update github.com/VictoriaMetrics/metrics from v1.16.0 to v1.17.0	2021-03-17 23:23:04 +02:00
Aliaksandr Valialkin	904dababcc	vendor: update github.com/VictoriaMetrics/metrics from v1.15.3 to v1.16.0 ... This adds the following new metrics for each VictoriaMetrics app: * process_resident_memory_anonymous_bytes - the RSS share for memory allocated by the process itself. This share cannot be freed by the OS, so it must be taken into account by OOM killer. * process_resident_memory_pagecache_bytes - the RSS share for page cache memory (aka memory-mapped files). This share can be freed by the OS at any time, so it must be ignored by OOM killer.	2021-03-17 17:59:40 +02:00
Aliaksandr Valialkin	20ac89c4e0	docs/CHANGELOG.md: cut v1.56.0	2021-03-17 02:04:55 +02:00
Aliaksandr Valialkin	bd0c6a095e	docs/CHANGELOG.md: do not mention reduction in query duration ... There was a signficant refactoring in the code responsible for time series search, so it can result in both speed ups and slow downs depending on used queries.	2021-03-17 01:56:58 +02:00
Aliaksandr Valialkin	7bc728bf53	app/vmselect: add vm_index_search_duration_seconds histogram for monitoring the performance of index search	2021-03-17 01:17:41 +02:00
Aliaksandr Valialkin	6378205415	lib/netutil: enable IPv6 UDP listening if -enableTCP6 command-line flag is passed to VictoriaMetrics ... This is a follow-up for 18cfc4be7b See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1131	2021-03-17 00:16:17 +02:00
Aliaksandr Valialkin	aba955fa16	Makefile: prepare vmutils-windows-*.zip archive on make release-vmutils command ... The archive contains the following executables for Windows: * vmagent * vmalert * vmauth * vmctl Other components - vmbackup, vmrestore, victoria-metrics - aren't supported for Windows yet	2021-03-16 20:52:41 +02:00
Aliaksandr Valialkin	f0a4157f89	app/vmselect/promql: do not crash if histogram_over_time() function name contains uppercase letters such as Histogram_over_time()	2021-03-16 12:24:21 +02:00
Aliaksandr Valialkin	dd7e82c34f	app/vmstorage: add -logNewSeries command-line flag for determining the source of series churn rate	2021-03-15 22:38:50 +02:00
Aliaksandr Valialkin	7c37e9aea9	app/{vminsert,vmagent}: a follow-up for b1aa8c3d8f ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1124	2021-03-15 21:37:55 +02:00
Aliaksandr Valialkin	fb935e6e2c	docs/CHANGELOG.md: typo fix: FATURE -> FEATURE	2021-03-15 14:58:19 +02:00
Aliaksandr Valialkin	2dae0a2c47	app/vmselect: add round_digits query arg to /api/v1/query and /api/v1/query_range handlers for limiting the number of decimal digits after the point	2021-03-15 12:36:33 +02:00
Aliaksandr Valialkin	7c002023d7	app/vmselect/prometheus: do not include datapoints with timestamps matching t-d when returning results from /api/v1/query?query=m[d]&time=t as Prometheus does	2021-03-12 12:16:50 +02:00
Aliaksandr Valialkin	a6a71ef861	lib/promscrape: add ability to configure proxy options via proxy_tls_config, proxy_basic_auth, proxy_bearer_token and proxy_bearer_token_file ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1116	2021-03-12 03:36:19 +02:00
Aliaksandr Valialkin	bc5c4add89	lib/httpserver: export vm_available_memory_bytes and vm_available_cpu_cores metrics ... These metrics are useful for tracking the available memory and CPU cores for VictoriaMetrics apps.	2021-03-10 12:02:42 +02:00
Aliaksandr Valialkin	2728b25783	docs/CHANGELOG.md: mention about the bugfix from 787242d7b0	2021-03-09 20:56:06 +02:00
Aliaksandr Valialkin	36fd007247	lib/proxy: set missing ServerName in TLS config for proxy_url. ... While at it, allow setting Proxy-Authorization for `proxy_url` via `basic_auth` and `bearer_token` configs.	2021-03-09 18:58:18 +02:00
Aliaksandr Valialkin	3fd8653b40	lib/promscrape: apply sample_limit after metric relabeling is applied as Prometheus does ... See the description for `sample_limit` option from Prometheus docs: Per-scrape limit on number of scraped samples that will be accepted. If more than this number of samples are present after metric relabeling the entire scrape will be treated as failed. 0 means no limit. https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config	2021-03-09 15:47:18 +02:00
Aliaksandr Valialkin	e44532b760	docs/CHANGELOG.md: mention about improved query performance at 18fe0ff14b	2021-03-09 13:12:25 +02:00
Aliaksandr Valialkin	fe8b12fbad	app/vmselect/promql: follow up for 433fff0006	2021-03-09 12:48:44 +02:00
Aliaksandr Valialkin	f6878eac36	vendor: update github.com/VictoriaMetrics/fasthttp from 1.0.12 to 1.0.13 ... This should fix a bug in vmagent with high CPU usage during failed scrapes with small `scrape_timeout`.	2021-03-09 11:45:02 +02:00
John Belmonte	364fdf4a56	spelling fix: adjacent (#1115)	2021-03-09 09:18:19 +02:00
Aliaksandr Valialkin	14a399dd06	lib/promscrape: add scrape_offset option to scrape_config ... This option can be used for specifying the particular offset per each scrape interval for target scraping	2021-03-08 12:03:33 +02:00
Aliaksandr Valialkin	1187ee5e16	lib/decimal: prevent exponent overflow when processing values close to zero ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1114	2021-03-05 18:52:47 +02:00
Aliaksandr Valialkin	47ac2051bb	app/vmauth: allow using regexps in url_map paths ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1112	2021-03-05 18:21:36 +02:00
Aliaksandr Valialkin	17b87725ed	lib/promscrape/discovery/kubernetes: reduce memory usage when Kubernetes service discovery is configured on a big number of scrape jobs ... Previously vmagent was creating a separate Kubernetes object cache per each scrape job. This could result in increased memory usage when monitoring a Kubernetes cluster with big number of objects (pods / nodes / services, etc.) as seen at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1113 Now it uses a shared map of scrape objects across multiple scrape jobs.	2021-03-05 17:29:55 +02:00
Aliaksandr Valialkin	a14053ffa0	app/vmselect/promql: add histogram_avg(), histogram_stddev() and histogram_stdvar() functions to MetricsQL	2021-03-04 14:12:07 +02:00
Aliaksandr Valialkin	423cd981fb	lib/promscrape: add -promscrape.cluster.replicationFactor command-line flag for replicating scrape targets among vmagent instances in the cluster	2021-03-04 10:20:15 +02:00
Aliaksandr Valialkin	3a68b94487	docs/CHANGELOG.md: cut v1.55.1 release	2021-03-03 11:49:10 +02:00
Aliaksandr Valialkin	4c3ef78c05	docs/CHANGELOG.md: mention recent bugfixes from commits 7906316741 and e154f4a644	2021-03-03 10:50:31 +02:00
Aliaksandr Valialkin	1252ca44d8	docs/CHANGELOG.md: cut v1.55.0	2021-03-02 21:37:58 +02:00
Aliaksandr Valialkin	22b1941cfc	lib/promscrape/discovery/ec2: follow-up after f6114345de	2021-03-02 13:46:26 +02:00
Aliaksandr Valialkin	937f382938	lib/protoparser/prometheus: properly unescape label values in Prometheus exposition format ... Unescape only `\n`, `\"` and `\\` sequences as Prometheus does. Other escape sequences shouldn't be unescaped.	2021-03-02 13:21:43 +02:00
Aliaksandr Valialkin	019d8e88d8	lib/protoparser/graphite: fix parsing of a Graphite line with empty tags such as foo; 1 2 ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1100	2021-03-01 17:16:35 +02:00
Aliaksandr Valialkin	2c43e846a9	docs/CHANGELOG.md: mention the out of range panic bugfix d6a41b6ea2	2021-03-01 16:54:24 +02:00
Aliaksandr Valialkin	732e729ef9	docs/CHANGELOG.md: mention the issue related to using Kubernetes watch API for service discovery	2021-03-01 01:42:11 +02:00
Aliaksandr Valialkin	cb943f35c7	app/vmagent: remove data race when applying rate limits to -remoteWrite.url with multiple queues	2021-03-01 00:29:07 +02:00
Aliaksandr Valialkin	1da1d502a8	docs/CHANGELOG.md: mention about https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1074	2021-02-28 19:31:44 +02:00
Aliaksandr Valialkin	9fc7726d84	lib/httpserver: make sure the gzipResponseWriter.Write() is called on Flush() and Close() calls ... This should fix the `http: superfluous response.WriteHeader call` issue See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1078	2021-02-28 19:22:50 +02:00
Aliaksandr Valialkin	e5ca8ac0db	lib/promscrape: add ability to spread scrape targets among multiple vmagent instances ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1084	2021-02-28 18:41:08 +02:00
Aliaksandr Valialkin	03903c1176	docs/CHANGELOG.md: mention 317b0cbed2	2021-02-28 14:02:49 +02:00
Aliaksandr Valialkin	7cc3d96a41	lib/fs: follow-up after f3a03c4164	2021-02-27 01:01:47 +02:00
Aliaksandr Valialkin	975dac9086	docs/CHANGELOG.md: mentioned a bugfix with extra_label handling during caching query results ... Related to 186c078fac	2021-02-27 00:20:29 +02:00
Aliaksandr Valialkin	8683ea85e6	lib/fs: properly handle stale NFS file handle error during file deletion ... This error can appear when -storageDataPath points to NFS volume and the given file has been already removed.	2021-02-26 23:25:14 +02:00
Aliaksandr Valialkin	d86e9b49c4	app/vmselect/promql: increase accuracy for buckets_limit() function for small limits by skipping the first and the last buckets during merge ... The first and the last buckets are usually `[0 ... leMin]` and `(leMax ... +Inf)`. If they are merged with adjancent buckets, then the resulting accuracy can suffer.	2021-02-26 22:56:36 +02:00
Aliaksandr Valialkin	c8f2f9b2e8	lib/promscrape: cleanup after 9b2246c29b ... Main points: * Revert changes outside lib/promscrape/discovery/kuberntes . These changes can be applied later in a separate commit * Minimize changes in lib/promscrape/discovery/kubernetes compared to a93e644001 * Corner case fixes.	2021-02-26 16:54:05 +02:00
Aliaksandr Valialkin	98854e5f2b	app/vmselect: add sign(q) and clamp(q, min, max) functions, which will be added in the upcoming Prometheus release ... See https://twitter.com/roidelapluie/status/1363428376162295811 The `last_over_time(m[d])` function already exists in MetricsQL.	2021-02-24 17:24:56 +02:00
Aliaksandr Valialkin	f050e3f492	docs/CHANGELOG.md: mention about a bugfix from 4805b80977	2021-02-24 11:49:10 +02:00
Aliaksandr Valialkin	f4135b0d14	app/vmselect/promql: properly calculate histogram_quantile() over zero buckets and only a single non-zero le="+Inf"` bucket like Prometheus does	2021-02-24 00:42:22 +02:00
Aliaksandr Valialkin	fa03e0d210	app/vmselect/promql: add increase_pure() function to MetricsQL ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/962	2021-02-22 19:14:15 +02:00
Aliaksandr Valialkin	dd1e53b119	lib/promrelabel: optimize relabeling performance for common cases	2021-02-22 00:51:13 +02:00
Aliaksandr Valialkin	ff5bbc4b88	lib/promscrape: export vm_promscrape_target_relabel_duration_seconds metric	2021-02-21 23:21:42 +02:00
Aliaksandr Valialkin	901e12024d	vendor: update github.com/VictoriaMetrics/metrics from v1.14.0 to v1.15.0 ... The v1.15.0 exports the following additional metrics: process_io_read_bytes_total - the number of bytes read via io syscalls such as read and pread process_io_written_bytes_total - the number of bytes written via io syscalls such as write and pwrite process_io_read_syscalls_total - the number of read syscalls such as read and pread process_io_write_syscalls_total - the number of write syscalls such as write and pwrite process_io_storage_read_bytes_total - the number of bytes read from storage layer process_io_storage_written_bytes_total - the number of bytes written to storage layer These metrics can be used for monitoring process io	2021-02-21 22:54:00 +02:00
Aliaksandr Valialkin	c2678754e4	app/vmagent: properly perform graceful shutdown, which was broken in the commit 1d1ba889fe ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065	2021-02-19 00:31:34 +02:00
Aliaksandr Valialkin	f26162ec99	lib/promscrape: add scrape_align_interval config option into scrape config ... This option allows aligning scrapes to a particular intervals.	2021-02-18 23:53:44 +02:00
Aliaksandr Valialkin	9c70c1f21f	app/vmselect/promql: reduce the probability of duplicate time series errors when querying Kubernetes metrics	2021-02-18 22:07:29 +02:00
Aliaksandr Valialkin	5e341ccb59	docs/CHANGELOG.md: cut v1.54.1	2021-02-18 19:09:59 +02:00
Aliaksandr Valialkin	a537c4f602	lib/storage: properly handle queries containing a filter on metric name plus any number of negative filters and zero non-negative filters ... Example: `node_cpu_seconds_total{mode!="idle"}`	2021-02-18 18:46:36 +02:00
Aliaksandr Valialkin	a7697cc88b	docs/CHANGELOG.md: cut v1.54.0	2021-02-18 14:52:38 +02:00
Aliaksandr Valialkin	edcdc39eb3	app/vmagent/remotewrite: cleanup after 1d1ba889fe ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1065	2021-02-17 21:42:55 +02:00
Aliaksandr Valialkin	4b110fa21c	docs/CHANGELOG.md: mention that prod binaries are built now with Go1.16 ... This is a follow-up for 4edfe76bef	2021-02-17 21:06:59 +02:00
Aliaksandr Valialkin	bea2f86b7b	docs/CHANGELOG.md: document new per-tenant metrics	2021-02-16 23:34:24 +02:00
Aliaksandr Valialkin	9f5ac603a7	lib/storage: reduce the minimum supported retention for inverted index from one month to one day	2021-02-15 15:12:29 +02:00
Aliaksandr Valialkin	38d7e96602	lib/promscrape/discovery/kubernetes: add __meta_kubernetes_endpoints_label_* and __meta_kuberntes_endpoints_annotation_* labels to role: endpoints ... This syncs kubernetes SD with Prometheus 2.25 See 617c56f55a	2021-02-15 02:51:16 +02:00
Aliaksandr Valialkin	c727d2219b	lib/storage: properly hanle regexp tag filters with dots, which can be converted to full string match filters. ... For example `{label=~"foo\.bar"}` should be converted to `{label="foo.bar"}`. Previously it has was mistakenly conveted to `{label="foo\.bar"}` . This could result in missing time series for such tag filters.	2021-02-14 23:38:14 +02:00
Aliaksandr Valialkin	73c95c4e5b	docs/CHANGELOG.md: mention about fixed multiarch build for Docker images ... Related commit: f9902b3372	2021-02-12 15:23:49 +02:00
Aliaksandr Valialkin	1e38ad6d20	app/vmauth: add ability to route requests from a single users to multiple targets depending on the requested path ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1064	2021-02-11 12:41:16 +02:00
Aliaksandr Valialkin	04faea8b45	app/vmselect: parallelize q1 <binary_op> q2 queries by running q1 and q2 in parallel ... This should reduce query execution times.	2021-02-10 22:59:39 +02:00
Aliaksandr Valialkin	553016ea99	lib/storage: disable composite index usage when querying old data	2021-02-10 14:57:50 +02:00
Aliaksandr Valialkin	c7dccebaef	lib/storage: optimize search by label filters matching big number of time series	2021-02-10 00:44:54 +02:00
Aliaksandr Valialkin	a4140de9e6	lib/mergeset: unconditionally cache indexdb blocks ... Production workloads show that indexdb blocks must be cached unconditionally for reducing CPU usage. This shouldn't increase memory usage too much, since unused blocks are removed from the cache every two minutes.	2021-02-09 00:47:50 +02:00
Aliaksandr Valialkin	4dca03501b	docs/CHANGELOG.md: mention about a bugfix for timezone data from df0cda3ab9	2021-02-08 15:58:25 +02:00
Aliaksandr Valialkin	c5770600a2	lib/cgroup: follow-up after b9bf3cbe3e	2021-02-08 15:54:38 +02:00
Aliaksandr Valialkin	803a00102a	app/{vminsert,vmselect}: accept requests to paths with /graphite and /prometheus prefixes ... This should improve compatibility with path prefixes from VictoriaMetrics cluster. See https://victoriametrics.github.io/Cluster-VictoriaMetrics.html#url-format	2021-02-04 20:01:18 +02:00
Aliaksandr Valialkin	b44edc7832	docs/CHANGELOG.md: mention recently added changes	2021-02-04 16:41:34 +02:00
Aliaksandr Valialkin	9fb38569eb	lib/httpserver: expose process_open_fds and process_max_fds metrics ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/402 See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1037	2021-02-04 16:40:50 +02:00
Aliaksandr Valialkin	814455138a	docs/CHANGELOG.md: cut v1.53.1	2021-02-03 23:45:21 +02:00
Aliaksandr Valialkin	ea00dac35f	docs/CHANGELOG.md: mention that {__graphite__="foo.*.bar"} syntax deprecates -search.treatDotsAsIsInRegexps command-line flag	2021-02-03 20:41:29 +02:00
Aliaksandr Valialkin	a5a1b9bd66	lib/storage: fix a bug, which breaks searching by Graphite wildcard filters	2021-02-03 20:14:22 +02:00
Aliaksandr Valialkin	9d41c06db1	docs/CHANGELOG.md: fix a link to Graphite Render API usage docs	2021-02-03 12:29:37 +02:00
Aliaksandr Valialkin	c7e03f30d8	docs: mention about Graphite render API implementation	2021-02-03 12:12:04 +02:00
Aliaksandr Valialkin	8c568b13b2	docs/CHANGELOG.md: cut v1.53.0	2021-02-03 03:42:31 +02:00
Aliaksandr Valialkin	157c02622b	app/vmselect: add ability to set Graphite-compatible filter via {__graphite__="foo.*.bar"} syntax	2021-02-03 01:21:54 +02:00
Aliaksandr Valialkin	4068f8d590	lib/promscrape: add vm_promscrape_service_discovery_duration_seconds metric	2021-02-02 16:15:25 +02:00
Aliaksandr Valialkin	bd11fd8f1d	lib/promscrape: add vm_promscrape_scrape_retries_total, vm_promscrape_discovery_retries_total and vm_promscrape_discovery_requests_total metrics	2021-02-01 20:06:27 +02:00
Aliaksandr Valialkin	b43b498fd8	app/vmselect: add ability to pass extra_label=<label>=<value> query arg to Prometheus Querying API ... This enforced `{label="value"}` label filter to the query. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1021	2021-02-01 18:04:17 +02:00
Aliaksandr Valialkin	5d87dbfd65	docs: document ability to query Graphite datasource from vmalert	2021-02-01 15:26:33 +02:00
Aliaksandr Valialkin	b2aa80e74b	app/vmagent: add -remoteWrite.roundDigits command-line option for limiting the number of digits after the point for stored values ... This commit also adds --vm-round-digits command-line option to vmctl tool.	2021-02-01 14:27:09 +02:00
Aliaksandr Valialkin	d5c180e680	app/vmctl: move vmctl code from github.com/VictoriaMetrics/vmctl ... It is better developing vmctl tool in VictoriaMetrics repository, so it could be released together with the rest of vmutils tools such as vmalert, vmagent, vmbackup, vmrestore and vmauth.	2021-02-01 01:10:20 +02:00
Aliaksandr Valialkin	929f09b90d	docs/CHANGELOG.md: typo fixes	2021-01-27 01:18:48 +02:00
Aliaksandr Valialkin	fc5b26d856	lib/promscrape: export vm_promscrape_scrapes_failed_per_url_total and vm_promscrape_scrapes_skipped_by_sample_limit_per_url_total metrics ... These metrics could be useful for determining imporperly working scrape targets. Note that these metrics are exported only for failing scrape targets. They aren't exposed for normally working targets.	2021-01-27 00:39:26 +02:00
Aliaksandr Valialkin	419ad74269	app/vmagent: add -remoteWrite.rateLimit command-line flag for limiting data rate to remote storage ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1035	2021-01-27 00:39:26 +02:00
Aliaksandr Valialkin	3fe848cdd7	lib/logger: add -loggerTimezone command-line flag for adjusting timezone for timestamps in log messages	2021-01-26 22:51:54 +02:00
Aliaksandr Valialkin	5481906db6	docs/CHANGELOG.md: mention about https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1027	2021-01-26 16:37:36 +02:00
Aliaksandr Valialkin	8cea3c3cc4	lib/promscrape: retry scrape and service discovery requests when the remote server closes http keep-alive connection	2021-01-22 13:22:33 +02:00
Aliaksandr Valialkin	3caac3d12c	docs/CHANGELOG.md: mention about the fix with too big HTTP reconnection rate to targets ... This has been fixed in 0a45220b0a	2021-01-22 12:09:16 +02:00
Aliaksandr Valialkin	8749c2dd92	app/vmselect: add -search.maxStepForPointsAdjustment command-line flag, which can be used for disabling adjustment for points returned from /api/v1/query_range handler if they have timestamps closer than -search.latencyOffset to the current time	2021-01-19 22:56:32 +02:00
Aliaksandr Valialkin	7ce87ebcb2	docs/CHANGELOG.md: cut v1.52.0	2021-01-13 12:58:51 +02:00
Aliaksandr Valialkin	1051d8aa2d	app/vmselect/promql: add ability to pass multiple labels to sort_by_label and sort_by_label_desc functions ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/992	2021-01-13 12:44:51 +02:00
Aliaksandr Valialkin	bdd0a1cdb2	lib/backup: increase backup chunk size from 128MB to 1GB ... This should reduce costs for object storage API calls by 8x. See https://cloud.google.com/storage/pricing#operations-pricing	2021-01-13 12:16:35 +02:00
Aliaksandr Valialkin	acf1a2c72b	app/vmselect/promql: properly parse escaped multibyte utf8 code sequences in metric names and labels names ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/990	2021-01-13 10:59:42 +02:00
Aliaksandr Valialkin	89315d719d	docs/CHANGELOG.md: document updated extra_label query arg behavior ... Follow-up for dc9d7aedd5	2021-01-13 00:58:20 +02:00
Aliaksandr Valialkin	7373986f9e	docs/CHANGELOG.md: mention that the minimum supported TLS version now is v1.2 ... Follow-up for 7bf5d48315	2021-01-13 00:44:39 +02:00
Aliaksandr Valialkin	fe3444b124	deployment/docker: upgrade base image for Docker packages from Alpine 3.13.1 to Alpine 3.12.3 in order to fix potential security issues ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1010	2021-01-12 21:57:01 +02:00
Aliaksandr Valialkin	1837f2f7d3	app/vmselect/promql: add tfirst_over_time(m[d]) and tlast_over_time(m[d]) MetricsQL functions for returning timestamps for the first and the last samples in m over d	2021-01-12 16:12:12 +02:00
Aliaksandr Valialkin	2140ccbdcc	docs/CHANGELOG.md: document big fixes from the commit 7976c22797	2021-01-12 13:44:17 +02:00
Aliaksandr Valialkin	2c44f9989a	lib/promscrape: properly show scrape duration on /targets page ... Previously it has been shown as 0.000s for any scrape duration.	2021-01-11 21:14:46 +02:00
Aliaksandr Valialkin	89611fa48c	docs/CHANGELOG.md: mention about a bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/989	2021-01-11 13:11:41 +02:00
Aliaksandr Valialkin	24ffad74c1	all: use net.Dial instead of fasthttp.Dial, because fasthttp.Dial limits the number of concurrent dials to 1000	2021-01-11 12:53:30 +02:00
Aliaksandr Valialkin	9dcb18e03d	app/vmstorage: disable final merge by default, since it may result in high disk IO and CPU usage without measurable benefits such as increased query performance and reduced disk space usage	2021-01-08 00:16:05 +02:00
Aliaksandr Valialkin	70df5f4975	docs/CHANGELOG.md: cut v1.51.0	2020-12-27 14:21:29 +02:00
Aliaksandr Valialkin	c86286ec1d	app/vmselect/promql: do not ajdust offset value provided in the query ... Previously it could be modified in order to improve response cache hit ratio. This is unneeded, since cache hit ratio should remain good because the query time range should be already aligned to multiple of `step` values. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/976	2020-12-27 14:09:25 +02:00
Aliaksandr Valialkin	59183f66d0	app/vmselect: refactor /api/v1/stats/top_queries ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/907	2020-12-25 16:44:29 +02:00
Aliaksandr Valialkin	932e53522d	docs/CHANGELOG.md: mention that vmalert now properly escapes multi-line queries when passing to Grafana ... A follow-up for 1de15ad490 See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/890	2020-12-25 11:12:06 +02:00
Aliaksandr Valialkin	4eb520a342	docs/CHANGELOG.md: mention about adding missing __meta_kubernetes_service_* labels for endpoints and endpointslices roles in kubernetes_sd_config ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/982	2020-12-24 11:33:00 +02:00
Aliaksandr Valialkin	820669da69	lib/promscrape: code prettifying for 8dd03ecf19	2020-12-24 10:56:10 +02:00
Aliaksandr Valialkin	9df60518bb	docs: mention that it is possible to set multiple -notifier.tlsInsecureSkipVerify command-line flags for vmalert ... See c3a92968343c2b3619f1ab935702d0e9b3a46733	2020-12-22 22:32:13 +02:00
Aliaksandr Valialkin	46dba00756	lib/storage: remove stale parts as soon as they go outside the configured retention ... Previously such parts could remain undeleted for long durations until they are merged with other parts. This should help for `-retentionPeriod` values smaller than one month.	2020-12-22 19:54:31 +02:00
Aliaksandr Valialkin	e15b20dde3	docs/CHANGELOG.md: cut v1.50.2	2020-12-19 15:32:34 +02:00
Aliaksandr Valialkin	13804bda8f	docs/CHANGELOG.md: mention about bugfix for populating template variables in vmalert ... See 404cbd1522 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/974	2020-12-19 14:16:04 +02:00
Aliaksandr Valialkin	88ac4dfc07	app/vmselect: properly parse negative combined offsets such as -1h2m3s ... Previously such offsets were parsed as `-1h + 2m + 3s`. Now they are parsed as `-(1h + 2m + 3s)`.	2020-12-19 01:23:46 +02:00
Aliaksandr Valialkin	17c2ce18fd	docs/CHANGELOG.md: there is no visible difference for CPU usage and disk IO usage on production workloads	2020-12-19 00:51:41 +02:00
Aliaksandr Valialkin	ebf8da3730	lib/{storage,mergeset}: tune background merge process in order to reduce CPU usage and disk IO usage	2020-12-18 20:01:08 +02:00
Aliaksandr Valialkin	e6666da4e7	docs/CHANGELOG.md: mention that Docker images for vmagent, vmalert, vmauth, vmbackup and vmrestore with tags containing -cluster suffix are no longer published ... See 441822c4cc for details	2020-12-18 20:01:03 +02:00
Aliaksandr Valialkin	9abb2d6c74	lib/protoparser/prometheus: follow-up commit after 7d38627b9f6f212ae602aea6a72f469fe3c70ba2 ... Document the bugfix in docs/CHANGELOG.md and add a test for the bugfix.	2020-12-16 23:40:17 +02:00
Aliaksandr Valialkin	795e32be4a	docs/CHANGELOG.md: cut v1.50.1 release	2020-12-15 21:11:39 +02:00
Aliaksandr Valialkin	e8f645bf52	docs/CHANGELOG.md: mention about bugfix, which properly removes unregistered targets from /targets page ... See bugfix at a4c7fcb5e1	2020-12-15 21:06:59 +02:00
Aliaksandr Valialkin	aa56b9217e	app/vmagent: add vmagent_remotewrite_blocks_sent_total and vmagent_remotewrite_bytes_sent_total metrics per each -remoteWrite.url	2020-12-15 20:39:57 +02:00
Aliaksandr Valialkin	a13a443bf7	docs/CHANGELOG.md: cut v1.50.0 release	2020-12-15 14:44:42 +02:00
Aliaksandr Valialkin	b730fc2667	lib/promscrape: properly handle scrape errors when stream parsing is enabled ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/967	2020-12-15 14:08:28 +02:00
Aliaksandr Valialkin	11fa458e39	app/vmselect/promql: return expected increase() result for the first point on the graph with value not exceeding 100	2020-12-15 13:40:46 +02:00
Aliaksandr Valialkin	2813d0b1e0	docs/CHANGELOG.md: mention that vmagent now accepts multiple -remoteWrite.sendTimeout and -remoteWrite.tlsInsecureSkipVerify command-line flags	2020-12-15 12:58:35 +02:00
Aliaksandr Valialkin	ae3675d3d0	docs/CHANGELOG.md: mention about adding query, first and value functions to alert templates ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/539	2020-12-14 20:17:11 +02:00
Aliaksandr Valialkin	5d426dfe0a	docs/CHANGELOG.md: mention bugfix for proper recovering from incorrectly stored persistent queue ... See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/964	2020-12-14 19:28:47 +02:00
Aliaksandr Valialkin	069c9ade52	app/{vmagent,vminsert}: follow-up for ce8c2dd1f1: return /targets page in HTML when requested via web browser	2020-12-14 14:06:00 +02:00
Aliaksandr Valialkin	5ebfc275e6	app/victoria-metrics: automatically reset response cache when samples with too timestamps older than now - search.cacheTimestampOffset are ingested	2020-12-14 13:08:28 +02:00
Aliaksandr Valialkin	081aa4ad68	docs/CHANGELOG.md: mention https://github.com/VictoriaMetrics/VictoriaMetrics/issues/955	2020-12-11 17:48:26 +02:00
Aliaksandr Valialkin	ba8ac08739	app/vmselect/graphite: properly handle wildcards and charsets inside curly braces ... For example, `foo{bar*,[a-f]a*b}` should match `foobar`, `foobar123`, `foofab`, etc. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/952	2020-12-11 17:24:12 +02:00
Aliaksandr Valialkin	3ca1ed0fde	docs/CHANGELOG.md: mention about memory leak fix in vmagent when big number of targets is discovered via service discovery	2020-12-09 10:35:26 +02:00
Aliaksandr Valialkin	fd9fd191b9	lib/promscrape/discovery/consul: properly pass Datacenter filter to Consul API server ... Previously it has been passed as `sdc` query arg, while it should be passed as `dc` query arg. See https://www.consul.io/api-docs/health#list-nodes-for-service for details. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574#issuecomment-740454170	2020-12-08 21:52:42 +02:00
Aliaksandr Valialkin	4146fc4668	all: properly handle CPU limits set on the host system/container ... This can reduce memory usage on systems with enabled CPU limits. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/946	2020-12-08 21:07:29 +02:00
Aliaksandr Valialkin	0f1b969aa6	lib/promscrape: export vm_promscrape_scrapers_{started|stopped}_total metrics for monitoring target churn rate	2020-12-08 11:57:52 +02:00
Aliaksandr Valialkin	05813259dc	lib/promscrape: export vm_promscrape_active_scrapers{type="<sd_type>"} metric for tracking the number of active scrapers per each service discovery type	2020-12-08 01:54:23 +02:00
Aliaksandr Valialkin	9c1c9d8e76	lib/promscrape: do not enable strict config parsing when -promscrape.config.dryRun command-line flag is passed ... Strict parsing for -promscrape.config can be enabled by passing `-promscrape.config.strictParse` command-line flag. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/944	2020-12-07 13:18:50 +02:00
Aliaksandr Valialkin	007dbf273d	app/vmselect/graphite: remove duplicate name tag from /tags/autoComplete/tags handler ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/942	2020-12-07 01:08:52 +02:00
Aliaksandr Valialkin	b0e4b234cb	lib/protoparser/influx: allow multiple whitespace chars between measurement, fields and timestamp in Influx line protocol	2020-12-06 12:01:27 +02:00
Aliaksandr Valialkin	ae1db8fa08	docs/CHANGELOG.md: cut v1.49.0	2020-12-05 13:49:04 +02:00
Aliaksandr Valialkin	d305cc2017	deployment/docker: update Go builder from v1.15.5 to v1.15.6 ... This fixes issues found in Go since v1.15.5 - see https://github.com/golang/go/issues?q=milestone%3AGo1.15.6+label%3ACherryPickApproved	2020-12-05 12:37:34 +02:00
Aliaksandr Valialkin	e2e8ef86d9	app/vmselect/promql: add count_eq_over_time(m[d], N) and count_ne_over_time(m[d], N) for calculating the number of samples in m over d that are equal / not equal to N	2020-12-05 12:30:46 +02:00
Aliaksandr Valialkin	9d787f9edd	all: do not print usage info for all the flags when incorrect command-line flag is passed ... This should improve usability for VictoriaMetrics apps that have big number of command-line flags, i.e. all the apps.	2020-12-03 21:47:37 +02:00
Aliaksandr Valialkin	66379cc69f	app/vmselect/promql: add label_uppercase(q, label1, ... labelN) and label_lowercase(q, label1, ... labelN) functions ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/936	2020-12-03 21:47:36 +02:00
Aliaksandr Valialkin	5009b25a03	lib/promscrape: code cleanup after c6dee6c52d ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/574	2020-12-03 19:50:53 +02:00
Aliaksandr Valialkin	1982505c2b	app/vmselect/promql: return nan from minute(m) when m equals to nan ... This aligns VictoriaMetrics behaviour with Prometheus behaviour. The issue has been spotted in https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/	2020-12-02 20:16:58 +02:00
Aliaksandr Valialkin	91a4c279cc	app/vmselect: return metric values from time() cmp_op metric query when cmp_op comparison is true ... This aligns MetricsQL behavior to Prometheus' one. The issue has been identified at https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/	2020-12-02 12:09:34 +02:00
Aliaksandr Valialkin	7590b8477b	vendor: upgrade github.com/VictoriaMetrics/metricsql from v0.7.2 to v0.7.3 ... This fixes parsing of hex numbers in MetricsQL such as 0x3b The bug has been detected at https://promlabs.com/promql-compliance-test-results/2020-12-01/victoriametrics/	2020-12-02 08:10:43 +02:00
Aliaksandr Valialkin	700bda8e2e	app/vmselect/promql: return nan from a >bool b if a is nan in the same way as Prometheus does	2020-12-02 00:28:26 +02:00
Aliaksandr Valialkin	a906b3862f	lib/protoparser/prometheus: properly parse OpenMetrics timestamps ... OpenMetrics timestamps are floating-point numbers, that represent Unix timestamp in seconds. This differs from Prometheus exposition format, where timestamps are integer numbers representing Unix timestamp in milliseconds.	2020-11-27 14:54:29 +02:00
Aliaksandr Valialkin	2cea4d403f	all: typo fix: thouthand->thousand	2020-11-26 13:33:46 +02:00
Aliaksandr Valialkin	4cb6bcd2d7	docs/CHANGELOG.md: cut v1.48.0 release	2020-11-26 02:05:57 +02:00
Aliaksandr Valialkin	6b1317b6a4	docs/CHANGELOG.md: add a link to Netflix Eureka - https://github.com/Netflix/eureka	2020-11-26 01:36:20 +02:00
Aliaksandr Valialkin	b7fcdb528d	app/{vmagent,victoria-metrics}: add -dryRun option and make more clear handling for -promscrape.config.dryRun	2020-11-25 22:59:13 +02:00
Aliaksandr Valialkin	8a057e705a	lib/storage: log metric name plus all its labels when the metric timestamp is outside the configured retention ... This should simplify debugging when the source of the metric with unexpected timestamp must be found.	2020-11-25 14:41:37 +02:00
Aliaksandr Valialkin	ae04378424	lib/protoparser/prometheus: properly parse "infinity" values in OpenMetrics format ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/924	2020-11-24 19:03:38 +02:00
Aliaksandr Valialkin	d0ffb49ee2	docs/CHANGELOG.md: mention that /tags/delSeries handler is supported after f0c207fae2	2020-11-24 12:34:56 +02:00
Aliaksandr Valialkin	b7f4fc6e0d	lib/protoparser/prometheus: properly parse metrics with exemplars ... Examplars have been introduced in OpenMetrics - see https://github.com/OpenObservability/OpenMetrics/blob/master/OpenMetrics.md#exemplars-1 Previously VictoriaMetrics couldn't parse the following metric foo{bar="baz"} 123 # exemplar here This commit fixes this. Note that VictoriaMetrics ignores the exemplar as for now.	2020-11-24 12:34:56 +02:00
Aliaksandr Valialkin	aa90b93778	lib/promscrape: expose __meta_ec2_ipv6_addresses label for ec2_sd_config like Prometheus will do in the next release	2020-11-23 16:56:42 +02:00
Aliaksandr Valialkin	de523c81b9	lib/promscrape: add filters option to dockerswarm_sd_config like Prometheus did in v2.23.0	2020-11-23 16:27:40 +02:00
Aliaksandr Valialkin	e614a14b21	docs: sync with cluster branch	2020-11-23 00:42:04 +02:00
Nikolay	eec76718e9	Adds eureka service discovery (#913) ... * Adds eureka service discovery https://github.com/VictoriaMetrics/VictoriaMetrics/issues/851 Netflix service discovery for AWS * Apply suggestions from code review Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>	2020-11-20 13:38:12 +02:00
Aliaksandr Valialkin	de7f315231	docs/CHANGELOG.md: mention that slow query log now contains remote client address	2020-11-19 12:41:17 +02:00
Aliaksandr Valialkin	97a0c80904	lib/logger: follow-up for 09105ff49c	2020-11-19 12:37:00 +02:00
Aliaksandr Valialkin	170e2f54ab	docs/CHANGELOG.md: mention about snap install victoriametrics	2020-11-18 19:49:54 +02:00
Aliaksandr Valialkin	8b116b619a	docs/CHANGELOG.md: sync with cluster branch	2020-11-18 19:46:05 +02:00
Aliaksandr Valialkin	1c477bc2fc	docs/CHANGELOG.md: cut v1.47.0 release	2020-11-16 21:00:06 +02:00
Aliaksandr Valialkin	30a922f383	docs/CHANGELOG.md: mention about Graphite Tags API implementation	2020-11-16 15:34:20 +02:00
Aliaksandr Valialkin	27a417bcd3	app/vmselect/prometheus: improve performance for /api/v1/labels and /api/v1/label/<labelName>/values on time ranges exceeding one day when match[] query arg is set	2020-11-16 13:51:59 +02:00
Aliaksandr Valialkin	5d6d2ef3a6	docs/CHANGELOG.md: mention about improved performance for /api/v1/series on a time range exceeding one day	2020-11-16 13:21:13 +02:00
Aliaksandr Valialkin	6c0d36e4a9	app/vmselect: propagate errors from vmstorage to response to the client if -search.denyPartialResponse command-line flag is set ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/891 This commit also adds `"isPartial":{true|false}` field to `/api/v1/*` responses. `"isPartial":true` is set when the response is based on a partial data because some of vmstorage nodes weren't available during query processing.	2020-11-14 12:47:48 +02:00
Aliaksandr Valialkin	ef9a8989fd	docs/Single-server-VictoriaMetrics.md: document /internal/force_flush endpoint	2020-11-13 18:43:10 +02:00
Aliaksandr Valialkin	67b41c080d	docs/CHANGELOG.md: mentioned that Go builder has been updated from v1.15.4 to v1.15.5 ... See 3fa9ab4a49 for details.	2020-11-13 16:22:12 +02:00
Aliaksandr Valialkin	1f19c167a4	app/vmselect/promql: assume that time series value doesnt change during gaps when calculating increase() and delta() ... This should remove unexpected spikes at the end of gaps. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/894	2020-11-13 14:59:24 +02:00
Aliaksandr Valialkin	57a4af98fa	lib/protoparser/promremotewrite: forward errors, which can occur during data ingestion, to the caller of ParseStream, so it could properly return HTTP 503 status code on non-nil error ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/896	2020-11-13 11:01:07 +02:00
Aliaksandr Valialkin	47a038401b	all: consistently return text-based HTTP responses with charset=utf-8 ... This is a follow-up for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/897	2020-11-13 10:35:41 +02:00
Aliaksandr Valialkin	348edd92fe	app/vmselect: add -search.treatDotsAsIsInRegexps command-line flag for automatic escaping of dots in regexp label filters	2020-11-11 12:39:07 +02:00
Aliaksandr Valialkin	e205975716	lib/promscrape: make a copy of ScrapeWork from discovered []ScrapeWork slice instead of referring to an item in this slice ... This should prevent from holding previously discovered []ScrapeWork slices when a part of discovered targets changes over time. This should reduce memory usage for the case when big number of discovered scrape targets changes over time. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825	2020-11-10 16:13:05 +02:00
Aliaksandr Valialkin	47390d8947	app/vmselect/promql: do not return data points in the end of the selected time range for time series ending in the middle of the selected time range ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/887 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/845	2020-11-10 14:51:44 +02:00
Aliaksandr Valialkin	0d7a3f4eb3	docs/CHANGELOG.md: mention abot explicit setting of extra labels in alert entities (see 3adf8c5a6f) ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/870	2020-11-10 00:40:51 +02:00
Aliaksandr Valialkin	bcd139362b	lib/promscrape: add -promscrape.dropOriginalLabels command-line flag for reducing memory usage when discovering big number of scrape targets ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/878 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/825	2020-11-10 00:19:57 +02:00
Aliaksandr Valialkin	41813eb87a	CHANGELOG.md: cut v1.46.0	2020-11-07 17:52:45 +02:00
Aliaksandr Valialkin	bb3b513bdd	docs/CHANGELOG.md: make docs-sync	2020-11-07 16:30:58 +02:00
Aliaksandr Valialkin	55e98e265e	docs/CHANGELOG.md: add CHANGELOG header	2020-11-07 01:15:00 +02:00
Aliaksandr Valialkin	dd6bfa50e9	app/vmselect/promql: code cleanup after 43823addea	2020-11-06 01:30:50 +02:00
Aliaksandr Valialkin	944c5ea331	docs: add CHANGELOG.md	2020-11-05 14:58:22 +02:00