Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-19 07:01:02 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
8,311 Commits 114 Branches 748 Tags 596 MiB
0bed453737
Commit Graph

3991 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
eecc5e8463
lib/storage: wait for up to 60 seconds before deciding to delete metricID entries from indexdb if metricID->metricName entry is missing during search 
The metricID->metricName entry can remain invisible for search for some time after registering new metricName.
This is expected condition. So wait for up to 60 seconds in the hope that the metricID->metricName
entry will become visible before deleting all the entries from indexdb, which are associated with the given metricID.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5959
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5948

See also 20812008a7
 2024-03-18 00:37:11 +02:00
Aliaksandr Valialkin
111d0aa2bf
app/{vmagent,vminsert}: add an ability to ignore input samples outside the current aggregation interval for stream aggregation 
See https://docs.victoriametrics.com/stream-aggregation.html#ignoring-old-samples
 2024-03-17 23:30:46 +02:00
Aliaksandr Valialkin
e70b644f1f
lib/streamaggr: ignore out of order samples when calculating increase, increase_prometheus, total and total_prometheus outputs 
Out of order samples may result in unexpected spikes for these outputs.
So it is better to ignore such samples.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5931
 2024-03-17 23:24:14 +02:00
Aliaksandr Valialkin
1f753a049a
lib/streamaggr: follow-up for 15e33d56f1 
- Properly set pushSample.timestamp when flushing de-duplicated samples to stream aggregation
  This is needed for https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5931

- Re-classify this change as feature instead of bugfix at docs/CHANGELOG.md

- Verify de-duplication logic for samples with different timestamps

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5643
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5939
 2024-03-17 23:23:57 +02:00
Aliaksandr Valialkin
c3dbcc7c0e
docs/CHANGELOG.md: clarify that -datasource.lookback commnad-line flag is no-op in the upcoming release 
Document the solution - to switch to eval_delay option at group config.

This is a follow-up for e80b44f19d
 2024-03-17 23:23:31 +02:00
Aliaksandr Valialkin
b481448c53
docs/CHANGELOG.md: document the bugfix from cb259116b4 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5802
 2024-03-17 23:23:11 +02:00
Aliaksandr Valialkin
7c4d7dc6dd
lib/promauth: properly set Host header in requests to scrape targets. 
The `Host` header must be set via net/http.Request.Host field, since net/http.Client
ignores this header if it is set via Request.Header.Set().

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5969
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5970
 2024-03-17 23:22:54 +02:00
hagen1778
efddda945e
app/vmctl: fix arguments order in httputils.TLSConfig 
follow-up after 9d5bf5ba5d

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-03-17 23:22:08 +02:00
Github Actions
eb82f329cb
Automatic update operator docs from VictoriaMetrics/operator@ae6e1b6 (#5964) 2024-03-17 23:16:52 +02:00
Daria Karavaieva
dbdbbacf93
docs/vmanomaly: fix 404 links (#5968) 2024-03-17 23:16:15 +02:00
hagen1778
770dd7affd
docs: mention missing vmalert change for memory usage reduction in 1.97.3 
521f9ffb43
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-03-17 23:15:23 +02:00
hagen1778
41a1efbea8
docs: follow-up 15e33d56f1 
Update documentation according to changes in deduplication logic.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-03-17 23:07:51 +02:00
Andrii Chubatiuk
a58a81d80b
lib/streamaggr: pick sample with bigger timestamp or value on deduplicator (#5939) 
Apply the same deduplication logic as in https://docs.victoriametrics.com/#deduplication
This would require more memory for deduplication, since we need to track timestamp
for each record. However, deduplication should become more consistent.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5643

---------

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
 2024-03-17 23:06:37 +02:00
Github Actions
61dc9f5c8d
Automatic update operator docs from VictoriaMetrics/operator@de88675 (#5958) 2024-03-17 20:48:56 +02:00
Zakhar Bessarab
5f1f1f2982
docs: explicitly mention VMUI is available in cluster (#5955) 
It is confusing for cluster users to find that VMUI is available at vmselect as it is only mentioned in the list of URLs. Explicit mention of vmselect URL in docs will make it easier to discover.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
 2024-03-17 20:48:07 +02:00
hagen1778
7b2f19a63e
docs: mention bug investigation in 1.99 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-03-17 20:46:48 +02:00
Hui Wang
349564fd82
vmalert: deprecate cmd-line flag -datasource.lookback (#5877) 
* vmalert: deprecate cmd-line flag `-datasource.lookback`

* fix lint

* review fixes

Signed-off-by: hagen1778 <roman@victoriametrics.com>

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit e80b44f19d)
 2024-03-12 16:17:40 +01:00
Aliaksandr Valialkin
abf347ec0f
docs: replace speed up with more clear accelerate wording 2024-03-12 03:03:15 +02:00
Aliaksandr Valialkin
f92d4609e2
lib/storage: optimize /api/v1/labels and /api/v1/label/.../values when match[] contains metric name 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2978
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5055
 2024-03-12 03:01:47 +02:00
nemobis
14b913eb51
docs: fix typo in stalenes (#5950) 
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
 2024-03-12 02:59:43 +02:00
Github Actions
221f0a000b
Automatic update operator docs from VictoriaMetrics/operator@9b1a6e6 (#5946) 2024-03-12 02:57:08 +02:00
Artem Navoiev
1348a822f8
docs:vmbackup fix typo sped -> speed 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-03-12 02:55:08 +02:00
Zakhar Bessarab
217893921c
docs: fix typo in flags description (#5942) 
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
 2024-03-12 02:45:46 +02:00
Github Actions
b2b034ddb7
Automatic update operator docs from VictoriaMetrics/operator@9caa896 (#5941) 2024-03-10 13:00:35 +02:00
Github Actions
9d57092a58
Automatic update operator docs from VictoriaMetrics/operator@403a78a (#5932) 2024-03-10 13:00:22 +02:00
hagen1778
86abbe5b67
app/vmctl: support TLS configuration for VictoriaMetrics destination 
VictoriaMetrics destination is specified via `--vm-*` cmd-line flags
and is used in opentsdb, influx, prometheus, remote-read modes.

updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5426

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 0b7ce70df4)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-03-08 21:15:23 +01:00
hagen1778
b1fe25792c
app/vmctl: follow-up b9f7c3169a 
* fix typos in flags description
* move the change to #tip section in changelog

b9f7c3169a
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 83a8c24281)
 2024-03-08 21:15:09 +01:00
Khushi Jain
98c0909a12
app/vmctl : Provide TLS config options for vm native protocol (#5824) 
Co-authored-by: Khushi Jain <khushi.jain@nokia.com>
(cherry picked from commit b9f7c3169a)
 2024-03-08 21:15:09 +01:00
Aliaksandr Valialkin
b21fb355fa
docs/vmauth.md: update -help output after e08b91baafc95da090f75e9c29a27d8f62a2b76e 2024-03-07 01:37:48 +02:00
Aliaksandr Valialkin
b9788d5cf8
docs/CHANGELOG.md: typo fixes 2024-03-07 01:35:41 +02:00
Aliaksandr Valialkin
d9e3670627
lib/promauth: set the Host header to tlsServerName if itsn't empty 
If tlsServerName isn't empty, then it is likely the https request is sent to IP instead of hostname.
In this case the request will fail, since Go automatically sets the Host header to the IP instead
of the desired hostname at tlsServerName. So set the Host header to tlsServerName if itsn't empty.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5802
 2024-03-07 01:23:40 +02:00
Aliaksandr Valialkin
e7e9721476
docs/vmauth.md: typo fixes after 7b2b980181 2024-03-07 01:08:40 +02:00
Aliaksandr Valialkin
93ef217857
app/vmauth: allow discovering backend ips behind shared hostname and spreading load among the discovered ips 
This is done with the `discover_backend_ips` option at `user` and `url_map` level.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5707
 2024-03-07 01:02:38 +02:00
Aliaksandr Valialkin
b2e182ca62
app/vmauth: add src_headers option at url_map, which allows routing incoming requests to different backends depending on request headers 2024-03-06 21:56:42 +02:00
Aliaksandr Valialkin
5bd4325135
app/vmauth: simplify configuration for src_query_args 
Use the shorter form:

src_query_args:
- arg1=value1
- arg2=value2

instead of

src_query_args:
- name: arg1
  value: value2
- name: arg2
  value: value2

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878
 2024-03-06 21:24:53 +02:00
Aliaksandr Valialkin
1df7e5b69d
deployment/docker: update Go builder from Go1.21.7 to Go1.22.1 
See https://github.com/golang/go/issues?q=milestone%3AGo1.22.1+label%3ACherryPickApproved
 2024-03-06 21:04:44 +02:00
Aliaksandr Valialkin
0929f255f0
docs/vmauth.md: mention that request query args can used for routing decisions 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878
 2024-03-06 20:58:10 +02:00
Aliaksandr Valialkin
87d59cdfa5
app/vmauth: add ability to route requests based on HTTP query args via src_query_args option 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5878
 2024-03-06 20:52:42 +02:00
Aliaksandr Valialkin
2711732944
app/vmselect/prometheus: do not drop match[] filters if -search.ignoreExtraFiltersAtLabelsAPI flag is set 
The `match[]` filter is mandatory at /api/v1/series, so it mustn't be dropped here.

There is no sense in dropping `match[]` filter together with `extra_label` and `extra_filters[]`
at /api/v1/labels and /api/v1/label/.../values if -search.ignoreExtraFiltersAtLabelsAPI commnad-line flag is set,
since:
- the `match[]` filter triggers slow path at these APIs;
- the `extra_label` and `extra_filters[]` filters narrow down the number of matched time series,
  so they improve performance comparing to the case when only `match[]` filter is left,
  while `extra_label` and `extra_filters[]` filters are dropped.

This is a follow-up for 0b7a23a91d
 2024-03-06 13:57:20 +02:00
Daria Karavaieva
98b56476d7
redirect alias (#5934) 2024-03-06 13:54:48 +02:00
Yury Molodov
56e61bb54b
vmui: improve tracing styles (#5926) 
Improved trace display for better visual separation of branches:
* Increased left padding for each element
* Added padding for the last element in the branch
 2024-03-06 13:54:07 +02:00
Github Actions
cfce708eb9
Automatic update operator docs from VictoriaMetrics/operator@f028fdf (#5929) 2024-03-06 13:34:37 +02:00
hagen1778
a4cdfaea28
docs: clarify deduplication is needed in multi-retention setup 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-03-06 13:33:40 +02:00
hagen1778
a364de3cfe
lib/writeconcurrencylimiter: mention dependency on CPU cores for -maxConcurrentInserts flag 
The change also removes misleading `default` value from README for `maxConcurrentInserts`
cmd-line flag.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

(cherry picked from commit 73f5fb0f0c)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-03-05 18:56:38 +01:00
Aliaksandr Valialkin
27b9e8ed3e
app/{vmagent,vminsert}: add -streamAggr.dropInputSamples command-line flag for dropping the specified labels from input samples before deduplication and streaming aggregation 2024-03-05 02:27:27 +02:00
hagen1778
b612c0536e
docs: re-fresh vmctl docs 
* add recommendation about network bandiwdth between vmctl, source and destination
* use more relevant time series selector in examples - see https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5835

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-03-05 00:49:48 +02:00
Aliaksandr Valialkin
c38c45d71f
app/{vminsert,vmagent}: allow using -streamAggr.dedupInterval without -streamAggr.config 
This allows performing online de-duplication of incoming samples
 2024-03-05 00:47:23 +02:00
Github Actions
b530f6bf73
Automatic update operator docs from VictoriaMetrics/operator@c8ff654 (#5918) 2024-03-04 19:12:59 +02:00
Aliaksandr Valialkin
48a425898a
lib/streamaggr: enable time alignment for aggregate flushed to multiples of interval 
For example, if `interval: 1m`, then data flush occurs at the end of every minute,
while `interval: 1h` leads to data flush at the end of every hour.

Add `no_align_flush_to_interval` option, which can be used for disabling the alignment.
 2024-03-04 06:23:35 +02:00
Aliaksandr Valialkin
3ba9b2225e
docs/stream-aggregation.md: add troubleshooting section with solutions for common problems in streaming aggregation 2024-03-04 03:04:59 +02:00
Aliaksandr Valialkin
492c6c3ff5
docs/CHANGELOG.md: typo fix 2024-03-02 04:52:41 +02:00
Aliaksandr Valialkin
d0e6541f35
docs/stream-aggregation.md: typo fixes 2024-03-02 04:35:37 +02:00
Aliaksandr Valialkin
b912a45220
docs/stream-aggregation.md: remove superflouous output_relabel_configs from the config example for histogram aggregation 2024-03-02 03:36:08 +02:00
Aliaksandr Valialkin
0d5d46f9db
lib/streamaggr: huge pile of changes 
- Reduce memory usage by up to 5x when de-duplicating samples across big number of time series.
- Reduce memory usage by up to 5x when aggregating across big number of output time series.
- Add lib/promutils.LabelsCompressor, which is going to be used by other VictoriaMetrics components
  for reducing memory usage for marshaled []prompbmarshal.Label.
- Add `dedup_interval` option at aggregation config, which allows setting individual
  deduplication intervals per each aggregation.
- Add `keep_metric_names` option at aggregation config, which allows keeping the original
  metric names in the output samples.
- Add `unique_samples` output, which counts the number of unique sample values.
- Add `increase_prometheus` and `total_prometheus` outputs, which ignore the first sample
  per each newly encountered time series.
- Use 64-bit hashes instead of marshaled labels as map keys when calculating `count_series` output.
  This makes obsolete https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5579
- Expose various metrics, which may help debugging stream aggregation:
  - vm_streamaggr_dedup_state_size_bytes - the size of data structures responsible for deduplication
  - vm_streamaggr_dedup_state_items_count - the number of items in the deduplication data structures
  - vm_streamaggr_labels_compressor_size_bytes - the size of labels compressor data structures
  - vm_streamaggr_labels_compressor_items_count - the number of entries in the labels compressor
  - vm_streamaggr_flush_duration_seconds - a histogram, which shows the duration of stream aggregation flushes
  - vm_streamaggr_dedup_flush_duration_seconds - a histogram, which shows the duration of deduplication flushes
  - vm_streamaggr_flush_timeouts_total - counter for timed out stream aggregation flushes,
    which took longer than the configured interval
  - vm_streamaggr_dedup_flush_timeouts_total - counter for timed out deduplication flushes,
    which took longer than the configured dedup_interval
- Actualize docs/stream-aggregation.md

The memory usage reduction increases CPU usage during stream aggregation by up to 30%.

This commit is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5850
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5898
 2024-03-02 03:15:43 +02:00
hagen1778
8ce65e93d0
docs: mention docs link for VictoriaLogs docker env 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-03-01 21:34:15 +02:00
Aliaksandr Valialkin
7559b6eb97
docs: update -help output after recent changes in VictoriaMetrics components 2024-03-01 05:31:13 +02:00
Aliaksandr Valialkin
0dbcc07d04
docs/CHANGELOG.md: typo fix 2024-03-01 04:41:39 +02:00
Aliaksandr Valialkin
5f0dd8743c
docs: bump the latest VictoriaMetrics release from v1.98.0 to v1.99.0 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.99.0
 2024-03-01 04:24:11 +02:00
Aliaksandr Valialkin
be1446f720
docs/LTS-releases.md: update latest LTS release to v1.97.3 and v1.93.13 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.97.3
and https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.13
 2024-03-01 04:21:25 +02:00
Aliaksandr Valialkin
2540b8fdf8
deployment: update VictoriaLogs docker image from v0.4.2-victorialogs to v0.5.0-victorialogs 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v0.5.0-victorialogs
 2024-03-01 04:18:59 +02:00
Aliaksandr Valialkin
d66680fc47
docs/CHANGELOG.md: cut v1.99.0 release 2024-03-01 03:33:44 +02:00
Aliaksandr Valialkin
dc91c07384
docs/CHANGELOG.md: document v1.97.3 LTS release 2024-03-01 03:31:19 +02:00
Aliaksandr Valialkin
e4a1d6774e
docs: consistently use https://docs.victoriametrics.com/lts-releases/ link for LTS releases 2024-03-01 02:52:41 +02:00
Aliaksandr Valialkin
0fac8d6e87
docs/CHANGELOG.md: document v1.93.13 LTS release 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.13
 2024-03-01 02:41:05 +02:00
Aliaksandr Valialkin
bde2de3298
docs/VictoriaLogs/CHANGELOG.md: typo fix: bufix -> bugfix 2024-03-01 02:41:04 +02:00
Aliaksandr Valialkin
9a470239cf
docs/VictoriaLogs/CHANGELOG.md: cut v0.5.0-victorialogs 2024-03-01 01:43:35 +02:00
Aliaksandr Valialkin
d84cdf98da
app/{vmselect,vlselect}/vmui: run make vmui-update vmui-logs-update after e130f29659 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5862
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5152
 2024-03-01 00:51:49 +02:00
Yury Molodov
c71c2b37c5
vmui: add gap display option for charts #5152 (#5862) 2024-03-01 00:51:48 +02:00
Aliaksandr Valialkin
728fb59e0d
docs: update docs after 0b7a23a91d 2024-03-01 00:18:46 +02:00
Aliaksandr Valialkin
6aea1e5093
app/{vmagent,vminsert}: follow-up for 434a5803e7 
Document the /opentelemetry/v1/metrics endpoint instead of /opentelemetry/api/v1/push,
since the /v1/metrics suffix is hardcoded in OpenTelemetry protocol specification.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5871
 2024-02-29 18:05:12 +02:00
Nikolay
abb773120d
app/{vmagent,vminsert}: adds /v1/metrics suffix for opentelemetry route path (#5871) 
* app/{vmagent,vminsert}: adds /v1/metrics suffix for opentelemetry route path
it must fix compatibility with opentemetry-collector [spec](https://opentelemetry.io/docs/specs/otlp/\#otlphttp-request)
this suffix is hard-coded and cannot be changed with collector configuration

* Apply suggestions from code review

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-02-29 17:59:57 +02:00
Github Actions
5e20be845f
Automatic update operator docs from VictoriaMetrics/operator@7e8f811 (#5902) 2024-02-29 17:53:11 +02:00
Alexander Marshalov
b8c588a7e2
fixed broken link in docs to google pub sub (#5901) 2024-02-29 17:11:46 +02:00
Hui Wang
25e454be4c
metricsql: fix label_join() when dst_label is equal to one of the `… (#5886) 
* metricsql: fix label_join() when `dst_label` is equal to one of the `src_label`

* Update app/vmselect/promql/transform.go

* Update docs/CHANGELOG.md

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-02-29 16:02:48 +02:00
Aliaksandr Valialkin
cfeae6fffb
docs/VictoriaLogs/CHANGELOG.md: document the bugfix at a5795f533d 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5897
 2024-02-29 15:34:57 +02:00
Aliaksandr Valialkin
7310605c27
docs/CHANGELOG.md: clarify the description for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5859 
This is a follow-up for 3c74aa6b3d
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5888
 2024-02-29 15:28:52 +02:00
Aliaksandr Valialkin
0d2d1897c2
app/vmbackupmanager: follow-up after f2885a5c41dfaeecfecba0aceb8ae65b15e15d1c 
- Document the change at docs/CHANGELOG.md
- Give more clear name for maxHealthChecksRetries constant
- Improve logging when storage isn't reachable
- Use http.NewRequestWithContext() instead of setting up a request without context
  and then creating a new request with context vi req.WithContext() call.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5859
Updates https://github.com/VictoriaMetrics/VictoriaMetrics-enterprise/pull/735
 2024-02-29 15:17:57 +02:00
Hui Wang
99acc5bbba
caseStudies: add NetEaseCloudMusic post (#5891) 2024-02-29 14:58:15 +02:00
Artem Navoiev
3d9f947e42
docs: update statistic per tenant, add use cases section (#5892) 
* docs: update statistic per tenant, add use cases section

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* prettify

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

* Update docs/PerTenantStatistic.md

---------

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-02-29 14:55:55 +02:00
Aliaksandr Valialkin
38b42489c6
docs/anomaly-detection/guides: bump alertmanager version from v0.25.0 to v0.27.0 after 3723c809a1 2024-02-29 14:51:39 +02:00
Aliaksandr Valialkin
7832d0800e
app/{vminsert,vmagent}: follow-up after 67a55b89a4 
- Document the ability to read OpenTelemetry data from Amazon Firehose at docs/CHANGELOG.md

- Simplify parsing Firehose data. There is no need in trying to optimize the parsing with fastjson
  and byte slice tricks, since OpenTelemetry protocol is really slooow because of over-engineering.
  It is better to write clear code for better maintanability in the future.

- Move Firehose parser from /lib/protoparser/firehose to lib/protoparser/opentelemetry/firehose,
  since it is used only by opentelemetry parser.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5893
 2024-02-29 14:47:20 +02:00
Github Actions
33a65d6988
Automatic update operator docs from VictoriaMetrics/operator@9f1c910 (#5895) 2024-02-29 13:56:31 +02:00
Aliaksandr Valialkin
4e5cbc5dc6
deployment/docker: downgrade Go builder from 1.22.0 to 1.21.7 
Go1.22.0 contains the bug https://github.com/golang/go/issues/65705 ,
which prevents vmagent from normal operation.
 2024-02-29 13:52:58 +02:00
Aliaksandr Valialkin
cb9b46be54
app/vmselect/prometheus: add -search.ignoreExtraFiltersAtLabelsAPI command-line flag for ignoring extra_filters and extra_label args at /api/v1/labels, /api/v1/label/.../values and /api/v1/series 2024-02-29 13:00:24 +02:00
lzfhust
2821993c9c
add xiaohongshu in Case studies (#5885) 2024-02-29 02:40:00 +02:00
Aliaksandr Valialkin
7d619deacc
docs/Release-Guide.md: https://github.com/VictoriaMetrics/victoriametrics-lts-rpm should contain the latest release 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5811
 2024-02-27 13:14:25 +02:00
Roman Khavronenko
baa32adc47
deployment: create a separate env for VictoriaLogs (#5857) 
* deployment: create a separate env for VictoriaLogs

The new environment consists of the following components:
* VictoriaLogs
* fluentbit for collecting logs and sending to VictoriaLogs
* VictoriaMetrics for scraping and storing metrics from fluentbit and VictoriaLogs
* Grafana with VictoriaLogs datasource for monitoring

-----------------

The motivation for creating a separate environment is to simplify existing environments
and make it easier to update or modify them in future.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-02-27 13:13:42 +02:00
Aliaksandr Valialkin
dc5b1e4dc1
lib/httpserver: return back the default value for -http.connTimeout to 2 minutes 
It has been appeared that there are VictoriaMetrics users, who rely on the fact that
VictoriaMetrics components were closing incoming connections to -httpListenAddr every 2 minutes
by default. So let's return back this value by default in order to fix the breaking change
made at d8c1db7953 .

See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1304#issuecomment-1961891450 .
 2024-02-24 00:20:11 +02:00
Aliaksandr Valialkin
2f0b2b942b
docs/Troubleshooting.md: add Too much disk space used chapter` 2024-02-24 00:19:51 +02:00
hagen1778
0ac2df6e66
docs: mention missing change 521f9ffb430edf6aea7720a966b5b079cf15e34e 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-02-24 00:19:13 +02:00
Aliaksandr Valialkin
83217b7473
app/vmselect: add -search.maxLabelsAPIDuration and -search.maxLabelsAPISeries options for fine-tuning CPU and RAM usage for /api/v1/series , /api/v1/labels and /api/v1/label/.../values 
This commit returns back limits for these endpoints, which have been removed at 5d66ee88bd ,
since it has been appeared that missing limits result in high CPU usage, while the introduced concurrency limiter
results in failed lightweight requests to these endpoints because of timeout when heavyweight requests are executed.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5055
 2024-02-23 02:56:58 +02:00
Yury Molodov
140aaafad0
vmui: add a time picker to the "Logs Explorer" page (#5808) 
* vmui: add a time picker to the "Logs Explorer" page #5673

* Update app/vmui/packages/vmui/src/pages/ExploreLogs/hooks/useFetchLogs.ts

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-02-23 01:38:54 +02:00
Yury Molodov
b2b9f6e900
vmui: fix display Popper.tsx (#5842) 
* vmui: fix display Popper.tsx

* vmui/docs: fix display Popper.tsx

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-02-23 01:33:12 +02:00
Aliaksandr Valialkin
50ea081d75
docs/Single-server-VictoriaMetrics.md: sync with docs/README.md after 79b57f625c 2024-02-23 01:28:42 +02:00
Aliaksandr Valialkin
fbae1533fc
docs/CHANGELOG.md: document d68bb658ce 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5833
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5834
 2024-02-23 01:26:02 +02:00
hagen1778
225fd781a5
docs: re-classify change of default http timeout to bugfix 
The change introduced in d8c1db7953 (diff-2bfab3db5cc1baf4c6d3ff6b19901926e3bdf4411ec685dac973e5fcff1c723b)
was backported to v1.97.2. Therefore, it is a `bugfix` and should be explicitly
mentioned in the changelog of v1.97.2

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-02-23 01:17:40 +02:00
Nikolay
22762d7a69
app/vmselect: change export/csv timestamp format for rfc3339 to respect milliseconds (#5853) 
* app/vmselect: adds milliseconds to the csv export response for rfc3339
* milliseconds is a standard prescion for VictoriaMetrics query request responses
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5837

* app/victoria-metrics: adds tests for csv export/import
follow-up after 3541a8d0cf96dd4f8563624c4aab6816615d0756

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
 2024-02-23 01:16:08 +02:00
Fred Navruzov
c9d8627676
- v1.11 doc updates (#5852) 
- fix dead links
 2024-02-23 01:11:05 +02:00
Dan Dascalescu
6b5e8e7089
docs: CSV RFC3339 format uses server timezone (#5839) 2024-02-23 01:07:55 +02:00
Aliaksandr Valialkin
a982ab6bfb
app/vmstorage: expose vm_snapshots metric, which shows the current number of snapshots 
While at it, refresh docs about snapshots - https://docs.victoriametrics.com/#how-to-work-with-snapshots
 2024-02-23 01:07:04 +02:00
Aliaksandr Valialkin
843f3ec94e
docs/MetricsQL.md: improve text formatting for better readability 2024-02-23 01:05:49 +02:00
Aliaksandr Valialkin
477fdc21aa
app/vmselect/promql: add count_values_over_time() MetricsQL function 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5847
 2024-02-23 01:05:31 +02:00
Alexander Marshalov
8322425364
[lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5814) 
* [lib/promutils, lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5801)

* fixed tests

* fixed test

* Revert "fixed test"

This reverts commit 8a29764806.

* Revert "fixed tests"

This reverts commit 9ce13d1042.

* Revert "[lib/promutils, lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5801)"

This reverts commit a7a04bd4

* [lib/httputils] fixed floating-point error when parsing time in RFC3339 format (#5801)

---------

Co-authored-by: Nikolay <nik@victoriametrics.com>
 2024-02-23 00:58:26 +02:00
Artem Navoiev
b46014c8ab
docs: change header from h1 to h2 1.97.2. The markdown requires the proper structure in hierachy of title so h1 can not be a child of h1,h2... only be a separate item, in our structure title is the parent h1 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-02-23 00:42:15 +02:00
Github Actions
e2bbab072a
Automatic update operator docs from VictoriaMetrics/operator@d88157b (#5845) 2024-02-23 00:41:10 +02:00
Github Actions
f08f33cd5a
Automatic update operator docs from VictoriaMetrics/operator@c393852 (#5844) 2024-02-23 00:40:38 +02:00
Github Actions
786679135b
Automatic update operator docs from VictoriaMetrics/operator@4791fd1 (#5843) 2024-02-23 00:40:04 +02:00
Roman Khavronenko
2e172b9361
vmctl : Provide TLS config options for Open TSDB datasource #5797 (#5832) 
Originally implemented here https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5797

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: khushijain21 <khushij393@gmail.com>
(cherry picked from commit bb1279bfc4)
 2024-02-20 16:27:52 +01:00
Daria Karavaieva
550b589790
Vmanomaly Quickstart Fix absolute links (#5831) 
* links fix

* typo fix

(cherry picked from commit 4034d081f4)
 2024-02-20 16:27:52 +01:00
Daria Karavaieva
a68518c0e0
Vmanomaly QuickStart (#5800) 
* first edit

* typo 1

* typo 2

* fixes 3

* fixes 4

* fixes 5

* fixes, cross links

* v1.10 config

* models why, self-monitoring fix

* config next steps

* fixes

* minor fix

(cherry picked from commit b2baf7d472)
 2024-02-20 16:27:52 +01:00
hagen1778
c065287757
docs: move recent changes to Tip 
These changes were mistakenly put to existing release

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit dc25c30fdc)
 2024-02-20 16:27:51 +01:00
hagen1778
4474c23aed
app/vmalert: consistently sort groups by name and filename on /groups page 
This should prevent non-deterministic sorting for groups with identical names.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit e2dad3a2ac)
 2024-02-20 13:51:31 +01:00
hagen1778
6c63fd831d
app/vmalert: follow-up after b60dcbe11f 
* support case-insensitive search
* reflect search condition in URL, so link can be sharable
* support filtering on /alerts page
* fix collapseAll/expandAll logic to respect only shown entries
* add changelog

b60dcbe11f
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 11b03d9fc8)
 2024-02-20 13:35:02 +01:00
Artem Navoiev
e0569a355b
docs: mention slack inviter and slack channel (#5817) 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 5b652bccad)
 2024-02-20 13:35:01 +01:00
Aliaksandr Valialkin
1de59610c8
docs/VictoriaLogs/CHANGELOG.md: document cafd6f08b3 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5400
 2024-02-18 23:18:12 +02:00
Aliaksandr Valialkin
c4b16e3749
docs/VictoriaLogs/CHANGELOG.md: document 333bda8702 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5447
 2024-02-18 23:13:30 +02:00
Aliaksandr Valialkin
b58c429044
app/vlselect: follow-up for 451d2abf50 
- Consistently return the first `limit` log entries if the total size of found log entries doesn't exceed 1Mb.
  See app/vlselect/logsql/sort_writer.go . Previously random log entries could be returned with each request.
- Document the change at docs/VictoriaLogs/CHANGELOG.md
- Document the `limit` query arg at docs/VictoriaLogs/querying/README.md
- Make the change less intrusive.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5674
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5778
 2024-02-18 23:06:08 +02:00
Dmytro Kozlov
2d674f98d4
Enable the limit query param for the /select/logsql/query (#5778) 
* app/vlselect: add limit for logs query

* app/vlselect: CHANGELOG.md

* app/vlselect: stop search process if limit is reached, update logic, remove default limit

* app/vlselect: fix tests

* app/vlselect: fix filter tests

* app/vlselect: fix tests
 2024-02-18 22:59:16 +02:00
Aliaksandr Valialkin
82e38e1627
lib/promscrape: add support for enable_compression option in the same way as Prometheus does 
Updates https://github.com/prometheus/prometheus/pull/13166
Updates https://github.com/prometheus/prometheus/issues/12319

Do not document enable_compression option at docs/sd_configs.md, since vmagent already supports
more clear disable_compression option - see https://docs.victoriametrics.com/vmagent/#scrape_config-enhancements
 2024-02-18 19:42:09 +02:00
Aliaksandr Valialkin
f0db7d474f
lib/promscrape/discovery/kuma: add support for client_id option 
See https://github.com/prometheus/prometheus/pull/13278
 2024-02-18 19:19:55 +02:00
Aliaksandr Valialkin
690f6c833c
vendor: update github.com/VictoriaMetrics/metricsql from v0.72.1 to v0.73.0 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5383
 2024-02-18 18:43:41 +02:00
Aliaksandr Valialkin
c1a3876e95
docs/MetricsQL.md: properly document how MetricsQL selects the lookbehind window 
- rate(m) isn't equivalent to rate(m[1i]) when step is smaller than the interval between samples.
- default_rollup(m) isn't equivalent to default_rollup(m[1i]) when step is smaller than the interval between samples.

These changes have been made in v1.85.3 as a part of https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3483 .
See the corresponding commit - 9fa3f1dc57 .

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5816
 2024-02-18 13:48:45 +02:00
Aliaksandr Valialkin
8053ff9135
docs/LTS-releases.md: cosmetic fixes 2024-02-17 18:09:43 +02:00
Aliaksandr Valialkin
55bba932d4
docs/CHANGELOG.md: document f8207e33a2 2024-02-17 17:55:01 +02:00
Aliaksandr Valialkin
cf96c3416d
docs/CHANGELOG.md: add missing for in the description of the TLS configuration features for vmctl 
This is a follow-up for 6a07cb1bdb and f973711e56
 2024-02-17 17:55:01 +02:00
Aliaksandr Valialkin
efbead081c
docs/Single-server-VictoriaMetrics.md: enumerate all the VictoriaMetrics components 2024-02-17 17:54:34 +02:00
Aliaksandr Valialkin
9f7c9e9033
docs/LTS-releases.md: add a dedicated page describing LTS lines of releases for VictoriaMetrics 2024-02-17 17:53:56 +02:00
hagen1778
e53f53aaf5
app/vmctl: follow-up after 0c293a66ec 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit f973711e56)
 2024-02-16 15:31:58 +01:00
Khushi Jain
9ce7f21a63
app/vmctl : support TLS config options for remote read mode (#5798) 
(cherry picked from commit 0c293a66ec)
 2024-02-16 15:31:58 +01:00
hagen1778
025e52adad
app/vmctl: follow-up after 7cd1b7d047 
* cleanup code
* update docs

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 6a07cb1bdb)
 2024-02-16 15:31:58 +01:00
Khushi Jain
02c8b5015c
app/vmctl : support TLS config options for InfluxDB datasource (#5783) 
* vmctl: TLS flags for influx DB

* added httputils function

* Add changelog and doc

---------

Co-authored-by: Khushi Jain <khushi.jain@nokia.com>
(cherry picked from commit 7cd1b7d047)
 2024-02-16 15:31:57 +01:00
Fred Navruzov
d8de87aeb0
docs: vmanomaly - updates of v1.10.0 and model type section (#5813) 
* - apply v1.10 changes
- chapter on model types (uni/multivariate and rolling)

* - update self-monitoring labels description
- fix typos

* fix duplicated text and link rendering

(cherry picked from commit 172e196ac9)
 2024-02-16 15:31:56 +01:00
hagen1778
5e4732cc2d
docs: update formatting for usage examples 
- Use `sh` format for examples
- Reduce length of progress bars

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 3170ad3f44)
 2024-02-16 15:31:56 +01:00
Aliaksandr Valialkin
33b2553c78
app/vmstorage: expose vm_last_partition_parts metrics, which may help identifying performance issues related to the increased number of parts in the last partition 2024-02-15 14:52:53 +02:00
Aliaksandr Valialkin
6fd9331b7f
docs/CHANGELOG.md: document v1.93.12 LTS release 2024-02-14 20:17:52 +02:00
Aliaksandr Valialkin
c617379789
docs/CHANGELOG.md: document v1.97.2 LTS release 2024-02-14 18:50:59 +02:00
Aliaksandr Valialkin
77a514f2cc
all: update Docker image tag for VictoriaMetrics components from v1.97.1 to v1.98.0 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.98.0
 2024-02-14 17:14:55 +02:00
Aliaksandr Valialkin
06da06dac0
lib/promrelabel: store the original labels before returning them them to promutils.PutLabels() 
This should reduce memory allocations.

This is a follow-up for b09bd6c42a

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5389
 2024-02-14 16:09:38 +02:00
Aliaksandr Valialkin
acc37ff28a
docs/CHANGELOG.md: cut v1.98.0 release 2024-02-14 16:00:59 +02:00
Aliaksandr Valialkin
9df42f000d
docs/CHANGELOG.md: various typo cleanups 2024-02-14 02:45:25 +02:00
Yury Molodov
b08a23c4a5
vmui: improve the context for autocomplete #5736 #5737 #5739 (#5804) 
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-02-14 02:36:06 +02:00
Aliaksandr Valialkin
5f2905d120
app/vmselect: add sum_eq_over_time, sum_gt_over_time and sum_le_over_time functions to MetricsQL 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4641
 2024-02-13 23:40:30 +02:00
Nikolay
0a2cc0e873
app/vmauth: properly release memory during config reload (#5805) 
* app/vmauth: properly release memory during config reload
previously metrics package hold a refrence for channels for users concurrent requests.
it case of churn at `name`  field of users configuration, new metric was created. But previous one wasn't deleted.
It prevented full parsed configuration from being garbace collected.

now all config related metrics are bound to corresponding metrics.Set and unregistered during config reload process.

It also must fix an issue with incorrect values for current concurrent user requests

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4690

* wip

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-02-13 20:49:57 +02:00
Aliaksandr Valialkin
89aceb2b2f
docs: add link to https://docs.victoriametrics.com/scrape_config_examples/ to docs about configuring target scraping at vmagent and single-node VictoriaMetrics 2024-02-13 20:48:04 +02:00
Aliaksandr Valialkin
2c44ea89fd
docs/vmbackupmanager.md: mention -license command-line flag instead of deprecated -eula 2024-02-13 20:41:33 +02:00
Aliaksandr Valialkin
3f74cadba4
docs/vmauth.md: add Config reload chapter, which explains how to reload -auth.config at vmauth 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1194
 2024-02-13 20:36:59 +02:00
Aliaksandr Valialkin
f6cb2021c3
docs/enterprise.md: remove the mention of deprecated -eula command-line flag 2024-02-13 18:47:07 +02:00
Aliaksandr Valialkin
dcacf209d9
vendor: update github.com/VictoriaMetrics/metricsql from v0.70.1 to v0.71.0 
This adds an ability to propagate label filters across label_set() and alias() functions.

This should help https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1827#issuecomment-1654095358
 2024-02-13 06:37:09 +02:00
Aliaksandr Valialkin
67091537ae
app/vmagent/remotewrite: add -remoteWrite.tlsHandshakeTimeout command-line flag for tuning tls handshake timeout to -remoteWrite.url 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1699
 2024-02-13 02:46:24 +02:00
Aliaksandr Valialkin
84e439c469
docs/vmauth.md: add missing dot 2024-02-13 01:09:11 +02:00
Aliaksandr Valialkin
6bc70a883d
app/vmauth: add support for mTLS-based routing of incoming requests to different backends depending on the subject field in the TLS certificate provided by the user 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1547
 2024-02-13 01:04:19 +02:00
Aliaksandr Valialkin
ebcbca0e00
docs/keyConcepts.md: do not duplicate the list of supported data ingestion protocols - just refer to the original list at https://docs.victoriametrics.com/#how-to-import-time-series-data 
This should simplify keeping docs in sync
 2024-02-12 22:55:27 +02:00
Aliaksandr Valialkin
8793817fbe
docs/vmbackup.md: remove the unneeded -storageDataPath command-line from the example for making server-side copy of the backup 2024-02-12 22:24:41 +02:00
Aliaksandr Valialkin
f5680a6857
all: upgrade Go builder from Go1.21.7 to Go1.22.0 
See https://go.dev/doc/go1.22
 2024-02-12 22:14:00 +02:00
Artem Navoiev
5815ea4160
docs vmbackupmanager update flags 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-02-12 20:16:00 +02:00
Aliaksandr Valialkin
12cf2d8b44
docs/enterprise.md: remove duplicate Enterprise word in the same sentence 2024-02-12 20:13:30 +02:00
Aliaksandr Valialkin
0f46a49912
docs/Single-server-VictoriaMetrics.md: substitute duplicate cases studies list with the link to the original list 2024-02-12 20:13:01 +02:00
Aliaksandr Valialkin
8ea4b8a3b4
docs: remove misleading either from the description of or label filters 2024-02-12 20:12:27 +02:00
Github Actions
c952c678b2
Automatic update operator docs from VictoriaMetrics/operator@e261c37 (#5788) 2024-02-12 20:11:07 +02:00
Roman Khavronenko
433c3726b2
app/vmalert: support filtering for /api/v1/rule like Prometheus does (#5787) 
Follow-up after 62e5e2a4c8

Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit 8850c7431d)
 2024-02-09 14:36:15 +01:00
Victor Amorim dos Santos
56b1d8e9ed
app/vmalert: support type param for filtering /api/v1/rules response by rule type (#5749) 
Co-authored-by: Hui Wang <haley@victoriametrics.com>
(cherry picked from commit 62e5e2a4c8)
 2024-02-09 14:36:14 +01:00
Aliaksandr Valialkin
46de5d7eb1
docs: update docs after ae8a867924 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1470
 2024-02-09 04:20:09 +02:00
Aliaksandr Valialkin
cf64597878
all: add support for specifying multiple -httpListenAddr options 2024-02-09 03:22:49 +02:00
Aliaksandr Valialkin
5bbf95f9b4
docs/CHANGELOG.md: typo fixes 2024-02-08 21:18:52 +02:00
Aliaksandr Valialkin
ae7da12280
lib/httpserver: do not close client connections every 2 minutes by default 
Closing client connections every 2 minutes doesn't help load balancing -
this just leads to "jumpy" connections between multiple backend servers,
e.g. the load isn't spread evenly among backend servers, and instead jumps
between the servers every 2 minutes.

It is still possible periodically closing client connections by specifying non-zero -http.connTimeout command-line flag.

This should help with https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1304#issuecomment-1636997037

This is a follow-up for d387da142e
 2024-02-08 21:10:54 +02:00
Aliaksandr Valialkin
9216d00a6c
docs/{vmagent,vmalert}: mention that /-/reload endpoint may be protected with -reloadAuthKey command-line flag 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1172
 2024-02-08 18:49:21 +02:00
Aliaksandr Valialkin
0d313dda8a
docs/Cluster-VictoriaMetrics.md: document that /api/v1/query?series_lector[d] returns raw samples 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1148
 2024-02-08 18:32:47 +02:00
Aliaksandr Valialkin
fc8328a87f
docs: sync with enterprise branch 2024-02-08 17:25:26 +02:00
Aliaksandr Valialkin
428109a427
docs: update -help output after 83e55456e2 2024-02-08 17:22:42 +02:00
Artem Navoiev
fbe3f51fee
docs: change ndjson links to https://jsonlines.org/ as original one was hacked (#5782) 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-02-08 17:22:31 +02:00
hagen1778
2aa86313bc
docs: sync flags description for vmbackup and vmbackupmanager 
Sync description after changes in 83e55456e2

Remove extra text in flags description for vmbackupmanager as it breaks layout
when rendered at docs.victoriametrics.com

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-02-08 17:12:53 +02:00
Aliaksandr Valialkin
44052f4fd5
docs/CHANGELOG.md: add a link to docs describing -disableReroutingOnUnavailable command-line flag 2024-02-08 17:05:27 +02:00
hagen1778
a2063e0e7a
docs: follow-up after 83e55456e2 
83e55456e2
Signed-off-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit e1926f286b)
 2024-02-08 15:58:34 +01:00
Khushi Jain
a076cb4a93
app/vmbackup: support client-side TLS configuration for create/delete snapshot API (#5738) 
(cherry picked from commit 83e55456e2)
 2024-02-08 15:58:34 +01:00
Aliaksandr Valialkin
d0b55e359a
docs/vmagent.md: add debugging scrape targets chapter 2024-02-08 16:32:54 +02:00
Aliaksandr Valialkin
582d431df2
app/victoria-metrics: properly send staleness markers on victoriametrics shutdown if -selfScrapeInterval > 0 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/943
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1526
 2024-02-08 15:30:40 +02:00
Aliaksandr Valialkin
202d8e2c40
docs: update -help output after 61d9df4c36 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/834
 2024-02-08 14:50:56 +02:00
Aliaksandr Valialkin
b18e608016
app/vmselect: add ability to reset rollup result cache on startup by passing -search.resetRollupResultCacheOnStartup command-line flag 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/834
 2024-02-08 14:42:15 +02:00
Aliaksandr Valialkin
84a3f4719d
docs/CHANGELOG.md: clarify the bugfix description for e1bf8440eb 2024-02-08 13:01:26 +02:00
Aliaksandr Valialkin
ae2a9c8195
lib/mergeset: prevent from possible too big indexBlockSize panic 
This panic could occur when samples with too long label values are ingested into VictoriaMetrics.
This could result in too long fistItem and commonPrefix values at blockHeader (up to 64kb each).
This may inflate the maximum index block size by 4 * maxIndexBlockSize.
 2024-02-08 12:55:58 +02:00
hagen1778
9b173c2f01
dashboards: follow-up 4369bc1df2 
* add more details to changelog
* simplify panels description
* remove capacity planning recommendation, as it proves it incompetent

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-02-08 12:55:42 +02:00
Hui Wang
0cd0ddc1c1
deployment/dashboards: fix Storage full ETA panels (#5747) 
During background downsampling, rate(vm_deduplicated_samples_total{type="merge"}) could be much bigger than 
rate(vm_rows_added_to_storage_total) and it could last quite some time,
 which causes negative values of Storage full ETA and confuses users, see playground.

Instead of trying to get more accurate results during downsampling, I think it's ok to ignore 
vm_deduplicated_samples_total at all, it's more reasonable to see Storage full ETA increase after downsampling.

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-02-08 12:54:31 +02:00
Aliaksandr Valialkin
def5573f92
app/vmselect/promql: properly handle precision errors in rollup functions 
changes(), increases_over_time() and resets() shouldn't take into account
value changes, which may occur because of precision errors.

The maximum guaranteed precision for raw samples stored in VictoriaMetrics is 12 decimal digits.
So do not count relative changes for values if they are smaller than 1e-12 comparing to the value.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/767
 2024-02-08 02:33:17 +02:00
Aliaksandr Valialkin
f4252cd9aa
docs/CHANGELOG.md: properly document the change at b74006e2ca 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5774
 2024-02-07 22:06:37 +02:00
Nihal
95287d269b
[vmsingle/vminsert]: change http success response code to 200 for -/reload request handler (#5776) 
* change vmsingle's response code to 200 for reload request handler

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

* change vmsingle's response code to 200 for the reload request handler

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

* change vmsingle's response code to 200 for the reload request handler. See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5774

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

---------

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>
 2024-02-07 22:06:24 +02:00
Github Actions
3c069607b3
Automatic update operator docs from VictoriaMetrics/operator@38dfad5 (#5772) 2024-02-07 21:55:11 +02:00
Aliaksandr Valialkin
5d9e0ab71e
docs/CHANGELOG.md: support empty command-line flag values in short array notation 
For example, -fooDuration=',10s,' is now supported - it sets three command-line flag values:

- the first and the last one are set to the default value for `-fooDuration`
- the second one is set to 10s
 2024-02-07 20:55:01 +02:00
Aliaksandr Valialkin
949481fe48
docs/relabeling.md: add examples on how to rename scraped metrics and how to add/change labels in scraped metrics 2024-02-07 20:29:04 +02:00
Aliaksandr Valialkin
caf706fcc0
all: update Go builder from Go1.21.6 to Go1.21.7 
See https://github.com/golang/go/issues?q=milestone%3AGo1.21.7+label%3ACherryPickApproved
 2024-02-07 04:01:05 +02:00
Aliaksandr Valialkin
18e1dfd7a8
docs/scrape_config_examples.md: add an example for scraping cadvisor in Kubernetes 2024-02-07 03:23:01 +02:00
Aliaksandr Valialkin
898414856f
docs/Cluster-VictoriaMetrics.md: document the /datadog/api/beta/sketches endpoint 
This is a follow-up for a1d1ccd6f2

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5584
 2024-02-07 01:36:08 +02:00
Aliaksandr Valialkin
82f4e4e070
app/{vmagent,vminsert}: follow-up after a1d1ccd6f2 
- Document the change at docs/CHANGELOG.md
- Copy changes from docs/Single-server-VictoriaMetrics.md to README.md
- Add missing handler for processing multitenant requests ( https://docs.victoriametrics.com/vmagent/#multitenancy )
- Substitute github.com/stretchr/testify dependency with 3 lines of code in the added tests
- Comment unclear code at lib/protoparser/datadogsketches/parser.go , so @AndrewChubatiuk could update it
  and add permalinks to the original source code there.
- Various code cleanups

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5584
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3091
 2024-02-07 01:31:52 +02:00
Daria Karavaieva
0ff3e6f3bc
Vmanomaly Guide - dashboard and query change (#5771) 
* dashboard fix

* query fix

* changed screenshots

* minor fixes
 2024-02-07 01:31:15 +02:00
hagen1778
eb90f2170a
docs/vmalert: mention step param when comapring rule results to raw queries 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-02-07 01:30:32 +02:00
Andrii Chubatiuk
c634859c4f
support datadog /api/beta/sketches API (#5584) 
Co-authored-by: Andrew Chubatiuk <andrew.chubatiuk@motional.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-02-07 01:30:00 +02:00
Yury Molodov
5778acf9eb
vmui: improve select component functionality (#5755) 
* vmui: fix select closing on click outside (#5728)

* vmui: clear entered text in select after selecting a value (#5727)

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-02-06 22:50:28 +02:00
Yury Molodov
0b5f5d456c
vmui: fix handling invalid timezone (#5758) 
* vmui: fix handling invalid timezone (#5732)

* vmui: switch browser timezone flag to isValid

---------

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-02-06 22:48:00 +02:00
Yury Molodov
0cf17068b8
vmui: fix graph dragging (#5769) 
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-02-06 22:32:43 +02:00
Aliaksandr Valialkin
ebd0f1b72e
vendor: update github.com/VictoriaMetrics/metricsql from v0.70.0 to v0.70.1 
This should help with the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5604
 2024-02-06 21:52:55 +02:00
Aliaksandr Valialkin
dc340a83a9
docs/scrape_config_examples.md: document that full urls can be used as in the targets list 2024-02-06 19:49:23 +02:00
Aliaksandr Valialkin
2fec67632f
docs/scrape_config_examples.md: add missing to specify phrase 2024-02-06 19:43:30 +02:00
Aliaksandr Valialkin
0504771db1
docs/CHANGELOG.md: add link to mTLS feature request 2024-02-06 19:32:51 +02:00
Aliaksandr Valialkin
bace92fab6
lib/httpserver: add support for mTLS for requests to -httpListenAddr 2024-02-06 17:47:27 +02:00
Aliaksandr Valialkin
79990cc89b
docs/CHANGELOG.md: move descriptions for recently added features from v1.97.1 to tip 2024-02-06 16:31:44 +02:00
Aliaksandr Valialkin
22524353ed
docs/scrape_config_examples.md: add examples for typical scrape_config usage 2024-02-06 16:01:34 +02:00
Aliaksandr Valialkin
01312003be
docs/Cluster-VictoriaMetrics.md: add a warning that -disableReroutingOnUnavailable command-line flag may result in long pause for data ingestion when some of vmstorage nodes are unavailable for long time 2024-02-06 16:01:07 +02:00
Aliaksandr Valialkin
bffcb40537
docs/enterprise.md: prioritize enterprise technical support over other enterprise features 2024-02-06 16:00:21 +02:00
hagen1778
8bc18b5a4c
docs/vmalert: update limit description 
Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-02-06 16:00:02 +02:00
hagen1778
702cc4514a
docs/vmalert: mention limit option in group params 
This param was supported for long time but was missing in the docs.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-02-06 15:59:28 +02:00
Fred Navruzov
eaf75a565f
update changelog with 1.0.0 helm chart ref (#5763) 2024-02-06 15:58:49 +02:00
Aliaksandr Valialkin
2bad79a97e
docs/sd_configs.md: improve readability a bit 2024-02-05 15:33:23 +02:00
Github Actions
c0b2ccb6ca
Automatic update operator docs from VictoriaMetrics/operator@bfc521d (#5761) 
Co-authored-by: Hui Wang <haley@victoriametrics.com>
 2024-02-05 15:33:22 +02:00
Aliaksandr Valialkin
63a43331a3
docs/Cluster-VictoriaMetrics.md: document -disableReroutingOnUnavailable command-line flag 
This is a follow-up for 88f0d1572e
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5713
 2024-02-05 15:17:09 +02:00
Aliaksandr Valialkin
67d166f181
docs/Cluster-VictoriaMetrics.md: move the Improve re-routing performance during restart to more appropriate place 
Previoulsy it was mistakenly inserted between `No downtime strategy` and 'Minimum downtime strategy' chapters.

This is a follow-up for 37997abd14
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5293
 2024-02-05 14:44:06 +02:00
Aliaksandr Valialkin
e2c58a9db1
docs/Cluster-VictoriaMetrics.md: mention -metrics.exposeMetadata command-line flag in Monitoring section 2024-02-05 11:48:05 +02:00
Aliaksandr Valialkin
103f527037
docs/CHANGELOG.md: add a link to all VictoriaMetrics dashboards for Grafana 2024-02-05 11:45:39 +02:00
Aliaksandr Valialkin
6af029fd9b
docs/Troubleshooting.md: clarify the section about GOGC tuning 
This is a follow-up for 487a94565b
 2024-02-05 11:45:39 +02:00
Aliaksandr Valialkin
b105a5a1f5
docs/CHANGELOG.md: add missing links to the corresponding dashboards 2024-02-05 11:45:06 +02:00
Aliaksandr Valialkin
19f9203c7a
docs: mention -metrics.exposeMetadata command-line flag in Monitoring sections 
This is a follow-up for 326a77c697
 2024-02-05 11:44:39 +02:00
Artem Navoiev
e3138a356c
docs: fix aliasis for url-example 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-02-05 11:43:58 +02:00
Fred Navruzov
be41ffb925
docs: vmanomaly - fix 1.9.2 references in pull cmd and docs (#5756) 
* fix 1.9.2 references in pull cmd and docs
* better readability of upgrade note
 2024-02-05 11:43:41 +02:00
hagen1778
bdbab7bed5
dashboards/all: add new panel CPU spent on GC 
It should help identifying cases when too much CPU is spent on garbage collection,
 and advice users on how this can be addressed.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-02-05 11:42:28 +02:00
hagen1778
2206309439
dashboards: add Targets scraped/s 
A new stat panel shows the number of targets scraped by the vmagent per-second.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-02-05 10:51:35 +02:00
Aliaksandr Valialkin
9f23231926
docs/CHANGELOG.md: document v1.93.11 LTS release 2024-02-01 18:21:35 +02:00
Aliaksandr Valialkin
d63bc81f29
docs/CHANGELOG.md: document v1.87.14 LTS release 2024-02-01 17:09:03 +02:00
Aliaksandr Valialkin
24437b6d70
all: update VictoriaMetrics Docker image from v1.97.0 to v1.97.1 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.97.1
 2024-02-01 16:15:14 +02:00
Aliaksandr Valialkin
dab3da9ee8
docs: mark v1.97.x as long-term support release 2024-02-01 15:16:52 +02:00
Aliaksandr Valialkin
e1ceb64432
docs/CHANGELOG.md: cut v1.97.1 2024-02-01 15:08:53 +02:00
Aliaksandr Valialkin
eff4dc3685
docs: sync -help output after recent changes 2024-02-01 15:06:19 +02:00
Aliaksandr Valialkin
d8f508cc21
docs/Single-server-VictoriaMetrics.md: clarify Security chapter a bit 2024-02-01 14:45:10 +02:00
Aliaksandr Valialkin
ee327cfd5d
docs/Single-server-VictoriaMetrics.md: run make docs-sync after 49d5e7fef5 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5262
 2024-02-01 14:42:07 +02:00
Aliaksandr Valialkin
6c56f49f9c
lib/prompbmarshal: return back custom protobuf marshaler for lib/prompbmarshal.WriteRequest 
The easyproto-based marshaler is 2x slower than the previous custom marshaler,
so let's stick with it. This improves the performance for sending data to remote storage at vmagent
and reduces CPU usage to pre-v1.97.0 levels.
 2024-02-01 06:34:46 +02:00
Aliaksandr Valialkin
cc626ae3b5
lib/promauth: follow-up for fca3b14b7b 
- Simplify the code for handling BasicAuthConfig at lib/promauth/config.go
- Move the description of the change into correct place at docs/CHANGELOG.md
- Put tests for username in front of tests for password at lib/promauth/config_test.go

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5720
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5511
 2024-01-31 19:47:53 +02:00
Nihal
bcd094ac8b
Support for username_file in scrape config (basic_auth) similar to Prometheus for having config compatibility (#5720) 
* adding support for username_file in basic_auth of scrape config

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

* adding support for username_file in basic_auth of scrape config. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5511

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

* adding support for username_file in basic_auth of scrape config

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

* adding support for username_file in basic_auth of scrape config

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

* adding support for username_file in basic_auth of scrape config

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>

---------

Signed-off-by: Syed Nihal <syed.nihal@nokia.com>
 2024-01-31 19:47:50 +02:00
Aliaksandr Valialkin
4f5cb17042
app/vmselect/netstorage: properly handle the case when an empty brsPool points to the end of brs.brs 
This case is possible after a new brsPool is allocated. The fix is to verify whether len(brsPool) >= len(brs.brs)
before trying to append a new item to brsPool and sharing its contents with brs.brs.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5733
 2024-01-31 10:31:51 +02:00
hagen1778
151247c9b9
dashboards/single: fix typo in query for version annotation 
The typo falsely produced many version change events.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
 2024-01-31 10:28:24 +02:00
Aliaksandr Valialkin
2033fe4caf
app/vmselect/promql: really keep metric names when keep_metric_names modifier is applied to binary operator 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5556
 2024-01-31 02:33:06 +02:00
Aliaksandr Valialkin
a0043e214f
docs/vmauth.md: typo fix after 68be182075 2024-01-31 00:13:56 +02:00
Aliaksandr Valialkin
f1effc1133
all: update VictoriaMetrics Docker image from v1.96.0 to v1.97.0 
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.97.0
 2024-01-30 23:37:42 +02:00
Aliaksandr Valialkin
6d766abde7
deployment/docker: upgrade base Docker image from Alpine 3.19.0 to 3.19.1 
See https://www.alpinelinux.org/posts/Alpine-3.19.1-released.html
 2024-01-30 22:47:29 +02:00
Aliaksandr Valialkin
966f1ad07f
docs/CHANGELOG.md: cut v1.97.0 2024-01-30 22:45:10 +02:00
Dan Dascalescu
d2c38e2588
docs: t...over_time functions return fractional seconds (#5715) 
* docs: t...over_time functions return fractional seconds

* Apply suggestions from code review

---------

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-01-30 22:20:16 +02:00
Roman Khavronenko
02e609b141
app/vmselect: set proper timestamp for cached instant responses (#5723) 
* app/vmselect: set proper timestamp for cached instant responses

The change updates `getSumInstantValues` to prefer timestamp
from the most recent results. Before, timestamp from cached series
was used.

The old behavior had negative impact on recording rules as they
were getting responses with shifted timestamps in past.
Subsequent recording or alerting rules fetching results of these
recording rules could get no result due to staleness interval.

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5659
Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
 2024-01-30 22:20:16 +02:00
Yury Molodov
587c5fec2d
vmui: add the ability to expand all tracing entries (#5677) (#5726) 2024-01-30 21:13:13 +02:00
Github Actions
317a2551ee
Automatic update Grafana datasource docs from VictoriaMetrics/grafana-datasource@40e4e15 (#5729) 2024-01-30 21:07:50 +02:00
Aliaksandr Valialkin
1a071c5228
docs/CHANGELOG.md: document the enhancement, which reduces initial memory usage when vmagent scrapes targets with large responses 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5567
 2024-01-30 20:51:29 +02:00
Aliaksandr Valialkin
dc9f4f0907
docs/CHANGELOG.md: refer to the related pull request for the bugfix for https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1945 2024-01-30 20:21:51 +02:00
Aliaksandr Valialkin
16787415b3
docs/CHANGELOG.md: document the bugfix addressed by the commit bc7cf4950b 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1945
 2024-01-30 20:16:32 +02:00
Artem Navoiev
17a4d310d0
docs: add alias for keyconcepts 
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
 2024-01-30 18:39:59 +02:00
Aliaksandr Valialkin
09c388a8e4
lib/promscrape: use the standard net/http.Client instead of fasthttp.Client for scraping targets in non-streaming mode 
While fasthttp.Client uses less CPU and RAM when scraping targets with small responses (up to 10K metrics),
it doesn't work well when scraping targets with big responses such as kube-state-metrics.
In this case it could use big amounts of additional memory comparing to net/http.Client,
since fasthttp.Client reads the full response in memory and then tries re-using the large buffer
for further scrapes.

Additionally, fasthttp.Client-based scraping had various issues with proxying, redirects
and scrape timeouts like the following ones:

- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1945
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5425
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2794
- https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1017

This should help reducing memory usage for the case when target returns big response
and this response is scraped by fasthttp.Client at first before switching to stream parsing mode
for subsequent scrapes. Now the switch to stream parsing mode is performed on the first scrape
after reading the response body in memory and noticing that its size exceeds the value passed
to -promscrape.minResponseSizeForStreamParse command-line flag.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5567

Overrides https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4931
 2024-01-30 18:39:55 +02:00