Commit Graph

4341 Commits

Author SHA1 Message Date
Andrii Chubatiuk
9a051fc80f
removed unneeded ref shortcodes, updated VM changelog to use relative markdown links (#6691)
### Describe Your Changes

Use relative markdown references, removed `{{< ref >}}` shortcodes

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-25 13:20:05 +02:00
Andrii Chubatiuk
6b97044d8a
view documentation locally (#6677)
- moved files from root to VictoriaMetrics folder to be able to mount
operator docs and VictoriaMetrics docs independently
- added ability to run website locally

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-25 12:27:05 +02:00
Andrii Chubatiuk
4a01709b90
added _index.md for hugo, which point to README.md (#6686)
### Describe Your Changes

Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.

### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-25 12:20:40 +02:00
Hui Wang
e0c62e5c50
security: upgrade base docker image (Alpine) from 3.20.1 to 3.20.2 (#6684)
See https://www.alpinelinux.org/posts/Alpine-3.20.1-released.html

>including security fix for:
OpenSSL CVE-2024-5535
2024-07-25 11:02:23 +02:00
Zakhar Bessarab
9f5eb25150
app/vmauth: change response code when all backend are not available (#6676)
### Describe Your Changes

Change response code to 502 to align it with behaviour of other existing
reverse proxies. Currently, the following reverse proxies will return
502 in case an upstream is not available: nginx, traefik, caddy, apache.


Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-07-25 10:24:05 +02:00
Zhu Jiekun
bafdc9dd78
docs: [vmagent] Add docs for remote write protocol control in Kafka integration (#6682)
### Describe Your Changes

Improve documentation to help:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6670

Currently, the documentation of Kafka Integration did not describe:
- How to switch between different remote write protocols in producer
side and consumer side.
2024-07-25 10:21:37 +02:00
Zakhar Bessarab
53c08cb8d7
docs/vmbackup: add a note for azure identity (#6675)
### Describe Your Changes

Add a note to clarify usage of azure credentials when multiple
credentials are available.

The user is required to specify AZURE_CLIENT_ID as otherwise Azure API
will return an error: "Multiple user assigned identities exist, please
specify the clientId / resourceId of the identity in the token request"

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-07-25 10:20:50 +02:00
Artem Navoiev
591c42fdbe
docs: fix broken links
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-25 10:19:59 +02:00
Aliaksandr Valialkin
b65f32350e
docs/CHANGELOG.md: document keep_original_host option at vmauth
This option has been added in the commit add2db12b2
2024-07-20 11:48:58 +02:00
Aliaksandr Valialkin
28af963940
docs/vmauth.md: document the case with default url_prefix additionally to url_map 2024-07-20 09:46:31 +02:00
Aliaksandr Valialkin
a50a29500f
app/vmauth: properly proxy requests to backend paths ending with /
Previously the traling / was incorrectly removed when proxying requests from http://vmauth/

While at it, add more tests for requestHandler()
2024-07-19 17:29:17 +02:00
Aliaksandr Valialkin
4e3acfbe9a
app/vmauth: properly proxy HTTP requests without body
The Request.Body for requests without body can be nil. This could break readTrackingBody.Read() logic,
which could incorrectly return "cannot read data after closing the reader" error in this case.
Fix this by initializing the readTrackingBody.r with zeroReader.

While at it, properly set Host header if it is specified in 'headers' section.
It must be set net/http.Request.Host instead of net/http.Request.Header.Set(),
since the net/http.Client overwrites the Host header with the value from req.Host
before sending the request.

While at it, add tests for requestHandler(). Additional tests for various requestHandler() cases
will be added in future commits.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6445
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5707
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5240
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6525
2024-07-19 16:26:07 +02:00
Aliaksandr Valialkin
3dd239fb87
docs/goals.md: clarify that complicated CI/CD tasks make debugging harder 2024-07-19 08:51:23 +02:00
Aliaksandr Valialkin
5436911abb
docs/vmauth.md: use /some_path/.* instead of /some_path/.+ in examples, so it matches /some_path/
The /some_path/.+ regexp matches /some_path/ followed by at least a single char.
This is unexpected by most users, since they expect it should match /some_path/.
Substitute .+ with .*, so this regexp matches /some_path/ .
2024-07-19 08:51:22 +02:00
Aliaksandr Valialkin
7eb24d98e1
docs/Single-server-VictoriaMetrics.md: recommend periodic running of make spellcheck command
This is a follow-up for fabf0b928e
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6562
2024-07-18 11:52:34 +02:00
Aliaksandr Valialkin
12393c64f7
docs/goals.md: make clear that complication of CI/CD pipeline is bad
CI/CD pipeline is usually hard to debug. So it is good to keep it as simple as possible.
2024-07-18 11:52:24 +02:00
Andrii Chubatiuk
13b9a2cfc1
Code tooltip updates (#6657)
### Describe Your Changes

recently added custom `promtextmetric` and `influxtextmetric` prismjs
plugins to vmdocs to convert this
<img width="1081" alt="Screenshot 2024-07-17 at 12 13 29"
src="https://github.com/user-attachments/assets/66d4ea18-10fe-45ef-80b4-989d0eb3bd92">
to this
<img width="1087" alt="Screenshot 2024-07-17 at 12 24 34"
src="https://github.com/user-attachments/assets/60d4ab44-79e5-4c63-b966-54b989ead1aa">




### Checklist

The following checks are **mandatory**:

- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-18 11:52:24 +02:00
Aliaksandr Valialkin
3cda945138
docs/Release-Guide.md: shorten release guide for Ansible roles and RPM packages
There is no need in enumerating all the files, which must be updated, since these files
may be moved to other locations. It is enough to mention that versions for all the VictoriaMetrics
components must be updated.
2024-07-17 23:36:30 +02:00
Aliaksandr Valialkin
47f62f2e75
docs/vmbackup.md: an attempt to fix formatting issue at https://docs.victoriametrics.com/vmbackup/#providing-credentials-via-env-variables 2024-07-17 23:01:32 +02:00
Aliaksandr Valialkin
009f71fc4c
docs/stream-aggregation.md: an attempt to fix yaml formatting at https://docs.victoriametrics.com/stream-aggregation/#stream-aggregation-config 2024-07-17 22:43:54 +02:00
Aliaksandr Valialkin
c57c169dc4
deployment: update VictoriaMetrics Docker image from v1.101.0 to v1.102.0
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.102.0
2024-07-17 22:35:00 +02:00
Aliaksandr Valialkin
1ebd4e8e43
docs/CHANGELOG.md: cut v1.102.0 release 2024-07-17 20:50:01 +02:00
Aliaksandr Valialkin
dc6565c105
docs/CHANGELOG.md: consistently use new url format for the MetricsQL docs
Use https://docs.victoriametrics.com/metricsql/ instead of https://docs.victoriametrics.com/MetricsQL.html .
This removes unnecessary redirect from https://docs.victoriametrics.com/MetricsQL.html to https://docs.victoriametrics.com/metricsql/

This is a follow-up for 6a4bd5049b
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6547
2024-07-17 20:32:48 +02:00
Aliaksandr Valialkin
6dc2bcff2b
docs/CHANGELOG.md: document v1.97.6 LTS release
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.97.6
2024-07-17 20:28:04 +02:00
Aliaksandr Valialkin
299c9af6de
docs/CHANGELOG.md: document v1.93.16 LTS release
See https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.93.16
2024-07-17 19:48:15 +02:00
Aliaksandr Valialkin
65437f23fc
docs/CHANGELOG.md: order the changes at tip, so they are easier to read 2024-07-17 18:56:06 +02:00
Fred Navruzov
231d950764
docs/vmanomaly - v1.13.3 patch notes (#6659)
### Describe Your Changes

Add patch note v1.13.3 to CHANGELOG doc page for `vmanomaly`

### Checklist

The following checks are **mandatory**:

- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
2024-07-17 18:56:06 +02:00
Aliaksandr Valialkin
116b02eb29
docs/VictoriaLogs/querying/README.md: add link to Grafana datasource for VictoriaLogs to the top of the page 2024-07-17 18:34:47 +02:00
Aliaksandr Valialkin
f32e27e815
docs/Single-server-VictoriaMetrics.md: explain why images must have the same filepath as the corresponding docs at docs/ folder
This is a follow-up for f2c1d30011
2024-07-17 18:26:46 +02:00
Aliaksandr Valialkin
326c9bc1aa
docs: fix spellcheck errors found by make spellcheck
Thanks to the spellcheck Makefile rule added by @arkid15r at fabf0b928e
2024-07-17 18:21:17 +02:00
Aliaksandr Valialkin
9b529c2742
lib/backup/azremote: follow-up for 5fd3aef549
- Mention that credentials can be configured via env variables at both vmbackup and vmrestore docs.

- Make clear that the AZURE_STORAGE_DOMAIN env var is optional at https://docs.victoriametrics.com/vmbackup/#providing-credentials-via-env-variables

- Use string literals as is for env variable names instead of indirecting them via string constants.
  This makes easier to read and understand the code. These environment variable names aren't going to change
  in the future, so there is no sense in hiding them under string constants with some other names.

- Refer to https://docs.victoriametrics.com/vmbackup/#providing-credentials-via-env-variables in error messages
  when auth creds are improperly configured. This should simplify figuring out how to fix the error.

- Simplify the code a bit at FS.newClient(), so it is easier to follow it now.
  While at it, remove the check when superflouos environment variables are set, since it is too fragile
  and it looks like it doesn't help properly configuring vmbackup / vmrestore.

- Remove envLookuper indirection - just use 'func(name string) (string, bool)' type inline.
  This simplifies code reading and understanding.

- Split TestFSInit() into TestFSInit_Failure() and TestFSInit_Success(). This simplifies the test code,
  so it should be easier to maintain in the future.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6518
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5984
2024-07-17 17:55:39 +02:00
Aliaksandr Valialkin
f7789b61e7
lib/protoparser/graphite: follow-up for 476faf5578
- Clarify the description of -graphite.sanitizeMetricName command-line flag at README.md
- Do not sanitize tag values - only metric names and tag names must be sanitized,
  since they are treated specially by Grafana. Grafana doesn't apply any restrictions on tag values.
- Properly replace more than two consecutive dots with a single dot.
- Disallow unicode letters in metric names and tag names, since neither Prometheus nor Grafana
  do not support them.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6489
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6077
2024-07-17 12:57:56 +02:00
rtm0
1b03d7e6de
Fix inconsistent error handling in Storage.AddRows() (#6583)
`Storage.AddRows()` returns an error only in one case: when
`Storage.updatePerDateData()` fails to unmarshal a `metricNameRaw`. But
the same error is treated as a warning when it happens inside
`Storage.add()` or returned by `Storage.prefillNextIndexDB()`.

This commit fixes this inconsistency by treating the error returned by
`Storage.updatePerDateData()` as a warning as well. As a result
`Storage.add()` does not need a return value anymore and so doesn't
`Storage.AddRows()`.

Additionally, this commit adds a unit test that checks all cases that
result in a row not being added to the storage.

---------

Signed-off-by: Artem Fetishev <wwctrsrx@gmail.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
2024-07-17 12:55:07 +02:00
Aliaksandr Valialkin
e1f6926d2a
docs/CONTRIBUTING.md: mention that docs/CHANGELOG.md shouldn't contain technical details for the changes
The purpose of docs/CHANGELOG.md is to provide VictoriaMetrics users clear and concise information
on what's changed at VictoriaMetrics components. Technical details of the change are unclear
to most of VictoriaMetrics users, who are not familiar with VictoriaMetrics source code.
These details complicate reading the docs/CHANGELOG.md by ordinary users, so do not clutter
the changelog with technical details. If the user wants technical details, he can click
the link to the related GitHub issue and/or pull request and dive into all the details he wants.
2024-07-17 11:51:08 +02:00
Aliaksandr Valialkin
31b8e9054d
app/vmauth: pool readTrackingBody structs in order to reduce pressure on Go GC
- use pool for readTrackingBody structs in order to reduce pressure on Go GC
- allow re-reading partially read request body
- add missing tests for various cases of readTrackingBody usage

This is a follow-up for ad6af95183 and 4d66e042e3.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6445
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6446
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6533
2024-07-17 11:34:32 +02:00
Aliaksandr Valialkin
111f7da946
Revert "app/vmauth: reader pool to reduce gc & mem alloc (#6533)"
This reverts commit 4d66e042e3.

Reasons for revert:

- The commit makes unrelated invalid changes to docs/CHANGELOG.md
- The changes at app/vmauth/main.go are too complex. It is better splitting them into two parts:
  - pooling readTrackingBody struct for reducing pressure on GC
  - avoiding to use readTrackingBody when -maxRequestBodySizeToRetry command-line flag is set to 0

Let's make this in the follow-up commits!

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6445
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6533
2024-07-17 11:34:31 +02:00
Aliaksandr Valialkin
2acbd69a56
docs: remove the remaining mentioning about snap packages after the commit 0a42c8fd8b
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6543
2024-07-16 18:38:28 +02:00
Zakhar Bessarab
096abd827f
app/vmagent/kafka: fix non-unique metric naming (#774)
* app/vmagent/kafka: fix non-unique metric naming

Fix panic when using multiple topics with the same name.

See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6636
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/changelog: document bugfix

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/vmagent: more examples for Kafka ingestion with multiple brokers/topics groups

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2024-07-16 18:06:33 +02:00
Aliaksandr Valialkin
617a7b4db6
lib/promscrape/discovery/yandexcloud: follow-up for 070abe5c71
- Obtain IAM token via GCE-like API instead of Amazon EC2 IMDSv2 API,
  since it looks like IMDBSv2 API isn't supported by Yandex Cloud
  according to https://yandex.cloud/en/docs/security/standard/authentication#aws-token :

  > So far, Yandex Cloud does not support version 2, so it is strongly recommended
  > to technically disable getting a service account token via the Amazon EC2 metadata service.

- Try obtaining IAM token via GCE-like API at first and then fall back to the deprecated Amazon EC2 IMDBSv1.
  This should prevent from auth errors for instances with disabled GCE-like auth API.
  This addresses @ITD27M01 concern at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5513#issuecomment-1867794884

- Make more clear the description of the change at docs/CHANGELOG.md , add reference to the related issue.

P.S. This change wasn't tested in prod because I have no access to Yandex Cloud.
It is recommended to test this change by @ITD27M01 and @vmazgo , who filed
the issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5513

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6524
2024-07-16 18:06:33 +02:00
Alexander Marshalov
4881496d1a
docs: fixed image paths in the guide "Anomaly Detection and Alerting Setup" (#6651) 2024-07-16 18:06:32 +02:00
f41gh7
146f69d6f8
docs: mention vmagent and vmgateway changes
Signed-off-by: f41gh7 <nik@victoriametrics.com>
2024-07-16 18:02:50 +02:00
Aliaksandr Valialkin
0f56ab8774
docs/CHANGELOG.md: clarify docs and changelog after e666d64f1d
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6453
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6525
2024-07-16 14:02:07 +02:00
Aliaksandr Valialkin
6d237da3f3
lib/promscrape: follow-up for 1e83598be3
- Clarify that the -promscrape.maxScrapeSize value is used for limiting the maximum
  scrape size if max_scrape_size option isn't set at https://docs.victoriametrics.com/sd_configs/#scrape_configs

- Fix query example for scrape_response_size_bytes metric at https://docs.victoriametrics.com/vmagent/#automatically-generated-metrics

- Mention about max_scrape_size option at the -help description for -promscrape.maxScrapeSize command-line flag

- Treat zero value for max_scrape_size option as 'no scrape size limit'

- Change float64 to int type for scrapeResponseSize struct fields and function args, since response size cannot be fractional

- Optimize isAutoMetric() function a bit

- Sort auto metrics in alphabetical order in isAutoMetric() and in scrapeWork.addAutoMetrics() functions
  for better maintainability in the future

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6434
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6429
2024-07-16 12:38:41 +02:00
Github Actions
081bf0cd3e
Automatic update Grafana datasource docs from VictoriaMetrics/victorialogs-datasource@3cbd5e9 (#6649) 2024-07-16 12:38:40 +02:00
Aliaksandr Valialkin
6a1ea788b9
Revert "docs: [vmagent] Add CHANGELOG for Statsd support in v1.102.0-rc1 (#6494)"
This reverts commit b37b288dce.

Reason for revert: statsd protocol support has been reverted - see 2da7dfc754
2024-07-16 10:59:18 +02:00
Aliaksandr Valialkin
8b1c38abde
app/vmauth: follow-up for 3a45bbb4e0
- Move the test for SRV discovery into a separate function. This allows verifying round-robin discovery across SRV records.
- Restore the original netutil.Resolver after the test finishes, so it doesn't interfere with other tests.
- Move the description of the bugfix into the correct place at docs/CHANGELOG.md - it should be placed under v1.102.0-rc2
  instead of v1.102.0-rc1.
- Remove unneeded code in URLPrefix.sanitizeAndInitialize(), since it is expected this function is called only once
  for finishing URLPrefix initializiation. In this case URLPrefix.nextDiscoveryDeadline and URLPrefix.n are equal to 0
  according to https://pkg.go.dev/sync/atomic#Uint64
- Properly fix the bug at URLPrefix.discoverBackendAddrsIfNeeded() - it is expected that hostToAddrs map uses
  the original hostname keys, including 'srv+' prefix, so it shouldn't be removed when looping over up.busOriginal.
  Instead, the 'srv+' prefix must be removed from the hostname only locally before passing the hostname to netutil.Resolver.LookupSRV.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6401
2024-07-16 10:41:08 +02:00
Artem Navoiev
36955ff066
docs/data-ingestion use img tag instead of markdown image. this allow docs that sync automation to correctly render images
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2024-07-16 10:41:06 +02:00
Aliaksandr Valialkin
468c04d3c2
app/vmauth: clarify the description for -idleConnTimeout command-line flag
This is a follow-up for d44058bcd6
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6388
2024-07-16 09:40:01 +02:00
Aliaksandr Valialkin
8b76a40715
lib/httpserver: skip basic auth check for additional request paths, which should call httpserver.CheckAuthFlag()
This is a follow-up for 61dce6f2a1

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6338
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6329
2024-07-16 01:08:41 +02:00
Aliaksandr Valialkin
aa52d6cd9b
app/vminsert: increase default value for -maxLabelValueLen command-line flag from 1KiB to 4KiB
It has been appeared that the standard Kubernetes monitoring can generate labels with sizes up to 4KiB

This is a follow-up for a5d1013042
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6176
2024-07-15 23:32:54 +02:00