The indexSearch.containsTimeRange() function is called for the current indexDB and the previous indexDB
every time when searching for metricIDs by label filters. This function consumes a lot of additional CPU time
for cases when queries with lightweight label filters are sent to VictoriaMetrics at high rate (e.g. thousands of RPS),
like in the issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7009 .
Optimize indexSearch.containsTimeRange() function in the following ways:
- Unconditionally return true if this function is called for the current indexDB, since there are very high
chances that the current indexDB contains the data with timestamps in the requested time range.
- Cache the minimum timestamp, which is missing in the indexed data for the previous indexDB.
This is safe to do, since the previous indexDB is readonly.
This optimization eliminates potentially slow lookup in the previous indexDB for typical
use cases when the requested time range is close to the current time.
Previously indexDB.doExtDB() was returning boolean value, which was indicating whether f callback was called.
There is no need in returning this boolean value, since the f callback can determine on itself whether it was called.
This simplifies the code a bit.
While at it, document indexDB.doExtDB().
Make function name and comments more clear.
d8f8822fa5
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Nikolay <nik@victoriametrics.com>
This fixes flaky test TestGetCommonTokensForOrFilters:
filter_or_test.go:143: unexpected tokens for field "_msg"; got ["foo" "bar"]; want ["bar" "foo"]
### Describe Your Changes
### Pull Request Description:
1. **HTML File Structure Optimization**: Adjusted the location of HTML
files for different builds to prevent redundant files in the final
output. See issue #6900
2. **Metadata Fixes**: Corrected metadata in HTML files for each build
configuration.
3. **Favicon Update**: Replaced PNG favicon (`14 KB` and `1.58 KB`) with
SVG (`1.35 KB`).
4. **Social Media Optimization**: Optimized the social preview image,
reducing its size by `60.2 KB`.
5. **Git Ignore Update**: Added `public/index.html` to `.gitignore` as
it is dynamically generated during the build process.
### Checklist
The following checks are **mandatory**:
- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
(cherry picked from commit 4e976f66f3)
### Describe Your Changes
Fix a typo; `filebeat.yml` -> `fluent-bit.conf`
### Checklist
The following checks are **mandatory**:
- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
Signed-off-by: Thomas Danielsson <thomas@elajt.se>
(cherry picked from commit 69b976f08e)
### Describe Your Changes
change the `/query_range` and `/query` requests method from `GET` to
`POST`. See #6288.
### Checklist
The following checks are **mandatory**:
- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
---------
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: hagen1778 <roman@victoriametrics.com>
(cherry picked from commit b0bdb92729)
…g from 1e3 to 5e3
This should improve visibility on errors produced by very long queries.
The change is classified as BUG in order to port it to LTS releases.
### Describe Your Changes
Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
---------
Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Mathias Palmersheim <mathias@victoriametrics.com>
(cherry picked from commit e115b85770)
### Describe Your Changes
Added missing `{` in vmalert rule.
### Checklist
The following checks are **mandatory**:
- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
(cherry picked from commit 75cc7922c3)
Improved VictoriaMetrics documentation for cloud
Related issue: https://github.com/VictoriaMetrics/cloud/issues/2143
### Describe Your Changes
Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.
### Checklist
The following checks are **mandatory**:
- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
(cherry picked from commit 4c228f1e18)
### Describe Your Changes
Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.
### Checklist
The following checks are **mandatory**:
- [ ] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
---------
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit 277fed9990)
* Previously, only metricID->metricName missing index records were
tracked with deadline But it was possible a case for missing
metricID->TSID index records. IndexDB metrics fix exposed misleading
metric for such missing records.
* This commit adds check for metricID->TSID missing index records. And
delete missing metricID entry if it hit 60 second deadline.
Related issue
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6931
Signed-off-by: f41gh7 <nik@victoriametrics.com>
Previously we manually imported system windows DDLs
and made direct syscall.
But golang exposes syscall wrappers with sys/windows package.
It seems, that direct syscall was broken at 1.23 golang release. It was
`GetDiskFreeSpace` syscall in our case.
This commit replaces all manual syscalls with wrappers
Related issue:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6973
Related golang issue:
https://github.com/golang/go/issues/69029
Signed-off-by: f41gh7 <nik@victoriametrics.com>
Also tried to make it catch "Authorisation" in the future, fixed a lot
of other misspells along the way, but didn't make it catch
"Authorisation" anyway.
- Fix misspelled "Authorization" header name
- Fix misspelled "organization"
- Fix more misspells
…pair
`alert_relabel_configs` in [notifier
config](https://docs.victoriametrics.com/vmalert/#notifier-configuration-file)
can drop alert labels when used to filter different tenant alert message
to different notifier.
alertmanager would report error like `msg="Failed to validate alerts"
err="at least one label pair required"` in this case, but the rest of
the alerts inside one request would still be valid in alertmanager, so
it's not severe.
(cherry picked from commit ae4d376e41)
### Describe Your Changes
Please provide a brief description of the changes you made. Be as
specific as possible to help others understand the purpose and impact of
your modifications.
### Checklist
The following checks are **mandatory**:
- [x] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit f9a8c09fe8)
Previously the query could return incorrect results, since the query timestamp was updated with every Query.Clone() call
during iterative search for the time range with up to limit=N rows.
While at it, optimize queries, which find low number of matching logs, while spend a lot of CPU time for searching
across big number of logs. The optimization reduces the upper bound of the time range to search if the current time range
contains zero matching rows.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785
Previously tokens from AND filters were extracted in random order. This could slow down
checking them agains bloom filters if the most specific tokens go at the beginning of the AND filters.
Preserve the original order of tokens when matching them against bloom filters,
so the user could control the performance of the query by putting the most specific AND filters
at the beginning of the query.
While at it, add tests for getCommonTokensForAndFilters() and getCommonTokensForOrFilters().
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6554
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6556
Previously the following query could miss rows matching !bar if these rows do not contain foo:
foo OR !bar
This is because of incorrect detection of common tokens for OR filters - all the unsupported filters
were skipped (including the NOT filter (aka `!`)), while in this case zero common tokens must be returned.
While at it, move repetiteve code in TestFilterAnd and TestFilterOr into f function.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6554
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/6556
