Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-15 08:23:34 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
cd2247b24a
VictoriaMetrics/lib/storage
History
Roman Khavronenko cd2247b24a
app/vmselect: limit the number of parallel workers by 32 (#5195) 
* app/vmselect: limit the number of parallel workers by 32

The change should improve performance and memory usage during query processing
on machines with big number of CPU cores. The number of parallel workers for
query processing is controlled via `-search.maxWorkersPerQuery` command-line flag.
By default, the number of workers is limited by the number of available CPU cores,
but not more than 32. The limit can be increased via `-search.maxWorkersPerQuery`.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

* wip

- The `-search.maxWorkersPerQuery` command-line flag doesn't limit resource usage,
  so move it from the `resource usage limits` to `troubleshooting` chapter at docs/Single-server-VictoriaMetrics.md

- Make more clear the description for the `-search.maxWorkersPerQuery` command-line flag

- Add the description of `-search.maxWorkersPerQuery` to docs/Cluster-VictoriaMetrics.md

- Limit the maximum value, which can be passed to `-search.maxWorkersPerQuery`, to GOMAXPROCS,
  because bigger values may worsen query performance and increase CPU usage

- Improve the the description of the change at docs/CHANGELOG.md. Mark it as FEATURE instead of BUGFIX,
  since it is closer to a feature than to a bugfix.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5087

---------

Signed-off-by: hagen1778 <roman@victoriametrics.com>
Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2023-10-26 09:15:27 +02:00
..
block_header_test.go
block_header.go lib/storage: verify that timestamps in block are in the range specified by blockHeader.{Min,Max}Timestamp when upacking the block 2022-09-06 13:07:49 +03:00
block_stream_merger.go Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:25:49 +02:00
block_stream_reader_test.go Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:25:49 +02:00
block_stream_reader_timing_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
block_stream_reader.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
block_stream_writer_timing_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
block_stream_writer.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
block_test.go lib/storage: use deterministic random generator in tests 2023-01-23 20:12:32 -08:00
block.go lib/storage: validate timestamps in the block only if they use encoding, which needs validation 2022-10-21 00:54:37 +03:00
dedup_test.go lib/storage: follow-up after 7c0ae3a86a 2022-12-08 18:18:49 -08:00
dedup_timing_test.go lib/storage: follow-up after 7c0ae3a86a 2022-12-08 18:18:49 -08:00
dedup.go app,lib: fix typos in comments (#3804) 2023-02-13 09:32:35 -08:00
filenames.go lib/storage: consistently use OS-independent separator in file paths 2023-03-25 14:34:36 -07:00
index_db_test.go lib/storage: log fatal error inside searchMetricName() instead of propagating it to the caller 2023-09-22 11:37:55 +02:00
index_db_timing_test.go Add flagutil.Duration to avoid conversion bugs (#4835) 2023-09-01 09:30:30 +02:00
index_db.go lib/storage: log fatal error inside searchMetricName() instead of propagating it to the caller 2023-09-22 11:37:55 +02:00
inmemory_part_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
inmemory_part_timing_test.go lib/storage: use deterministic random generator in tests 2023-01-23 20:12:32 -08:00
inmemory_part.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
merge_test.go Makefile: update golangci-lint from v1.51.2 to v1.54.2 2023-09-01 10:25:49 +02:00
merge_timing_test.go lib/storage: properly free up resources from newTestStorage() by calling stopTestStorage() 2023-07-13 17:13:34 -07:00
merge.go lib/storage: optimization: do not scan block for rows outside retention if it is covered by the retention 2022-12-03 22:14:20 -08:00
metaindex_row_test.go
metaindex_row.go all: subsitute ioutil.ReadAll with io.ReadAll 2022-08-22 00:16:04 +03:00
metric_name_test.go
metric_name.go app/vmselect/promql: add the ability to copy all the labels from one side of group_left()/group_right() operation 2023-07-17 16:58:30 -07:00
part_header.go lib/storage: fix a bug, which prevents from reading pre-v1.90.0 parts 2023-04-14 22:33:29 -07:00
part_search_test.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
part_search_timing_test.go lib/storage: skip missing tsids in the current block header by using binary search 2022-12-14 22:07:55 -08:00
part_search.go lib/storage: optimize partSearch.searchBHS() for common case when the TSID for the current block header is bigger or equal to the current tsid 2022-12-19 10:31:39 -08:00
part.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
partition_search_test.go lib/storage: properly free up resources from newTestStorage() by calling stopTestStorage() 2023-07-13 17:13:34 -07:00
partition_search.go lib/storage: fixes finalDedup for backfilled data (#3737) 2023-02-01 09:57:02 -08:00
partition_test.go lib/storage: stop exposing vm_merge_need_free_disk_space metric 2023-09-25 17:00:14 +02:00
partition.go lib/{mergeset,storage}: consistently reset isInMerge field in parts passed to mergeParts() before returning from the function 2023-10-02 20:34:52 +02:00
raw_block.go
raw_row.go lib/{storage,mergeset}: convert InitFromFilePart to MustInitFromFilePart 2023-04-14 15:47:20 -07:00
search_test.go lib/storage: replace OpenStorage() with MustOpenStorage() 2023-04-14 23:04:42 -07:00
search.go lib/storage: log fatal error inside searchMetricName() instead of propagating it to the caller 2023-09-22 11:37:55 +02:00
storage_test.go app/vmselect: limit the number of parallel workers by 32 (#5195) 2023-10-26 09:15:27 +02:00
storage_timing_test.go lib/storage: replace OpenStorage() with MustOpenStorage() 2023-04-14 23:04:42 -07:00
storage.go lib/storage: follow-up after 188cfe3a85 2023-10-26 09:04:49 +02:00
table_search_test.go lib/storage: properly free up resources from newTestStorage() by calling stopTestStorage() 2023-07-13 17:13:34 -07:00
table_search_timing_test.go lib/storage: properly free up resources from newTestStorage() by calling stopTestStorage() 2023-07-13 17:13:34 -07:00
table_search.go lib/storage: do not pass retentionMsecs and isReadOnly args explicitly - access them via Storage arg 2022-10-24 01:32:56 +03:00
table_test.go flagutil: Make .Msecs private (#4906) 2023-09-03 10:37:57 +02:00
table_timing_test.go lib/storage: properly free up resources from newTestStorage() by calling stopTestStorage() 2023-07-13 17:13:34 -07:00
table.go lib/storage: do not create flock.lock files at partition directories, since it is created at the Storage level 2023-07-06 17:26:37 -07:00
tag_filters_test.go lib/regexutil: add Simplify() function for simplifying the regular expression 2022-08-26 11:57:43 +03:00
tag_filters_timing_test.go lib/regexutil: add Simplify() function for simplifying the regular expression 2022-08-26 11:57:43 +03:00
tag_filters.go lib/storage: optimize matching speed for non-trivial regexp filters 2022-10-01 12:07:18 +03:00
time_test.go
time.go app,lib: fix typos in comments (#3804) 2023-02-13 09:32:35 -08:00
tsid_test.go
tsid.go