Aliaksandr Valialkin
7ffa66d249
lib/{mergeset,storage}: take into account byte slice capacity when returning the size of in-memory part
...
This results in more correct reporting of memory usage for in-memory parts
2022-12-03 22:31:34 -08:00
Aliaksandr Valialkin
db049fed84
lib/mergeset: cleanup after de6dd1cd5a
...
Remove unused getInmemoryPart and putInmemoryPart functions
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2249
2022-08-04 18:34:38 +03:00
Aliaksandr Valialkin
3f999b11f7
lib/mergeset: remove aux buffers from inmemoryPart
...
This should reduce the size of inmemoryPart items and may improve performance a bit during registering new time series
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2247
2022-03-03 17:12:25 +02:00
Aliaksandr Valialkin
ecf68da79e
lib/mergeset: eliminate copying of itemsData and lensData from storageBlock to inmemoryBlock
...
This should improve performance when registering new time series.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2247
2022-03-03 17:12:25 +02:00
Aliaksandr Valialkin
f4e466955d
lib/mergeset: move storageBlock from inmemoryPart to a sync.Pool
...
The lifetime of storageBlock is much shorter comparing to the lifetime of inmemoryPart,
so sync.Pool usage should reduce overall memory usage and improve performance
because of better locality of reference when marshaling inmemoryBlock to inmemoryPart.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2247
2022-03-03 17:12:25 +02:00
Aliaksandr Valialkin
b47f18f555
lib/{mergeset,storage}: tune compression levels for small blocks
...
This should reduce CPU usage spent on compression
2022-02-25 15:34:13 +02:00
Aliaksandr Valialkin
b805a675f3
lib/{mergeset,storage}: switch from sync.Pool to chan-based pool for inmemoryPart objects
...
This should reduce memory usage on systems with big number of CPU cores,
since every inmemoryPart object occupies at least 64KB of memory and sync.Pool maintains
a separate pool inmemoryPart objects per each CPU core.
Though the new scheme for the pool worsens per-cpu cache locality, this should be amortized
by big sizes of inmemoryPart objects.
2021-07-06 16:33:25 +03:00
Aliaksandr Valialkin
587132555f
lib/mergeset: reduce memory usage for inmemoryBlock by using more compact items representation
...
This also should reduce CPU time spent by GC, since inmemoryBlock.items don't have pointers now,
so GC doesn't need visiting them.
2021-02-21 22:09:10 +02:00
Aliaksandr Valialkin
41f512af1c
all: add vm_data_size_bytes metrics for easy monitoring of on-disk data size and on-disk inverted index size
2019-07-04 19:43:04 +03:00
Aliaksandr Valialkin
1836c415e6
all: open-sourcing single-node version
2019-05-23 00:18:06 +03:00
