Aliaksandr Valialkin
c4194020ef
app/vmagent: do not allow sending unpacked requests with sizes exceeding -maxInsertRequestSize
2020-02-25 19:35:43 +02:00
Aliaksandr Valialkin
2471340e0d
app/vmagent: add ability to accept Influx line protocol data via TCP and UDP
...
Just set `-influxListenAddr` command-line flag
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/333
2020-02-25 19:18:01 +02:00
Aliaksandr Valialkin
7a045125cc
lib/fs: typo fix: read blocks bigger than 8KB via pread()
call instead of using mmap
2020-02-25 18:04:06 +02:00
Aliaksandr Valialkin
13ee8271d0
lib/envflag: substitute dots with underscores in env var names if -envflag.enable is set
...
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/311
2020-02-24 21:15:11 +02:00
Aliaksandr Valialkin
7ee7614e90
app/vmagent: initial implementation for vmagent
2020-02-23 17:31:54 +02:00
Aliaksandr Valialkin
110cce24d9
lib/storage: add vm_ prefix to deduplicated_samples_total
metric
2020-02-21 19:33:36 +02:00
Aliaksandr Valialkin
9d279e26a7
lib/protoparser/prometheus: skip leading whitespace from tag names
2020-02-16 19:06:23 +02:00
Aliaksandr Valialkin
a2b81b71b9
lib/storage: typo fix
2020-02-16 15:53:48 +02:00
Aliaksandr Valialkin
ad4cb9f3ca
lib/storage: prevent from clobbering nin-nil lastError in Storage.add
2020-02-16 15:51:35 +02:00
Aliaksandr Valialkin
846d7fa7e9
app/vmselect: add sort_by_label(q, label)
and sort_by_label_desc(q, label)
functions
...
This is implementation of https://github.com/prometheus/prometheus/pull/1533 for VictoriaMetrics.
2020-02-13 17:01:50 +02:00
Aliaksandr Valialkin
e3b18ca1ab
lib/mergeset: skip createing temporary part objects when merging source inmemory parts
...
This should reduce CPU usage when adding new entries to inverted index.
This should alos prevent from creating stalled cleaner goroutines for the created temporary parts,
since they were never closed.
This should fix the following issue: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/316 .
2020-02-13 14:09:13 +02:00
Aliaksandr Valialkin
347aaba79d
lib/{storage,mergeset}: use time.Ticker instead of time.Timer where appropriate
...
It has been appeared that time.Timer was used in places where time.Ticker must be used instead.
This could result in blocked goroutines as in the https://github.com/VictoriaMetrics/VictoriaMetrics/issues/316 .
2020-02-13 13:21:48 +02:00
Aliaksandr Valialkin
e7d1037210
docs: migrate ExtendedPromQL->MetricsQL in order to be more consistent
2020-02-10 23:03:31 +02:00
Aliaksandr Valialkin
fcdd95a6ef
lib/envflag: check for incorrect flag values read from environment vars
2020-02-10 16:09:03 +02:00
Aliaksandr Valialkin
9c5db9400c
lib/envflag: add -envflag.enable
command-line flag for enabling reading flags from environment vars
...
By default flags are read only from command line. They can be read from environment vars if `-envflag.enable` is set.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/311
2020-02-10 16:09:01 +02:00
Aliaksandr Valialkin
1010a57882
all: allow setting flags via environment vars
...
Now flags can be set via environment vars with the same names as flags.
Command-line flags override flags set via env vars.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/311
2020-02-10 13:31:21 +02:00
Aliaksandr Valialkin
ea66212c93
lib/storage: move -dedup.minScrapeInterval
flag outside lib/storage, so it doesnt show up in vminsert
in cluster version
2020-02-10 13:07:25 +02:00
Aliaksandr Valialkin
8b360a25e9
lib/logger: initialize output to os.Stderr by default
2020-02-04 22:43:26 +02:00
Aliaksandr Valialkin
1f271a9815
lib/logger: add -loggerOutput
command-line flag
...
This flag allows changing log output from `stderr` to `stdout` if `-loggerOutput=stdout` is set.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/306
2020-02-04 21:48:24 +02:00
Aliaksandr Valialkin
49ab3fa076
lib/logger: do not clutter -loggerFormat=json
output with stack trace
...
This should improve json parsing
2020-02-04 21:40:20 +02:00
Aliaksandr Valialkin
56d6b8ed0a
lib/storage: do not deduplicate blocks with less than 32 samples during merge
...
This should improve deduplication accuracy for blocks with higher number of samples.
2020-02-04 18:41:37 +02:00
Aliaksandr Valialkin
7cde594696
all: do not clash flag description with back-quoted flag types
...
See https://golang.org/pkg/flag/#PrintDefaults for more details.
2020-02-04 15:56:01 +02:00
Aliaksandr Valialkin
9b25a2fb67
lib/fs: remove unused readerAt
interface
2020-01-31 15:13:00 +02:00
Aliaksandr Valialkin
e3adc095bd
all: add -dedup.minScrapeInterval
command-line flag for data de-duplication
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/86
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/278
2020-01-31 01:18:54 +02:00
Aliaksandr Valialkin
a45f25699c
lib/storage: re-use indexSearch inside Storage.prefetchMetricNames
2020-01-31 01:18:53 +02:00
Aliaksandr Valialkin
cb5c39ee70
lib/fs: optimize small reads for ReaderAt.MustReadAt
by reading from memory-mapped space instead of reading from file descriptor
...
This should improve performance when reading many small blocks.
2020-01-30 15:16:16 +02:00
Aliaksandr Valialkin
da19fffa08
all: rename ReadAt* to MustReadAt* in order to dont clash with io.ReaderAt
2020-01-30 15:16:16 +02:00
Aliaksandr Valialkin
1332ddc15e
lib/storage: pass missing AccountID and ProjectID to searchMetricName
2020-01-30 15:16:16 +02:00
Aliaksandr Valialkin
4ed5e9a7ce
lib/storage: pre-fetch metricNames for the found metricIDs in Search.Init
...
This should speed up Search.NextMetricBlock loop for big number of found time series.
2020-01-30 15:16:16 +02:00
Aliaksandr Valialkin
cb2a2f281f
lib/mergeset: properly update lastAccesstime
in indexBlockCache entries
...
This is a follow-up for 6665f10e7b
2020-01-29 21:21:01 +02:00
Aliaksandr Valialkin
170c1c3a4e
app/vmselect/promql: add keep_next_value(q)
for filling gaps with the next non-empty value
2020-01-29 00:48:14 +02:00
Aliaksandr Valialkin
a9c1d5b351
app/vminsert: moved -maxInsertRequestSize
command-line flag out of lib/prompb
in order to prevent its inclusion in vmselect
and vmstorage
apps
2020-01-28 22:53:50 +02:00
Aliaksandr Valialkin
81ba371eaf
lib/logger: fix improperly set skipframes for all the logging functions
2020-01-26 18:34:58 +02:00
Aliaksandr Valialkin
9f595cb2b1
lib/httpserver: log the caller of httpserver.Errorf
...
Previously log message contained `httpserver.Errorf`, not it contains the caller of `httpserver.Errorf`, which is more useful.
2020-01-25 20:18:06 +02:00
Aliaksandr Valialkin
36a1a21d6e
lib/protoparser: add parser for Prometheus exposition text format
...
This parser will be used by vmagent
2020-01-24 20:11:19 +02:00
Aliaksandr Valialkin
0cda6afa8e
app/vminsert: move ingestion protocol parsers to lib/protoparser, so they could be re-used in the upcoming vmagent
2020-01-24 16:55:18 +02:00
Aliaksandr Valialkin
ea53a21b02
all: consistently log durations in seconds with millisecond precision
...
This should improve logs readability
2020-01-22 18:35:24 +02:00
Aliaksandr Valialkin
40e564eb9c
app/vmselect/promql: add range_over_time(m[d])
function for calculating value range for m
over d
2020-01-21 19:05:29 +02:00
Aliaksandr Valialkin
9eaa2ab871
app/vmselect/promql: add label_match(q, label, regexp)
and label_mismatch(q, label, regexp)
functions for filtering out time series with labels matching the given regexp
2020-01-21 15:00:35 +02:00
Aliaksandr Valialkin
62b041e90a
lib/{mergeset,storage}: properly update lastAccessTime
in index and data block cache entries
2020-01-20 15:00:10 +02:00
Aliaksandr Valialkin
607d4418b8
lib/uint64set: add missing bucket32.b16his values
2020-01-18 14:26:23 +02:00
Aliaksandr Valialkin
e3379537cd
lib/uint64set: optimize Set.Union
...
This should improve performance for queries over big number of time series
2020-01-18 13:47:34 +02:00
Aliaksandr Valialkin
5077efd3f7
lib/uint64set: add benchmarks for Set.Union
2020-01-18 13:47:33 +02:00
Aliaksandr Valialkin
a851c75703
lib/storage: skip recovering timestamps order for lossless compression (PrecisionBits=64)
2020-01-17 23:59:19 +02:00
Aliaksandr Valialkin
2084921e64
all: use github.com/klauspost/compress/gzip
instead of compress/gzip
...
`github.com/klauspost/compress/gzip` is more optimized than `compress/gzip`.
This gives better gzip compression and decompression speeds.
2020-01-17 23:59:17 +02:00
Aliaksandr Valialkin
ab4d5d72eb
lib/uint64set: reduce memory allocations in Set.AppendTo
2020-01-17 22:33:00 +02:00
Aliaksandr Valialkin
476c7fb109
lib/storage: reduce memory allocations when merging metricID sets
2020-01-17 22:10:56 +02:00
Aliaksandr Valialkin
29d21259f0
lib/uint64set: typo fix in Set.Intersect
2020-01-17 18:11:46 +02:00
Aliaksandr Valialkin
ed1d259b10
lib/uint64set: optimize Intersect, Subtract and Union functions
...
This should improve performance for queries over big number of time series.
2020-01-17 16:16:43 +02:00
Aliaksandr Valialkin
68d35357b1
lib/uint64set: improve benchmark for Set.Intersect
2020-01-17 16:16:43 +02:00