Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-20 23:46:23 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
1,713 Commits 114 Branches 748 Tags 625 MiB
1659135752
Commit Graph

80 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
124f78857b app/{vminsert,vmagent}: improve data ingestion speed over a single connection 
Process data obtianed from a single connection on all the available CPU cores.
 2020-09-28 04:13:08 +03:00
Aliaksandr Valialkin
95688cbfc5 all: add native format for data export and import 
The data can be exported via [/api/v1/export/native](https://victoriametrics.github.io/#how-to-export-data-in-native-format) handler
and imported via [/api/v1/import/native](https://victoriametrics.github.io/#how-to-import-data-in-native-format) handler.
 2020-09-27 19:54:07 +03:00
Aliaksandr Valialkin
c00627c103 app/vminsert: code prettifying 2020-09-26 04:13:18 +03:00
Aliaksandr Valialkin
2380e9b017 app/{vminsert,vmagent}: allow passing timestamp via timestamp query arg when ingesting data to /api/v1/import/prometheus 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/750
 2020-09-11 13:27:14 +03:00
Aliaksandr Valialkin
f41b36bb9a app/{vminsert,vmagent}: allow adding extra labels when importing data via Prometheus, CSV and JSON line formats 
Extra labels may be added to the imported data by passing `extra_label=name=value` query args.
Multiple query args may be passed in order to add multiple extra labels.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/719
 2020-09-02 19:43:21 +03:00
Aliaksandr Valialkin
576da0fe46 app/{vminsert,vmagent}: improve documentation for -influxListenAddr command-line flag 2020-08-14 18:04:44 +03:00
Aliaksandr Valialkin
0f63da3698 app/{vmagent,vminsert}: properly preserve db tag from query string passed to Influx line protocol query 
Previously `db` tag from the query string wasn't added to metrics after encountering `db` tag in the Influx line

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/653
 2020-07-28 21:25:19 +03:00
Aliaksandr Valialkin
215eba0b82 app/vminsert: flush bufs if needed after the current row is added 
Previously the data for the added row could be overwritten by the flush
before the row addition is complete.
 2020-07-26 12:10:11 +03:00
Aliaksandr Valialkin
a090627059 app/vminsert: limit memory usage when ingesting data in big packets 2020-07-24 23:32:40 +03:00
Aliaksandr Valialkin
4c8b49b193 app/vminsert: export vm_relabel_metrics_dropped_total metric that shows the number of metrics dropped due to relabeling 2020-07-23 14:57:53 +03:00
Aliaksandr Valialkin
9257eee982 app/vminsert: do not call ApplyRelabeling function if relabeling is disabled 
This should reduce CPU usage a bit when `-relabelConfig` isn't set
 2020-07-23 13:39:44 +03:00
Aliaksandr Valialkin
2f612e0c67 app/vminsert: fix relabeling for metrics ingested via Influx line protocol 
Previously the enabled relabeling with `-relabelConfig` command-line flag could result in missing labels
if a single Influx line protocol message contains multiple field values.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/638
 2020-07-23 13:23:14 +03:00
Aliaksandr Valialkin
b35cb293f5 lib/httpserver: log remote address in error message from httpserver.Errorf 
This should improve detection of the root cause of errors.
Thanks to Anant for the idea.
 2020-07-20 14:11:22 +03:00
Aliaksandr Valialkin
8c14ca93fa app/vminsert/influx: properly handle the case when certain labels with empty values are removed by ApplyRelabeling() call 
Previously this could lead to `out of range` panic
 2020-07-17 00:07:06 +03:00
Aliaksandr Valialkin
4081e2295e app/{vminsert,vmagent}: add -influxSkipMeasurement command-line flag for using field name as metric name 
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/626
 2020-07-14 14:17:24 +03:00
Aliaksandr Valialkin
cba820e390 app/{vminsert,vmagent}: add ability to import data in Prometheus exposition format via /api/v1/import/prometheus 2020-07-10 12:14:07 +03:00
Aliaksandr Valialkin
0b2086b7a5 app/vminsert: prevent from adding and/or selecting labels with empty values 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/600
 2020-07-02 23:14:11 +03:00
Aliaksandr Valialkin
91b3482894 app/vminsert: add ability to apply relabeling to all the incoming metrics if -relabelConfig command-line arg points to a file with a list of relabel_config entries 
See https://victoriametrics.github.io/#relabeling
 2020-07-02 20:39:28 +03:00
Aliaksandr Valialkin
d5dddb0953 all: use %w instead of %s for wrapping errors in fmt.Errorf 
This will simplify examining the returned errors such as httpserver.ErrorWithStatusCode .
See https://blog.golang.org/go1.13-errors for details.
 2020-06-30 23:05:11 +03:00
Aliaksandr Valialkin
b542e50680 app/vminsert: export metrics for determining ingested rows with dropped or truncated labels 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/565
 2020-06-19 01:10:37 +03:00
Aliaksandr Valialkin
432187ac3b app/vminsert: add /-/reload handler in the same way as for vmagent 2020-04-30 02:15:39 +03:00
Aliaksandr Valialkin
1fe66fb3cc app/{vmagent,vminsert}: add support for importing csv data via /api/v1/import/csv 2020-03-10 21:15:35 +02:00
Aliaksandr Valialkin
032c88561b app/vminsert/prompush: limit memory usage by pushing promscrape data in smaller blocks 2020-03-03 19:58:54 +02:00
Aliaksandr Valialkin
6386aeb1e0 app/vmagent: add ability to accept Influx line protocol data via TCP and UDP 
Just set `-influxListenAddr` command-line flag

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/333
 2020-02-25 19:12:49 +02:00
Aliaksandr Valialkin
4c4448b66e app/vminsert: add /targets handler, which exposes Prometheus targets defined in -promscrape.config file 2020-02-25 18:13:11 +02:00
Aliaksandr Valialkin
04762344c6 app/vmagent: initial implementation for vmagent 2020-02-23 13:36:03 +02:00
Aliaksandr Valialkin
42864bb52f all: do not clash flag description with back-quoted flag types 
See https://golang.org/pkg/flag/#PrintDefaults for more details.
 2020-02-04 15:46:52 +02:00
Aliaksandr Valialkin
371e86194d app/vminsert: moved -maxInsertRequestSize command-line flag out of lib/prompb in order to prevent its inclusion in vmselect and vmstorage apps 2020-01-28 23:02:08 +02:00
Aliaksandr Valialkin
6456c93dbb app/vminsert: move ingestion protocol parsers to lib/protoparser, so they could be re-used in the upcoming vmagent 2020-01-24 16:53:00 +02:00
Aliaksandr Valialkin
227cf53ef9 app/vminsert: increase default value for -insert.maxQueueDuration from 30s to 60s 
This should help catching up with high ingestion rate after VictoriaMetrics restart.
 2020-01-18 14:39:36 +02:00
Aliaksandr Valialkin
c9472e4f3a all: use github.com/klauspost/compress/gzip instead of compress/gzip 
`github.com/klauspost/compress/gzip` is more optimized than `compress/gzip`.
This gives better gzip compression and decompression speeds.
 2020-01-17 23:58:46 +02:00
Aliaksandr Valialkin
d88725f133 app/{vminsert,vmselect}: improve error messages when VictoriaMetrics cannot handle too high number of concurrent inserts / selects 2020-01-17 13:24:37 +02:00
Aliaksandr Valialkin
0d7505b00e all: mention command-line flags used for limiting the incoming request size in error messages 
This should improve error logs usability.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/287
 2020-01-16 13:03:30 +02:00
Aliaksandr Valialkin
4f0a645f77 vendor: update github.com/valyala/fastjson from v1.4.2 to v1.4.5 
This should fix parsing Inf values in `/api/v1/import`. The previous attempt to fix this in VictoriaMetrics v1.32.1 was unsuccessful.
 2020-01-10 23:15:15 +02:00
Aliaksandr Valialkin
d03827c57d app/vminsert: return StatusNoContent http response for /api/v1/import to be consistent with other insert handlers 2019-12-19 01:21:54 +02:00
Aliaksandr Valialkin
bc645152cb app/vminsert: simultaneously accept telnet put and HTTP /api/put OpenTSDB metrics at -opentsdbListenAddr 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/266
 2019-12-14 00:30:12 +02:00
Aliaksandr Valialkin
68e1cf8942 app/vminsert: add /api/v1/import handler 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6
 2019-12-09 20:59:04 +02:00
Aliaksandr Valialkin
8501b4a48d app/vminsert: consistency renaming for counters 2019-12-09 16:43:10 +02:00
Aliaksandr Valialkin
293b541784 make vendor-update 2019-12-07 23:10:16 +02:00
Aliaksandr Valialkin
84b57e8974 app/vminsert/influx: add a test case from https://community.librenms.org/t/integration-with-victoriametrics/9689 2019-12-07 23:00:40 +02:00
Aliaksandr Valialkin
811b7a8303 app/vminsert/influx: allow empty measurement in Influx line protocol 
In this case metric names are mapped directly from field names without any prefixes.
 2019-11-30 23:18:41 +02:00
Aliaksandr Valialkin
e83fe938c8 all: make fmt 2019-10-17 20:04:34 +03:00
Aliaksandr Valialkin
97ce4e03a5 all: add support for GOARCH=386 and fix all the issues related to 32-bit architectures such as GOARCH=arm 
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/212
 2019-10-17 18:23:23 +03:00
Aliaksandr Valialkin
d32f88c378 app/vminsert/opentsdbhttp: remove FATAL prefix from logger.Fatalf errors for the sake of consistency with other logger.Fatalf calls 2019-09-19 22:15:59 +03:00
Aliaksandr Valialkin
fb909cf710 app/vminsert/influx: set db label only if Influx line doesnt have db tag 2019-08-24 13:52:48 +03:00
Aliaksandr Valialkin
9c9221d1b2 app/vminsert: skip empty tags 2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
70ca018a57 app/vminsert/opentsdbhttp: skip invalid rows and continue parsing the remaining rows 
Invalid rows are logged and counted in `vm_rows_invalid_total{type="opentsdb-http"}` metric
 2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
4266091e4f app/vminsert/opentsdb: skip invalid rows and continue parsing the remaining rows 
Invalid rows are logged and counted in `vm_rows_invalid_total{type="opentsdb"}` metric
 2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
8001d29b6e app/vminsert/graphite: skip invalid rows and continue parsing the remaining rows 
Invalid rows are logged and counted in `vm_rows_invalid_total{type="graphite"}` metric
 2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
9d3f1fcbb9 app/vminsert/influx: skip invalid rows and continue parsing the remaining rows 
Invalid influx lines are logged and counted in `vm_rows_invalid_total{type="influx"}` metric.
 2019-08-24 13:36:29 +03:00