Aliaksandr Valialkin
70ca018a57
app/vminsert/opentsdbhttp: skip invalid rows and continue parsing the remaining rows
...
Invalid rows are logged and counted in `vm_rows_invalid_total{type="opentsdb-http"}` metric
2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
4266091e4f
app/vminsert/opentsdb: skip invalid rows and continue parsing the remaining rows
...
Invalid rows are logged and counted in `vm_rows_invalid_total{type="opentsdb"}` metric
2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
8001d29b6e
app/vminsert/graphite: skip invalid rows and continue parsing the remaining rows
...
Invalid rows are logged and counted in `vm_rows_invalid_total{type="graphite"}` metric
2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
9d3f1fcbb9
app/vminsert/influx: skip invalid rows and continue parsing the remaining rows
...
Invalid influx lines are logged and counted in `vm_rows_invalid_total{type="influx"}` metric.
2019-08-24 13:36:29 +03:00
Aliaksandr Valialkin
ba7b3806be
app/vminsert/influx: do not allow escaping newline char, since they dont occur in real life
...
The prefious report with escaped newline chars in influx line protocol was false alarm.
2019-08-23 18:42:05 +03:00
Aliaksandr Valialkin
7fa88c6efc
app/vminsert/opentsdbhttp: allow timestamp as float64 and as string, since it occurs in real life
2019-08-23 18:35:41 +03:00
Aliaksandr Valialkin
4da34b11f8
app/vminsert/influx: handle \r\n aka crlf influx line endings from windows world
...
Such lines exist in real life.
2019-08-23 18:28:49 +03:00
Aliaksandr Valialkin
a18317adbc
app/vminsert/influx: allow escaping newline char
...
Though newline char isn't mentioned in escape rules at https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_tutorial/ ,
there are reports that such chars occur in real life
2019-08-23 15:14:46 +03:00
Aliaksandr Valialkin
44d7fc599d
app/vminsert/influx: skip comments starting with # in influx line protocol
2019-08-23 14:43:09 +03:00
Aliaksandr Valialkin
ac004665b5
all: return 503 http error if service is temporarily unavailable
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/156
2019-08-23 09:55:07 +03:00
Aliaksandr Valialkin
8c03a8c4b4
app/vminsert: allow setting the maximum number of labels per time series via -maxLabelsPerTimeseries
2019-08-23 08:45:26 +03:00
Aliaksandr Valialkin
5f33fc8e46
app/vminsert: add ability to ingest data via HTTP OpenTSDB /api/put requests
...
This is manual merge of the https://github.com/VictoriaMetrics/VictoriaMetrics/pull/152
Thanks to nustinov@gmail.com for the initial pull request.
2019-08-22 12:28:32 +03:00
Aliaksandr Valialkin
ec8125606d
app/vminsert/opentsdb: fix BenchmarkRowsUnmarshal by adding missing put prefixes to each line
2019-08-21 19:14:47 +03:00
Aliaksandr Valialkin
5c83f8e203
app: add vm_concurrent_ metrics for visibility in concurrency limiters for vminsert and vmselect
2019-08-05 18:30:57 +03:00
Aliaksandr Valialkin
a9f39168d2
app/vminsert/influx: round automatically generated timestamp according to the given precision arg
2019-08-02 00:24:06 +03:00
Aliaksandr Valialkin
1eade9b358
app/vminsert: add vm_rows_per_insert summary metric
...
This metric should help tuning batch sizes on clients writing data to VictoriaMetrics
2019-07-27 13:21:46 +03:00
Aliaksandr Valialkin
bf6cbb762c
app/vminsert: improve error messages for Influx, OpenTSDB and Graphite parsing
...
Include in the error message the line which failed to parse.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/127
2019-07-26 22:08:52 +03:00
Aliaksandr Valialkin
575d2f0a91
app/vminsert: use netutil.TCPListener for collecting network-related metrics for Graphite and OpenTSDB TCP traffic
2019-07-15 22:58:00 +03:00
Jiri Tyr
e0fccc6c60
Change the default influxMeasurementFieldSeparator
2019-06-26 13:22:03 +03:00
Aliaksandr Valialkin
1f7d9a213a
app/vminsert: fix inifinite loop when reading two lines without newline in the end
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/82
2019-06-26 02:51:56 +03:00
Aliaksandr Valialkin
4b2523fb40
app/vminsert/opentsdb: remove unused const maxReadPacketSize; update https://github.com/VictoriaMetrics/VictoriaMetrics/issues/69
2019-06-20 14:30:06 +03:00
Aliaksandr Valialkin
a8d4224828
app/vminsert/graphite: allow skipping timestamps in Graphite plaintext protocol
...
In this case VictoriaMetrics uses the ingestion time as a timestamp.
2019-06-18 19:04:04 +03:00
Aliaksandr Valialkin
86d3d907a5
app/vminsert/influx: add -influxSkipSingleField flag for using {measurement} instead of {measurement}{separator}{field_name} for Influx lines with a single field
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/66
2019-06-17 19:05:57 +03:00
Aliaksandr Valialkin
269285848f
app/vminsert/influx: add -influxMeasurementFieldSeparator flag for the ability to change separator for {measurement}{separator}{field_name} metric name
2019-06-14 10:00:12 +03:00
Aliaksandr Valialkin
c0baa977cf
app/vminsert/concurrencylimiter: typo fix in the error message
2019-06-08 22:43:33 +03:00
Aliaksandr Valialkin
f4252f87e6
app/vminsert: really fix #60
...
ReadLinesBlock may accept dstBuf with non-zero length. In this case the last line without trailing newline isn't read.
Fix this by comparing len(dstBuf) to 0 instead of its original length.
2019-06-07 23:37:03 +03:00
Aliaksandr Valialkin
0b78d228d2
app/vminsert: properly read trailing line without newline in the end
...
This fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/60
2019-06-07 23:17:59 +03:00
Aliaksandr Valialkin
17f0a53068
app/vminsert: explain that /query request emulation is required for TSBS benchmark
2019-06-03 18:40:27 +03:00
Aliaksandr Valialkin
56b952c456
app/vminsert: add -maxConcurrentInserts command-line flag for limiting the number of concurrent inserts
2019-05-29 12:41:23 +03:00
Aliaksandr Valialkin
968d094524
app/vminsert: reduce memory usage for Influx, Graphite and OpenTSDB protocols
...
Do not buffer per-connection data and just store it as it arrives
2019-05-28 18:47:23 +03:00
Aliaksandr Valialkin
e307a4d92c
lib/timerpool: use timer pool in concurrency limiters
...
This should reduce the number of memory allocations in highly loaded system
2019-05-28 17:20:10 +03:00
Aliaksandr Valialkin
0eae39daa7
app/vminsert: properly reset InsertCtx.mrs - they must be empty after Reset call
2019-05-28 16:08:01 +03:00
Aliaksandr Valialkin
feb6b203a4
app/vminsert/influx: try converting string values to numeric values, since Influx agents may send numeric values as strings
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/34
2019-05-26 22:11:19 +03:00
Aliaksandr Valialkin
1836c415e6
all: open-sourcing single-node version
2019-05-23 00:18:06 +03:00
