Aliaksandr Valialkin
8b56b849e9
app/vminsert: return StatusNoContent http response for /api/v1/import
to be consistent with other insert handlers
2019-12-19 01:22:01 +02:00
Aliaksandr Valialkin
a7bf8e77af
app/vminsert: simultaneously accept telnet put
and HTTP /api/put
OpenTSDB metrics at -opentsdbListenAddr
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/266
2019-12-14 00:42:18 +02:00
Aliaksandr Valialkin
cffaeda0f1
all: publish Docker images for the following GOARCH: amd64, arm, arm64, ppc64le and 386
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/258
2019-12-11 23:33:11 +02:00
Aliaksandr Valialkin
c81a89a8ed
app/vminsert: add /api/v1/import
handler
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6
2019-12-09 22:37:49 +02:00
Aliaksandr Valialkin
0c304439d4
app/vminsert: consistency renaming for counters
2019-12-09 16:44:26 +02:00
Aliaksandr Valialkin
c217a53c35
make vendor-update
2019-12-07 23:11:26 +02:00
Aliaksandr Valialkin
3534e71c96
app/vminsert/influx: add a test case from https://community.librenms.org/t/integration-with-victoriametrics/9689
2019-12-07 23:00:54 +02:00
Aliaksandr Valialkin
1595dcd3d9
app/vminsert/influx: allow empty measurement in Influx line protocol
...
In this case metric names are mapped directly from field names without any prefixes.
2019-11-30 21:59:07 +02:00
Aliaksandr Valialkin
4c63caa37c
deployment/docker/certs: update TLS certs source from alpine:3.9 to alpine:3.10
2019-11-29 19:55:36 +02:00
Aliaksandr Valialkin
87b39222be
Revert "lib/fs: do not postpone directory removal on NFS error"
...
This reverts commit 21aeb02b46649ac9906cb37733f7b155a77a0db9.
2019-11-12 16:29:50 +02:00
Aliaksandr Valialkin
5f52eb7653
lib/fs: do not postpone directory removal on NFS error
...
Continue trying to remove NFS directory on temporary errors for up to a minute.
The previous async removal process breaks in the following case during VictoriaMetrics start
- VictoriaMetrics opens index, finds incomplete merge transactions and starts replaying them.
- The transaction instructs removing old directories for parts, which were already merged into bigger part.
- VictoriaMetrics removes these directories, but their removal is delayed due to NFS errors.
- VictoriaMetrics scans partition directory after all the incomplete merge transactions are finished
and finds directories, which should be removed, but weren't still removed due to NFS errors.
- VictoriaMetrics panics when it finds unexpected empty directory.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/162
2019-11-10 13:27:16 +02:00
Aliaksandr Valialkin
78fc35c9b1
all: make fmt
2019-10-17 20:05:12 +03:00
Aliaksandr Valialkin
5b01b7fb01
all: add support for GOARCH=386 and fix all the issues related to 32-bit architectures such as GOARCH=arm
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/212
2019-10-17 18:27:49 +03:00
Aliaksandr Valialkin
8d398af92f
app/vminsert/netstorage: mention the data size that cannot be sent to vmstorage
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/175
2019-09-25 12:53:41 +03:00
Aliaksandr Valialkin
73ac7b8dd6
app/vminsert/netstorage: make sure the conn exists before closing it in storageNode.closeBrokenConn
...
The conn can be missing or already closed during the call to storageNode.closeBrokenConn.
Prevent `nil pointer dereference` panic by verifying whether the conn is already closed.
Thanks to @CH-anhngo for reporting the issue.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/189
2019-09-25 10:36:50 +03:00
Aliaksandr Valialkin
b0c738ae8b
app/vminsert/opentsdbhttp: remove FATAL prefix from logger.Fatalf errors for the sake of consistency with other logger.Fatalf calls
2019-09-19 22:16:11 +03:00
Aliaksandr Valialkin
550a12415a
app/vminsert/netstorage: log network errors when sending data to vmstorage nodes
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/175
2019-09-13 22:26:24 +03:00
Aliaksandr Valialkin
ed50b8792b
app/vminsert/netstorage: reduce the maximum buffer size for rerouted rows, so it occupies less RAM
2019-09-11 14:50:30 +03:00
Aliaksandr Valialkin
2f4c950fe9
app/vminsert/netstorage: send per-storageNode bufs to vmstorage nodes in parallel
...
This should improve the maximum ingestion throughput.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/175
2019-09-11 14:50:19 +03:00
Aliaksandr Valialkin
694cc59ed1
app/vminsert/netstorage: dynamically adjust timeouts for sending packets from vminsert to vmstorage depending on packet size
...
Bigger packets will have more chances to be sent to vmstorage.
2019-09-11 14:50:14 +03:00
Aliaksandr Valialkin
2c654258ef
lib/fs: add MustStopDirRemover for waiting until pending directories are removed on graceful shutdown
...
This patch is mainly required for laggy NFS. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/162
2019-09-05 11:17:17 +03:00
Aliaksandr Valialkin
38711526d3
app/vminsert/influx: set db
label only if Influx line doesnt have db
tag
2019-08-24 13:55:01 +03:00
Aliaksandr Valialkin
1ee536f9fd
app/vminsert: skip empty tags
2019-08-24 13:36:41 +03:00
Aliaksandr Valialkin
a283023d16
app/vminsert/opentsdbhttp: skip invalid rows and continue parsing the remaining rows
...
Invalid rows are logged and counted in `vm_rows_invalid_total{type="opentsdb-http"}` metric
2019-08-24 13:36:41 +03:00
Aliaksandr Valialkin
38b9615c53
app/vminsert/opentsdb: skip invalid rows and continue parsing the remaining rows
...
Invalid rows are logged and counted in `vm_rows_invalid_total{type="opentsdb"}` metric
2019-08-24 13:36:41 +03:00
Aliaksandr Valialkin
2a8fc41bab
app/vminsert/graphite: skip invalid rows and continue parsing the remaining rows
...
Invalid rows are logged and counted in `vm_rows_invalid_total{type="graphite"}` metric
2019-08-24 13:36:41 +03:00
Aliaksandr Valialkin
22685ef94d
app/vminsert/influx: skip invalid rows and continue parsing the remaining rows
...
Invalid influx lines are logged and counted in `vm_rows_invalid_total{type="influx"}` metric.
2019-08-24 13:36:41 +03:00
Aliaksandr Valialkin
425a81a6c7
app/vminsert/influx: do not allow escaping newline char, since they dont occur in real life
...
The prefious report with escaped newline chars in influx line protocol was false alarm.
2019-08-23 18:43:00 +03:00
Aliaksandr Valialkin
8da8dd0876
app/vminsert/opentsdbhttp: allow timestamp as float64 and as string, since it occurs in real life
2019-08-23 18:35:52 +03:00
Aliaksandr Valialkin
0ea21eb9dc
app/vminsert/influx: handle \r\n
aka crlf
influx line endings from windows world
...
Such lines exist in real life.
2019-08-23 18:28:54 +03:00
Aliaksandr Valialkin
b3502b2b39
app/vminsert/influx: allow escaping newline char
...
Though newline char isn't mentioned in escape rules at https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_tutorial/ ,
there are reports that such chars occur in real life
2019-08-23 15:14:58 +03:00
Aliaksandr Valialkin
f1f8fce4f7
app/vminsert/influx: skip comments starting with #
in influx line protocol
2019-08-23 14:43:24 +03:00
Aliaksandr Valialkin
697de90893
app/vminsert: do not drop data in reroutedBuf if all the storage nodes are unhealthy
2019-08-23 10:38:19 +03:00
Aliaksandr Valialkin
a5dc54efc3
app/vminsert: properly limit the size of reroutedBuf
2019-08-23 10:29:51 +03:00
Aliaksandr Valialkin
c197641978
all: return 503 http error if service is temporarily unavailable
...
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/156
2019-08-23 09:49:50 +03:00
Aliaksandr Valialkin
e734076f0f
app/vminsert: allow setting the maximum number of labels per time series via -maxLabelsPerTimeseries
2019-08-23 08:47:18 +03:00
Aliaksandr Valialkin
0697164b4f
app/vminsert: add ability to ingest data via HTTP OpenTSDB /api/put
requests
...
This is manual merge of the https://github.com/VictoriaMetrics/VictoriaMetrics/pull/152
Thanks to nustinov@gmail.com for the initial pull request.
2019-08-22 12:46:54 +03:00
Aliaksandr Valialkin
4d555c7c87
app/vminsert/opentsdb: fix BenchmarkRowsUnmarshal by adding missing put
prefixes to each line
2019-08-21 19:15:04 +03:00
Aliaksandr Valialkin
8e05758ff5
app: add vm_concurrent_
metrics for visibility in concurrency limiters for vminsert and vmselect
2019-08-05 18:30:29 +03:00
Aliaksandr Valialkin
8ed84a4713
app/vminsert/influx: round automatically generated timestamp according to the given precision
arg
2019-08-02 00:24:39 +03:00
Aliaksandr Valialkin
b7089705b7
app/vminsert: add vm_rows_per_insert
summary metric
...
This metric should help tuning batch sizes on clients writing data to VictoriaMetrics
2019-07-27 13:28:20 +03:00
Aliaksandr Valialkin
1fd4e9fb5c
app/vminsert: improve error messages for Influx, OpenTSDB and Graphite parsing
...
Include in the error message the line which failed to parse.
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/127
2019-07-26 22:09:21 +03:00
Aliaksandr Valialkin
54f035d4ce
all: small updates after PR #114
2019-07-24 17:43:43 +03:00
Aliaksandr Valialkin
cb8104cf77
app: clarify error messages when -storageNode
arg is missing in vminsert and vmselect
2019-07-20 10:21:59 +03:00
Aliaksandr Valialkin
2b4254d01f
app/vminsert: use netutil.TCPListener for collecting network-related metrics for Graphite and OpenTSDB TCP traffic
2019-07-15 22:58:35 +03:00
Jiri Tyr
e7a0bf1a71
Change the default influxMeasurementFieldSeparator
2019-06-26 13:22:54 +03:00
Aliaksandr Valialkin
d5cb9fddd8
app/vminsert: fix inifinite loop when reading two lines without newline in the end
...
Fixes https://github.com/VictoriaMetrics/VictoriaMetrics/issues/82
2019-06-26 02:52:56 +03:00
Aliaksandr Valialkin
3838d224d5
app/vminsert/opentsdb: remove unused const maxReadPacketSize
; update https://github.com/VictoriaMetrics/VictoriaMetrics/issues/69
2019-06-20 14:30:02 +03:00
Aliaksandr Valialkin
d4ed6189d4
app/vminsert/graphite: allow skipping timestamps in Graphite plaintext protocol
...
In this case VictoriaMetrics uses the ingestion time as a timestamp.
2019-06-18 19:05:46 +03:00
Aliaksandr Valialkin
e40224d5de
lib/flagutil: add NewArray helper func
2019-06-18 10:44:09 +03:00