VictoriaMetrics/lib
Zakhar Bessarab 5b3cbd4db1
app/vlinsert: add support of loki push protocol (#4482)
* app/vlinsert: add support of loki push protocol

- implemented loki push protocol for both Protobuf and JSON formats
- added examples in documentation
- added example docker-compose

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert: move protobuf metric into its own file

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/promtail: update reference to docker image

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/promtail: make volume name unique

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: add license reference

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/promtail: fix volume name

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/VictoriaLogs/data-ingestion: add stream fields for loki JSON ingestion example

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: move entities to places where those are used

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: refactor to use common components

- use CommonParameters from insertutils
- stop ingestion after first error similar to elasticsearch and jsonline

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: address review feedback

- add missing logstorage.PutLogRows calls
- refactor tenant ID parsing to use common function
- reduce number of allocations for parsing by reusing  logfields slices
- add tests and benchmarks for requests processing funcs

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-20 16:49:43 -07:00
..
appmetrics all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:48:26 -07:00
auth app/vminsert: allows parsing tenant id from labels (#3009) 2022-09-30 17:28:35 +03:00
awsapi .golangci.yml: properly enable revive linter and fix all the warnings it detects 2023-02-26 12:19:58 -08:00
backup fix removing storage data dir before restoring from backup (#598) 2023-07-06 22:32:12 -07:00
blockcache
bloomfilter all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:48:26 -07:00
bufferedwriter app/vmselect: move common http functionality from app/vmselect/searchutils to lib/httputils 2023-07-06 17:22:23 -07:00
buildinfo
bytesutil docs: follow-up after aec4b5db81 (#4638) 2023-07-19 14:48:17 -07:00
cgroup lib/cgroup: add SetGOGC() function 2023-07-06 17:24:31 -07:00
consts
decimal lib/decimal: use consistent randomizer in tests 2023-01-23 19:24:05 -08:00
encoding all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:48:26 -07:00
envflag docs: follow-up after aec4b5db81 (#4638) 2023-07-19 14:48:17 -07:00
envtemplate allowed using dashes and dots in environment variables names (#4009) 2023-03-24 17:57:19 -07:00
fastnum
fasttime
filestream lib/filestream: change Create() to MustCreate() 2023-04-14 15:14:24 -07:00
flagutil lib/flagutil: ArrayString: support commas inside quoted strings and inside [], {} and () braces 2023-03-28 21:25:07 -07:00
formatutil app/vmbackupmanager: add metrics for better observability (#488) 2022-12-20 14:18:43 -08:00
fs lib/fs: add ReaderAt.Path() function 2023-07-06 17:25:19 -07:00
handshake lib/handshake: do not pollute logs with cannot read hello messages on TCP health checks 2023-05-18 10:37:59 -07:00
htmlcomponents app/vmselect: remove dependency on lib/promscrape from app/vmselect 2023-01-03 23:27:36 -08:00
httpserver docs: make httpAuth.* flags description less ambiguous (#4588) 2023-07-09 12:36:14 -07:00
httputils lib/httputils: fix test after b49d04b3dc 2023-07-06 22:21:43 -07:00
influxutils lib/flagutil: rename Array to ArrayString 2022-10-01 18:28:19 +03:00
ingestserver lib/netutil: init implimentation of proxy protocol (#3687) 2023-01-26 23:25:22 -08:00
leveledbytebufferpool
logger add error handler for parsing prometheus text format to vmagent and v… (#3693) 2023-01-23 22:36:23 -08:00
logjson app/vlinsert/jsonline: code prettifying 2023-07-06 21:35:55 -07:00
logstorage app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 16:49:43 -07:00
lrucache
memory docs: follow-up after aec4b5db81 (#4638) 2023-07-19 14:48:17 -07:00
mergeset lib/mergeset: skip common prefix in binarySearchKey() function 2023-07-13 22:05:14 -07:00
metricsql
netutil docs: follow-up after aec4b5db81 (#4638) 2023-07-19 14:48:17 -07:00
persistentqueue app/vmagent,lib/persistentqueue: show warning message if --remoteWrite.maxDiskUsagePerURL flag lower than 500MB (#4196) 2023-05-08 15:45:21 -07:00
procutil lib/procutil: stop immediately after receiving the second SIGINT or SIGTERM signal 2022-10-20 21:58:49 +03:00
promauth lib/promscrape: disable support for service discovery and metrics scrape via http2 2023-07-06 16:04:31 -07:00
prompb
prompbmarshal
promrelabel all: add support for or filters in series selectors 2023-07-15 23:56:18 -07:00
promscrape all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:48:26 -07:00
promutils fix parse for invalid partial RFC3339 format (#4539) 2023-07-06 22:09:35 -07:00
protoparser lib/promutils: properly return error when incorrect Prometheus label names are passed to NewLabelsFromString() 2023-05-12 17:02:06 -07:00
proxy
pushmetrics fixed typos in documentation and commandline flags descriptions (#4275) 2023-05-10 02:22:06 -07:00
querytracer lib/querytracer: fix remaining tests after 49ebc48809 2022-12-08 18:18:50 -08:00
regexutil app,lib: fix typos in comments (#3804) 2023-02-13 09:32:35 -08:00
slicesutil app/vlinsert: add support of loki push protocol (#4482) 2023-07-20 16:49:43 -07:00
snapshot app/vmbackup: prevent password leaks (#3672) 2023-01-18 11:40:52 -08:00
storage all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:48:26 -07:00
streamaggr Revert "lib/streamaggr: discard samples with timestamps outside of aggregation interval (#4199)" 2023-05-08 21:50:19 -07:00
syncwg
tenantmetrics app/vminsert: allows parsing tenant id from labels (#3009) 2022-09-30 17:28:35 +03:00
timerpool
uint64set lib/uint64set: use repeatable randomizer in tests 2023-01-23 19:24:05 -08:00
vmselectapi lib/vmselectapi: move the code for checking the expected client errors into a isExpectedError() function 2023-07-06 16:37:59 -07:00
workingsetcache all: replace atomic.Value with atomic.Pointer[T] 2023-07-19 17:48:26 -07:00
writeconcurrencylimiter lib/writeconcurrencylimiter: initialize concurrencyLimitCh before exporting vm_concurrent_insert_capacity and vm_concurrent_insert_current metrics 2023-02-07 11:08:39 -08:00