Commit Graph

160 Commits

Author SHA1 Message Date
Zakhar Bessarab
5b3cbd4db1
app/vlinsert: add support of loki push protocol (#4482)
* app/vlinsert: add support of loki push protocol

- implemented loki push protocol for both Protobuf and JSON formats
- added examples in documentation
- added example docker-compose

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert: move protobuf metric into its own file

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/promtail: update reference to docker image

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/promtail: make volume name unique

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: add license reference

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* deployment/docker/victorialogs/promtail: fix volume name

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs/VictoriaLogs/data-ingestion: add stream fields for loki JSON ingestion example

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: move entities to places where those are used

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: refactor to use common components

- use CommonParameters from insertutils
- stop ingestion after first error similar to elasticsearch and jsonline

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* app/vlinsert/loki: address review feedback

- add missing logstorage.PutLogRows calls
- refactor tenant ID parsing to use common function
- reduce number of allocations for parsing by reusing  logfields slices
- add tests and benchmarks for requests processing funcs

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-20 16:49:43 -07:00
Roman Khavronenko
80768d53dd
docs: follow-up after aec4b5db81 (#4638)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-07-19 14:48:17 -07:00
Aliaksandr Valialkin
2042ea7bf3
docs/VictoriaLogs/CHANGELOG.md: remove newlines inside FEATURE descriptions, since they dont play nicely when copy-n-pasting to https://github.com/VictoriaMetrics/VictoriaMetrics/releases/ 2023-07-17 23:31:37 -07:00
Aliaksandr Valialkin
15ccde8dd3
all: use the new release for VictoriaLogs - v0.2.0-victorialogs 2023-07-17 23:20:07 -07:00
Aliaksandr Valialkin
8e42b2294c
docs/VictoriaLogs: add CHANGELOG.md 2023-07-17 23:14:23 -07:00
Aliaksandr Valialkin
da2ef397fa
lib/logstorage: add abilty to speficy offset for the selected _time filter
The following syntax is supported: _time:filter offset off
For example:

- _time:5m offset 1h - 5-minute duration one hour before the current time
- _time:2023 offset 2w - 2023 year with the 2 weeks offset in the past
2023-07-17 19:07:14 -07:00
Aliaksandr Valialkin
6751a08071
lib/logstorage: support for short form of _time:(now-duration, now] filter: _time:duration 2023-07-17 18:23:43 -07:00
Aliaksandr Valialkin
8fdfd13a29
lib/logstorage: LogsQL: replace exact_prefix("...") with exact("..."*)
This makes LogsQL queries more consistent with i("...") and i("..."*) syntax
2023-07-17 17:19:45 -07:00
Aliaksandr Valialkin
8715e331c8
docs/VictoriaLogs/data-ingestion/README.md: document how to verify whether the data is successfully ingested into VictoriaLogs 2023-07-16 22:54:05 -07:00
Artem Navoiev
30e84f4c66
docs: logs disable table of content for pages with no h2-h6, add aliases
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-16 00:08:06 -07:00
Artem Navoiev
6363a7bb80
docs: victorialogs add front-matter for readme pages
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-16 00:08:06 -07:00
Artem Navoiev
2963cbbb74
docs: victorialogs add front-matter for data-ingestion items
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-16 00:08:06 -07:00
Aliaksandr Valialkin
6a65af5112
all: replace ElasticSearch -> Elasticsearch for the sake of consistency
This is a follow-up for 7f6b5dc47b
2023-07-14 10:52:43 -07:00
Aliaksandr Valialkin
99ec7ef28f
docs/VictoriaLogs: use exact number when comparing efficiency of VictoriaLogs and Elasticsearch 2023-07-14 10:47:29 -07:00
subham sarkar
9d3cfc03cd
docs: Update README.md (#4613)
Fix grammatical mistakes and also s/ElasticSearch/Elasticsearch
2023-07-14 10:46:13 -07:00
Artem Navoiev
c245e38f8d
fix alias
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-14 10:44:04 -07:00
Artem Navoiev
6b98511ac9
add front-matter for some of victorialogs pages
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-14 10:44:04 -07:00
Aliaksandr Valialkin
2636c35cec
docs/VictoriaLogs/FAQ.md: small fixes 2023-07-12 01:10:40 -07:00
Aliaksandr Valialkin
b99fcb7d7a
docs/VictoriaLogs: add FAQ 2023-07-12 00:30:33 -07:00
Aliaksandr Valialkin
f8299ee9d8
docs/VictoriaLogs/README.md: make it clear that VictoriaLogs is open source 2023-07-12 00:30:33 -07:00
Aliaksandr Valialkin
a5f55259f6
docs/VictoriaLogs: make more prominent the information about returned log fields in query responses
Thanks to @candlerb for suggestions on how to improve VictoriaLogs docs
at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4609#issuecomment-1629758426
2023-07-10 15:01:54 -07:00
Aliaksandr Valialkin
371036eb35
docs/VictoriaLogs: small clarifications 2023-07-09 12:36:38 -07:00
Zakhar Bessarab
ddd918b93c
docs: make httpAuth.* flags description less ambiguous (#4588)
* docs: make `httpAuth.*` flags description less ambiguous

Currently, it may confuse users whether `httpAuth.*` flags are used by HTTP client or server configuration(see https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4586 for example).

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* docs: fix a typo

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-09 12:36:14 -07:00
Aliaksandr Valialkin
85c134feec
docs/VictoriaLogs/LogsQL.md: various fixes according to https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4579 2023-07-06 22:24:41 -07:00
Artem Navoiev
7ff0ac1a33
docs: add command-line flags (#4550)
* add command-line flags

Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-06 22:08:21 -07:00
Craig Rodrigues
4e0ca99da9
docs/VictoriaLogs: Fix curl command for fetching binary 2023-07-06 21:50:05 -07:00
Artem Navoiev
d48e697bbd
fix jsonline endpoint in docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-06 21:40:59 -07:00
Aliaksandr Valialkin
b215e6e044
docs/VictoriaLogs/QuickStart.md: add an option to run VictoriaLogs from pre-built binaries 2023-07-06 21:39:12 -07:00
Aliaksandr Valialkin
166456f340
docs/VictoriaLogs/README.md: add Upgrading section 2023-07-06 21:39:12 -07:00
Aliaksandr Valialkin
881dc29ef8
docs/VictoriaLogs/README.md: typo fix 2023-07-06 21:39:12 -07:00
Aliaksandr Valialkin
a3956b3d9c
all: update VictoriaLogs docker tag to v0.1.0-victorialogs 2023-07-06 21:39:12 -07:00
Aliaksandr Valialkin
4b10432435
app/vlselect: handle vmui at /select/vmui path instead of /vmui
This simplifies routing at auth proxies such as vmauth to vlselect component,
which serves VMUI - just route all the requests, which start with /select/, to vlselect.
2023-07-06 21:36:28 -07:00
Aliaksandr Valialkin
33e4d51636
docs/VictoriaLogs/data-ingestion/README.md: remove trailing spaces 2023-07-06 21:35:55 -07:00
Aliaksandr Valialkin
08634ae612
app/vlinsert/jsonline: code prettifying 2023-07-06 21:35:55 -07:00
Aliaksandr Valialkin
64b8aa108b
docs/VictoriaLogs/README.md: small fixes 2023-07-06 21:35:55 -07:00
Aliaksandr Valialkin
7206f91201
docs/VictoriaLogs/data-ingestion: small fixes 2023-07-06 21:35:22 -07:00
Aliaksandr Valialkin
ee720d1bbc
docs/VictoriaLogs/querying: rename VMUI -> 'Web UI' in order to reduce confusion with VictoriaMetrics UI 2023-07-06 21:35:03 -07:00
Aliaksandr Valialkin
2b43e5be80
docs/VictoriaLogs: small fixes 2023-07-06 21:34:41 -07:00
Alexander Marshalov
c6dba396db
added docs for vmui in victorialogs (#4494) 2023-07-06 21:33:49 -07:00
Alexander Marshalov
c12b5250c7
added more info and examples about data ingestion and collectors to VictoriaLogs docs (#4490) 2023-07-06 21:31:56 -07:00
Alexander Marshalov
d9d759bc90
jsonline support for data ingestion in vlinsert (#4487)
added json lines / json stream format for ingestion to vlinsert
2023-07-06 21:30:35 -07:00
Artem Navoiev
a972567faa
docs: change wording in victorialogs benchmarks section
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-06 21:29:55 -07:00
Artem Navoiev
b33f7a3181
docs: change wording in victorialogs benchmarks section
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-06 21:29:34 -07:00
Artem Navoiev
d9b625719f
fix typos
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-06 21:29:11 -07:00
Zakhar Bessarab
7ab9a4d5dc
docs/VictoriaLogs: add benchmarks section into readme
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-06 21:28:48 -07:00
Zakhar Bessarab
ed18c503ac
docs/VictoriaLogs: add reference to benchmark setup in readme
Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-06 21:28:27 -07:00
Aliaksandr Valialkin
efee71986f
app/vlselect/logsql: sort query results by _time if their summary size doesnt exceed -select.maxSortBufferSize 2023-07-06 21:25:00 -07:00
Aliaksandr Valialkin
c77f729680
docs/VictoriaLogs/README.md: typo fix 2023-07-06 21:23:45 -07:00
Aliaksandr Valialkin
74ac4722a9
docs/VictoriaLogs/README.md: add missing link to quick start docs 2023-07-06 21:23:21 -07:00
Aliaksandr Valialkin
fd6c2dd02e
docs/VictoriaLogs: change the structure of the docs in order to be more maintainable
The change is based on https://github.com/VictoriaMetrics/VictoriaMetrics/pull/4477
2023-07-06 21:22:59 -07:00
Aliaksandr Valialkin
1c39af56ab
app/victoria-logs: add ability to debug data ingestion by passing debug query arg to data ingestion API 2023-07-06 21:19:58 -07:00
Aliaksandr Valialkin
5fdf82657f
docs/VictoriaLogs: mention that VictoriaLogs supports multitenancy and out of order logs ingestion 2023-07-06 21:19:33 -07:00
Aliaksandr Valialkin
8b479e9122
docs/VictoriaLogs/LogsQL.md: typo fixes 2023-07-06 17:33:18 -07:00
Aliaksandr Valialkin
f4137a28ff
docs/VictoriaLogs/LogsQL.md: typo fixes 2023-07-06 17:33:18 -07:00
Aliaksandr Valialkin
cbfbaf37f3
docs/VictoriaLogs/LogsQL.md: typo fix 2023-07-06 17:32:51 -07:00
Aliaksandr Valialkin
c96cdc75c2
docs/VictoriaLogs/LogsQL.md: typo fix 2023-07-06 17:32:27 -07:00
Aliaksandr Valialkin
4d1c2b1722
docs/VictoriaLogs/LogsQL.md: typo fixes 2023-07-06 17:32:06 -07:00
Aliaksandr Valialkin
b66fbf8509
docs/VictoriaLogs/README.md: typo fixes 2023-07-06 17:31:45 -07:00
Aliaksandr Valialkin
137cbc5529
docs/VictoriaLogs/README.md: document how to run docker image for VictoriaLogs 2023-07-06 17:31:14 -07:00
Aliaksandr Valialkin
374890294e
app/victoria-logs: initial code release 2023-07-06 17:30:05 -07:00