VictoriaMetrics/docs/VictoriaLogs/data-ingestion/opentelemetry.md
Zhu Jiekun 143d137265
docs: [VictoriaLogs] OTel Collector elasticsearchexporter header note (#7074)
### Describe Your Changes

By default, the `elasticsearchexporter` in OTel Collector puts the log
message under a field other than `_msg` (e.g., `Body`). Without
specifying via an HTTP header, those logs may not be queried correctly.
See also:
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/6785.

This pull request updates the example configuration and notes for the
`elasticsearchexporter`.

### Checklist

The following checks are **mandatory**:

- [X] My change adheres [VictoriaMetrics contributing
guidelines](https://docs.victoriametrics.com/contributing/).

(cherry picked from commit fea4433362)
2024-09-24 16:58:16 +02:00

4.0 KiB

weight title disableToc menu aliases
4 OpenTelemetry setup true
docs
parent weight
victorialogs-data-ingestion 4
/VictoriaLogs/data-ingestion/OpenTelemetry.html

VictoriaLogs supports both client open-telemetry SDK and collector.

Client SDK

Specify EndpointURL for http-exporter builder.

Consider the following example for golang SDK:

 // Create the OTLP log exporter that sends logs to configured destination
 logExporter, err := otlploghttp.New(ctx,
  otlploghttp.WithEndpointURL("http://victorialogs:9428/insert/opentelemetry/v1/logs"),
 )

Optionally, stream fields could be defined via headers:

 // Create the OTLP log exporter that sends logs to configured destination
 logExporter, err := otlploghttp.New(ctx,
  otlploghttp.WithEndpointURL("http://victorialogs:9428/insert/opentelemetry/v1/logs"),
   otlploghttp.WithHeaders(map[string]string{"VL-Stream-Fields": "telemetry.sdk.language,severity"}),
 )

Given config defines 2 stream fields - severity and telemetry.sdk.language.

See also HTTP headers

Collector configuration

VictoriaLogs supports given below OpenTelemetry collector exporters:

Elasticsearch

exporters:
  elasticsearch:
    endpoints:
      - http://victorialogs:9428/insert/elasticsearch
    headers:
      VL-Msg-Field: "Body" # Optional.
receivers:
  filelog:
    include: [/tmp/logs/*.log]
    resource:
      region: us-east-1
service:
  pipelines:
    logs:
      receivers: [filelog]
      exporters: [elasticsearch]

Please note that every ingested log entry must contain at least a _msg field with the actual log message. By default, the Elasticsearch exporter may place the log message in the Body field. In this case, you can specify the field mapping via:

    headers:
      VL-Msg-Field: "Body"

VictoriaLogs also support specify AccountID, ProjectID, log timestamp and other fields via HTTP headers.

Loki

exporters:
  loki:
    endpoint: http://victorialogs:9428/insert/loki/api/v1/push
receivers:
  filelog:
    include: [/tmp/logs/*.log]
    resource:
      region: us-east-1
service:
  pipelines:
    logs:
      receivers: [filelog]
      exporters: [loki]

OpenTelemetry

Specify logs endpoint for OTLP/HTTP exporter in configuration file for sending the collected logs to VictoriaLogs:

exporters:
  otlphttp:
    logs_endpoint: http://localhost:9428/insert/opentelemetry/v1/logs

Optionally, stream fields could be defined via headers:

exporters:
  otlphttp:
    logs_endpoint: http://localhost:9428/insert/opentelemetry/v1/logs
    headers:
      VL-Stream-Fields: telemetry.sdk.language,severity

See also HTTP headers

Substitute localhost:9428 address inside exporters.otlphttp.logs_endpoint with the real address of VictoriaLogs.

The ingested log entries can be queried according to these docs.

See also: