mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-12-18 22:52:11 +01:00
Added docs for telegraf proxmox and vector (#6619)
### Describe Your Changes initial docs to implement #6618 more platforms can be added on this branch or on future commits. ### Checklist The following checks are **mandatory**: - [x ] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/).
This commit is contained in:
parent
6278c6ddf2
commit
22f269ad0f
71
docs/data-ingestion/Proxmox.md
Normal file
71
docs/data-ingestion/Proxmox.md
Normal file
@ -0,0 +1,71 @@
|
|||||||
|
---
|
||||||
|
title: Proxmox
|
||||||
|
weight: 1
|
||||||
|
sort: 1
|
||||||
|
menu:
|
||||||
|
docs:
|
||||||
|
identifier: "proxmox"
|
||||||
|
parent: "data-ingestion"
|
||||||
|
weight: 1
|
||||||
|
# sort: 1
|
||||||
|
aliases:
|
||||||
|
- /data-ingestion/proxmox.html
|
||||||
|
- /data-ingestion/Proxmox.html
|
||||||
|
---
|
||||||
|
|
||||||
|
# Proxmox Data Ingestion
|
||||||
|
Since Proxmox Virtual Environment(PVE) and Proxmox Backup Server(PBS) support sending data using the InfluxDB We can use the InfluxDB write support built into VictoriaMetrics
|
||||||
|
Currently PVE and PBS only support using an Authorization Token for authentication and does not support basic auth or a username and password.
|
||||||
|
|
||||||
|
## Proxmox Virtual Environment (PVE)
|
||||||
|
If want help Sending your data to Managed VictoriaMetrics check out [our blog](https://victoriametrics.com/blog/proxmox-monitoring-with-dbaas/).
|
||||||
|
|
||||||
|
1. Login to PVE as an administrator
|
||||||
|
2. Go to DataCenter > MetricServer > Add > InfluxDB
|
||||||
|
|
||||||
|
![PVE Metric Navigation](pve-nav.webp)
|
||||||
|
|
||||||
|
|
||||||
|
3. Set the parameters as follows:
|
||||||
|
- Name: VictoriaMetrics (can be changed to any string)
|
||||||
|
- Server: the hostname or IP of your VictoriaMetrics Instance
|
||||||
|
- Port: This will vary depending how you are sending data to VictoriaMetrics, but the defaults for all components are listed in the [data ingestion documentation](https://docs.victoriametrics.com/data-ingestion.html)
|
||||||
|
- Protocol: use HTTPS if you have TLS/SSL configured otherwise use HTTP
|
||||||
|
- Organization: leave empty since it doesn't get used
|
||||||
|
- Bucket: leave empty since it doesn't get used
|
||||||
|
- Token: your token from vmauth or leave blank if you don't have authentication enabled
|
||||||
|
- If you need to ignore TLS/SSL errors check the advanced box and uncheck the verify certificate box
|
||||||
|
4. Click the `Create` button
|
||||||
|
|
||||||
|
![PVE Metric Form](pve-form.webp)
|
||||||
|
|
||||||
|
5. Run `system_uptime{object="nodes"}` in vmui or in the explore view in Grafana to verify metrics from PVE are being sent to VictoriaMetrics.
|
||||||
|
You should see 1 time series per node in your PVE cluster.
|
||||||
|
|
||||||
|
## Proxmox Backup Server (PBS)
|
||||||
|
1. Login to PBS as an administrator
|
||||||
|
2. Go to Configuration > Metrics Server > Add > InfluxDB
|
||||||
|
|
||||||
|
|
||||||
|
![PBS Metric Navigation](pbs-nav.webp)
|
||||||
|
|
||||||
|
|
||||||
|
3. Set the parameters as follows:
|
||||||
|
- Name: VictoriaMetrics (can be set to any string)
|
||||||
|
- URL: http(s)://<ip_or_host>:<port>
|
||||||
|
- set the URL to https if you have TLS enabled and http if you do not
|
||||||
|
- Port: This will vary depending how you are sending data to VictoriaMetrics, but the defaults for all components are listed in the [data ingestion documentation](https://docs.victoriametrics.com/data-ingestion.html)
|
||||||
|
- Organization: leave empty since it doesn't get used
|
||||||
|
- Bucket: leave empty since it doesn't get used
|
||||||
|
- Token: your token from vmauth or leave blank if you don't have authentication enabled
|
||||||
|
4. Click the `Create` button
|
||||||
|
|
||||||
|
|
||||||
|
![PBS Metric Form](pbs-form.webp)
|
||||||
|
|
||||||
|
|
||||||
|
5. Run `cpustat_idle{object="host"}` in vmui or in the explore view in Grafana to verify metrics from PBS are being to VictoriaMetrics.
|
||||||
|
|
||||||
|
|
||||||
|
# References
|
||||||
|
- [Blog Post for configuring Managed VictoriaMetrics and Proxmox VE](https://victoriametrics.com/blog/proxmox-monitoring-with-dbaas/)
|
35
docs/data-ingestion/README.md
Normal file
35
docs/data-ingestion/README.md
Normal file
@ -0,0 +1,35 @@
|
|||||||
|
---
|
||||||
|
# sort: 14
|
||||||
|
title: Data Ingestion
|
||||||
|
weight: 0
|
||||||
|
menu:
|
||||||
|
docs:
|
||||||
|
parent: 'victoriametrics'
|
||||||
|
identifier: 'data-ingestion'
|
||||||
|
weight: 7
|
||||||
|
aliases:
|
||||||
|
- /data-ingestion.html
|
||||||
|
- /data-ingestion.html
|
||||||
|
- /dataingestion/
|
||||||
|
---
|
||||||
|
|
||||||
|
# Data Ingestion
|
||||||
|
In This Folder you will find instructions for sending data to VictoriaMetrics from a variety of platforms.
|
||||||
|
If your tool is not listed it is likely you can ingest your data into VictoriaMetrics using one of the protocols listed in our [Prominent features]({{< ref "/Single-server-VictoriaMetrics.md#prominent-features" >}}) section.
|
||||||
|
|
||||||
|
If you are unsure what port number to use when pushing data to VictoriaMetrics single node, vminsert, vmagent, and vmauth we have listed the default ports below.
|
||||||
|
|
||||||
|
- VictoriaMetrics Single: 8428
|
||||||
|
- vmagent: 8429
|
||||||
|
- vmauth: 8427
|
||||||
|
- vminsert: 8482
|
||||||
|
|
||||||
|
In the rest of the documentation we will assume you have configured your push endpoint to use TLS/SSL on port 443 so the urls in the rest of the documentation will look like `https://<victoriametrics_url>` instead of `http://<victoriametrics_url>:8428` for VictoriaMetrics single.
|
||||||
|
|
||||||
|
## Documented Collectors/Agents
|
||||||
|
* [Telegraf]({{< relref "Telegraf.md" >}})
|
||||||
|
* [Vector]({{< relref "Vector.md" >}})
|
||||||
|
|
||||||
|
## Supported Platforms
|
||||||
|
* [Proxmox Virtual Environment and Proxmox Backup Server]({{< relref "Proxmox.md" >}})
|
||||||
|
|
105
docs/data-ingestion/Telegraf.md
Normal file
105
docs/data-ingestion/Telegraf.md
Normal file
@ -0,0 +1,105 @@
|
|||||||
|
---
|
||||||
|
title: Telegraf
|
||||||
|
weight: 1
|
||||||
|
sort: 1
|
||||||
|
menu:
|
||||||
|
docs:
|
||||||
|
identifier: "telegraf"
|
||||||
|
parent: "data-ingestion"
|
||||||
|
weight: 1
|
||||||
|
# sort: 1
|
||||||
|
aliases:
|
||||||
|
- /data-ingestion/telegraf.html
|
||||||
|
- /data-ingestion/Telegraf.html
|
||||||
|
---
|
||||||
|
# Telegraf Setup
|
||||||
|
You will need to add the following output section to a Telegraf configuration file and reload Telegraf to enable shipping data from Telegraf to VictoriaMetrics.
|
||||||
|
All the options examples below can be combined to fit your use case
|
||||||
|
|
||||||
|
To avoid storing Passwords in configuration files you can store as a key value pair in `/etc/default/telegraf` on Linux as follows
|
||||||
|
```
|
||||||
|
victoriametrics_url="https://metrics.example.com"
|
||||||
|
victoriametrics_user="telegraf"
|
||||||
|
victoriametrics_password="password"
|
||||||
|
victoriametrics_token="my_token"
|
||||||
|
```
|
||||||
|
and they can be referenced in a Telegraf configuration file by prepending the variable name with `$` ex. `$victoriametrics_url` will be translated to `https://metrics.example.com` if it referenced in a Telegraf configuration using the values from `/etc/default/telegraf` in the values seen above.
|
||||||
|
Otherwise please replace the variables below to fit your setup.
|
||||||
|
|
||||||
|
If you want to mimic this behavior on windows please read [Influx Data's blog on storing variables in the registry](https://www.influxdata.com/blog/using-telegraf-on-windows/)
|
||||||
|
|
||||||
|
## Minimum Configuration with no Authentication
|
||||||
|
```toml
|
||||||
|
[[outputs.influxdb]]
|
||||||
|
urls = ["$victoriametrics_url"]
|
||||||
|
database = "victoriametrics"
|
||||||
|
skip_database_creation = true
|
||||||
|
exclude_retention_policy_tag = true
|
||||||
|
content_encoding = "gzip"
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## HTTP Basic Authentication (Username and Password)
|
||||||
|
This is the same as the minimum configuration, but adds the `username` and `password` options
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[[outputs.influxdb]]
|
||||||
|
urls = ["$victoriametrics_url"]
|
||||||
|
username = "$victoriametrics_user"
|
||||||
|
password = "$victoriametrics_password"
|
||||||
|
database = "victoriametrics"
|
||||||
|
skip_database_creation = true
|
||||||
|
exclude_retention_policy_tag = true
|
||||||
|
content_encoding = "gzip"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Bearer Authentication (Token)
|
||||||
|
|
||||||
|
This is the same as the minimum configuration but adds the authorization header
|
||||||
|
|
||||||
|
```
|
||||||
|
[[outputs.influxdb]]
|
||||||
|
urls = ["$victoriametrics_url"]
|
||||||
|
http_headers = {"Authorization" = "Bearer $victoriametrics_token"}
|
||||||
|
database = "victoriametrics"
|
||||||
|
skip_database_creation = true
|
||||||
|
exclude_retention_policy_tag = true
|
||||||
|
content_encoding = "gzip"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Route certain metrics
|
||||||
|
If you only want to route certain metrics to VictoriaMetrics use the `namepass` option with a comma separated listed of the measurements you wish to send to VictoriaMetrics.
|
||||||
|
|
||||||
|
```
|
||||||
|
[[outputs.influxdb]]
|
||||||
|
urls = ["$victoriametrics_url"]
|
||||||
|
username = "$victoriametrics_user"
|
||||||
|
password = "$victoriametrics_password"
|
||||||
|
database = "victoriametrics"
|
||||||
|
skip_database_creation = true
|
||||||
|
exclude_retention_policy_tag = true
|
||||||
|
content_encoding = "gzip"
|
||||||
|
namepass = ["cpu","disk","measurement1","measurement2"]
|
||||||
|
```
|
||||||
|
|
||||||
|
## Ignore TLS/SSL Certificate errors
|
||||||
|
This is the same as the minimum configuration but adds `insecure_skip_verify = true` to the configuration to ignore TLS certificate errors.
|
||||||
|
This is not recommended since it can allow sending metrics to a compromised site.
|
||||||
|
|
||||||
|
```
|
||||||
|
[[outputs.influxdb]]
|
||||||
|
urls = ["$victoriametrics_url"]
|
||||||
|
username = "$victoriametrics_user"
|
||||||
|
password = "$victoriametrics_password"
|
||||||
|
database = "victoriametrics"
|
||||||
|
skip_database_creation = true
|
||||||
|
exclude_retention_policy_tag = true
|
||||||
|
content_encoding = "gzip"
|
||||||
|
insecure_skip_verify = true
|
||||||
|
```
|
||||||
|
|
||||||
|
# References
|
||||||
|
- [Install Telegraf](https://docs.influxdata.com/telegraf/v1/install/)
|
||||||
|
- [InfluxDBv1 output for Telegraf](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/influxdb)
|
||||||
|
- [Storing Telegraf variables in the windows registry](https://www.influxdata.com/blog/using-telegraf-on-windows/)
|
||||||
|
- [Telegraf variables](https://docs.influxdata.com/telegraf/v1/configuration/#example-telegraf-environment-variables)
|
121
docs/data-ingestion/Vector.md
Normal file
121
docs/data-ingestion/Vector.md
Normal file
@ -0,0 +1,121 @@
|
|||||||
|
---
|
||||||
|
title: Vector
|
||||||
|
weight: 1
|
||||||
|
sort: 1
|
||||||
|
menu:
|
||||||
|
docs:
|
||||||
|
identifier: "Vector"
|
||||||
|
parent: "data-ingestion"
|
||||||
|
weight: 1
|
||||||
|
# sort: 1
|
||||||
|
aliases:
|
||||||
|
- /data-ingestion/Vector.html
|
||||||
|
- /data-ingestion/vector.html
|
||||||
|
---
|
||||||
|
# Vector
|
||||||
|
To Send data to Vector you need to configure with a Prometheus remote write sink and forward metrics to that sink from at least 1 source.
|
||||||
|
You will need to replace the values in `<>` with your to match your setup.
|
||||||
|
|
||||||
|
## Minimum Config
|
||||||
|
```yaml
|
||||||
|
sources:
|
||||||
|
host_metrics_source:
|
||||||
|
type: host_metrics
|
||||||
|
sinks:
|
||||||
|
victoriametrics_sink:
|
||||||
|
type: prometheus_remote_write
|
||||||
|
inputs:
|
||||||
|
- host_metrics_source
|
||||||
|
endpoint: "https://<victoriametrics_url>/api/v1/write"
|
||||||
|
healthcheck:
|
||||||
|
enabled: false
|
||||||
|
```
|
||||||
|
|
||||||
|
## Basic Authentication
|
||||||
|
|
||||||
|
This adds support for basic authentication by defining the auth strategy, user, and password fields:
|
||||||
|
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
sources:
|
||||||
|
host_metrics_source:
|
||||||
|
type: host_metrics
|
||||||
|
sinks:
|
||||||
|
victoriametrics_sink:
|
||||||
|
type: prometheus_remote_write
|
||||||
|
inputs:
|
||||||
|
- host_metrics_source
|
||||||
|
endpoint: "https://<victoriametrics_url>/api/v1/write"
|
||||||
|
auth:
|
||||||
|
strategy: "basic"
|
||||||
|
user: "<victoriametrics_user"
|
||||||
|
password: "<victoriametrics_password>"
|
||||||
|
healthcheck:
|
||||||
|
enabled: false
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
## Bearer / Token Authentication
|
||||||
|
|
||||||
|
This adds support for bearer/token authentication by defining the auth strategy and token fields:
|
||||||
|
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
sources:
|
||||||
|
host_metrics_source:
|
||||||
|
type: host_metrics
|
||||||
|
sinks:
|
||||||
|
victoriametrics_sink:
|
||||||
|
type: prometheus_remote_write
|
||||||
|
inputs:
|
||||||
|
- host_metrics_source
|
||||||
|
endpoint: "https://<victoriametrics_url>/api/v1/write"
|
||||||
|
auth:
|
||||||
|
strategy: "bearer"
|
||||||
|
token: "<victoriametrics_token>"
|
||||||
|
healthcheck:
|
||||||
|
enabled: false
|
||||||
|
```
|
||||||
|
|
||||||
|
## VictoriaMetrics and VictoriaLogs
|
||||||
|
|
||||||
|
This combines the Bearer Authentication section with the [VictoriaLogs docs for Vector](https://docs.victoriametrics.com/victorialogs/data-ingestion/vector/),
|
||||||
|
so you can send metrics and logs with 1 agent to multiple sources:
|
||||||
|
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
sources:
|
||||||
|
host_metrics_source:
|
||||||
|
type: host_metrics
|
||||||
|
journald_source:
|
||||||
|
type: journald
|
||||||
|
sinks:
|
||||||
|
victoriametrics_sink:
|
||||||
|
type: prometheus_remote_write
|
||||||
|
inputs:
|
||||||
|
- host_metrics_source
|
||||||
|
endpoint: "https://<victoriametrics_url>/api/v1/write"
|
||||||
|
auth:
|
||||||
|
strategy: "bearer"
|
||||||
|
token: "<token>"
|
||||||
|
healthcheck:
|
||||||
|
enabled: false
|
||||||
|
victorialogs_sink:
|
||||||
|
inputs:
|
||||||
|
- journald_source
|
||||||
|
type: elasticsearch
|
||||||
|
endpoints:
|
||||||
|
- "https://<victorialogs_url>/insert/elasticsearch/"
|
||||||
|
mode: bulk
|
||||||
|
api_version: "v8"
|
||||||
|
healthcheck:
|
||||||
|
enabled: false
|
||||||
|
query:
|
||||||
|
_msg_field: "message"
|
||||||
|
_time_field: "timestamp"
|
||||||
|
_stream_fields: "host,container_name"
|
||||||
|
```
|
||||||
|
|
||||||
|
# References
|
||||||
|
- [Vector documentation](https://vector.dev/docs/)
|
||||||
|
- [VictoriaLogs documenation for using vector]({{< ref "/victorialogs/data-ingestion/vector" >}})
|
BIN
docs/data-ingestion/pbs-form.webp
Normal file
BIN
docs/data-ingestion/pbs-form.webp
Normal file
Binary file not shown.
After Width: | Height: | Size: 9.2 KiB |
BIN
docs/data-ingestion/pbs-nav.webp
Normal file
BIN
docs/data-ingestion/pbs-nav.webp
Normal file
Binary file not shown.
After Width: | Height: | Size: 15 KiB |
BIN
docs/data-ingestion/pve-form.webp
Normal file
BIN
docs/data-ingestion/pve-form.webp
Normal file
Binary file not shown.
After Width: | Height: | Size: 14 KiB |
BIN
docs/data-ingestion/pve-nav.webp
Normal file
BIN
docs/data-ingestion/pve-nav.webp
Normal file
Binary file not shown.
After Width: | Height: | Size: 25 KiB |
Loading…
Reference in New Issue
Block a user