mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-12-18 14:40:26 +01:00
Added docs for telegraf proxmox and vector (#6619)
### Describe Your Changes initial docs to implement #6618 more platforms can be added on this branch or on future commits. ### Checklist The following checks are **mandatory**: - [x ] My change adheres [VictoriaMetrics contributing guidelines](https://docs.victoriametrics.com/contributing/).
This commit is contained in:
parent
6278c6ddf2
commit
22f269ad0f
71
docs/data-ingestion/Proxmox.md
Normal file
71
docs/data-ingestion/Proxmox.md
Normal file
@ -0,0 +1,71 @@
|
||||
---
|
||||
title: Proxmox
|
||||
weight: 1
|
||||
sort: 1
|
||||
menu:
|
||||
docs:
|
||||
identifier: "proxmox"
|
||||
parent: "data-ingestion"
|
||||
weight: 1
|
||||
# sort: 1
|
||||
aliases:
|
||||
- /data-ingestion/proxmox.html
|
||||
- /data-ingestion/Proxmox.html
|
||||
---
|
||||
|
||||
# Proxmox Data Ingestion
|
||||
Since Proxmox Virtual Environment(PVE) and Proxmox Backup Server(PBS) support sending data using the InfluxDB We can use the InfluxDB write support built into VictoriaMetrics
|
||||
Currently PVE and PBS only support using an Authorization Token for authentication and does not support basic auth or a username and password.
|
||||
|
||||
## Proxmox Virtual Environment (PVE)
|
||||
If want help Sending your data to Managed VictoriaMetrics check out [our blog](https://victoriametrics.com/blog/proxmox-monitoring-with-dbaas/).
|
||||
|
||||
1. Login to PVE as an administrator
|
||||
2. Go to DataCenter > MetricServer > Add > InfluxDB
|
||||
|
||||
![PVE Metric Navigation](pve-nav.webp)
|
||||
|
||||
|
||||
3. Set the parameters as follows:
|
||||
- Name: VictoriaMetrics (can be changed to any string)
|
||||
- Server: the hostname or IP of your VictoriaMetrics Instance
|
||||
- Port: This will vary depending how you are sending data to VictoriaMetrics, but the defaults for all components are listed in the [data ingestion documentation](https://docs.victoriametrics.com/data-ingestion.html)
|
||||
- Protocol: use HTTPS if you have TLS/SSL configured otherwise use HTTP
|
||||
- Organization: leave empty since it doesn't get used
|
||||
- Bucket: leave empty since it doesn't get used
|
||||
- Token: your token from vmauth or leave blank if you don't have authentication enabled
|
||||
- If you need to ignore TLS/SSL errors check the advanced box and uncheck the verify certificate box
|
||||
4. Click the `Create` button
|
||||
|
||||
![PVE Metric Form](pve-form.webp)
|
||||
|
||||
5. Run `system_uptime{object="nodes"}` in vmui or in the explore view in Grafana to verify metrics from PVE are being sent to VictoriaMetrics.
|
||||
You should see 1 time series per node in your PVE cluster.
|
||||
|
||||
## Proxmox Backup Server (PBS)
|
||||
1. Login to PBS as an administrator
|
||||
2. Go to Configuration > Metrics Server > Add > InfluxDB
|
||||
|
||||
|
||||
![PBS Metric Navigation](pbs-nav.webp)
|
||||
|
||||
|
||||
3. Set the parameters as follows:
|
||||
- Name: VictoriaMetrics (can be set to any string)
|
||||
- URL: http(s)://<ip_or_host>:<port>
|
||||
- set the URL to https if you have TLS enabled and http if you do not
|
||||
- Port: This will vary depending how you are sending data to VictoriaMetrics, but the defaults for all components are listed in the [data ingestion documentation](https://docs.victoriametrics.com/data-ingestion.html)
|
||||
- Organization: leave empty since it doesn't get used
|
||||
- Bucket: leave empty since it doesn't get used
|
||||
- Token: your token from vmauth or leave blank if you don't have authentication enabled
|
||||
4. Click the `Create` button
|
||||
|
||||
|
||||
![PBS Metric Form](pbs-form.webp)
|
||||
|
||||
|
||||
5. Run `cpustat_idle{object="host"}` in vmui or in the explore view in Grafana to verify metrics from PBS are being to VictoriaMetrics.
|
||||
|
||||
|
||||
# References
|
||||
- [Blog Post for configuring Managed VictoriaMetrics and Proxmox VE](https://victoriametrics.com/blog/proxmox-monitoring-with-dbaas/)
|
35
docs/data-ingestion/README.md
Normal file
35
docs/data-ingestion/README.md
Normal file
@ -0,0 +1,35 @@
|
||||
---
|
||||
# sort: 14
|
||||
title: Data Ingestion
|
||||
weight: 0
|
||||
menu:
|
||||
docs:
|
||||
parent: 'victoriametrics'
|
||||
identifier: 'data-ingestion'
|
||||
weight: 7
|
||||
aliases:
|
||||
- /data-ingestion.html
|
||||
- /data-ingestion.html
|
||||
- /dataingestion/
|
||||
---
|
||||
|
||||
# Data Ingestion
|
||||
In This Folder you will find instructions for sending data to VictoriaMetrics from a variety of platforms.
|
||||
If your tool is not listed it is likely you can ingest your data into VictoriaMetrics using one of the protocols listed in our [Prominent features]({{< ref "/Single-server-VictoriaMetrics.md#prominent-features" >}}) section.
|
||||
|
||||
If you are unsure what port number to use when pushing data to VictoriaMetrics single node, vminsert, vmagent, and vmauth we have listed the default ports below.
|
||||
|
||||
- VictoriaMetrics Single: 8428
|
||||
- vmagent: 8429
|
||||
- vmauth: 8427
|
||||
- vminsert: 8482
|
||||
|
||||
In the rest of the documentation we will assume you have configured your push endpoint to use TLS/SSL on port 443 so the urls in the rest of the documentation will look like `https://<victoriametrics_url>` instead of `http://<victoriametrics_url>:8428` for VictoriaMetrics single.
|
||||
|
||||
## Documented Collectors/Agents
|
||||
* [Telegraf]({{< relref "Telegraf.md" >}})
|
||||
* [Vector]({{< relref "Vector.md" >}})
|
||||
|
||||
## Supported Platforms
|
||||
* [Proxmox Virtual Environment and Proxmox Backup Server]({{< relref "Proxmox.md" >}})
|
||||
|
105
docs/data-ingestion/Telegraf.md
Normal file
105
docs/data-ingestion/Telegraf.md
Normal file
@ -0,0 +1,105 @@
|
||||
---
|
||||
title: Telegraf
|
||||
weight: 1
|
||||
sort: 1
|
||||
menu:
|
||||
docs:
|
||||
identifier: "telegraf"
|
||||
parent: "data-ingestion"
|
||||
weight: 1
|
||||
# sort: 1
|
||||
aliases:
|
||||
- /data-ingestion/telegraf.html
|
||||
- /data-ingestion/Telegraf.html
|
||||
---
|
||||
# Telegraf Setup
|
||||
You will need to add the following output section to a Telegraf configuration file and reload Telegraf to enable shipping data from Telegraf to VictoriaMetrics.
|
||||
All the options examples below can be combined to fit your use case
|
||||
|
||||
To avoid storing Passwords in configuration files you can store as a key value pair in `/etc/default/telegraf` on Linux as follows
|
||||
```
|
||||
victoriametrics_url="https://metrics.example.com"
|
||||
victoriametrics_user="telegraf"
|
||||
victoriametrics_password="password"
|
||||
victoriametrics_token="my_token"
|
||||
```
|
||||
and they can be referenced in a Telegraf configuration file by prepending the variable name with `$` ex. `$victoriametrics_url` will be translated to `https://metrics.example.com` if it referenced in a Telegraf configuration using the values from `/etc/default/telegraf` in the values seen above.
|
||||
Otherwise please replace the variables below to fit your setup.
|
||||
|
||||
If you want to mimic this behavior on windows please read [Influx Data's blog on storing variables in the registry](https://www.influxdata.com/blog/using-telegraf-on-windows/)
|
||||
|
||||
## Minimum Configuration with no Authentication
|
||||
```toml
|
||||
[[outputs.influxdb]]
|
||||
urls = ["$victoriametrics_url"]
|
||||
database = "victoriametrics"
|
||||
skip_database_creation = true
|
||||
exclude_retention_policy_tag = true
|
||||
content_encoding = "gzip"
|
||||
```
|
||||
|
||||
|
||||
## HTTP Basic Authentication (Username and Password)
|
||||
This is the same as the minimum configuration, but adds the `username` and `password` options
|
||||
|
||||
```toml
|
||||
[[outputs.influxdb]]
|
||||
urls = ["$victoriametrics_url"]
|
||||
username = "$victoriametrics_user"
|
||||
password = "$victoriametrics_password"
|
||||
database = "victoriametrics"
|
||||
skip_database_creation = true
|
||||
exclude_retention_policy_tag = true
|
||||
content_encoding = "gzip"
|
||||
```
|
||||
|
||||
## Bearer Authentication (Token)
|
||||
|
||||
This is the same as the minimum configuration but adds the authorization header
|
||||
|
||||
```
|
||||
[[outputs.influxdb]]
|
||||
urls = ["$victoriametrics_url"]
|
||||
http_headers = {"Authorization" = "Bearer $victoriametrics_token"}
|
||||
database = "victoriametrics"
|
||||
skip_database_creation = true
|
||||
exclude_retention_policy_tag = true
|
||||
content_encoding = "gzip"
|
||||
```
|
||||
|
||||
## Route certain metrics
|
||||
If you only want to route certain metrics to VictoriaMetrics use the `namepass` option with a comma separated listed of the measurements you wish to send to VictoriaMetrics.
|
||||
|
||||
```
|
||||
[[outputs.influxdb]]
|
||||
urls = ["$victoriametrics_url"]
|
||||
username = "$victoriametrics_user"
|
||||
password = "$victoriametrics_password"
|
||||
database = "victoriametrics"
|
||||
skip_database_creation = true
|
||||
exclude_retention_policy_tag = true
|
||||
content_encoding = "gzip"
|
||||
namepass = ["cpu","disk","measurement1","measurement2"]
|
||||
```
|
||||
|
||||
## Ignore TLS/SSL Certificate errors
|
||||
This is the same as the minimum configuration but adds `insecure_skip_verify = true` to the configuration to ignore TLS certificate errors.
|
||||
This is not recommended since it can allow sending metrics to a compromised site.
|
||||
|
||||
```
|
||||
[[outputs.influxdb]]
|
||||
urls = ["$victoriametrics_url"]
|
||||
username = "$victoriametrics_user"
|
||||
password = "$victoriametrics_password"
|
||||
database = "victoriametrics"
|
||||
skip_database_creation = true
|
||||
exclude_retention_policy_tag = true
|
||||
content_encoding = "gzip"
|
||||
insecure_skip_verify = true
|
||||
```
|
||||
|
||||
# References
|
||||
- [Install Telegraf](https://docs.influxdata.com/telegraf/v1/install/)
|
||||
- [InfluxDBv1 output for Telegraf](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/influxdb)
|
||||
- [Storing Telegraf variables in the windows registry](https://www.influxdata.com/blog/using-telegraf-on-windows/)
|
||||
- [Telegraf variables](https://docs.influxdata.com/telegraf/v1/configuration/#example-telegraf-environment-variables)
|
121
docs/data-ingestion/Vector.md
Normal file
121
docs/data-ingestion/Vector.md
Normal file
@ -0,0 +1,121 @@
|
||||
---
|
||||
title: Vector
|
||||
weight: 1
|
||||
sort: 1
|
||||
menu:
|
||||
docs:
|
||||
identifier: "Vector"
|
||||
parent: "data-ingestion"
|
||||
weight: 1
|
||||
# sort: 1
|
||||
aliases:
|
||||
- /data-ingestion/Vector.html
|
||||
- /data-ingestion/vector.html
|
||||
---
|
||||
# Vector
|
||||
To Send data to Vector you need to configure with a Prometheus remote write sink and forward metrics to that sink from at least 1 source.
|
||||
You will need to replace the values in `<>` with your to match your setup.
|
||||
|
||||
## Minimum Config
|
||||
```yaml
|
||||
sources:
|
||||
host_metrics_source:
|
||||
type: host_metrics
|
||||
sinks:
|
||||
victoriametrics_sink:
|
||||
type: prometheus_remote_write
|
||||
inputs:
|
||||
- host_metrics_source
|
||||
endpoint: "https://<victoriametrics_url>/api/v1/write"
|
||||
healthcheck:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
## Basic Authentication
|
||||
|
||||
This adds support for basic authentication by defining the auth strategy, user, and password fields:
|
||||
|
||||
|
||||
```yaml
|
||||
sources:
|
||||
host_metrics_source:
|
||||
type: host_metrics
|
||||
sinks:
|
||||
victoriametrics_sink:
|
||||
type: prometheus_remote_write
|
||||
inputs:
|
||||
- host_metrics_source
|
||||
endpoint: "https://<victoriametrics_url>/api/v1/write"
|
||||
auth:
|
||||
strategy: "basic"
|
||||
user: "<victoriametrics_user"
|
||||
password: "<victoriametrics_password>"
|
||||
healthcheck:
|
||||
enabled: false
|
||||
|
||||
```
|
||||
|
||||
## Bearer / Token Authentication
|
||||
|
||||
This adds support for bearer/token authentication by defining the auth strategy and token fields:
|
||||
|
||||
|
||||
```yaml
|
||||
sources:
|
||||
host_metrics_source:
|
||||
type: host_metrics
|
||||
sinks:
|
||||
victoriametrics_sink:
|
||||
type: prometheus_remote_write
|
||||
inputs:
|
||||
- host_metrics_source
|
||||
endpoint: "https://<victoriametrics_url>/api/v1/write"
|
||||
auth:
|
||||
strategy: "bearer"
|
||||
token: "<victoriametrics_token>"
|
||||
healthcheck:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
## VictoriaMetrics and VictoriaLogs
|
||||
|
||||
This combines the Bearer Authentication section with the [VictoriaLogs docs for Vector](https://docs.victoriametrics.com/victorialogs/data-ingestion/vector/),
|
||||
so you can send metrics and logs with 1 agent to multiple sources:
|
||||
|
||||
|
||||
```yaml
|
||||
sources:
|
||||
host_metrics_source:
|
||||
type: host_metrics
|
||||
journald_source:
|
||||
type: journald
|
||||
sinks:
|
||||
victoriametrics_sink:
|
||||
type: prometheus_remote_write
|
||||
inputs:
|
||||
- host_metrics_source
|
||||
endpoint: "https://<victoriametrics_url>/api/v1/write"
|
||||
auth:
|
||||
strategy: "bearer"
|
||||
token: "<token>"
|
||||
healthcheck:
|
||||
enabled: false
|
||||
victorialogs_sink:
|
||||
inputs:
|
||||
- journald_source
|
||||
type: elasticsearch
|
||||
endpoints:
|
||||
- "https://<victorialogs_url>/insert/elasticsearch/"
|
||||
mode: bulk
|
||||
api_version: "v8"
|
||||
healthcheck:
|
||||
enabled: false
|
||||
query:
|
||||
_msg_field: "message"
|
||||
_time_field: "timestamp"
|
||||
_stream_fields: "host,container_name"
|
||||
```
|
||||
|
||||
# References
|
||||
- [Vector documentation](https://vector.dev/docs/)
|
||||
- [VictoriaLogs documenation for using vector]({{< ref "/victorialogs/data-ingestion/vector" >}})
|
BIN
docs/data-ingestion/pbs-form.webp
Normal file
BIN
docs/data-ingestion/pbs-form.webp
Normal file
Binary file not shown.
After Width: | Height: | Size: 9.2 KiB |
BIN
docs/data-ingestion/pbs-nav.webp
Normal file
BIN
docs/data-ingestion/pbs-nav.webp
Normal file
Binary file not shown.
After Width: | Height: | Size: 15 KiB |
BIN
docs/data-ingestion/pve-form.webp
Normal file
BIN
docs/data-ingestion/pve-form.webp
Normal file
Binary file not shown.
After Width: | Height: | Size: 14 KiB |
BIN
docs/data-ingestion/pve-nav.webp
Normal file
BIN
docs/data-ingestion/pve-nav.webp
Normal file
Binary file not shown.
After Width: | Height: | Size: 25 KiB |
Loading…
Reference in New Issue
Block a user