mirror of
https://github.com/VictoriaMetrics/VictoriaMetrics.git
synced 2024-11-23 12:31:07 +01:00
hide deprecated docs from menu (#5095)
This commit is contained in:
Alexander Marshalov
GitHub
parent
e330ab7ad9
commit
7b35eaa853
@ -1,5 +1,5 @@
|
|||||||
---
|
---
|
||||||
sort: 100
|
sort: -1 # hide page from menu
|
||||||
weight: 100
|
weight: 100
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@ -37,6 +37,11 @@
|
|||||||
color: #b35cec !important;
|
color: #b35cec !important;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// hidden in menu pages
|
||||||
|
[data-sort="-1"] {
|
||||||
|
display: none;
|
||||||
|
}
|
||||||
|
|
||||||
// Content
|
// Content
|
||||||
.markdown-body h1,
|
.markdown-body h1,
|
||||||
.markdown-body h2,
|
.markdown-body h2,
|
||||||
|
|||||||
@ -1,86 +1,9 @@
|
|||||||
---
|
---
|
||||||
sort: 1
|
sort: -1 # hide page from menu
|
||||||
weight: 1
|
|
||||||
title: VictoriaMetrics Operator Overview
|
|
||||||
menu:
|
|
||||||
docs:
|
|
||||||
parent: "operator"
|
|
||||||
weight: 1
|
|
||||||
aliases:
|
aliases:
|
||||||
- /operator/VictoriaMetrics-Operator.html
|
- /operator/VictoriaMetrics-Operator.html
|
||||||
---
|
---
|
||||||
|
|
||||||
# VictoriaMetrics Operator Overview
|
This page is deprecated.
|
||||||
|
|
||||||
## Overview
|
Please see [Operator page](./README.md) instead.
|
||||||
|
|
||||||
Design and implementation inspired by [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator). It's great a tool for managing monitoring configuration of your applications. VictoriaMetrics operator has api capability with it.
|
|
||||||
So you can use familiar CRD objects: `ServiceMonitor`, `PodMonitor`, `PrometheusRule` and `Probe`. Or you can use VictoriaMetrics CRDs:
|
|
||||||
- `VMServiceScrape` - defines scraping metrics configuration from pods backed by services.
|
|
||||||
- `VMPodScrape` - defines scraping metrics configuration from pods.
|
|
||||||
- `VMRule` - defines alerting or recording rules.
|
|
||||||
- `VMProbe` - defines a probing configuration for targets with blackbox exporter.
|
|
||||||
|
|
||||||
Besides, operator allows you to manage VictoriaMetrics applications inside kubernetes cluster and simplifies this process [quick-start](https://docs.victoriametrics.com/operator/quick-start.html)
|
|
||||||
With CRD (Custom Resource Definition) you can define application configuration and apply it to your cluster [crd-objects](https://docs.victoriametrics.com/operator/api.html).
|
|
||||||
|
|
||||||
Operator simplifies VictoriaMetrics cluster installation, upgrading and managing.
|
|
||||||
|
|
||||||
It has integration with VictoriaMetrics `vmbackupmanager` - advanced tools for making backups. Check backup [docs](https://docs.victoriametrics.com/operator/backups.html)
|
|
||||||
|
|
||||||
## Use cases
|
|
||||||
|
|
||||||
For kubernetes-cluster administrators, it simplifies installation, configuration and management for `VictoriaMetrics` application. The main feature of operator is its ability to delegate the configuration of applications monitoring to the end-users.
|
|
||||||
|
|
||||||
For applications developers, its great possibility for managing observability of applications. You can define metrics scraping and alerting configuration for your application and manage it with an application deployment process. Just define app_deployment.yaml, app_vmpodscrape.yaml and app_vmrule.yaml. That's it, you can apply it to a kubernetes cluster. Check [quick-start](/operator/quick-start.html) for an example.
|
|
||||||
|
|
||||||
## Operator vs helm-chart
|
|
||||||
|
|
||||||
VictoriaMetrics provides [helm charts](https://github.com/VictoriaMetrics/helm-charts). Operator makes the same, simplifies it and provides advanced features.
|
|
||||||
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
Operator configured by env variables, list of it can be found at [link](https://docs.victoriametrics.com/operator/vars.html)
|
|
||||||
|
|
||||||
It defines default configuration options, like images for components, timeouts, features.
|
|
||||||
|
|
||||||
|
|
||||||
## Kubernetes' compatibility versions
|
|
||||||
|
|
||||||
Operator tested at kubernetes versions from 1.16 to 1.22.
|
|
||||||
|
|
||||||
For clusters version below 1.16 you must use legacy CRDs from [path](https://github.com/VictoriaMetrics/operator/tree/master/config/crd/legacy)
|
|
||||||
and disable CRD controller with flag: `--controller.disableCRDOwnership=true`
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
- cannot apply crd at kubernetes 1.18 + version and kubectl reports error:
|
|
||||||
```console
|
|
||||||
Error from server (Invalid): error when creating "release/crds/crd.yaml": CustomResourceDefinition.apiextensions.k8s.io "vmalertmanagers.operator.victoriametrics.com" is invalid: [spec.validation.openAPIV3Schema.properties[spec].properties[initContainers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property]
|
|
||||||
Error from server (Invalid): error when creating "release/crds/crd.yaml": CustomResourceDefinition.apiextensions.k8s.io "vmalerts.operator.victoriametrics.com" is invalid: [
|
|
||||||
```
|
|
||||||
upgrade to the latest release version. There is a bug with kubernetes objects at the early releases.
|
|
||||||
|
|
||||||
|
|
||||||
## Development
|
|
||||||
|
|
||||||
- [operator-sdk](https://github.com/operator-framework/operator-sdk) version v1.0.0+
|
|
||||||
- golang 1.15 +
|
|
||||||
- minikube or kind
|
|
||||||
|
|
||||||
start:
|
|
||||||
```console
|
|
||||||
make run
|
|
||||||
```
|
|
||||||
|
|
||||||
for test execution run:
|
|
||||||
```console
|
|
||||||
#unit tests
|
|
||||||
|
|
||||||
make test
|
|
||||||
|
|
||||||
# you need minikube or kind for e2e, do not run it on live cluster
|
|
||||||
#e2e tests with local binary
|
|
||||||
make e2e-local
|
|
||||||
```
|
|
||||||
|
|||||||
@ -1,89 +1,10 @@
|
|||||||
---
|
---
|
||||||
sort: 2
|
sort: -1 # hide page from menu
|
||||||
weight: 2
|
|
||||||
title: Additional Scrape Configuration
|
|
||||||
menu:
|
|
||||||
docs:
|
|
||||||
parent: "operator"
|
|
||||||
weight: 2
|
|
||||||
aliases:
|
aliases:
|
||||||
- /operator/additional-scrape.html
|
- /operator/additional-scrape.html
|
||||||
---
|
---
|
||||||
|
|
||||||
# Additional Scrape Configuration
|
This page is deprecated.
|
||||||
|
|
||||||
AdditionalScrapeConfigs is an additional way to add scrape targets in VMAgent CRD.
|
|
||||||
There are two options for adding targets into VMAgent: inline configuration into CRD or defining it as a Kubernetes Secret.
|
|
||||||
|
|
||||||
No validation happens during the creation of configuration. However, you must validate job specs, and it must follow job spec configuration.
|
|
||||||
Please check [scrape_configs documentation](https://docs.victoriametrics.com/sd_configs.html#scrape_configs) as references.
|
|
||||||
|
|
||||||
## Inline Additional Scrape Configuration in VMAgent CRD
|
|
||||||
|
|
||||||
You need to add scrape configuration directly to the vmagent spec.inlineScrapeConfig. It is raw text in YAML format.
|
|
||||||
See example below
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
cat <<EOF | kubectl apply -f -
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMAgent
|
|
||||||
metadata:
|
|
||||||
name: example-vmagent
|
|
||||||
spec:
|
|
||||||
serviceScrapeSelector: {}
|
|
||||||
replicas: 1
|
|
||||||
serviceAccountName: vmagent
|
|
||||||
inlineScrapeConfig: |
|
|
||||||
- job_name: "prometheus"
|
|
||||||
static_configs:
|
|
||||||
- targets: ["localhost:9090"]
|
|
||||||
remoteWrite:
|
|
||||||
- url: "http://vmagent-example-vmsingle.default.svc:8429/api/v1/write"
|
|
||||||
EOF
|
|
||||||
```
|
|
||||||
|
|
||||||
**Note**: Do not use passwords and tokens with inlineScrapeConfig use Secret instead of
|
|
||||||
|
|
||||||
|
|
||||||
## Define Additional Scrape Configuration as a Kubernetes Secret
|
|
||||||
|
|
||||||
You need to define Kubernetes Secret with a key.
|
|
||||||
|
|
||||||
The key is `prometheus-additional.yaml` in the example below
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
cat <<EOF | kubectl apply -f -
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Secret
|
|
||||||
metadata:
|
|
||||||
name: additional-scrape-configs
|
|
||||||
stringData:
|
|
||||||
prometheus-additional.yaml: |
|
|
||||||
- job_name: "prometheus"
|
|
||||||
static_configs:
|
|
||||||
- targets: ["localhost:9090"]
|
|
||||||
EOF
|
|
||||||
```
|
|
||||||
|
|
||||||
After that, you need to specify the secret's name and key in VMAgent CRD in `additionalScrapeConfigs` section
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
cat <<EOF | kubectl apply -f -
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMAgent
|
|
||||||
metadata:
|
|
||||||
name: example-vmagent
|
|
||||||
spec:
|
|
||||||
serviceScrapeSelector: {}
|
|
||||||
replicas: 1
|
|
||||||
serviceAccountName: vmagent
|
|
||||||
additionalScrapeConfigs:
|
|
||||||
name: additional-scrape-configs
|
|
||||||
key: prometheus-additional.yaml
|
|
||||||
remoteWrite:
|
|
||||||
- url: "http://vmagent-example-vmsingle.default.svc:8429/api/v1/write"
|
|
||||||
EOF
|
|
||||||
```
|
|
||||||
|
|
||||||
**Note**: You can specify only one Secret in the VMAgent CRD configuration so use it for all additional scrape configurations.
|
|
||||||
|
|
||||||
|
Please see [Additional scrape configuration](./resources/vmagent.md#additional-scrape-configuration)
|
||||||
|
section in VMAgent docs instead.
|
||||||
|
|||||||
@ -1,155 +1,10 @@
|
|||||||
---
|
---
|
||||||
sort: 5
|
sort: -1 # hide page from menu
|
||||||
weight: 5
|
|
||||||
title: Backups
|
|
||||||
menu:
|
|
||||||
docs:
|
|
||||||
parent: "operator"
|
|
||||||
weight: 5
|
|
||||||
aliases:
|
aliases:
|
||||||
- /operator/backups.html
|
- /operator/backups.html
|
||||||
---
|
---
|
||||||
|
|
||||||
# Backups
|
This page is deprecated.
|
||||||
|
|
||||||
## vmbackupmanager
|
Please see [Backup automation for VMSingle](./resources/vmsingle.md#backup-automation) or
|
||||||
|
[Backup automation for VMCluster](./resources/vmcluster.md#backup-automation) instead.
|
||||||
You can check vmbackupmanager [documentation](https://docs.victoriametrics.com/vmbackupmanager.html). It contains a description of the service and its features. This documentation covers vmbackumanager integration in vmoperator
|
|
||||||
|
|
||||||
### vmbackupmanager is a part of VictoriaMetrics Enterprise offer
|
|
||||||
|
|
||||||
*Before using it, you must have signed contract and accept ESA https://victoriametrics.com/legal/esa/*
|
|
||||||
|
|
||||||
## Usage examples
|
|
||||||
|
|
||||||
`VMSingle` and `VMCluster` has built-in backup configuration, it uses `vmbackupmanager` - proprietary tool for backups.
|
|
||||||
It supports incremental backups (hourly, daily, weekly, monthly) with popular object storages (aws s3, google cloud storage).
|
|
||||||
|
|
||||||
The configuration example is the following:
|
|
||||||
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Secret
|
|
||||||
metadata:
|
|
||||||
name: remote-storage-keys
|
|
||||||
type: Opaque
|
|
||||||
stringData:
|
|
||||||
credentials: |-
|
|
||||||
[default]
|
|
||||||
aws_access_key_id = your_access_key_id
|
|
||||||
aws_secret_access_key = your_secret_access_key
|
|
||||||
---
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMSingle
|
|
||||||
metadata:
|
|
||||||
name: example-vmsingle
|
|
||||||
spec:
|
|
||||||
# Add fields here
|
|
||||||
retentionPeriod: "1"
|
|
||||||
vmBackup:
|
|
||||||
# This is Enterprise Package feature you need to have signed contract to use it
|
|
||||||
# and accept the ESA https://victoriametrics.com/legal/esa/
|
|
||||||
acceptEULA: true
|
|
||||||
destination: "s3://your_bucket/folder"
|
|
||||||
credentialsSecret:
|
|
||||||
name: remote-storage-keys
|
|
||||||
key: credentials
|
|
||||||
---
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMCluster
|
|
||||||
metadata:
|
|
||||||
name: example-vmcluster-persistent
|
|
||||||
spec:
|
|
||||||
retentionPeriod: "4"
|
|
||||||
replicationFactor: 2
|
|
||||||
vmstorage:
|
|
||||||
replicaCount: 2
|
|
||||||
vmBackup:
|
|
||||||
# This is Enterprise Package feature you need to have signed contract to use it
|
|
||||||
# and accept the ESA https://victoriametrics.com/legal/esa/
|
|
||||||
acceptEULA: true
|
|
||||||
destination: "s3://your_bucket/folder"
|
|
||||||
credentialsSecret:
|
|
||||||
name: remote-storage-keys
|
|
||||||
key: credentials
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
NOTE: for cluster version operator adds suffix for destination: `"s3://your_bucket/folder"`, it becomes `"s3://your_bucket/folder/$(POD_NAME)"`.
|
|
||||||
It's needed to make consistent backups for each storage node.
|
|
||||||
|
|
||||||
You can read more about backup configuration options and mechanics [here](https://docs.victoriametrics.com/vmbackup.html)
|
|
||||||
|
|
||||||
Possible configuration options for backup crd can be found at [link](https://docs.victoriametrics.com/operator/api.html#vmbackup)
|
|
||||||
|
|
||||||
|
|
||||||
## Restoring backups
|
|
||||||
|
|
||||||
There are several ways to restore with [vmrestore](https://docs.victoriametrics.com/vmrestore.html) or [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager.html).
|
|
||||||
|
|
||||||
|
|
||||||
### Manually mounting disk
|
|
||||||
You have to stop `VMSingle` by scaling it replicas to zero and manually restore data to the database directory.
|
|
||||||
|
|
||||||
Steps:
|
|
||||||
1. edit `VMSingle` CRD, set replicaCount: 0
|
|
||||||
1. wait until database stops
|
|
||||||
1. ssh to some server, where you can mount `VMSingle` disk and mount it manually
|
|
||||||
1. restore files with `vmrestore`
|
|
||||||
1. umount disk
|
|
||||||
1. edit `VMSingle` CRD, set replicaCount: 1
|
|
||||||
1. wait database start
|
|
||||||
|
|
||||||
### Using VMRestore init container
|
|
||||||
|
|
||||||
1. add init container with vmrestore command to `VMSingle` CRD, example:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMSingle
|
|
||||||
metadata:
|
|
||||||
name: vmsingle-restored
|
|
||||||
namespace: monitoring-system
|
|
||||||
spec:
|
|
||||||
initContainers:
|
|
||||||
- name: vmrestore
|
|
||||||
image: victoriametrics/vmrestore:latest
|
|
||||||
volumeMounts:
|
|
||||||
- mountPath: /victoria-metrics-data
|
|
||||||
name: data
|
|
||||||
- mountPath: /etc/vm/creds
|
|
||||||
name: secret-remote-storage-keys
|
|
||||||
readOnly: true
|
|
||||||
args:
|
|
||||||
- -storageDataPath=/victoria-metrics-data
|
|
||||||
- -src=s3://your_bucket/folder/latest
|
|
||||||
- -credsFilePath=/etc/vm/creds/credentials
|
|
||||||
vmBackup:
|
|
||||||
# This is Enterprise Package feature you need to have signed contract to use it
|
|
||||||
# and accept the ESA https://victoriametrics.com/legal/esa/
|
|
||||||
acceptEULA: true
|
|
||||||
destination: "s3://your_bucket/folder"
|
|
||||||
extraArgs:
|
|
||||||
runOnStart: "true"
|
|
||||||
image:
|
|
||||||
repository: victoriametrics/vmbackupmanager
|
|
||||||
tag: v1.83.0-enterprise
|
|
||||||
credentialsSecret:
|
|
||||||
name: remote-storage-keys
|
|
||||||
key: credentials
|
|
||||||
```
|
|
||||||
1. apply it, and db will be restored from s3
|
|
||||||
|
|
||||||
1. remove initContainers and apply crd.
|
|
||||||
|
|
||||||
Note that using `VMRestore` will require adjusting `src` for each pod because restore will be handled per-pod.
|
|
||||||
|
|
||||||
### Using VMBackupmanager init container
|
|
||||||
|
|
||||||
Using VMBackupmanager restore in Kubernetes environment is described [here](https://docs.victoriametrics.com/vmbackupmanager.html#how-to-restore-in-kubernetes).
|
|
||||||
|
|
||||||
Advantages of using `VMBackupmanager` include:
|
|
||||||
- automatic adjustment of `src` for each pod when backup is requested
|
|
||||||
- graceful handling of case when no restore is required - `VMBackupmanager` will exit with successful status code and won't prevent pod from starting
|
|
||||||
|
|||||||
@ -1,48 +1,10 @@
|
|||||||
---
|
---
|
||||||
sort: 14
|
sort: -1 # hide page from menu
|
||||||
weight: 14
|
|
||||||
title: Configuration synchronization
|
|
||||||
menu:
|
|
||||||
docs:
|
|
||||||
parent: "operator"
|
|
||||||
weight: 14
|
|
||||||
aliases:
|
aliases:
|
||||||
- /operator/configuration_syncronization.html
|
- /operator/configuration_syncronization.html
|
||||||
---
|
---
|
||||||
|
|
||||||
# Configuration synchronization
|
This page is deprecated.
|
||||||
|
|
||||||
## Basic concepts
|
Please see [Configuration synchronization](./resources/README.md#configuration-synchronization)
|
||||||
VictoriaMetrics applications, like many other applications with configuration file deployed at Kubernetes, uses `ConfigMaps` and `Secrets` for configuration files.
|
in Resources section instead.
|
||||||
Usually, it's out of application scope to watch for configuration on-disk changes.
|
|
||||||
Applications reload their configuration by a signal from a user or some other tool, that knows how to watch for updates.
|
|
||||||
At Kubernetes, the most popular design for this case is a sidecar container, that watches for configuration file changes and sends an HTTP request to the application.
|
|
||||||
|
|
||||||
`Configmap` or `Secret` that mounted at `Pod` holds a copy of its content.
|
|
||||||
Kubernetes component `kubelet` is responsible for content synchronization between an object at Kubernetes API and a file served on disk.
|
|
||||||
It's not efficient to sync its content immediately, and `kubelet` eventually synchronizes it. There is a configuration option, that controls this period.
|
|
||||||
|
|
||||||
That's why, `VMAgent`, `VMAlert` and other applications managed by operator don't receive changes immediately. It usually takes 1-2 min, before content will be updated.
|
|
||||||
|
|
||||||
It may trigger errors when an application was deleted, but `VMAgent` still tries to scrape it.
|
|
||||||
|
|
||||||
## Possible mitigations
|
|
||||||
|
|
||||||
The naive solution for this case decrease the synchronization period. But it configures globally and may be hard for operator users.
|
|
||||||
|
|
||||||
That's why operator uses a few hacks.
|
|
||||||
|
|
||||||
For `ConfigMap` updates, operator changes annotation with a time of `Configmap` content update. It triggers `ConfigMap`'s content synchronization by kubelet immediately.
|
|
||||||
It's the case for `VMAlert`, it uses `ConfigMap` as a configuration source.
|
|
||||||
|
|
||||||
For `Secret` it doesn't work. And operator offers its implementation for side-car container. It can be configured with env variable for operator:
|
|
||||||
```
|
|
||||||
- name: VM_USECUSTOMCONFIGRELOADER
|
|
||||||
value: "true"
|
|
||||||
```
|
|
||||||
If it's defined, operator uses own [config-reloader](https://github.com/VictoriaMetrics/operator/tree/master/internal/config-reloader) instead of [prometheus-config-reload](https://github.com/prometheus-operator/prometheus-operator/tree/main/cmd/prometheus-config-reloader).
|
|
||||||
|
|
||||||
It watches corresponding `Secret` for changes with Kubernetes API watch call and writes content into emptyDir.
|
|
||||||
This emptyDir shared with the application.
|
|
||||||
In case of content changes, `config-reloader` sends HTTP requests to the application.
|
|
||||||
It greatly reduces the time for configuration synchronization.
|
|
||||||
|
|||||||
@ -1,241 +1,9 @@
|
|||||||
---
|
---
|
||||||
sort: 6
|
sort: -1 # hide page from menu
|
||||||
weight: 6
|
|
||||||
title: Design
|
|
||||||
menu:
|
|
||||||
docs:
|
|
||||||
parent: "operator"
|
|
||||||
weight: 6
|
|
||||||
aliases:
|
aliases:
|
||||||
- /operator/design.html
|
- /operator/design.html
|
||||||
---
|
---
|
||||||
|
|
||||||
# Design
|
This page is deprecated.
|
||||||
|
|
||||||
This document describes the design and interaction between the custom resource definitions (CRD) that the Victoria
|
Please see [Custom Resources](./resources/README.md) instead.
|
||||||
Metrics Operator introduces.
|
|
||||||
|
|
||||||
Operator introduces the following custom resources:
|
|
||||||
|
|
||||||
* [VMSingle](#vmsingle)
|
|
||||||
* [VMCluster](#vmcluster)
|
|
||||||
* [VMAgent](#vmagent)
|
|
||||||
* [VMAlert](#vmalert)
|
|
||||||
* [VMServiceScrape](#vmservicescrape)
|
|
||||||
* [VMPodScrape](#vmpodscrape)
|
|
||||||
* [VMAlertmanager](#vmalertmanager)
|
|
||||||
* [VMAlertmanagerConfig](#vmalertmanagerconfig)
|
|
||||||
* [VMRule](#vmrule)
|
|
||||||
* [VMPrometheusConverter](#vmprometheusconverter)
|
|
||||||
* [VMProbe](#vmprobe)
|
|
||||||
* [VMNodeScrape](#vmnodescrape)
|
|
||||||
* [VMStaticScrape](#vmstaticscrape)
|
|
||||||
* [VMAuth](#vmauth)
|
|
||||||
* [VMUser](#vmuser)
|
|
||||||
|
|
||||||
## VMSingle
|
|
||||||
|
|
||||||
The `VMSingle` CRD declaratively defines a [single-node VM](https://github.com/VictoriaMetrics/VictoriaMetrics)
|
|
||||||
installation to run in a Kubernetes cluster.
|
|
||||||
|
|
||||||
For each `VMSingle` resource, the Operator deploys a properly configured `Deployment` in the same namespace.
|
|
||||||
The VMSingle `Pod`s are configured to mount an empty dir or `PersistentVolumeClaimSpec` for storing data.
|
|
||||||
Deployment update strategy set to [recreate](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#recreate-deployment).
|
|
||||||
No more than one replica allowed.
|
|
||||||
|
|
||||||
For each `VMSingle` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
|
|
||||||
name `<VMSingle-name>`.
|
|
||||||
|
|
||||||
## VMCluster
|
|
||||||
|
|
||||||
The `VMCluster` CRD defines a [cluster version VM](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster).
|
|
||||||
|
|
||||||
For each `VMCluster` resource, the Operator creates `VMStorage` as `StatefulSet`, `VMSelect` as `StatefulSet` and `VMInsert`
|
|
||||||
as deployment. For `VMStorage` and `VMSelect` headless services are created. `VMInsert` is created as service with clusterIP.
|
|
||||||
|
|
||||||
There is a strict order for these objects creation and reconciliation:
|
|
||||||
1. `VMStorage` is synced - the Operator waits until all its pods are ready;
|
|
||||||
1. Then it syncs `VMSelect` with the same manner;
|
|
||||||
1. `VMInsert` is the last object to sync.
|
|
||||||
|
|
||||||
All statefulsets are created with [OnDelete](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#on-delete)
|
|
||||||
update type. It allows to manually manage the rolling update process for Operator by deleting pods one by one and waiting
|
|
||||||
for the ready status.
|
|
||||||
|
|
||||||
Rolling update process may be configured by the operator env variables.
|
|
||||||
The most important is `VM_PODWAITREADYTIMEOUT=80s` - it controls how long to wait for pod's ready status.
|
|
||||||
|
|
||||||
## VMAgent
|
|
||||||
|
|
||||||
The `VMAgent` CRD declaratively defines a desired [VMAgent](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmagent)
|
|
||||||
setup to run in a Kubernetes cluster.
|
|
||||||
|
|
||||||
For each `VMAgent` resource Operator deploys a properly configured `Deployment` in the same namespace.
|
|
||||||
The VMAgent `Pod`s are configured to mount a `Secret` prefixed with `<VMAgent-name>` containing the configuration
|
|
||||||
for VMAgent.
|
|
||||||
|
|
||||||
For each `VMAgent` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
|
|
||||||
name `<VMAgent-name>`.
|
|
||||||
|
|
||||||
The CRD specifies which `VMServiceScrape` should be covered by the deployed VMAgent instances based on label selection.
|
|
||||||
The Operator then generates a configuration based on the included `VMServiceScrape`s and updates the `Secret` which
|
|
||||||
contains the configuration. It continuously does so for all changes that are made to the `VMServiceScrape`s or the
|
|
||||||
`VMAgent` resource itself.
|
|
||||||
|
|
||||||
If no selection of `VMServiceScrape`s is provided - Operator leaves management of the `Secret` to the user,
|
|
||||||
so user can set custom configuration while still benefiting from the Operator's capabilities of managing VMAgent setups.
|
|
||||||
|
|
||||||
## VMAlert
|
|
||||||
|
|
||||||
The `VMAlert` CRD declaratively defines a desired [VMAlert](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmalert)
|
|
||||||
setup to run in a Kubernetes cluster.
|
|
||||||
|
|
||||||
For each `VMAlert` resource, the Operator deploys a properly configured `Deployment` in the same namespace.
|
|
||||||
The VMAlert `Pod`s are configured to mount a list of `Configmaps` prefixed with `<VMAlert-name>-number` containing
|
|
||||||
the configuration for alerting rules.
|
|
||||||
|
|
||||||
For each `VMAlert` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
|
|
||||||
name `<VMAlert-name>`.
|
|
||||||
|
|
||||||
The CRD specifies which `VMRule`s should be covered by the deployed VMAlert instances based on label selection.
|
|
||||||
The Operator then generates a configuration based on the included `VMRule`s and updates the `Configmaps` containing
|
|
||||||
the configuration. It continuously does so for all changes that are made to `VMRule`s or to the `VMAlert` resource itself.
|
|
||||||
|
|
||||||
Alerting rules are filtered by selector `ruleNamespaceSelector` in `VMAlert` CRD definition. For selecting rules from all
|
|
||||||
namespaces you must specify it to empty value:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
spec:
|
|
||||||
ruleNamespaceSelector: {}
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
## VMServiceScrape
|
|
||||||
|
|
||||||
The `VMServiceScrape` CRD allows to define a dynamic set of services for monitoring. Services
|
|
||||||
and scraping configurations can be matched via label selections. This allows an organization to introduce conventions
|
|
||||||
for how metrics should be exposed. Following these conventions new services will be discovered automatically without
|
|
||||||
need to reconfigure.
|
|
||||||
|
|
||||||
Monitoring configuration based on `discoveryRole` setting. By default, `endpoints` is used to get objects from kubernetes api.
|
|
||||||
It's also possible to use `discoveryRole: service` or `discoveryRole: endpointslices`
|
|
||||||
|
|
||||||
`Endpoints` objects are essentially lists of IP addresses.
|
|
||||||
Typically, `Endpoints` objects are populated by `Service` object. `Service` object discovers `Pod`s by a label
|
|
||||||
selector and adds those to the `Endpoints` object.
|
|
||||||
|
|
||||||
A `Service` may expose one or more service ports backed by a list of one or multiple endpoints pointing to
|
|
||||||
specific `Pod`s. The same reflected in the respective `Endpoints` object as well.
|
|
||||||
|
|
||||||
The `VMServiceScrape` object discovers `Endpoints` objects and configures VMAgent to monitor `Pod`s.
|
|
||||||
|
|
||||||
The `Endpoints` section of the `VMServiceScrapeSpec` is used to configure which `Endpoints` ports should be scraped.
|
|
||||||
For advanced use cases, one may want to monitor ports of backing `Pod`s, which are not a part of the service endpoints.
|
|
||||||
Therefore, when specifying an endpoint in the `endpoints` section, they are strictly used.
|
|
||||||
|
|
||||||
> Note: `endpoints` (lowercase) is the field in the `VMServiceScrape` CRD, while `Endpoints` (capitalized) is the Kubernetes object kind.
|
|
||||||
|
|
||||||
Both `VMServiceScrape` and discovered targets may belong to any namespace. It is important for cross-namespace monitoring
|
|
||||||
use cases, e.g. for meta-monitoring. Using the `serviceScrapeSelector` of the `VMAgentSpec`
|
|
||||||
one can restrict the namespaces from which `VMServiceScrape`s are selected from by the respective VMAgent server.
|
|
||||||
Using the `namespaceSelector` of the `VMServiceScrape` one can restrict the namespaces from which `Endpoints` can be
|
|
||||||
discovered from. To discover targets in all namespaces the `namespaceSelector` has to be empty:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
spec:
|
|
||||||
namespaceSelector: {}
|
|
||||||
```
|
|
||||||
|
|
||||||
## VMPodScrape
|
|
||||||
|
|
||||||
The `VMPodScrape` CRD allows to declaratively define how a dynamic set of pods should be monitored.
|
|
||||||
Use label selections to match pods for scraping. This allows an organization to introduce conventions
|
|
||||||
for how metrics should be exposed. Following these conventions new services will be discovered automatically without
|
|
||||||
need to reconfigure.
|
|
||||||
|
|
||||||
A `Pod` is a collection of one or more containers which can expose Prometheus metrics on a number of ports.
|
|
||||||
|
|
||||||
The `VMPodScrape` object discovers pods and generates the relevant scraping configuration.
|
|
||||||
|
|
||||||
The `PodMetricsEndpoints` section of the `VMPodScrapeSpec` is used to configure which ports of a pod are going to be
|
|
||||||
scraped for metrics and with which parameters.
|
|
||||||
|
|
||||||
Both `VMPodScrapes` and discovered targets may belong to any namespace. It is important for cross-namespace monitoring
|
|
||||||
use cases, e.g. for meta-monitoring. Using the `namespaceSelector` of the `VMPodScrapeSpec` one can restrict the
|
|
||||||
namespaces from which `Pods` are discovered from. To discover targets in all namespaces the `namespaceSelector` has to
|
|
||||||
be empty:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
spec:
|
|
||||||
namespaceSelector:
|
|
||||||
any: true
|
|
||||||
```
|
|
||||||
|
|
||||||
## VMAlertmanager
|
|
||||||
|
|
||||||
The `VMAlertmanager` CRD declaratively defines a desired Alertmanager setup to run in a Kubernetes cluster.
|
|
||||||
It provides options to configure replication and persistent storage.
|
|
||||||
|
|
||||||
For each `Alertmanager` resource, the Operator deploys a properly configured `StatefulSet` in the same namespace.
|
|
||||||
The Alertmanager pods are configured to include a `Secret` called `<alertmanager-name>` which holds the used
|
|
||||||
configuration file in the key `alertmanager.yaml`.
|
|
||||||
|
|
||||||
When there are two or more configured replicas the Operator runs the Alertmanager instances in high availability mode.
|
|
||||||
|
|
||||||
## VMAlertmanagerConfig
|
|
||||||
|
|
||||||
The `VMAlertmanagerConfig` provides way to configure `VMAlertmanager` configuration with CRD. It allows to define different configuration parts,
|
|
||||||
which will be merged by operator into config. It behaves like other config parts - `VMServiceScrape` and etc.
|
|
||||||
|
|
||||||
## VMRule
|
|
||||||
|
|
||||||
The `VMRule` CRD declaratively defines a desired Prometheus rule to be consumed by one or more VMAlert instances.
|
|
||||||
|
|
||||||
Alerts and recording rules can be saved and applied as YAML files, and dynamically loaded without requiring any restart.
|
|
||||||
|
|
||||||
|
|
||||||
## VMPrometheusConverter
|
|
||||||
|
|
||||||
By default, the Operator converts and updates existing prometheus-operator API objects:
|
|
||||||
|
|
||||||
`ServiceMonitor` into `VMServiceScrape`
|
|
||||||
`PodMonitor` into `VMPodScrape`
|
|
||||||
`PrometheusRule` into `VMRule`
|
|
||||||
`Probe` into `VMProbe`
|
|
||||||
Removing prometheus-operator API objects wouldn't delete any converted objects. So you can safely migrate or run
|
|
||||||
two operators at the same time.
|
|
||||||
|
|
||||||
|
|
||||||
## VMProbe
|
|
||||||
|
|
||||||
The `VMProbe` CRD provides probing target ability with a prober. The most common prober is [blackbox exporter](https://github.com/prometheus/blackbox_exporter).
|
|
||||||
By specifying configuration at CRD, operator generates config for `VMAgent` and syncs it. It's possible to use static targets
|
|
||||||
or use standard k8s discovery mechanism with `Ingress`.
|
|
||||||
You have to configure blackbox exporter before you can use this feature. The second requirement is `VMAgent` selectors,
|
|
||||||
it must match your `VMProbe` by label or namespace selector. `VMAgent` probeSelector must match `VMProbe` labels.
|
|
||||||
See more details about selectors [here](https://docs.victoriametrics.com/operator/quick-start.html#object-selectors).
|
|
||||||
|
|
||||||
## VMNodeScrape
|
|
||||||
|
|
||||||
The `VMNodeScrape` CRD provides discovery mechanism for scraping metrics kubernetes nodes.
|
|
||||||
By specifying configuration at CRD, operator generates config for `VMAgent` and syncs it. It's useful for cadvisor scraping,
|
|
||||||
node-exporter or other node-based exporters. `VMAgent` nodeScrapeSelector must match `VMNodeScrape` labels.
|
|
||||||
See more details about selectors [here](https://docs.victoriametrics.com/operator/quick-start.html#object-selectors).
|
|
||||||
|
|
||||||
## VMStaticScrape
|
|
||||||
|
|
||||||
The `VMStaticScrape` CRD provides mechanism for scraping metrics from static targets, configured by CRD targets.
|
|
||||||
By specifying configuration at CRD, operator generates config for `VMAgent` and syncs it. It's useful for external targets management,
|
|
||||||
when service-discovery is not available. `VMAgent` staticScrapeSelector must match `VMStaticScrape` labels.
|
|
||||||
See more details about selectors [here](https://docs.victoriametrics.com/operator/quick-start.html#object-selectors).
|
|
||||||
|
|
||||||
## VMAuth
|
|
||||||
|
|
||||||
The `VMAuth` CRD provides mechanism for exposing application with authorization to outside world or to other applications inside kubernetes cluster.
|
|
||||||
For first case, user can configure `ingress` setting at `VMAuth` CRD. For second one, operator will create secret with `username` and `password` at `VMUser` CRD name.
|
|
||||||
So it will be possible to access these credentials from any application by targeting corresponding kubernetes secret.
|
|
||||||
|
|
||||||
## VMUser
|
|
||||||
|
|
||||||
The `VMUser` CRD describes user configuration, its authentication methods `basic auth` or `Authorization` header. User access permissions, with possible routing information.
|
|
||||||
User can define routing target with `static` config, by entering target `url`, or with `CRDRef`, in this case, operator queries kubernetes API, retrieves information about CRD and builds proper url.
|
|
||||||
|
|||||||
@ -1,93 +1,9 @@
|
|||||||
---
|
---
|
||||||
sort: 8
|
sort: -1 # hide page from menu
|
||||||
weight: 8
|
|
||||||
title: Managing application versions
|
|
||||||
menu:
|
|
||||||
docs:
|
|
||||||
parent: "operator"
|
|
||||||
weight: 8
|
|
||||||
aliases:
|
aliases:
|
||||||
- /operator/managing-versions.html
|
- /operator/managing-versions.html
|
||||||
---
|
---
|
||||||
|
|
||||||
|
This page is deprecated.
|
||||||
|
|
||||||
# Managing application versions
|
Please see [Managing versions of VM](./resources/README.md#managing-versions-of-vm) instead.
|
||||||
|
|
||||||
## VMAlert, VMAgent, VMAlertmanager, VMSingle version
|
|
||||||
|
|
||||||
|
|
||||||
for those objects you can specify following settings at `spec.Image`
|
|
||||||
|
|
||||||
for instance, to set `VMSingle` version add `spec.image.tag` name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
cat <<EOF | kubectl apply -f -
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMSingle
|
|
||||||
metadata:
|
|
||||||
name: example-vmsingle
|
|
||||||
spec:
|
|
||||||
image:
|
|
||||||
repository: victoriametrics/victoria-metrics
|
|
||||||
tag: v1.39.2
|
|
||||||
pullPolicy: Always
|
|
||||||
retentionPeriod: "1"
|
|
||||||
EOF
|
|
||||||
```
|
|
||||||
|
|
||||||
Also, you can specify `imagePullSecrets` if you are pulling images from private repo:
|
|
||||||
```yaml
|
|
||||||
cat <<EOF | kubectl apply -f -
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMSingle
|
|
||||||
metadata:
|
|
||||||
name: example-vmsingle
|
|
||||||
spec:
|
|
||||||
imagePullSecrets:
|
|
||||||
- name: my-repo-secret
|
|
||||||
image:
|
|
||||||
repository: my-repo-url/victoria-metrics
|
|
||||||
tag: v1.39.2
|
|
||||||
retentionPeriod: "1"
|
|
||||||
EOF
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
# VMCluster
|
|
||||||
|
|
||||||
for `VMCluster` you can specify tag and repository setting per cluster object.
|
|
||||||
But `imagePullSecrets` is global setting for all `VMCluster` specification.
|
|
||||||
```yaml
|
|
||||||
cat << EOF | kubectl apply -f -
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMCluster
|
|
||||||
metadata:
|
|
||||||
name: example-vmcluster
|
|
||||||
spec:
|
|
||||||
imagePullSecrets:
|
|
||||||
- name: my-repo-secret
|
|
||||||
# Add fields here
|
|
||||||
retentionPeriod: "1"
|
|
||||||
vmstorage:
|
|
||||||
replicaCount: 2
|
|
||||||
image:
|
|
||||||
repository: victoriametrics/vmstorage
|
|
||||||
tag: v1.39.2-cluster
|
|
||||||
pullPolicy: Always
|
|
||||||
vmselect:
|
|
||||||
replicaCount: 2
|
|
||||||
image:
|
|
||||||
repository: victoriametrics/vmselect
|
|
||||||
tag: v1.39.2-cluster
|
|
||||||
pullPolicy: Always
|
|
||||||
vminsert:
|
|
||||||
replicaCount: 2
|
|
||||||
image:
|
|
||||||
repository: victoriametrics/vminsert
|
|
||||||
tag: v1.39.2-cluster
|
|
||||||
pullPolicy: Always
|
|
||||||
EOF
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@ -1,250 +1,9 @@
|
|||||||
---
|
---
|
||||||
sort: 10
|
sort: -1 # hide page from menu
|
||||||
weight: 10
|
|
||||||
title: Relabeling
|
|
||||||
menu:
|
|
||||||
docs:
|
|
||||||
parent: "operator"
|
|
||||||
weight: 10
|
|
||||||
aliases:
|
aliases:
|
||||||
- /operator/relabeling.html
|
- /operator/relabeling.html
|
||||||
---
|
---
|
||||||
|
|
||||||
# Relabeling
|
This page is deprecated.
|
||||||
|
|
||||||
## VMAgent relabel
|
Please see [Relabeling](./resources/vmagent.md#relabeling) section in VMAgent docs instead.
|
||||||
|
|
||||||
|
|
||||||
`VMAgent` supports global relabeling for all metrics and per remoteWrite target relabel config.
|
|
||||||
|
|
||||||
> Note in some cases, you don't need relabeling,
|
|
||||||
> key=value label pairs can be added to the all scrapped metrics with `spec.externalLabels` for `VMAgent`.
|
|
||||||
>
|
|
||||||
```yaml
|
|
||||||
# simple label add config
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMAgent
|
|
||||||
metadata:
|
|
||||||
name: stack
|
|
||||||
spec:
|
|
||||||
externalLabels:
|
|
||||||
clusterid: some_cluster
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
It supports relabeling with custom configMap or inline defined at CRD
|
|
||||||
|
|
||||||
## Configmap example
|
|
||||||
|
|
||||||
Quick tour how to create `ConfigMap` with relabeling configuration
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
cat << EOF | kubectl apply -f -
|
|
||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: vmagent-relabel
|
|
||||||
data:
|
|
||||||
global-relabel.yaml: |
|
|
||||||
- target_label: bar
|
|
||||||
- source_labels: [aa]
|
|
||||||
separator: "foobar"
|
|
||||||
regex: "foo.+bar"
|
|
||||||
target_label: aaa
|
|
||||||
replacement: "xxx"
|
|
||||||
- action: keep
|
|
||||||
source_labels: [aaa]
|
|
||||||
- action: drop
|
|
||||||
source_labels: [aaa]
|
|
||||||
target-1-relabel.yaml: |
|
|
||||||
- action: keep_if_equal
|
|
||||||
source_labels: [foo, bar]
|
|
||||||
- action: drop_if_equal
|
|
||||||
source_labels: [foo, bar]
|
|
||||||
EOF
|
|
||||||
```
|
|
||||||
|
|
||||||
Second, add `relabelConfig` to `VMagent` spec for global relabeling with name of `Configmap` - `vmagent-relabel` and key `global-relabel.yaml`.
|
|
||||||
For relabeling per remoteWrite target, add `urlRelabelConfig` name of `Configmap` - `vmagent-relabel` and key `target-1-relabel.yaml` to one of remoteWrite target for relabeling only
|
|
||||||
for those target.
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
cat <<EOF | kubectl apply -f -
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMAgent
|
|
||||||
metadata:
|
|
||||||
name: example-vmagent
|
|
||||||
spec:
|
|
||||||
serviceScrapeNamespaceSelector: {}
|
|
||||||
podScrapeNamespaceSelector: {}
|
|
||||||
podScrapeSelector: {}
|
|
||||||
serviceScrapeSelector: {}
|
|
||||||
replicaCount: 1
|
|
||||||
serviceAccountName: vmagent
|
|
||||||
relabelConfig:
|
|
||||||
name: "vmagent-relabel"
|
|
||||||
key: "global-relabel.yaml"
|
|
||||||
remoteWrite:
|
|
||||||
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
|
|
||||||
- url: "http://vmsingle-example-vmsingle.default.svc:8429/api/v1/write"
|
|
||||||
urlRelabelConfig:
|
|
||||||
name: "vmagent-relabel"
|
|
||||||
key: "target-1-relabel.yaml"
|
|
||||||
EOF
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
## Inline example
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
cat <<EOF | kubectl apply -f -
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMAgent
|
|
||||||
metadata:
|
|
||||||
name: example-vmagent
|
|
||||||
spec:
|
|
||||||
serviceScrapeNamespaceSelector: {}
|
|
||||||
podScrapeNamespaceSelector: {}
|
|
||||||
podScrapeSelector: {}
|
|
||||||
serviceScrapeSelector: {}
|
|
||||||
replicaCount: 1
|
|
||||||
serviceAccountName: vmagent
|
|
||||||
inlineRelabelConfig:
|
|
||||||
- target_label: bar
|
|
||||||
- source_labels: [aa]
|
|
||||||
separator: "foobar"
|
|
||||||
regex: "foo.+bar"
|
|
||||||
target_label: aaa
|
|
||||||
replacement: "xxx"
|
|
||||||
- action: keep
|
|
||||||
source_labels: [aaa]
|
|
||||||
- action: drop
|
|
||||||
source_labels: [aaa]
|
|
||||||
remoteWrite:
|
|
||||||
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
|
|
||||||
- url: "http://vmsingle-example-vmsingle.default.svc:8429/api/v1/write"
|
|
||||||
inlineUrlRelabelConfig:
|
|
||||||
- action: keep_if_equal
|
|
||||||
source_labels: [foo, bar]
|
|
||||||
- action: drop_if_equal
|
|
||||||
source_labels: [foo, bar]
|
|
||||||
EOF
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
## Combined example
|
|
||||||
|
|
||||||
It's also possible to use both features in combination.
|
|
||||||
|
|
||||||
First will be added relabeling configs from `inlineRelabelConfig`, then `relabelConfig` from configmap.
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
cat << EOF | kubectl apply -f -
|
|
||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: vmagent-relabel
|
|
||||||
data:
|
|
||||||
global-relabel.yaml: |
|
|
||||||
- target_label: bar
|
|
||||||
- source_labels: [aa]
|
|
||||||
separator: "foobar"
|
|
||||||
regex: "foo.+bar"
|
|
||||||
target_label: aaa
|
|
||||||
replacement: "xxx"
|
|
||||||
- action: keep
|
|
||||||
source_labels: [aaa]
|
|
||||||
- action: drop
|
|
||||||
source_labels: [aaa]
|
|
||||||
target-1-relabel.yaml: |
|
|
||||||
- action: keep_if_equal
|
|
||||||
source_labels: [foo, bar]
|
|
||||||
- action: drop_if_equal
|
|
||||||
source_labels: [foo, bar]
|
|
||||||
EOF
|
|
||||||
```
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
cat <<EOF | kubectl apply -f -
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMAgent
|
|
||||||
metadata:
|
|
||||||
name: example-vmagent
|
|
||||||
spec:
|
|
||||||
serviceScrapeNamespaceSelector: {}
|
|
||||||
podScrapeNamespaceSelector: {}
|
|
||||||
podScrapeSelector: {}
|
|
||||||
serviceScrapeSelector: {}
|
|
||||||
replicaCount: 1
|
|
||||||
serviceAccountName: vmagent
|
|
||||||
inlineRelabelConfig:
|
|
||||||
- target_label: bar1
|
|
||||||
- source_labels: [aa]
|
|
||||||
relabelConfig:
|
|
||||||
name: "vmagent-relabel"
|
|
||||||
key: "global-relabel.yaml"
|
|
||||||
remoteWrite:
|
|
||||||
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
|
|
||||||
- url: "http://vmsingle-example-vmsingle.default.svc:8429/api/v1/write"
|
|
||||||
urlRelabelConfig:
|
|
||||||
name: "vmagent-relabel"
|
|
||||||
key: "target-1-relabel.yaml"
|
|
||||||
inlineUrlRelabelConfig:
|
|
||||||
- action: keep_if_equal
|
|
||||||
source_labels: [foo1, bar2]
|
|
||||||
EOF
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
Resulted configmap:
|
|
||||||
```yaml
|
|
||||||
apiVersion: v1
|
|
||||||
data:
|
|
||||||
global_relabeling.yaml: |
|
|
||||||
- target_label: bar1
|
|
||||||
- source_labels:
|
|
||||||
- aa
|
|
||||||
- target_label: bar
|
|
||||||
- source_labels: [aa]
|
|
||||||
separator: "foobar"
|
|
||||||
regex: "foo.+bar"
|
|
||||||
target_label: aaa
|
|
||||||
replacement: "xxx"
|
|
||||||
- action: keep
|
|
||||||
source_labels: [aaa]
|
|
||||||
- action: drop
|
|
||||||
source_labels: [aaa]
|
|
||||||
url_rebaling-1.yaml: |
|
|
||||||
- source_labels:
|
|
||||||
- foo1
|
|
||||||
- bar2
|
|
||||||
action: keep_if_equal
|
|
||||||
- action: keep_if_equal
|
|
||||||
source_labels: [foo, bar]
|
|
||||||
- action: drop_if_equal
|
|
||||||
source_labels: [foo, bar]
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
finalizers:
|
|
||||||
- apps.victoriametrics.com/finalizer
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/component: monitoring
|
|
||||||
app.kubernetes.io/instance: example-vmagent
|
|
||||||
app.kubernetes.io/name: vmagent
|
|
||||||
managed-by: vm-operator
|
|
||||||
name: relabelings-assets-vmagent-example-vmagent
|
|
||||||
namespace: default
|
|
||||||
ownerReferences:
|
|
||||||
- apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
blockOwnerDeletion: true
|
|
||||||
controller: true
|
|
||||||
kind: VMAgent
|
|
||||||
name: example-vmagent
|
|
||||||
uid: 7e9fb838-65da-4443-a43b-c00cd6c4db5b
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
## Additional information
|
|
||||||
|
|
||||||
`VMAgent` also has some extra options for relabeling actions, you can check it [docs](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmagent/README.md#relabeling)
|
|
||||||
|
|||||||
@ -1,44 +1,9 @@
|
|||||||
---
|
---
|
||||||
sort: 11
|
sort: -1 # hide page from menu
|
||||||
weight: 11
|
|
||||||
title: CRD Validation
|
|
||||||
menu:
|
|
||||||
docs:
|
|
||||||
parent: "operator"
|
|
||||||
weight: 11
|
|
||||||
aliases:
|
aliases:
|
||||||
- /operator/resources-validation.html
|
- /operator/resources-validation.html
|
||||||
---
|
---
|
||||||
|
|
||||||
# CRD Validation
|
This page is deprecated.
|
||||||
|
|
||||||
## Description
|
Please see [CRD Validation](./configuration.md#crd-validation) instead.
|
||||||
Operator supports validation admission webhook [docs](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/)
|
|
||||||
|
|
||||||
It checks resources configuration and returns errors to caller before resource will be created at kubernetes api.
|
|
||||||
This should reduce errors and simplify debugging.
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
Validation hooks at operator side must be enabled with flags:
|
|
||||||
```
|
|
||||||
--webhook.enable
|
|
||||||
# optional configuration for certDir and tls names.
|
|
||||||
--webhook.certDir=/tmp/k8s-webhook-server/serving-certs/
|
|
||||||
--webhook.keyName=tls.key
|
|
||||||
--webhook.certName=tls.crt
|
|
||||||
```
|
|
||||||
|
|
||||||
You have to mount correct certificates at give directory.
|
|
||||||
It can be simplified with cert-manager and kustomize command: `kustomize build config/deployments/webhook/ `
|
|
||||||
|
|
||||||
|
|
||||||
## Requirements
|
|
||||||
|
|
||||||
- Valid certificate with key must be provided to operator
|
|
||||||
- Valid CABundle must be added to the `ValidatingWebhookConfiguration`
|
|
||||||
|
|
||||||
|
|
||||||
## Useful links
|
|
||||||
- [k8s admission webhooks](https://banzaicloud.com/blog/k8s-admission-webhooks/)
|
|
||||||
- [olm webhooks](https://docs.openshift.com/container-platform/4.5/operators/user/olm-webhooks.html)
|
|
||||||
|
|||||||
@ -1,152 +1,10 @@
|
|||||||
---
|
---
|
||||||
sort: 13
|
sort: -1 # hide page from menu
|
||||||
weight: 13
|
|
||||||
title: Managing configuration for VMAlertmanager
|
|
||||||
menu:
|
|
||||||
docs:
|
|
||||||
parent: "operator"
|
|
||||||
weight: 13
|
|
||||||
aliases:
|
aliases:
|
||||||
- /operator/vmalertmanager-configuration.html
|
- /operator/vmalertmanager-configuration.html
|
||||||
---
|
---
|
||||||
|
|
||||||
# Managing configuration for VMAlertmanager
|
This page is deprecated.
|
||||||
|
|
||||||
The operator generates a configuration file for `VMAlertmanager` based on user input at the definition of `CRD`.
|
Please see [VMAlertmanager](./resources/vmalertmanager.md)
|
||||||
Generated config stored at `Secret` created by the operator, it has the following name template `vmalertmanager-CRD_NAME-config`.
|
and [VMAlertmanagerConfig](./resources/vmalertmanagerconfig.md) instead.
|
||||||
This configuration file is mounted at `VMAlertmanager` `Pod`. A special side-car container tracks its changes and sends config-reload signals to `alertmanager` container.
|
|
||||||
|
|
||||||
## Using secret
|
|
||||||
|
|
||||||
Basically, you can use the global configuration defined at manually created `Secret`. This `Secret` must be created before `VMAlertmanager`.
|
|
||||||
|
|
||||||
Name of the `Secret` must be defined at `VMAlertmanager` `spec.configSecret` option.
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Secret
|
|
||||||
metadata:
|
|
||||||
name: vmalertmanager-example-alertmanager
|
|
||||||
labels:
|
|
||||||
app: vm-operator
|
|
||||||
type: Opaque
|
|
||||||
stringData:
|
|
||||||
alertmanager.yaml: |
|
|
||||||
global:
|
|
||||||
resolve_timeout: 5m
|
|
||||||
route:
|
|
||||||
receiver: 'webhook'
|
|
||||||
receivers:
|
|
||||||
- name: 'webhook'
|
|
||||||
webhook_configs:
|
|
||||||
- url: 'http://alertmanagerwh:30500/'
|
|
||||||
---
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMAlertmanager
|
|
||||||
metadata:
|
|
||||||
name: example-alertmanager
|
|
||||||
spec:
|
|
||||||
replicaCount: 2
|
|
||||||
configSecret: vmalertmanager-example-alertmanager
|
|
||||||
```
|
|
||||||
|
|
||||||
## Using inline raw config
|
|
||||||
|
|
||||||
Also, if there is no secret data at configuration, or you just want to redefine some global variables for `alertmanager`.
|
|
||||||
You can define configuration at `spec.configRawYaml` section of `VMAlertmanager` configuration:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMAlertmanager
|
|
||||||
metadata:
|
|
||||||
name: example-alertmanager
|
|
||||||
spec:
|
|
||||||
replicaCount: 2
|
|
||||||
configRawYaml: |
|
|
||||||
global:
|
|
||||||
resolve_timeout: 5m
|
|
||||||
route:
|
|
||||||
receiver: 'default'
|
|
||||||
group_interval: 5m
|
|
||||||
repeat_interval: 12h
|
|
||||||
receivers:
|
|
||||||
- name: 'default'
|
|
||||||
```
|
|
||||||
|
|
||||||
If both `configSecret` and `configRawYaml` are defined, only configuration from `configRawYaml` will be used. Values from `configRawYaml` will be ignored.
|
|
||||||
|
|
||||||
## Using VMAlertmanagerConfig
|
|
||||||
|
|
||||||
`VMAlertmanagerConfig` allows delegating notification configuration to the kubernetes cluster users.
|
|
||||||
The application owner may configure notifications by defining it at `VMAlertmanagerConfig`.
|
|
||||||
With the combination of `VMRule` and `VMServiceScrape` it allows delegating configuration observability to application owners, and uses popular `GitOps` practice.
|
|
||||||
|
|
||||||
Operator combines `VMAlertmanagerConfigs` into a single configuration file for `VMAlertmanager`.
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMAlertmanagerConfig
|
|
||||||
metadata:
|
|
||||||
name: example-email-web
|
|
||||||
namespace: production
|
|
||||||
spec:
|
|
||||||
route:
|
|
||||||
receiver: email
|
|
||||||
group_interval: 1m
|
|
||||||
routes:
|
|
||||||
- receiver: email
|
|
||||||
matchers:
|
|
||||||
- {severity =~ "warning|critical", app_name = "blog"}
|
|
||||||
receivers:
|
|
||||||
- name: email
|
|
||||||
email_configs:
|
|
||||||
- to: some-email@example.com
|
|
||||||
from: alerting@example.com
|
|
||||||
smarthost: example.com:25
|
|
||||||
text: ALARM
|
|
||||||
```
|
|
||||||
|
|
||||||
### Special Case
|
|
||||||
|
|
||||||
VMAlertmanagerConfig has enforced namespace matcher.
|
|
||||||
Alerts must have a proper namespace label, with the same value as name of namespace for VMAlertmanagerConfig.
|
|
||||||
It can be disabled, by setting the following value to the VMAlertmanager: spec.disableNamespaceMatcher: true.
|
|
||||||
|
|
||||||
## Extra configuration files
|
|
||||||
|
|
||||||
`VMAlertmanager` specification has the following fields, that can be used to configure without editing raw configuration file:
|
|
||||||
|
|
||||||
- `spec.templates` - list of keys in `ConfigMaps`, that contains template files for `alertmanager`, e.g.:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMAlertmanager
|
|
||||||
metadata:
|
|
||||||
name: example-alertmanager
|
|
||||||
spec:
|
|
||||||
replicaCount: 2
|
|
||||||
templates:
|
|
||||||
- Name: alertmanager-templates
|
|
||||||
Key: my-template-1.tmpl
|
|
||||||
- Name: alertmanager-templates
|
|
||||||
Key: my-template-2.tmpl
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: alertmanager-templates
|
|
||||||
data:
|
|
||||||
my-template-1.tmpl: |
|
|
||||||
{{ define "hello" -}}
|
|
||||||
hello, Victoria!
|
|
||||||
{{- end }}
|
|
||||||
my-template-2.tmpl: """
|
|
||||||
```
|
|
||||||
|
|
||||||
These templates will be automatically added to `VMAlertmanager` configuration and will be automatically reloaded on changes in source `ConfigMap`.
|
|
||||||
- `spec.configMaps` - list of `ConfigMap` names (in the same namespace) that will be mounted at `VMAlertmanager`
|
|
||||||
workload and will be automatically reloaded on changes in source `ConfigMap`. Mount path is `/etc/vm/configs/<configmap-name>`.
|
|
||||||
|
|
||||||
## Behavior without provided config
|
|
||||||
|
|
||||||
If no configuration is provided, operator configures stub configuration with blackhole route.
|
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user