Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-14 16:12:15 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Created Operator folder in docs (#2034)

Browse Source 
* Created Operator folder in docs

Transferred Operator documentation

* Removed Contributing and Release

* Changed sort numbering

* Renamed folder

Operator -> operator

* 1

1

* Name change

Operator -> operator

* Removed colon symbol

* Useful links transformed to links style

* "updated at..." is no longer a header

* delete manager patch.yaml

* delete kustomization.yaml

* removed part with links

* community and contributions part removed

* Delete readme

* Docs navigation removed
This commit is contained in:
Yurii Kravets 2022-01-21 12:05:58 +02:00 committed by Aliaksandr Valialkin
parent 034012c80f
commit 4b7650832e
No known key found for this signature in database
GPG Key ID: A72BEC6CD3D0DED1
15 changed files with 5131 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
docs/operator/README.md Normal file
View File

@ -0,0 +1,20 @@
---
sort: 22
---
# VictoriaMetrics Operator
1. [VictoriaMetrics Operator](VictoriaMetrics-Operator.html)
2. [Additional Scrape Configuration](additional-scrape.html)
3. [API Docs](api.html)
4. [Authorization and exposing components](auth.html)
5. [vmbackupmanager](backups.html)
6. [Design](design.html)
7. [High Availability](high-availability.html)
8. [VMAlert, VMAgent, VMAlertmanager, VMSingle version](managing-versions.html)
9. [Victoria Metrics Operator Quick Start](quick-start.html)
10. [VMAgent relabel](relabeling.html)
11. [CRD Validation](resources-validation.html)
12. [Security](security.html)
13. [Auto Generated vars for package config](vars.html)
14. [CONTRIBUTING](CONTRIBUTING.html)

79
docs/operator/VictoriaMetrics-Operator.md Normal file
View File

@ -0,0 +1,79 @@
---
sort: 1
---
# VictoriaMetrics operator
## Overview
Design and implementation inspired by [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator). It's great a tool for managing monitoring configuration of your applications. VictoriaMetrics operator has api capability with it.
So you can use familiar CRD objects: `ServiceMonitor`, `PodMonitor`, `PrometheusRule` and `Probe`. Or you can use VictoriaMetrics CRDs:
- `VMServiceScrape` - defines scraping metrics configuration from pods backed by services.
- `VMPodScrape` - defines scraping metrics configuration from pods.
- `VMRule` - defines alerting or recording rules.
- `VMProbe` - defines a probing configuration for targets with blackbox exporter.
Besides, operator allows you to manage VictoriaMetrics applications inside kubernetes cluster and simplifies this process [quick-start](/Operator/quick-start.html)
With CRD (Custom Resource Definition) you can define application configuration and apply it to your cluster [crd-objects](/Operator/api.html).
Operator simplifies VictoriaMetrics cluster installation, upgrading and managing.
It has integration with VictoriaMetrics `vmbackupmanager` - advanced tools for making backups. Check backup [docs](/Operator/backups.html)
## Use cases
For kubernetes-cluster administrators, it simplifies installation, configuration, management for `VictoriaMetrics` application. And the main feature of operator - is ability to delegate applications monitoring configuration to the end-users.
For applications developers, its great possibility for managing observability of applications. You can define metrics scraping and alerting configuration for your application and manage it with an application deployment process. Just define app_deployment.yaml, app_vmpodscrape.yaml and app_vmrule.yaml. That's it, you can apply it to a kubernetes cluster. Check [quick-start](/Operator/quick-start.html) for an example.
## Operator vs helm-chart
VictoriaMetrics provides [helm charts](https://github.com/VictoriaMetrics/helm-charts). Operator makes the same, simplifies it and provides advanced features.
## Configuration
Operator configured by env variables, list of it can be found at [link](/vars.html)
It defines default configuration options, like images for components, timeouts, features.
## Kubernetes' compatibility versions
operator tested at kubernetes versions
from 1.16 to 1.22
For clusters version below 1.16 you must use legacy CRDs from [path](config/crd/legacy)
and disable CRD controller with flag: `--controller.disableCRDOwnership=true`
## Troubleshooting
- cannot apply crd at kubernetes 1.18 + version and kubectl reports error:
```bash
Error from server (Invalid): error when creating "release/crds/crd.yaml": CustomResourceDefinition.apiextensions.k8s.io "vmalertmanagers.operator.victoriametrics.com" is invalid: [spec.validation.openAPIV3Schema.properties[spec].properties[initContainers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property]
Error from server (Invalid): error when creating "release/crds/crd.yaml": CustomResourceDefinition.apiextensions.k8s.io "vmalerts.operator.victoriametrics.com" is invalid: [
```
upgrade to the latest release version. There is a bug with kubernetes objects at the early releases.
## Development
- operator-sdk verson v1.0.0 + [https://github.com/operator-framework/operator-sdk]
- golang 1.15 +
- minikube or kind
start:
```bash
make run
```
for test execution run:
```bash
#unit tests
make test
# you need minikube or kind for e2e, do not run it on live cluster
#e2e tests with local binary
make e2e-local
```

85
docs/operator/additional-scrape.MD Normal file
View File

@ -0,0 +1,85 @@
---
sort: 2
---
# Additional Scrape Configuration
AdditionalScrapeConfigs allows specifying a key of a Secret containing
additional Prometheus scrape configurations or define scrape configuration at CRD spec.
Scrape configurations specified
are appended to the configurations generated by the operator.
Job configurations specified must have the form as specified in the official
[Prometheus documentation](
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config).
As scrape configs are appended, the user is responsible to make sure it is
valid.
## Creating an additional configuration inline at CRD
Add needed scrape configuration directly to the vmagent spec.inlineScrapeConfig
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
name: example-vmagent
spec:
serviceScrapeSelector: {}
replicas: 1
serviceAccountName: vmagent
inlineScrapeConfig: |
- job_name: "prometheus"
static_configs:
- targets: ["localhost:9090"]
remoteWrite:
- url: "http://vmagent-example-vmsingle.default.svc:8429/api/v1/write"
EOF
```
NOTE: Do not use password and tokens with inlineScrapeConfig.
## Creating an additional configuration with secret
First, you will need to create the additional configuration.
Below we are making a simple "prometheus" config. Name this
`prometheus-additional.yaml` or something similar.
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
name: additional-scrape-configs
stringData:
prometheus-additional.yaml: |
- job_name: "prometheus"
static_configs:
- targets: ["localhost:9090"]
EOF
```
Finally, reference this additional configuration in your `vmagent.yaml` CRD.
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
name: example-vmagent
spec:
serviceScrapeSelector: {}
replicas: 1
serviceAccountName: vmagent
additionalScrapeConfigs:
name: additional-scrape-configs
key: prometheus-additional.yaml
remoteWrite:
- url: "http://vmagent-example-vmsingle.default.svc:8429/api/v1/write"
EOF
```
NOTE: Use only one secret for ALL additional scrape configurations.

2003
docs/operator/api.MD Normal file
View File

File diff suppressed because it is too large Load Diff

182
docs/operator/auth.MD Normal file
View File

@ -0,0 +1,182 @@
---
sort: 4
---
# Authorization and exposing components
## Exposing components
CRD objects doesn't have `ingress` configuration. Instead, you can use `VMAuth` as proxy between ingress-controller and VM app components.
It adds missing authorization and access control features and enforces it.
Access can be given with `VMUser` definition. It supports basic auth and bearer token authentication.
```yaml
cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAuth
metadata:
name: main-router
spec:
userNamespaceSelector: {}
userSelector: {}
ingress: {}
EOF
```
Advanced configuration with cert-manager annotations:
```yaml
cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAuth
metadata:
name: router-main
spec:
podMetadata:
labels:
component: vmauth
userSelector: {}
userNamespaceSelector: {}
replicaCount: 2
resources:
requests:
cpu: "250m"
memory: "350Mi"
limits:
cpu: "500m"
memory: "850Mi"
ingress:
tlsSecretName: vmauth-tls
annotations:
cert-manager.io/cluster-issuer: base
class_name: nginx
tlsHosts:
- vm-access.example.com
EOF
```
simple static routing with read-only access to vmagent for username - `user-1` with password `Asafs124142`
```yaml
# curl vmauth:8427/metrics -u 'user-1:Asafs124142'
cat << EOF | kubectl apply -f
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMUser
metadata:
name: user-1
spec:
password: Asafs124142
targetRefs:
- static:
url: http://vmagent-base.default.svc:8429
paths: ["/targets/api/v1","/targets","/metrics"]
EOF
```
With bearer token access:
```yaml
# curl vmauth:8427/metrics -H 'Authorization: Bearer Asafs124142'
cat << EOF | kubectl apply -f
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMUser
metadata:
name: user-2
spec:
bearerToken: Asafs124142
targetRefs:
- static:
url: http://vmagent-base.default.svc:8429
paths: ["/targets/api/v1","/targets","/metrics"]
EOF
```
It's also possible to use service discovery for objects:
```yaml
# curl vmauth:8427/metrics -H 'Authorization: Bearer Asafs124142'
cat << EOF | kubectl apply -f
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMUser
metadata:
name: user-3
spec:
bearerToken: Asafs124142
targetRefs:
- crd:
kind: VMAgent
name: base
namespace: default
paths: ["/targets/api/v1","/targets","/metrics"]
EOF
```
Cluster components supports auto path generation for single tenant view:
```yaml
cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMUser
metadata:
name: vmuser-tenant-1
spec:
bearerToken: some-token
targetRefs:
- crd:
kind: VMCluster/vminsert
name: test-persistent
namespace: default
target_path_suffix: "/insert/1"
- crd:
kind: VMCluster/vmselect
name: test-persistent
namespace: default
target_path_suffix: "/select/1"
- static:
url: http://vmselect-test-persistent.default.svc:8481/
paths:
- /internal/resetRollupResultCache
EOF
```
For each `VMUser` operator generates corresponding secret with username/password or bearer token at the same namespace as `VMUser`.
## Basic auth for targets
To authenticate a `VMServiceScrape`s over a metrics endpoint use [`basicAuth`](../api.html#basicauth)
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMServiceScrape
metadata:
labels:
k8s-apps: basic-auth-example
name: basic-auth-example
spec:
endpoints:
- basicAuth:
password:
name: basic-auth
key: password
username:
name: basic-auth
key: user
port: metrics
selector:
matchLabels:
app: myapp
EOF
```
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
name: basic-auth
data:
password: dG9vcg== # toor
user: YWRtaW4= # admin
type: Opaque
EOF
```

134
docs/operator/backups.MD Normal file
View File

@ -0,0 +1,134 @@
---
sort: 5
---
# vmbackupmanager
## vmbackupmanager is proprietary software.
Before using it, you must have signed contract and accept EULA https://victoriametrics.com/assets/VM_EULA.pdf
## Usage examples
`VMSingle` and `VMCluster` has built-in backup configuration, it uses `vmbackupmanager` - proprietary tool for backups.
It supports incremental backups (hours, daily, etc) with popular object storages (aws s3, google cloud storage).
You can enable it with the simple configuration, define secret
```yaml
---
apiVersion: v1
kind: Secret
metadata:
name: remote-storage-keys
type: Opaque
stringData:
credentials: |-
[default]
aws_access_key_id = your_access_key_id
aws_secret_access_key = your_secret_access_key
---
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMSingle
metadata:
name: example-vmsingle
spec:
# Add fields here
retentionPeriod: "1"
vmBackup:
# This is Enterprise Package feature you need to have signed contract to use it
# and accept the EULA https://victoriametrics.com/assets/VM_EULA.pdf
acceptEULA: true
destination: "s3://your_bucket/folder"
credentialsSecret:
name: remote-storage-keys
key: credentials
---
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMCluster
metadata:
name: example-vmcluster-persistent
spec:
retentionPeriod: "4"
replicationFactor: 2
vmstorage:
replicaCount: 2
vmBackup:
# This is Enterprise Package feature you need to have signed contract to use it
# and accept the EULA https://victoriametrics.com/assets/VM_EULA.pdf
acceptEULA: true
destination: "s3://your_bucket/folder"
credentialsSecret:
name: remote-storage-keys
key: credentials
```
NOTE: for cluster version operator adds suffix for `destination: "s3://your_bucket/folder"`, it becomes `"s3://your_bucket/folder/$(POD_NAME)"`.
It's needed to make consistent backups for each storage node.
You can read more about backup configuration options and mechanics [here](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmbackup)
Possible configuration options for backup crd can be found at [link](/docs/api.html#vmbackup)
## Restoring backups
It can be done with [vmrestore](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmrestore)
There two ways:
First:
You have to stop `VMSingle` by scaling it replicas to zero and manually restore data to the database directory.
Steps:
1) edit `VMSingle` CRD, set replicaCount: 0
2) wait until database stops
3) ssh to some server, where you can mount `VMSingle` disk and mount it manually
4) restore files with `vmrestore`
5) umount disk
6) edit `VMSingle` CRD, set replicaCount: 1
7) wait database start
Second:
1) add init container with vmrestore command to `VMSingle` CRD, example:
```yaml
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMSingle
metadata:
name: vmsingle-restored
namespace: monitoring-system
spec:
initContainers:
- name: vmrestore
image: victoriametrics/vmrestore:latest
volumeMounts:
- mountPath: /victoria-metrics-data
name: data
- mountPath: /etc/vm/creds
name: secret-remote-storage-keys
readOnly: true
args:
- -storageDataPath=/victoria-metrics-data
- -src=s3://your_bucket/folder/latest
- -credsFilePath=/etc/vm/creds/credentials
vmBackup:
# This is Enterprise Package feature you need to have signed contract to use it
# and accept the EULA https://victoriametrics.com/assets/VM_EULA.pdf
acceptEULA: true
destination: "s3://your_bucket/folder"
extraArgs:
runOnStart: "true"
image:
repository: victoriametrics/vmbackupmanager
tag: v1.67.0-enterprise
credentialsSecret:
name: remote-storage-keys
key: credentials
```
2) apply it, and db will be restored from s3
3) remove initContainers and apply crd.

229
docs/operator/design.MD Normal file
View File

@ -0,0 +1,229 @@
---
sort: 6
---
# Design
This document describes the design and interaction between the custom resource definitions (CRD) that the Victoria
Metrics Operator introduces.
Operator introduces the following custom resources:
* [VMSingle](#vmsingle)
* [VMCluster](#vmcluster)
* [VMAgent](#vmagent)
* [VMAlert](#vmalert)
* [VMServiceScrape](#vmservicescrape)
* [VMPodScrape](#vmpodscrape)
* [VMAlertmanager](#vmalertmanager)
* [VMAlertmanagerConfig](#vmalertmanagerconfig)
* [VMRule](#vmrule)
* [VMProbe](#vmprobe)
* [VMNodeScrape](#vmodescrape)
* [VMStaticScrape](#vmstaticscrape)
* [VMAuth](#vmauth)
* [VMUser](#vmuser)
## VMSingle
The `VMSingle` CRD declaratively defines a [single-node VM](https://github.com/VictoriaMetrics/VictoriaMetrics)
installation to run in a Kubernetes cluster.
For each `VMSingle` resource, the Operator deploys a properly configured `Deployment` in the same namespace.
The VMSingle `Pod`s are configured to mount an empty dir or `PersistentVolumeClaimSpec` for storing data.
Deployment update strategy set to [recreate](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#recreate-deployment).
No more than one replica allowed.
For each `VMSingle` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
name `<VMSingle-name>`.
## VMCluster
The `VMCluster` CRD defines a [cluster version VM](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster).
For each `VMCluster` resource, the Operator creates `VMStorage` as `StatefulSet`, `VMSelect` as `StatefulSet` and `VMInsert`
as deployment. For `VMStorage` and `VMSelect` headless services are created. `VMInsert` is created as service with clusterIP.
There is a strict order for these objects creation and reconciliation:
1. `VMStorage` is synced - the Operator waits until all its pods are ready;
2. Then it syncs `VMSelect` with the same manner;
3. `VMInsert` is the last object to sync.
All statefulsets are created with [OnDelete](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#on-delete)
update type. It allows to manually manage the rolling update process for Operator by deleting pods one by one and waiting
for the ready status.
Rolling update process may be configured by the operator env variables.
The most important is `VM_PODWAITREADYTIMEOUT=80s` - it controls how long to wait for pod's ready status.
## VMAgent
The `VMAgent` CRD declaratively defines a desired [VMAgent](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmagent)
setup to run in a Kubernetes cluster.
For each `VMAgent` resource Operator deploys a properly configured `Deployment` in the same namespace.
The VMAgent `Pod`s are configured to mount a `Secret` prefixed with `<VMAgent-name>` containing the configuration
for VMAgent.
For each `VMAgent` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
name `<VMAgent-name>`.
The CRD specifies which `VMServiceScrape` should be covered by the deployed VMAgent instances based on label selection.
The Operator then generates a configuration based on the included `VMServiceScrape`s and updates the `Secret` which
contains the configuration. It continuously does so for all changes that are made to the `VMServiceScrape`s or the
`VMAgent` resource itself.
If no selection of `VMServiceScrape`s is provided - Operator leaves management of the `Secret` to the user,
so user can set custom configuration while still benefiting from the Operator's capabilities of managing VMAgent setups.
## VMAlert
The `VMAlert` CRD declaratively defines a desired [VMAlert](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmalert)
setup to run in a Kubernetes cluster.
For each `VMAlert` resource, the Operator deploys a properly configured `Deployment` in the same namespace.
The VMAlert `Pod`s are configured to mount a list of `Configmaps` prefixed with `<VMAlert-name>-number` containing
the configuration for alerting rules.
For each `VMAlert` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with
name `<VMAlert-name>`.
The CRD specifies which `VMRule`s should be covered by the deployed VMAlert instances based on label selection.
The Operator then generates a configuration based on the included `VMRule`s and updates the `Configmaps` containing
the configuration. It continuously does so for all changes that are made to `VMRule`s or to the `VMAlert` resource itself.
Alerting rules are filtered by selector `ruleNamespaceSelector` in `VMAlert` CRD definition. For selecting rules from all
namespaces you must specify it to empty value:
```yaml
spec:
ruleNamespaceSelector: {}
```
## VMServiceScrape
The `VMServiceScrape` CRD allows to define a dynamic set of services for monitoring. Services
and scraping configurations can be matched via label selections. This allows an organization to introduce conventions
for how metrics should be exposed. Following these conventions new services will be discovered automatically without
need to reconfigure.
Monitoring configuration based on `discoveryRole` setting. By default, `endpoints` is used to get objects from kubernetes api.
Its also possible to use `discoveryRole: service` or `discoveryRole: endpointslices`
`Endpoints` objects are essentially lists of IP addresses.
Typically, `Endpoints` objects are populated by `Service` object. `Service` object discovers `Pod`s by a label
selector and adds those to the `Endpoints` object.
A `Service` may expose one or more service ports backed by a list of one or multiple endpoints pointing to
specific `Pod`s. The same reflected in the respective `Endpoints` object as well.
The `VMServiceScrape` object discovers `Endpoints` objects and configures VMAgent to monitor `Pod`s.
The `Endpoints` section of the `VMServiceScrapeSpec` is used to configure which `Endpoints` ports should be scraped.
For advanced use cases, one may want to monitor ports of backing `Pod`s, which are not a part of the service endpoints.
Therefore, when specifying an endpoint in the `endpoints` section, they are strictly used.
> Note: `endpoints` (lowercase) is the field in the `VMServiceScrape` CRD, while `Endpoints` (capitalized) is the Kubernetes object kind.
Both `VMServiceScrape` and discovered targets may belong to any namespace. It is important for cross-namespace monitoring
use cases, e.g. for meta-monitoring. Using the `serviceScrapeSelector` of the `VMAgentSpec`
one can restrict the namespaces from which `VMServiceScrape`s are selected from by the respective VMAgent server.
Using the `namespaceSelector` of the `VMServiceScrape` one can restrict the namespaces from which `Endpoints` can be
discovered from. To discover targets in all namespaces the `namespaceSelector` has to be empty:
```yaml
spec:
namespaceSelector: {}
```
## VMPodScrape
The `VMPodScrape` CRD allows to declaratively define how a dynamic set of pods should be monitored.
Use label selections to match pods for scraping. This allows an organization to introduce conventions
for how metrics should be exposed. Following these conventions new services will be discovered automatically without
need to reconfigure.
A `Pod` is a collection of one or more containers which can expose Prometheus metrics on a number of ports.
The `VMPodScrape` object discovers pods and generates the relevant scraping configuration.
The `PodMetricsEndpoints` section of the `VMPodScrapeSpec` is used to configure which ports of a pod are going to be
scraped for metrics and with which parameters.
Both `VMPodScrapes` and discovered targets may belong to any namespace. It is important for cross-namespace monitoring
use cases, e.g. for meta-monitoring. Using the `namespaceSelector` of the `VMPodScrapeSpec` one can restrict the
namespaces from which `Pods` are discovered from. To discover targets in all namespaces the `namespaceSelector` has to
be empty:
```yaml
spec:
namespaceSelector:
any: true
```
## VMAlertmanager
The `VMAlertmanager` CRD declaratively defines a desired Alertmanager setup to run in a Kubernetes cluster.
It provides options to configure replication and persistent storage.
For each `Alertmanager` resource, the Operator deploys a properly configured `StatefulSet` in the same namespace.
The Alertmanager pods are configured to include a `Secret` called `<alertmanager-name>` which holds the used
configuration file in the key `alertmanager.yaml`.
When there are two or more configured replicas the Operator runs the Alertmanager instances in high availability mode.
## VMAlertmanagerConfig
The `VMAlertmanagerConfig` provides way to configure `VMAlertmanager` configuration with CRD. It allows to define different configuration parts,
which will be merged by operator into config. It behaves like other config parts - `VMServiceScrape` and etc.
## VMRule
The `VMRule` CRD declaratively defines a desired Prometheus rule to be consumed by one or more VMAlert instances.
Alerts and recording rules can be saved and applied as YAML files, and dynamically loaded without requiring any restart.
## VMPrometheusConverter
By default, the Operator converts and updates existing prometheus-operator API objects:
`ServiceMonitor` into `VMServiceScrape`
`PodMonitor` into `VMPodScrape`
`PrometheusRule` into `VMRule`
`Probe` into `VMProbe`
Removing prometheus-operator API objects wouldn't delete any converted objects. So you can safely migrate or run
two operators at the same time.
## VMProbe
The `VMProbe` CRD provides probing target ability with a prober. The most common prober is [blackbox exporter](https://github.com/prometheus/blackbox_exporter).
By specifying configuration at CRD, operator generates config for `VMAgent` and syncs it. Its possible to use static targets
or use standard k8s discovery mechanism with `Ingress`.
You have to configure blackbox exporter before you can use this feature. The second requirement is `VMAgent` selectors,
it must match your `VMProbe` by label or namespace selector.
## VMNodeScrape
The `VMNodeScrape` CRD provides discovery mechanism for scraping metrics kubernetes nodes.
By specifying configuration at CRD, operator generates config for `VMAgent` and syncs it. Its useful for cadvisor scraping,
node-exporter or other node-based exporters. `VMAgent` nodeScrapeSelector must match `VMNodeScrape` labels.
## VMStaticScrape
The `VMStaticScrape` CRD provides mechanism for scraping metrics from static targets, configured by CRD targets.
By specifying configuration at CRD, operator generates config for `VMAgent` and syncs it. It's useful for external targets management,
when service-discovery is not available. `VMAgent` staticScrapeSelector must match `VMStaticScrape` labels.
## VMAuth
The `VMAuth` CRD provides mechanism for exposing application with authorization to outside world or to other applications inside kubernetes cluster.
For first case, user can configure `ingress` setting at `VMAuth` CRD. For second one, operator will create secret with `username` and `password` at `VMUser` CRD name.
So it will be possible to access this credentials from any application by targeting corresponding kubernetes secret.
## VMUser
The `VMUser` CRD describes user configuration, its authentication methods `basic auth` or `Authorization` header. User access permissions, with possible routing information.
User can define routing target with `static` config, by entering target `url`, or with `CRDRef`, in this case, operator queries kubernetes API, retrieves information about CRD and builds proper url.

322
docs/operator/high-availability.MD Normal file
View File

@ -0,0 +1,322 @@
---
sort: 7
---
# High Availability
High availability is not only important for customer-facing software but if the monitoring infrastructure is not highly available, then there is a risk that operations people are not notified for alerts. Therefore high availability must be just as thought through for the monitoring stack, as for anything else.
## VMAgent
To run VMAgent in a highly available manner you have to configure deduplication at Victoria Metrics first [doc](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/docs/Single-server-VictoriaMetrics.md#deduplication)
Then increase replicas for VMAgent.
create `VMSingle` with dedup flag
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMSingle
metadata:
name: example-vmsingle-persisted
spec:
retentionPeriod: "1"
extraArgs:
dedup.minScrapeInterval: 60s
EOF
```
create `VMAgent` with 2 replicas
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
name: example-vmagent
spec:
serviceScrapeNamespaceSelector: {}
podScrapeNamespaceSelector: {}
podScrapeSelector: {}
serviceScrapeSelector: {}
scrapeInterval: 60s
vmAgentExternalLabelName: vmagent-ha
replicaCount: 2
remoteWrite:
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
EOF
```
Sharding for `VMAgent` distributes scraping between multiple deployments of `VMAgent`.
more info https://victoriametrics.github.io/vmagent.html#scraping-big-number-of-targets
Example usage:
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
name: example-vmagent
spec:
serviceScrapeNamespaceSelector: {}
podScrapeNamespaceSelector: {}
podScrapeSelector: {}
serviceScrapeSelector: {}
scrapeInterval: 60s
vmAgentExternalLabelName: vmagent-ha
shardCount: 5
replicaCount: 2
remoteWrite:
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
EOF
```
This configuration produces 5 deployments with 2 replicas at each. Each deployment has own shard num
and scrapes only 1/5 of all targets.
## VMAlert
It can be launched with multiple replicas without an additional configuration, alertmanager is responsible for alert deduplication.
Note, if you want to use `VMAlert` with high-available `VMAlertmanager`, which has more then 1 replica. You have to specify all pod fqdns
at `VMAlert.spec.notifiers.[url]`. Or you can use service discovery for notifier, examples:
alertmanager:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: vmalertmanager-example-alertmanager
labels:
app: vm-operator
type: Opaque
stringData:
alertmanager.yaml: |
global:
resolve_timeout: 5m
route:
group_by: ['job']
group_wait: 30s
group_interval: 5m
repeat_interval: 12h
receiver: 'webhook'
receivers:
- name: 'webhook'
webhook_configs:
- url: 'http://alertmanagerwh:30500/'
---
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAlertmanager
metadata:
name: example
namespace: default
labels:
usage: dedicated
spec:
replicaCount: 2
configSecret: vmalertmanager-example-alertmanager
configSelector: {}
configNamespaceSelector: {}
```
vmalert with fqdns:
```yaml
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAlert
metadata:
name: example-ha
namespace: default
spec:
datasource:
url: http://vmsingle-example.default.svc:8429
notifiers:
- url: http://vmalertmanager-example-0.vmalertmanager-example.default.svc:9093
- url: http://vmalertmanager-example-1.vmalertmanager-example.default.svc:9093
```
vmalert with service discovery:
```yaml
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAlert
metadata:
name: example-ha
namespace: default
spec:
datasource:
url: http://vmsingle-example.default.svc:8429
notifiers:
- selector:
namespaceSelector:
matchNames:
- default
labelSelector:
matchLabels:
usage: dedicated
```
## VMSingle
It doesn't support high availability by default, for such purpose use VMCluster or duplicate the setup.
## VMCluster
Cluster version provides a full set of high availability features - metrics replication, node failover, horizontal scaling.
For using cluster version you have to create corresponding CRD object:
```yaml
cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMCluster
metadata:
name: example-vmcluster-persistent
spec:
retentionPeriod: "4"
replicationFactor: 2
vmstorage:
replicaCount: 2
storageDataPath: "/vm-data"
podMetadata:
labels:
owner: infra
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: "app.kubernetes.io/name"
operator: In
values:
- "vmstorage"
topologyKey: "kubernetes.io/hostname"
storage:
volumeClaimTemplate:
spec:
resources:
requests:
storage: 10Gi
resources:
limits:
cpu: "2"
memory: 2048Mi
vmselect:
replicaCount: 2
cacheMountPath: "/select-cache"
podMetadata:
labels:
owner: infra
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: "app.kubernetes.io/name"
operator: In
values:
- "vmselect"
topologyKey: "kubernetes.io/hostname"
storage:
volumeClaimTemplate:
spec:
resources:
requests:
storage: 2Gi
resources:
limits:
cpu: "1"
memory: "500Mi"
vminsert:
replicaCount: 2
podMetadata:
labels:
owner: infra
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: "app.kubernetes.io/name"
operator: In
values:
- "vminsert"
topologyKey: "kubernetes.io/hostname"
resources:
limits:
cpu: "1"
memory: "500Mi"
EOF
```
Then wait for the cluster becomes ready
```bash
kubectl get vmclusters -w
NAME INSERT COUNT STORAGE COUNT SELECT COUNT AGE STATUS
example-vmcluster-persistent 2 2 2 2s expanding
example-vmcluster-persistent 2 2 2 30s operational
```
Get links for connection by executing command:
```bash
kubectl get svc -l app.kubernetes.io/instance=example-vmcluster-persistent
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
vminsert-example-vmcluster-persistent ClusterIP 10.96.34.94 <none> 8480/TCP 69s
vmselect-example-vmcluster-persistent ClusterIP None <none> 8481/TCP 79s
vmstorage-example-vmcluster-persistent ClusterIP None <none> 8482/TCP,8400/TCP,8401/TCP 85s
```
Now you can connect vmagent to vminsert and vmalert to vmselect
>NOTE do not forget to create rbac for vmagent
```yaml
cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
name: example-vmagent
spec:
serviceScrapeNamespaceSelector: {}
serviceScrapeSelector: {}
podScrapeNamespaceSelector: {}
podScrapeSelector: {}
# Add fields here
replicaCount: 1
remoteWrite:
- url: "http://vminsert-example-vmcluster-persistent.default.svc.cluster.local:8480/insert/0/prometheus/api/v1/write"
EOF
```
Config for vmalert
```yaml
cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAlert
metadata:
name: example-vmalert
spec:
# Add fields here
replicas: 1
datasource:
url: "http://vmselect-example-vmcluster-persistent.default.svc.cluster.local:8481/select/0/prometheus"
notifier:
url: "http://alertmanager-operated.default.svc:9093"
evaluationInterval: "10s"
ruleSelector: {}
EOF
```
## Alertmanager
The final step of the high availability scheme is Alertmanager, when an alert triggers, actually fires alerts against *all* instances of an Alertmanager cluster.
The Alertmanager, starting with the `v0.5.0` release, ships with a high availability mode. It implements a gossip protocol to synchronize instances of an Alertmanager cluster regarding notifications that have been sent out, to prevent duplicate notifications. It is an AP (available and partition tolerant) system. Being an AP system means that notifications are guaranteed to be sent at least once.
The Victoria Metrics Operator ensures that Alertmanager clusters are properly configured to run highly available on Kubernetes.

BIN
docs/operator/logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 15 KiB

82
docs/operator/managing-versions.MD Normal file
View File

@ -0,0 +1,82 @@
---
sort: 8
---
# VMAlert, VMAgent, VMAlertmanager, VMSingle version
for those objects you can specify following settings at `spec.Image`
for instance, to set `VMSingle` version add `spec.image.tag` name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases)
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMSingle
metadata:
name: example-vmsingle
spec:
image:
repository: victoriametrics/victoria-metrics
tag: v1.39.2
pullPolicy: Always
retentionPeriod: "1"
EOF
```
Also, you can specify `imagePullSecrets` if you are pulling images from private repo:
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMSingle
metadata:
name: example-vmsingle
spec:
imagePullSecrets:
- name: my-repo-secret
image:
repository: my-repo-url/victoria-metrics
tag: v1.39.2
retentionPeriod: "1"
EOF
```
# VMCluster
for `VMCluster` you can specify tag and repository setting per cluster object.
But `imagePullSecrets` is global setting for all `VMCluster` specification.
```yaml
cat << EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMCluster
metadata:
name: example-vmcluster
spec:
imagePullSecrets:
- name: my-repo-secret
# Add fields here
retentionPeriod: "1"
vmstorage:
replicaCount: 2
image:
repository: victoriametrics/vmstorage
tag: v1.39.2-cluster
pullPolicy: Always
vmselect:
replicaCount: 2
image:
repository: victoriametrics/vmselect
tag: v1.39.2-cluster
pullPolicy: Always
vminsert:
replicaCount: 2
image:
repository: victoriametrics/vminsert
tag: v1.39.2-cluster
pullPolicy: Always
EOF
```

1519
docs/operator/quick-start.MD Normal file
View File

File diff suppressed because it is too large Load Diff

240
docs/operator/relabeling.MD Normal file
View File

@ -0,0 +1,240 @@
---
sort: 10
---
# VMAgent relabel
`VMAgent` supports global relabeling for all metrics and per remoteWrite target relabel config.
> Note in some cases, you don't need relabeling,
> key=value label pairs can be added to the all scrapped metrics with `spec.externalLabels` for `VMAgent`.
>
```yaml
# simple label add config
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
name: stack
spec:
externalLabels:
clusterid: some_cluster
```
It supports relabeling with custom configMap or inline defined at CRD
## Configmap example
Quick tour how to to create `Confimap` with relabeling configuration
```yaml
cat << EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: vmagent-relabel
data:
global-relabel.yaml: |
- target_label: bar
- source_labels: [aa]
separator: "foobar"
regex: "foo.+bar"
target_label: aaa
replacement: "xxx"
- action: keep
source_labels: [aaa]
- action: drop
source_labels: [aaa]
target-1-relabel.yaml: |
- action: keep_if_equal
source_labels: [foo, bar]
- action: drop_if_equal
source_labels: [foo, bar]
EOF
```
Second, add `relabelConfig` to `VMagent` spec for global relabeling with name of `Configmap` - `vmagent-relabel` and key `global-relabel.yaml`.
For relabeling per remoteWrite target, add `urlRelabelConfig` name of `Configmap` - `vmagent-relabel` and key `target-1-relabel.yaml` to one of remoteWrite target for relabeling only
for those target.
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
name: example-vmagent
spec:
serviceScrapeNamespaceSelector: {}
podScrapeNamespaceSelector: {}
podScrapeSelector: {}
serviceScrapeSelector: {}
replicaCount: 1
serviceAccountName: vmagent
relabelConfig:
name: "vmagent-relabel"
key: "global-relabel.yaml"
remoteWrite:
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
- url: "http://vmsingle-example-vmsingle.default.svc:8429/api/v1/write"
urlRelabelConfig:
name: "vmagent-relabel"
key: "target-1-relabel.yaml"
EOF
```
## Inline example
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
name: example-vmagent
spec:
serviceScrapeNamespaceSelector: {}
podScrapeNamespaceSelector: {}
podScrapeSelector: {}
serviceScrapeSelector: {}
replicaCount: 1
serviceAccountName: vmagent
inlineRelabelConfig:
- target_label: bar
- source_labels: [aa]
separator: "foobar"
regex: "foo.+bar"
target_label: aaa
replacement: "xxx"
- action: keep
source_labels: [aaa]
- action: drop
source_labels: [aaa]
remoteWrite:
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
- url: "http://vmsingle-example-vmsingle.default.svc:8429/api/v1/write"
inlineUrlRelabelConfig:
- action: keep_if_equal
source_labels: [foo, bar]
- action: drop_if_equal
source_labels: [foo, bar]
EOF
```
## Combined example
Its also possible to use both features in combination.
First will be added relabeling configs from `inlineRelabelConfig`, then `relabelConfig` from configmap.
```yaml
cat << EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: vmagent-relabel
data:
global-relabel.yaml: |
- target_label: bar
- source_labels: [aa]
separator: "foobar"
regex: "foo.+bar"
target_label: aaa
replacement: "xxx"
- action: keep
source_labels: [aaa]
- action: drop
source_labels: [aaa]
target-1-relabel.yaml: |
- action: keep_if_equal
source_labels: [foo, bar]
- action: drop_if_equal
source_labels: [foo, bar]
EOF
```
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
name: example-vmagent
spec:
serviceScrapeNamespaceSelector: {}
podScrapeNamespaceSelector: {}
podScrapeSelector: {}
serviceScrapeSelector: {}
replicaCount: 1
serviceAccountName: vmagent
inlineRelabelConfig:
- target_label: bar1
- source_labels: [aa]
relabelConfig:
name: "vmagent-relabel"
key: "global-relabel.yaml"
remoteWrite:
- url: "http://vmsingle-example-vmsingle-persisted.default.svc:8429/api/v1/write"
- url: "http://vmsingle-example-vmsingle.default.svc:8429/api/v1/write"
urlRelabelConfig:
name: "vmagent-relabel"
key: "target-1-relabel.yaml"
inlineUrlRelabelConfig:
- action: keep_if_equal
source_labels: [foo1, bar2]
EOF
```
Resulted configmap:
```yaml
apiVersion: v1
data:
global_relabeling.yaml: |
- target_label: bar1
- source_labels:
- aa
- target_label: bar
- source_labels: [aa]
separator: "foobar"
regex: "foo.+bar"
target_label: aaa
replacement: "xxx"
- action: keep
source_labels: [aaa]
- action: drop
source_labels: [aaa]
url_rebaling-1.yaml: |
- source_labels:
- foo1
- bar2
action: keep_if_equal
- action: keep_if_equal
source_labels: [foo, bar]
- action: drop_if_equal
source_labels: [foo, bar]
kind: ConfigMap
metadata:
finalizers:
- apps.victoriametrics.com/finalizer
labels:
app.kubernetes.io/component: monitoring
app.kubernetes.io/instance: example-vmagent
app.kubernetes.io/name: vmagent
managed-by: vm-operator
name: relabelings-assets-vmagent-example-vmagent
namespace: default
ownerReferences:
- apiVersion: operator.victoriametrics.com/v1beta1
blockOwnerDeletion: true
controller: true
kind: VMAgent
name: example-vmagent
uid: 7e9fb838-65da-4443-a43b-c00cd6c4db5b
```
## Additional information
`VMAgent` also has some extra options for relabeling actions, you can check it [docs](https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmagent/README.md#relabeling)

36
docs/operator/resources-validation.MD Normal file
View File

@ -0,0 +1,36 @@
---
sort: 11
---
# CRD Validation
## Description
Operator supports validation admission webhook [docs](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/)
It checks resources configuration and returns errors to caller before resource will be created at kubernetes api.
This should reduce errors and simplify debugging.
## Configuration
Validation hooks at operator side must be enabled with flags:
```
--webhook.enable
# optional configuration for certDir and tls names.
--webhook.certDir=/tmp/k8s-webhook-server/serving-certs/
--webhook.keyName=tls.key
--webhook.certName=tls.crt
```
You have to mount correct certificates at give directory.
It can be simplified with cert-manager and kustomize command: `kustomize build config/deployments/webhook/ `
## Requirements
- Valid certificate with key must be provided to operator
- Valid CABundle must be added to the `ValidatingWebhookConfiguration`
## Useful links
- [k8s admission webhooks](https://banzaicloud.com/blog/k8s-admission-webhooks/)
- [olm webhooks](https://docs.openshift.com/container-platform/4.5/operators/user/olm-webhooks.html)

85
docs/operator/security.MD Normal file
View File

@ -0,0 +1,85 @@
---
sort: 12
---
# Security
VictoriaMetrics operator provides several security features, such as [PodSecurityPolicies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/), [PodSecurityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/).
## PodSecurityPolicy.
By default, operator creates serviceAccount for each cluster resource and binds default `PodSecurityPolicy` to it.
Default psp:
```yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: vmagent-example-vmagent
spec:
allowPrivilegeEscalation: false
fsGroup:
rule: RunAsAny
hostNetwork: true
requiredDropCapabilities:
- ALL
runAsUser:
rule: RunAsAny
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
volumes:
- persistentVolumeClaim
- secret
- emptyDir
- configMap
- projected
- downwardAPI
- nfs
```
This behaviour may be disabled with env variable passed to operator:
```yaml
- name: VM_PSPAUTOCREATEENABLED
value: "false"
```
User may also override default pod security policy with setting: `spec.podSecurityPolicyName: "psp-name"`.
## PodSecurityContext
`PodSecurityContext` can be configured with spec setting. It may be useful for mounted volumes, with `VMSingle` for example:
```yaml
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMSingle
metadata:
name: vmsingle-f
namespace: monitoring-system
spec:
retentionPeriod: "2"
removePvcAfterDelete: true
securityContext:
runAsUser: 1000
fsGroup: 1000
runAsGroup: 1000
extraArgs:
dedup.minScrapeInterval: 10s
storage:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 25Gi
resources:
requests:
cpu: "0.5"
memory: "512Mi"
limits:
cpu: "1"
memory: "1512Mi"
```

115
docs/operator/vars.MD Normal file
View File

@ -0,0 +1,115 @@
---
sort: 13
---
# Auto Generated vars for package config
**_updated at Tue Oct 12 07:44:08 UTC 2021_**
| varible name | variable default value | variable required | variable description |
| --- | --- | --- | --- |
| VM_USECUSTOMCONFIGRELOADER | false | false | enables custom config reloader for vmauth and vmagent,it should speed-up config reloading process. |
| VM_CUSTOMCONFIGRELOADERIMAGE | victoriametrics/operator:config-reloader-0.1.0 | false | - |
| VM_PSPAUTOCREATEENABLED | true | false | - |
| VM_VMALERTDEFAULT_IMAGE | victoriametrics/vmalert | false | - |
| VM_VMALERTDEFAULT_VERSION | v1.67.0 | false | - |
| VM_VMALERTDEFAULT_PORT | 8080 | false | - |
| VM_VMALERTDEFAULT_USEDEFAULTRESOURCES | true | false | - |
| VM_VMALERTDEFAULT_RESOURCE_LIMIT_MEM | 500Mi | false | - |
| VM_VMALERTDEFAULT_RESOURCE_LIMIT_CPU | 200m | false | - |
| VM_VMALERTDEFAULT_RESOURCE_REQUEST_MEM | 200Mi | false | - |
| VM_VMALERTDEFAULT_RESOURCE_REQUEST_CPU | 50m | false | - |
| VM_VMALERTDEFAULT_CONFIGRELOADERCPU | 100m | false | - |
| VM_VMALERTDEFAULT_CONFIGRELOADERMEMORY | 25Mi | false | - |
| VM_VMALERTDEFAULT_CONFIGRELOADIMAGE | jimmidyson/configmap-reload:v0.3.0 | false | - |
| VM_VMAGENTDEFAULT_IMAGE | victoriametrics/vmagent | false | - |
| VM_VMAGENTDEFAULT_VERSION | v1.67.0 | false | - |
| VM_VMAGENTDEFAULT_CONFIGRELOADIMAGE | quay.io/prometheus-operator/prometheus-config-reloader:v0.48.1 | false | - |
| VM_VMAGENTDEFAULT_PORT | 8429 | false | - |
| VM_VMAGENTDEFAULT_USEDEFAULTRESOURCES | true | false | - |
| VM_VMAGENTDEFAULT_RESOURCE_LIMIT_MEM | 500Mi | false | - |
| VM_VMAGENTDEFAULT_RESOURCE_LIMIT_CPU | 200m | false | - |
| VM_VMAGENTDEFAULT_RESOURCE_REQUEST_MEM | 200Mi | false | - |
| VM_VMAGENTDEFAULT_RESOURCE_REQUEST_CPU | 50m | false | - |
| VM_VMAGENTDEFAULT_CONFIGRELOADERCPU | 100m | false | - |
| VM_VMAGENTDEFAULT_CONFIGRELOADERMEMORY | 25Mi | false | - |
| VM_VMSINGLEDEFAULT_IMAGE | victoriametrics/victoria-metrics | false | - |
| VM_VMSINGLEDEFAULT_VERSION | v1.67.0 | false | - |
| VM_VMSINGLEDEFAULT_PORT | 8429 | false | - |
| VM_VMSINGLEDEFAULT_USEDEFAULTRESOURCES | true | false | - |
| VM_VMSINGLEDEFAULT_RESOURCE_LIMIT_MEM | 1500Mi | false | - |
| VM_VMSINGLEDEFAULT_RESOURCE_LIMIT_CPU | 1200m | false | - |
| VM_VMSINGLEDEFAULT_RESOURCE_REQUEST_MEM | 500Mi | false | - |
| VM_VMSINGLEDEFAULT_RESOURCE_REQUEST_CPU | 150m | false | - |
| VM_VMSINGLEDEFAULT_CONFIGRELOADERCPU | 100m | false | - |
| VM_VMSINGLEDEFAULT_CONFIGRELOADERMEMORY | 25Mi | false | - |
| VM_VMCLUSTERDEFAULT_USEDEFAULTRESOURCES | true | false | - |
| VM_VMCLUSTERDEFAULT_VMSELECTDEFAULT_IMAGE | victoriametrics/vmselect | false | - |
| VM_VMCLUSTERDEFAULT_VMSELECTDEFAULT_VERSION | v1.67.0-cluster | false | - |
| VM_VMCLUSTERDEFAULT_VMSELECTDEFAULT_PORT | 8481 | false | - |
| VM_VMCLUSTERDEFAULT_VMSELECTDEFAULT_RESOURCE_LIMIT_MEM | 1000Mi | false | - |
| VM_VMCLUSTERDEFAULT_VMSELECTDEFAULT_RESOURCE_LIMIT_CPU | 500m | false | - |
| VM_VMCLUSTERDEFAULT_VMSELECTDEFAULT_RESOURCE_REQUEST_MEM | 500Mi | false | - |
| VM_VMCLUSTERDEFAULT_VMSELECTDEFAULT_RESOURCE_REQUEST_CPU | 100m | false | - |
| VM_VMCLUSTERDEFAULT_VMSTORAGEDEFAULT_IMAGE | victoriametrics/vmstorage | false | - |
| VM_VMCLUSTERDEFAULT_VMSTORAGEDEFAULT_VERSION | v1.67.0-cluster | false | - |
| VM_VMCLUSTERDEFAULT_VMSTORAGEDEFAULT_VMINSERTPORT | 8400 | false | - |
| VM_VMCLUSTERDEFAULT_VMSTORAGEDEFAULT_VMSELECTPORT | 8401 | false | - |
| VM_VMCLUSTERDEFAULT_VMSTORAGEDEFAULT_PORT | 8482 | false | - |
| VM_VMCLUSTERDEFAULT_VMSTORAGEDEFAULT_RESOURCE_LIMIT_MEM | 1500Mi | false | - |
| VM_VMCLUSTERDEFAULT_VMSTORAGEDEFAULT_RESOURCE_LIMIT_CPU | 1000m | false | - |
| VM_VMCLUSTERDEFAULT_VMSTORAGEDEFAULT_RESOURCE_REQUEST_MEM | 500Mi | false | - |
| VM_VMCLUSTERDEFAULT_VMSTORAGEDEFAULT_RESOURCE_REQUEST_CPU | 250m | false | - |
| VM_VMCLUSTERDEFAULT_VMINSERTDEFAULT_IMAGE | victoriametrics/vminsert | false | - |
| VM_VMCLUSTERDEFAULT_VMINSERTDEFAULT_VERSION | v1.67.0-cluster | false | - |
| VM_VMCLUSTERDEFAULT_VMINSERTDEFAULT_PORT | 8480 | false | - |
| VM_VMCLUSTERDEFAULT_VMINSERTDEFAULT_RESOURCE_LIMIT_MEM | 500Mi | false | - |
| VM_VMCLUSTERDEFAULT_VMINSERTDEFAULT_RESOURCE_LIMIT_CPU | 500m | false | - |
| VM_VMCLUSTERDEFAULT_VMINSERTDEFAULT_RESOURCE_REQUEST_MEM | 200Mi | false | - |
| VM_VMCLUSTERDEFAULT_VMINSERTDEFAULT_RESOURCE_REQUEST_CPU | 150m | false | - |
| VM_VMALERTMANAGER_CONFIGRELOADERIMAGE | jimmidyson/configmap-reload:v0.3.0 | false | - |
| VM_VMALERTMANAGER_CONFIGRELOADERCPU | 100m | false | - |
| VM_VMALERTMANAGER_CONFIGRELOADERMEMORY | 25Mi | false | - |
| VM_VMALERTMANAGER_ALERTMANAGERDEFAULTBASEIMAGE | prom/alertmanager | false | - |
| VM_VMALERTMANAGER_ALERTMANAGERVERSION | v0.22.2 | false | - |
| VM_VMALERTMANAGER_LOCALHOST | 127.0.0.1 | false | - |
| VM_VMALERTMANAGER_USEDEFAULTRESOURCES | true | false | - |
| VM_VMALERTMANAGER_RESOURCE_LIMIT_MEM | 256Mi | false | - |
| VM_VMALERTMANAGER_RESOURCE_LIMIT_CPU | 100m | false | - |
| VM_VMALERTMANAGER_RESOURCE_REQUEST_MEM | 56Mi | false | - |
| VM_VMALERTMANAGER_RESOURCE_REQUEST_CPU | 30m | false | - |
| VM_DISABLESELFSERVICESCRAPECREATION | false | false | - |
| VM_VMBACKUP_IMAGE | victoriametrics/vmbackupmanager | false | - |
| VM_VMBACKUP_VERSION | v1.67.0-enterprise | false | - |
| VM_VMBACKUP_PORT | 8300 | false | - |
| VM_VMBACKUP_USEDEFAULTRESOURCES | true | false | - |
| VM_VMBACKUP_RESOURCE_LIMIT_MEM | 500Mi | false | - |
| VM_VMBACKUP_RESOURCE_LIMIT_CPU | 500m | false | - |
| VM_VMBACKUP_RESOURCE_REQUEST_MEM | 200Mi | false | - |
| VM_VMBACKUP_RESOURCE_REQUEST_CPU | 150m | false | - |
| VM_VMBACKUP_LOGLEVEL | INFO | false | - |
| VM_VMAUTHDEFAULT_IMAGE | victoriametrics/vmauth | false | - |
| VM_VMAUTHDEFAULT_VERSION | v1.67.0 | false | - |
| VM_VMAUTHDEFAULT_CONFIGRELOADIMAGE | quay.io/prometheus-operator/prometheus-config-reloader:v0.48.1 | false | - |
| VM_VMAUTHDEFAULT_PORT | 8427 | false | - |
| VM_VMAUTHDEFAULT_USEDEFAULTRESOURCES | true | false | - |
| VM_VMAUTHDEFAULT_RESOURCE_LIMIT_MEM | 300Mi | false | - |
| VM_VMAUTHDEFAULT_RESOURCE_LIMIT_CPU | 200m | false | - |
| VM_VMAUTHDEFAULT_RESOURCE_REQUEST_MEM | 100Mi | false | - |
| VM_VMAUTHDEFAULT_RESOURCE_REQUEST_CPU | 50m | false | - |
| VM_VMAUTHDEFAULT_CONFIGRELOADERCPU | 100m | false | - |
| VM_VMAUTHDEFAULT_CONFIGRELOADERMEMORY | 25Mi | false | - |
| VM_ENABLEDPROMETHEUSCONVERTER_PODMONITOR | true | false | - |
| VM_ENABLEDPROMETHEUSCONVERTER_SERVICESCRAPE | true | false | - |
| VM_ENABLEDPROMETHEUSCONVERTER_PROMETHEUSRULE | true | false | - |
| VM_ENABLEDPROMETHEUSCONVERTER_PROBE | true | false | - |
| VM_ENABLEDPROMETHEUSCONVERTEROWNERREFERENCES | false | false | - |
| VM_HOST | 0.0.0.0 | false | - |
| VM_LISTENADDRESS | 0.0.0.0 | false | - |
| VM_DEFAULTLABELS | managed-by=vm-operator | false | - |
| VM_LABELS | - | false | - |
| VM_CLUSTERDOMAINNAME | "" | false | Will use DNS search paths if unset |
| VM_PODWAITREADYTIMEOUT | 80s | false | - |
| VM_PODWAITREADYINTERVALCHECK | 5s | false | - |
| VM_PODWAITREADYINITDELAY | 10s | false | - |