From 7b35eaa8537e56c6d4165c3ee53e269aa76cc37e Mon Sep 17 00:00:00 2001 From: Alexander Marshalov <_@marshalov.org> Date: Tue, 3 Oct 2023 14:28:33 +0200 Subject: [PATCH] hide deprecated docs from menu (#5095) --- docs/ExtendedPromQL.md | 2 +- docs/_includes/extra/styles.scss | 5 + docs/operator/VictoriaMetrics-Operator.md | 83 +----- docs/operator/additional-scrape.md | 87 +----- docs/operator/backups.md | 153 +---------- docs/operator/configuration_syncronization.md | 46 +--- docs/operator/design.md | 238 +---------------- docs/operator/managing-versions.md | 90 +------ docs/operator/relabeling.md | 247 +----------------- docs/operator/resources-validation.md | 41 +-- docs/operator/vmalertmanager-configuration.md | 150 +---------- 11 files changed, 37 insertions(+), 1105 deletions(-) diff --git a/docs/ExtendedPromQL.md b/docs/ExtendedPromQL.md index 80208247a..7a3bbecd5 100644 --- a/docs/ExtendedPromQL.md +++ b/docs/ExtendedPromQL.md @@ -1,5 +1,5 @@ --- -sort: 100 +sort: -1 # hide page from menu weight: 100 --- diff --git a/docs/_includes/extra/styles.scss b/docs/_includes/extra/styles.scss index 97597e825..e42980ed1 100644 --- a/docs/_includes/extra/styles.scss +++ b/docs/_includes/extra/styles.scss @@ -37,6 +37,11 @@ color: #b35cec !important; } +// hidden in menu pages +[data-sort="-1"] { + display: none; +} + // Content .markdown-body h1, .markdown-body h2, diff --git a/docs/operator/VictoriaMetrics-Operator.md b/docs/operator/VictoriaMetrics-Operator.md index a0d523d55..5f62dfee6 100644 --- a/docs/operator/VictoriaMetrics-Operator.md +++ b/docs/operator/VictoriaMetrics-Operator.md @@ -1,86 +1,9 @@ --- -sort: 1 -weight: 1 -title: VictoriaMetrics Operator Overview -menu: - docs: - parent: "operator" - weight: 1 +sort: -1 # hide page from menu aliases: - /operator/VictoriaMetrics-Operator.html --- -# VictoriaMetrics Operator Overview +This page is deprecated. -## Overview - -Design and implementation inspired by [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator). It's great a tool for managing monitoring configuration of your applications. VictoriaMetrics operator has api capability with it. -So you can use familiar CRD objects: `ServiceMonitor`, `PodMonitor`, `PrometheusRule` and `Probe`. Or you can use VictoriaMetrics CRDs: -- `VMServiceScrape` - defines scraping metrics configuration from pods backed by services. -- `VMPodScrape` - defines scraping metrics configuration from pods. -- `VMRule` - defines alerting or recording rules. -- `VMProbe` - defines a probing configuration for targets with blackbox exporter. - -Besides, operator allows you to manage VictoriaMetrics applications inside kubernetes cluster and simplifies this process [quick-start](https://docs.victoriametrics.com/operator/quick-start.html) -With CRD (Custom Resource Definition) you can define application configuration and apply it to your cluster [crd-objects](https://docs.victoriametrics.com/operator/api.html). - -Operator simplifies VictoriaMetrics cluster installation, upgrading and managing. - -It has integration with VictoriaMetrics `vmbackupmanager` - advanced tools for making backups. Check backup [docs](https://docs.victoriametrics.com/operator/backups.html) - -## Use cases - -For kubernetes-cluster administrators, it simplifies installation, configuration and management for `VictoriaMetrics` application. The main feature of operator is its ability to delegate the configuration of applications monitoring to the end-users. - -For applications developers, its great possibility for managing observability of applications. You can define metrics scraping and alerting configuration for your application and manage it with an application deployment process. Just define app_deployment.yaml, app_vmpodscrape.yaml and app_vmrule.yaml. That's it, you can apply it to a kubernetes cluster. Check [quick-start](/operator/quick-start.html) for an example. - -## Operator vs helm-chart - -VictoriaMetrics provides [helm charts](https://github.com/VictoriaMetrics/helm-charts). Operator makes the same, simplifies it and provides advanced features. - - -## Configuration - -Operator configured by env variables, list of it can be found at [link](https://docs.victoriametrics.com/operator/vars.html) - -It defines default configuration options, like images for components, timeouts, features. - - -## Kubernetes' compatibility versions - -Operator tested at kubernetes versions from 1.16 to 1.22. - -For clusters version below 1.16 you must use legacy CRDs from [path](https://github.com/VictoriaMetrics/operator/tree/master/config/crd/legacy) -and disable CRD controller with flag: `--controller.disableCRDOwnership=true` - -## Troubleshooting - -- cannot apply crd at kubernetes 1.18 + version and kubectl reports error: -```console -Error from server (Invalid): error when creating "release/crds/crd.yaml": CustomResourceDefinition.apiextensions.k8s.io "vmalertmanagers.operator.victoriametrics.com" is invalid: [spec.validation.openAPIV3Schema.properties[spec].properties[initContainers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property] -Error from server (Invalid): error when creating "release/crds/crd.yaml": CustomResourceDefinition.apiextensions.k8s.io "vmalerts.operator.victoriametrics.com" is invalid: [ -``` - upgrade to the latest release version. There is a bug with kubernetes objects at the early releases. - - -## Development - -- [operator-sdk](https://github.com/operator-framework/operator-sdk) version v1.0.0+ -- golang 1.15 + -- minikube or kind - -start: -```console -make run -``` - -for test execution run: -```console -#unit tests - -make test - -# you need minikube or kind for e2e, do not run it on live cluster -#e2e tests with local binary -make e2e-local -``` +Please see [Operator page](./README.md) instead. diff --git a/docs/operator/additional-scrape.md b/docs/operator/additional-scrape.md index cfa1d4bcf..3fb0896e5 100644 --- a/docs/operator/additional-scrape.md +++ b/docs/operator/additional-scrape.md @@ -1,89 +1,10 @@ --- -sort: 2 -weight: 2 -title: Additional Scrape Configuration -menu: - docs: - parent: "operator" - weight: 2 +sort: -1 # hide page from menu aliases: - /operator/additional-scrape.html --- -# Additional Scrape Configuration - -AdditionalScrapeConfigs is an additional way to add scrape targets in VMAgent CRD. -There are two options for adding targets into VMAgent: inline configuration into CRD or defining it as a Kubernetes Secret. - -No validation happens during the creation of configuration. However, you must validate job specs, and it must follow job spec configuration. -Please check [scrape_configs documentation](https://docs.victoriametrics.com/sd_configs.html#scrape_configs) as references. - -## Inline Additional Scrape Configuration in VMAgent CRD - -You need to add scrape configuration directly to the vmagent spec.inlineScrapeConfig. It is raw text in YAML format. -See example below - -```yaml -cat <`. - -## VMCluster - -The `VMCluster` CRD defines a [cluster version VM](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster). - -For each `VMCluster` resource, the Operator creates `VMStorage` as `StatefulSet`, `VMSelect` as `StatefulSet` and `VMInsert` -as deployment. For `VMStorage` and `VMSelect` headless services are created. `VMInsert` is created as service with clusterIP. - -There is a strict order for these objects creation and reconciliation: -1. `VMStorage` is synced - the Operator waits until all its pods are ready; -1. Then it syncs `VMSelect` with the same manner; -1. `VMInsert` is the last object to sync. - -All statefulsets are created with [OnDelete](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#on-delete) -update type. It allows to manually manage the rolling update process for Operator by deleting pods one by one and waiting -for the ready status. - -Rolling update process may be configured by the operator env variables. -The most important is `VM_PODWAITREADYTIMEOUT=80s` - it controls how long to wait for pod's ready status. - -## VMAgent - -The `VMAgent` CRD declaratively defines a desired [VMAgent](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmagent) -setup to run in a Kubernetes cluster. - -For each `VMAgent` resource Operator deploys a properly configured `Deployment` in the same namespace. -The VMAgent `Pod`s are configured to mount a `Secret` prefixed with `` containing the configuration -for VMAgent. - -For each `VMAgent` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with -name ``. - -The CRD specifies which `VMServiceScrape` should be covered by the deployed VMAgent instances based on label selection. -The Operator then generates a configuration based on the included `VMServiceScrape`s and updates the `Secret` which -contains the configuration. It continuously does so for all changes that are made to the `VMServiceScrape`s or the -`VMAgent` resource itself. - -If no selection of `VMServiceScrape`s is provided - Operator leaves management of the `Secret` to the user, -so user can set custom configuration while still benefiting from the Operator's capabilities of managing VMAgent setups. - -## VMAlert - -The `VMAlert` CRD declaratively defines a desired [VMAlert](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmalert) -setup to run in a Kubernetes cluster. - -For each `VMAlert` resource, the Operator deploys a properly configured `Deployment` in the same namespace. -The VMAlert `Pod`s are configured to mount a list of `Configmaps` prefixed with `-number` containing -the configuration for alerting rules. - -For each `VMAlert` resource, the Operator adds `Service` and `VMServiceScrape` in the same namespace prefixed with -name ``. - -The CRD specifies which `VMRule`s should be covered by the deployed VMAlert instances based on label selection. -The Operator then generates a configuration based on the included `VMRule`s and updates the `Configmaps` containing -the configuration. It continuously does so for all changes that are made to `VMRule`s or to the `VMAlert` resource itself. - -Alerting rules are filtered by selector `ruleNamespaceSelector` in `VMAlert` CRD definition. For selecting rules from all -namespaces you must specify it to empty value: - -```yaml -spec: - ruleNamespaceSelector: {} -``` - - -## VMServiceScrape - -The `VMServiceScrape` CRD allows to define a dynamic set of services for monitoring. Services -and scraping configurations can be matched via label selections. This allows an organization to introduce conventions -for how metrics should be exposed. Following these conventions new services will be discovered automatically without -need to reconfigure. - -Monitoring configuration based on `discoveryRole` setting. By default, `endpoints` is used to get objects from kubernetes api. -It's also possible to use `discoveryRole: service` or `discoveryRole: endpointslices` - - `Endpoints` objects are essentially lists of IP addresses. -Typically, `Endpoints` objects are populated by `Service` object. `Service` object discovers `Pod`s by a label -selector and adds those to the `Endpoints` object. - -A `Service` may expose one or more service ports backed by a list of one or multiple endpoints pointing to -specific `Pod`s. The same reflected in the respective `Endpoints` object as well. - -The `VMServiceScrape` object discovers `Endpoints` objects and configures VMAgent to monitor `Pod`s. - -The `Endpoints` section of the `VMServiceScrapeSpec` is used to configure which `Endpoints` ports should be scraped. -For advanced use cases, one may want to monitor ports of backing `Pod`s, which are not a part of the service endpoints. -Therefore, when specifying an endpoint in the `endpoints` section, they are strictly used. - -> Note: `endpoints` (lowercase) is the field in the `VMServiceScrape` CRD, while `Endpoints` (capitalized) is the Kubernetes object kind. - -Both `VMServiceScrape` and discovered targets may belong to any namespace. It is important for cross-namespace monitoring -use cases, e.g. for meta-monitoring. Using the `serviceScrapeSelector` of the `VMAgentSpec` -one can restrict the namespaces from which `VMServiceScrape`s are selected from by the respective VMAgent server. -Using the `namespaceSelector` of the `VMServiceScrape` one can restrict the namespaces from which `Endpoints` can be -discovered from. To discover targets in all namespaces the `namespaceSelector` has to be empty: - -```yaml -spec: - namespaceSelector: {} -``` - -## VMPodScrape - -The `VMPodScrape` CRD allows to declaratively define how a dynamic set of pods should be monitored. -Use label selections to match pods for scraping. This allows an organization to introduce conventions -for how metrics should be exposed. Following these conventions new services will be discovered automatically without -need to reconfigure. - -A `Pod` is a collection of one or more containers which can expose Prometheus metrics on a number of ports. - -The `VMPodScrape` object discovers pods and generates the relevant scraping configuration. - -The `PodMetricsEndpoints` section of the `VMPodScrapeSpec` is used to configure which ports of a pod are going to be -scraped for metrics and with which parameters. - -Both `VMPodScrapes` and discovered targets may belong to any namespace. It is important for cross-namespace monitoring -use cases, e.g. for meta-monitoring. Using the `namespaceSelector` of the `VMPodScrapeSpec` one can restrict the -namespaces from which `Pods` are discovered from. To discover targets in all namespaces the `namespaceSelector` has to -be empty: - -```yaml -spec: - namespaceSelector: - any: true -``` - -## VMAlertmanager - -The `VMAlertmanager` CRD declaratively defines a desired Alertmanager setup to run in a Kubernetes cluster. -It provides options to configure replication and persistent storage. - -For each `Alertmanager` resource, the Operator deploys a properly configured `StatefulSet` in the same namespace. -The Alertmanager pods are configured to include a `Secret` called `` which holds the used -configuration file in the key `alertmanager.yaml`. - -When there are two or more configured replicas the Operator runs the Alertmanager instances in high availability mode. - -## VMAlertmanagerConfig - -The `VMAlertmanagerConfig` provides way to configure `VMAlertmanager` configuration with CRD. It allows to define different configuration parts, -which will be merged by operator into config. It behaves like other config parts - `VMServiceScrape` and etc. - -## VMRule - -The `VMRule` CRD declaratively defines a desired Prometheus rule to be consumed by one or more VMAlert instances. - -Alerts and recording rules can be saved and applied as YAML files, and dynamically loaded without requiring any restart. - - -## VMPrometheusConverter - -By default, the Operator converts and updates existing prometheus-operator API objects: - -`ServiceMonitor` into `VMServiceScrape` -`PodMonitor` into `VMPodScrape` -`PrometheusRule` into `VMRule` -`Probe` into `VMProbe` -Removing prometheus-operator API objects wouldn't delete any converted objects. So you can safely migrate or run -two operators at the same time. - - -## VMProbe - - The `VMProbe` CRD provides probing target ability with a prober. The most common prober is [blackbox exporter](https://github.com/prometheus/blackbox_exporter). - By specifying configuration at CRD, operator generates config for `VMAgent` and syncs it. It's possible to use static targets - or use standard k8s discovery mechanism with `Ingress`. - You have to configure blackbox exporter before you can use this feature. The second requirement is `VMAgent` selectors, - it must match your `VMProbe` by label or namespace selector. `VMAgent` probeSelector must match `VMProbe` labels. - See more details about selectors [here](https://docs.victoriametrics.com/operator/quick-start.html#object-selectors). - -## VMNodeScrape - -The `VMNodeScrape` CRD provides discovery mechanism for scraping metrics kubernetes nodes. -By specifying configuration at CRD, operator generates config for `VMAgent` and syncs it. It's useful for cadvisor scraping, -node-exporter or other node-based exporters. `VMAgent` nodeScrapeSelector must match `VMNodeScrape` labels. - See more details about selectors [here](https://docs.victoriametrics.com/operator/quick-start.html#object-selectors). - -## VMStaticScrape - -The `VMStaticScrape` CRD provides mechanism for scraping metrics from static targets, configured by CRD targets. -By specifying configuration at CRD, operator generates config for `VMAgent` and syncs it. It's useful for external targets management, -when service-discovery is not available. `VMAgent` staticScrapeSelector must match `VMStaticScrape` labels. -See more details about selectors [here](https://docs.victoriametrics.com/operator/quick-start.html#object-selectors). - -## VMAuth - - The `VMAuth` CRD provides mechanism for exposing application with authorization to outside world or to other applications inside kubernetes cluster. -For first case, user can configure `ingress` setting at `VMAuth` CRD. For second one, operator will create secret with `username` and `password` at `VMUser` CRD name. -So it will be possible to access these credentials from any application by targeting corresponding kubernetes secret. - -## VMUser - - The `VMUser` CRD describes user configuration, its authentication methods `basic auth` or `Authorization` header. User access permissions, with possible routing information. - User can define routing target with `static` config, by entering target `url`, or with `CRDRef`, in this case, operator queries kubernetes API, retrieves information about CRD and builds proper url. +Please see [Custom Resources](./resources/README.md) instead. diff --git a/docs/operator/managing-versions.md b/docs/operator/managing-versions.md index 8a1d32869..ec54291fe 100644 --- a/docs/operator/managing-versions.md +++ b/docs/operator/managing-versions.md @@ -1,93 +1,9 @@ --- -sort: 8 -weight: 8 -title: Managing application versions -menu: - docs: - parent: "operator" - weight: 8 +sort: -1 # hide page from menu aliases: - /operator/managing-versions.html --- +This page is deprecated. -# Managing application versions - -## VMAlert, VMAgent, VMAlertmanager, VMSingle version - - -for those objects you can specify following settings at `spec.Image` - -for instance, to set `VMSingle` version add `spec.image.tag` name from [releases](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) - -```yaml -cat < Note in some cases, you don't need relabeling, -> key=value label pairs can be added to the all scrapped metrics with `spec.externalLabels` for `VMAgent`. -> -```yaml -# simple label add config -apiVersion: operator.victoriametrics.com/v1beta1 -kind: VMAgent -metadata: - name: stack -spec: - externalLabels: - clusterid: some_cluster -``` - - - - It supports relabeling with custom configMap or inline defined at CRD - -## Configmap example - - Quick tour how to create `ConfigMap` with relabeling configuration - - ```yaml -cat << EOF | kubectl apply -f - -apiVersion: v1 -kind: ConfigMap -metadata: - name: vmagent-relabel -data: - global-relabel.yaml: | - - target_label: bar - - source_labels: [aa] - separator: "foobar" - regex: "foo.+bar" - target_label: aaa - replacement: "xxx" - - action: keep - source_labels: [aaa] - - action: drop - source_labels: [aaa] - target-1-relabel.yaml: | - - action: keep_if_equal - source_labels: [foo, bar] - - action: drop_if_equal - source_labels: [foo, bar] -EOF -``` - -Second, add `relabelConfig` to `VMagent` spec for global relabeling with name of `Configmap` - `vmagent-relabel` and key `global-relabel.yaml`. - For relabeling per remoteWrite target, add `urlRelabelConfig` name of `Configmap` - `vmagent-relabel` and key `target-1-relabel.yaml` to one of remoteWrite target for relabeling only -for those target. - -```yaml -cat <`. - -## Behavior without provided config - -If no configuration is provided, operator configures stub configuration with blackhole route. +Please see [VMAlertmanager](./resources/vmalertmanager.md) +and [VMAlertmanagerConfig](./resources/vmalertmanagerconfig.md) instead.