Kiu/VictoriaMetrics
Kiu/VictoriaMetrics
1
0
Fork 0
mirror of https://github.com/VictoriaMetrics/VictoriaMetrics.git synced 2024-12-15 00:13:30 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

doc/operator: update formatting for backup section, add FAQ section (#3416)

Browse Source 
* doc/operator: update formatting for backup section, add FAQ section

* doc/operator: address review feedback

* doc/operator: add note about difference between `VMRestore` and `VMBackupmanager` as init containers

* Update docs/operator/backups.MD

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>

Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
This commit is contained in:
Zakhar Bessarab 2022-11-29 22:19:46 +04:00 committed by Aliaksandr Valialkin
parent a547226b53
commit 5edaff5165
No known key found for this signature in database
GPG Key ID: A72BEC6CD3D0DED1
2 changed files with 101 additions and 68 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
docs/operator/FAQ.md Normal file
View File

@ -0,0 +1,22 @@
---
sort: 15
---
# FAQ
## How to change VMStorage PVC storage class
With Helm chart deployment:
1. Update the PVCs manually
2. Run `kubectl delete statefulset --cascade=orphan {vmstorage-sts}` which will delete the sts but keep the pods
3. Update helm chart with the new storage class in the volumeClaimTemplate
4. Run the helm chart to recreate the sts with the updated value
With Operator deployment:
1. Update the PVCs manually
2. Run `kubectl delete vmcluster --cascade=orphan {cluster-name}`
3. Run `kubectl delete statefulset --cascade=orphan {vmstorage-sts}`
4. Update VMCluster spec to use new storage class
5. Apply cluster configuration

147
docs/operator/backups.MD
View File

@ -6,18 +6,21 @@ sort: 5
## vmbackupmanager ## vmbackupmanager
## vmbackupmanager is proprietary software. You can check vmbackupmanager [documentation](https://docs.victoriametrics.com/vmbackupmanager.html). It contains a description of the service and its features. This documentation covers vmbackumanager integration in vmoperator
Before using it, you must have signed contract and accept EULA https://victoriametrics.com/assets/VM_EULA.pdf ### vmbackupmanager is a part of VictoriaMetrics Enterprise offer
*Before using it, you must have signed contract and accept EULA https://victoriametrics.com/assets/VM_EULA.pdf*
## Usage examples ## Usage examples
`VMSingle` and `VMCluster` has built-in backup configuration, it uses `vmbackupmanager` - proprietary tool for backups. `VMSingle` and `VMCluster` has built-in backup configuration, it uses `vmbackupmanager` - proprietary tool for backups.
It supports incremental backups (hours, daily, etc) with popular object storages (aws s3, google cloud storage). It supports incremental backups (hourly, daily, weekly, monthly) with popular object storages (aws s3, google cloud storage).
You can enable it with the simple configuration, define secret The configuration example is the following:
```yaml
```yaml
--- ---
apiVersion: v1 apiVersion: v1
kind: Secret kind: Secret
@ -63,74 +66,82 @@ spec:
credentialsSecret: credentialsSecret:
name: remote-storage-keys name: remote-storage-keys
key: credentials key: credentials
```
```
NOTE: for cluster version operator adds suffix for `destination: "s3://your_bucket/folder"`, it becomes `"s3://your_bucket/folder/$(POD_NAME)"`. NOTE: for cluster version operator adds suffix for destination: `"s3://your_bucket/folder"`, it becomes `"s3://your_bucket/folder/$(POD_NAME)"`.
It's needed to make consistent backups for each storage node. It's needed to make consistent backups for each storage node.
You can read more about backup configuration options and mechanics [here](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmbackup) You can read more about backup configuration options and mechanics [here](https://docs.victoriametrics.com/vmbackup.html)
Possible configuration options for backup crd can be found at [link](https://docs.victoriametrics.com/operator/api.html#vmbackup) Possible configuration options for backup crd can be found at [link](https://docs.victoriametrics.com/operator/api.html#vmbackup)
## Restoring backups ## Restoring backups
There are several ways to restore with [vmrestore](https://docs.victoriametrics.com/vmrestore.html) or [vmbackupmanager](https://docs.victoriametrics.com/vmbackupmanager.html).
It can be done with [vmrestore](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/app/vmrestore) ### Manually mounting disk
You have to stop `VMSingle` by scaling it replicas to zero and manually restore data to the database directory.
There two ways:
First:
You have to stop `VMSingle` by scaling it replicas to zero and manually restore data to the database directory.
Steps:
1) edit `VMSingle` CRD, set replicaCount: 0
2) wait until database stops
3) ssh to some server, where you can mount `VMSingle` disk and mount it manually
4) restore files with `vmrestore`
5) umount disk
6) edit `VMSingle` CRD, set replicaCount: 1
7) wait database start
Second:
1) add init container with vmrestore command to `VMSingle` CRD, example: Steps:
```yaml 1. edit `VMSingle` CRD, set replicaCount: 0
apiVersion: operator.victoriametrics.com/v1beta1 2. wait until database stops
kind: VMSingle 3. ssh to some server, where you can mount `VMSingle` disk and mount it manually
metadata: 4. restore files with `vmrestore`
name: vmsingle-restored 5. umount disk
namespace: monitoring-system 6. edit `VMSingle` CRD, set replicaCount: 1
spec: 7. wait database start
initContainers:
- name: vmrestore ### Using VMRestore init container
image: victoriametrics/vmrestore:latest
volumeMounts:
- mountPath: /victoria-metrics-data
name: data
- mountPath: /etc/vm/creds
name: secret-remote-storage-keys
readOnly: true
args:
- -storageDataPath=/victoria-metrics-data
- -src=s3://your_bucket/folder/latest
- -credsFilePath=/etc/vm/creds/credentials
vmBackup:
# This is Enterprise Package feature you need to have signed contract to use it
# and accept the EULA https://victoriametrics.com/assets/VM_EULA.pdf
acceptEULA: true
destination: "s3://your_bucket/folder"
extraArgs:
runOnStart: "true"
image:
repository: victoriametrics/vmbackupmanager
tag: v1.67.0-enterprise
credentialsSecret:
name: remote-storage-keys
key: credentials
``` 1. add init container with vmrestore command to `VMSingle` CRD, example:
2) apply it, and db will be restored from s3
```yaml
3) remove initContainers and apply crd. apiVersion: operator.victoriametrics.com/v1beta1
kind: VMSingle
metadata:
name: vmsingle-restored
namespace: monitoring-system
spec:
initContainers:
- name: vmrestore
image: victoriametrics/vmrestore:latest
volumeMounts:
- mountPath: /victoria-metrics-data
name: data
- mountPath: /etc/vm/creds
name: secret-remote-storage-keys
readOnly: true
args:
- -storageDataPath=/victoria-metrics-data
- -src=s3://your_bucket/folder/latest
- -credsFilePath=/etc/vm/creds/credentials
vmBackup:
# This is Enterprise Package feature you need to have signed contract to use it
# and accept the EULA https://victoriametrics.com/assets/VM_EULA.pdf
acceptEULA: true
destination: "s3://your_bucket/folder"
extraArgs:
runOnStart: "true"
image:
repository: victoriametrics/vmbackupmanager
tag: v1.83.0-enterprise
credentialsSecret:
name: remote-storage-keys
key: credentials
```
2. apply it, and db will be restored from s3
3. remove initContainers and apply crd.
Note that using `VMRestore` will require adjusting `src` for each pod because restore will be handled per-pod.
### Using VMBackupmanager init container
Using VMBackupmanager restore in Kubernetes environment is described [here](https://docs.victoriametrics.com/vmbackupmanager.html#how-to-restore-in-kubernetes).
Advantages of using `VMBackupmanager` include:
- automatic adjustment of `src` for each pod when backup is requested
- graceful handling of case when no restore is required - `VMBackupmanager` will exit with successful status code and won't prevent pod from starting