VictoriaMetrics/docs/Release-Guide.md

weight	title	menu	aliases
30	Release process guidance	docs parent weight victoriametrics 30	/Release-Guide.html

Pre-reqs

1. Make sure you have enterprise remote configured

git remote add enterprise <url>

1. Make sure you have singing key configured
2. Make sure you have github token with at least read:org, repo, write:packages permissions exported under GITHUB_TOKEN env variable. You can create token here
3. Make sure you're authorized for pushing docker images

For MacOS users

Make sure you have GNU version of utilities zip, tar, sha256sum. To install them run the following commands:

brew install coreutils
brew install gnu-tar
export PATH="/usr/local/opt/coreutils/libexec/gnubin:$PATH"

Docker may need additional configuration changes:

docker buildx create --use --name=qemu
docker buildx inspect --bootstrap  

By default, docker on MacOS has limited amount of resources (CPU, mem) to use. Bumping the limits may significantly improve build speed.

Release version and Docker images

1. Make sure all the changes are documented in CHANGELOG.md. Ideally, every change must be documented in the commit with the change. Alternatively, the change must be documented immediately after the commit, which adds the change.
2. Run make vmui-update and make vmui-logs-update commands to re-build static files for vmui. See commit example.
3. Make sure all the changes are synced between master, cluster, enterprise-single-node and enterprise-cluster branches. Changes in these branches must be synced immediately after they are committed in at least a single branch.
4. Make sure that the release branches have no security issues.
5. Update release versions if needed in SECURITY.md.
6. Run PKG_TAG=v1.xx.y make docs-update-version command to update version help tooltips.
7. Cut new version in CHANGELOG.md and make it merged. See example in this commit.
8. Cherry-pick bug fixes relevant for LTS releases.
9. Make sure you get all changes fetched git fetch --all.
10. Create the following release tags:
    • git tag -s v1.xx.y in master branch
    • git tag -s v1.xx.y-cluster in cluster branch
    • git tag -s v1.xx.y-enterprise in enterprise-single-node branch
    • git tag -s v1.xx.y-enterprise-cluster in enterprise-cluster branch
11. Run TAG=v1.xx.y make publish-release. This command performs the following tasks: a) Build and package binaries in *.tar.gz release archives with the corresponding _checksums.txt files inside bin directory. This step can be run manually with the command make release from the needed git tag. b) Build and publish multi-platform Docker images for the given TAG, TAG-cluster, TAG-enterprise and TAG-enterprise-cluster. The multi-platform Docker image is built for the following platforms:
    • linux/amd64
    • linux/arm64
    • linux/arm
    • linux/ppc64le
    • linux/386 This step can be run manually with the command make publish from the needed git tag.
12. Verify that created images are stable and don't introduce regressions on test environment.
13. Push the tags v1.xx.y and v1.xx.y-cluster created at previous steps to public GitHub repository at https://github.com/VictoriaMetrics/VictoriaMetrics. Push the tags v1.xx.y, v1.xx.y-cluster, v1.xx.y-enterprise and v1.xx.y-enterprise-cluster to the corresponding branches in private repository. Important note: do not push enterprise tags to public GitHub repository - they must be pushed only to private repository.
14. Run TAG=v1.xx.y make github-create-release github-upload-assets. This command performs the following tasks: a) Create draft GitHub release with the name TAG. This step can be run manually with the command TAG=v1.xx.y make github-create-release. The release id is stored at /tmp/vm-github-release file. b) Upload all the binaries and checksums created at step 9a to that release. This step can be run manually with the command make github-upload-assets. It is expected that the needed release id is stored at /tmp/vm-github-release file, which must be created at the step a. If the upload process is interrupted by any reason, then the following recovery steps must be performed:
    • To delete the created draft release by running the command make github-delete-release. This command expects that the id of the release to delete is located at /tmp/vm-github-release file created at the step a.
    • To run the command TAG=v1.xx.y make github-create-release github-upload-assets, so new release is created and all the needed assets are re-uploaded to it.
15. Test new images on sandbox.
16. Go to https://github.com/VictoriaMetrics/VictoriaMetrics/releases and verify that draft release with the name TAG has been created and this release contains all the needed binaries and checksums.
17. Update the release description with the content of CHANGELOG for this release.
18. Publish release by pressing "Publish release" green button in GitHub's UI.
19. Update GitHub tickets related to the new release. Usually, such tickets have label waiting for release. Close such tickets by mentioning which release they were included into, and remove the label. See example here.
20. Bump VictoriaMetrics version at deployment/docker/docker-compose.yml and at deployment/docker/docker-compose-cluster.yml.
21. Bump VictoriaMetrics version mentioned in docs.
22. Follow the instructions in release follow-up.

Public Announcement

• Publish message in Slack at https://victoriametrics.slack.com
• Post at X (Twitter) at https://x.com/VictoriaMetrics
• Post in Reddit at https://www.reddit.com/r/VictoriaMetrics/
• Post in LinkedIn at https://www.linkedin.com/company/victoriametrics/
• Publish message in Telegram at https://t.me/VictoriaMetrics_en and https://t.me/VictoriaMetrics_ru1

Operator

The operator repository https://github.com/VictoriaMetrics/operator/

Bump the version of images

• Bump Version field in file internal/config/config.go with new release version for:
  • vmalert in BaseOperatorConf.VMAlertDefault.Version,
  • vmagent in BaseOperatorConf.VMAgentDefault.Version,
  • vmsingle in BaseOperatorConf.VMSingleDefault.Version,
  • vmselect in BaseOperatorConf.VMClusterDefault.VMSelectDefault.Version,
  • vmstorage in BaseOperatorConf.VMClusterDefault.VMStorageDefault.Version,
  • vminsert in BaseOperatorConf.VMClusterDefault.VMInsertDefault.Version,
  • vmbackupmanager in BaseOperatorConf.VMBackup.Version (should be enterprise version),
  • vmauth in BaseOperatorConf.VMAuthDefault.Version.
• Run make operator-conf.
• Rename "Next release" section in docs/CHANGELOG.md to the new release version and create new empty "Next release" section.
• Commit and push changes to master.
• Create and push a new tag with the new release version.
• Create github release from this tag with "Release notes" from docs/CHANGELOG.md for this version in description.

Helm Charts

The helm chart repository https://github.com/VictoriaMetrics/helm-charts/

Bump the version of images

Bump tag field in values.yaml with new release version. Bump appVersion field in Chart.yaml with new release version. Add new line to "Next release" section in CHANGELOG.md about version update (the line must always start with "-"). Do NOT change headers in CHANGELOG.md. Bump version field in Chart.yaml with incremental semver version (based on the CHANGELOG.md analysis).

Do these updates to the following charts:

1. Update vmagent chart version in values.yaml and Chart.yaml
2. Update vmalert chart version in values.yaml and Chart.yaml
3. Update vmauth chart version in values.yaml and Chart.yaml
4. Update cluster chart versions in values.yaml, bump version for vmselect, vminsert and vmstorage and Chart.yaml
5. Update k8s-stack chart versions in values.yaml, bump version for vmselect, vminsert, vmstorage, vmsingle, vmalert, vmagent and Chart.yaml
6. Update single-node chart version in values.yaml and Chart.yaml
7. Update vmgateway chart version in values.yaml and Chart.yaml

Once updated, run the following commands:

1. Commit and push changes to master.
2. Run "Release" action on Github:
3. Merge new PRs "Automatic update CHANGELOGs and READMEs" and "Synchronize docs" after pipelines are complete.

Ansible Roles

1. Update the version of VictoriaMetrics components at https://github.com/VictoriaMetrics/ansible-playbooks.
2. Commit changes.
3. Create a new tag with git tag -sm <TAG> <TAG>.
4. Push the changes with the new tag. This automatically publishes the new versions to galaxy.ansible.com.