Commit Graph

108 Commits

Author SHA1 Message Date
Aliaksandr Valialkin
c285fca256
Makefile: allow specifying the needed concurrency for make via MAKE_CONCURRENCY env var 2023-11-21 01:11:36 +02:00
Aliaksandr Valialkin
6b75523468
Makefile: speedup release, publish and crossbuild rules by using parallel make 2023-11-20 23:07:11 +02:00
Aliaksandr Valialkin
00f8c55efd
Makefile: remove package-base dependency from publish rule, since this dep is set inside all the publish-* dependencies
This is a follow-up for d4099a75be
2023-11-15 12:58:30 +01:00
Aliaksandr Valialkin
996e746c2c
Makefile: update golangci-lint version from v1.54.2 to v1.55.1
See https://github.com/golangci/golangci-lint/releases/tag/v1.55.1
2023-11-02 21:42:35 +01:00
Haleygo
130e0ea5f0
vmalert-tool: implement unittest (#4789)
1. split package rule under /app/vmalert, expose needed objects
2. add vmalert-tool with unittest subcmd

https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2945
2023-10-16 14:12:06 +02:00
Aliaksandr Valialkin
0bc0d2610c
go.mod: increase the minimum supported Go version from Go1.19 to Go1.20 2023-09-07 12:18:12 +02:00
Aliaksandr Valialkin
d8afd7fe98
Makefile: update golangci-lint from v1.51.2 to v1.54.2
See https://github.com/golangci/golangci-lint/releases/tag/v1.54.2
2023-09-01 10:25:49 +02:00
Aliaksandr Valialkin
0609d8c8c5
Makefile: cleanup bin/ directory at the beginning of make publish-release command
This is needed in order to prevent from non-build artifacts to be uploaded to Github release page
2023-08-29 13:16:20 +02:00
Roman Khavronenko
c81e90223c
dashboards: provide copies of Grafana dashboards alternated with Vict… (#4905)
dashboards: provide copies of Grafana dashboards alternated with VictoriaMetrics datasource

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-08-29 11:20:16 +02:00
Aliaksandr Valialkin
3e5e55d235
Revert "make: add goimports task (#4582)"
This reverts commit 20b18e9feb.

Reason for revert: running goimports on `make check-all` introduces the following issues:

- It runs only on modified files, which weren't commited yet into git repository.
  This means the formatting for the remaining files becomes different comparing to the formatting
  for the changed files. This also means that the goimports has no any effect
  at github actions and when the changed code is already commited to git repository.
- `gomiports` performs formatting in the same way as gofmt, so `make fmt` becomes unnecessary.
  But when `gofmt` is substituted with `goimports`, then it performs unnecessary formatting for *.qtpl.go files.
  It is possible to make a hack, which will prepare a list of all the *.go files at lib/ and app/
  without the *.qtpl.go files, and then feed this list to `goimports`, but this looks too fragile
  for the task of just fixing the ordering of Go imports.

So it is better to leave source code formatting as is with `gofmt`, while manually fixing improper ordering
of Go import from time to time in dedicated commits until better solution arises.
2023-07-13 12:18:02 -07:00
Zakhar Bessarab
64149782ac
make: add goimports task (#4582)
* make: add goimports task

Adds task to fix imports formatting implace.
Formats imports into:
- native library
- external libraries
- local packages based on github.com/VictoriaMetrics/VictoriaMetrics prefix

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* make: add goimports install task

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* make: run goimports only for changed files

Applying goimports to all existing files would create a lot of problems with cherry-picking changes between different branches used for development. To avoid this it was decided to only run goimports on changed files to fix formatting gradually.

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

* make: update goimports to run on all changed files

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>

---------

Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>
2023-07-13 12:17:31 -07:00
Denys Holius
ef29a41824
docs: use printf for Makefile:copy-docs section (#4548)
printf handles new line char for cross-platform use
2023-07-06 22:20:53 -07:00
Max Golionko
75d9914916
CI: disable docker scan, enable auto release to sandbox (#4476)
* disable docker scan

* disable nightly, enable auto release to sandbox

* remove whitespace
2023-07-06 22:02:48 -07:00
Artem Navoiev
258575441e
revert version of golanci
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-06 21:47:35 -07:00
Artem Navoiev
28301d1b41
remove deleted repo from the docs
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
2023-07-06 21:47:07 -07:00
Aliaksandr Valialkin
4f6eb775df
Makefile: remove trailing whitespace from copy-docs comments 2023-07-06 11:01:39 -07:00
Artem Navoiev
8dd2a263b9
desribe old link param
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>
(cherry picked from commit bf4711ecba)
2023-06-02 13:25:49 +02:00
Artem Navoiev
5e6fe50c53
return information about cluster
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

(cherry picked from commit 8f09569cb8)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-02 13:25:48 +02:00
Artem Navoiev
3185b8048c
update docs-sync Makefile command, add hugo front-matter
Signed-off-by: Artem Navoiev <tenmozes@gmail.com>

(cherry picked from commit f791811b15)
Signed-off-by: hagen1778 <roman@victoriametrics.com>
2023-06-02 13:24:41 +02:00
Aliaksandr Valialkin
ef4e5e4fc7
Makefile: add missing vmcluster-windows-amd64 target
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70
2023-03-25 15:34:21 -07:00
Aliaksandr Valialkin
fc3d826d7f
all: add Windows build for VictoriaMetrics
This commit changes background merge algorithm, so it becomes compatible with Windows file semantics.

The previous algorithm for background merge:

1. Merge source parts into a destination part inside tmp directory.
2. Create a file in txn directory with instructions on how to atomically
   swap source parts with the destination part.
3. Perform instructions from the file.
4. Delete the file with instructions.

This algorithm guarantees that either source parts or destination part
is visible in the partition after unclean shutdown at any step above,
since the remaining files with instructions is replayed on the next restart,
after that the remaining contents of the tmp directory is deleted.

Unfortunately this algorithm doesn't work under Windows because
it disallows removing and moving files, which are in use.

So the new algorithm for background merge has been implemented:

1. Merge source parts into a destination part inside the partition directory itself.
   E.g. now the partition directory may contain both complete and incomplete parts.
2. Atomically update the parts.json file with the new list of parts after the merge,
   e.g. remove the source parts from the list and add the destination part to the list
   before storing it to parts.json file.
3. Remove the source parts from disk when they are no longer used.

This algorithm guarantees that either source parts or destination part
is visible in the partition after unclean shutdown at any step above,
since incomplete partitions from step 1 or old source parts from step 3 are removed
on the next startup by inspecting parts.json file.

This algorithm should work under Windows, since it doesn't remove or move files in use.
This algorithm has also the following benefits:

- It should work better for NFS.
- It fits object storage semantics.

The new algorithm changes data storage format, so it is impossible to downgrade
to the previous versions of VictoriaMetrics after upgrading to this algorithm.

Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3236
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3821
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70
2023-03-19 23:28:26 -07:00
Aliaksandr Valialkin
7d37ab63a5
Makefile: update golangci-lint from v1.51.1 to v1.51.2
See https://github.com/golangci/golangci-lint/releases/tag/v1.51.2
2023-03-12 17:08:34 -07:00
Aliaksandr Valialkin
6149e4c319
Makefile: update golangci-lint from v1.50.1 to v1.51.1 2023-02-07 11:08:38 -08:00
Aliaksandr Valialkin
87a4943bea
Makefile: remove trailing space after golangci-lint run command
It is left after ec2c82e800
2023-01-05 18:09:05 -08:00
Luke Palmer
51d956225d
Lint and errcheck using golangci-lint (#3558) 2023-01-05 17:00:17 -08:00
Aliaksandr Valialkin
a6a2e35a83
Makefile: update golangci-lint version from v1.48.0 to v1.50.1 2022-12-20 13:10:42 -08:00
Aliaksandr Valialkin
2e714aa2d3
Makefile: publish release docker images at DockerHub with the stable tag
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2911
2022-12-20 12:27:24 -08:00
Aliaksandr Valialkin
9d3c121feb
deployment/docker: use cluster-latest tag for docker images from cluster components
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3505
2022-12-20 10:38:51 -08:00
Zakhar Bessarab
de0e4f1f7e
make: make openssl output parsing symbol number agnostic (#3429) 2022-12-02 19:10:12 -08:00
Aliaksandr Valialkin
e7c2d057e6
Makefile: remove docs/*.tmp files after running sed command there 2022-10-06 15:25:27 +03:00
Aliaksandr Valialkin
9f1632b30b
Makefile: add missing "=" char between "-i" flag and its value for sed
This is a follow-up after 78af27f955
2022-10-06 15:10:52 +03:00
Roman Khavronenko
92f4d6a338
docs: when modifying docs in place allow storing backups (#3205)
The stored backups would help to identify docs corruption
but aren't needed for commiting. So `.tmp` backup files
are also git-ignored.

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-10-06 15:10:52 +03:00
Roman Khavronenko
238401e3ed
docs: udpate Datadog section (#3190)
The `docs-sync` command was updated to modify images path
for assets in `docs/` folder. The change allows to refer images from `docs`
without copying them to the root folder.

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-10-05 22:36:35 +03:00
Aliaksandr Valialkin
ae2f669e73
Makefile: run errcheck for all the app/... subdirs 2022-09-30 18:40:06 +03:00
Aliaksandr Valialkin
47ebf6e836
Makefile: fix -compat value passed to go mod tidy 2022-09-19 15:14:01 +03:00
Roman Khavronenko
d071e39694
bump Go version to 1.19.1 (#3108)
The reason is to cover vulnerability GO-2022-0969
Found in: net/http@go1.18.5
Fixed in: net/http@go1.19.1
More info: https://pkg.go.dev/vuln/GO-2022-0969

Signed-off-by: hagen1778 <roman@victoriametrics.com>

Signed-off-by: hagen1778 <roman@victoriametrics.com>
2022-09-14 13:43:27 +03:00
Aliaksandr Valialkin
21d608b210
Makefile: remove github-create-release and github-upload-assets commands from publish-release
This is a follow-up for b9231c715a
2022-09-08 21:03:00 +03:00
Aliaksandr Valialkin
6cbe85a08a
Makefile: consistently use go install instead of go get for installing various binaries needed during build/test/check of the code
`go install` is the preferred way for installing go binaries starting
from the minimum supported Go version for VictoriaMetrics - Go1.18 -
see https://tip.golang.org/doc/go1.18#go-command
2022-09-08 18:46:08 +03:00
Aliaksandr Valialkin
c083286a92
Makefile: check for vulnerabilities in used Go packages with govulncheck when running make check-all
See https://go.dev/blog/vuln
2022-09-08 18:46:00 +03:00
Aliaksandr Valialkin
f6b3ad0c37
docs/Release-Guide.md: require manual push of the created release tags to public Github
The automated push of release tags to Github require specifying the remote repository name
when doing `git push <remote-repo-name> v1.xx.y`.
The remote repository name can differ in different environments,
so it cannot be put into Makefile rule.

TODO: create a Makefile rule, which generates standard remote names for public
and private repositories in Git, so `git push` for release tags could be automated then.
2022-09-08 15:00:34 +03:00
Aliaksandr Valialkin
6fe1f895f0
Makefile: properly push public tags 2022-09-02 23:19:03 +03:00
Aliaksandr Valialkin
0be941c99b
Makefile: push v1.xx.y and v1.xx.y-cluster tags to github before creating the v1.xx.y release at github
Otherwise Github creates the v1.xx.y tag on itself when creating the release
2022-09-02 21:49:10 +03:00
Max Golionko
0359fce052
simplify release process (#3012)
* simplify release process

* address comments

* address comments

* wip

* wip

* wip

Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>
2022-08-31 02:28:45 +03:00
Aliaksandr Valialkin
40c2fae617
vendor: make vendor-update 2022-08-15 00:54:50 +03:00
Aliaksandr Valialkin
bfa490e921
Makefile: update golangci-lint from v1.47.1 to v1.48.0
This is needed for adding support for Go 1.19
2022-08-07 22:33:40 +03:00
Aliaksandr Valialkin
573f6c8d6c
Makefile: remove redundant -mod=vendor option when running Go tools
The `-mod=vendor` is automatically set when there is a `vendor` directory
starting from Go1.14 - see https://go.dev/doc/go1.14#go-command

Since the minimum supported Go version for VictoriaMetrics is Go1.17,
then the `-mod=vendor` option is no longer needed.
2022-08-07 20:40:41 +03:00
Denys Holius
9b4024cd62
deployment/docker/Makefile: added docker-scan (#2916)
* deployment/docker/Makefile: added docker-scan

docker-scan based on native 'docker scan' function that use snyk.io, see https://docs.docker.com/engine/scan/

* set to call 'docker-scan after release binaries but before publishing
2022-08-02 13:26:43 +03:00
Denys Holius
d635169c90
Update golangci version to latest v1.47.1 (#2890)
See https://github.com/golangci/golangci-lint/releases/tag/v1.47.1
2022-07-19 19:31:16 +03:00
Aliaksandr Valialkin
da6c85a2f6
all: follow-up for d99ba3481b 2022-07-13 17:17:08 +03:00
Dmytro Kozlov
4e4def9df8
Rename release packages (#2810)
* makefile: add os to each release file

* makefile: update vmutils arm64

* makefile: update victoria-metrics release process

* makefile: update publish with os

* makefile: update publish with os

* makefile: change tar library

* update release logic

* copy all releases

* sort command by GOOS

* rollback commands

* rollback OSARCH

* fix commands

* cleanup

* fix windows build

* sort build by GOOS, update README.md
2022-07-13 17:11:01 +03:00