12 Commits

Author	SHA1	Message	Date
Zakhar Bessarab	760cdcec68	lib/backup: fix issue with inconsistent copying of appliedRetention.txt (#5027) ... * lib/backup: fix issue with inconsistent copying of appliedRetention.txt appliedRetention.txt can be modified in place, so it should be always copied just the same as parts.json Updates: https://github.com/victoriaMetrics/victoriaMetrics/issues/5005 Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> * docs: add changelog entry for appliedRetention.txt copying fix Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> --------- Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com>	2023-09-21 11:25:19 +02:00
Aliaksandr Valialkin	4d01bc6d52	lib/backup: properly copy parts.json files inside indexdb directory additional to data directory ... This is a follow-up for 264ffe3fa1 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5005 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/pull/5006	2023-09-18 16:16:50 +02:00
Aliaksandr Valialkin	f93a7b8457	lib/backup/common: consistently use canonical path with / directory separators at Part.Path ... Previously Part.Path could contain `\` directory separators on Windows OS, which could result in incorrect filepaths generation when making backups at object storage. Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4704 This is a follow-up for f2df8ad480	2023-09-18 16:15:34 +02:00
Zakhar Bessarab	264ffe3fa1	lib/backup: force copying of parts.json (#5006) ... * lib/backup: force copying of parts.json Copying of parts.json is required because `part.key()` comparison can create same key value for files with different contents. This will result in inconsistent backup being created or restored. See: https://github.com/VictoriaMetrics/VictoriaMetrics/issues/5005 Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> * lib/backup: ensure parts.json is only copied once Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> --------- Signed-off-by: Zakhar Bessarab <z.bessarab@victoriametrics.com> Co-authored-by: Nikolay <nik@victoriametrics.com>	2023-09-15 19:04:38 +02:00
Aliaksandr Valialkin	a0f695f5de	app/vmbackup: add ability to make server-side copying of existing backups	2023-08-13 17:24:24 -07:00
Nikolay	4786f036de	lib/backup: fixes path generation for windows (#4133) ... replaces custom fsync function with standard Fsync methods for files. fixes pattern matching for parts and properly generate backup path for local fs. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70	2023-05-03 10:48:53 +02:00
Nikolay	06eff5a72c	removes FileSize from backup part key (#1872) ... * removes FileSize from backup part key it should fix download restoration for backups * Update lib/backup/common/part.go Co-authored-by: Aliaksandr Valialkin <valyala@victoriametrics.com>	2021-12-01 11:01:28 +02:00
Aliaksandr Valialkin	bdd0a1cdb2	lib/backup: increase backup chunk size from 128MB to 1GB ... This should reduce costs for object storage API calls by 8x. See https://cloud.google.com/storage/pricing#operations-pricing	2021-01-13 12:16:35 +02:00
Aliaksandr Valialkin	b449607181	lib/backup: add MustStop() method for all remote filesystems	2020-10-09 15:32:19 +03:00
Aliaksandr Valialkin	f358fb72d1	app/{vmbackup,vmrestore}: add backup complete file to backup when it is complete and check for this file before restoring from backup ... This should prevent from restoring from incomplete backups. Add `-skipBackupCompleteCheck` command-line flag to `vmrestore` in order to be able restoring from old backups without `backup complete` file.	2020-01-09 15:35:38 +02:00
Oleg Kovalov	b4f44befa3	fix misspelled words (#229)	2019-11-12 00:16:42 +02:00
Aliaksandr Valialkin	1e46961d68	app/{vmbackup,vmrestore}: add vmbackup and vmrestore tools for creating backups on s3 or gcs from instant snapshots ... Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/203 Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/38	2019-11-08 21:21:07 +02:00