Command backup (Backup with hardened multipart) - EpicGazel/zpaqfranz GitHub Wiki

As everyone knows (or may be not) my very first contribute to zpaq was a rudimentary implementation of multipart archives, then merged by Mahoney (with his usual high skill).

Unfortunately, however, zpaq is more an archiver rather than a backup system: there are no realistic ways to check the integrity of multipart archives, zpaq does not handle very well

  • the zero length: if you press control-C during compression, a 0-bytes long pippo_00000XX.zpaq is (can) be made
  • the "hole" (a missing "piece", pippo001, pippo002, pippo007, pippo008...)
  • mixing different backups. You can replace one piece of a zpaq multipart archive with another, and zpaq will joyfully consider it, without noticing the error (!). Since each session is "self-sufficient" zpaq not only does not warn the user, but in the case of encryption (i.e., with -key) nasty thing happens.
  • cannot really (quickly) check the archive for missing parts: if a "piece" is lost, it is possible that everything (from those version to the last) is lost too. Even more, if you hold data from third-party clients, for testing an encrypted archive you need the password, which you simply don't have. And 99.9 percent of backups are encrypted, even the one on LAN-connected NASes. speed. If you have a 10.000 "pieces" backup, splitted in 10.000 "chunks", with zpaq you really cannot say if everything is OK, unless you run a (lengthy) full-scale test, this can take hours (ex. virtual machine disks)

There are critical cases where you want to do cloud backups on systems that do NOT allow the --append of rsync (OK, rclone and robocopy, I'm talking about you)

Similarly, computing hashes on inexpensive VPS cloud systems, usually with very slow disks is difficult, already for sizes around ~50GB

This new release creates a text-based index file that keeps the list of multiparts, their size, their MD5 and their quick hash

Multipart backup with zpaqfranz

zpaqfranz backup z:\prova.zpaq *.cpp

Will create automagically create

  • a multipart archive starting from prova_00000001.zpaq, prova_00000002.zpaq, prova_00000001.zpaq...
  • a textfile index prova_00000000_backup.txt
  • a binary index prova_00000000_backup.index

Now you can use the testbackup command to test

Is this a panacea?

Of course NOT Personally, I don't like splitting backups into many different parts at all, the risk of one being lost, garbled or corrupted is high However, in certain cases, there is no alternative. I do not mention one of the most well-known Internet service providers, to avoid publicity (after all... they do not pay me :)

Better a or backup ?

I use both of them I am thinking of an evolution of multipart with error correction (not detect, correction), but the level of priority is modest

Why the ancient MD5?

Unless now zpaqfranz use XXH3 for this kind of detection (-checktxt) But, sometimes, you must choose the fastest among "usual" ones (spoiler: some cheap cloud vendors)
Use -backupxxh3 to use XXH3 instead