This post is part of a series on backup software implementation. See the backup-impl tag for a list of all posts in the series.

Export and import of backups

If one cares about the longevity of backed up data, it seems sensible to try to worry about the inevitable situation when fundamental decisions made for existing backups need to be changed:

• What backup software is used?
• How is data split into chunks? How big are the chunks?
• How is data compressed?
• How is data encrypted?

If, say, a new compression algorithm is developed that results in significantly smaller compressed data, one may want to re-compress all ones existing backups. Or one may want to switch to a new encryption method that's more secure that what has been used so far.

Or one may find much better backup software in the future.

For these and other reasons, one may want to convert one's existing backups to a new form. This is a problem that version control systems have had for a while, and the same approach would work for backups: an "export format" that's independent of the backup software (see [git export}(https://git-scm.com/docs/git-fast-export) for an example).

Thus, if one backup system can export existing backups in a common format, and another can import, then converting backups should be quite easy. (For version control systems, there's a lot of history and details that vary between systems that make this somewhat difficult, but in principle it's easy.)

I have not designed a backup export format yet. It's too early for that, I think, even if I first had this idea years ago. The first step would be to gather needs and wants, and that is a job in itself. My current list:

• the format should enable streaming, to avoid needing large amounts of backup space
• likewise, the format should enable incremental conversion
• the format should allow filtering
  • e.g., to drop all cat pictures

Example:

obnam1 export --all | filter-out-cat-photos | obnam2 import

I'm sure people can come up with any number of innovative ways to use such a filtering system. For me, I like the export/import approach because it allows me to change my backup parameters after the fact, and breaks the lock-in to the backup system I've chosen to use. I do not, however, know of any implementation of the concept.

Feedback

I'm not looking for suggestions on what backup software to use. Please don't suggest solutions.

I would be happy to hear other people's thoughts about backup software implementation. Or what needs and wants they have for backup solutions.

If you have any feedback on this post, please post them in the fediverse thread for this post.