TL;DR

I'm going to start implementing bits of backup software again. It may never become a usable program, certainly not in the near future, but it might if you help.

Obnam 1, 2004-2017

In 2004 I started implementing my first backup program. I'd written various shell scripts around tar to make and restore backups to floppies and a floppy tape drive, but those weren't serious projects. I did not want to rely on them for anything important. They mostly did their job, but only if used in the right way and nothing went even a little wrong.

The project would eventually be called Obnam. It was written in Python (my preferred programming language at the time), and would use a copy-on-write B-tree data structure, invented by Ohad Rodeh. The same data structure is used in the Btrfs file system in the Linux kernel.

I released Obnam 1.0 in 2012. I used it myself for many years, and was mostly happy with it. It had several other users too, which was nice. However, it was slow, mostly for architectural reasons, and the implementation and software architecture were such that I had trouble maintaining it, never mind making large, architectural changes. It was no longer fun, and a hobby project needs to be fun.

In 2017 I ended the project, and asked those using it to move to other backup software.

I thought I was free and would never think about backup software implementation again. More fool me.

Obnam 2, 2021-2023

It turned that I find the technical problems of implementing backup software irresistible. In 2021 I started again, this time using the Rust programming language (my new preference), and an entirely different software architecture. The new project used an SQLite database file for each backup to keep track of file metadata, including listing the individual data chunks that made up the contents of each file. This worked well, and was sufficiently fast.

Unfortunately, I had again moved quickly to get a usable program. I cut corners, and this again made it harder to develop the software, or even to maintain it. Combined with unrelated stress from work and life events, I didn't feel like fixing the mess I had made. I effectively, though not explicitly, abandoned the new version on 2023. I've made some minor changes to keep the code building and passing its test suite, but I've also not actually used it for a long while.

But at least after two such failures, I would finally be free and could put all of this past me. Twice fool me.

Obnam 3, 2025-

Apparently I am hard of learning.

I still can't stop thinking about backup software implementation. Last year I wrote a series of blog posts on the topic, to get the thoughts out of my head.

That wasn't enough. I want to try my hand at actually implementing some of my thoughts, if only to prove to myself they're feasible. So I'm starting a new project, Obnam 3.

This time, my goal is to build a strong, stable foundation without any corner cutting. However long it takes. I'm not in a hurry. As long as I have fun. I may build up from the foundation and make something that is actually useful. Something that can make and restore backups.

I don't expect to get there this year.

Caveat: I don't have much time to allocate to this. Initially, I'll spend up to three hours on Sundays, unless I'm busy with something else, or don't feel like it. This is a commitment to myself, not to anyone else.

Invitation to join the fun

I will do this development in the open. I'll blog each development session on the Obnam blog. I'll create a comment chain on the fediverse for each blog post.

I invite anyone who finds this kind of thing interesting to follow and to provide feedback. I'll also welcome patches. I'll use a Radicle repository for development, but you can just use plain Git. It will be most convenient for me to receive issues and patches via Radicle, but I'll accept them via any means that reaches me.

I don't expect to post about this here on my personal blog until there is something significant to say.