A critical feature for me was offline access. Not “I hope my IMAP client cached that email” offline access, but real “I can read any email, ever” offline access. The problem is that every client I’m familiar with that could do that basically worked locally. The price paid for offline access was single computer access. In theory, multiple desktops could fetch mail and process it, but the tags and filters would get out of sync. I don’t know anyone who has pulled this off. I needed to build a system where despite running independent clients against independent mail stores, the state of the universe would be kept roughly in sync.

My idea was inspired a bit by distributed version control systems. Every computer would have a mail store, but could sync with any other store. This required giving every message a unique id (UUID) when it enters the mailtanium “universe” and tracking changes, not just current state. I also added a checkpoint feature. This way, when we connect to another computer, we ask for all the changes since the last checkpoint we remember asking about. We download any missing messages and tag changes since that point, then instruct the remote to create a new checkpoint. All checkpoints are saved; they’re a lot like commits in source control. On the local end, we also have a checkpoint, which we use to determine which messages need to be uploaded. Checkpoints themselves are identified by UUID. A server can support many, many clients because each checkpoint takes only a few bytes of storage, and its the clients that remember their last checkpoint.

The protocol is optimized in some ways for the simple use case of a client syncing against a remote server, but it also allows clients to sync against each other. So instead of downloading all my mail from my server onto my laptop, I could sync it against my desktop before leaving home (much faster link), then sync against the server on the road. I won’t get multiple copies of mail, and stuff marked read on my laptop will be marked read on my desktop when I finally return home, even without syncing the desktop and laptop explicitly.

One issue in any design like this is merging conflicts. What should happen when two clients both change a message, then sync? Fortunately, we’re not dealing with source code here and the possible changes are limited. So a very simple, last to sync wins, is acceptable. The theory is that most of the changes you make are going to be redundant anyway, like marking a message read on two machines. You’re not going to tag the same message important on one system and boring on another. The last thing I want to do is force the user to perform merges.

I also wanted the sync protocol to be robust against network disruptions. After the initial checkpoint sync, the list of missing messages and attachments is saved to the client’s todo list. We record the new checkpoint on the server as soon as we’ve saved the list (but not the contents) of missing messages. The client then works through the todo list. If we get disconnected we can start again (at that point, not the beginning) and if we’re far behind we’ll fill in more recent messages first.

Unlike a DVCS, our history is always linear and may not always even be the same (read a message, unread, read, sync and the read/unread/read history may not appear on all client machines, only the final read state). In general, I’m only interested in the current state, and making sure that the same current state appears on all synced clients in short order. The checkpointing and logging system is just an optimization to avoid comparing the state of the entire mail store on each sync. Some other settings are also synced, like my mailing list filters, using a similar last writer wins system.