Among other developmental heresies, I’m also a believer in everybody working in the same branch. I’ve dropped hints from time to time, and of course OpenBSD practitioners are familiar with this ideology, but I’ve only tried explaining it in full to a few coworkers. Who sat through my talk alternating between being shocked and appalled. Good times.

There’s not much of a narrative here, just some scattered thoughts. Now revised with a few more thoughts. No promises about the cohesion, however. This post started out as a longer form followup to Why OpenBSD doesn’t use GitHub but it’s gone in a slightly different direction. (Wow, that email is three years old.)

branching

Branching seems like a great idea. I want to build a feature; you want to build a feature. I make a branch; you make a branch. And then it’s off to the races. The code is just pouring forth. Too many cooks in the kitchen? Not when you have infinite kitchens! Nothing can stop us from finishing on schedule. Nay, ahead of schedule.

Boom. We’re done. All that’s left is to merge. Oh. Hmm. Merging.

merging

So what’s merging? It’s the worst.

Any discussion of merging trouble soon turns to merge conflicts. Because merge conflicts are death. But there are fates worse than death. There are successful merges. Some thoughts by Old New Thing on the matter. Especially part two.

--- a/db.c
+++ b/db.c
@@ -1 +1 @@
-int dbSchemaVersion = 328;
+int dbSchemaVersion = 329;

Twitch. Twitch. And so we build workarounds into our development process.

--- a/db.c
+++ b/db.c
@@ -1 +1 @@
-int dbSchemaVersion = 372; // building a better tomorrow
+int dbSchemaVersion = 373; // oh god, we were so wrong

One can hope to join a team where the village elders have passed down the wisdom of experience, but omg cargo cult superstitious old people probably not.

text

Even if the text of the code merges the way we’d like, that’s not a guarantee the implemented features will integrate correctly. Alice adds a confirmation prompt to every save file dialog. Bob adds five new save dialogs. Do the new dialogs confirm or not? Of course they do. Twice.

Text may not be the best representation of a program’s semantics, but it’s often the best we have. Alas, merge tools certainly operate in a world of text. And worse, the operation of most merge tools reinforces this notion. Pick the left line. Pick the right line. Simply using a side by side merge tool suggests that the final product should be a linear combination/interleaving of lines.

I actually much prefer the simpler alternative where all the code is dumped in the file and it’s left to me and vi to sort things out. By starting the process in an editor, I’m not subconsciously made to feel that post-merge edits are the result of choosing poorly. (If only I had picked left-right instead of right-left, maybe I wouldn’t need to edit this file...)

interference

The problem is that the intersection (or perhaps, interference pattern) of two moving targets is difficult to predict. Much easier is to instead evaluate each feature against the static past, so that’s what happens, despite the obvious shortcomings.

The solution is to bring all the code together as early and often as possible. The error bound for our estimate of the combined effect of two changes is proportional to the magnitude of each change. A series of small changes means our baseline is constantly readjusting, leading to fewer surprises.

slow

It certainly sounds like forcing everyone to play in the same sandbox will slow things down. And it does. But it also speeds things up. Features never get trapped in the next branch; users are never forced to decide which branch to run. Availability is essentially simultaneous with completion. Slower development, but faster shipping.

social

For an open source project, branching is counter intuitively antisocial. For instance, I usually tell people I’m running OpenBSD, but that’s kind of a lie. I’m actually running teduBSD, which is like OpenBSD but has some changes to make it even better. Of course, you can’t have teduBSD because I’m selfish. I’m also lazy, and only inclined to make my changes work for me, not everyone else. But eventually the effort involved in maintaining my secret branch exceeds the effort necessary to fine tune my changes and integrate them back into the main source tree. To the extent that tooling may help me maintain a branch, it facilitates antisocial development and hoarding.

It’s often stated that technology can’t solve social problems. But anti-technology can solve anti-social problems. Haha.

less

Of course, to be a stickler about it, unless the entire team is sitting around one computer the instant somebody starts editing a file, they’ve created a branch. Yeah, ok. Pretend the phrase is “branch less” if that’s how it’s going to be. As in keep the branch small and don’t let it get to far away.

wip

As a consequence, WIP projects get pushed into the tree sooner. It’s tempting to hold features back, refining them until perfect, but this limits their exposure. Ultimately, committing the WIP and polishing in tree leads to faster development.

toggles

Another solution is feature toggles (flags), where all the code is added to the main branch, but disabled until it’s ready to go. OpenBSD doesn’t really use this approach either. After all, knobs are for knobs. But sometimes code will coexist in the tree while it’s being worked on. The decision of which web server to use might be considered a megaflag.

other

Theo’s OpenBSD release engineering talk touched on some of the same material, but focused just on the release process.

Trunk based development is a more coherent explanation, although OpenBSD does something a little different. We don’t cherry pick changes from a single trunk to pull back to a release. The release is tagged, and then it’s done.

Feature Toggles are one of the worst kinds of Technical Debt. Another perspective. There’s actually been a lot written on this topic, more than I expected. I seem to live in a bubble where branch and merge is the only way things are done. This article has a good set of links for further reading.