flak rss random

origins of libressl

While I still remember the timeline and before I get confused by outsiders trying to rewrite history, here’s the official unofficial history of libressl. If there’s any one person to blame for causing libressl to happen, I’d have to say that it’s me. That’s not to say it was my idea, just that I instigated. This is how it began; who knows how it ends?

Heartbleed is the obvious place to start, but at the time it was just another in a serious of annual catastrophic OpenSSL bugs. The ease and speed with which 20 different exploits showed up on github was a new twist, but the “drop everything, set your hair on fire, patch OpenSSL” mania was otherwise about par for the course. There was also the wrinkle that the bug was in a default activated feature used by zero people that one couldn’t disable without recompiling the whole library. That probably would have lead to a tightened review process for any further OpenSSL updates in order to avoid more poison slipping in, but not a fork.

The lights in the situation room are still dark. The coffee maker is still cold.

Then I looked into mitigating Heartbleed with malloc.conf and fell down the rabbit hole. The nature of the freelists thwarted our malloc from doing what it could have done to help. Sigh, another feature that slipped in, but still not cause for a fork. (It was supposed to be a very short post. “If you set malloc.conf -> J, you can effectively mitigate Heartbleed.” There would have ended our little story.)

We aren’t yet having meetings in the situation room, but the people who meet in the situation room are starting to talk amongst themselves. Theo is posting nastygrams to the list.

Very quickly thereafter I discovered nginx without freelists didn’t work. Took a bit of work, but the freelist reuse bug was isolated and fixed.

Now people are starting to file into the situation room.

At this point somebody pointed out that bug 2167 had been known to the OpenSSL devs for years. One line patches had been provided, but no action had been taken.

Is everybody in the situation room? Good. What’s our next move?

The Rubicon lay somewhere between the discovery of freelists and the unfixed bug. Maybe the bug was always the final straw, maybe our momentum would have carried us here anyway. That unfixed bug (still unfixed in OpenSSL even now, two weeks later, despite OpenBSD, FreeBSD, and Debian all patching it out of tree) galvanized the team. It was clear that a fork was the only solution and that working with upstream would be a futile effort. If they wouldn’t apply this one simple patch, how would we ever convince them to drop their precious support for Visual C++ 5.0?

I didn’t really have anything to do with the planning of libressl. My secret plan, even before Heartbleed and going back to #gotofail, had been to write libztls (Zeromus TLS), but that would have taken a lot longer to get off the ground, and the Valhalla Rampage charged ahead in the mean time. Once that happened, I forgot all about libztls and started nuking from orbit with abandon. Also for the record, I’m not certain who invented the name, but it didn’t come from a G+ (WTF uses G+?) comment on April 15. I have an email from Miod, subject “LibreSSL battle plan“, dated April 11.

I might also mention that I was away, without Internet, the whole weekend before Hearbleed. I returned home Monday night and wanted nothing more than to take a shower, but my bathroom was flooded with water from a broken water heater from an apartment above. I had nothing better to do while waiting for maintenance than poke around with malloc.conf. If it hadn’t been for that, I probably never would have bothered. So there you have it, a broken water heater is the true cause of the libressl fork.

Posted 22 Apr 2014 14:10 by tedu Updated: 06 May 2014 02:43