flak rss random

rethinking openbsd security

OpenBSD aims to be a secure operating system. In the past few months there were quite a few security errata, however. That’s not too unusual, but some of the recent ones were a bit special. One might even say bad. The OpenBSD approach to security has a few aspects, two of which might be avoiding errors and minimizing the risk of mistakes. Other people have other ideas about how to build secure systems. I think it’s worth examining whether the OpenBSD approach works, or if this is evidence that it’s doomed to failure.

I picked a few errata, not all of them, that were interesting and happened to suit my narrative.

libc auth

The auth functions would exec helpers without checking argv. Report. Patch.

This is embarrassingly simple, yet probably not obvious in code review. In part I think there’s some confusion as to who is responsible for checking inputs. You could look at any of the three components involved, calling program, libc, login_passwd, and reasonably conclude that somebody else is doing the checks. In the end I guess we decided libc is at fault since that’s where the patch went, but that code doesn’t look immediately wrong to me.

A more interesting part of the story is that even with the libc bug, login_passwd should not have been exploitable in this manner, except for another bug. In 2001, login_passwd was rewritten to support kerberos, and what is perhaps the real bug introduced. A request for challenge response auth (think s/key) would return authenticated, instead of silence. Many years later, the kerberos code was removed, but the refactor to support it remained, and so did the introduced bug.

This bug wouldn’t have been a complete non issue (and there were some other related issues with argv parsing), but the impact certainly would have been reduced had the kerberos refactor been more thoroughly undone.

argv parsing can be tricky to get right in security contexts. A fair number of suggestions how to prevent problems wouldn’t have helped. I’ll note that this vuln popped up after Fixing Filenames made the rounds again, but the leading dash had nothing to do with a filename.


Naughty environment variables weren’t removed in ld.so. Report. Patch.

Something that looks like a memory bug. Except mostly not. The error here is tying the success of one operation (splitting environment variable) to the removing that variable. Getting a little a head of ourselves.

Advocates of various type systems no doubt believe they would handle these operations in the correct order, but I’m not sure that helps. The C code didn’t fail because it was missing error handling or failed to notice the allocation failure.


ftp would follow redirects to local files. Report. Patch.

The NetBSD ftp bug where it would exec redirects is my goto example for runaway features. Same bug (but with fortunately minor consequences) strikes again! Stay indoors. Don’t add features to your programs.

smtpd from

smtpd failed to validate some sender addresses. Report. Patch. Commentary.

I think Gilles’s commentary covers most of this, but I’ll recap some backstory. Long ago, everybody had their mail stored in /var/mail in mbox files. This is problematic because there’s the possibility of corruption if your mua deletes an email while the mda is delivering a new one. (Not to mention other problems, like From mangling.) So you need to lock the mbox file. But file locking isn’t reliable over NFS. So you need lockfiles instead. But you need everyone to agree on the particular locking protocol, and while the mbox files are owned by the user, the directory is owned by root. So you need to exec a setuid helper to actually deliver to the mbox. Well, that’s one problem. Another holdover from the long ago is that the mda program is configurable and may not just be a program, but a shell pipeline. People set their mda to “spam-assassin | mail.mda” and that’s not something you can just pass to execve(). Some of that’s probably not 100% accurate, but close enough.

There’s a lot of historic complexity here. And it’s unfortunately close to the surface. Mail delivery has a long history and people build up very complex systems and workflows on top of it, which makes building a drop in replacement very difficult. Despite various levels of privilege separation, the less privileged processes are still mostly trusted by the parent, which runs as root. It will execute the commands and arguments that it receives.

Avoiding this seems as simple as switching to maildir, but that requires other changes elsewhere. The included mua mail can’t read maildir. Personally, I’d say this is long over due, but mbox still works for some people, and the upgrade path is perhaps not fully transparent or automatic.

smtpd read

An out of bounds read in smtpd could be leveraged into command execution. Report. Patch.

A real deal memory safety issue. By sending back some funny status lines, a remote smtp server could inject commands into the smtpd queue. When an email is queued for redelivery, smtpd adds some information about the destination to its header, so it knows which command to run (see above). In an attack similar to http request smuggling, if you can create a bounce with unexpected commands in the header, smtpd will execute when it attempts redelivery.

As above, one problem is that the most sensitive parts of smtpd are in very close proximity to the parts under attack. The most serious problem in this instance I think is that smtpd chose to store its own metadata inline with email data. This makes parsing desync attacks possible. If the quoted reply from the server were saved in a completely separate file from the one containing delivery instructions, the out of bounds read would have been mostly harmless.

This vulnerability stands out to me because I think the inherent danger of commingling data from different trust levels was never recognized. It’s clearly perilous.


The conclusion was of course preordained, but here we go anyway.

I think several of these errata help demonstrate that principles like eliminating legacy interfaces and reducing complexity are vital to maintaining security. For the most part, the failure was in not following through, not because the principle itself is flawed or untenable. Some things slip through, but I’d argue not because the vigilance necessary is superhuman. There’s a danger here of providing useless advice like just don’t make mistakes and git gud, but I think the OpenBSD approach is more actionable than that.

Even OpenBSD is subject to compromise for the sake of practicality, which is how some legacy designs stick around. So the lesson perhaps is to really stick with the principles that work, and not just when convenient. But not always an easy choice to make.

Of the three vulns I’d classify as most serious, auth and two smtpd, all were more or less exploitable only due to design issues beyond the original bug. They would have been just minor oopsies, which gives me hope that we don’t always have to be perfect. Alas, it can be hard to identify design flaws in the abstract. And all the parts of a system can look secure, but the weak points remain in the joinery.

Privilege separation is a key component of OpenBSD security, and interprocess communication is at the heart of it. More focus on what can go wrong with corrupted processes would help. Secure browsers are evolving sandboxes which require longer and longer exploit chains. smtpd in particular is supposed to be secure against memory corruption in the network processes, but the ease with which a child can control the parent is pretty alarming.

Only one errata strikes me as a vulnerability that would have been prevented by using a safer language. Yes, there’s likely some programmatic idiom which if followed religiously may have helped in some cases, but I’d need more evidence to be convinced people would encode the relevant invariants by default.

Writing a mail server is tricky business if you are constrained by the designs that came before.

Posted 31 Mar 2020 04:20 by tedu Updated: 04 Apr 2020 09:15
Tagged: openbsd programming security thoughts