I’ve been a rather happy lua user for a few years. In particular, the luajit implementation. But as part of an ongoing overhaul of this and that, I decided to rewrite all my lua code in go. Or wait, let me rephrase that.

As our signature service, flak needs to be at the vanguard of web technology. In order to maintain our commitment to performance and availability, we are constantly reviewing new languages and frameworks, and as we move into the future, it becomes necessary to reevaluate old decisions. Adapting new cloud ready technologies allows us to continue delivering the fast and stable responses our customer base deserves. In this ever changing world with our universal pursuit of extraordinarily high standards, no aspect of a blog could be more important than the language its framework is written in.

flak has gone through a few iterations over the years, gradually swapping out one component for another. Mostly, I’d start using a library written by somebody else to get started, then eventually replace it with my own version. On the one hand, this meant I had a good grasp on what needed to be done for a rewrite. On the other hand, since everything was bespoke, I wouldn’t find many libraries to help.

There are two mostly custom text template engines in flak. One is a custom markdown to HTML converter that posts are written in. The second is the HTML templates that the site itself is written in. Neither is particularly advanced or complicated, but for a blog, this represents the majority of the functionality. Tackling the second was pretty easy. Go includes a basic HTML template library. It has its quirks, but it’s definitely usable. This required rewriting the templates, but there’s only a few of them. About 200 lines of HTML. The first engine was more trouble. Translating hundreds of old posts into a more fashionable flavor of markdown wasn’t happening. Instead, 100 lines of gross regexes were translated from lua to go.

There’s almost a one to one correspondence in route handlers. I think a number of frameworks have converged in the same direction, but function showpage(request, response) and addHandler("/url", showpage) (lua version) was a very easy mapping to go http handlers. Other language features, like anonymous functions and closures, are also present in go and can be mechanically translated. In a few places, I’d written some polymorphic functions in lua, but it was probably better to split them into distinct functions anyway.

Some parts of flak that needed to be rewritten had already been so. Syntax highlighting in flak was accomplished with pygments. This had been one of the slowest and most awkward dependent components for a while. I’d already written a minimal replacement for that in go, so that was dropped in. And having already rewritten my RSS reader from lua to go, I had some basic login auth code ready, too.

There are some SQL statements to query the database. The same SQL statements return the same data when executed from go. The main difference here is that in lua one can return each row as a table with column names as keys. Go requires unpacking each row with Scan(). This requires a bit more care, in that column order matters. And select * is a timebomb waiting to explode. On the other hand, the difference between string and template.HTML would have resolved numerous troubles throughout flak’s early development.

Go and lua take a similar bytes can be strings approach to ünicöde, which makes me happy.

One might say that flak was a victim of aggressively premature optimization. Even though the database was sqlite3, I was treating it like it was a million miles away, going to great lengths to avoid querying it. One of the problems with caching is it handles common requests, but that’s no reason to live with high latency for uncommon requests, right? So I’d try to break each page apart into pieces that could be cached individually. This was probably taken a bit too far.

The new flak makes no effort at caching, except that which can be added by a front end proxy. Nevertheless, page render time is in the low millisecond range. Good enough, I think. Conveniently, go makes it trivial to scale to at least a modest number of cores, something which required a multiple process architecture for lua.

One feature of luajit I relied upon was instant startup. I could write a script to restart flak whenever any file changed and see instant results in the browser. With go, the cycle is a bit longer since I have to wait for it it recompile. Can use gomacro here, but it’s not transparent. On the bright side, since I inevitably type psot in all new code, catching this in the compiler helps. Using a strict module for lua had been a life saver, but a compiler works even better.

One of the quirks of the old web framework I’d written was that it would rewrite URLs to add prefixes. So that a link to /random would become /flak/random in the final HTML. At the time, I thought I wanted to run a dozen applications off the same domain, while still wanted the convenience of using unprefixed paths internally. But this required a precarious set of patterns. A form would only be rewritten if the action came first, not the method. I didn’t feel like implementing this in go, even if it meant changing URLs.

Moving to URLs is trouble for RSS readers, however. Other sites seem to simply republish all the old links with the new URLs, but that seems obnoxious. So there’s a quick hack in place to restrict RSS entries to posts published after the changeover.

Keeping dependencies light has had its benefits. I knew I could rewrite everything, having written it the first time. Sometimes one gets trapped by a requirement to use a particularly library’s functionality, and then migrating to a new language requires finding an equivalent replacement.