flak rss random

another tale of go.mod bloat

It’s been one year since our previous adventure, so it’s time for another round of guess why that dependency shows up in the tarball. This time we’re looking at honk, an ActivityPub server that’s supposed to be idiosyncratic with minimal dependencies, so you can keep all your attention focused where it’s needed.

Looking at go.mod, it doesn’t look so bad, but a closer inspection of go.sum reveals a lot of packages that don’t seem to belong. goldmark (a markdown parser) and gopkg.in/yaml.v3 and the list goes on, and they have nothing to do with any functionality I’m aware of. This is all being dragged in by gofeed, one way or another. For reasons, I’m also checking some Mastodon RSS feeds, and this seemed like a quick way to turn XML into an array of links, but now we’ve hit the whole iceberg.

A momentary digression while I discuss a bug in go doc that infuriates me, and provides some context. I take my laptop out to the park looking to do some distraction free coding, and I run go doc mux to check something, but I get an error: “go: .../testify@v1.8.1: Get proxy.golang.org/...: dial tcp: lookup proxy.golang.org: no such host”. I just ran go build ten seconds ago and it worked, I have all the dependencies needed, but somehow go doc has gotten itself into a state where it absolutely needs to check something online. This is not a warning that maybe something will be out of date or missing, it’s a hard error. It is not possible to read any documentation until I pull out my phone and switch on the hotspot, all so that go doc can fetch whatever this file is. The offline doc reader that’s terminally online. My dream is that by sufficiently trimming dependencies, I will avoid such situations in the future.

Back to our review of go.mod, I notice we’re pulling in modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd. Chatbot, is a module last updated over seven years ago likely to be “modern”, whatever that means? Why is this here?

Well, it turns out gofeed also includes a JSON feed parser. I am certain I have never seen a JSON RSS feed, but some wanker didn’t like that there’s both RSS and Atom, and decided that N+1 would make things better. (This functionality did not exist when I started using gofeed, but it got sucked in with an update along the way.) And gofeed uses json-iterator for parsing because gotta go fast, which uses concurrent because sync.Map was not added until go 1.9. The current release of go is 1.24. Why are we still doing this?

Now, I already had some code (types, mostly) to generate my own RSS feeds with xml.Encoder and it turns out that can be used with xml.Decoder very simply. Just sprinkle in a few new functions, and I can drop the entire gofeed dependency. What does the difference look like? The release tarballs:

-rw-r--r--  1 tedu  staff  2392731 May 21 19:07 honk-1.4.2.tgz
-rw-r--r--  1 tedu  staff  1353918 May 21 19:07 honk-e10334d4b7c0.tgz

I just cut my bandwidth costs in half.

I have a philosophy that the amount of code should be roughly proportional to the provided functionality. And despite being nearly half the code, this certainly did not amount to anywhere near half the functionality of the final program.

In fact, looking at the added code and removed dependencies commits, there are fewer code lines added than dependency lines removed. Not the code for the dependencies, but merely the list of their names was longer than the code needed to implement the utilized functionality. Something about bananas and jungles.

Posted 22 May 2025 07:27 by tedu Updated: 22 May 2025 07:27
Tagged: go programming