We have a new edition of POSIX, which means new rules, new regulations, new red tape. But not new lines. Which is to say, posix at long last banishes new lines in file names.

more...

What if we want a grep that doesn’t stuck but we don’t want to resort to wild hacks like editing the source? What if there was some way to flush stdout automatically?

The auto flusher is a very simple preload.

#include <stdio.h>
#include <unistd.h>
#include <pthread.h>

static void *
flusher(void *arg)
{
        while (1) {
                sleep(3);
                fflush(stdout);
        }
}

__attribute__((constructor))
void
herewego(void)
{
        pthread_t thread;

        pthread_create(&thread, NULL, flusher, NULL);
}

Magic constructor attribute for initializers in C, and then we create a thread which loops around occasionally flushing stdout, so if there’s any data left lingering, it will eventually find it’s way out instead of waiting forever.

$ cc -shared -lpthread -o libflusher.so flusher.c
$ env LD_PRELOAD=./libflusher.so grep ...

Sometimes pipes get stuck. To recap, if we run this command...

~/src/usr.bin/grep> tail -200 util.c | grep unsigned | grep int
grep_revstr(unsigned char *str, int len)

... we get the expected output. If we run it this way, however, there’s no output.

~/src/usr.bin/grep> tail -f -200 util.c | grep unsigned | grep int

The file isn’t being appended, meaning tail won’t exit, so we don’t expect the pipeline to ever finish, but we would like to see the current results.

Fortunately, grep provides an lbflag we can set. In this case, if the input is a pipe, we’ll assume there’s something fancy going on, and switch to line buffering.

        struct stat sb;
        fstat(0, &sb);
        if ((sb.st_mode & S_IFMT) == S_IFIFO)
                lbflag = 1;

Easy fix. (Or one can read the manual.)

But this is excessive. What about those times when we want the efficiency of buffering? There are lots of possible pipelines that don’t involve tail. What we can do instead is take a peek at some of our neighbors and see what’s really going on. Not perfectly accurate, but probably good enough.

static void
checklbflag()
{
        pid_t pgrp = getpgrp();
        struct kinfo_proc ki[24];
        size_t len = sizeof(ki);
        int mib[6] = { CTL_KERN, KERN_PROC, KERN_PROC_PGRP, pgrp,
                sizeof(ki[0]), sizeof(ki)/sizeof(ki[0]) };

        sysctl(mib, 6, ki, &len, NULL, 0);
        for (size_t i = 0; i < len / sizeof(ki[0]); i++)
                if (strcmp(ki[i].p_comm, "tail") == 0)
                        lbflag = 1;
}

Alas, this requires including <sys/sysctl.h> which will slow down our compiles.

It is also possible, using forbidden magic, to determine whether the pipe we are reading from is being written to by tail, as opposed to a tail at the end of the pipeline, but that’s left as an exercise for the reader.

As noted, Valve uses the same flicker effect for broken lights in all their games. What if I want a broken web page?

more...

The Honk Foundation is pleased to announce the initial release of Enterprise Social, previously known as Honk Enterprise, previously known as honk3. It’s reworked and retuned, reimagined and remade. A social network for the future.

more...

A response of sorts to commit robbery.

I’ve been on both sides of commit misattribution. Sometimes my commit gets sniped. Sometimes I’m the one doing the sniping. It’s very rarely intentional.

In a project like OpenBSD, we’d have three Todds and four Jasons, which led to frequent confusion. Forgetting exactly who contributed what would happen occasionally. Emails get forwarded around. And I think every new OpenBSD developer goes through the experience of having half their commits sniped for the first month because other developers think you’re submitting a patch, not requesting review.

I’d guess 1% of commits are misattributed in some way. There’s an acknowledgement, but it’s incomplete. Or it’s omitted. Or it credits somebody else entirely. I think that’s not bad overall, though it does mean 1% of first time contributors have a suboptimal experience. It sucks to be the statistic.

Could the 1% be improved? Sure, just require a few more TPS reports before commits are permitted. If the solution is that we simply need to ask the maintainers to care more, are we sure they don’t care? There’s already enough people burning out and dropping out of open source work.

I think Ariel’s story could have been framed better, with a rather less villainous counterparty. I almost didn’t read it, because I don’t care about commit drama, but the story of the debugging was really interesting. I’m glad I read that! But hey, your blog, your story. That said, I don’t think it’s possible to generalize from one event to sweeping conclusions like this is why people don’t contribute to open source. This is, maybe, why 1% of people don’t contribute to open source.

Big Stomper is out to get you. Better watch what you say, or you’ll get stomped.

Of course, they won’t admit to stomping you. They’ll even deny it. Make it look like an accident or natural causes, an unfortunate rockslide. But you’ll be stomped just the same. All for speaking out about their misdeeds.

Not everyone believes Big Stomper is even responsible, but I hang out on the Stomp Watch forum, where we catalog and discuss the evil things Big Stomper is up to. So we know that when something bad happens, Big Stomper is to blame. Sometimes we debate if the stomping is carried out in house, or contracted out, but we are all agreed that the murky lack of evidence is a clear sign that professionals are involved. These are not amateur stompers.

Because I know what Big Stomper is capable of, it would be irresponsible not to warn you what happens to anyone who reveals their secrets. Everyone at Stomp Watch would love to discuss your revelations, but you need to know that you will definitely get stomped. Big Stomper does this to dissuade future leakers.

Some people (there’s always that “someone” on the internet, right?) will tell that it’s safe to come forward, these are just rumors and wild speculation. Well, they’re not the ones risking their lives, are they? Anyone inside Big Stomper who feels obligated to disclose the truth should come check out Stomp Watch first to know the risks.

For some of the software I write, I try to make occasional releases, and for the go software I vendor the dependencies so it’s all there. I was just reminded that I hadn’t made a release of something in five years, so I reran my release script, and ended up with a tarball that was 10x bigger than the previous one. Some terrible choices have been made.

 317140 May  6  2019 humungus-0.9.6.tgz
3094567 Apr 30 22:25 humungus-0.9.7.tgz

There’s been five years of development, but that’s really not that much. I most certainly have not 10xed the functionality. Where did this all come from?

I poked around the vendor directory for a bit for some large files. One of them is hsluv-snapshot-rev4.json, a 1.5MB json file. I wouldn’t recommend clicking on that link, unless your browser likes displaying single lines of text 1.5MB bytes long.

I have no idea what this file does. The only color I need is for a little green menu in the terminal admin interface. I checked, and go-colorful was a dependency in previous releases, but it didn’t have this file at the time. Apparently I still don’t need it. I just deleted it after running go vendor and everything still works. That reduced the size of the final release tarball by 500K.

2MB (post compression) still to go, or thereabouts. I was in a hurry, so I didn’t actually trim anything more down. The majority of the code is the golang/sys/unix repository. I still have to find out what’s dragging this in, and then delete or rewrite it.

Ah, there it is. tcell nonblock_bsd.go imports sys/unix. 9.5MB of vendored source code to set a flag on a file descriptor.

I have tried to be mindful of the number of dependencies I’m adding. I keep an eye on go.mod to make sure it doesn’t explode. But I’ve been lax in running du on the vendor directory. Even limited sets of dependencies are subject to extreme bloat.

I added ActivityPub support to humungus using the vocabulary defined by ForgeFed so now I can follow my commits with honk. Statements dreamed up by the utterly deranged.

more...

Memory leaks have plagued C programs for as long as the language has existed. Many solutions have been proposed, even going so far as to suggest we should rewrite C programs in other languages. But there’s a better way.

Presented here is a simple solution that will eliminate the memory leaks from every C program. Link this into your program, and memory leaks are a thing of the past.

#include <dlfcn.h>
#include <stdio.h>

struct leaksaver {
        struct leaksaver *next;
        void *pointer;
} *bigbucket;

void *
malloc(size_t len)
{
        static void *(*nextmalloc)(size_t);
        nextmalloc = dlsym(RTLD_NEXT, "malloc");
        void *ptr = nextmalloc(len);
        if (ptr) {
                struct leaksaver *saver = nextmalloc(sizeof(*saver));
                saver->pointer = ptr;
                saver->next = bigbucket;
                bigbucket = saver;
        }
        return ptr;
}

It is now entirely optional to call free. If you don’t call free, memory usage will increase over time, but technically, it’s not a leak. As an optimization, you may choose to call free to reduce memory, but again, strictly optional.

Problem sovled!