Serving the fediverse from memory

There is a page on this site at /social. It shows what I have been posting to the fediverse, back to October 2023, and it is the only part of this site whose content I do not write in a text editor.

It exists for a small and fairly stubborn reason. I post to a server I run, and I wanted reading it to happen on a site I run, without asking anyone to sign up for anything or install an app to see it.

What the fediverse is, for anyone who has not gone near it

The fediverse is a set of social networks that talk to each other over a shared protocol called ActivityPub.¹ Mastodon is the best known part of it.

The bit people find strange at first is that there is no single mastodon.com to sign up to. There are thousands of servers, each run by somebody, and an account on one can follow an account on another in the same way an email address at one provider can write to an address at a different one. What holds it together is a protocol rather than a company, so there is no middle for anyone to own.

Mine runs Pleroma, a lighter implementation of the same idea that speaks the same API Mastodon does. The server is social.dotmavriq.life and it has one real user on it, which is me.

Why I bother

There is no algorithm, which sounds like a small thing and turns out not to be. Posts show up in the order people wrote them, nothing is weighing what I should see against what kept me scrolling last week, and when I reach the end of what the people I follow have written, I have reached the end. Nothing is trying to keep me there.

What I did not expect was what that does to the way people write. Earlier this week I posted about a Red Admiral sitting on the path just outside our place in Carcavelos. The next day I went looking for pictures of Clavinets, the Hohner keyboard that is all over seventies funk records, and somewhere in the results I found what I was fairly sure was an excellent potato gratin. It turned out to be Tartiflette, so I cooked it for a guest on Wednesday. Nobody writes that kind of thing to grow an audience, mostly because there is no mechanism there that would reward them for trying.

It is what the internet used to feel like to me, before it turned into somewhere you go to earn a living or to watch people have opinions in public. Mostly it was people leaving small things around where somebody else might come across them. I met most of the friends I still have that way, and it turns out that still works fine. It just stopped being what you get by default.

So the page is not a growth channel and I am not building anything with it. I like the thing, and I would rather the record of it sat on a domain I control.

What gets filtered, and why

The page is not a mirror of the timeline. Four rules cut it down, and each one is about noise rather than image.

Replies are dropped. A reply is half of a conversation, and the other half is not on the page. Reading one is like hearing someone answer a question you did not hear asked.

Boosts are dropped. A boost is somebody else's writing. It belongs to them, and passing it through my site as though it were mine is a bit rude even when I liked it.

Posts that open with a mention are dropped. These are replies wearing a different hat. If a post starts by naming somebody, it is aimed at them and not at whoever wandered onto my website.

A second account on the instance only appears when it brings a photograph. That account is mostly there for pictures, so the pictures are what gets through.

Nothing is filtered for being unflattering. There is no rule that removes a post because I later thought better of it, and I have not gone back to tidy the archive. The rules are about whether a post is a piece of writing or a piece of a conversation.

The first version, and what it actually did

The first version fetched the timeline from the visitor's browser, filtered it there, and rendered it with JavaScript. That is a reasonable place to start. The site had no need to know about the fediverse, and the browser was perfectly capable of making an HTTP request.

Here is what it meant in practice.

The page painted three grey rectangles and the word "loading", then posts appeared a moment later. Every single visitor's browser fetched my Pleroma instance directly, so the server's traffic scaled with my readership rather than with my posting. Each fetch pulled twenty posts and the filters threw away about eleven of them, so most of what crossed the wire existed to be discarded.

And it stopped. You would reach the bottom of the first screenful and the page would tell you that is everything, which was not true. There were nearly three hundred posts behind that message.

That last part is the one that actually bothered me, because it was not a slow page or an ugly page. It was a page confidently reporting the wrong thing. A failed request and an exhausted archive were being collapsed into the same message, so any hiccup on the instance truncated three years of posting and said so with total confidence.

The thing I should have checked first

I had been asking for the public timeline and doing the filtering myself. The API will do most of it.

The account statuses endpoint takes exclude_replies, exclude_reblogs and only_media as query parameters.² Pleroma implements the Mastodon API, so the instance applies them before it sends anything. I tested it against my own server and got forty posts back with zero replies, zero boosts and zero mention-openers in them.

Covering the entire archive went from roughly forty requests to ten. Nothing is fetched to be thrown away. The only rule I still apply myself is the mention-opener one, because there is no query parameter for it.

I had written a filtering pipeline to reproduce something the server was already willing to do, which is a specific kind of mistake I would like to make less often: solving a problem in my code because I never asked whether it was a problem at all.

Where it ended up

There is now a small service next to the site, about two hundred lines of Lua on OpenResty, whose entire job is to hold the archive.

It crawls the timeline once, keeps the filtered result in memory, writes a snapshot to disk, and after that asks the instance only for posts newer than the newest one it holds. In steady state that is two requests every ten minutes, and they usually come back empty. Rendering the page reads a string out of memory and writes it to the socket.

The numbers, measured on the live site:

The archive is 284 posts and roughly 300KB, which fits in memory with room to spare for a decade of posting.

Two properties matter more to me than the speed. The posts are in the HTML, so the page works with JavaScript switched off and a search engine can read it. And the instance can be down for a week without /social noticing, because nothing in a page render depends on it.

No infinite scroll

There is a button. It says how many posts you are looking at and how many there are, and when you reach the end it says so.

I did not want infinite scroll. It is the interaction pattern of the exact thing I like the fediverse for not being, and a page that quietly keeps going forever is a page that has decided on your behalf that you are not finished. Clicking is a reasonable thing to ask of somebody who wants to read further.

Three things I got wrong

A null that was true. In Lua, everything except nil and false is truthy, and the JSON decoder turns null into a sentinel object rather than nil. Every post carries reblog: null, so if post.reblog then return false end rejected every post I had ever written. The endpoint returned 200 OK with an empty list and no error anywhere, which is the worst way for a bug to behave.

A 682MB image for two hundred lines of Lua. The HTTP library I needed is three files, but the tool that fetches it needs a compiler toolchain. Building it in a stage that gets thrown away brought that down to 226MB.

I took the site down for about a minute. I had locked the new container down with cap_drop: ALL, and nginx wants CAP_CHOWN at startup to take ownership of its temp directories. It crash-looped, the site was configured to wait for it, and so the site never started.

The interesting part was the symptom. The domain kept answering 200 the whole time, because the reverse proxy fell through to an old static site that still had a route for it. Production looked completely healthy from outside while the actual application was not running at all. I fixed the cause by making the container run unprivileged so it never needs the capability, but the thing I actually took away is that "the URL returns 200" was never the check I thought it was.

Still open

Clicking a hashtag on that page still does the old thing. It goes straight from the browser to the instance, filters in the browser, and inherits every problem the feed just stopped having. I measured one that took fifteen seconds and came back a 500.

The fix is sitting right there, because the service already holds every post and every post already contains its hashtags. That is an index, built once, and then clicking a tag is a lookup rather than a request. It is next.

What it was actually about

I framed this as a performance problem for longer than I should have, and spent a while trying to make a fetch faster.

It was never that. It is 300 kilobytes of my own writing, finite and unchanging except for the few sentences I add to the end of it each week, and I had built something that went across the internet to ask for it again every time a person opened the page. Something that small, that stable and that much mine belongs in memory, and once it was there the pagination stopped being arithmetic I could get wrong, most of the failure modes stopped existing, and the page got fast as a side effect rather than as the goal.