Yet another PDS implementation?

It would've been funnier if I could've been number 69 (yeah I'm easily amused, why?) but alas, 'twas not to be. Image lifted from @pds.dad and saved lovingly to my folder of random memery.

So. It's not actually happening. Yet. Just kind of randomly pontificating into the void, pondering my orb, shaving the yaks, the usual.

But why?!

Well, since I got @peedee.es done I've been toying with supporting more than just the reference PDS. Because while the reference PDS is like, the reference, like, you know, dude - it's also not exactly a shining beacon of efficient use of resources. I mean, a CAR import will use anywhere from 20 to 44 times the amount of memory as the CAR file size. So importing a 256Mb repository (not unheard of) is going to mean you need an absolute fuckton of memory just to pull it off.

And that's where things kind of fall flat when you're trying to run it using a SaaS model because guess what? Memory is restricted based on how much you pay me. Because without upper bound, you end up either with noisy neighbours, or with oversubscribed physical nodes. And that's bad(tm).

Okay so, like Tranquil. Uses less memory, but has it's own issues. Cocoon? Same thing. The reference PDS needs a sidecar that pretty much absorbs and takes over imports, exports, rate limits, and to just provide a slightly sane API that the SaaS platform can use to "just do stuff". Tranquil? Needs the same sidecar. With fewer moving parts, but the parts that do move are way more complex because in order to do certain things, there's direct grubbing in Postgres involved. Cocoon? Same story. Still needs that sidecar.

And every time I look at a different PDS implementation, the design of that sidecar grows. And grows. And is like, turning very quietly into it's own PDS that just sort of pulls a "The Thing" and horks the good bits out of every implementation and absorbs it for it's own (not nefarious) purposes.

Which is kind of not the greatest because that sidecar uses memory too.

So whatcha gonna do then?

Well, for now, not a damn thing. I'm going on vacation soon, for a whole month. No social media, no computer, I'm going to most likely hate it and probably end up getting my fix in an internet cafe, but that aside.

But after that...

Software As A Service

Generally on any SaaS platform, you don't get your very own little app running; there is only one app, which gets scaled out horizontally, that can handle every "tenant". And there you have it.

The idea!

A single PDS binary. That handles everyone's "PDS". Instead of getting the whole house? You rent a room. And that's your PDS. Data can be kept isolated, and since this PDS binary can run multiple instances of itself, if one borks out? It'll keep working. If an entire physical node goes kaka? It still works. If the entire datacenter burns down? Yeah, okay, you got me, then we have a problem.

From my side, this is ideal because as things grow (or a "room PDS" grows), I can just throw more hardware at it. I guess it's like, similar to what Bluesky has done with their PDS stack + Entryway except all rolled into a single binary

The wailing and gnashing of teeth

I can hear it. I can hear it, vaguely in the distance. "But my data, next to someone else's data?!". Yeah no, see, it's isolated in storage. Like, you got the key to your room, and nobody else does. Unless you were dumb enough to give it them.

Why this is either absolutely brilliant or incredibly stupid

See the image above? Yeah. Either this is brilliant because it gets me what I want (which also indirectly means I can cut pricing down even further) and it will be fully spec compliant which is also nice - ooooor - it's really stupid because why the fuck would I want to build it and support it and test drive it and open source it and probably have to deal with all the fucknutters who are all "ehrmehgehrd there's an AGENTS.md in there fuck you blahblahwhaaambulance".

So the jury is out on this.

You said agents?

I said agents. Well, AGENTS.md but you know what that means. Yes, I use agents in my development work. I think after 30 odd years of hand-rolling code I'm allowed to, you know, not do the grunt work? It honestly doesn't make a difference if I sit here typing it or whether an agent does it. Oh, yeah, no, it means I have to read it during review - and that's fine, I can always edit things myself afterwards, or ask the agent to stop being a dumb muppet.

Suffice to say, it leaves me with so much more time to spend on actually engineering what I want, in the way that I want, with the end goal of it actually doing what I want. In an acceptable time frame, no less.

Anyway.

In closing

So yeah. There you have it. Either this is going to go really well, or it's going to be really stupid and I'll have to make some sort of "welllll it didn't work out" post about it.

Who knows.

I don't...