Skip to content
Tag

Ruby

Articles and publications tagged Ruby across the Atmosphere.

78articles23publications
MacKuba blog
MacKuba blog
Oct 15, 2025
How I ran one Ruby app on three SQL databases for six months
Since June 2023, I’ve been running a service written in Ruby (Sinatra) that provides several Bluesky custom feeds (initially built with a feed for the iOS/Mac developers community in mind, later expanded to many other feeds). If you don’t know much about Bluesky feeds, you make them by basically running a server which somehow collects and picks existing posts from Bluesky using some kind of algorithm (chronological or by popularity, based on keyword matching, personal likes, whatever you want), and then exposes a specific API endpoint. The Bluesky AppView (API server) then calls your service passing some request parameters, and your service responds with a list of URIs of posts (which the API server then turns into full post JSON and returns to the client app). This lets you share such feed with anyone on the platform, so they can add it to their app and use it like any built-in feed. (If you’re interested, check out my example feed service project.) In order to provide such service, in practice you need to connect to the Bluesky “firehose” streaming API which sends you all posts made by anyone on the network, and then save either those which are needed for your algorithm, or save all of them and filter later. I chose the latter, since that lets me retry the matching at any time after I modify the keyword lists and see what would be added after that change (and also some of the feeds I now run require having all posts). I also use the same database/service to generate e.g. the total daily/weekly stats here. All posts made on Bluesky is much less than all posts on Twitter, of course, but it’s still a lot of posts. At the moment (October 2025), there are around 3.5M posts made on average every day; at the last “all time high” in November 2024, it was around 7.5M per day. A post is up to 300 characters of (Unicode) text, but since I also store the other metadata that’s in the record JSON, like timestamp, reply/quote references, embeds like images and link cards, language tags etc., it adds up to a bit less than 1 KB of storage per post on average. In addition to that, the firehose stream (if you use the original CBOR stream from a relay, not Jetstream, which is a JSON-serving proxy) includes a lot of overhead data that you don’t need in a service like that, plus all the other types of events like handle changes, likes, follows, blocks, reposts, and so on. The total input traffic is around 15 Mbit/s average right now in October 2025 (or around 5 TB per month), and it used to be around twice that for a moment last year. (Jetstream sends around an order of magnitude less, especially if you ask it to send filtered data, e.g. only the posts.) On disk, the millions of posts per day add up to a few gigabytes per day. Since I was running this on a VPS with a 256 GB disk (Netcup, RS 1000 – reflink), I have a cron job set up to regularly prune all older posts and keep only e.g. last 40 days worth of them (since I don’t really need to keep the older posts …
19
RubyDatabases