Possum - Querying public data from your PDS

Posts

I really like the idea of having a place on your personal website where you can just share your thoughts and opinions. When im visiting someone's blog I always look for a /posts somewhere in the page.

I tried to make something similar during the first iteration of my website: During the build process I converted some markdown files into HTML and called it a day. It worked alright for a while, but figuring out how to make syntax highlight work on my code blocks was not worth the effort at the time. Thats when I started using Leaflet for my blog posts instead.

standard.site

Firstly, Leaflet feels really nice to use. The text editor UX is great, its possible to edit my posts on mobile, and code blocks are have proper syntax highlight. The main diference is that my posts are not stored on my website's codebase anymore, they live in my Personal Data Server (PDS), together with the rest of my records.

What I like the most about ATproto, is that its all publicly available. Maybe, if I could find a way to query my articles, I could display them on my website again! They all use the standard.site.document lexicon definition, the fields are well documented and can be easily turned into a valid Gleam type. After some research I found that I can make HTTP requests directly to my server. Now that I learned how to use the API, all I needed to do was to parse the response body using the decode module from gleam's standard library.

const url = "https://bsky.social/xrpc/com.atproto.repo.listRecords"
  <> "?repo=kacaii.dev"
  <> "&collection=site.standard.document"

fn fetch_articles() -> effect.Effect(Message) {
  let decoder = {
    let value_decoder = decode.at(["value"], article_decoder())
    use articles <- decode.field("records", decode.list(value_decoder))
    decode.success(articles)
  }

  let handler = rsvp.expect_json(decoder, ApiReturnedArticles)
  rsvp.get(url, handler)
}

Good enough, now I can continue writing on Leaflet peacefully, and my website will take care of displaying a preview of my latests posts (including the one you are reading right now).

Community

That was just a HTTP request, the usage is simple but took a bit of setup in order to do hit the right endpoint. I needed to know what my Decentralized Identifier (DID) is, and you also need where my PDS is located. If I can take care of that, I believe this could help other projects. There are quite a lot of gleam developers interested in the AT Protocol but not many atproto related packages, I want to make the process of accessing that dataset easier for both communities.

I decided to make a library for building this kind of requests. Its common in gleam to use the sans-io pattern: Build the requests, and let the user decide when, and how to send them. The following code will result in a response that contains the user's DID.

const com_atproto_identity = "xrpc/com.atproto.identity"

/// Resolves an atproto handle (hostname) to a DID.
pub fn resolve_handle(
  handle: handle.Handle,
  pds host: String,
) -> request.Request(String) {
  request.Request(
    method: http.Get,
    headers: [#("accept", "application/json")],
    body: "",
    scheme: http.Https,
    host:,
    port: option.None,
    path: com_atproto_identity <> ".resolveHandle",
    query: option.None,
  )
  |> request.set_query([#("handle", handle.to_string(handle))])
}

You can set the host parameter to an API like slingshot, or even bluesky. I included some examples on the documentation for the library that can serve as quick references. I want to avoid the usage of invalid identifiers, so I also included a few submodules containing basic string parsers. The AT Protocol community was nice enough to include an basic regex pattern on the documentation, it may not be suitable for large-scale apps*¹, but its good enought for a small projects.

Now that you have access to your DID, you'll need to know where your PDS is located. This can be done with the PLC Directory. Just send a GET request there containing the DID you just acquired as the path, and it will return plenty of useful information, including the address of your Personal Data Server!

import gleam/dynamic/decode
import possum
import possum/did

let assert Ok(did) = did.parse("did:plc:z72i7hdynmk6r22z27h6tvur")
let request = possum.get_plc_data(did)

// You can use this decoder for the endpoint field.
let decoder =
  decode.at(["services", "atproto_pds", "endpoint"], decode.string)

Fetching the records

Now that you have access to both your DID and your PDS's endpoint, it will become really easy to query data from it. Just set the request host to your PDS, pass your DID as parameter to the functions and decode the response received into a custom type for your application.

import blog/atproto
import gleam/dynamic/decode
import gleam/option
import possum
import rsvp

fn fetch_documents(atproto: atproto.AtProto) -> effect.Effect(Message) {
  let decoder = {
    let value_decoder = decode.at(["value"], document_decoder())
    use documents <- decode.field("records", decode.list(value_decoder))
    decode.success(documents)
  }

  let request =
    possum.list_records(
      atproto.did,
      pds: atproto.pds,
      collection: atproto.standard_documents,
      limit: option.None,
      cursor: option.None,
      reverse: option.None,
    )

  let handler = rsvp.expect_json(decoder, ApiReturnedDocuments)
  rsvp.send(request, handler)
}

It can also be used to fetch blobs*² from your PDS. The at_uri module contains utilities for AT-Uri string parsing. You can either store the response body in a BitArray or point the src attribute to where the image is located. You can easily use it with lustre!

let image = case model {
  Pending ->
    html.img([
      // Using the favicon as a placeholder
      attribute.src("/favicon.webp"),
      attribute.class("aspect-square size-20 rounded-sm border opacity-50"),
      attribute.class("animate-pulse"),
    ])

  Model(profile:) ->
    html.img([
      attribute.class("bg-muted aspect-square size-20 rounded-sm border"),
      attribute.class("hover:brightness-110"),

      // Pointing source attribute to a file that lives in my PDS
      possum.get_blob(atproto.did, profile.avatar_cid, pds: atproto.pds)
        |> request.to_uri
        |> uri.to_string
        |> attribute.src,
    ])
}

Possum is available on hex package manager. I received a lot of support from both gleam and tangled communities, and im quite happy with how the project is going. Constructive feedback is always welcome and I'm looking forward to learn more about the ATproto ecossystem.