Stillwater

Latency you can read at a glance, for every region on one screen.

Requests a day, on one pipeline

was 600 million before the rewrite

Client

Stillwater

Role

Staff engineer, ingest pipeline

Timeline

Two years, 2019 to 2021

Built with

Go, Rust, Postgres

Stillwater latency screen: an orange heat map of checkout latency over 24 hours, with four figures above it

The problem

The ingest pipeline fell over at 600 million requests a day and the fix on the table was a bigger cluster.

What I did

I rewrote the hot path in Go around one append-only log, with back-pressure instead of retries. We shipped it a region at a time, old and new side by side.

What changed

4.1 billion requests a day on fewer machines than before, and a latency screen people could read from across the room.

Next project

Sandeel

Have a number that needs moving? Tell me what it is today.

Next free in January 2027. Send the number, where it is measured, and what it costs you. I reply within a day.

Start a conversation

idris@halloran.example