Capability
← All capabilitiesReal-time data systems
Pipelines that keep up with a source that never stops and never waits for you.
Systems where the data arrives whether or not you are ready: a chain tip, a security event stream, a market feed. The hard parts are not throughput but correctness under interruption — exactly-once semantics where they matter, checkpointing that survives a crash mid-block, and backfill that can run against the same code as the live path without racing it.
How we approach it
The design decision that separates a real indexer from a polling script is treating live-follow and historical-backfill as two modes of one program rather than two programs. Two codebases drift, and the drift shows up as a silent gap in the data six months later. Running one binary in two modes, each with its own persisted checkpoint, means the backfill exercises the same parsing and the same writes as the tip follower — so a bug is found once, not twice, and never in production only.
Stack
Rust · PostgreSQL · Redis · Kafka · ClickHouse · WebSocket
AI security operations platform
Client undisclosed · 23 services in the deployed topology
A 23-service pipeline that ingests security telemetry, enriches it in flight, and runs its own models over the stream.
Token launchpad indexer
Client undisclosed · 12 services in production
A Rust indexer following the tip of a BNB Chain launchpad and DEX, with historical backfill running the same binary.