Engineering · 7 min

Anatomy of the Augur ingest stack

Postgres for state, Node workers for adapters, a service-role Supabase client for cross-tenant writes. The full pipeline from poll-loop to canonical_events.

2026-05-25

A short tour of how an OSINT event becomes an alert in Augur.

1. Adapters

One TypeScript module per source under packages/ingest-adapters. USGS reads a GeoJSON summary every 30s. EMSC reads a FDSN feed. NHC reads the ATCF advisory text every 5 minutes. Each adapter exports a single function: fetch(since: Date): Promise<CanonicalEvent[]>.

2. Canonical events

Adapters output a flat, source-agnostic shape: { source, type, ts, severity, geo, payload, external_id }. The external_id is the source-provided unique key. Combined with source we get a stable dedupe key even if the same event re-emerges on a later poll.

3. Severity scoring

The adapter computes a 0-100 severity at ingest time using source-specific signals (USGS magnitude + PAGER, NHC cone + landfall, GDELT goldstein + tone). The full rationale lives in the alert-fatigue post.

4. Postgres state

We use Postgres for everything — event log, dedupe via partial unique indexes, alert history, dwell-time tracking, audit log. The big advantage is that there's exactly one consistent backend; no eventual-consistency stories to debug.

5. Dispatcher

Every 5 minutes a cron job hits POST /api/alerts/dispatch. The handler loads recent canonical events, joins them against active watch zones using haversine (circles) or ray-cast (polygons), respects per-zone snooze + dwell-time + severity threshold, and fires through each user's alert channels. Dedupe via (user_id, zone_id, event_id) partial unique index.

6. Service-role boundary

The dispatcher writes across tenants, so it uses the Supabase service-role key. Reads from user-facing dashboards use the RLS-scoped client. The service-role key never leaves the backend (Vercel env, VPS env files).

The whole pipeline is ~3kLOC of TypeScript. We're hiring (/careers) — if this stack appeals, email hiring@augur.news.

← Back to blog · Start free