The sync engine

The sync engine is what makes inventory, topology and the dashboard reflect reality without the console ever calling a cloud provider directly from a page request. Everything a screen shows was produced by a background pass and written to the Store.

Three triggers, one engine

• Scheduled — GET /api/cron/sync runs on a schedule (Vercel cron), authenticated by a CRON_SECRET Bearer header. Locally, an in-process pump drives the same runner unless CC_SYNC_DISABLED=1.
• Connect-kick — the first time a connection turns healthy, a sync pass starts within seconds so inventory appears almost immediately after you connect.
• On-demand — the 'Sync now' button (POST /api/cloud/accounts/{id}/sync) claims that account's units and runs them within a bounded time budget; anything deferred finishes on the next scheduled pass.

How a pass works

A pass claims work units with a lease so two passes never do the same work. Each provider has a persisted token bucket: when a provider rate-limits, the engine backs off with jitter and the bucket refills over time, so sync is bounded and polite. Delta cursors let a pass fetch only what changed since last time. All of this state lives in the database, so a pass that is interrupted (a serverless function timing out) resumes cleanly on the next run.

Freshness and SLAs

Each resource type carries a freshness record — when it was last synced, whether it is stale against its SLA, and the SLA window itself. The console shows a 'last synced' marker and, when a connection is in warning, keeps serving the last known-good inventory (stale-but-served) rather than showing nothing.

Soft-delete reconciliation

When a resource disappears upstream, the engine marks its row soft-deleted with the reason removed_upstream rather than erasing it. When you disconnect an account, its rows are marked account_disconnected. Both remain queryable with includeDeleted=1 on `GET /api/cloud/resources`, so history is never silently lost and the two reasons stay distinguishable.