Docs / infrastructure.md · mirrored from the repo

Cloudflare infrastructure — setup & purpose

Status: operational reference. What’s provisioned on Cloudflare, why each piece exists, how it’s set up, and how it maps to the two-plane architecture. Companion to infra/README.md (the runbook), docs/CI-CD.md (how it deploys), and docs/BACK-OFFICE.md / docs/ARCHITECTURE.md (why).

1. Principles

• Everything Cloudflare by default — Workers + Workflows + Agents SDK on D1 / R2 / KV / Vectorize / Queues / Workers AI + AI Gateway. External tools only when CF genuinely can’t.
• Alchemy IaC is the source of truth for backing resources (infra/alchemy.run.ts, pure TypeScript). Alchemy provisions the resources; Wrangler deploys the Workers/Pages that bind them by id/name in each wrangler.jsonc. We don’t make Alchemy fight the deploy workflow.
• CI is the deployer, not Cloudflare’s Git integration — path-scoped, uniform across Pages + Workers (CI-CD.md).
• Reproducible per environment — -staging now, -prod later; same code, different resources.

2. ⚠️ Account safety — the one footgun

There are two Cloudflare accounts; the canonical one is dash-ayushman-99 (b00772422a2e2a143f0b2058b85155bc). Alchemy picks the first account arbitrarily when a token spans several, so alchemy.run.ts hard-fails unless CLOUDFLARE_ACCOUNT_ID is pinned to the canonical id. Local wrangler + MCP were once bound to the wrong empty account — always target dash-ayushman-99 (memory cloudflare-account-topology).

3. Resource inventory (staging)

Provisioned by Alchemy (adopt: true everywhere, so re-runs re-adopt rather than error):

Plan requirements: Queues + Vectorize need the Workers Paid plan; R2 must be enabled on the account.

Secrets Store entries (values from env — GitHub Actions secrets in CI, infra/.env locally; never the repo). Provider API keys live inside the AI Gateway (BYOK), not here:

4. Compute + edge (deployed by Wrangler/CI, not Alchemy)

• Pages — namaste-ji-console-staging (+ namaste-ji-console prod), namaste-ji-brand. Direct Upload (Git auto-build OFF) so CI is the only deployer.
• Workers — console-api-staging/-prod (Hono), events-calendar, creative-head, content-creator. Wrangler v4, Node 22.
• Cloudflare Access — the whole back office sits behind Access at bo-staging.namasteji.org (same-origin: Pages UI on /*, Worker on /api/*; workers_dev off).
• SSL (free plan): Universal SSL covers root + first-level subdomains only → BO hosts stay first-level (bo-staging, bo); never second-level (CI-6).

flowchart TB
  subgraph Edge
    ACC[Cloudflare Access] --- PGS[Pages: console UI]
    PGS --- API[Worker: console-api]
  end
  subgraph Agents
    EC[events-calendar]; CH[creative-head]; CC[content-creator]
  end
  API --> D1c[(D1 console)]:::pii
  CC --> D1cat[(D1 catalog)]; CC --> R2[(R2 content)]; CC --> VEC[(Vectorize)]
  CC --> RQ[[render queue]]; CC --> EQ[[embed queue]] --> DLQ[[DLQ]]
  EC --> D1e[(D1 events)]; CH --> D1ch[(D1 creative-head)]
  API --> KV[(KV config)]
  EC & CH & CC --> AIG{{AI Gateway}} --> LLM[(LLM / embed providers · BYOK)]
  API & EC & CH & CC --> SS[(Secrets Store)]
  classDef pii fill:#fbeaf0,stroke:#993556;

5. Setup (one-time)

1. Enable R2 and confirm the Workers Paid plan on dash-ayushman-99.
2. Create a scoped API token for the canonical account; fill infra/.env (copy from .env.example) with CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID (pinned), ALCHEMY_PASSWORD.
3. Provision: npm --workspace=infra run deploy:staging.

   Local path has a space (“Other Projects”) which breaks the bundled alchemy CLI — run the program directly: cd infra && set -a && . ./.env && set +a && node alchemy.run.ts.

4. Copy the printed resource ids into the consuming Workers’ wrangler.jsonc (staging database_id, etc.) and uncomment the R2/Queue/Vectorize/Secrets-Store binding blocks.
5. Apply migrations via Wrangler/CI (Alchemy owns the resource, not the schema): console RBAC in services/console-api/migrations/, catalog in packages/catalog/migrations/, per-agent schemas in each agent’s migrations/.

6. Deploy flow (see CI-CD.md)

• Staging auto-deploys on merge to main (path-scoped): deploy-console.yml (+ per-agent workflows). Prod is manual + gated by the production GitHub Environment (deploy-console-prod.yml). Alchemy runs via infra.yml (workflow_dispatch).
• CI CLOUDFLARE_API_TOKEN needs Workers + Pages + Secrets Store scope. Set GitHub secrets with gh secret set NAME --body "$VALUE" (piping over stdin can add a trailing newline → 6111 Invalid Authorization header).

7. Alchemy state

State is local (.alchemy/, gitignored). Because every resource uses adopt: true and AI Gateway create is idempotent, the program is safe to run from anywhere even without shared state — an empty-state run re-adopts. A CloudflareStateStore (drift detection + destroy tracking shared between local and CI) is a future enhancement, not required for create/adopt.

8. Roadmap / open

• Production substrate (-prod resources) mirrored from staging via the same program + stage.
• Feed Lab needs (docs/console/FEED-LAB.md): a FeedPolicy KV store (draft→prod labels), raw-embedding persistence (R2 parquet or D1 blob, FL-4) alongside Vectorize, and per-user Durable Objects for taste/session state (region-pinned PII).
• Vectorize dim 1024 is provisional (pending the multimodal embedding-model choice).
• Move brand Pages-project creation from the Wrangler bootstrap into Alchemy; add PR-preview deploys.