PAYG: pay-as-you-go billing — metered automation/AI/API + one-time free grant (#6589)

## Summary

Pay-as-you-go (PAYG) billing for Stirling-PDF SaaS. Manual PDF editing
stays free forever; only **automation, AI, and API** usage is metered.
Every team gets a **one-time lifetime free grant** (default 500 PDFs)
before any billing; past that, a team adds a card and pays per metered
document, with a self-set monthly spending cap.

This branch combines and supersedes the in-flight BE (#6574) and FE
(#6579) work plus the SaaS edge functions (Stirling-PDF-SaaS PR, now on
`v3`), hardened into a single reviewable feature after a pre-merge
dead-code/security review.

## Billing model

- **Always free:** manual / JWT web-tool usage is `BYPASSED` — never
metered, no matter where it's triggered.
- **Billable categories:** `AUTOMATION`, `AI`, `API`.
- **One-time lifetime free grant** (`pricing_policy.free_tier_units`,
default 500): never resets, survives subscribing. It gates unsubscribed
teams (billable API calls hard-stop with a 402 once exhausted) and
decides the free-vs-paid split of every job.
- **Subscribed:** paid documents (beyond the grant) are metered to a
Stripe Billing Meter; an optional monthly spending cap degrades billable
categories when reached.
- **Dedup:** the same file pushed through several steps within a
workflow window counts **once** (lineage join), so API/AI chaining on
one file isn't double-charged.

## What's included

**Database** — Flyway migrations `V11`→`V21` with matching Supabase
twins: pricing policy + per-team sidecar (`payg_team_extensions`:
subscription id, Stripe customer, free-grant counter), append-only
`wallet_ledger`, shadow charges, subscription-state RPCs (`V14`), audit
logs (`V15`), billing category (`V16`), one-time lifetime free grant
(`V19`), launch-grant seed (`V20`), drop of the unused
`wallet_category_summary` view (`V21`).

**Charge pipeline** — `PaygChargeInterceptor` (open/join a process,
split the free grant, write the ledger DEBIT), `JobChargeService`
(consume the grant under a row lock, restore it on a first-step refund,
meter only the paid portion on completion), `StaleJobCloser` fallback
(idempotent close → meter).

**Entitlement** — `EntitlementService` (per-team cached snapshot:
grant-gated for free teams, monthly-cap-gated for subscribed) +
`EntitlementGuard` (401 `SIGNUP_REQUIRED` / 402 `FEATURE_DEGRADED` /
`PAYG_LIMIT_REACHED`).

**Metering** — `PaygMeterReportingService` writes a durable
`payg_meter_event_log` row around every POST to the `meter-payg-units`
edge fn (pending → posted/failed); `PaygMeterReconcileScheduler` retries
unposted events under the same idempotency key inside Stripe's 24h dedup
window.

**Billing facts** — `TeamBillingService` reads the synced `stripe.*`
mirror (subscription window, per-document rate; the unsubscribed-team
estimate resolves the rate by Price `lookup_key = plan:processor`).

**Wallet API** — `PaygWalletController`: `GET /api/v1/payg/wallet`,
`PATCH /api/v1/payg/cap`.

**Frontend** — PAYG Plan page (two-card free layout + subscribed views),
`useWallet`, upgrade modal with lazy-loaded Stripe Embedded Checkout and
a shared `SpendCapControl`, customer-portal link, 402/401 interceptor
toast, en-GB i18n. (Per-member usage shows each teammate's spend; the
activity feed is behind a flag until polished.)

**SaaS edge functions** (`Stirling-PDF-SaaS` `v3`) —
`create-checkout-session`, `create-payg-team-subscription`,
`create-customer-portal-session`, `meter-payg-units`,
`payg-subscription-webhook`, `stripe-sync`, plus the stripe-sync
`migrate` + scoped-`backfill` scripts. All price lookup is DB-driven (no
`STRIPE_PAYG_PRICE_ID_*` env vars).

## Release prerequisites (prod)

1. Apply Flyway migrations (`V11`→`V21`) and the Supabase migration
twins.
2. Stripe Sync Engine: run `stripe-sync:migrate`, then a **scoped**
backfill — `product`, `price`, `customer`, `subscription` only (not
`all`, which rate-limits).
3. Register 2 PAYG webhook endpoints (each its own signing secret):
`stripe-sync` (product/price/customer/subscription `.*`) and
`payg-subscription-webhook` (`customer.subscription.created`/`.deleted`
drive state; `.updated` + `invoice.*` observed). Keep the legacy
`stripe-webhook` only if credits/self-hosted flows still run.
4. Stripe Billing Meter: `event_name = payg_doc_units`, value key
`processed_documents`.
5. Env: `PAYG_METER_ENDPOINT` + `SUPABASE_EDGE_FUNCTION_SECRET`
(backend); the webhook signing secrets (edge fns). The default pricing
policy must point at the PAYG Stripe Price(s); `V20` seeds
`free_tier_units = 500`.

## Testing

- `:saas:test` green, `:saas:spotlessCheck` clean, edge-fn Deno tests
green, FE saas typecheck clean (the remaining errors are pre-existing
`proprietary/*` + `prototypes/*`, untouched here). Cucumber shadow-mode
suite + CI workflow included.

## Pre-merge review

An independent dead-code/security pass came back **clean on security**
(team-derived authz / no IDOR, leader-only cap mutation, no
billing-category downgrade, dev/mock hooks gated to
`import.meta.env.DEV` + `/dev/`, no secrets/injection, fail-open
metering by design). The dead/unwired code it flagged has been removed
in this branch (unenforced sub-cap control, an unused JDBC DAO + its
view, dead methods).

## Follow-ups (tracked, not blocking)

- **Enforce per-member sub-caps** — the control was removed because it
read for display but never gated a request; the per-member usage display
and `cap_units` column are retained for when enforcement is wired.
- **API/AI chaining billing model + `ProcessType` enum** — confirm
same-file dedup covers API chaining; define per-tool AI charging; decide
whether the unused enum values stay.
- **Activity feed** — hidden behind a flag until the meter-event surface
is polished.

---------

Co-authored-by: Reece <[email protected]>
This commit is contained in:
ConnorYoh
2026-06-11 15:56:01 +01:00
committed by GitHub
co-authored by Reece
parent 11ab762f57
commit cf513c255b
89 changed files with 11389 additions and 1034 deletions
@@ -3090,6 +3090,18 @@ integration = "Integration Configuration"
security = "Security Configuration"
system = "System Configuration"
[config.payg]
label = "Billing & usage"
section = "Pay-as-you-go"
[connectionMode.status]
localOffline = "Offline mode running"
localOnline = "Offline mode running"
saas = "Connected to Stirling Cloud"
selfhostedChecking = "Connected to self-hosted server (checking...)"
selfhostedOffline = "Self-hosted server unreachable"
selfhostedOnline = "Connected to self-hosted server"
[convert]
tags = "transform,change,convert,PDF to Word,PDF to Excel,PDF to image,Word to PDF,Excel to PDF,PowerPoint to PDF,HTML to PDF,export,import,file conversion,format change,save as"
autoRotate = "Auto Rotate"
@@ -5132,6 +5144,239 @@ title = "Page Ranges"
bullet1 = "<strong>all</strong> → selects all pages"
title = "Special Keywords"
[pageSelection.tooltip.syntax]
text = "Use numbers, ranges, keywords, and progressions (n starts at 0). Parentheses are supported."
title = "Syntax Basics"
[pageSelection.tooltip.syntax.bullets]
keywords = "Keywords: odd, even"
numbers = "Numbers/ranges: 5, 10-20"
progressions = "Progressions: 3n, 4n+1"
[pageSelection.tooltip.tips]
bullet1 = "Page numbers start from 1 (not 0)"
bullet2 = "Spaces are automatically removed"
bullet3 = "Invalid expressions are ignored"
text = "Keep these guidelines in mind:"
title = "Tips"
[payg]
subtitle = "Pay-as-you-go — you only pay for what you process. Billing period {{start}} {{end}}."
[payg.activity]
subtitle = "Only AI and automation draw from your budget. Everyday tools are free and aren't listed here."
title = "Recent billable activity"
units = "units"
viewAll = "View all"
[payg.cap]
amount = "Cap amount"
currency = "Currency"
custom = "Custom"
degradeAt = "Limit spend at"
degradeAtDesc = "Set below 100% to leave yourself headroom."
docsEstimate = "≈ {{docs}} processed PDFs / month"
docsRate = "at {{rate}} / PDF"
noCapDesc = "Usage is billed without an upper limit. You can re-enable a cap at any time."
noCapLabel = "No cap"
perMonth = "/ month"
preview = "{{money}} ≈ {{units}} PDFs per month"
previewNote = "At current pricing. Final translation happens server-side on save."
previewShort = "≈ {{units}} PDFs per month at current pricing."
save = "Update cap"
subtitle = "Set your maximum spend. We convert this to PDFs using the current price tier."
title = "Monthly spending cap"
warnAt = "Warn me at"
warnAtDesc = "Notify when usage crosses this threshold."
[payg.checkout]
connecting = "Connecting to Stripe…"
errorTitle = "Stripe error"
[payg.checkout.error]
startFailed = "Couldn't start checkout session"
[payg.checkout.mock]
backend = "Backend is in mock mode — no real Stripe session was created."
continue = "Continue with mock subscription"
noKey = "VITE_STRIPE_PUBLISHABLE_KEY is unset. Real iframe mounts here once configured."
title = "Stripe Embedded Checkout (mock mode)"
[payg.confirm]
body = "Your team can now run automation, AI, and API operations beyond your 500 free PDFs."
capValue = "{{symbol}}{{amount}} / month"
noCap = "No cap"
note = "You can change your cap, cancel, or open the Stripe customer portal any time from this page."
summaryLabel = "Monthly ceiling"
title = "Welcome to the Processor plan"
[payg.error]
body = "We couldn't reach the billing service. Refresh the page to try again."
title = "Couldn't load your plan"
[payg.exhausted]
body = "You've used all 500 of your free PDFs. Upgrade to Processor to keep going."
cta = "Go to billing"
title = "You've used your free PDFs"
[payg.free.cta]
benefit1Body = "chain tools, schedule runs, batch process"
benefit1Title = "Automation pipelines"
benefit2Body = "summarise, classify, redact, AI-OCR"
benefit2Title = "AI tools"
benefit3Body = "call any Stirling endpoint programmatically"
benefit3Title = "API access"
button = "Turn on Processor →"
reassurance = "No minimum · Set a $0 cap to test · Cancel anytime"
subtitle = "Keep going past your 500 free PDFs with automation, AI, and the API. Set a monthly ceiling — you stay in control."
title = "Turn on the Processor plan"
[payg.free.editor]
eyebrow = "Editor plan · Always free"
[payg.free.explainer]
alwaysFreeBody = "Manual tools — viewing, editing, merging, splitting, signing, watermarks, compression, conversion, manual OCR. Use them as much as you want, no matter where you trigger them from."
alwaysFreeForYouBody = "Manual tools — viewing, editing, merging, splitting, signing, watermarks, compression, conversion, manual OCR. Use them as much as you want, never counted."
alwaysFreeForYouLabel = "Always free for you"
alwaysFreeLabel = "Always free"
countsBody = "Automation pipelines (chained tools, scheduled runs), AI tools (summaries, classification, AI-OCR), and API calls (programmatic access). Past your free PDFs you'll need Processor."
countsLabel = "Counts toward your 500 free"
sharedBody = "Automation pipelines, AI tools, and API calls share a single one-time allowance of 500 free PDFs across the whole team. Once it's used up, ask your owner to enable Processor."
sharedLabel = "Shared with the team"
[payg.free.header]
subtitle = "Editor plan — manual tools are always free. Pay only for automation, AI & API. Billing period {{period}}."
[payg.free.hero]
capSuffix = "/ {{limit}} free PDFs"
eyebrow = "Your free PDFs"
metaCategories = "Automation · AI · API requests"
neverResets = "One-time — never resets"
[payg.free.member]
body = "Your team owner can enable the Processor plan and set a monthly ceiling. Until then, manual tools are free for you to use as much as you like — automation, AI, and API access share the team's one-time allowance of 500 free PDFs."
ownerOnly = "Only your team owner can turn on Processor. Manual tools stay free for you to use as much as you like."
title = "Want to process more than your 500 free PDFs?"
[payg.free.proc]
eyebrow = "Processor plan · metered"
[payg.free.state]
approachingLimit = "Approaching limit"
limitReached = "Limit reached"
plentyLeft = "Plenty left"
[payg.gates]
ai = "AI tools (AI Create, suggestions, AI-OCR)"
automation = "Automations & pipelines"
client = "Browser-only tools (viewer, page editor, file management)"
offsite = "Server tools (compress, OCR, convert, watermark…)"
pauses = "pauses at cap"
subtitle = "Your everyday tools keep working. Only AI and automation pause until the cap resets or is raised."
title = "What happens when the cap is reached"
[payg.member]
askLeader = "Only your team owner can change the cap."
[payg.members]
docs = "PDFs"
subtitle = "Billable PDFs each teammate has processed this period."
title = "Team member usage"
[payg.role]
leader = "Team owner"
member = "Member"
[payg.signupRequired]
body = "Stirling PDF gives every signed-up account 500 free operations — enough to keep most workflows humming without paying a cent. You're currently using Stirling as a guest, which doesn't include billable tools like AI, automations, or hosted processing."
cancel = "Not now"
cta = "Sign up free"
subtext = "Creating an account is free and takes a few seconds. No credit card required."
title = "Sign up to use {{category}}"
[payg.signupRequired.category]
ai = "AI features"
api = "this tool"
automation = "automations"
default = "this feature"
[payg.state]
degraded = "Cap reached"
full = "Healthy"
warned = "Approaching cap"
[payg.stripe]
open = "Open billing portal"
subtitle = "Receipts, invoices, payment method, billing currency."
title = "Manage billing in Stripe"
[payg.stripe.toast.unavailable]
body = "Billing portal isn't available right now. Try again in a moment."
title = "Billing portal unavailable"
[payg.upgrade]
closeAria = "Close"
[payg.upgrade.button]
back = "← Back"
cancel = "Cancel"
continue = "Continue →"
finish = "Finish"
[payg.upgrade.cap]
customPlaceholder = "Or enter your own amount ($0 keeps it free)"
help = "We'll never charge above this. Set $0 if you want to keep everything free while testing."
noCapHint = "You can still cancel anytime from the customer portal."
noCapLabel = "No cap — I'll manage spend manually"
perMonthSuffix = "/ month"
presetsAria = "Monthly cap preset"
title = "Set your monthly spend ceiling"
usdNote = "Estimated in USD. You can adjust your cap any time after subscribing — in your own currency."
[payg.upgrade.checkout]
capLabel = "Monthly ceiling:"
capValue = "{{symbol}}{{amount}} / month"
edit = "Edit"
help = "Stripe handles your card details. Stirling never sees them."
loading = "Loading checkout…"
noCap = "No cap"
title = "Add your payment method"
[payg.upgrade.footer]
capHint = "You can change your cap any time later."
checkoutHint = "Card details handled by Stripe — never touched by Stirling."
confirmHint = "Your wallet will refresh automatically in a moment."
[payg.upgrade.help]
aiBody = "summarise, classify, redact, AI-OCR"
aiTitle = "AI tools"
apiBody = "programmatic access to any Stirling endpoint"
apiTitle = "API calls"
automationBody = "chained tools or scheduled runs that don't need clicks"
automationTitle = "Automation pipelines"
footnote = "Manual tools — viewing, editing, merging, splitting, watermarking, compressing, manual OCR — are always free, even past 500. The distinction is the type of work, not where you click."
title = "What we count toward billing"
[payg.upgrade.promise]
body = "You only pay for automation pipelines, AI tools, and API calls — the work that goes beyond a single click. Edit, merge, split, sign, compress as much as you want, no charge."
highlight = "Manual tools stay free, always."
[payg.upgrade.steps]
cap = "Set monthly ceiling"
payment = "Add payment method"
[payg.upgrade.title]
confirm = "You're subscribed"
default = "Upgrade to Processor plan"
[payg.usage]
credit = "{{amount}} account credit"
docs = "documents"
resetsIn = "Resets in {{days}} days"
resetsTomorrow = "Resets tomorrow"
spent = "{{spend}} of {{cap}} used"
thisPeriod = "This billing period"
[payment]
autoClose = "This window will close automatically..."
canCloseWindow = "You can now close this window."