4 Commits

Include renames

Author	SHA1	Message	Date
ConnorYohandGitHub	28b81828b5	PAYG: PricingPolicyService + admin REST + 30s read cache (#6469) ... ## What this is PR-I1 service half from `notes/PAYG_DESIGN.md`. Built on top of the data model from #6460 — answers "what pricing policy applies to this team right now?" with a fast cache and an admin write surface. ## Scope | Piece | Where | |---|---| | `PricingPolicyService` — `getEffectivePolicy(teamId)` with 30s Caffeine cache + admin write paths | `app/saas/.../payg/policy/PricingPolicyService.java` | | `PolicyChangedEvent` — published after admin writes for in-process cache invalidation | `app/saas/.../payg/policy/PolicyChangedEvent.java` | | Admin REST — list / get / create / set-default / set team override / get effective | `app/saas/.../payg/policy/admin/PricingPolicyAdminController.java` + DTOs | | `PricingPolicyRepository.clearDefaultFlag()` — atomic clear for set-default | repository update | | `SaasJpaConfigScanTest` — drift guard against the JPA scan paths going stale (carried over from the #6460 review concern) | new test | | V12 default-policy seed (`v1-initial`, 25 pages/unit, 5 MiB/unit, per-`JobSource` step limits) | `V12__seed_default_payg_policy.sql` | ## Lookup precedence 1. `PaygTeamExtensions.pricingPolicyId` set → return that policy 2. Else return the `pricing_policy` row with `is_default = TRUE` 3. Override row points at a deleted policy → log warn, fall back to default (safety net for racing deletes) 4. No default → `IllegalStateException` (V12 seed guarantees one exists) ## Cache behaviour - 30s `expireAfterWrite` Caffeine, max 10k entries, keyed by `teamId`. - **Single correctness model: the TTL.** Cross-instance propagation is at-most-30-seconds. The writer instance sees its own change immediately via the `PolicyChangedEvent` after-commit publish. Other instances pick it up on the next TTL expiry. - Admin reads use `getEffectivePolicyUncached` so admins always see their own write straight back. **Why no LISTEN/NOTIFY runner.** An earlier cut of this PR included a Postgres `LISTEN policy_changed` runner so cross-instance propagation was instant. Dropped — admin policy changes are events-per-week and the 30s TTL is already the correctness floor; the listener was ~250 lines of nontrivial code (raw JDBC outside HikariCP, daemon thread, reconnect loop, lock-protected connection lifecycle) for a use case that isn't on the hot path. Trade-off is documented in `notes/PAYG_DESIGN.md` §9 with three concrete triggers that would justify reintroducing it (aggressive cap enforcement, Redis landing for other reasons, real-time admin UI). ## Writes — transactional, fire `PolicyChangedEvent` after commit - `create(draft)` — rejects pre-set `policy_id` or `is_default=true` (promotion must go through `setDefault` so the partial unique idx is freed first). - `setDefault(id)` — atomically clears the existing default via `clearDefaultFlag()` then flips the new row. Idempotent: silent no-op if the row is already default. - `setTeamOverride(teamId, policyId | null)` — validates the policy exists before save; `null` clears the override. `publishOnCommit` uses `TransactionSynchronizationManager.afterCommit` so listeners never see pre-commit state. Outside a transaction (test paths) falls through to immediate publish. ## Admin REST surface — `/api/v1/admin/payg/...` All endpoints `@PreAuthorize("hasRole('ADMIN')")`: - `GET /policies` — list all - `GET /policies/{id}` — read one - `POST /policies` — create new (non-default) - `POST /policies/{id}/set-default` — atomic promote - `PUT /teams/{teamId}/policy-override` — set or clear per-team override - `GET /teams/{teamId}/effective-policy` — cache-bypassing live read Validation errors → 400, unknown rows → 404. ## Counterpart Supabase PR [`Stirling-PDF-SaaS#298`](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/298) — seeds the same V1 default policy on the Supabase side via `20260528000002_payg_seed_default_policy.sql`. ## Tests - 17 × `PricingPolicyServiceTest` — lookup precedence, cache hit/miss, invalidation on event, mutation paths publishing event, error cases. - 14 × `PricingPolicyAdminControllerTest` — every endpoint's happy path + error mapping, DTO defensive-copy invariant. - 2 × `SaasJpaConfigScanTest` — reflection-based guard that `payg.repository` is in `@EnableJpaRepositories` and `payg` is in `@EntityScan`. Without this, new sub-packages can silently fail to wire at runtime — same class of bug that the #6460 review caught. Full `:saas:test` BUILD SUCCESSFUL. ## Design doc `notes/PAYG_DESIGN.md` §7.4 PR-I1 — completes the service half (the schema half landed in #6460). §9 carries the 30s-TTL trade-off note.	2026-05-29 16:27:01 +00:00
ConnorYohandGitHub	83ea07ed6a	saas: DocumentClassifier + PAYG data model (#6460) ... # Description of Changes Two layers — the `DocumentClassifier` utility plus the full data model for the new billing engine. Nothing wires the entities into application behaviour yet; services and controllers land in follow-up PRs. **Companion PR:** [Stirling-PDF-SaaS#296](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/296) — Supabase migration for the v3 dev branch, schema-equivalent to the Flyway migration in this PR. ## 1. DocumentClassifier (under `payg.docs`) `DocumentClassifier` computes the doc-unit cost of an uploaded file (or multi-file input) under a `PricingPolicy`. PDFs read page count via `stirling.software.jpdfium.PdfDocument`; non-PDFs are bytes-only. Formula: `max(ceil(pages / docPagesPerUnit), ceil(bytes / docBytesPerUnit))` clamped to `[1, fileUnitCap]`. Multi-file is the sum of raw per-file units capped at `fileUnitCap × file_count`. Two floors, by design: the classifier returns `docUnits` with an absolute `1` floor for non-empty input; the policy-level `minChargeUnits` is intentionally applied later, at process-open time in `JobChargeService`, per design § 3.4 (`unitsForProcess = max(policy.min_charge_units, docUnits)`). Documented in the interface + impl javadoc. Upload bytes are materialised through `TempFileManager.createManagedTempFile` so jpdfium gets a `Path`; the temp file auto-deletes on close. Twelve tests, all in-memory fixtures generated with PDFBox at test time — no committed binary blobs. ## 2. PAYG data model (under `payg.*`) JPA entities, repositories, and a Flyway migration covering the full schema in §6 of the design. **Enums** (`payg.model`): `JobSource`, `ProcessType`, `JobStatus`, `JobStepStatus`, `ArtifactKind`, `LedgerEntryType`, `LedgerBucket`, `ReferenceType`, `EntitlementState`, `FeatureSet`, `FeatureGate`, `WalletEngine`, `CapPeriod`, `AutoGroupStrategy`. **Entities + repositories:** | Entity | Table | Notes | |---|---|---| | `PricingPolicy` | `pricing_policy` | Promoted from a record. `stepLimits` is `Map<JobSource, Integer>` persisted via normalised child table `pricing_policy_step_limit`. `stripePriceIds` is `Set<String>` persisted via `pricing_policy_stripe_price` — currency comes from `stripe.prices` via Sync Engine, not stored locally. | | `ProcessingJob` | `processing_job` | UUID PK. Tracks lineage window via `step_count` and `last_step_at`. | | `ProcessingJobStep` | `processing_job_step` | Per-tool-call audit. | | `JobArtifactHash` | `job_artifact_hash` | Composite key `(job_id, content_hash, kind)`. `content_hash VARCHAR(128)` so multiple signature schemes coexist as `"type:value"` storage keys. Lineage detector queries this. | | `WalletLedgerEntry` | `wallet_ledger` | Append-only, signed `amount_units`. Two unique indexes kill double-posting. | | `WalletPolicy` | `wallet_policy` | Per-team engine + cap + degradation rules + lineage strategy. No `@Version` — admin-only writes (documented in javadoc). | | `WalletEntitlementSnapshot` | `wallet_entitlement_snapshot` | Composite key `(team_id, user_id)`; `user_id = 0` is the team-wide sentinel. No `@Version` — full-row recompute via `EntitlementService.recompute` (documented in javadoc). | | `PaygShadowCharge` | `payg_shadow_charge` | Per-job diff while in `PAYG_SHADOW` engine mode. | | `PaygTeamExtensions` | `payg_team_extensions` | Sidecar 1:1 with `teams` carrying `pricing_policy_id` (per-team override) + `stripe_customer_id`. Sidecar pattern (mirrors `saas_team_extensions`) so OSS Hibernate ddl-auto never sees PAYG columns on `teams`. | **Column adds:** - `team_memberships.cap_units` (optional per-member sub-cap) **Width split (intentional, documented in V11):** per-row deltas (`wallet_ledger.amount_units`, `processing_job.charged_units`) are `INTEGER` because no single charge realistically approaches 2B units. Cap and period-rollup columns (`team_memberships.cap_units`, `wallet_policy.cap_units`, `wallet_entitlement_snapshot.period_spend_units / period_cap_units`) are `BIGINT` because they accumulate across a billing period and admins may legitimately set headroom-cap values into the millions. **JPA wiring:** `SaasJpaConfig` was updated to include `stirling.software.saas.payg.repository` in `@EnableJpaRepositories.basePackages` and `stirling.software.saas.payg` in `@EntityScan` (covers `payg.policy` / `payg.job` / `payg.wallet` / `payg.entitlement` / `payg.shadow` recursively). New `SaasJpaConfigScanTest` reads the annotations reflectively and asserts every expected package is wired — catches the next time someone adds a new sub-package without updating the scan paths. **Migration:** `V11__saas_payg_model.sql` (purely additive). Schema-equivalent to the Supabase migration in the companion PR — including the `VARCHAR(128) content_hash` width that's needed for the multi-signature-scheme storage encoding the lineage layer uses. ## 3. Smoke tests `PaygEntitiesSmokeTest` exercises each entity via the no-arg ctor JPA requires, plus getter/setter round-trips and composite-key equality — catches Lombok/annotation regressions without needing a database. Real-DB integration coverage lands alongside the services that consume each entity. ## Why this is safe to land now - All schema changes are additive — no existing rows modified, no columns dropped. - The entities are not yet referenced from any production code path; they exist for the next PRs to build on. - The v3 Supabase dev branch picks up the schema via the companion PR; the main repo's Flyway migration applies the same shape when an instance boots against a freshly-migrated v3 database. ## Open decisions made - **Step-limits keyed by `JobSource`** rather than by `ProcessType`. Captures the "self-hosted gets a different knob" framing in earlier feedback. Trivially overridable per pricing policy version. - **Step limits + Stripe price IDs normalised into child tables** rather than JSONB on `pricing_policy` (per Connor's review on #296). Typed columns, queryable directly, no JSON parsing. - **Currency dropped from `pricing_policy_stripe_price`** — it lives on `stripe.prices.currency` and is resolved via Sync Engine. App is currency-blind. ## Rollback Straight `git revert` on this PR. The Supabase migration in #296 is additive and can be left in place safely — the running app ignores tables it doesn't reference. --- ## Checklist - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings - [x] I have run `task check` (via `./gradlew :saas:test` with `ENABLE_SAAS=true`) — passes	2026-05-29 12:03:01 +00:00
ConnorYohandGitHub	a0e0e88f07	saas: harden CreditService Stripe ordering + lint @AutoJobPostMapping weights (#6458) ... # Description of Changes Two narrowly-scoped hardening changes to the credits engine. ## 1. CreditService — move Stripe meter call to `afterCommit` The Stripe metered-usage call sits inside the surrounding `@Transactional`, holding the `user_credits` row lock for the duration of an HTTP round-trip to Supabase. Under load this starves concurrent debits; a transient Stripe blip rolls back a (correct) free-credit consumption and forces the caller to retry. The Stripe call now runs in a `TransactionSynchronization.afterCommit` hook — DB commits first, Stripe fires immediately after. If Stripe fails after commit, we log + increment a new `credits.stripe_report.failures` counter; the idempotency key is stable, so a manual replay recovers without double-charging. Applied to both `consumeCreditBySupabaseId` and `consumeCreditWithWaterfall`. **Dead-code removed:** - Unreachable UUID fallback for MDC `requestId` — `CorrelationIdFilter` already guarantees the key on every request. - The `"Unable to report usage to Stripe"` `RuntimeException` and its catch block — the afterCommit refactor eliminates the throw path. - `StripeRollbackOnFailureTest` — pinned the rollback-on-Stripe-fail behaviour this refactor replaces. ## 2. `@AutoJobPostMapping` — build-time lint for `resourceWeight` `UnifiedCreditInterceptor` multiplies `resourceWeight` into the per-call charge. An endpoint that falls through to the annotation default produces a charge derived from a value nobody chose. - Annotation default flipped from `1` to `Integer.MIN_VALUE` (sentinel). Both runtime readers (`UnifiedCreditInterceptor`, `AutoJobAspect`) already clamp into `[1, 100]` so behaviour is unchanged. - New `AutoJobPostMappingWeightTest` scans the classpath and fails the build if any method leaves the sentinel. - Initial run caught 11 endpoints relying on the default. Explicit weights now declared, chosen by comparing to peer endpoints: - `EditTextController` — LARGE - `EmailController#sendEmailWithAttachment` — SMALL - `ConvertPDFToMarkdown` — MEDIUM - `AttachmentController` (extract/list/rename/delete) — SMALL × 4 - `ConvertImgPDFController` (cbr/cbz ↔ pdf) — MEDIUM × 2, LARGE × 2 ## Tests - `StripeUsageIdempotencyKeyTest` — pins the `(supabaseId, overage, requestId)` idempotency key shape so Stripe always dedupes a retry. - `StripeAfterCommitOrderingTest` — pins that `afterCommit` fires after commit and NOT on rollback. - `AutoJobPostMappingWeightTest` — the lint itself, plus a self-check that the classpath scan finds at least 10 `@AutoJobPostMapping` methods (guards against the lint passing vacuously). Build verified: `ENABLE_SAAS=true ./gradlew :stirling-pdf:test :saas:test`. --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) — no translation changes - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) — internal-billing change, no public docs impact - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) — N/A ### Translations (if applicable) - [ ] Not applicable ### UI Changes (if applicable) - [ ] Not applicable ### Testing (if applicable) - [x] I have run `task check` (via `./gradlew :stirling-pdf:test :saas:test` with `ENABLE_SAAS=true`) — passes - [x] I have tested my changes locally	2026-05-28 14:57:59 +00:00
3 people	9d081d1792	SaaS Consolidation (#6384) ... Co-authored-by: ConnorYoh <[email protected]>	2026-05-21 16:05:35 +01:00