mirror of
https://github.com/arsvendg/Stirling-PDF.git
synced 2026-07-14 10:34:06 +02:00
ConnorYohandGitHub
3807cdfbc6
> 📌 **Stacked on [#6464](https://github.com/Stirling-Tools/Stirling-PDF/pull/6464)** (lineage primitives, still in review). #6469 has merged so its commits are no longer in this PR's diff. Once #6464 merges, a final rebase collapses the lineage-primitives commits out of this diff too — leaving only the B-1 work. ## What this is Process tracking + shadow charging engine. Bundles PR-I7 service half with the non-filter piece of PR-I7a so the pieces ship together — none of them is useful in isolation. **Review focus:** the new files in: - \`app/saas/src/main/java/stirling/software/saas/payg/job/\` (\`JobService\`, \`JobContext\`, \`JoinOrOpenResult\`, \`StaleJobCloser\`) - \`app/saas/src/main/java/stirling/software/saas/payg/charge/\` (\`JobChargeService\`, \`ChargeContext\`, \`ChargeOutcome\`, \`JobInput\`) - \`app/saas/src/main/java/stirling/software/saas/payg/lineage/LineagePruneScheduler.java\` - their tests The 8 files inherited from #6464 (lineage primitives) are unchanged from there — they ride along in this diff until #6464 lands. The remaining work for shadow-in-staging is the ingress/egress filter that wires controllers into this engine — that's PR B-2. ## Scope ### \`JobService\` — persistence + lineage policy - **\`joinOrOpen\`** — the multi-input "any-match-joins, newest wins" rule. Hash every input via the lineage detector; if any matches an open process in the workflow window, attach to the one with the freshest \`lastStepAt\`. Step-limit overflow on the matched job spawns a fresh process; the new job's input signatures are still recorded so \`mostRecentMatchWins\` routes future calls forward. - **\`recordOutput\`** — post-tool-success path. Records OUTPUT signatures so the next call that takes this file as input lineage-matches into the same process. - **\`appendStep\`** — audit-trail step row written after a tool completes. - **\`close\`** — idempotent; safe to call from multiple paths (explicit, FE on-unload, scheduler). Returns the same row on re-close, no state mutation. - **\`findStale\` / \`closeStale\`** — workflow-window-based stale closure used by the scheduler. ### \`JobChargeService\` — the orchestrator (shadow variant) \`openProcess\` resolves the effective policy via \`PricingPolicyService\` (now in main via #6469), derives the step-limit for the current \`JobSource\` (with a defensive fallback if the policy is missing an entry), delegates to \`JobService.joinOrOpen\`, and on OPENED runs the \`DocumentClassifier\` + writes a \`payg_shadow_charge\` row. Applies the policy-level \`minChargeUnits\` floor per design § 3.4. Shadow variant only — never debits the ledger, never posts a Stripe meter event. The real-charging follow-up reuses the same orchestration and swaps the side-effect. \`legacyCreditsCharged\` on the shadow row stays \`0\` until the legacy \`CreditService\` is wired in (PR B-2), where the comparison becomes meaningful. ### Schedulers (both plain \`@Scheduled\`) - **\`StaleJobCloser\`** — fixed-rate 60 s. Closes \`OPEN\` jobs idle past the workflow window. API users never have to call close explicitly — this is the safety net. - **\`LineagePruneScheduler\`** — hourly cron, retention 1 h. Deletes \`job_artifact_hash\` rows older than the retention window. - **No \`@SchedulerLock\` / no \`shedlock\` table** — consistent with the 5 existing unguarded \`@Scheduled\` tasks in \`:saas\` (\`CreditResetScheduler\` and friends, none of which are guarded today). Cluster-correctness across all 7 saas schedulers is tracked in design § 9 as a separate focused cleanup. Underlying operations are idempotent — duplicate firings on multi-pod would be wasted DB load, not data corruption. ### Records (call-shape glue for PR B-2's filter) - \`JobContext\` / \`JoinOrOpenResult\` — input/output for \`JobService\`. - \`ChargeContext\` / \`ChargeOutcome\` — input/output for \`JobChargeService\`. - \`JobInput\` — paired \`(MultipartFile, materialised Path)\` so the upcoming ingress filter can pass both views without re-materialising. ## Tests **26 new, all green.** - 14 × \`JobServiceTest\` — no-match → opens new, single-match → joins existing, multi-input any-match-joins, multi-match newest-wins (older job never even looked up), step-limit hit spawns fresh job (and original \`stepCount\` is NOT mutated), empty inputs reject, stale-signature handling, recordOutput delegation, close idempotency, closeStale, appendStep persistence. - 7 × \`JobChargeServiceTest\` — JOINED skips classifier + shadow write entirely, OPENED writes shadow row + classifies (single + multi file paths), \`minChargeUnits\` floor applied, step-limit resolved per-\`JobSource\` from policy, missing source entry falls back to conservative default of 10. - 2 × \`StaleJobCloserTest\`, 3 × \`LineagePruneSchedulerTest\` — scheduler-wiring smoke + constructor-validation tests. \`ENABLE_SAAS=true ./gradlew :saas:test\` — BUILD SUCCESSFUL. ## What's not in this PR (lands in PR B-2) - **Tool ingress/egress servlet filter.** The highest-risk piece — materialises the request body into a \`JobInput\`, calls \`JobChargeService.openProcess\` from every \`@AutoJobPostMapping\`, records OUTPUT after success. Edge cases to validate: multipart parts, async controllers, streaming responses, errored 5xx paths, very large files. Design decisions for the filter are being worked through in \`notes/PAYG_FILTER_DESIGN.md\` before any code is written. - **Wire shadow path into legacy \`CreditService\`.** Every legacy debit writes a comparison row carrying both the PAYG would-be units and the legacy actual credits, populating \`diffPct\`. ## Design doc \`notes/PAYG_DESIGN.md\` — PR-I7 + PR-I7a status updated to reflect this bundle. § 9 carries the cluster-correctness deferral note alongside the existing LISTEN/NOTIFY trade-off note. § 7.5.1 readiness summary shows the path now needs just **1 more PR** (the filter half + CreditService wire-in, both bundled into PR B-2).
Stirling PDF - The Open-Source PDF Platform
Stirling PDF is a powerful, open-source PDF editing platform. Run it as a personal desktop app, in the browser, or deploy it on your own servers with a private API. Edit, sign, redact, convert, and automate PDFs without sending documents to external services.
Key Capabilities
- Everywhere you work - Desktop client, browser UI, and self-hosted server with a private API.
- 50+ PDF tools - Edit, merge, split, sign, redact, convert, OCR, compress, and more.
- Automation & workflows - No-code pipelines direct in UI with APIs to process millions of PDFs.
- Enterprise‑grade - SSO, auditing, and flexible on‑prem deployments.
- Developer platform - REST APIs available for nearly all tools to integrate into your existing systems.
- Global UI - Interface available in 40+ languages.
For a full feature list, see the docs: https://docs.stirlingpdf.com
Quick Start
docker run -p 8080:8080 docker.stirlingpdf.com/stirlingtools/stirling-pdf
Then open: http://localhost:8080
For full installation options (including desktop and Kubernetes), see our Documentation Guide.
Resources
Support
- Community Discord
- Bug Reports: Github issues
Contributing
We welcome contributions! Please see CONTRIBUTING.md for guidelines.
This project uses Task as a unified command runner for all build, dev, and test commands. Run task install to get started, or see the Developer Guide for full details.
For adding translations, see the Translation Guide.
License
Stirling PDF is open-core. See LICENSE for details.