mirror of
https://github.com/arsvendg/Stirling-PDF.git
synced 2026-07-14 10:34:06 +02:00
ConnorYohandGitHub
22dacbed01
## What this PR does Wires the **B-1 shadow charging engine** into real HTTP request flow. After this lands, flipping an internal team to ``PAYG_SHADOW`` via SQL begins populating ``payg_shadow_charge`` automatically — with **zero impact** on the legacy credit deduction path. **This is the load-bearing PR for shadow mode.** Without it, B-1's engine sits idle — nothing in the codebase calls ``JobChargeService.openProcess()`` from a real HTTP request. Stacks on top of #6477 (PR B-1). ## Components | Class | Role | |---|---| | ``PaygResponseBodyWrapperFilter`` | Servlet filter, installs tee'ing response wrapper. Defers wrapper close to ``AsyncListener`` for ``DeferredResult`` / ``CompletableFuture`` controllers so the lifetime spans the async window. | | ``PaygResponseBodyWrapper`` | ``HttpServletResponseWrapper`` — in-memory ``ByteArrayOutputStream`` up to 10 MiB; spills to ``TempFile`` above. ``materialisedPath()`` always returns a uniform ``Path`` interface. | | ``PaygChargeInterceptor`` | ``AsyncHandlerInterceptor`` mirroring ``UnifiedCreditInterceptor`` shape. ``preHandle`` gates on ``@AutoJobPostMapping``, materialises multipart inputs, calls ``JobChargeService.openProcess``. ``afterCompletion`` branches on HTTP status. | | ``PaygOutputExtractor`` | Pulls PDFs out of the response body. Direct ``application/pdf`` returns body verbatim; ``application/zip`` iterates entries and keeps each ``.pdf`` entry whose first bytes match the ``%PDF-`` magic. | | ``PaygWebMvcConfig`` | Registers filter at end of Spring filter chain (after security); interceptor after ``UnifiedCreditInterceptor``. | | ``PaygFilterProperties`` | ``payg.filter.enabled`` master switch + in-memory threshold + optional max-bytes ceiling. | ## Status branching in afterCompletion | HTTP status | Action | |---|---| | **2xx** | Append OK step; extract PDFs from response; ``JobService.recordOutput`` per PDF | | **4xx** | Append FAILED step with ``errorCode``. No refund — customer paid for the attempt. No OUTPUT recording. | | **5xx + OPENED** (first-step) | ``JobChargeService.markFirstStepFailed`` → shadow row flipped to ``REFUNDED``, process CLOSED. Refund counter incremented. | | **5xx + JOINED** (mid-chain) | ``JobChargeService.decrementStepCount`` — step slot returned without resetting ``lastStepAt`` (workflow window stays active for retry). | ## New ``JobChargeService`` methods - **``markFirstStepFailed(jobId, reason)``** — flips shadow row to ``REFUNDED`` with ``refundedAt`` + ``refundReason``, closes the process. Idempotent. Mimics the eventual Stripe ``meter_event_adjustment(cancel)`` flow that real-mode will invoke at the same callsite. **Refund implies close** so a same-input retry can't lineage-join into a refunded chain for free work. - **``decrementStepCount(jobId)``** — defensive floor at 1; never drives count negative. ## Schema - Backend: ``V13__payg_shadow_charge_status.sql`` adds ``status`` (``CHARGED`` | ``REFUNDED``) + ``refunded_at`` + ``refund_reason``. ``DEFAULT 'CHARGED'`` so existing B-1 rows stay correct without backfill. - Supabase: matching migration in [Stirling-PDF-SaaS#payg-shadow-charge-status](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/tree/payg-shadow-charge-status) ## Fail-open semantics in shadow Any ``RuntimeException`` in ``preHandle`` / ``afterCompletion`` is logged at WARN, increments ``payg.filter.errors``, and lets the customer's tool call proceed unbilled. This **reverses to fail-closed** when ``wallet_policy.engine = PAYG`` (real charging) — that reversal lives inside ``JobChargeService`` and ships with the cap evaluator PR (PR-C1 in PAYG_DESIGN.md). ## Observability Micrometer metrics: - ``payg.filter.errors`` Counter — internal failures (preHandle + afterCompletion). Alert source. - ``payg.filter.calls`` Counter, tagged ``disposition`` (``OPENED`` | ``JOINED`` | ``SHORT_CIRCUIT``) - ``payg.filter.refunds`` Counter — first-step 5xx refunds - ``payg.filter.duration`` Timer — preHandle + afterCompletion wall-clock per request ## Test coverage (38 tests across 4 classes) - **PaygResponseBodyWrapperTest** (12 tests) — in-memory, spill, threshold crossing mid-chunk, writer vs outputStream exclusivity, ``resetBuffer`` with and without spill, close idempotency, single-byte writes across threshold. - **PaygOutputExtractorTest** (7 tests) — direct PDF, parametrised content type, ZIP with mixed entries + magic-byte gate, corrupt ZIP fail-open, empty ZIP. - **PaygChargeInterceptorTest** (13 tests) — all preHandle short-circuits, OPENED disposition stash, fail-open on chargeService exception, 2xx recordOutputs path, 5xx OPENED → markFirstStepFailed, 5xx JOINED → decrementStepCount, 4xx FAILED step append, max-bytes ceiling skip, PIPELINE header detection. - **JobChargeServiceTest extended** (+6 tests) — markFirstStepFailed happy path, idempotency, missing-shadow-row case, long-reason trim; decrementStepCount happy path, floor-at-1 defence, missing-job no-op. ## What's NOT in this PR (deliberate) - **No SpringBootTest layer.** The saas module doesn't have bootstrap test infrastructure (Supabase JWT config + H2 schema harness). Integration confidence comes from the manual staging deploy + SQL-flip of an internal team. Bootstrap-test infra is a focused follow-up if needed. - **No saas-mode Behave / docker-compose.** Per design §17 — deferred. Existing ``testing/cucumber/`` infrastructure doesn't yet have a saas-profile compose target; that's its own PR when warranted. - **No CreditService wire-in** (per design §13 decision). Per-row comparison data moves to the reconciliation report PR (PR-S2). ``legacy_credits_charged`` + ``diff_pct`` columns stay at 0 in shadow rows. - **No reconciliation report endpoint.** Direct SQL queries against ``payg_shadow_charge`` cover the data-access need until patterns emerge. ## Rollback levers | Symptom | Lever | |---|---| | Some / all tool calls breaking due to filter | ``payg.filter.enabled=false`` + restart (~20s) | | Shadow rows look wrong for a specific team | ``UPDATE wallet_policy SET engine = 'LEGACY' WHERE team_id = ?`` | | Mass shadow weirdness | ``UPDATE wallet_policy SET engine = 'LEGACY'`` | | Memory exhaustion from response tee | Lower ``payg.filter.response.in-memory-threshold-bytes`` | ## Test plan - [ ] CI green (build + tests) - [ ] Aikido / Snyk / SonarCloud clean - [ ] Manual: deploy to staging - [ ] Manual: flip one internal team via ``UPDATE wallet_policy SET engine = 'PAYG_SHADOW' WHERE team_id = ?`` - [ ] Manual: hit ``/api/v1/security/add-password`` with that team's JWT; verify a ``payg_shadow_charge`` row appears with ``status='CHARGED'`` - [ ] Manual: trigger a 503 (e.g. via temporary backend kill mid-request); verify the resulting row is ``status='REFUNDED'`` + the process is ``CLOSED`` - [ ] Manual: hit ``/api/v1/general/split`` with a multi-page PDF; verify one OUTPUT signature per inner PDF appears in ``job_artifact_hash`` - [ ] Manual: chain ``add-password`` → ``compress`` on the output; verify the second call JOINS the first process (no new shadow row) and the inner output OUTPUT signature is what drove the lineage join ## Stacks on / references - Stacks on: #6477 (B-1 — shadow charging engine) - Schema mirror: Stirling-PDF-SaaS#payg-shadow-charge-status branch - Design doc: ``notes/PAYG_FILTER_DESIGN.md`` (all 19 decisions DECIDED)
Stirling PDF - The Open-Source PDF Platform
Stirling PDF is a powerful, open-source PDF editing platform. Run it as a personal desktop app, in the browser, or deploy it on your own servers with a private API. Edit, sign, redact, convert, and automate PDFs without sending documents to external services.
Key Capabilities
- Everywhere you work - Desktop client, browser UI, and self-hosted server with a private API.
- 50+ PDF tools - Edit, merge, split, sign, redact, convert, OCR, compress, and more.
- Automation & workflows - No-code pipelines direct in UI with APIs to process millions of PDFs.
- Enterprise‑grade - SSO, auditing, and flexible on‑prem deployments.
- Developer platform - REST APIs available for nearly all tools to integrate into your existing systems.
- Global UI - Interface available in 40+ languages.
For a full feature list, see the docs: https://docs.stirlingpdf.com
Quick Start
docker run -p 8080:8080 docker.stirlingpdf.com/stirlingtools/stirling-pdf
Then open: http://localhost:8080
For full installation options (including desktop and Kubernetes), see our Documentation Guide.
Resources
Support
- Community Discord
- Bug Reports: Github issues
Contributing
We welcome contributions! Please see CONTRIBUTING.md for guidelines.
This project uses Task as a unified command runner for all build, dev, and test commands. Run task install to get started, or see the Developer Guide for full details.
For adding translations, see the Translation Guide.
License
Stirling PDF is open-core. See LICENSE for details.