mirror of
https://github.com/arsvendg/Stirling-PDF.git
synced 2026-07-14 10:34:06 +02:00
1fa1293b3935106193cd9ee377c81c0c5ce75d3a
478
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  LudyandGitHub
|
1ce765ab1e | refactor: replace legacy Paths usage with Path.of (#6441) | ||
 Anthony StirlingandGitHub
|
946c032fb5 | Change default language to en-US and add US language (#6621) | ||
|
88adb7adad |
create agent (#6520)
Added the create agent. Use [these prompts](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/blob/main/docgen/backend/default_templates/sample_prompts.md) to test or try your own :) Here’s the one I use ``` Hey, I need to generate an employee expense report for reimbursement. Company: Summit Consulting Partners Company address: 88 Riverside Plaza, Suite 1400, New York, NY 10069 Accounting department email: [email protected] Employee details: * Employee Name: Michael Tran * Employee ID: EMP-1047 * Department: Client Services * Report Date: January 20th, 2026 * Reporting Period: January 5th, 2026 – January 16th, 2026 * Manager Approver: Laura Simmons Trip purpose: Client onsite meetings with Atlantic Energy Solutions in Boston, MA. Expense items: * Flight (NYC to Boston roundtrip) — $325.40 — January 5th, 2026 — Airline ticket * Hotel (3 nights at Harborview Hotel) — $822.75 — January 5th-8th, 2026 * Taxi from airport to hotel — $48.00 — January 5th, 2026 * Client dinner (3 attendees) — $186.20 — January 6th, 2026 * Parking at JFK Airport — $72.00 — January 5th-8th, 2026 * Breakfast (per diem not used) — $18.50 — January 7th, 2026 * Uber to client office — $22.10 — January 7th, 2026 * Printing + presentation materials — $46.90 — January 8th, 2026 * Lunch with client — $39.75 — January 8th, 2026 * Office supplies (notebooks, pens) — $27.60 — January 10th, 2026 * Mileage reimbursement (client visit in NJ, 42 miles @ $0.67/mile) — $28.14 — January 14th, 2026 * Team lunch meeting (internal) — $64.30 — January 15th, 2026 Reimbursement method should be direct deposit. Add a notes section stating: "All receipts attached. Expenses are business-related and comply with company travel policy." ``` --------- Co-authored-by: Anthony Stirling <[email protected]> |
||
|
EthanHealy01andGitHub
|
5fca2f199a | Feature/pdf ingestion jpdfium (#6525) | ||
 James BruntonandGitHub
|
3675db5907 |
Add new triggers, sources and sinks to policies (#6543)
# Description of Changes Add new triggers: - Schedule (fires every X amount of time) - Folder watch (fires whenever the OS tells us a folder has a new file in it; on Mac this is technically a 2s schedule but that's just how Java implements it) Add new sources: - Folder (reads from this directory) Add new sinks: - Inline (stores in FileStorage) - Folder (stores in specified directory) Still want to do S3 buckets and web hooks and stuff, but they can come in a future PR. I'm hoping this should make it sufficient to be able to integrate with processing folders frontend etc. I've also changed it so that policies can have multiple sources and triggers at once, which seems like it might be useful. |
||
|
Anthony StirlingandGitHub
|
3ecd95b779 |
Add MCP server with OAuth/API-key auth (#6570)
Adds an optional MCP server (proprietary module) that exposes Stirling's PDF operations and AI capabilities to MCP clients. Off by default, zero footprint when disabled. ### What - New `/mcp` endpoint: streamable-HTTP + JSON-RPC 2.0; 8 tools (describe_operation, pages/convert/misc/security category tools, AI, upload, download). - Runs real operations over an internal loopback; results returned inline as base64 (small) or by fileId (large). ### Auth (two modes) - OAuth2 resource server: RFC 9728 protected-resource metadata, RFC 8707 audience binding, JWKS, `mcp.tools.read/write` scopes; binds each token to a provisioned Stirling account. - API-key mode: reuses Stirling per-user `X-API-KEY` (no IdP needed). ### Security - Per-user file ownership in FileStorage: async/queued writes scoped to the submitting user; legacy/owner-less files stay readable. - Admin allow/block list controls which operations are exposed. - Python engine gated behind a shared secret (`X-Engine-Auth`). - MCP filter chain is isolated and cannot weaken the main app's security. - Hardened: no upstream error-body leakage, log injection sanitized, fileId path/sidecar enumeration blocked. ### Config / footprint - Disabled by default (`mcp.enabled=false`); all beans `@ConditionalOnProperty`. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
 ConnorYohandGitHub
|
84aca12055 |
PR-S4: shadow-mode hardening (review follow-ups) (#6523)
## What this PR does Bundles the **low-risk polish items** from the [multi-agent review of #6519](https://github.com/Stirling-Tools/Stirling-PDF/pull/6519). Each change is independent, mechanical, and ships with focused unit-test coverage. The medium-severity items (\`?async=true\` OUTPUT recording, JSON-consumes endpoint coverage, SpringBootTest harness) are tracked separately in [\`notes/PAYG_DESIGN.md\` §7.5 PR-S4](https://github.com/Stirling-Tools/Stirling-PDF/blob/payg-s4-hardening/notes/PAYG_DESIGN.md) — they need design decisions + bigger infrastructure work, so this PR sticks to the mechanical wins. Stacked on #6519. When that merges to main, this rebases cleanly — no code changes. ## Changes | Area | What | Why | |---|---|---| | **\`tool_id\` becomes route pattern** | \`PaygChargeInterceptor.resolveToolId()\` prefers \`HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE\` over \`request.getRequestURI()\`. Truncates to 128 + WARN log + \`payg.filter.errors\` increment when truncation fires. | Audit rollups aggregate by endpoint instead of by every individual request's path-variable / matrix-param variant. Silent truncation now louder. | | **Direct PDF magic-byte check** | \`PaygOutputExtractor.extract()\` magic-checks the body even for direct \`application/pdf\` responses. | Asymmetric with the ZIP-entry path which always magic-checks. A tool that emits \`application/pdf\` for a JSON / HTML payload would otherwise pollute \`job_artifact_hash\`. | | **DESKTOP_APP detection** | \`X-Stirling-Client: desktop\` header → \`JobSource.DESKTOP_APP\`. | The enum value was unreachable from \`determineSource()\`; Tauri shell traffic was mis-classified as WEB. No anti-spoof — V12 step limits are identical for WEB/DESKTOP_APP so the worst-case abuse value is zero today. | | **\`max-bytes\` sensible default** | 500 MiB instead of \`null\` (unbounded). | Covers the largest realistic Stirling responses (full split-to-ZIP on a 1000-page document) while preventing pathological cases from tying up the interceptor for minutes. Set to \`null\` to disable. | | **\`BufferedOutputStream\` for spill** | Wraps the spill \`OutputStream\` in 64 KiB \`BufferedOutputStream\`. | Previously every Tomcat chunk (default 8 KiB) was a separate syscall. Big spilled responses get a syscall-bound speedup. | | **Duration timer per phase** | \`payg.filter.duration\` tagged \`phase=preHandle\` vs \`phase=afterCompletion\`. | Two distinct latency distributions were blended into one histogram; hard to alert on. | ## Tests | Test | What it covers | |---|---| | \`PaygOutputExtractorTest.pdfContentType_butBodyMissingPdfMagic_returnsEmpty\` | New direct-PDF magic-byte gate. | | \`PaygChargeInterceptorTest.preHandle_desktopClientHeader_setsJobSourceDesktopApp\` | New \`X-Stirling-Client: desktop\` → DESKTOP_APP path. | | \`PaygChargeInterceptorTest.preHandle_toolId_prefersBestMatchingPattern\` | Route pattern wins over URI when both are set. | | \`PaygChargeInterceptorTest.preHandle_toolId_truncatesAndCountsWhenLongerThan128\` | Oversized values truncate + increment errors counter. | Full saas suite green (210 tests), coverage targets met. ## What's NOT in this PR (deliberately) - **\`?async=true\` OUTPUT recording.** The JobExecutorService returns a synchronous \`JobResponse{jobId}\` body before the async tool actually runs; \`afterCompletion\` fires too early. Needs a design decision: short-circuit PAYG when \`async=true\` OR hook into \`TaskManager\` completion. Tracked in PR-S4 design doc. - **JSON-consumes endpoint coverage.** The \`MultipartHttpServletRequest\` cast skips endpoints with \`consumes = APPLICATION_JSON_VALUE\` (e.g. \`ConvertPdfJsonController.exportPartialPdf\`). Fix is either extract a request-body hash for JSON or add a CI lint forbidding non-multipart \`@AutoJobPostMapping\`. Design discussion needed. - **SpringBootTest harness for filter + interceptor wiring.** Saas module doesn't have one yet. Separate work — PR-S3 takes a different approach (docker-compose + Behave); a SpringBootTest layer would be additive in-process coverage. These are tracked in \`notes/PAYG_DESIGN.md §7.5\` so they don't slip. ## Tracked in \`notes/PAYG_DESIGN.md\` §7.5 PR-S4. |
||
|
Anthony StirlingandGitHub
|
71361f0d33 | Minor: Office doc changes (#6571) | ||
|
Anthony StirlingandGitHub
|
6478c400db | Fix font loss in rearrange/overlay/autosplit/OCR from PDFBox (#6545) | ||
|
Anthony StirlingandGitHub
|
502f6c1e4d | fix folder causing 500 toast when deleted on another machine (#6551) | ||
|
Anthony StirlingandGitHub
|
1e739b6f6f |
SaaS-aware API landing page (#6585)
# Description of Changes OLD (and still current in selfhosted) <img width="610" height="869" alt="image" src="https://github.com/user-attachments/assets/f8019298-b4ee-4a68-b928-a9746b64ac1c" /> New (in SaaS mode) <img width="635" height="876" alt="image" src="https://github.com/user-attachments/assets/6ee4946f-1d7b-42ec-a6f7-75e85739e348" /> --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
ConnorYohandGitHub
|
98967bfa86 |
PAYG: V14 + V15 — subscription_id, free-tier, RPCs, audit logs (#6532)
## Summary Two Flyway migrations + matching JPA entity updates. **Part 1 of 2** in the Stripe/Supabase wire-up (PR-SB-1 in `payg-stripe-supabase-plan.html`); the companion SaaS PR carries the twin Supabase migrations + new edge functions. ### V14 — payg_subscription_state.sql - `payg_team_extensions.payg_subscription_id` — the single switch that decides whether a team is billed. NULL = free-tier or block; NOT NULL = post Stripe meter events. - `pricing_policy.free_tier_units_per_cycle` — per-policy free allowance before a card is required. Default 0. - `payg_link_subscription(team_id, customer_id, sub_id)` RPC — idempotent. - `payg_unlink_subscription(team_id, reason)` RPC — called on `subscription.deleted`. - AFTER-INSERT trigger on `teams` so every new signup gets a `payg_team_extensions` sidecar row from creation. - Backfill for existing teams without a sidecar row. - RLS: SELECT permissive (any team member), UPDATE restricted to LEADER. Service-role bypasses (backend reads + day-1 migration writes). ### V15 — payg_audit_logs.sql - `payg_meter_event_log` — backend audit of every Stripe meter event POST attempt (idempotency-key UNIQUE; index on unposted rows for nightly reconcile). - `payg_subscription_change_log` — written by V14 RPCs on every link/unlink. ### Entity updates - `PaygTeamExtensions.paygSubscriptionId` — read-only field; RPC functions are the only writers. - `PricingPolicy.freeTierUnitsPerCycle` — read by upcoming `PaygTeamUsageService` (PR-SB-4). ### Behaviour change **None yet.** The columns + functions sit unused until PR-SB-4 wires `PaygMeterReportingService` and the free-tier gate into `JobChargeService`. This PR is pure schema + JPA wiring. ## Test plan - [x] `./gradlew :saas:test` — BUILD SUCCESSFUL - [x] Manual schema review: column types, FK directions, RLS scope - [ ] Apply against v3-Supabase via `supabase db push` (after companion SaaS PR merges) - [ ] Smoke-test the trigger: `INSERT INTO teams(...)` → assert `payg_team_extensions` row appears - [ ] Smoke-test RPCs: SQL-only test of `payg_link_subscription` + `payg_unlink_subscription` produces expected row + audit entries ## References - `notes/PAYG_DESIGN.md` (revision note 2026-06-03) - `payg-stripe-supabase-plan.html` §3.1 (RPC functions), §3.5 (RLS), §3.10 (audit-log tables) |
||
|
ConnorYohandGitHub
|
ff96a80947 |
PAYG B-3 / S-3: cucumber suite for shadow-mode flows + CI workflow (#6522)
## What this PR is End-to-end cucumber coverage for the PAYG shadow charging engine (the filter + interceptor stack from #6519), wired into CI via a new `docker-compose-tests-saas.yml` workflow that runs only on PAYG-touching PRs. Stacked on #6519. ## Automated scenarios (run by `docker-compose-tests-saas.yml`) See [`testing/cucumber/features/payg/shadow_charges.feature`](../tree/payg-s3-cucumber/testing/cucumber/features/payg/shadow_charges.feature): | Scenario | Validates | |---|---| | First tool call writes a CHARGED row | Filter + interceptor fire end-to-end | | Lineage join — second call on output | `JobService.joinOrOpen` matching; no new shadow row | | 4xx leaves the row CHARGED | "Customer paid for the attempt" semantics | | ZIP-returning tool records per-PDF OUTPUT | `PaygOutputExtractor` unpacks + records signatures | | Multi-file input writes a single shadow row | Multi-input group sizing | | `X-Stirling-Automation` sets PIPELINE source | Header → `JobSource` detection | All 6 run locally via `./testing/test-payg.sh` and will run on CI for any PR that touches `app/saas/**`, the PAYG cucumber features, the saas compose stack, or the workflow itself. ## Manual-only scenarios — documented in design doc, not in this suite Two parts of the shadow engine are deliberately not automated; the engine paths are unit-tested in `PaygChargeInterceptorTest.afterCompletion_5xx_opened_*`, and the manual procedures (which require a temporary throw endpoint or a container restart with a flag flipped) live in [`notes/PAYG_DESIGN.md` §7.5.2 "PAYG cucumber: manual-only scenarios"](../tree/payg-s3-cucumber/notes/PAYG_DESIGN.md). - **5xx first-step failure → REFUNDED + CLOSED.** No reliably-5xx-ing endpoint exists; manual procedure adds a throw endpoint, runs, asserts, removes. - **Kill-switch (`PAYG_FILTER_ENABLED=false`).** Needs a container restart mid-suite; manual procedure tears down, flips env, brings up, asserts zero shadow rows. If either gets a hot-reload path (test-only throw endpoint shipped behind a profile gate, or admin endpoint for the kill switch), automate it in a follow-up and drop the manual procedure. ## CI workflow `.github/workflows/docker-compose-tests-saas.yml` (new) — self-contained, not wired into `build.yml`'s `files-changed` matrix so the saas-cucumber job fails and succeeds independently. Triggers only on PAYG-relevant paths. No JaCoCo coverage in v1 (saas compose doesn't have the coverage override; can add later). ## Test infrastructure (recap) - **`testing/compose/docker-compose-saas.yml`** — Stirling-PDF backend with `STIRLING_FLAVOR=saas` + Postgres holding the `stirling_pdf` schema. Supabase JWT auto-config disabled; API-key auth via `SECURITY_CUSTOMGLOBALAPIKEY` is the live path the cucumber tests exercise. - **`testing/compose/payg/saas-init.sql`** + **`saas-seed.sql`** — schema bootstrap + idempotent seed (team / user / wallet_policy). - **`testing/cucumber/features/payg/shadow_charges.feature`** — the 6 scenarios above. - **`testing/cucumber/features/steps/payg_step_definitions.py`** — step defs using `requests` (HTTP) + `psycopg` (direct DB inspection). Direct DB reads are deliberate — we want to see the filter's side effects, not relay them through another API layer. - **`testing/test-payg.sh`** — companion runner to `testing/test.sh`. Brings up the saas compose, waits for health, seeds, runs behave, tears down. - **`behave.ini`** excludes `features/payg` from the default behave run (the saas-cucumber CI job invokes it explicitly). ## Why a separate harness from `testing/test.sh` The existing `test.sh` covers the proprietary-flavour stack (no PAYG tables, no saas profile). Coupling two CI matrices that fail and succeed independently into one script is asking for trouble. Keep the saas-cucumber job focused on its own concerns; once the harness is mature, the wider team can decide whether to merge them. ## Tracked in `notes/PAYG_DESIGN.md` §7.5 (PR-S3) + §7.5.2 (manual scenarios). |
||
|
Anthony StirlingandGitHub
|
800a411167 |
Hide endpoints (#6586)
# Description of Changes Hides the /api/v1/credits endpoints from the generated OpenAPI/Swagger docs. The root GET /api/v1/credits and GET /api/v1/credits/usage now carry @Hidden (the 8 admin credit endpoints were already hidden), so the whole Credit Management controller is gone from the docs. Adds a single global AI tag to the OpenAPI definition. Why We don't want the credits or AI endpoints surfaced in the public API docs yet as they are not ready for public use, but we do want the AI endpoints pre-grouped under one AI tag so they land cleanly when we later un-hide them but dont clutter PDF APIs. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
James BruntonandGitHub
|
51478e5051 |
Policies backend (#6527)
# Description of Changes Add a backend for running any multi-step PDF operations. This is designed to be used for the upcoming Policies feature, along with anything else that will require automated running of PDF operations, like the Automate tool or Processing Folders. The implementation is not complete. I've tried to get all the infrastructure in there so that we can add in whichever triggers we need in the future (like cron triggers or watching folders on disk) but currently it just supports manual triggering of the policy. The basis of this work was the operation running from the Stirling Engine, which this PR removes in favour of this new system. The only currently accessible frontend way to test this work is to ask the AI chat to execute multiple operations on a PDF, but I've also extensively tested with direct API calls to make sure that the policies work and persist properly. |
||
|
Anthony StirlingandGitHub
|
69e62d8949 |
exclude unused Redis auto-config (#6547)
fix(health): exclude unused Redis auto-config so default /actuator/health stays UP --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony StirlingandGitHub
|
d202c9c32f |
Minor: sanitize SVG (#6572)
# Description of Changes Sanitize SVG --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
 briosandGitHub
|
0dff192281 |
perf(compression): add vite-plugin-compression for gzip and Brotli support (#6279)
Signed-off-by: Balázs Szücs <[email protected]> |
||
|
EthanHealy01andGitHub
|
1698769928 |
Improvements to agent chat markdown rendering. (#6507)
### To test
- Ask the agent to “list all the things you can do and put them in a
markdown table”. I know we’re explicitly asking it for markdown, but I
don’t want to update the system prompt to ask it to make tables when
necessary because it’ll probably turn everything into a table, not sure
though, we can test in future.
- Notice how the loading is different
- Notice how the user chat is in a bubble but the agent chat is flat
(super standard design practice in AI tools, and looks much better when
the agent outputs mardown, expecially tables and needs room to do so)
- Ask it to do something different, then close the chat, and see that
the agent is marked as running and has a green outline and a green dot.
- Play around with resizing the chat to make it bigger/smaller
Open to any and all criticisms on any of the design choices, and of
course the usual, code etc.
Resizing
<img width="1572" height="812" alt="Screenshot 2026-06-01 at 2 47 53 PM"
src="https://github.com/user-attachments/assets/ec0ac1d0-01da-4025-bf7e-eea4eb544181"
/>
Loading (cool animation not visible through screenshot obviously)
<img width="559" height="141" alt="Screenshot 2026-06-01 at 2 53 41 PM"
src="https://github.com/user-attachments/assets/99f0b1f5-1719-4d78-8947-21b142293052"
/>
Removed bubbles for agent chat (maybe controversial, let me know) and
markdown now renders properly again
<img width="654" height="1060" alt="Screenshot 2026-06-01 at 2 55 01 PM"
src="https://github.com/user-attachments/assets/445f0889-a632-4751-9a16-f80ae388c632"
/>
|
||
|
ConnorYohandGitHub
|
22dacbed01 |
PAYG B-2: shadow-mode filter + interceptor (engine activation) (#6519)
## What this PR does Wires the **B-1 shadow charging engine** into real HTTP request flow. After this lands, flipping an internal team to ``PAYG_SHADOW`` via SQL begins populating ``payg_shadow_charge`` automatically — with **zero impact** on the legacy credit deduction path. **This is the load-bearing PR for shadow mode.** Without it, B-1's engine sits idle — nothing in the codebase calls ``JobChargeService.openProcess()`` from a real HTTP request. Stacks on top of #6477 (PR B-1). ## Components | Class | Role | |---|---| | ``PaygResponseBodyWrapperFilter`` | Servlet filter, installs tee'ing response wrapper. Defers wrapper close to ``AsyncListener`` for ``DeferredResult`` / ``CompletableFuture`` controllers so the lifetime spans the async window. | | ``PaygResponseBodyWrapper`` | ``HttpServletResponseWrapper`` — in-memory ``ByteArrayOutputStream`` up to 10 MiB; spills to ``TempFile`` above. ``materialisedPath()`` always returns a uniform ``Path`` interface. | | ``PaygChargeInterceptor`` | ``AsyncHandlerInterceptor`` mirroring ``UnifiedCreditInterceptor`` shape. ``preHandle`` gates on ``@AutoJobPostMapping``, materialises multipart inputs, calls ``JobChargeService.openProcess``. ``afterCompletion`` branches on HTTP status. | | ``PaygOutputExtractor`` | Pulls PDFs out of the response body. Direct ``application/pdf`` returns body verbatim; ``application/zip`` iterates entries and keeps each ``.pdf`` entry whose first bytes match the ``%PDF-`` magic. | | ``PaygWebMvcConfig`` | Registers filter at end of Spring filter chain (after security); interceptor after ``UnifiedCreditInterceptor``. | | ``PaygFilterProperties`` | ``payg.filter.enabled`` master switch + in-memory threshold + optional max-bytes ceiling. | ## Status branching in afterCompletion | HTTP status | Action | |---|---| | **2xx** | Append OK step; extract PDFs from response; ``JobService.recordOutput`` per PDF | | **4xx** | Append FAILED step with ``errorCode``. No refund — customer paid for the attempt. No OUTPUT recording. | | **5xx + OPENED** (first-step) | ``JobChargeService.markFirstStepFailed`` → shadow row flipped to ``REFUNDED``, process CLOSED. Refund counter incremented. | | **5xx + JOINED** (mid-chain) | ``JobChargeService.decrementStepCount`` — step slot returned without resetting ``lastStepAt`` (workflow window stays active for retry). | ## New ``JobChargeService`` methods - **``markFirstStepFailed(jobId, reason)``** — flips shadow row to ``REFUNDED`` with ``refundedAt`` + ``refundReason``, closes the process. Idempotent. Mimics the eventual Stripe ``meter_event_adjustment(cancel)`` flow that real-mode will invoke at the same callsite. **Refund implies close** so a same-input retry can't lineage-join into a refunded chain for free work. - **``decrementStepCount(jobId)``** — defensive floor at 1; never drives count negative. ## Schema - Backend: ``V13__payg_shadow_charge_status.sql`` adds ``status`` (``CHARGED`` | ``REFUNDED``) + ``refunded_at`` + ``refund_reason``. ``DEFAULT 'CHARGED'`` so existing B-1 rows stay correct without backfill. - Supabase: matching migration in [Stirling-PDF-SaaS#payg-shadow-charge-status](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/tree/payg-shadow-charge-status) ## Fail-open semantics in shadow Any ``RuntimeException`` in ``preHandle`` / ``afterCompletion`` is logged at WARN, increments ``payg.filter.errors``, and lets the customer's tool call proceed unbilled. This **reverses to fail-closed** when ``wallet_policy.engine = PAYG`` (real charging) — that reversal lives inside ``JobChargeService`` and ships with the cap evaluator PR (PR-C1 in PAYG_DESIGN.md). ## Observability Micrometer metrics: - ``payg.filter.errors`` Counter — internal failures (preHandle + afterCompletion). Alert source. - ``payg.filter.calls`` Counter, tagged ``disposition`` (``OPENED`` | ``JOINED`` | ``SHORT_CIRCUIT``) - ``payg.filter.refunds`` Counter — first-step 5xx refunds - ``payg.filter.duration`` Timer — preHandle + afterCompletion wall-clock per request ## Test coverage (38 tests across 4 classes) - **PaygResponseBodyWrapperTest** (12 tests) — in-memory, spill, threshold crossing mid-chunk, writer vs outputStream exclusivity, ``resetBuffer`` with and without spill, close idempotency, single-byte writes across threshold. - **PaygOutputExtractorTest** (7 tests) — direct PDF, parametrised content type, ZIP with mixed entries + magic-byte gate, corrupt ZIP fail-open, empty ZIP. - **PaygChargeInterceptorTest** (13 tests) — all preHandle short-circuits, OPENED disposition stash, fail-open on chargeService exception, 2xx recordOutputs path, 5xx OPENED → markFirstStepFailed, 5xx JOINED → decrementStepCount, 4xx FAILED step append, max-bytes ceiling skip, PIPELINE header detection. - **JobChargeServiceTest extended** (+6 tests) — markFirstStepFailed happy path, idempotency, missing-shadow-row case, long-reason trim; decrementStepCount happy path, floor-at-1 defence, missing-job no-op. ## What's NOT in this PR (deliberate) - **No SpringBootTest layer.** The saas module doesn't have bootstrap test infrastructure (Supabase JWT config + H2 schema harness). Integration confidence comes from the manual staging deploy + SQL-flip of an internal team. Bootstrap-test infra is a focused follow-up if needed. - **No saas-mode Behave / docker-compose.** Per design §17 — deferred. Existing ``testing/cucumber/`` infrastructure doesn't yet have a saas-profile compose target; that's its own PR when warranted. - **No CreditService wire-in** (per design §13 decision). Per-row comparison data moves to the reconciliation report PR (PR-S2). ``legacy_credits_charged`` + ``diff_pct`` columns stay at 0 in shadow rows. - **No reconciliation report endpoint.** Direct SQL queries against ``payg_shadow_charge`` cover the data-access need until patterns emerge. ## Rollback levers | Symptom | Lever | |---|---| | Some / all tool calls breaking due to filter | ``payg.filter.enabled=false`` + restart (~20s) | | Shadow rows look wrong for a specific team | ``UPDATE wallet_policy SET engine = 'LEGACY' WHERE team_id = ?`` | | Mass shadow weirdness | ``UPDATE wallet_policy SET engine = 'LEGACY'`` | | Memory exhaustion from response tee | Lower ``payg.filter.response.in-memory-threshold-bytes`` | ## Test plan - [ ] CI green (build + tests) - [ ] Aikido / Snyk / SonarCloud clean - [ ] Manual: deploy to staging - [ ] Manual: flip one internal team via ``UPDATE wallet_policy SET engine = 'PAYG_SHADOW' WHERE team_id = ?`` - [ ] Manual: hit ``/api/v1/security/add-password`` with that team's JWT; verify a ``payg_shadow_charge`` row appears with ``status='CHARGED'`` - [ ] Manual: trigger a 503 (e.g. via temporary backend kill mid-request); verify the resulting row is ``status='REFUNDED'`` + the process is ``CLOSED`` - [ ] Manual: hit ``/api/v1/general/split`` with a multi-page PDF; verify one OUTPUT signature per inner PDF appears in ``job_artifact_hash`` - [ ] Manual: chain ``add-password`` → ``compress`` on the output; verify the second call JOINS the first process (no new shadow row) and the inner output OUTPUT signature is what drove the lineage join ## Stacks on / references - Stacks on: #6477 (B-1 — shadow charging engine) - Schema mirror: Stirling-PDF-SaaS#payg-shadow-charge-status branch - Design doc: ``notes/PAYG_FILTER_DESIGN.md`` (all 19 decisions DECIDED) |
||
|
ConnorYohandGitHub
|
3807cdfbc6 |
PAYG: process tracking + shadow charging engine (PR B-1) (#6477)
> 📌 **Stacked on [#6464](https://github.com/Stirling-Tools/Stirling-PDF/pull/6464)** (lineage primitives, still in review). #6469 has merged so its commits are no longer in this PR's diff. Once #6464 merges, a final rebase collapses the lineage-primitives commits out of this diff too — leaving only the B-1 work. ## What this is Process tracking + shadow charging engine. Bundles PR-I7 service half with the non-filter piece of PR-I7a so the pieces ship together — none of them is useful in isolation. **Review focus:** the new files in: - \`app/saas/src/main/java/stirling/software/saas/payg/job/\` (\`JobService\`, \`JobContext\`, \`JoinOrOpenResult\`, \`StaleJobCloser\`) - \`app/saas/src/main/java/stirling/software/saas/payg/charge/\` (\`JobChargeService\`, \`ChargeContext\`, \`ChargeOutcome\`, \`JobInput\`) - \`app/saas/src/main/java/stirling/software/saas/payg/lineage/LineagePruneScheduler.java\` - their tests The 8 files inherited from #6464 (lineage primitives) are unchanged from there — they ride along in this diff until #6464 lands. The remaining work for shadow-in-staging is the ingress/egress filter that wires controllers into this engine — that's PR B-2. ## Scope ### \`JobService\` — persistence + lineage policy - **\`joinOrOpen\`** — the multi-input "any-match-joins, newest wins" rule. Hash every input via the lineage detector; if any matches an open process in the workflow window, attach to the one with the freshest \`lastStepAt\`. Step-limit overflow on the matched job spawns a fresh process; the new job's input signatures are still recorded so \`mostRecentMatchWins\` routes future calls forward. - **\`recordOutput\`** — post-tool-success path. Records OUTPUT signatures so the next call that takes this file as input lineage-matches into the same process. - **\`appendStep\`** — audit-trail step row written after a tool completes. - **\`close\`** — idempotent; safe to call from multiple paths (explicit, FE on-unload, scheduler). Returns the same row on re-close, no state mutation. - **\`findStale\` / \`closeStale\`** — workflow-window-based stale closure used by the scheduler. ### \`JobChargeService\` — the orchestrator (shadow variant) \`openProcess\` resolves the effective policy via \`PricingPolicyService\` (now in main via #6469), derives the step-limit for the current \`JobSource\` (with a defensive fallback if the policy is missing an entry), delegates to \`JobService.joinOrOpen\`, and on OPENED runs the \`DocumentClassifier\` + writes a \`payg_shadow_charge\` row. Applies the policy-level \`minChargeUnits\` floor per design § 3.4. Shadow variant only — never debits the ledger, never posts a Stripe meter event. The real-charging follow-up reuses the same orchestration and swaps the side-effect. \`legacyCreditsCharged\` on the shadow row stays \`0\` until the legacy \`CreditService\` is wired in (PR B-2), where the comparison becomes meaningful. ### Schedulers (both plain \`@Scheduled\`) - **\`StaleJobCloser\`** — fixed-rate 60 s. Closes \`OPEN\` jobs idle past the workflow window. API users never have to call close explicitly — this is the safety net. - **\`LineagePruneScheduler\`** — hourly cron, retention 1 h. Deletes \`job_artifact_hash\` rows older than the retention window. - **No \`@SchedulerLock\` / no \`shedlock\` table** — consistent with the 5 existing unguarded \`@Scheduled\` tasks in \`:saas\` (\`CreditResetScheduler\` and friends, none of which are guarded today). Cluster-correctness across all 7 saas schedulers is tracked in design § 9 as a separate focused cleanup. Underlying operations are idempotent — duplicate firings on multi-pod would be wasted DB load, not data corruption. ### Records (call-shape glue for PR B-2's filter) - \`JobContext\` / \`JoinOrOpenResult\` — input/output for \`JobService\`. - \`ChargeContext\` / \`ChargeOutcome\` — input/output for \`JobChargeService\`. - \`JobInput\` — paired \`(MultipartFile, materialised Path)\` so the upcoming ingress filter can pass both views without re-materialising. ## Tests **26 new, all green.** - 14 × \`JobServiceTest\` — no-match → opens new, single-match → joins existing, multi-input any-match-joins, multi-match newest-wins (older job never even looked up), step-limit hit spawns fresh job (and original \`stepCount\` is NOT mutated), empty inputs reject, stale-signature handling, recordOutput delegation, close idempotency, closeStale, appendStep persistence. - 7 × \`JobChargeServiceTest\` — JOINED skips classifier + shadow write entirely, OPENED writes shadow row + classifies (single + multi file paths), \`minChargeUnits\` floor applied, step-limit resolved per-\`JobSource\` from policy, missing source entry falls back to conservative default of 10. - 2 × \`StaleJobCloserTest\`, 3 × \`LineagePruneSchedulerTest\` — scheduler-wiring smoke + constructor-validation tests. \`ENABLE_SAAS=true ./gradlew :saas:test\` — BUILD SUCCESSFUL. ## What's not in this PR (lands in PR B-2) - **Tool ingress/egress servlet filter.** The highest-risk piece — materialises the request body into a \`JobInput\`, calls \`JobChargeService.openProcess\` from every \`@AutoJobPostMapping\`, records OUTPUT after success. Edge cases to validate: multipart parts, async controllers, streaming responses, errored 5xx paths, very large files. Design decisions for the filter are being worked through in \`notes/PAYG_FILTER_DESIGN.md\` before any code is written. - **Wire shadow path into legacy \`CreditService\`.** Every legacy debit writes a comparison row carrying both the PAYG would-be units and the legacy actual credits, populating \`diffPct\`. ## Design doc \`notes/PAYG_DESIGN.md\` — PR-I7 + PR-I7a status updated to reflect this bundle. § 9 carries the cluster-correctness deferral note alongside the existing LISTEN/NOTIFY trade-off note. § 7.5.1 readiness summary shows the path now needs just **1 more PR** (the filter half + CreditService wire-in, both bundled into PR B-2). |
||
|
|
35a712a278 |
smart redaction (#6195)
Co-authored-by: James Brunton <[email protected]> |
||
 
|
895dcbbafd |
Update Backend 3rd Party Licenses (#6407)
Co-authored-by: Anthony Stirling <[email protected]> Signed-off-by: stirlingbot[bot] <stirlingbot[bot]@users.noreply.github.com> |
||
|
James BruntonandGitHub
|
b705c5b84c |
Switch to use JPDFium v1.0.2, which signs the Mac binaries (#6521)
# Description of Changes Currently, it's not possible to develop the backend on Mac without manually signing the JPDFium binaries yourself since macOS will reject running the unsigned binaries. [We've now updated JPDFium to sign the Mac binaries in v1.0.2](https://github.com/Stirling-Tools/JPDFium/releases/tag/v1.0.2), so update to use that version. |
||
|
James BruntonandGitHub
|
1264f4cfed |
Set up document management for Stirling Engine (#6476)
# Description of Changes Change Stirling Engine to support deleting documents automatically. This happens both on user logout and after an amount of time specified by the Java when ingesting a document (allowing for personal documents to have short lifetimes but org documents to be left in the db with no expiry date). Also sets up an [ACL policy](https://en.wikipedia.org/wiki/Access-control_list) for the documents so the database knows which users have access to which documents. This is not fully implemented in the Java, so currently all docs are treated as having a single owner, the uploader, but theoretically when we need to support org storage, we shouldn't need to change the db schema. |
||
|
ConnorYohandGitHub
|
e6974d52f7 |
PAYG: hash-lineage detection primitives (modular extractor / store / detector) (#6464)
## What this is
Three orthogonal interfaces — each with one production impl — for
detecting whether an incoming tool call should join an existing process
via content-hash lineage. Groundwork for PR-I7a: nothing in this PR
calls the detector yet; the ingress/egress filter that wires it into
every controller lands separately.
Built to be modular along three axes. Swapping any of them should not
require changes elsewhere:
| Axis | Interface | V1 impl | Plausible future impl |
|---|---|---|---|
| Hash algorithm | `LineageSignatureExtractor` |
`ByteHashSignatureExtractor` (SHA-256) | `PdfMetadataSignatureExtractor`
(PDF `/ID`, content-stream hash) |
| Storage backend | `JobLineageStore` | `JpaJobLineageStore` |
`RedisJobLineageStore` |
| Matching policy | `HashLineageDetector` | `DefaultHashLineageDetector`
| strategy-driven variant (any-match-joins for multi-input — lives in
JobService) |
## Interfaces
### `LineageSignatureExtractor` — what counts as a fingerprint
```java
public interface LineageSignatureExtractor {
Set<LineageSignature> extract(Path file) throws IOException;
String name();
}
```
File-based (not stream-based) so a future PDF-aware extractor can open
the same file via jpdfium / PDFBox and pull `/ID[0]` or a content-stream
hash. Multiple extractors compose at the detector layer — Spring
auto-wires all `LineageSignatureExtractor` beans, the detector unions
their results.
Production impl: **`ByteHashSignatureExtractor`** — SHA-256 over the
file via 64 KiB-buffered `DigestInputStream`. Hardware-accelerated by
the JVM (Intel SHA-NI, ARM SHA).
### `JobLineageStore` — where signatures live
```java
public interface JobLineageStore {
void record(UUID jobId, Set<LineageSignature> signatures, ArtifactKind kind);
Optional<LineageMatch> findOpenJobForSignatures(Long userId, Set<LineageSignature> candidates, Duration window);
int pruneOlderThan(Instant cutoff);
}
```
Knows nothing about storage technology. Production impl
**`JpaJobLineageStore`** runs a single joined query against
`job_artifact_hash` ⋈ `processing_job` — status + window filtering
happen at the database. The query is bounded by `Limit.of(1)` on the hot
path so a job set sharing a popular signature doesn't materialise
unwanted rows. A future `RedisJobLineageStore` (or write-through hybrid)
is a drop-in.
### `HashLineageDetector` — the high-level API
```java
public interface HashLineageDetector {
Optional<LineageMatch> detect(Long userId, Path inputFile) throws IOException;
void record(UUID jobId, Path file, ArtifactKind kind) throws IOException;
}
```
**`DefaultHashLineageDetector`** delegates extraction to every
registered `LineageSignatureExtractor`, storage to the configured
`JobLineageStore`, and reads `payg.lineage.workflow-window` (default
`PT5M`) from config. When a single extractor throws (e.g. a future
PDF-aware extractor against a malformed PDF), the other extractors still
contribute — failures don't block the byte-hash from landing.
## Profile gating
All three `@Component` beans (`JpaJobLineageStore`,
`ByteHashSignatureExtractor`, `DefaultHashLineageDetector`) are
`@Profile("saas")` — consistent with every other `:saas` bean. Without
this guard the JPA store would fail to wire against its profile-gated
repository in non-saas profiles that pull `:saas` onto the classpath.
## Tests
Run entirely in-memory; no database required.
- **`LineageSignatureTest`** — storage-key encoding round-trips, rejects
malformed `"type:value"` keys.
- **`ByteHashSignatureExtractorTest`** — identical bytes → identical
sigs; empty file hashes to the well-known SHA-256-of-empty constant; 10
MiB file streams without OOM.
- **`DefaultHashLineageDetectorTest`** — same-user / within-window /
status=OPEN filtering, multi-signature matching (one extractor sees
`pdf-id` and matches even when bytes differ), most-recent-job-wins,
record+detect round-trip, extractor-throwing-doesn't-break-others.
**`InMemoryJobLineageStore`** (in test sources) implements the same
`JobLineageStore` interface as the JPA impl, plus a `registerJob` hook
for tests to model job state. Same contract — proves the abstraction is
portable. When the Redis impl lands it gets the same contract tests.
## What's not in this PR (deliberate)
- The tool ingress/egress filter that wires the detector into every
controller — separate, focused review.
- `JobChargeService.openProcess()` — uses the detector, part of the
charging machinery, separate PR.
- Prune scheduler that calls `pruneOlderThan` — small follow-up
alongside the `shedlock` foundational table.
- PDF-aware extractor (`PdfMetadataSignatureExtractor`) — to be added
when we measure how often byte-hash-only misses real workflows.
- Multi-input "any-match-joins" lineage policy — that's a `JobService`
decision (PR-I7), not a primitive.
## Self-review pass applied
An independent code-review on this PR caught:
- **HIGH:** Missing `@Profile("saas")` on the three `@Component` beans →
fixed.
- **MEDIUM:** `pruneOlderThan` missing `@Transactional` (its
`@Modifying` query would have thrown
`InvalidDataAccessApiUsageException`) → fixed.
- **MEDIUM:** `findOpenJobsForSignatures` fetching the whole match set
just to `get(0)` → now takes `Limit`, JPA store passes `Limit.of(1)` on
the hot path.
- **LOW:** `InMemoryJobLineageStore` used both `synchronized` methods
and `ConcurrentHashMap` → dropped the redundant `ConcurrentHashMap`.
Deferred: project-wide UTC unification (`LocalDateTime.now()`
system-zone is the established convention; flipping one file mid-stack
caused a real test failure — proper fix needs its own audit).
## Rollback
Straight `git revert`. No callers yet; deleting these classes wouldn't
break anything.
---
## Checklist
- [x] Tests pass: `ENABLE_SAAS=true ./gradlew :saas:test`
- [x] No new warnings
- [x] Self-review performed (HIGH + MEDIUM findings addressed)
|
||
|
Anthony StirlingandGitHub
|
b355ccec9e | Add Valkey cluster backplane and sticky-410 ownership (clusters) (#6472) | ||
|
briosandGitHub
|
2c0ebc28a7 | perf(api): optimize static asset caching, enable ETag support, and expand response compression mime types. (#6273) | ||
|
Anthony StirlingandGitHub
|
30e782e29c | Disable Save-to-server when storage off, fix QR port 0 (#6473) | ||
|
ConnorYohandGitHub
|
28b81828b5 |
PAYG: PricingPolicyService + admin REST + 30s read cache (#6469)
## What this is PR-I1 service half from `notes/PAYG_DESIGN.md`. Built on top of the data model from #6460 — answers "what pricing policy applies to this team right now?" with a fast cache and an admin write surface. ## Scope | Piece | Where | |---|---| | `PricingPolicyService` — `getEffectivePolicy(teamId)` with 30s Caffeine cache + admin write paths | `app/saas/.../payg/policy/PricingPolicyService.java` | | `PolicyChangedEvent` — published after admin writes for in-process cache invalidation | `app/saas/.../payg/policy/PolicyChangedEvent.java` | | Admin REST — list / get / create / set-default / set team override / get effective | `app/saas/.../payg/policy/admin/PricingPolicyAdminController.java` + DTOs | | `PricingPolicyRepository.clearDefaultFlag()` — atomic clear for set-default | repository update | | `SaasJpaConfigScanTest` — drift guard against the JPA scan paths going stale (carried over from the #6460 review concern) | new test | | V12 default-policy seed (`v1-initial`, 25 pages/unit, 5 MiB/unit, per-`JobSource` step limits) | `V12__seed_default_payg_policy.sql` | ## Lookup precedence 1. `PaygTeamExtensions.pricingPolicyId` set → return that policy 2. Else return the `pricing_policy` row with `is_default = TRUE` 3. Override row points at a deleted policy → log warn, fall back to default (safety net for racing deletes) 4. No default → `IllegalStateException` (V12 seed guarantees one exists) ## Cache behaviour - 30s `expireAfterWrite` Caffeine, max 10k entries, keyed by `teamId`. - **Single correctness model: the TTL.** Cross-instance propagation is at-most-30-seconds. The writer instance sees its own change immediately via the `PolicyChangedEvent` after-commit publish. Other instances pick it up on the next TTL expiry. - Admin reads use `getEffectivePolicyUncached` so admins always see their own write straight back. **Why no LISTEN/NOTIFY runner.** An earlier cut of this PR included a Postgres `LISTEN policy_changed` runner so cross-instance propagation was instant. Dropped — admin policy changes are events-per-week and the 30s TTL is already the correctness floor; the listener was ~250 lines of nontrivial code (raw JDBC outside HikariCP, daemon thread, reconnect loop, lock-protected connection lifecycle) for a use case that isn't on the hot path. Trade-off is documented in `notes/PAYG_DESIGN.md` §9 with three concrete triggers that would justify reintroducing it (aggressive cap enforcement, Redis landing for other reasons, real-time admin UI). ## Writes — transactional, fire `PolicyChangedEvent` after commit - `create(draft)` — rejects pre-set `policy_id` or `is_default=true` (promotion must go through `setDefault` so the partial unique idx is freed first). - `setDefault(id)` — atomically clears the existing default via `clearDefaultFlag()` then flips the new row. Idempotent: silent no-op if the row is already default. - `setTeamOverride(teamId, policyId | null)` — validates the policy exists before save; `null` clears the override. `publishOnCommit` uses `TransactionSynchronizationManager.afterCommit` so listeners never see pre-commit state. Outside a transaction (test paths) falls through to immediate publish. ## Admin REST surface — `/api/v1/admin/payg/...` All endpoints `@PreAuthorize("hasRole('ADMIN')")`: - `GET /policies` — list all - `GET /policies/{id}` — read one - `POST /policies` — create new (non-default) - `POST /policies/{id}/set-default` — atomic promote - `PUT /teams/{teamId}/policy-override` — set or clear per-team override - `GET /teams/{teamId}/effective-policy` — cache-bypassing live read Validation errors → 400, unknown rows → 404. ## Counterpart Supabase PR [`Stirling-PDF-SaaS#298`](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/298) — seeds the same V1 default policy on the Supabase side via `20260528000002_payg_seed_default_policy.sql`. ## Tests - 17 × `PricingPolicyServiceTest` — lookup precedence, cache hit/miss, invalidation on event, mutation paths publishing event, error cases. - 14 × `PricingPolicyAdminControllerTest` — every endpoint's happy path + error mapping, DTO defensive-copy invariant. - 2 × `SaasJpaConfigScanTest` — reflection-based guard that `payg.repository` is in `@EnableJpaRepositories` and `payg` is in `@EntityScan`. Without this, new sub-packages can silently fail to wire at runtime — same class of bug that the #6460 review caught. Full `:saas:test` BUILD SUCCESSFUL. ## Design doc `notes/PAYG_DESIGN.md` §7.4 PR-I1 — completes the service half (the schema half landed in #6460). §9 carries the 30s-TTL trade-off note. |
||
|
ConnorYohandGitHub
|
83ea07ed6a |
saas: DocumentClassifier + PAYG data model (#6460)
# Description of Changes Two layers — the `DocumentClassifier` utility plus the full data model for the new billing engine. Nothing wires the entities into application behaviour yet; services and controllers land in follow-up PRs. **Companion PR:** [Stirling-PDF-SaaS#296](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/296) — Supabase migration for the v3 dev branch, schema-equivalent to the Flyway migration in this PR. ## 1. DocumentClassifier (under `payg.docs`) `DocumentClassifier` computes the doc-unit cost of an uploaded file (or multi-file input) under a `PricingPolicy`. PDFs read page count via `stirling.software.jpdfium.PdfDocument`; non-PDFs are bytes-only. Formula: `max(ceil(pages / docPagesPerUnit), ceil(bytes / docBytesPerUnit))` clamped to `[1, fileUnitCap]`. Multi-file is the sum of raw per-file units capped at `fileUnitCap × file_count`. Two floors, by design: the classifier returns `docUnits` with an absolute `1` floor for non-empty input; the policy-level `minChargeUnits` is intentionally applied later, at process-open time in `JobChargeService`, per design § 3.4 (`unitsForProcess = max(policy.min_charge_units, docUnits)`). Documented in the interface + impl javadoc. Upload bytes are materialised through `TempFileManager.createManagedTempFile` so jpdfium gets a `Path`; the temp file auto-deletes on close. Twelve tests, all in-memory fixtures generated with PDFBox at test time — no committed binary blobs. ## 2. PAYG data model (under `payg.*`) JPA entities, repositories, and a Flyway migration covering the full schema in §6 of the design. **Enums** (`payg.model`): `JobSource`, `ProcessType`, `JobStatus`, `JobStepStatus`, `ArtifactKind`, `LedgerEntryType`, `LedgerBucket`, `ReferenceType`, `EntitlementState`, `FeatureSet`, `FeatureGate`, `WalletEngine`, `CapPeriod`, `AutoGroupStrategy`. **Entities + repositories:** | Entity | Table | Notes | |---|---|---| | `PricingPolicy` | `pricing_policy` | Promoted from a record. `stepLimits` is `Map<JobSource, Integer>` persisted via normalised child table `pricing_policy_step_limit`. `stripePriceIds` is `Set<String>` persisted via `pricing_policy_stripe_price` — currency comes from `stripe.prices` via Sync Engine, not stored locally. | | `ProcessingJob` | `processing_job` | UUID PK. Tracks lineage window via `step_count` and `last_step_at`. | | `ProcessingJobStep` | `processing_job_step` | Per-tool-call audit. | | `JobArtifactHash` | `job_artifact_hash` | Composite key `(job_id, content_hash, kind)`. `content_hash VARCHAR(128)` so multiple signature schemes coexist as `"type:value"` storage keys. Lineage detector queries this. | | `WalletLedgerEntry` | `wallet_ledger` | Append-only, signed `amount_units`. Two unique indexes kill double-posting. | | `WalletPolicy` | `wallet_policy` | Per-team engine + cap + degradation rules + lineage strategy. No `@Version` — admin-only writes (documented in javadoc). | | `WalletEntitlementSnapshot` | `wallet_entitlement_snapshot` | Composite key `(team_id, user_id)`; `user_id = 0` is the team-wide sentinel. No `@Version` — full-row recompute via `EntitlementService.recompute` (documented in javadoc). | | `PaygShadowCharge` | `payg_shadow_charge` | Per-job diff while in `PAYG_SHADOW` engine mode. | | `PaygTeamExtensions` | `payg_team_extensions` | Sidecar 1:1 with `teams` carrying `pricing_policy_id` (per-team override) + `stripe_customer_id`. Sidecar pattern (mirrors `saas_team_extensions`) so OSS Hibernate ddl-auto never sees PAYG columns on `teams`. | **Column adds:** - `team_memberships.cap_units` (optional per-member sub-cap) **Width split (intentional, documented in V11):** per-row deltas (`wallet_ledger.amount_units`, `processing_job.charged_units`) are `INTEGER` because no single charge realistically approaches 2B units. Cap and period-rollup columns (`team_memberships.cap_units`, `wallet_policy.cap_units`, `wallet_entitlement_snapshot.period_spend_units / period_cap_units`) are `BIGINT` because they accumulate across a billing period and admins may legitimately set headroom-cap values into the millions. **JPA wiring:** `SaasJpaConfig` was updated to include `stirling.software.saas.payg.repository` in `@EnableJpaRepositories.basePackages` and `stirling.software.saas.payg` in `@EntityScan` (covers `payg.policy` / `payg.job` / `payg.wallet` / `payg.entitlement` / `payg.shadow` recursively). New `SaasJpaConfigScanTest` reads the annotations reflectively and asserts every expected package is wired — catches the next time someone adds a new sub-package without updating the scan paths. **Migration:** `V11__saas_payg_model.sql` (purely additive). Schema-equivalent to the Supabase migration in the companion PR — including the `VARCHAR(128) content_hash` width that's needed for the multi-signature-scheme storage encoding the lineage layer uses. ## 3. Smoke tests `PaygEntitiesSmokeTest` exercises each entity via the no-arg ctor JPA requires, plus getter/setter round-trips and composite-key equality — catches Lombok/annotation regressions without needing a database. Real-DB integration coverage lands alongside the services that consume each entity. ## Why this is safe to land now - All schema changes are additive — no existing rows modified, no columns dropped. - The entities are not yet referenced from any production code path; they exist for the next PRs to build on. - The v3 Supabase dev branch picks up the schema via the companion PR; the main repo's Flyway migration applies the same shape when an instance boots against a freshly-migrated v3 database. ## Open decisions made - **Step-limits keyed by `JobSource`** rather than by `ProcessType`. Captures the "self-hosted gets a different knob" framing in earlier feedback. Trivially overridable per pricing policy version. - **Step limits + Stripe price IDs normalised into child tables** rather than JSONB on `pricing_policy` (per Connor's review on #296). Typed columns, queryable directly, no JSON parsing. - **Currency dropped from `pricing_policy_stripe_price`** — it lives on `stripe.prices.currency` and is resolved via Sync Engine. App is currency-blind. ## Rollback Straight `git revert` on this PR. The Supabase migration in #296 is additive and can be left in place safely — the running app ignores tables it doesn't reference. --- ## Checklist - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings - [x] I have run `task check` (via `./gradlew :saas:test` with `ENABLE_SAAS=true`) — passes |
||
|
EthanHealy01andGitHub
|
763595a5a3 |
feat: add Agents UI to proprietary right sidebar (#6454)
Update UI to include agents Run `task dev:all` to test |
||
|
Anthony StirlingandGitHub
|
398617391b | Fix SSO auto-login and custom metadata settings not persisting on restart (#6468) | ||
|
ConnorYohandGitHub
|
a0e0e88f07 |
saas: harden CreditService Stripe ordering + lint @AutoJobPostMapping weights (#6458)
# Description of Changes Two narrowly-scoped hardening changes to the credits engine. ## 1. CreditService — move Stripe meter call to `afterCommit` The Stripe metered-usage call sits inside the surrounding `@Transactional`, holding the `user_credits` row lock for the duration of an HTTP round-trip to Supabase. Under load this starves concurrent debits; a transient Stripe blip rolls back a (correct) free-credit consumption and forces the caller to retry. The Stripe call now runs in a `TransactionSynchronization.afterCommit` hook — DB commits first, Stripe fires immediately after. If Stripe fails after commit, we log + increment a new `credits.stripe_report.failures` counter; the idempotency key is stable, so a manual replay recovers without double-charging. Applied to both `consumeCreditBySupabaseId` and `consumeCreditWithWaterfall`. **Dead-code removed:** - Unreachable UUID fallback for MDC `requestId` — `CorrelationIdFilter` already guarantees the key on every request. - The `"Unable to report usage to Stripe"` `RuntimeException` and its catch block — the afterCommit refactor eliminates the throw path. - `StripeRollbackOnFailureTest` — pinned the rollback-on-Stripe-fail behaviour this refactor replaces. ## 2. `@AutoJobPostMapping` — build-time lint for `resourceWeight` `UnifiedCreditInterceptor` multiplies `resourceWeight` into the per-call charge. An endpoint that falls through to the annotation default produces a charge derived from a value nobody chose. - Annotation default flipped from `1` to `Integer.MIN_VALUE` (sentinel). Both runtime readers (`UnifiedCreditInterceptor`, `AutoJobAspect`) already clamp into `[1, 100]` so behaviour is unchanged. - New `AutoJobPostMappingWeightTest` scans the classpath and fails the build if any method leaves the sentinel. - Initial run caught 11 endpoints relying on the default. Explicit weights now declared, chosen by comparing to peer endpoints: - `EditTextController` — LARGE - `EmailController#sendEmailWithAttachment` — SMALL - `ConvertPDFToMarkdown` — MEDIUM - `AttachmentController` (extract/list/rename/delete) — SMALL × 4 - `ConvertImgPDFController` (cbr/cbz ↔ pdf) — MEDIUM × 2, LARGE × 2 ## Tests - `StripeUsageIdempotencyKeyTest` — pins the `(supabaseId, overage, requestId)` idempotency key shape so Stripe always dedupes a retry. - `StripeAfterCommitOrderingTest` — pins that `afterCommit` fires after commit and NOT on rollback. - `AutoJobPostMappingWeightTest` — the lint itself, plus a self-check that the classpath scan finds at least 10 `@AutoJobPostMapping` methods (guards against the lint passing vacuously). Build verified: `ENABLE_SAAS=true ./gradlew :stirling-pdf:test :saas:test`. --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) — no translation changes - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) — internal-billing change, no public docs impact - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) — N/A ### Translations (if applicable) - [ ] Not applicable ### UI Changes (if applicable) - [ ] Not applicable ### Testing (if applicable) - [x] I have run `task check` (via `./gradlew :stirling-pdf:test :saas:test` with `ENABLE_SAAS=true`) — passes - [x] I have tested my changes locally |
||
|
Anthony StirlingandGitHub
|
c80a5db5f5 | folder and file fixes (#6461) | ||
|
|
8bd78d2624 |
Add landscape page size options (#6248)
# Description of Changes Adds orientation (portrait/landscape) to the Adjust Page Scale tool. - Orientation as a separate parameter (per review), sent through to the backend - ScalePagesController simplified; PDFWithPageSize gains the orientation field - Regenerated tool_models.py; frontend + backend tests added --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. --------- Co-authored-by: ConnorYoh <[email protected]> |
||
|
Anthony StirlingandGitHub
|
b3c4b8b463 | Add S3 storage and cluster artifact backend (#6457) | ||
|
James BruntonandGitHub
|
44fbf8c587 |
Various bug fixes found while testing SaaS build (#6459)
# Description of Changes Various fixes and improvements I made while testing the SaaS code: - Changes the new `.env.saas` file to live in `app/` and match the semantics of the other `.env` files - Adds top-level `task dev:saas` command to spawn SaaS frontend & backend - Deletes dead SaaS code and improves some overriding logic - Fixes refreshing issue when coming back to the tab - Fix the Compare tool's selection logic - Make Compare handle error cases properly - Fixes the location of the "Dismiss All Errors" button (was rendering on top of the top-bar with a transparent background previously so it looked rubbish) - Fixes file selection in PDF Editor |
||
|
Anthony StirlingandGitHub
|
76840d8a57 | Add CI DB migration smoke test against v2.0/v2.5/v2.10 updates (#6453) | ||
|
ConnorYohandGitHub
|
43b67d213d |
feat(oauth2): opt-in claim-dump diagnostics for OIDC login failures (#6456)
# Description of Changes ## What & why Customers using ADFS (or any generic OIDC provider that doesn't emit `email`) hit `Attribute value for 'email' cannot be null` during OAuth2 login with no visibility into what claims the provider actually sent. The only available remedy was guessing at `security.oauth2.useAsUsername` until something worked. This PR adds a new opt-in `security.oauth2.debugLogging` flag (default `false`). When enabled, `CustomOAuth2UserService` logs: - All ID token claims (sorted, with values) - All UserInfo endpoint claims (if any) - The merged attribute key set Spring exposes to `getAttribute()` - The value the configured `useAsUsername` actually resolved to - A **`Hint:`** line listing the claim keys present in the token that map to a valid `UsernameAttribute` enum value — i.e. exactly what the operator could put in `useAsUsername` to make login work Logged at `INFO` on the success path and `ERROR` on failure (inside the existing `catch (IllegalArgumentException)` block that throws `OAuth2AuthenticationException`). The block is wrapped with a `[OAUTH2 DEBUG] ... [/OAUTH2 DEBUG]` banner and ends with a PII warning so operators don't leave it on in production. Default off → zero observable change for anyone not actively troubleshooting. ## Files changed | File | Why | |---|---| | `app/common/.../ApplicationProperties.java` | New `debugLogging` field on the `OAUTH2` config class with javadoc warning about PII | | `app/core/src/main/resources/settings.yml.template` | Documents `oauth2.debugLogging` so it appears on next startup | | `app/proprietary/.../security/service/CustomOAuth2UserService.java` | Emits the claim dump + suggestion hint when the flag is on | | `app/proprietary/.../security/service/CustomOAuth2UserServiceDebugLoggingTest.java` (new) | Unit test: mocks the OIDC delegate, asserts off-path is silent and on-path emits the dump with the right Hint contents | ## End-to-end verification Ran the bundled `testing/compose/docker-compose-keycloak-oauth.yml` Keycloak realm, configured `security.oauth2.useAsUsername: mail` (Keycloak emits `email`, not `mail`) and `provider: demarest` (matches the original customer bug report). Triggered the OAuth flow at `http://localhost:8080/oauth2/authorization/demarest` and confirmed: - The ERROR-level dump fires with the full 19-claim ID token decoded - `-- Value at 'mail' : <NULL — this is why login fails>` correctly identifies the missing claim - `-- Hint:` correctly suggests `[email, family_name, given_name, preferred_username]` (the four keys present that map to valid `UsernameAttribute` values) - Auth still fails with the original `OAuth2AuthenticationException` — no change to control flow, just added diagnostic logging Unit test (`CustomOAuth2UserServiceDebugLoggingTest`) covers both branches. ## Reviewer notes - **No new public APIs.** The flag is config-only; no servlet endpoints exposed. - **PII is logged when the flag is on.** This is the whole point — operators need to see the claims to fix their config — but it's gated, defaults off, and the dump self-documents with a `WARNING: ... Set security.oauth2.debugLogging=false once troubleshooting is complete.` footer. - **Why log everything, not just sub/email?** Because the operator doesn't know in advance which claim they actually want. ADFS uses `upn` in some configs and `preferred_username` in others; Azure AD uses `oid`; the customer here had neither. Dumping the full set is the only way to make the diagnostic self-service. - **Out of scope for this PR (follow-ups):** - The `UsernameAttribute` enum doesn't include `upn` / `unique_name` (common ADFS claims). If the customer's token only has `upn`, the Hint will be empty even though the operator can see `upn` in the dump. Worth a separate PR to extend the enum. - The known-provider validator in `Provider.java` (rejects e.g. `useAsUsername: mail` for `provider: keycloak` at startup) bypasses our diagnostic for those provider names. ADFS customers using `provider: <name>` fall into the `default` branch so are not affected — but it's a sharp edge worth documenting. --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) — N/A, backend-only change - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] Doc-repo update (if functionality has heavily changed) — diagnostic flag is self-documenting via the `settings.yml.template` comment and the in-log warning; happy to add a doc-repo entry if reviewers want one - [ ] Translation tags — N/A ### UI Changes (if applicable) - [ ] N/A — backend-only ### Testing (if applicable) - [x] Unit test added (`CustomOAuth2UserServiceDebugLoggingTest`) covering on/off paths and Hint correctness - [x] End-to-end verified locally against bundled Keycloak compose with intentionally misconfigured `useAsUsername` - [x] Full `:proprietary:test` suite passes |
||
|
Anthony StirlingandGitHub
|
d42b779644 | Add server-side folders and files page UI (#6383) | ||
|
Anthony StirlingandGitHub
|
4564ed5bec | Add cluster backplane abstraction and interfaces (#6449) | ||
 
|
7743720f0a |
chore: fix "Endoints" typo in health route comment (#6446)
`app/common/src/main/java/stirling/software/common/util/RequestUriUtils.java:175` section comment said "Health Endoints" → "Health Endpoints". Comment-only. Co-authored-by: Matt Van Horn <[email protected]> |
||
|
Anthony StirlingandGitHub
|
56ff1e5092 |
impl migration to pdfium for split (#6410)
## Summary Migrates `SplitPDFController`, `SplitPdfByChaptersController`, `SplitPdfBySizeController` from PDFBox to JPDFium. `SplitPdfBySectionsController` and `AutoSplitPdfController` are intentionally left on PDFBox (require JPDFium 1.0.2 features that don't exist yet). ## Benchmark (audited on `audit/jpdfium-split`, file `app/core/src/test/java/stirling/software/SPDF/bench/SplitBenchmark.java`) | Workload | PDFBox heap | JPDFium heap | PDFBox wall | JPDFium wall | |---|---|---|---|---| | 100 pp, chunk 10 | +21-26 MB | **+0.02 MB** | 80-106 ms | **25 ms** | | 300 pp, chunk 10 | +59 MB | **+1.0 MB** | 232 ms | **76 ms** | **98-99.9% heap reduction. 3-4.2x faster wall.** ## Hybrid - AcroForm-bearing splits keep PDFBox `FormUtils.pruneOrphanedFormFields` post-pass (FPDF_ImportPagesByIndex drops AcroForm dict). Sub-bench shows +1.0 MB / +27 ms - tightly bounded. - Metadata extraction stays on PDFBox. - `SplitPdfBySectionsController` - JPDFium `PdfPageSplitter` only does 2-up halving, not arbitrary MxN. - `AutoSplitPdfController` - needs PDFRenderer + zxing for QR markers. ## Test plan - [ ] 30/30 unit tests pass with PDFBox `Loader.loadPDF` as the oracle (assert page counts + document totals) - [ ] Existing cucumber feature `split.feature` continues to pass - [ ] AcroForm-bearing PDF round-trips without orphaned widgets (covered by existing FormUtils tests) |
||
 Reece BrowneandGitHub
|
4047d02086 | chore: address restructure PR review feedback (#6423) | ||
|
0a50e765b7 |
Restructure/frontend editor (#6404)
## Move editor under `frontend/editor/`
Pure restructure: `frontend/` becomes the workspace, `frontend/editor/`
holds
the PDF editor. 1775 file renames + 40 wiring edits. No logic changes.
### Why
`frontend/` is currently the editor — its `src/`, `public/`,
`src-tauri/`,
config files all sit at the root. Promoting `frontend/` to a
workspace and putting the editor in a sibling folder leaves room for
future
apps to drop in alongside it, sharing one `package.json` /
`node_modules` /
lint config / Storybook.
### What moves
frontend/
├── editor/ ← NEW: everything editor-specific
│ ├── src/ ← was frontend/src/
│ ├── public/ ← was frontend/public/
│ ├── src-tauri/ ← was frontend/src-tauri/
│ ├── index.html, vite.config.ts, vitest.config.ts, playwright.config.ts
│ ├── tsconfig*.json, tailwind.config.js, postcss.config.js
│ ├── scripts/
│ ├── .env, .env.desktop, .env.saas
│ └── DeveloperGuide.md
├── package.json, package-lock.json, node_modules/ ← workspace install
├── eslint.config.mjs, .prettierrc, .prettierignore ← shared tooling
├── .gitignore
└── README.md
### Wiring edits (40 files)
- `.taskfiles/frontend.yml`, `desktop.yml`, `e2e.yml`
- `build.gradle`, `app/core/build.gradle`
- `eslint.config.mjs`, `frontend/package.json`, `.gitignore`,
`.prettierignore`
- `docker/frontend/Dockerfile`
- 8 `.github/workflows/*.yml`, plus `.github/dependabot.yml`,
`.github/config/.files.yaml`, `.github/labeler-config-srvaroa.yml`
- `scripts/translations/**`
- Docs: `AGENTS.md`, `CLAUDE.md`, `ADDING_TOOLS.md`,
`DeveloperGuide.md`,
`WINDOWS_SIGNING.md`, `devGuide/HowToAddNewLanguage.md`,
`frontend/README.md`,
`frontend/editor/DeveloperGuide.md`
Plus 3 renamed + edited: `editor/vite.config.ts` (env path +
node_modules
walk-up), `editor/scripts/setup-env.mts` (renamed from `.ts` for
`import.meta.url`), `editor/scripts/build-provisioner.mjs` (resolve
src-tauri
relative to script).
### Verification
| Check | Result |
|---|---|
| `task frontend:typecheck:all` (6 variants) | exit 0 |
| `task frontend:lint` (eslint + dpdm) | exit 0 |
| `task frontend:format:check` | exit 0 |
| `task frontend:test` | 657 tests pass, 50 files |
| `task frontend:build:{core,proprietary,saas,desktop,prototypes}` | all
green |
| `task desktop:build` | full Tauri pipeline →
`Stirling-PDF_2.11.0_x64_en-US.msi` |
| `playwright test --list --project=stubbed` | 172 tests discovered |
`task desktop:build` exercises the heaviest path — Rust + WiX + MSI
bundle
against the moved `editor/src-tauri/`. If anything in the restructure
was
wrong it wouldn't have built.
### Test plan
- [ ] `frontend-validation.yml` green
- [ ] `e2e-stubbed.yml` green
- [ ] `tauri-build.yml` green on at least one platform
- [ ] `check_toml.yml` runs on a translation-touching PR
---------
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
|
||
|
Anthony StirlingandGitHub
|
48027ee9d6 | JDK25 - Integrate Stirling-Tools JPDFium for PDF operations (#6387) | ||
|
|
9d081d1792 |
SaaS Consolidation (#6384)
Co-authored-by: ConnorYoh <[email protected]> |
||
 dependabot[bot]andGitHub
|
ea91a35518 |
build(deps): bump org.postgresql:postgresql from 42.7.10 to 42.7.11 (#6375)
Signed-off-by: dependabot[bot] <[email protected]> |
||
|
|
2399da6893 |
migrate exportUpdatedPages from bytes to stream (#6201)
Co-authored-by: ConnorYoh <[email protected]> |