mirror of
https://github.com/arsvendg/Stirling-PDF.git
synced 2026-07-14 10:34:06 +02:00
69ee39fa6eae1379b92c71af49cc8c6fcd508514
5193
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
69ee39fa6e | Fix settings: dark borders, update dropdown z-index, dead accessibility link (#6528) | ||
|
|
353b5c807c | sort comments sidebar in visual reading order (#6439) (#6514) | ||
|
|
22dacbed01 |
PAYG B-2: shadow-mode filter + interceptor (engine activation) (#6519)
## What this PR does Wires the **B-1 shadow charging engine** into real HTTP request flow. After this lands, flipping an internal team to ``PAYG_SHADOW`` via SQL begins populating ``payg_shadow_charge`` automatically — with **zero impact** on the legacy credit deduction path. **This is the load-bearing PR for shadow mode.** Without it, B-1's engine sits idle — nothing in the codebase calls ``JobChargeService.openProcess()`` from a real HTTP request. Stacks on top of #6477 (PR B-1). ## Components | Class | Role | |---|---| | ``PaygResponseBodyWrapperFilter`` | Servlet filter, installs tee'ing response wrapper. Defers wrapper close to ``AsyncListener`` for ``DeferredResult`` / ``CompletableFuture`` controllers so the lifetime spans the async window. | | ``PaygResponseBodyWrapper`` | ``HttpServletResponseWrapper`` — in-memory ``ByteArrayOutputStream`` up to 10 MiB; spills to ``TempFile`` above. ``materialisedPath()`` always returns a uniform ``Path`` interface. | | ``PaygChargeInterceptor`` | ``AsyncHandlerInterceptor`` mirroring ``UnifiedCreditInterceptor`` shape. ``preHandle`` gates on ``@AutoJobPostMapping``, materialises multipart inputs, calls ``JobChargeService.openProcess``. ``afterCompletion`` branches on HTTP status. | | ``PaygOutputExtractor`` | Pulls PDFs out of the response body. Direct ``application/pdf`` returns body verbatim; ``application/zip`` iterates entries and keeps each ``.pdf`` entry whose first bytes match the ``%PDF-`` magic. | | ``PaygWebMvcConfig`` | Registers filter at end of Spring filter chain (after security); interceptor after ``UnifiedCreditInterceptor``. | | ``PaygFilterProperties`` | ``payg.filter.enabled`` master switch + in-memory threshold + optional max-bytes ceiling. | ## Status branching in afterCompletion | HTTP status | Action | |---|---| | **2xx** | Append OK step; extract PDFs from response; ``JobService.recordOutput`` per PDF | | **4xx** | Append FAILED step with ``errorCode``. No refund — customer paid for the attempt. No OUTPUT recording. | | **5xx + OPENED** (first-step) | ``JobChargeService.markFirstStepFailed`` → shadow row flipped to ``REFUNDED``, process CLOSED. Refund counter incremented. | | **5xx + JOINED** (mid-chain) | ``JobChargeService.decrementStepCount`` — step slot returned without resetting ``lastStepAt`` (workflow window stays active for retry). | ## New ``JobChargeService`` methods - **``markFirstStepFailed(jobId, reason)``** — flips shadow row to ``REFUNDED`` with ``refundedAt`` + ``refundReason``, closes the process. Idempotent. Mimics the eventual Stripe ``meter_event_adjustment(cancel)`` flow that real-mode will invoke at the same callsite. **Refund implies close** so a same-input retry can't lineage-join into a refunded chain for free work. - **``decrementStepCount(jobId)``** — defensive floor at 1; never drives count negative. ## Schema - Backend: ``V13__payg_shadow_charge_status.sql`` adds ``status`` (``CHARGED`` | ``REFUNDED``) + ``refunded_at`` + ``refund_reason``. ``DEFAULT 'CHARGED'`` so existing B-1 rows stay correct without backfill. - Supabase: matching migration in [Stirling-PDF-SaaS#payg-shadow-charge-status](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/tree/payg-shadow-charge-status) ## Fail-open semantics in shadow Any ``RuntimeException`` in ``preHandle`` / ``afterCompletion`` is logged at WARN, increments ``payg.filter.errors``, and lets the customer's tool call proceed unbilled. This **reverses to fail-closed** when ``wallet_policy.engine = PAYG`` (real charging) — that reversal lives inside ``JobChargeService`` and ships with the cap evaluator PR (PR-C1 in PAYG_DESIGN.md). ## Observability Micrometer metrics: - ``payg.filter.errors`` Counter — internal failures (preHandle + afterCompletion). Alert source. - ``payg.filter.calls`` Counter, tagged ``disposition`` (``OPENED`` | ``JOINED`` | ``SHORT_CIRCUIT``) - ``payg.filter.refunds`` Counter — first-step 5xx refunds - ``payg.filter.duration`` Timer — preHandle + afterCompletion wall-clock per request ## Test coverage (38 tests across 4 classes) - **PaygResponseBodyWrapperTest** (12 tests) — in-memory, spill, threshold crossing mid-chunk, writer vs outputStream exclusivity, ``resetBuffer`` with and without spill, close idempotency, single-byte writes across threshold. - **PaygOutputExtractorTest** (7 tests) — direct PDF, parametrised content type, ZIP with mixed entries + magic-byte gate, corrupt ZIP fail-open, empty ZIP. - **PaygChargeInterceptorTest** (13 tests) — all preHandle short-circuits, OPENED disposition stash, fail-open on chargeService exception, 2xx recordOutputs path, 5xx OPENED → markFirstStepFailed, 5xx JOINED → decrementStepCount, 4xx FAILED step append, max-bytes ceiling skip, PIPELINE header detection. - **JobChargeServiceTest extended** (+6 tests) — markFirstStepFailed happy path, idempotency, missing-shadow-row case, long-reason trim; decrementStepCount happy path, floor-at-1 defence, missing-job no-op. ## What's NOT in this PR (deliberate) - **No SpringBootTest layer.** The saas module doesn't have bootstrap test infrastructure (Supabase JWT config + H2 schema harness). Integration confidence comes from the manual staging deploy + SQL-flip of an internal team. Bootstrap-test infra is a focused follow-up if needed. - **No saas-mode Behave / docker-compose.** Per design §17 — deferred. Existing ``testing/cucumber/`` infrastructure doesn't yet have a saas-profile compose target; that's its own PR when warranted. - **No CreditService wire-in** (per design §13 decision). Per-row comparison data moves to the reconciliation report PR (PR-S2). ``legacy_credits_charged`` + ``diff_pct`` columns stay at 0 in shadow rows. - **No reconciliation report endpoint.** Direct SQL queries against ``payg_shadow_charge`` cover the data-access need until patterns emerge. ## Rollback levers | Symptom | Lever | |---|---| | Some / all tool calls breaking due to filter | ``payg.filter.enabled=false`` + restart (~20s) | | Shadow rows look wrong for a specific team | ``UPDATE wallet_policy SET engine = 'LEGACY' WHERE team_id = ?`` | | Mass shadow weirdness | ``UPDATE wallet_policy SET engine = 'LEGACY'`` | | Memory exhaustion from response tee | Lower ``payg.filter.response.in-memory-threshold-bytes`` | ## Test plan - [ ] CI green (build + tests) - [ ] Aikido / Snyk / SonarCloud clean - [ ] Manual: deploy to staging - [ ] Manual: flip one internal team via ``UPDATE wallet_policy SET engine = 'PAYG_SHADOW' WHERE team_id = ?`` - [ ] Manual: hit ``/api/v1/security/add-password`` with that team's JWT; verify a ``payg_shadow_charge`` row appears with ``status='CHARGED'`` - [ ] Manual: trigger a 503 (e.g. via temporary backend kill mid-request); verify the resulting row is ``status='REFUNDED'`` + the process is ``CLOSED`` - [ ] Manual: hit ``/api/v1/general/split`` with a multi-page PDF; verify one OUTPUT signature per inner PDF appears in ``job_artifact_hash`` - [ ] Manual: chain ``add-password`` → ``compress`` on the output; verify the second call JOINS the first process (no new shadow row) and the inner output OUTPUT signature is what drove the lineage join ## Stacks on / references - Stacks on: #6477 (B-1 — shadow charging engine) - Schema mirror: Stirling-PDF-SaaS#payg-shadow-charge-status branch - Design doc: ``notes/PAYG_FILTER_DESIGN.md`` (all 19 decisions DECIDED) |
||
|
|
3807cdfbc6 |
PAYG: process tracking + shadow charging engine (PR B-1) (#6477)
> 📌 **Stacked on [#6464](https://github.com/Stirling-Tools/Stirling-PDF/pull/6464)** (lineage primitives, still in review). #6469 has merged so its commits are no longer in this PR's diff. Once #6464 merges, a final rebase collapses the lineage-primitives commits out of this diff too — leaving only the B-1 work. ## What this is Process tracking + shadow charging engine. Bundles PR-I7 service half with the non-filter piece of PR-I7a so the pieces ship together — none of them is useful in isolation. **Review focus:** the new files in: - \`app/saas/src/main/java/stirling/software/saas/payg/job/\` (\`JobService\`, \`JobContext\`, \`JoinOrOpenResult\`, \`StaleJobCloser\`) - \`app/saas/src/main/java/stirling/software/saas/payg/charge/\` (\`JobChargeService\`, \`ChargeContext\`, \`ChargeOutcome\`, \`JobInput\`) - \`app/saas/src/main/java/stirling/software/saas/payg/lineage/LineagePruneScheduler.java\` - their tests The 8 files inherited from #6464 (lineage primitives) are unchanged from there — they ride along in this diff until #6464 lands. The remaining work for shadow-in-staging is the ingress/egress filter that wires controllers into this engine — that's PR B-2. ## Scope ### \`JobService\` — persistence + lineage policy - **\`joinOrOpen\`** — the multi-input "any-match-joins, newest wins" rule. Hash every input via the lineage detector; if any matches an open process in the workflow window, attach to the one with the freshest \`lastStepAt\`. Step-limit overflow on the matched job spawns a fresh process; the new job's input signatures are still recorded so \`mostRecentMatchWins\` routes future calls forward. - **\`recordOutput\`** — post-tool-success path. Records OUTPUT signatures so the next call that takes this file as input lineage-matches into the same process. - **\`appendStep\`** — audit-trail step row written after a tool completes. - **\`close\`** — idempotent; safe to call from multiple paths (explicit, FE on-unload, scheduler). Returns the same row on re-close, no state mutation. - **\`findStale\` / \`closeStale\`** — workflow-window-based stale closure used by the scheduler. ### \`JobChargeService\` — the orchestrator (shadow variant) \`openProcess\` resolves the effective policy via \`PricingPolicyService\` (now in main via #6469), derives the step-limit for the current \`JobSource\` (with a defensive fallback if the policy is missing an entry), delegates to \`JobService.joinOrOpen\`, and on OPENED runs the \`DocumentClassifier\` + writes a \`payg_shadow_charge\` row. Applies the policy-level \`minChargeUnits\` floor per design § 3.4. Shadow variant only — never debits the ledger, never posts a Stripe meter event. The real-charging follow-up reuses the same orchestration and swaps the side-effect. \`legacyCreditsCharged\` on the shadow row stays \`0\` until the legacy \`CreditService\` is wired in (PR B-2), where the comparison becomes meaningful. ### Schedulers (both plain \`@Scheduled\`) - **\`StaleJobCloser\`** — fixed-rate 60 s. Closes \`OPEN\` jobs idle past the workflow window. API users never have to call close explicitly — this is the safety net. - **\`LineagePruneScheduler\`** — hourly cron, retention 1 h. Deletes \`job_artifact_hash\` rows older than the retention window. - **No \`@SchedulerLock\` / no \`shedlock\` table** — consistent with the 5 existing unguarded \`@Scheduled\` tasks in \`:saas\` (\`CreditResetScheduler\` and friends, none of which are guarded today). Cluster-correctness across all 7 saas schedulers is tracked in design § 9 as a separate focused cleanup. Underlying operations are idempotent — duplicate firings on multi-pod would be wasted DB load, not data corruption. ### Records (call-shape glue for PR B-2's filter) - \`JobContext\` / \`JoinOrOpenResult\` — input/output for \`JobService\`. - \`ChargeContext\` / \`ChargeOutcome\` — input/output for \`JobChargeService\`. - \`JobInput\` — paired \`(MultipartFile, materialised Path)\` so the upcoming ingress filter can pass both views without re-materialising. ## Tests **26 new, all green.** - 14 × \`JobServiceTest\` — no-match → opens new, single-match → joins existing, multi-input any-match-joins, multi-match newest-wins (older job never even looked up), step-limit hit spawns fresh job (and original \`stepCount\` is NOT mutated), empty inputs reject, stale-signature handling, recordOutput delegation, close idempotency, closeStale, appendStep persistence. - 7 × \`JobChargeServiceTest\` — JOINED skips classifier + shadow write entirely, OPENED writes shadow row + classifies (single + multi file paths), \`minChargeUnits\` floor applied, step-limit resolved per-\`JobSource\` from policy, missing source entry falls back to conservative default of 10. - 2 × \`StaleJobCloserTest\`, 3 × \`LineagePruneSchedulerTest\` — scheduler-wiring smoke + constructor-validation tests. \`ENABLE_SAAS=true ./gradlew :saas:test\` — BUILD SUCCESSFUL. ## What's not in this PR (lands in PR B-2) - **Tool ingress/egress servlet filter.** The highest-risk piece — materialises the request body into a \`JobInput\`, calls \`JobChargeService.openProcess\` from every \`@AutoJobPostMapping\`, records OUTPUT after success. Edge cases to validate: multipart parts, async controllers, streaming responses, errored 5xx paths, very large files. Design decisions for the filter are being worked through in \`notes/PAYG_FILTER_DESIGN.md\` before any code is written. - **Wire shadow path into legacy \`CreditService\`.** Every legacy debit writes a comparison row carrying both the PAYG would-be units and the legacy actual credits, populating \`diffPct\`. ## Design doc \`notes/PAYG_DESIGN.md\` — PR-I7 + PR-I7a status updated to reflect this bundle. § 9 carries the cluster-correctness deferral note alongside the existing LISTEN/NOTIFY trade-off note. § 7.5.1 readiness summary shows the path now needs just **1 more PR** (the filter half + CreditService wire-in, both bundled into PR B-2). |
||
|
|
35a712a278 |
smart redaction (#6195)
Co-authored-by: James Brunton <[email protected]> |
||
|
|
7f3ca7ea70 |
Fix mockServiceWorker.js reformatting (#6526)
# Description of Changes `mockServiceWorker.js` is a third-party managed file, which is included in our `.prettierignore` file, and is rewritten to be in the module's standard format whenever `msw` runs. At some point, it was reformatted in our style, but shouldn't have been. This puts it back to `msw`'s style, which should make it stop appearing in diffs. |
||
|
|
895dcbbafd |
Update Backend 3rd Party Licenses (#6407)
Co-authored-by: Anthony Stirling <[email protected]> Signed-off-by: stirlingbot[bot] <stirlingbot[bot]@users.noreply.github.com> |
||
|
|
b705c5b84c |
Switch to use JPDFium v1.0.2, which signs the Mac binaries (#6521)
# Description of Changes Currently, it's not possible to develop the backend on Mac without manually signing the JPDFium binaries yourself since macOS will reject running the unsigned binaries. [We've now updated JPDFium to sign the Mac binaries in v1.0.2](https://github.com/Stirling-Tools/JPDFium/releases/tag/v1.0.2), so update to use that version. |
||
|
|
1264f4cfed |
Set up document management for Stirling Engine (#6476)
# Description of Changes Change Stirling Engine to support deleting documents automatically. This happens both on user logout and after an amount of time specified by the Java when ingesting a document (allowing for personal documents to have short lifetimes but org documents to be left in the db with no expiry date). Also sets up an [ACL policy](https://en.wikipedia.org/wiki/Access-control_list) for the documents so the database knows which users have access to which documents. This is not fully implemented in the Java, so currently all docs are treated as having a single owner, the uploader, but theoretically when we need to support org storage, we shouldn't need to change the db schema. |
||
|
|
71633861d0 |
Make zoom key command behave the same regardless of mouse position (#6508)
# Description of Changes Make zoom key command behave the same regardless of mouse position. Previously only zoomed the editor if the mouse was over the editor. |
||
|
|
e6974d52f7 |
PAYG: hash-lineage detection primitives (modular extractor / store / detector) (#6464)
## What this is
Three orthogonal interfaces — each with one production impl — for
detecting whether an incoming tool call should join an existing process
via content-hash lineage. Groundwork for PR-I7a: nothing in this PR
calls the detector yet; the ingress/egress filter that wires it into
every controller lands separately.
Built to be modular along three axes. Swapping any of them should not
require changes elsewhere:
| Axis | Interface | V1 impl | Plausible future impl |
|---|---|---|---|
| Hash algorithm | `LineageSignatureExtractor` |
`ByteHashSignatureExtractor` (SHA-256) | `PdfMetadataSignatureExtractor`
(PDF `/ID`, content-stream hash) |
| Storage backend | `JobLineageStore` | `JpaJobLineageStore` |
`RedisJobLineageStore` |
| Matching policy | `HashLineageDetector` | `DefaultHashLineageDetector`
| strategy-driven variant (any-match-joins for multi-input — lives in
JobService) |
## Interfaces
### `LineageSignatureExtractor` — what counts as a fingerprint
```java
public interface LineageSignatureExtractor {
Set<LineageSignature> extract(Path file) throws IOException;
String name();
}
```
File-based (not stream-based) so a future PDF-aware extractor can open
the same file via jpdfium / PDFBox and pull `/ID[0]` or a content-stream
hash. Multiple extractors compose at the detector layer — Spring
auto-wires all `LineageSignatureExtractor` beans, the detector unions
their results.
Production impl: **`ByteHashSignatureExtractor`** — SHA-256 over the
file via 64 KiB-buffered `DigestInputStream`. Hardware-accelerated by
the JVM (Intel SHA-NI, ARM SHA).
### `JobLineageStore` — where signatures live
```java
public interface JobLineageStore {
void record(UUID jobId, Set<LineageSignature> signatures, ArtifactKind kind);
Optional<LineageMatch> findOpenJobForSignatures(Long userId, Set<LineageSignature> candidates, Duration window);
int pruneOlderThan(Instant cutoff);
}
```
Knows nothing about storage technology. Production impl
**`JpaJobLineageStore`** runs a single joined query against
`job_artifact_hash` ⋈ `processing_job` — status + window filtering
happen at the database. The query is bounded by `Limit.of(1)` on the hot
path so a job set sharing a popular signature doesn't materialise
unwanted rows. A future `RedisJobLineageStore` (or write-through hybrid)
is a drop-in.
### `HashLineageDetector` — the high-level API
```java
public interface HashLineageDetector {
Optional<LineageMatch> detect(Long userId, Path inputFile) throws IOException;
void record(UUID jobId, Path file, ArtifactKind kind) throws IOException;
}
```
**`DefaultHashLineageDetector`** delegates extraction to every
registered `LineageSignatureExtractor`, storage to the configured
`JobLineageStore`, and reads `payg.lineage.workflow-window` (default
`PT5M`) from config. When a single extractor throws (e.g. a future
PDF-aware extractor against a malformed PDF), the other extractors still
contribute — failures don't block the byte-hash from landing.
## Profile gating
All three `@Component` beans (`JpaJobLineageStore`,
`ByteHashSignatureExtractor`, `DefaultHashLineageDetector`) are
`@Profile("saas")` — consistent with every other `:saas` bean. Without
this guard the JPA store would fail to wire against its profile-gated
repository in non-saas profiles that pull `:saas` onto the classpath.
## Tests
Run entirely in-memory; no database required.
- **`LineageSignatureTest`** — storage-key encoding round-trips, rejects
malformed `"type:value"` keys.
- **`ByteHashSignatureExtractorTest`** — identical bytes → identical
sigs; empty file hashes to the well-known SHA-256-of-empty constant; 10
MiB file streams without OOM.
- **`DefaultHashLineageDetectorTest`** — same-user / within-window /
status=OPEN filtering, multi-signature matching (one extractor sees
`pdf-id` and matches even when bytes differ), most-recent-job-wins,
record+detect round-trip, extractor-throwing-doesn't-break-others.
**`InMemoryJobLineageStore`** (in test sources) implements the same
`JobLineageStore` interface as the JPA impl, plus a `registerJob` hook
for tests to model job state. Same contract — proves the abstraction is
portable. When the Redis impl lands it gets the same contract tests.
## What's not in this PR (deliberate)
- The tool ingress/egress filter that wires the detector into every
controller — separate, focused review.
- `JobChargeService.openProcess()` — uses the detector, part of the
charging machinery, separate PR.
- Prune scheduler that calls `pruneOlderThan` — small follow-up
alongside the `shedlock` foundational table.
- PDF-aware extractor (`PdfMetadataSignatureExtractor`) — to be added
when we measure how often byte-hash-only misses real workflows.
- Multi-input "any-match-joins" lineage policy — that's a `JobService`
decision (PR-I7), not a primitive.
## Self-review pass applied
An independent code-review on this PR caught:
- **HIGH:** Missing `@Profile("saas")` on the three `@Component` beans →
fixed.
- **MEDIUM:** `pruneOlderThan` missing `@Transactional` (its
`@Modifying` query would have thrown
`InvalidDataAccessApiUsageException`) → fixed.
- **MEDIUM:** `findOpenJobsForSignatures` fetching the whole match set
just to `get(0)` → now takes `Limit`, JPA store passes `Limit.of(1)` on
the hot path.
- **LOW:** `InMemoryJobLineageStore` used both `synchronized` methods
and `ConcurrentHashMap` → dropped the redundant `ConcurrentHashMap`.
Deferred: project-wide UTC unification (`LocalDateTime.now()`
system-zone is the established convention; flipping one file mid-stack
caused a real test failure — proper fix needs its own audit).
## Rollback
Straight `git revert`. No callers yet; deleting these classes wouldn't
break anything.
---
## Checklist
- [x] Tests pass: `ENABLE_SAAS=true ./gradlew :saas:test`
- [x] No new warnings
- [x] Self-review performed (HIGH + MEDIUM findings addressed)
|
||
|
|
256d1a86d2 |
UI changes to update and support auto updating (#6075)
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
919f0ade99 |
Portal (#6391)
# Description of Changes ## What & why This PR introduces the **Stirling developer portal** — a new control-plane frontend that sits alongside the existing PDF editor — plus the shared design system and workspace structure needed to host both apps in one frontend. The portal is the parent product surface: where users connect sources, compose pipelines, wire agents, and manage usage / billing / infrastructure, with the PDF editor as one capability inside it. This PR lays the **foundation** — workspace reshape, design system, app shell, navigation, and a mock-driven home — rather than wiring real backends (those surfaces are placeholders for follow-up phases). ## What's in this PR **1. Frontend repo reshape (`frontend/src/` → `frontend/editor/`)** The existing editor app moved under `frontend/editor/`, so `editor`, `portal`, and `shared` are siblings in one workspace. All references were updated accordingly: `LICENSE`, `.dockerignore`, `.gitignore`, build/sign shell scripts, the GH language-check script, the Taskfile, and Docker config. **No editor source logic changed — path references only.** **2. New shared design system (`frontend/shared/`)** - **Design tokens** in `tokens.css` as the single runtime source of truth (light/dark, category accents, gradients). `tokens.ts` now holds only the `Tier` type — the old JS palette mirror was removed (nothing consumed it and it had drifted). - ~30 framework-light **components** (Card, Button, Input, Select, Tabs, Modal, Drawer, Toast, MetricCard, StatusBadge, Skeleton, EmptyState, …) with Storybook stories. - **Typed data catalogues**: `endpoints.ts` (10 verticals / 64 endpoints) and `ops.ts`. **3. New developer portal app (`frontend/portal/`)** - App shell: `Header`, `Sidebar`, `AssistantPanel`, search modal, notifications, tier switcher, theme toggle, MSW toggle. - **Tier-aware** home (free / pay-as-you-go / enterprise): KPI strip, 30-day usage chart, onboarding checklist, quick actions, recent activity, region health, product grid, and a curated **"Popular use cases"** teaser. - **Documents** view hosting the full, tab-filterable endpoint catalogue. - Placeholder views for Sources / Pipelines / Agents / Editor / Infrastructure / Usage & Billing / Developer Docs / Settings (follow-up phases). - **MSW-mocked** API layer: `api/*` issues real `fetch`, intercepted by mocks in dev/Storybook; pointing at a real backend is just a matter of not registering MSW. `react-router` URLs; Tier / View / UI contexts. **4. Tooling & guardrails** - ESLint extended to `portal` + `shared`, with **layering-boundary rules**: `shared/` may depend only on third-party packages and itself (no `@app` / `@portal` / `@core` / `@proprietary` / Tauri), so it stays cleanly extractable into a standalone package later. - `dpdm` circular-dependency check now walks editor + portal + shared (the old glob matched only 2 files). - New **devDependencies only** — Storybook (+ a11y/docs/themes addons), MSW. No runtime dependencies added. - New tasks: `frontend:dev:portal`, `frontend:build:portal`. ## Testing done locally - `tsc` for both `portal` and `shared` projects — clean - `eslint --max-warnings=0` across the whole frontend — clean - `dpdm` circular-dependency check — no cycles - Editor builds clean: `vite build editor --mode core` (✓ built, only the pre-existing >500 kB chunk-size advisory) - Editor runs in dev (core mode) with **zero console errors**; portal runs in dev across all three tiers ## Notes for reviewers - The change is overwhelmingly **additive**: `shared/` and `portal/` are brand-new; the existing editor is path-reference changes only. - The portal is intentionally **mock-driven** at this stage — real backends and the remaining views land in follow-up phases. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [x] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. --------- Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
|
b355ccec9e | Add Valkey cluster backplane and sticky-410 ownership (clusters) (#6472) | ||
|
|
de9d6ad3f5 | Add CI coverage summaries and aggregate JaCoCo report (#6451) | ||
|
|
2c0ebc28a7 | perf(api): optimize static asset caching, enable ETag support, and expand response compression mime types. (#6273) | ||
|
|
d1486c7762 |
ci(github-actions): replace deprecated app-id input with client-id (#6485)
# Description of Changes This change updates the GitHub App token generation step in the custom `setup-bot` GitHub Action to use the new `client-id` input instead of the deprecated `app-id` input when invoking `actions/create-github-app-token`. ### What was changed - Replaced the deprecated `app-id` parameter with `client-id` in `.github/actions/setup-bot/action.yml`. - Continued sourcing the value from the existing `inputs.app-id` action input to avoid broader interface changes. ### Why the change was made - `actions/create-github-app-token` has deprecated the `app-id` input and now expects `client-id`. - Updating the workflow removes the deprecation warning and ensures compatibility with current and future versions of the action. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
|
2ccff6f73f | fix(update-service): correct GitHub branch reference for version retrieval (#6333) | ||
|
|
78da227eba | fix: Use frontend/editor for locales paths (#6483) | ||
|
|
30e782e29c | Disable Save-to-server when storage off, fix QR port 0 (#6473) | ||
|
|
2b0905887b | Add desktop multi-window support (#6463) | ||
|
|
28b81828b5 |
PAYG: PricingPolicyService + admin REST + 30s read cache (#6469)
## What this is PR-I1 service half from `notes/PAYG_DESIGN.md`. Built on top of the data model from #6460 — answers "what pricing policy applies to this team right now?" with a fast cache and an admin write surface. ## Scope | Piece | Where | |---|---| | `PricingPolicyService` — `getEffectivePolicy(teamId)` with 30s Caffeine cache + admin write paths | `app/saas/.../payg/policy/PricingPolicyService.java` | | `PolicyChangedEvent` — published after admin writes for in-process cache invalidation | `app/saas/.../payg/policy/PolicyChangedEvent.java` | | Admin REST — list / get / create / set-default / set team override / get effective | `app/saas/.../payg/policy/admin/PricingPolicyAdminController.java` + DTOs | | `PricingPolicyRepository.clearDefaultFlag()` — atomic clear for set-default | repository update | | `SaasJpaConfigScanTest` — drift guard against the JPA scan paths going stale (carried over from the #6460 review concern) | new test | | V12 default-policy seed (`v1-initial`, 25 pages/unit, 5 MiB/unit, per-`JobSource` step limits) | `V12__seed_default_payg_policy.sql` | ## Lookup precedence 1. `PaygTeamExtensions.pricingPolicyId` set → return that policy 2. Else return the `pricing_policy` row with `is_default = TRUE` 3. Override row points at a deleted policy → log warn, fall back to default (safety net for racing deletes) 4. No default → `IllegalStateException` (V12 seed guarantees one exists) ## Cache behaviour - 30s `expireAfterWrite` Caffeine, max 10k entries, keyed by `teamId`. - **Single correctness model: the TTL.** Cross-instance propagation is at-most-30-seconds. The writer instance sees its own change immediately via the `PolicyChangedEvent` after-commit publish. Other instances pick it up on the next TTL expiry. - Admin reads use `getEffectivePolicyUncached` so admins always see their own write straight back. **Why no LISTEN/NOTIFY runner.** An earlier cut of this PR included a Postgres `LISTEN policy_changed` runner so cross-instance propagation was instant. Dropped — admin policy changes are events-per-week and the 30s TTL is already the correctness floor; the listener was ~250 lines of nontrivial code (raw JDBC outside HikariCP, daemon thread, reconnect loop, lock-protected connection lifecycle) for a use case that isn't on the hot path. Trade-off is documented in `notes/PAYG_DESIGN.md` §9 with three concrete triggers that would justify reintroducing it (aggressive cap enforcement, Redis landing for other reasons, real-time admin UI). ## Writes — transactional, fire `PolicyChangedEvent` after commit - `create(draft)` — rejects pre-set `policy_id` or `is_default=true` (promotion must go through `setDefault` so the partial unique idx is freed first). - `setDefault(id)` — atomically clears the existing default via `clearDefaultFlag()` then flips the new row. Idempotent: silent no-op if the row is already default. - `setTeamOverride(teamId, policyId | null)` — validates the policy exists before save; `null` clears the override. `publishOnCommit` uses `TransactionSynchronizationManager.afterCommit` so listeners never see pre-commit state. Outside a transaction (test paths) falls through to immediate publish. ## Admin REST surface — `/api/v1/admin/payg/...` All endpoints `@PreAuthorize("hasRole('ADMIN')")`: - `GET /policies` — list all - `GET /policies/{id}` — read one - `POST /policies` — create new (non-default) - `POST /policies/{id}/set-default` — atomic promote - `PUT /teams/{teamId}/policy-override` — set or clear per-team override - `GET /teams/{teamId}/effective-policy` — cache-bypassing live read Validation errors → 400, unknown rows → 404. ## Counterpart Supabase PR [`Stirling-PDF-SaaS#298`](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/298) — seeds the same V1 default policy on the Supabase side via `20260528000002_payg_seed_default_policy.sql`. ## Tests - 17 × `PricingPolicyServiceTest` — lookup precedence, cache hit/miss, invalidation on event, mutation paths publishing event, error cases. - 14 × `PricingPolicyAdminControllerTest` — every endpoint's happy path + error mapping, DTO defensive-copy invariant. - 2 × `SaasJpaConfigScanTest` — reflection-based guard that `payg.repository` is in `@EnableJpaRepositories` and `payg` is in `@EntityScan`. Without this, new sub-packages can silently fail to wire at runtime — same class of bug that the #6460 review caught. Full `:saas:test` BUILD SUCCESSFUL. ## Design doc `notes/PAYG_DESIGN.md` §7.4 PR-I1 — completes the service half (the schema half landed in #6460). §9 carries the 30s-TTL trade-off note. |
||
|
|
2c01f41142 |
Update indexeddb to v9 to unify SaaS and OSS users (#6474)
# Description of Changes The production SaaS is currently on v8 of IndexedDB due to various schema changes for Smart Folders, which haven't made their way into OSS. OSS is currently on v4 of IndexedDB, so if we release an OSS build to the SaaS deployment, existing users will not be able to use it because the DB version is 'too old'. This PR updates the IDB version number to v9 so both OSS and SaaS users will be able to upgrade to it. Theoretically both types of user should be able to keep their IDB files without issue. SaaS previously actively wiped the user's files in an old version (v6/v7) and users who haven't used it since then will have their DBs wiped, but that'd happen anyway if they use current SaaS so I don't think that matters. |
||
|
|
4d5eeb103f |
Fix username display issues (#6471)
# Description of Changes Main fixes: - Fix the display of the username in the bottom left - Now displays as "User" when not logged in on self-hosted (desktop) and "Guest" on SaaS when logged in anonymously - Now updates properly when the user logs in/out in SaaS, desktop and self-hosted - Fix incremental build issues in the desktop app that have been here since the start (I hope at least - I think the issue is that the JLink is built read-only and then on subsequent builds you get OS errors when trying to override the JLink with the new version. There's no real need for it to be read-only that I know of, so we might as well just make it R/W and ship like that) |
||
|
|
83ea07ed6a |
saas: DocumentClassifier + PAYG data model (#6460)
# Description of Changes Two layers — the `DocumentClassifier` utility plus the full data model for the new billing engine. Nothing wires the entities into application behaviour yet; services and controllers land in follow-up PRs. **Companion PR:** [Stirling-PDF-SaaS#296](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/296) — Supabase migration for the v3 dev branch, schema-equivalent to the Flyway migration in this PR. ## 1. DocumentClassifier (under `payg.docs`) `DocumentClassifier` computes the doc-unit cost of an uploaded file (or multi-file input) under a `PricingPolicy`. PDFs read page count via `stirling.software.jpdfium.PdfDocument`; non-PDFs are bytes-only. Formula: `max(ceil(pages / docPagesPerUnit), ceil(bytes / docBytesPerUnit))` clamped to `[1, fileUnitCap]`. Multi-file is the sum of raw per-file units capped at `fileUnitCap × file_count`. Two floors, by design: the classifier returns `docUnits` with an absolute `1` floor for non-empty input; the policy-level `minChargeUnits` is intentionally applied later, at process-open time in `JobChargeService`, per design § 3.4 (`unitsForProcess = max(policy.min_charge_units, docUnits)`). Documented in the interface + impl javadoc. Upload bytes are materialised through `TempFileManager.createManagedTempFile` so jpdfium gets a `Path`; the temp file auto-deletes on close. Twelve tests, all in-memory fixtures generated with PDFBox at test time — no committed binary blobs. ## 2. PAYG data model (under `payg.*`) JPA entities, repositories, and a Flyway migration covering the full schema in §6 of the design. **Enums** (`payg.model`): `JobSource`, `ProcessType`, `JobStatus`, `JobStepStatus`, `ArtifactKind`, `LedgerEntryType`, `LedgerBucket`, `ReferenceType`, `EntitlementState`, `FeatureSet`, `FeatureGate`, `WalletEngine`, `CapPeriod`, `AutoGroupStrategy`. **Entities + repositories:** | Entity | Table | Notes | |---|---|---| | `PricingPolicy` | `pricing_policy` | Promoted from a record. `stepLimits` is `Map<JobSource, Integer>` persisted via normalised child table `pricing_policy_step_limit`. `stripePriceIds` is `Set<String>` persisted via `pricing_policy_stripe_price` — currency comes from `stripe.prices` via Sync Engine, not stored locally. | | `ProcessingJob` | `processing_job` | UUID PK. Tracks lineage window via `step_count` and `last_step_at`. | | `ProcessingJobStep` | `processing_job_step` | Per-tool-call audit. | | `JobArtifactHash` | `job_artifact_hash` | Composite key `(job_id, content_hash, kind)`. `content_hash VARCHAR(128)` so multiple signature schemes coexist as `"type:value"` storage keys. Lineage detector queries this. | | `WalletLedgerEntry` | `wallet_ledger` | Append-only, signed `amount_units`. Two unique indexes kill double-posting. | | `WalletPolicy` | `wallet_policy` | Per-team engine + cap + degradation rules + lineage strategy. No `@Version` — admin-only writes (documented in javadoc). | | `WalletEntitlementSnapshot` | `wallet_entitlement_snapshot` | Composite key `(team_id, user_id)`; `user_id = 0` is the team-wide sentinel. No `@Version` — full-row recompute via `EntitlementService.recompute` (documented in javadoc). | | `PaygShadowCharge` | `payg_shadow_charge` | Per-job diff while in `PAYG_SHADOW` engine mode. | | `PaygTeamExtensions` | `payg_team_extensions` | Sidecar 1:1 with `teams` carrying `pricing_policy_id` (per-team override) + `stripe_customer_id`. Sidecar pattern (mirrors `saas_team_extensions`) so OSS Hibernate ddl-auto never sees PAYG columns on `teams`. | **Column adds:** - `team_memberships.cap_units` (optional per-member sub-cap) **Width split (intentional, documented in V11):** per-row deltas (`wallet_ledger.amount_units`, `processing_job.charged_units`) are `INTEGER` because no single charge realistically approaches 2B units. Cap and period-rollup columns (`team_memberships.cap_units`, `wallet_policy.cap_units`, `wallet_entitlement_snapshot.period_spend_units / period_cap_units`) are `BIGINT` because they accumulate across a billing period and admins may legitimately set headroom-cap values into the millions. **JPA wiring:** `SaasJpaConfig` was updated to include `stirling.software.saas.payg.repository` in `@EnableJpaRepositories.basePackages` and `stirling.software.saas.payg` in `@EntityScan` (covers `payg.policy` / `payg.job` / `payg.wallet` / `payg.entitlement` / `payg.shadow` recursively). New `SaasJpaConfigScanTest` reads the annotations reflectively and asserts every expected package is wired — catches the next time someone adds a new sub-package without updating the scan paths. **Migration:** `V11__saas_payg_model.sql` (purely additive). Schema-equivalent to the Supabase migration in the companion PR — including the `VARCHAR(128) content_hash` width that's needed for the multi-signature-scheme storage encoding the lineage layer uses. ## 3. Smoke tests `PaygEntitiesSmokeTest` exercises each entity via the no-arg ctor JPA requires, plus getter/setter round-trips and composite-key equality — catches Lombok/annotation regressions without needing a database. Real-DB integration coverage lands alongside the services that consume each entity. ## Why this is safe to land now - All schema changes are additive — no existing rows modified, no columns dropped. - The entities are not yet referenced from any production code path; they exist for the next PRs to build on. - The v3 Supabase dev branch picks up the schema via the companion PR; the main repo's Flyway migration applies the same shape when an instance boots against a freshly-migrated v3 database. ## Open decisions made - **Step-limits keyed by `JobSource`** rather than by `ProcessType`. Captures the "self-hosted gets a different knob" framing in earlier feedback. Trivially overridable per pricing policy version. - **Step limits + Stripe price IDs normalised into child tables** rather than JSONB on `pricing_policy` (per Connor's review on #296). Typed columns, queryable directly, no JSON parsing. - **Currency dropped from `pricing_policy_stripe_price`** — it lives on `stripe.prices.currency` and is resolved via Sync Engine. App is currency-blind. ## Rollback Straight `git revert` on this PR. The Supabase migration in #296 is additive and can be left in place safely — the running app ignores tables it doesn't reference. --- ## Checklist - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings - [x] I have run `task check` (via `./gradlew :saas:test` with `ENABLE_SAAS=true`) — passes |
||
|
|
61ebe977d3 |
Auto-delete CI linting comments on success (#6465)
# Description of Changes Set CI backend & engine comments to auto-delete once the CI has passed. Also redesign the engine CI to call `task engine:check` like it should have been, and make it post a comment when the tool models need to be updated. Also makes the comment wording more consistent between the three languages. |
||
|
|
763595a5a3 |
feat: add Agents UI to proprietary right sidebar (#6454)
Update UI to include agents Run `task dev:all` to test |
||
|
|
398617391b | Fix SSO auto-login and custom metadata settings not persisting on restart (#6468) | ||
|
|
a0e0e88f07 |
saas: harden CreditService Stripe ordering + lint @AutoJobPostMapping weights (#6458)
# Description of Changes Two narrowly-scoped hardening changes to the credits engine. ## 1. CreditService — move Stripe meter call to `afterCommit` The Stripe metered-usage call sits inside the surrounding `@Transactional`, holding the `user_credits` row lock for the duration of an HTTP round-trip to Supabase. Under load this starves concurrent debits; a transient Stripe blip rolls back a (correct) free-credit consumption and forces the caller to retry. The Stripe call now runs in a `TransactionSynchronization.afterCommit` hook — DB commits first, Stripe fires immediately after. If Stripe fails after commit, we log + increment a new `credits.stripe_report.failures` counter; the idempotency key is stable, so a manual replay recovers without double-charging. Applied to both `consumeCreditBySupabaseId` and `consumeCreditWithWaterfall`. **Dead-code removed:** - Unreachable UUID fallback for MDC `requestId` — `CorrelationIdFilter` already guarantees the key on every request. - The `"Unable to report usage to Stripe"` `RuntimeException` and its catch block — the afterCommit refactor eliminates the throw path. - `StripeRollbackOnFailureTest` — pinned the rollback-on-Stripe-fail behaviour this refactor replaces. ## 2. `@AutoJobPostMapping` — build-time lint for `resourceWeight` `UnifiedCreditInterceptor` multiplies `resourceWeight` into the per-call charge. An endpoint that falls through to the annotation default produces a charge derived from a value nobody chose. - Annotation default flipped from `1` to `Integer.MIN_VALUE` (sentinel). Both runtime readers (`UnifiedCreditInterceptor`, `AutoJobAspect`) already clamp into `[1, 100]` so behaviour is unchanged. - New `AutoJobPostMappingWeightTest` scans the classpath and fails the build if any method leaves the sentinel. - Initial run caught 11 endpoints relying on the default. Explicit weights now declared, chosen by comparing to peer endpoints: - `EditTextController` — LARGE - `EmailController#sendEmailWithAttachment` — SMALL - `ConvertPDFToMarkdown` — MEDIUM - `AttachmentController` (extract/list/rename/delete) — SMALL × 4 - `ConvertImgPDFController` (cbr/cbz ↔ pdf) — MEDIUM × 2, LARGE × 2 ## Tests - `StripeUsageIdempotencyKeyTest` — pins the `(supabaseId, overage, requestId)` idempotency key shape so Stripe always dedupes a retry. - `StripeAfterCommitOrderingTest` — pins that `afterCommit` fires after commit and NOT on rollback. - `AutoJobPostMappingWeightTest` — the lint itself, plus a self-check that the classpath scan finds at least 10 `@AutoJobPostMapping` methods (guards against the lint passing vacuously). Build verified: `ENABLE_SAAS=true ./gradlew :stirling-pdf:test :saas:test`. --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) — no translation changes - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) — internal-billing change, no public docs impact - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) — N/A ### Translations (if applicable) - [ ] Not applicable ### UI Changes (if applicable) - [ ] Not applicable ### Testing (if applicable) - [x] I have run `task check` (via `./gradlew :stirling-pdf:test :saas:test` with `ENABLE_SAAS=true`) — passes - [x] I have tested my changes locally |
||
|
|
c80a5db5f5 | folder and file fixes (#6461) | ||
|
|
4fa67afc3d |
Fix Tauri artifact copy path so installers upload (smoke + release) (#6466)
## Summary Regression from #6404 (Restructure/frontend editor). Two CI workflows copy the built installers to the wrong directory, so installer artifacts (MSI / DMG / DEB / RPM / AppImage) silently vanish: - **`tauri-build.yml`** (PR/desktop smoke builds) - uploads zero installer artifacts. - **`multiOSReleases.yml`** (production releases) - the empty artifacts are downloaded by `create-release` and fed to `action-gh-release`, so a release would publish **only the JARs, no desktop installers**. ## Root cause #6404 moved the Tauri project from `frontend/` to `frontend/editor/` and updated every **absolute** path (`projectPath`, `cd`, `Get-ChildItem`) to add the `editor/` segment - but left the **relative** copy targets `../../../dist`. Those resolve against the (now one level deeper) working dir after `cd ./frontend/editor/src-tauri/target`: | | resolves to | |---|---| | before #6404 (`frontend/src-tauri/target`) | repo-root `dist/` ✅ | | after #6404 (`frontend/editor/src-tauri/target`) | `frontend/dist/` ❌ (missing) | The `cp` fails, repo-root `dist/` (from `mkdir -p ./dist`) stays empty, and the upload finds nothing. `find -exec cp` failing is non-fatal, so jobs still report success - that's why it went unnoticed. No release has shipped broken yet: the last release (v2.11.0, 2026-05-19) predates #6404 (2026-05-22). ## Fix Copy to an absolute `$GITHUB_WORKSPACE/dist` in both workflows so the `cd` can't drift the destination again. This matches where the upload / signature-verify steps already read from. ## Evidence (run 26574078559, all 3 OS legs) ``` cp: cannot create regular file '../../../dist/Stirling-PDF-windows-x86_64.msi': No such file or directory ##[warning]No files were found with the provided path: ./dist/*. No artifacts will be uploaded. ``` The Tauri builds themselves succeeded - only the copy/upload was broken. ## Test plan - [ ] `tauri-build` on this PR uploads non-empty `Stirling-PDF-<name>` artifacts on Windows/macOS/Linux. - [ ] Next release (or a `workflow_dispatch` of multiOSReleases) attaches MSI/DMG/DEB/RPM/AppImage to the release. |
||
|
|
8bd78d2624 |
Add landscape page size options (#6248)
# Description of Changes Adds orientation (portrait/landscape) to the Adjust Page Scale tool. - Orientation as a separate parameter (per review), sent through to the backend - ScalePagesController simplified; PDFWithPageSize gains the orientation field - Regenerated tool_models.py; frontend + backend tests added --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. --------- Co-authored-by: ConnorYoh <[email protected]> |
||
|
|
b3c4b8b463 | Add S3 storage and cluster artifact backend (#6457) | ||
|
|
57af5b9dc2 |
Fix Tauri testing (#6462)
# Description of Changes #6402 introduced a Rust test `refresh_token_fallback.rs`, but it wasn't moved properly after the restructure of the `frontend/` folder in #6404. This PR moves the file to the right place, and also hooks up Task and CI rules for `cargo test` since nothing was actually running the test in the first place. |
||
|
|
44fbf8c587 |
Various bug fixes found while testing SaaS build (#6459)
# Description of Changes Various fixes and improvements I made while testing the SaaS code: - Changes the new `.env.saas` file to live in `app/` and match the semantics of the other `.env` files - Adds top-level `task dev:saas` command to spawn SaaS frontend & backend - Deletes dead SaaS code and improves some overriding logic - Fixes refreshing issue when coming back to the tab - Fix the Compare tool's selection logic - Make Compare handle error cases properly - Fixes the location of the "Dismiss All Errors" button (was rendering on top of the top-bar with a transparent background previously so it looked rubbish) - Fixes file selection in PDF Editor |
||
|
|
76840d8a57 | Add CI DB migration smoke test against v2.0/v2.5/v2.10 updates (#6453) | ||
|
|
d459ded168 |
Add cancel button to kill long-running AI tasks (#6351)
# Description of Changes Adds a cancel button to the AI chat to allow the user to abort long-running AI tasks. Just disconnects the SSE stream (all the backend code already interrupts when it notices the stream is dead). |
||
|
|
43b67d213d |
feat(oauth2): opt-in claim-dump diagnostics for OIDC login failures (#6456)
# Description of Changes ## What & why Customers using ADFS (or any generic OIDC provider that doesn't emit `email`) hit `Attribute value for 'email' cannot be null` during OAuth2 login with no visibility into what claims the provider actually sent. The only available remedy was guessing at `security.oauth2.useAsUsername` until something worked. This PR adds a new opt-in `security.oauth2.debugLogging` flag (default `false`). When enabled, `CustomOAuth2UserService` logs: - All ID token claims (sorted, with values) - All UserInfo endpoint claims (if any) - The merged attribute key set Spring exposes to `getAttribute()` - The value the configured `useAsUsername` actually resolved to - A **`Hint:`** line listing the claim keys present in the token that map to a valid `UsernameAttribute` enum value — i.e. exactly what the operator could put in `useAsUsername` to make login work Logged at `INFO` on the success path and `ERROR` on failure (inside the existing `catch (IllegalArgumentException)` block that throws `OAuth2AuthenticationException`). The block is wrapped with a `[OAUTH2 DEBUG] ... [/OAUTH2 DEBUG]` banner and ends with a PII warning so operators don't leave it on in production. Default off → zero observable change for anyone not actively troubleshooting. ## Files changed | File | Why | |---|---| | `app/common/.../ApplicationProperties.java` | New `debugLogging` field on the `OAUTH2` config class with javadoc warning about PII | | `app/core/src/main/resources/settings.yml.template` | Documents `oauth2.debugLogging` so it appears on next startup | | `app/proprietary/.../security/service/CustomOAuth2UserService.java` | Emits the claim dump + suggestion hint when the flag is on | | `app/proprietary/.../security/service/CustomOAuth2UserServiceDebugLoggingTest.java` (new) | Unit test: mocks the OIDC delegate, asserts off-path is silent and on-path emits the dump with the right Hint contents | ## End-to-end verification Ran the bundled `testing/compose/docker-compose-keycloak-oauth.yml` Keycloak realm, configured `security.oauth2.useAsUsername: mail` (Keycloak emits `email`, not `mail`) and `provider: demarest` (matches the original customer bug report). Triggered the OAuth flow at `http://localhost:8080/oauth2/authorization/demarest` and confirmed: - The ERROR-level dump fires with the full 19-claim ID token decoded - `-- Value at 'mail' : <NULL — this is why login fails>` correctly identifies the missing claim - `-- Hint:` correctly suggests `[email, family_name, given_name, preferred_username]` (the four keys present that map to valid `UsernameAttribute` values) - Auth still fails with the original `OAuth2AuthenticationException` — no change to control flow, just added diagnostic logging Unit test (`CustomOAuth2UserServiceDebugLoggingTest`) covers both branches. ## Reviewer notes - **No new public APIs.** The flag is config-only; no servlet endpoints exposed. - **PII is logged when the flag is on.** This is the whole point — operators need to see the claims to fix their config — but it's gated, defaults off, and the dump self-documents with a `WARNING: ... Set security.oauth2.debugLogging=false once troubleshooting is complete.` footer. - **Why log everything, not just sub/email?** Because the operator doesn't know in advance which claim they actually want. ADFS uses `upn` in some configs and `preferred_username` in others; Azure AD uses `oid`; the customer here had neither. Dumping the full set is the only way to make the diagnostic self-service. - **Out of scope for this PR (follow-ups):** - The `UsernameAttribute` enum doesn't include `upn` / `unique_name` (common ADFS claims). If the customer's token only has `upn`, the Hint will be empty even though the operator can see `upn` in the dump. Worth a separate PR to extend the enum. - The known-provider validator in `Provider.java` (rejects e.g. `useAsUsername: mail` for `provider: keycloak` at startup) bypasses our diagnostic for those provider names. ADFS customers using `provider: <name>` fall into the `default` branch so are not affected — but it's a sharp edge worth documenting. --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) — N/A, backend-only change - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] Doc-repo update (if functionality has heavily changed) — diagnostic flag is self-documenting via the `settings.yml.template` comment and the in-log warning; happy to add a doc-repo entry if reviewers want one - [ ] Translation tags — N/A ### UI Changes (if applicable) - [ ] N/A — backend-only ### Testing (if applicable) - [x] Unit test added (`CustomOAuth2UserServiceDebugLoggingTest`) covering on/off paths and Hint correctness - [x] End-to-end verified locally against bundled Keycloak compose with intentionally misconfigured `useAsUsername` - [x] Full `:proprietary:test` suite passes |
||
|
|
d42b779644 | Add server-side folders and files page UI (#6383) | ||
|
|
4564ed5bec | Add cluster backplane abstraction and interfaces (#6449) | ||
|
|
7743720f0a |
chore: fix "Endoints" typo in health route comment (#6446)
`app/common/src/main/java/stirling/software/common/util/RequestUriUtils.java:175` section comment said "Health Endoints" → "Health Endpoints". Comment-only. Co-authored-by: Matt Van Horn <[email protected]> |
||
|
|
620f246ca4 |
test(frontend): add URL-encoded filename parsing case (#6436)
# Description of Changes <!-- Please provide a summary of the changes, including: - What was changed - Why the change was made - Any challenges encountered Closes #(issue_number) --> --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [x] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [x] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [x] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [x] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [x] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [x] I have run `task check` to verify linters, typechecks, and tests pass - [x] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
|
dd24440b73 |
test(frontend): cover MIME precedence in non-PDF type detection (#6438)
# Description of Changes <!-- Please provide a summary of the changes, including: - What was changed - Why the change was made - Any challenges encountered Closes #(issue_number) --> --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [x] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [x] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [x] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [x] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [x] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [x] I have run `task check` to verify linters, typechecks, and tests pass - [x] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
|
c0374266e7 |
ci: harden workflow runners and update checkout action pins (#6445)
# Description of Changes Add runner hardening and housekeeping across workflows. - .github/dependabot.yml: add /docker/unoserver and /engine to Dependabot update paths. - .github/workflows/_runner-pick.yml and .github/workflows/build.yml: add step-security/harden-runner (egress-policy: audit) to audit outbound calls from runners. - .github/workflows/ai-engine.yml, .github/workflows/aur-publish.yml, .github/workflows/package-managers.yml: update actions/checkout usage to the newer v6.0.2 reference. - .github/workflows/package-managers.yml: minor YAML formatting tidy (release types array). These changes improve CI security by auditing egress, update the checkout action to a newer release, and expand Dependabot coverage. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
|
730ed43af9 | Fix login loop (#6402) | ||
|
|
56ff1e5092 |
impl migration to pdfium for split (#6410)
## Summary Migrates `SplitPDFController`, `SplitPdfByChaptersController`, `SplitPdfBySizeController` from PDFBox to JPDFium. `SplitPdfBySectionsController` and `AutoSplitPdfController` are intentionally left on PDFBox (require JPDFium 1.0.2 features that don't exist yet). ## Benchmark (audited on `audit/jpdfium-split`, file `app/core/src/test/java/stirling/software/SPDF/bench/SplitBenchmark.java`) | Workload | PDFBox heap | JPDFium heap | PDFBox wall | JPDFium wall | |---|---|---|---|---| | 100 pp, chunk 10 | +21-26 MB | **+0.02 MB** | 80-106 ms | **25 ms** | | 300 pp, chunk 10 | +59 MB | **+1.0 MB** | 232 ms | **76 ms** | **98-99.9% heap reduction. 3-4.2x faster wall.** ## Hybrid - AcroForm-bearing splits keep PDFBox `FormUtils.pruneOrphanedFormFields` post-pass (FPDF_ImportPagesByIndex drops AcroForm dict). Sub-bench shows +1.0 MB / +27 ms - tightly bounded. - Metadata extraction stays on PDFBox. - `SplitPdfBySectionsController` - JPDFium `PdfPageSplitter` only does 2-up halving, not arbitrary MxN. - `AutoSplitPdfController` - needs PDFRenderer + zxing for QR markers. ## Test plan - [ ] 30/30 unit tests pass with PDFBox `Loader.loadPDF` as the oracle (assert page counts + document totals) - [ ] Existing cucumber feature `split.feature` continues to pass - [ ] AcroForm-bearing PDF round-trips without orphaned widgets (covered by existing FormUtils tests) |
||
|
|
05b80fbe4f |
Working local Saas (#6450)
Env file for setting backend saas and taskfile for running it |
||
|
|
4047d02086 | chore: address restructure PR review feedback (#6423) | ||
|
|
5f78083470 | Fix unresolved Material Symbols icon names in frontend (#6443) | ||
|
|
fc048872d7 |
build(deps): bump idna from 3.12 to 3.15 in /testing/cucumber in the pip group across 1 directory (#6403)
Bumps the pip group with 1 update in the /testing/cucumber directory: [idna](https://github.com/kjd/idna). Updates `idna` from 3.12 to 3.15 <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/kjd/idna/blob/master/HISTORY.md">idna's changelog</a>.</em></p> <blockquote> <h2>3.15 (2026-05-12)</h2> <ul> <li>Enforce DNS-length cap on individual labels early in <code>check_label</code>, short-circuiting contextual-rule processing for oversized input while staying compatible with UTS 46 usage.</li> <li>Tidy core helpers: hoist bidi category sets to module-level frozensets (avoiding per-codepoint list construction), simplify length checks, and reuse the shared <code>_unicode_dots_re</code> from <code>idna.core</code> in the codec module.</li> <li>Use <code>raise ... from err</code> for proper exception chaining and switch internal string formatting to f-strings.</li> <li>Allow <code>flit_core</code> 4.x in the build backend.</li> <li>Expand the ruff lint set (flake8-bugbear, flake8-simplify, pyupgrade, perflint) and apply the surfaced fixes; pin lint CI to Python 3.14.</li> <li>Add Dependabot configuration for GitHub Actions.</li> <li>Convert README and HISTORY from reStructuredText to Markdown.</li> <li>Reference CVE-2026-45409 for the 3.14 advisory in place of the initial GHSA identifier.</li> </ul> <p>Thanks to Felix Yan, Stan Ulbrych, and metsw24-max for contributions to this release.</p> <h2>3.14 (2026-05-10)</h2> <ul> <li>Removed opportunity to process long inputs into quadratic time by rejecting oversize inputs up-front. Closes a bypass of the CVE-2024-3651 mitigation. [CVE-2026-45409]</li> </ul> <p>Thanks to Stan Ulbrych for reporting the issue.</p> <h2>3.13 (2026-04-22)</h2> <ul> <li>Correct classification error for codepoint U+A7F1</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/kjd/idna/commit/af30a092e158181d0b35ac66dfa813788126bdd8"><code>af30a09</code></a> Release 3.15</li> <li><a href="https://github.com/kjd/idna/commit/30314d4628744ca14cf2b5820564e5127a9f86f2"><code>30314d4</code></a> Pre-release 3.15rc0</li> <li><a href="https://github.com/kjd/idna/commit/05d4b219aa9eddc47371fcbd2000f0301016f3e9"><code>05d4b21</code></a> Merge pull request <a href="https://redirect.github.com/kjd/idna/issues/237">#237</a> from kjd/convert-docs-to-markdown</li> <li><a href="https://github.com/kjd/idna/commit/2987fdba1962bbb2358399e0084ba062b98a0bee"><code>2987fdb</code></a> Convert README and HISTORY from reStructuredText to Markdown</li> <li><a href="https://github.com/kjd/idna/commit/59fa8002d514bf4a5ce7b58f67b9ec587d53fa9c"><code>59fa800</code></a> Merge pull request <a href="https://redirect.github.com/kjd/idna/issues/236">#236</a> from kjd/dependabot/github_actions/actions-f3e34333ea</li> <li><a href="https://github.com/kjd/idna/commit/def69834ced5d4b3c50439d8b99c4c856ec19ca2"><code>def6983</code></a> Merge branch 'master' into dependabot/github_actions/actions-f3e34333ea</li> <li><a href="https://github.com/kjd/idna/commit/bbd8004a797185d8c56bb555cd5c88fde05e0631"><code>bbd8004</code></a> Merge pull request <a href="https://redirect.github.com/kjd/idna/issues/234">#234</a> from StanFromIreland/patch-1</li> <li><a href="https://github.com/kjd/idna/commit/edd07c05024344a6ccb517414ccb36683aee99fc"><code>edd07c0</code></a> Bump github/codeql-action from 3.35.2 to 4.35.2 in the actions group</li> <li><a href="https://github.com/kjd/idna/commit/5557db030c11bdec50d62aa5f631d705d33ba123"><code>5557db0</code></a> Merge branch 'master' into patch-1</li> <li><a href="https://github.com/kjd/idna/commit/f11746cf4981d25123ef7830d3ee60f07de8ae3d"><code>f11746c</code></a> Merge pull request <a href="https://redirect.github.com/kjd/idna/issues/235">#235</a> from StanFromIreland/patch-2</li> <li>Additional commits viewable in <a href="https://github.com/kjd/idna/compare/v3.12...v3.15">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/Stirling-Tools/Stirling-PDF/network/alerts). </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Anthony Stirling <[email protected]> |