mirror of
https://github.com/arsvendg/Stirling-PDF.git
synced 2026-07-14 10:34:06 +02:00
ConnorYohandGitHub
ff96a80947
## What this PR is End-to-end cucumber coverage for the PAYG shadow charging engine (the filter + interceptor stack from #6519), wired into CI via a new `docker-compose-tests-saas.yml` workflow that runs only on PAYG-touching PRs. Stacked on #6519. ## Automated scenarios (run by `docker-compose-tests-saas.yml`) See [`testing/cucumber/features/payg/shadow_charges.feature`](../tree/payg-s3-cucumber/testing/cucumber/features/payg/shadow_charges.feature): | Scenario | Validates | |---|---| | First tool call writes a CHARGED row | Filter + interceptor fire end-to-end | | Lineage join — second call on output | `JobService.joinOrOpen` matching; no new shadow row | | 4xx leaves the row CHARGED | "Customer paid for the attempt" semantics | | ZIP-returning tool records per-PDF OUTPUT | `PaygOutputExtractor` unpacks + records signatures | | Multi-file input writes a single shadow row | Multi-input group sizing | | `X-Stirling-Automation` sets PIPELINE source | Header → `JobSource` detection | All 6 run locally via `./testing/test-payg.sh` and will run on CI for any PR that touches `app/saas/**`, the PAYG cucumber features, the saas compose stack, or the workflow itself. ## Manual-only scenarios — documented in design doc, not in this suite Two parts of the shadow engine are deliberately not automated; the engine paths are unit-tested in `PaygChargeInterceptorTest.afterCompletion_5xx_opened_*`, and the manual procedures (which require a temporary throw endpoint or a container restart with a flag flipped) live in [`notes/PAYG_DESIGN.md` §7.5.2 "PAYG cucumber: manual-only scenarios"](../tree/payg-s3-cucumber/notes/PAYG_DESIGN.md). - **5xx first-step failure → REFUNDED + CLOSED.** No reliably-5xx-ing endpoint exists; manual procedure adds a throw endpoint, runs, asserts, removes. - **Kill-switch (`PAYG_FILTER_ENABLED=false`).** Needs a container restart mid-suite; manual procedure tears down, flips env, brings up, asserts zero shadow rows. If either gets a hot-reload path (test-only throw endpoint shipped behind a profile gate, or admin endpoint for the kill switch), automate it in a follow-up and drop the manual procedure. ## CI workflow `.github/workflows/docker-compose-tests-saas.yml` (new) — self-contained, not wired into `build.yml`'s `files-changed` matrix so the saas-cucumber job fails and succeeds independently. Triggers only on PAYG-relevant paths. No JaCoCo coverage in v1 (saas compose doesn't have the coverage override; can add later). ## Test infrastructure (recap) - **`testing/compose/docker-compose-saas.yml`** — Stirling-PDF backend with `STIRLING_FLAVOR=saas` + Postgres holding the `stirling_pdf` schema. Supabase JWT auto-config disabled; API-key auth via `SECURITY_CUSTOMGLOBALAPIKEY` is the live path the cucumber tests exercise. - **`testing/compose/payg/saas-init.sql`** + **`saas-seed.sql`** — schema bootstrap + idempotent seed (team / user / wallet_policy). - **`testing/cucumber/features/payg/shadow_charges.feature`** — the 6 scenarios above. - **`testing/cucumber/features/steps/payg_step_definitions.py`** — step defs using `requests` (HTTP) + `psycopg` (direct DB inspection). Direct DB reads are deliberate — we want to see the filter's side effects, not relay them through another API layer. - **`testing/test-payg.sh`** — companion runner to `testing/test.sh`. Brings up the saas compose, waits for health, seeds, runs behave, tears down. - **`behave.ini`** excludes `features/payg` from the default behave run (the saas-cucumber CI job invokes it explicitly). ## Why a separate harness from `testing/test.sh` The existing `test.sh` covers the proprietary-flavour stack (no PAYG tables, no saas profile). Coupling two CI matrices that fail and succeed independently into one script is asking for trouble. Keep the saas-cucumber job focused on its own concerns; once the harness is mature, the wider team can decide whether to merge them. ## Tracked in `notes/PAYG_DESIGN.md` §7.5 (PR-S3) + §7.5.2 (manual scenarios).
106 lines
6.3 KiB
Gherkin
106 lines
6.3 KiB
Gherkin
Feature: PAYG shadow-mode charging
|
|
# End-to-end coverage for the PAYG shadow charging engine via the filter +
|
|
# interceptor stack landed in PR #6519. Runs against the saas-profile
|
|
# docker-compose target (testing/compose/docker-compose-saas.yml).
|
|
#
|
|
# Each scenario sets up a test team in PAYG_SHADOW mode, hits a real tool
|
|
# endpoint, then asserts the shape of the rows written to payg_shadow_charge
|
|
# and processing_job / processing_job_step.
|
|
#
|
|
# Lives under features/payg so it's only loaded when the saas-cucumber job
|
|
# explicitly includes this directory (separate from the main behave run
|
|
# which boots the proprietary-flavor stack and has no PAYG tables).
|
|
|
|
Background:
|
|
Given the SaaS stack is running with PAYG enabled
|
|
And team "payg-cucumber-team" exists with wallet_policy.engine = "PAYG_SHADOW"
|
|
And I am authenticated as a member of team "payg-cucumber-team"
|
|
|
|
Scenario: First tool call writes a CHARGED shadow row
|
|
Given there are no existing shadow charges for team "payg-cucumber-team"
|
|
When I POST a single-page PDF to "/api/v1/security/add-password"
|
|
Then the response status is 200
|
|
And exactly 1 shadow charge row exists for team "payg-cucumber-team"
|
|
And the latest shadow charge row has status "CHARGED"
|
|
And the latest shadow charge row has payg_units >= 1
|
|
And the latest shadow charge row's job is OPEN
|
|
And the latest job has 1 step recorded with status "OK"
|
|
|
|
Scenario: Lineage join — second call on the same output joins the first process
|
|
Given there are no existing shadow charges for team "payg-cucumber-team"
|
|
When I POST a single-page PDF to "/api/v1/security/add-password"
|
|
And I take the response body as "step1-output"
|
|
And I POST "step1-output" to "/api/v1/security/sanitize-pdf"
|
|
Then exactly 1 shadow charge row exists for team "payg-cucumber-team"
|
|
# The second call joined the first process — no new shadow row
|
|
And the latest job has step_count = 2
|
|
And the latest job is OPEN
|
|
|
|
Scenario: First-step 5xx refunds the shadow row + closes the process
|
|
# No reliably-5xx-ing real tool endpoint exists (every malformed input is
|
|
# caught as 4xx by GlobalExceptionHandler), so we drive the refund path
|
|
# through PaygCucumberThrowController — a @Profile("payg-cucumber") stub
|
|
# that always throws IllegalStateException → 500. The profile is active
|
|
# only inside docker-compose-saas.yml, never in production.
|
|
Given there are no existing shadow charges for team "payg-cucumber-team"
|
|
When I POST a single-page PDF to "/api/v1/payg-cucumber/throw-500"
|
|
Then the response status is 500
|
|
And exactly 1 shadow charge row exists for team "payg-cucumber-team"
|
|
And the latest shadow charge row has status "REFUNDED"
|
|
And the latest shadow charge row's refunded_at is not null
|
|
And the latest shadow charge row's refund_reason starts with "first-step-5xx"
|
|
And the latest job is CLOSED
|
|
And the latest job has 1 step recorded with status "FAILED"
|
|
|
|
# ──────────────────────────────────────────────────────────────────────────
|
|
# NOTE: The kill-switch scenario (PAYG_FILTER_ENABLED=false) is NOT
|
|
# automated — it requires restarting the stirling-pdf-saas container with
|
|
# a different env var mid-run, which couples test setup to compose state
|
|
# more than it's worth for a flag that only flips during incident response.
|
|
# See notes/PAYG_DESIGN.md §7.5.2 "PAYG cucumber: manual-only scenarios".
|
|
# ──────────────────────────────────────────────────────────────────────────
|
|
|
|
Scenario: 4xx leaves the shadow row CHARGED (customer pays for the attempt)
|
|
# /sanitize-pdf on an encrypted PDF without the password reliably 400s
|
|
# via GlobalExceptionHandler's PdfPasswordException → ProblemDetail path.
|
|
# We chain: first call encrypts a PDF (CHARGED), second call tries to
|
|
# sanitize WITHOUT the password and 400s. The 4xx assertion is on the
|
|
# SECOND call's behaviour.
|
|
Given there are no existing shadow charges for team "payg-cucumber-team"
|
|
When I POST a single-page PDF to "/api/v1/security/add-password"
|
|
And I take the response body as "encrypted"
|
|
And I POST "encrypted" to "/api/v1/security/sanitize-pdf"
|
|
Then the response status is >= 400 and < 500
|
|
# Note: shadow_charges count is 1 because the second call lineage-joins
|
|
# the first (its input matches the first's output). The 4xx therefore
|
|
# appears as a FAILED step on the existing process, not a new shadow row.
|
|
And exactly 1 shadow charge row exists for team "payg-cucumber-team"
|
|
And the latest shadow charge row has status "CHARGED"
|
|
And the latest job has step_count = 2
|
|
And the latest step's error_code matches the response status
|
|
|
|
Scenario: ZIP-returning tool records OUTPUT signatures per inner PDF
|
|
# /split-pages with multiple page numbers returns a ZIP. Stirling sends
|
|
# application/octet-stream rather than application/zip — the extractor
|
|
# sniffs the PK\x03\x04 magic so the ZIP unpack path still fires.
|
|
Given there are no existing shadow charges for team "payg-cucumber-team"
|
|
When I POST a 3-page PDF to "/api/v1/general/split-pages" with form fields:
|
|
| pageNumbers | 1,2 |
|
|
Then the response status is 200
|
|
And exactly 1 shadow charge row exists for team "payg-cucumber-team"
|
|
And the latest job has at least 2 OUTPUT artifact hashes recorded
|
|
|
|
Scenario: Multi-file input writes a single shadow row sized by the group
|
|
Given there are no existing shadow charges for team "payg-cucumber-team"
|
|
When I POST two single-page PDFs as a multi-file payload to "/api/v1/general/merge-pdfs"
|
|
Then the response status is 200
|
|
And exactly 1 shadow charge row exists for team "payg-cucumber-team"
|
|
And the latest shadow charge row has payg_units >= 1
|
|
|
|
Scenario: PIPELINE header sets the job source
|
|
Given there are no existing shadow charges for team "payg-cucumber-team"
|
|
When I POST a single-page PDF with header "X-Stirling-Automation: true" to "/api/v1/security/add-password"
|
|
Then the response status is 200
|
|
And exactly 1 shadow charge row exists for team "payg-cucumber-team"
|
|
And the latest job's source is "PIPELINE"
|