Aller au contenu

Story ING-006: Cross-cutting integration test pack (isolation, large file, magic byte, crash matrix, prod flag)

Module: ingestion Slice: Tier 1 sub-slice ING-001-e (architecture §11) Side: [BACKEND] Version target: V0.0.0 Priority: 6 Depends on: ING-002, ING-003, ING-004, ING-005 (all behaviors implemented) Can develop concurrently with: ING-005 (test scaffolding can begin; assertions pinned once ING-005 lands) Merge order: Last — gates v0.0.0 ING release Estimated complexity: M PRD User Stories: N/A — Technical Story (release-gate test pack) Wireframe: N/A — backend only

Objective

Bind the v0.0.0 ING release with a regression-grade integration test pack covering the cross-cutting concerns called out in architecture §12 (Risks A1–A10) and §13 (Test Infrastructure): cross-tenant SHA-256 isolation, 5 000-row large-file handling within memory budget, magic-byte spoofing rejection, the full crash-recovery matrix, and a prod-profile feature-flag-closed smoke. These tests catch regressions any of ING-002..005 would otherwise let through.

Backend Scope

Entities

None new. Tests read existing ingestion_import, ingestion_import_row, customer, contract, and audit_log rows for assertions.

Migrations

None.

Service Layer

No production code. New test classes only (under backend/src/test/java/.../ingestion/): - IngestionCrossTenantIsolationIT (architecture §13.3 sample) — same SHA-256 dropped in tenant A and B; assert each gets its own COMMITTED row and no cross-pollution. - IngestionLargeFileMemoryIT — generated 5 000-row CSV; assert end-to-end commit ≤10 s parse-validate + ≤5 s commit (architecture §8) and peak heap <256 MB via JFR. - IngestionMagicByteSpoofIT — .csv-named file whose bytes are OOXML → PARSE_FAILED with FORMAT_NON_SUPPORTE; .xlsx-named file whose bytes are plain CSV → PARSE_FAILED with FORMAT_NON_SUPPORTE. - IngestionCrashRecoveryMatrixIT — three scenarios per architecture §5.1: (a) crash after INSERT PICKED_UP before parse → recovery kicks in; (b) crash after UPDATE PARSE_FAILED before MinIO move → next tick repeats the idempotent COPY+DELETE; (c) crash after COMMIT business TX before MinIO move → next tick repeats the move (file moves to processed even though the import is already COMMITTED). - IngestionProdFlagClosedSmokeIT (architecture §13.5 sample) — spring.profiles.active=prod + feature.artci.phone-auth.granted=false; tick → no-op + WARN log + file untouched. - IngestionDoubleProfileSmokeIT — runs the happy path AC-001 from ING-004 twice, once with SHARED datasource, once with BYO datasource provisioned at test setup (architecture §4.2). Asserts same results.

API Endpoints

None.

Validation Rules

None new. These tests assert existing validation behavior.

Multi-Tenant Considerations

The whole point of this slice: prove tenant isolation under SHA-256 collision (the highest-risk regression per architecture §12 A10). Test fixture seeds 3 tenants by default. BYO smoke covered explicitly in IngestionDoubleProfileSmokeIT.

Audit & Logging

No new events. Tests assert audit_log contents: - Cross-tenant test: exactly one ING_COMMITTED per tenant; zero ING_DUPLICATE_FILE. - Large-file test: exactly one ING_COMMITTED with rows_created=5000. - Crash matrix: ING_PARSE_FAILED for (a); the original ING_COMMITTED/ING_PARSE_FAILED from before crash for (b) and (c); no duplicate audit emission on the second tick.

Frontend Scope

N/A — backend only.

UI States (ALL REQUIRED)

State Behavior French Copy
Loading N/A — backend only —
Empty N/A — backend only —
Error N/A — backend only —
Offline / low network N/A — backend only —
Success N/A — backend only —

Responsive Behavior

N/A.

Interactions

N/A.

Acceptance Criteria

AC-001: Cross-tenant SHA-256 isolation
Given the same byte-identical CSV dropped by tenant A and tenant B
When IngestionService.tick() runs
Then both tenants have exactly 1 `ingestion_import` row with status=COMMITTED
  And neither tenant has a status=DUPLICATE row
  And `customer` rows for tenant A have only tenant_id=A; rows for tenant B have only tenant_id=B
  And the SQL `SELECT count(*) FROM customer WHERE tenant_id=B AND source_customer_id IN (SELECT source_customer_id FROM customer WHERE tenant_id=A)` returns the number of distinct source ids (rows ARE the same source ids — they are not the SAME rows in DB).

AC-002: Large file (5 000 rows) commits within performance + memory budget
Given a generated 5 000-row CSV in `/<tenantA>/inbox/`
When IngestionService.tick() runs
Then parse + validate completes in ≤10 s
  And the commit transaction completes in ≤5 s
  And max heap usage during the run is <256 MB (asserted via JFR snapshot)
  And `ingestion_import.rows_created = 5000`.

AC-003: Crash recovery matrix — all three crash points reach a clean steady state by tick #2
Given test scenarios (a) crash after PICKED_UP INSERT, (b) crash after PARSE_FAILED UPDATE but before MinIO move, (c) crash after COMMIT but before MinIO move
When the application boots and the next tick runs
Then (a): row is now PARSE_FAILED via startup recovery, file moved to /rejected/, audit emitted
  And (b): row remains PARSE_FAILED, file is now in /rejected/ (idempotent re-move), no duplicate audit
  And (c): row remains COMMITTED, file is now in /processed/ (idempotent re-move), no duplicate audit and no duplicate customer/contract rows.

AC-004: Prod feature-flag-closed smoke + magic-byte spoof rejection
Given `spring.profiles.active=prod` and `feature.artci.phone-auth.granted=false`
  And a file `<tenantA>/inbox/clean-5-rows.csv`
When IngestionService.tick() runs
Then zero `ingestion_import` rows are created, the file stays in inbox, and the `[LEGAL-REVIEW] cron skipped` WARN is logged

And given (separately) the flag opened and a spoofed file `<tenantA>/inbox/spoof.csv` whose first bytes are `50 4B 03 04` (OOXML)
When the next tick runs
Then `ingestion_import.status=PARSE_FAILED` with `parse_error_detail.anomaly_counts_by_code.FORMAT_NON_SUPPORTE >= 1`
  And the file is in /<tenantA>/rejected/ with sidecar JSON containing `"FORMAT_NON_SUPPORTE"`.

Compliance Rules

  • Critical rule #1 (tenant isolation) — IngestionCrossTenantIsolationIT is the dedicated regression guard.
  • Architecture §7.6 A1 (zip-bomb / XXE) + A5 (POI heap) — IngestionLargeFileMemoryIT enforces the 256 MB ceiling on the highest-stress path.
  • Architecture §7.6 A4 (cross-tenant key escape) — covered by AC-001's SQL invariant.
  • ARTCI compliance posture (V0 PRD §6.4) — IngestionProdFlagClosedSmokeIT is the binary release gate.

Standards & Conventions

  • docs/standards/critical-rules.md (#1, #3).
  • docs/standards/multi-datasource-patterns.md § integration test patterns under SHARED + BYO.
  • docs/standards/java-spring-guidelines.md § integration test slicing + JFR.
  • docs/standards/git-workflow.md — branch feat/ingestion-ING-006; PR title test(ingestion): cross-cutting integration pack (ING-006).

Testing Requirements

Unit Tests

  • N/A (this slice is integration tests by definition).

Integration Tests

All six test classes listed under "Service Layer" above. CI must run them on every PR touching ingestion/.

What QA Will Validate

  • ./gradlew :backend:integrationTest --tests "*Ingestion*IT" green on a clean checkout.
  • CI report shows JFR-asserted heap line for IngestionLargeFileMemoryIT.
  • BYO datasource smoke variant passes — confirms architecture §4.2 promise that no ING code change is needed across DB profiles.

Out of Scope

  • Performance load testing beyond 5 000 rows (V1).
  • Chaos / fault-injection testing on MinIO availability (V1).
  • ShedLock multi-instance scheduling regression tests (V1 — single-instance deploy in v0.0.0).
  • DOB drift detection (ING_DOB_DRIFT_DETECTED) tests — V0.0.2.
  • Atelier flow tests — V0.0.3.
  • UI tests — N/A (no UI).
  • End-to-end OP consumer assertions (only that the event was published; OP-side cohort refresh behavior is tested in OP's own story).

Definition of Done

  • [ ] All backend endpoints implemented and returning correct responses (N/A — no endpoints added)
  • [ ] All frontend pages render correctly at 360 / 768 / 1024 px (N/A — backend only)
  • [ ] All 5 UI states implemented (N/A — backend only)
  • [ ] French micro-copy matches design spec exactly (N/A — backend only)
  • [ ] All acceptance criteria pass manually
  • [ ] Unit + integration tests written and passing (six IT classes above)
  • [ ] Tenant isolation verified (dedicated IT class + SQL invariant assertion)
  • [ ] [CUSTOMER stories] payload + Slow-3G + resumability (N/A — backend only)
  • [ ] Audit log entries present for every required event (every test asserts the expected audit_log rows present and no duplicates)
  • [ ] No silent failures on network errors (crash matrix IT covers MinIO + DB partial-failure cases)