Aller au contenu

Story AUDIT-003: Compliance-report stub (degraded CSV export) + ingest metrics

Module: audit-log Slice: AUDIT-003 (Architecture §11) Side: [BACKEND] + [ADMIN] Version target: V0.1.0 Priority: 6 Depends on: AUDIT-001a (audit_log + audit_export tables exist), AUDIT-001b (outbox/builder for the self-event + reconciliation), AUDIT-002 (real data exists to export) Can develop concurrently with: None Merge order: After AUDIT-002 Estimated complexity: M PRD User Stories: AUDIT-US-04 (export — degraded V0 subset), AUDIT-US-08 (self-auditing — export part), AUDIT-US-09 (ingest health — metrics-only stub) Wireframe: N/A — backend only (no UI; platform admin calls the API directly)

Objective

Put down the V0.1.0 compliance-report stub: a POST /api/control/audit/exports endpoint that produces a degraded export — a flat CSV per tenant + date range, streamed to MinIO and returned as a presigned URL — plus a GET to poll status and download. No manifest, no SHA-256, no JSONL in V0 (that is the V1 full version, AUDIT-006). Also expose the Prometheus ingest-health metrics and emit the AUDIT_RECONCILED self-event when the outbox drains. Alerting rules are NOT wired here (V1, AUDIT-009) — V0 is log-only/metrics-only.

Backend Scope

Entities

  • Fills audit_export rows (table created in AUDIT-001a): status flows PENDING → RUNNING → COMPLETED|FAILED, plus event_count, jsonl_object_key (reused for the CSV key in V0), error_message, timestamps.

Migrations

None new (the audit_export table ships in AUDIT-001a).

Service Layer

All under root package com.altarys.papillon.pcs.controlplane.audit: - AuditExportController (...audit.web) — POST /api/control/audit/exports (returns 202 + exportId + pollUrl) and GET /api/control/audit/exports/{id} (status + presigned download URL). Auth: Keycloak JWT role papillon-platform-admin. - CsvExportWorker (...audit.internal) — async job. Cursor-scans audit_log for the requested tenant_id + date range (using the (tenant_id, business_committed_at DESC) index), streams a flat CSV to the MinIO audit-exports bucket, flips audit_export.status = COMPLETED, and fires the AUDIT_EXPORT_COMPLETED self-event. On failure: status FAILED + error_message; no partial bundle persisted (ops watcher GC; Architecture §5.4). - AuditMetrics (...audit.internal) — Micrometer instruments (Architecture §10.4): audit_outbox_incomplete_count (gauge), audit_oldest_incomplete_age_seconds (gauge), audit_freshness_seconds (histogram by producer_module), audit_ingest_failures_total (counter), audit_export_duration_seconds, audit_export_bytes_total. Scrapeable by Prometheus. - Reconciliation self-event: when the spring-modulith republisher (wired in AUDIT-001b) drains the incomplete-count back to 0, emit AUDIT_RECONCILED with the drained count.

API Endpoints

Method Path Request Response Auth
POST /api/control/audit/exports {tenantId (req), dateRangeStart (req), dateRangeEnd (req), eventTypes?, correlationId?, caseKey?} 202 {exportId, status:"PENDING", pollUrl} papillon-platform-admin
GET /api/control/audit/exports/{id} — 200 {exportId, status, scope, eventCount, download:{csvUrl, expiresAt}} papillon-platform-admin

Validation Rules

  • AC-04.3: tenantId, dateRangeStart, dateRangeEnd are required; no export without a tenantId (no unbounded cross-tenant export).
  • Presigned URL TTL = AUDIT_EXPORT_PRESIGN_TTL_SECONDS (default 900 s / 15 min); a fresh GET regenerates a fresh URL.
  • V0 degradation is explicit: CSV only, no manifest.json, no jsonl_sha256. The response must not claim integrity verification it does not provide.

Multi-Tenant Considerations

  • Export is strictly tenant-scoped via the required tenantId (FR-04). The cursor scan filters on tenant_id; no cross-tenant rows can appear in a bundle.
  • Works identically for SHARED and BYO tenants because all audit rows live on the platform DB (AUDIT-ADR-01).

Audit & Logging

  • Emits AUDIT_EXPORT_COMPLETED (PRD §B.1, AC-04.5 / AC-08.2) when an export completes, carrying {export_id, scope, event_count, requested_by} (no jsonl_sha256 in V0).
  • Emits AUDIT_RECONCILED (AC-01.4 / AC-09.3) when the outbox drains to zero.
  • (V0 does NOT emit AUDIT_QUERY_EXECUTED — the query API is V1.)

Frontend Scope

N/A — backend only. Platform admin invokes the API directly; no operator/admin console screen in V0 (PRD §C.5).

Acceptance Criteria

AC-04.5 / AC-08.2: Export is itself audited
Given a platform admin triggers an export and it completes
When the worker finishes writing the bundle
Then AUDIT writes one AUDIT_EXPORT_COMPLETED self-event with {export_id, scope, event_count, requested_by}

AC-003.1: Degraded CSV export round-trip (derived from Architecture §11 AUDIT-003 acceptance)
Given a tenant with audit rows and a date range
When the admin POSTs an export and polls the GET until COMPLETED
Then a flat CSV is downloadable via a presigned URL, the URL expires 15 minutes after issuance, and the row count matches the scope
And an export of 1 000 events completes within 5 seconds

AC-01.4: Reconciliation drains and is recorded
Given entries queued during an ingest-degradation window
When AUDIT recovers and the republisher drains the queue to zero
Then AUDIT emits AUDIT_RECONCILED with the drained count
And audit_outbox_incomplete_count is scrapeable by Prometheus and returns to 0

Compliance Rules

  • CIMA Reg. 01-24 §16.5 + CI fiscal retention — the export is the mechanism by which platform admin hands a regulator/lawyer a slice of the proof base. V0 ships a degraded (non-integrity-verified) CSV; the integrity-verifiable JSONL+manifest+SHA-256 version is V1 (AUDIT-006) — this limitation must be documented so the §16.5 claim is not over-stated (AC-05.4 honest scope).
  • AUDIT-US-09 alerting — V0 exposes metrics only; the Grafana alert rules + on-call routing are V1. [LEGAL-REVIEW] not newly triggered; retention-duration PRÉLIMINAIRE note from memo F5 still applies to any fixed log duration.

Standards & Conventions

  • docs/standards/api-guidelines.md — response envelope, cursor pagination, X-API-Version header convention.
  • docs/standards/java-spring-guidelines.md — async worker, controller, named-interface beans.
  • docs/standards/multi-datasource-patterns.md — platform-DB scan for all tenants.
  • docs/standards/critical-rules.md — rule 1 (tenant scoping on the export scan), rule 3 (catalog: AUDIT_EXPORT_COMPLETED, AUDIT_RECONCILED).

Testing Requirements

Unit Tests

  • AuditExportController rejects a POST missing tenantId / date range (AC-04.3).
  • CsvExportWorker flips status correctly on success and on failure.

Integration Tests

  • AC-003.1: POST an export for 1 000 events → poll GET until COMPLETED (within 5 s) → download CSV via presigned URL → assert row count → assert URL is dead after 15 min.
  • AC-04.5: assert exactly one AUDIT_EXPORT_COMPLETED row after a completed export.
  • AC-01.4: simulate an ingest-degradation window, queue entries, resume, assert the queue drains and an AUDIT_RECONCILED row appears with the right count; assert audit_outbox_incomplete_count returns to 0.
  • Tenant isolation: an export scoped to tenant A contains zero tenant-B rows.

What QA Will Validate

  • Degraded CSV export works end-to-end and is tenant-scoped.
  • Export and reconciliation self-events are written.
  • Ingest metrics are scrapeable (alert rules deliberately absent in V0).

Out of Scope

  • Full JSONL + manifest.json + SHA-256 integrity export (V1, AUDIT-006).
  • Filtered query API + AUDIT_QUERY_EXECUTED dedup self-event (V1, AUDIT-004).
  • By-case lookup + correlation expansion (V1, AUDIT-005).
  • Active AUDIT_INGEST_DEGRADED alerting + Grafana rules + on-call routing (V1, AUDIT-009).
  • papillon_audit_reader role consumption by a read pool (V1, AUDIT-007).
  • Any UI / console screen (V1+; tenant-side viewer is V2).
  • Automatic 10-year purge (V2).

Definition of Done

  • [ ] POST /api/control/audit/exports (202) + GET /exports/{id} implemented under papillon-platform-admin
  • [ ] Degraded CSV streamed to the audit-exports MinIO bucket; presigned URL expires after 15 min
  • [ ] Export of 1 000 events completes within 5 s (AC-003.1)
  • [ ] AUDIT_EXPORT_COMPLETED self-event written on completion (AC-04.5 / AC-08.2)
  • [ ] AUDIT_RECONCILED emitted on drain-to-zero; ingest metrics scrapeable by Prometheus (AC-01.4)
  • [ ] Export rejects a missing tenantId / date range (AC-04.3)
  • [ ] Tenant isolation verified (export scan scoped by tenant_id; both V1 DB profiles)
  • [ ] V0 degradation documented (no manifest/SHA-256) so the §16.5 claim is not over-stated
  • [ ] No silent failures: a worker crash flips status to FAILED with an error message, no partial bundle served