Aller au contenu

Architecture — ingestion (ING) module

Module prefix: ING · Plane: Application · Scope of this document: v0.0.0 only (story ING-001). v0.0.1 → V1 features (error UX, atelier, UI upload, full validation set, MinIO retention browse) are designed-for but not architected in detail here; checkpoints called out inline.

Authored by: ARCHITECT session, 2026-05-21. Builds on docs/architecture/PROGRESS.md cross-module decisions D1–D23 — those are reused and not re-debated.

Required PRD reads (already absorbed): docs/prd/ingestion-prd.md (V0 envelope at top + V1 §§A/B/C as forward-compat reference); docs/prd/prd-v0.md §4 (ING) + §5 + §8; docs/prd/prd-v1.md; docs/architecture/PROGRESS.md; docs/architecture/tenant-admin-arch.md (datasource resolver, TenantConfigCache, MinIO bucket); docs/architecture/audit-log-arch.md (PiiHasher, outbox, AuditEventBuilder); docs/architecture/notifications-arch.md (peer V0 module shape).


0. v0.0.0 scope summary (executive)

Concern v0.0.0 status
Trigger Single platform-wide @Scheduled task at 0 0 2 * * * Africa/Abidjan. Iterates active tenants sequentially (bounded parallelism = 1 in v0.0.0).
File source Platform-owned MinIO bucket papillon-ingestion-inbox, per-tenant prefix /<tenant_id>/inbox/. Tenant uploads via direct MinIO credentials issued by TENANT.
Accepted formats .csv (UTF-8, auto-detect ;/,) and .xlsx. Magic-byte verified.
Volume limits ≤5 MB file, ≤5 000 data rows. Same as V1 ING-R-002.
Validation set Core only (V0 PRD §4 ING simplified): mandatory = identifiant_client, identifiant_contrat, nom, prenoms, telephone, montant_du, date_echeance. Other V1 fields (date_naissance, branche, email, numero_police, immatriculation, statut_source, informations_complementaires) parsed if present, never blocking. code_agence ignored (single implicit agency in v0.0.0; multi-agency arrives v0.0.2).
Phone normalization E.164 CI (+225XXXXXXXXXX), V1 ING-R-005 algorithm. Blocking on failure.
Anomaly handling All-or-nothing reject — any blocking anomaly anywhere ⇒ entire file rejected, no commit, file moved to /<tenant_id>/rejected/<timestamp>__<filename>, ING_PARSE_FAILED audit with anomaly summary. No atelier, no operator feedback UI in v0.0.0.
Dedup SHA-256 per tenant_id (V1 ING-R-003). Re-drop of identical file ⇒ moved to /<tenant_id>/duplicate/, audit, no work.
Commit All-or-nothing single DB transaction: upsert customer + upsert contract + insert ingestion_import_row per line + emit audit + publish ImportCommitted Spring application event for OP.
Idempotency / crash recovery At-most-once commit per file via SHA-256 dedup + per-file lifecycle FSM on ingestion_import. Cron-crash during pickup ⇒ leaves file in /inbox/; next run picks it up unchanged.
Audit emission ING_UPLOAD_RECEIVED, ING_PARSE_FAILED, ING_COMMITTED, ING_CUSTOMER_CREATED/UPDATED, ING_CONTRACT_CREATED/UPDATED, ING_DUPLICATE_FILE. Via spring-modulith-events-jdbc outbox (D4 + D7).
Production gate Feature flag feature.artci.phone-auth.granted (default false) gates the cron on prod profile only. Pre-prod ingestion authorized. [LEGAL-REVIEW] warning emitted in startup logs while false.
Multi-tenant isolation Every query carries tenant_id (D1 + critical rule #1). MinIO keys partitioned by tenant_id. SHA-256 unique per tenant only.
DB profile coverage SHARED v0.0.0; BYO v0.0.4 — same DDL applies via D9 Flyway tenant/ migrations.
Not in v0.0.0 (designed-for, inactive) Atelier de correction (ingestion_staged_row table, PATCH /rows/{id} endpoint, all-rows-correctable workflow) → v0.0.3. UI upload endpoint POST /imports (multipart) → v0.0.3. MinIO file-history download /source-file → v0.0.3. Per-agency scoping (code_agence, ING-R-009, ING-R-017) → v0.0.2. Operator notification of rejection (badge, email) → v0.0.1 (badge) / V2 (email/SMS). date_naissance blocking validation → reintroduced v0.0.2 (V0 PRD §4 ING row "v0.0.2 reintroduit règles V1"). Full V1 validation set → v0.0.3.

1. System Context

1.1 Position in the platform

ING is the data ingress of the application plane. In v0.0.0 it has no human-facing UI — operators drop files in a MinIO inbox and the cron does the rest. ING is the upstream producer for OP (campaign cohort selection); every downstream renewal flow starts from a commit made by ING.

flowchart LR
  subgraph "External (per tenant)"
    Op([Cabinet admin])
    MinIO[(MinIO bucket<br/>papillon-ingestion-inbox<br/>/&lt;tenant_id&gt;/inbox/)]
  end
  subgraph "Control plane"
    TENANT[tenant-admin]
    AUTH[identity]
    AUDIT[audit-log]
  end
  subgraph "Application plane"
    ING[ingestion]
    OP[operator-console]
  end

  Op -- "scp / aws s3 cp<br/>(MinIO creds from TENANT)" --> MinIO
  ING -. "@Scheduled 02:00 Abidjan" .-> ING
  ING -- "list, get, move (inbox/rejected/processed/duplicate)" --> MinIO
  ING -- "tenant_id, country_code lookup" --> TENANT
  ING -- "upsert customer, contract" --> ING
  ING -- "outbox: ING_*" --> AUDIT
  ING -- "ApplicationEvent:<br/>ImportCommitted" --> OP

1.2 Cross-module dependencies (v0.0.0)

Direction Counterpart Contract Available
In TENANT TenantConfigCache bean — list active tenants + read tenant.country_code (informational; v0.0.0 = CI only). v0.0.0
In TENANT MinIO bucket papillon-ingestion-inbox exists. Cross-module impact — TENANT S0 must provision this bucket (extension of existing MinIO setup). v0.0.0
In AUDIT AuditEventType enum (add ING_* entries) + AuditEventBuilder named interface + PiiHasher for phone/name fields in audit details. v0.0.0
In AUDIT Catalog admits new event types: ING_UPLOAD_RECEIVED, ING_PARSE_FAILED, ING_COMMITTED, ING_CUSTOMER_CREATED, ING_CUSTOMER_UPDATED, ING_CONTRACT_CREATED, ING_CONTRACT_UPDATED, ING_DUPLICATE_FILE. Cross-module impact — AUDIT catalog rework needed before ING-001 dev-start (parallels OQ-X9 / NOTIF).
In (none AUTH) Cron has no user JWT; runs under actor_type=SYSTEM / actor_id="ingestion-cron". No SignedLinkService use. v0.0.0
Out OP ImportCommitted {importId, tenantId, rowsCreated, rowsUpdated, rowsSkipped, occurredAt, correlationId} Spring application event. OP listens to refresh cohort eligibility cache (consumed silently in v0.0.0; UI use in v0.0.1+). v0.0.0
Out (table) Direct DB writes to shared tables customer and contract (D-shared canonical owners; ING is the writer of identity fields, NOT the schema owner). v0.0.0

1.3 correlation_id policy

D16 locks correlation_id origin = OP at OP_CAMPAIGN_VALIDATED. ING events occur upstream of any campaign — there is no business correlation chain to inherit. ING therefore mints a batch-scoped correlation_id = import_id (UUIDv7) for all audit events of one import. This is not a renewal correlation chain; it is a debugging aid that lets OP/AUDIT join all rows of one file. Downstream OP campaigns will mint their own correlation_id. ImportCommitted carries importId but does not propagate correlation_id into OP — OP will start a fresh chain.


2. Data Model

2.1 ERD (v0.0.0)

erDiagram
  ingestion_import ||--o{ ingestion_import_row : has
  ingestion_import }o--|| tenant : "tenant_id (platform DB FK)"
  ingestion_import_row }o--|| customer : "FK"
  ingestion_import_row }o--|| contract : "FK"
  customer ||--o{ contract : has

  ingestion_import {
    uuid id PK "UUIDv7"
    uuid tenant_id FK
    varchar filename "≤255"
    char file_hash_sha256 "64 hex"
    varchar file_mime "text/csv or application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
    int file_size_bytes
    text file_minio_inbox_key "original /<tid>/inbox/<filename>"
    text file_minio_processed_key "post-success move, nullable"
    text file_minio_rejected_key "post-reject move, nullable"
    varchar status "ENUM: PICKED_UP, PARSE_FAILED, COMMITTED, DUPLICATE"
    int row_count_total "nullable until parse"
    int rows_created
    int rows_updated
    int rows_skipped
    jsonb parse_error_detail "nullable; only set on PARSE_FAILED"
    timestamptz picked_up_at
    timestamptz finished_at "PARSE_FAILED or COMMITTED reached"
  }

  ingestion_import_row {
    uuid id PK "UUIDv7"
    uuid import_id FK
    uuid tenant_id FK "denormalized"
    int row_number "1-based in source file"
    uuid customer_id FK
    uuid contract_id FK
    varchar action "ENUM: CREATED, UPDATED, SKIPPED"
    timestamptz committed_at
  }

2.2 Schema-stability commitment

Per V0 PRD §4 ING "additive non-breaking" + AUDIT-ADR-03 pattern: the v0.0.0 schema is forward-compatible. v0.0.3 will introduce ingestion_staged_row as a new table (additive); v0.0.2 will introduce agency_id columns on contract (TENANT-002 owns that migration). No ALTER TABLE on existing ING tables is anticipated v0.0.0 → V1 except adding nullable columns.

V1 forward-compat checks honored:

  • file_hash_sha256 is unique per (tenant_id, file_hash_sha256) — never globally (V1 §B.6).
  • tenant_id denormalized on ingestion_import_row so it can be joined without going through ingestion_import.
  • No UNIQUE on filename (same filename re-uploaded with different content must be allowed).

2.3 Shared entities written by ING (NOT owned)

ING writes customer and contract; their schema is governed by other modules' V1 PRDs (per V1 PRD §27). v0.0.0 ING-writable columns:

customer (tenant DB; D1): - id (UUIDv7), tenant_id, source_customer_id (V1 ING-R-015 dedup key with tenant_id), nom, prenoms, telephone (E.164), email (nullable), created_at, updated_at. - UNIQUE (tenant_id, source_customer_id). - date_naissance column exists (V1 schema); v0.0.0 writes NULL when absent in file.

contract (tenant DB; D1): - id (UUIDv7), tenant_id, customer_id, source_contract_id, branche (nullable in v0.0.0; reintroduced v0.0.2), date_echeance (DATE), montant_du (BIGINT, XOF minor units = whole XOF integer), numero_police (nullable), immatriculation (nullable), statut_source (nullable), informations_complementaires (nullable), last_import_id, created_at, updated_at. - UNIQUE (tenant_id, customer_id, source_contract_id) (V1 ING-R-012 inter-file dedup key). - agency_id column added by TENANT-002 in v0.0.2 (NOT NULL with FK), default to "default agency" auto-provisioned in v0.0.0 by TENANT-001 in the same migration that creates the column.

2.4 Indexing strategy

Table Index Rationale
ingestion_import PRIMARY KEY (id)
ingestion_import UNIQUE (tenant_id, file_hash_sha256) V1 ING-R-003 dedup. Composite leftmost = tenant_id ⇒ doubles as scope.
ingestion_import (tenant_id, status, picked_up_at DESC) Future history query (v0.0.3); cheap to keep now.
ingestion_import_row PRIMARY KEY (id)
ingestion_import_row (import_id) Join from header.
ingestion_import_row (tenant_id, customer_id) Trace queries from AUDIT case views (V1).
customer UNIQUE (tenant_id, source_customer_id) Upsert key (V1 ING-R-015).
contract UNIQUE (tenant_id, customer_id, source_contract_id) Upsert key (V1 ING-R-016).
contract (tenant_id, date_echeance) OP cohort selection (renewal window query).

2.5 Storage volume estimate (v0.0.0 pilot)

Per V0 PRD § 9 pilot scale ≤ 10 tenants × ≤ 5 000 rows × ≤ daily import. Worst-case month ≈ 1.5 M ingestion_import_row rows. Postgres handles this trivially; no partitioning needed before V1.


3. API Design

3.1 v0.0.0 endpoints

ING exposes no HTTP endpoint in v0.0.0. The cron is the only entry point. No multipart upload, no rows API, no commit API, no template download.

Reserved namespace /api/ingestion/... (per PROGRESS.md API namespaces) is claimed but unused until v0.0.3 (UI upload + atelier).

3.2 Why no HTTP in v0.0.0

V0 PRD §4 ING line: "Pas d'upload UI" + v0.0.3 cell: "Upload UI disponible (admin cabinet téléverse via web). L'atelier de correction complet V1 est layered ici." Adding the endpoint now without the atelier UI behind it would force the operator into a curl-or-Postman workflow without a feedback channel — explicitly out of v0.0.0 scope.

3.3 Out-of-band file drop

Operator drops files into MinIO via: - AWS CLI v2 (S3-compatible) using credentials issued by TENANT S0 onboarding (one IAM-equivalent user per tenant, scoped to /<tenant_id>/inbox/* read-write + /<tenant_id>/rejected/* /<tenant_id>/processed/* /<tenant_id>/duplicate/* read-only). - Or rclone / mc / SFTP-frontend (V1 — not part of v0.0.0).

Bucket key layout:

papillon-ingestion-inbox/
├── <tenant_id_A>/
│   ├── inbox/         ← operator writes here (drop zone)
│   ├── processed/     ← cron moves committed files here, key = <yyyy-MM-dd>__<original-filename>
│   ├── rejected/      ← cron moves parse-failed files here, key = <yyyy-MM-dd>T<HH-mm-ss>__<original-filename>; sidecar .errors.json
│   └── duplicate/     ← cron moves SHA-256 duplicates here
├── <tenant_id_B>/...

4. Multi-Tenant Strategy

4.1 Tenant resolution (cron context)

Cron has no JWT, no signed-link, no HTTP request. Tenant context is set explicitly per iteration:

for tenant in tenantConfigCache.listActiveTenants() {
  try (var ctx = TenantContext.scope(tenant.id, tenant.countryCode)) {
    ingestOnce(tenant);
  }
}

TenantContext.scope(...) is the same request-scoped holder used by TenantFilter (D1). actor_type = SYSTEM, actor_id = "ingestion-cron" on all audit emissions.

4.2 Isolation across V0 DB profiles

  • SHARED (v0.0.0): all ING tables in the shared tenant DB; WHERE tenant_id = :tid on every query (no exception). Critical rule #1.
  • BYO (v0.0.4): ING tables present in tenant's BYO DB via D9 Flyway tenant/ migrations applied at provisioning. Datasource resolver routes per-tenant; cron iterates tenants from platform DB, then routes business writes to the per-tenant resolved datasource. Audit always writes to platform DB (D12). No code change ING-side between v0.0.0 and v0.0.4 — the datasource resolver handles it.

4.3 MinIO bucket isolation

Single bucket, per-tenant prefix. Tenant MinIO credentials are scoped at the prefix level (MinIO ABAC policy / pre-signed IAM-equivalent). Cross-tenant key leaks structurally impossible because the cron always builds the key as f"{tenantId}/inbox/{filename}" from the iteration variable.

4.4 Agency sub-scoping (out of v0.0.0)

In v0.0.0 every tenant has exactly one auto-provisioned "default agency" (TENANT-001 creates it). contract.agency_id is set to this default for every committed row. v0.0.2 introduces real agencies + code_agence column resolution. The architecture is non-breaking: TENANT-002 migration adds the code_agence validation step inside ING; existing default_agency_id rows remain valid.


5. Connectivity & Resumability

ING is operator-side / system-side only. No customer flow; the low-bandwidth resumability rules of connectivity-low-bandwidth.md do not apply.

5.1 Cron crash recovery

The cron is the unit of resumability. Per-file lifecycle:

Stage What happens If cron crashes here
1. List /<tid>/inbox/ MinIO LIST. No DB write. File stays in inbox. Next run lists it.
2. Compute SHA-256, dedup check Stream-hash file via MinIO GET; SELECT on ingestion_import for (tenant_id, file_hash_sha256). No write. File stays in inbox. Hash is deterministic; re-runs match.
3. INSERT ingestion_import row with status='PICKED_UP' Transaction commits. INSERT either committed or not. On not-committed: next run re-picks. On committed: see 4.
4. Parse + validate (in-memory) No write. PICKED_UP row orphaned — recovered by startup job IngestionRecoverStuckImports (resets to ignored or re-parse).
5a. PARSE_FAILED branch UPDATE status='PARSE_FAILED', populate parse_error_detail, emit ING_PARSE_FAILED via outbox, MinIO COPY+DELETE inbox→rejected. If DB UPDATE done but MinIO move not: file stays in inbox; next run sees SHA-256 already present with status PARSE_FAILED → repeats the move step (idempotent COPY+DELETE).
5b. COMMIT branch Single transaction: upsert customer + contract + insert ingestion_import_rows + UPDATE ingestion_import.status='COMMITTED' + publish ImportCommitted (outbox). After commit: MinIO COPY+DELETE inbox→processed. If DB tx committed but MinIO move not done: file stays in inbox; next run sees SHA-256 already present with status COMMITTED → MinIO move step is repeated (idempotent).
5c. DUPLICATE branch (SHA-256 match) UPDATE … no, just emit ING_DUPLICATE_FILE audit and MinIO move inbox→duplicate. New ingestion_import row created with status='DUPLICATE' linking to the original via originating_import_id (column NULL otherwise). Same idempotency as 5a/5b.

Startup recovery job IngestionStartupRecovery (runs once at boot, before the scheduler enables): finds ingestion_import rows where status='PICKED_UP' and picked_up_at < now - 5 min, marks them PARSE_FAILED with reason recovered_from_crash, moves the inbox file to /rejected/ with sidecar JSON { "reason": "ingestion crashed during parse, please re-drop the file" }. Audit ING_PARSE_FAILED is emitted. No data corruption possible because no business write happens before stage 5b's single transaction.

5.2 Why no Redis / outbox for the cron itself

The cron is a single Spring @Scheduled bean. spring-modulith-events-jdbc (D4) handles the outbound audit + cross-module publication. The cron's inbound flow is naturally idempotent via SHA-256 dedup + MinIO file location, so it needs no extra outbox. No new infrastructure is introduced by ING.


6. Sequence Diagrams

6.1 Happy path — clean file

sequenceDiagram
  autonumber
  participant SCH as @Scheduled (02:00 Abidjan)
  participant ING as IngestionService
  participant MIN as MinIO
  participant DB as Tenant DB
  participant PDB as Platform DB
  participant OB as spring-modulith outbox
  participant OP as operator-console (listener)

  SCH->>ING: tick
  ING->>PDB: TenantConfigCache.listActiveTenants()
  loop for each tenant
    ING->>MIN: LIST /<tid>/inbox/
    alt no files
      ING-->>SCH: skip
    else file present
      ING->>MIN: GET object stream
      ING->>ING: stream SHA-256
      ING->>DB: SELECT ingestion_import WHERE (tenant_id, file_hash_sha256)
      Note over ING: no match → proceed
      ING->>DB: INSERT ingestion_import (status=PICKED_UP)
      ING->>ING: parse + validate (in-memory)
      Note over ING: 0 blocking anomalies
      ING->>DB: BEGIN TX
      ING->>DB: upsert customer + contract per row
      ING->>DB: INSERT ingestion_import_row per row
      ING->>DB: UPDATE ingestion_import SET status=COMMITTED, counters, finished_at
      ING->>OB: publish ING_COMMITTED + ING_CUSTOMER_* + ING_CONTRACT_* + ImportCommitted
      ING->>DB: COMMIT
      OB-->>PDB: AUDIT INSERT (post-commit)
      OB-->>OP: ImportCommitted (post-commit)
      ING->>MIN: COPY inbox→processed; DELETE inbox key
    end
  end

6.2 Parse-failed / reject path

sequenceDiagram
  autonumber
  participant SCH as @Scheduled
  participant ING as IngestionService
  participant MIN as MinIO
  participant DB as Tenant DB
  participant PDB as Platform DB

  SCH->>ING: tick
  ING->>MIN: GET file
  ING->>DB: INSERT ingestion_import (PICKED_UP)
  ING->>ING: parse + validate
  Note over ING: ≥1 blocking anomaly (TELEPHONE_INVALIDE, MONTANT_ABSENT, ...)
  ING->>DB: UPDATE ingestion_import SET status=PARSE_FAILED, parse_error_detail=<anomaly summary>, finished_at
  ING->>PDB: outbox publish ING_PARSE_FAILED (via spring-modulith)
  ING->>MIN: PUT /<tid>/rejected/<ts>__<filename>.errors.json (sidecar)
  ING->>MIN: COPY inbox→rejected; DELETE inbox key

6.3 Duplicate-file path

sequenceDiagram
  autonumber
  participant ING as IngestionService
  participant MIN as MinIO
  participant DB as Tenant DB

  ING->>MIN: GET file, stream SHA-256
  ING->>DB: SELECT ingestion_import WHERE (tenant_id, file_hash_sha256)
  Note over DB: row found (status=COMMITTED)
  ING->>DB: INSERT ingestion_import (status=DUPLICATE, originating_import_id=<original>)
  ING->>OB: ING_DUPLICATE_FILE (audit)
  ING->>MIN: COPY inbox→duplicate; DELETE inbox key

6.4 Crash mid-parse → next tick recovers

sequenceDiagram
  autonumber
  participant T1 as Run #1 (crashes)
  participant SR as IngestionStartupRecovery
  participant T2 as Run #2 (next 02:00 OR boot)
  participant DB as Tenant DB
  participant MIN as MinIO

  T1->>DB: INSERT ingestion_import (PICKED_UP)
  T1->>T1: parse... [CRASH]
  Note over T1: pod restart
  SR->>DB: SELECT status=PICKED_UP AND picked_up_at < now-5min
  SR->>DB: UPDATE → PARSE_FAILED, parse_error_detail={reason:"recovered_from_crash"}
  SR->>MIN: move inbox→rejected with sidecar
  Note over SR: file rejected with clear operator message; re-drop required
  T2->>MIN: LIST /<tid>/inbox/ → empty (file already moved)

6.5 Feature flag closed (production gate)

sequenceDiagram
  autonumber
  participant SCH as @Scheduled
  participant ING as IngestionService
  participant LOG as logs

  SCH->>ING: tick (profile=prod)
  ING->>ING: read feature.artci.phone-auth.granted
  Note over ING: == false
  ING->>LOG: WARN [LEGAL-REVIEW] ingestion cron skipped — ARTCI phone-auth pending. files in inbox NOT processed.
  ING-->>SCH: return without listing

7. Security

7.1 Auth

  • No operator auth in v0.0.0 — the cron is SYSTEM.
  • MinIO credentials issued per tenant at onboarding (TENANT-001), scoped to that tenant's prefixes only. Credentials rotation is V1.
  • No customer flow ⇒ no signed-link in this module.

7.2 RBAC (irrelevant in v0.0.0 — cron only)

When UI upload arrives v0.0.3: - CABINET_ADMIN → upload any agency's file. - OPERATOR → upload restricted to assigned agencies (re-introduces V1 ING-R-017). - READ → blocked (HTTP 403).

7.3 Encryption

  • In transit: TLS to MinIO (S3 SDK over HTTPS).
  • At rest: MinIO bucket configured with SSE (per cybersecurity guidance C.3). Postgres on encrypted volume.
  • Field-level: not in v0.0.0 (PII columns plain text, accepted v1 PRD A.7 + per TV-shared "réévaluer avant >10 tenants ou si BYO-DB exige chiffrement champ").

7.4 Audit (v0.0.0 events)

All emitted via outbox (D4, D7). Failure to audit MUST NOT roll back the business action (D7).

event_type When actor_type/actor_id subject_type/subject_id Details JSON (PII rules)
ING_UPLOAD_RECEIVED After INSERT ingestion_import (PICKED_UP) SYSTEM / ingestion-cron INGESTION_IMPORT / id { filename, file_size_bytes, file_hash_sha256, minio_inbox_key }. No row content.
ING_PARSE_FAILED After UPDATE to PARSE_FAILED SYSTEM / ingestion-cron INGESTION_IMPORT / id { reason_code, anomaly_counts_by_code, sample_row_numbers (first 5) }. Phone/name never in detail.
ING_DUPLICATE_FILE When SHA-256 already exists for tenant SYSTEM / ingestion-cron INGESTION_IMPORT / id { originating_import_id, originating_committed_at, file_hash_sha256 }. Per V1 §C.3 — never expose contract data.
ING_COMMITTED After successful single transaction SYSTEM / ingestion-cron INGESTION_IMPORT / id { rows_created, rows_updated, rows_skipped, row_count_total }.
ING_CUSTOMER_CREATED Per new customer row (loop inside commit listener) SYSTEM / ingestion-cron CUSTOMER / customer_id { source_customer_id, name_hash, phone_hash }. PiiHasher applied (D-shared). No plaintext.
ING_CUSTOMER_UPDATED Per updated customer row SYSTEM / ingestion-cron CUSTOMER / customer_id { source_customer_id, changed_fields[] }. Old/new values not included (V1 ING §B.8 minimization: only year if DOB ever differs).
ING_CONTRACT_CREATED Per new contract row SYSTEM / ingestion-cron CONTRACT / contract_id { source_contract_id, customer_id, date_echeance, montant_du }. Amount + deadline are commercial, not PII.
ING_CONTRACT_UPDATED Per updated contract row SYSTEM / ingestion-cron CONTRACT / contract_id { source_contract_id, changed_fields[], old_values_redacted_for_PII }.

V1 ING-R-015 ING_DOB_DRIFT_DETECTED is out of v0.0.0 (DOB is optional in v0.0.0 validation). Reintroduced v0.0.2 when DOB becomes required again.

7.5 Secret management

  • MinIO root creds: env var (existing infra from TENANT S0). Per-tenant scoped users provisioned by TENANT, NOT by ING.
  • DB creds: D15 — ING runs under papillon_app role (INSERT+SELECT on audit_log, full DDL forbidden, append-only on audit_log).

7.6 Threat surfaces specific to ingestion

Threat Mitigation
Malicious file in inbox (zip-bomb, XXE) Streaming parse with hard caps (5 MB raw + 5 000 rows). XLSX read via Apache POI SAX event API (no DOM expansion). No external entity resolution.
Magic-byte mismatch (.csv w/ OOXML inside) Magic-byte check before parsing (V1 ING-R-001 alinea); reject with FORMAT_NON_SUPPORTE.
Cross-tenant key escape MinIO bucket policy scopes credentials per prefix; cron always uses iteration-bound tenantId. Code reviewer must reject any concatenation that builds a key from file content.
PII leakage via audit details Mandatory PiiHasher for name, phone. No raw DOB in audit. No row content in ING_PARSE_FAILED (only counts + row numbers + reason codes).
Forge a duplicate-file race (same hash) UNIQUE (tenant_id, file_hash_sha256) constraint at DB level; duplicate INSERT attempt fails atomically; treated as DUPLICATE path.

8. Performance & Bandwidth Budget

ING is operator-side / batch. No bandwidth budget applies.

Concern Target
Parse + validate latency ≤10 s for 5 000 rows (V1 NFR). Achieved with streaming CSV parser + Apache POI SAX for XLSX.
Commit transaction ≤5 s for 5 000 rows. Bulk INSERT with Statement#addBatch / Spring JdbcClient batched updates.
Cron iteration latency At pilot scale (≤10 tenants), full sweep < 60 s when inbox is empty; < 5 min when every tenant has a 5 000-row file. Sequential is fine.
MinIO bandwidth Stream GET (no full buffer). Files <5 MB so memory footprint negligible.
DB connection use One business connection per tenant iteration (sequential), released before moving to next. Pool size 10 already provisioned for the platform.

No caching layer needed.


9. Migration Strategy

Per D9, ING contributes only to db/migration/tenant/ (business data lives in the tenant DB).

9.1 v0.0.0 migrations

db/migration/tenant/
├── V010__ingestion_create_tables.sql           ← creates ingestion_import, ingestion_import_row, indexes
├── V011__ingestion_customer_contract_columns.sql ← adds source_customer_id, source_contract_id, last_import_id; unique indexes
                                                    (or carved into customer/contract owners' migrations — TBD with TENANT)

customer and contract base tables are created by TENANT-001 migration (TENANT owns the schema). ING adds only the dedup-key columns + unique indexes via additive ALTER. Coordinate with TENANT ARCHITECT.

9.2 Audit event catalog migration (platform DB)

db/migration/platform/:

V004__audit_catalog_add_ing_events.sql  ← extends AuditEventType enum check constraint to admit ING_* event types

This is AUDIT's migration (AUDIT-ADR-03 freezes the catalog Day 1 — but the catalog explicitly lists ING_* codes from the start; this Vxxx file just seeds them if AUDIT's enum is data-driven). Confirm with AUDIT ARCHITECT whether enum is hard-coded (no migration needed) or table-driven.

9.3 Reversibility

All v0.0.0 migrations are additive. Rollback drops the new tables + new columns. No data destruction for existing tenants (pre-v0.0.0 the platform has no tenants).

9.4 Cross-profile (SHARED vs BYO)

Same DDL applies to both, per D9. No dialect drift; v0.0.4 BYO provisioning runs the same tenant/ migration set.


10. Integration Points

Integration v0.0.0 contract Notes
MinIO S3-compatible SDK (io.minio:minio or software.amazon.awssdk:s3 against MinIO endpoint). Operations: LIST, GET (stream), COPY, DELETE, PUT (errors sidecar). Bucket name in env: MINIO_INGESTION_BUCKET=papillon-ingestion-inbox. Shared with TENANT (compliance docs) + AUDIT (export bundles).
AUDIT In-process Spring named-interface bean AuditEventBuilder. Outbox via spring-modulith-events-jdbc. D7 — audit failure doesn't block business commit.
TENANT TenantConfigCache.listActiveTenants() + findById(tenantId). Bean dependency. D-shared.
OP Publish ImportCommitted via ApplicationEventPublisher (post-commit, transactional). OP listens silently in v0.0.0; UI rendering in v0.0.1+.
CSV parser com.fasterxml.jackson.dataformat:jackson-dataformat-csv (UTF-8, delimiter auto-detect by sniffing first line for ; vs ,). Streaming; no full-file buffer.
XLSX parser org.apache.poi:poi-ooxml with SAX event mode (XSSFReader + SharesStringsTable). Hard cap iteration at 5 001 rows. Avoid DOM-mode XSSFWorkbook (memory).
Phone normalization Inline regex chain (V1 ING-R-005). libphonenumber is overkill for CI-only v0.0.0. Swap to libphonenumber-java if multi-country lands earlier than expected.

No payment provider / SMS / WhatsApp / webhook integration in ING. No CSV/XLSX file ingestion uploaded by anyone other than the cron in v0.0.0.


11. Vertical Slice Decomposition

PO input — not for DEVELOPER. The DEVELOPER reads only the story file at docs/stories/v<VERSION>/<module>/<STORY-ID>.md. This section exists for the PRODUCT_OWNER to decompose into self-contained stories.

Tier 0 — Dev environment + test infrastructure (ING-S0, must merge first)

Scope: Verify the MinIO papillon-ingestion-inbox bucket exists (provisioned by TENANT S0; if not, extend docker-compose.yml MinIO init script). Add per-tenant prefix layout (<tid>/inbox/processed/rejected/duplicate) creation helper. Add TestContainers fixture for MinIO + Postgres tenant DB. Add a sample CSV fixture (fixtures/ingestion/clean-5-rows.csv) and a tainted XLSX fixture (fixtures/ingestion/blocking-anomalies.xlsx). Add Liquibase/Flyway migration files V010 + V011 (empty bodies committed by ING-S0; ING-001 fills them).

Files: - backend/docker-compose.yml (extend MinIO init script if needed) - backend/src/test/resources/fixtures/ingestion/clean-5-rows.csv - backend/src/test/resources/fixtures/ingestion/blocking-anomalies.xlsx - backend/src/test/java/.../ingestion/IngestionTestContainersConfig.java - backend/src/main/resources/db/migration/tenant/V010__ingestion_create_tables.sql - backend/src/main/resources/db/migration/tenant/V011__ingestion_customer_contract_columns.sql

Complexity: S. Mostly file scaffolding + verifying inherited TENANT S0 infrastructure suffices.

Dependencies: TENANT-S0 (MinIO + Postgres x2 + Flyway tooling), AUDIT-S0 (TestContainers patterns to copy).

ACs: (1) ./gradlew test boots an ephemeral MinIO + Postgres tenant DB; (2) migration V010 creates the two ING tables; (3) fixture CSV/XLSX files parseable by Apache POI + Jackson CSV in a smoke test.


Tier 1 — ING-001 v0.0.0 cron + pickup + parse + validate + commit + audit + reject

Scope: The single user story per V0 PRD §8 table — implements the entire v0.0.0 ING contract above. Split candidate (5 distinct testable behaviors — see sub-slices).

Files (illustrative): - backend/src/main/java/.../ingestion/cron/IngestionScheduler.java (@Scheduled) - backend/src/main/java/.../ingestion/IngestionService.java (orchestrator) - backend/src/main/java/.../ingestion/parse/CsvIngestionParser.java - backend/src/main/java/.../ingestion/parse/XlsxIngestionParser.java - backend/src/main/java/.../ingestion/validate/RowValidator.java (V0 subset of V1 ING-R-001..016) - backend/src/main/java/.../ingestion/validate/PhoneNormalizer.java - backend/src/main/java/.../ingestion/storage/IngestionInboxClient.java (MinIO wrapper; list/get/move) - backend/src/main/java/.../ingestion/commit/IngestionCommitTransaction.java (single TX) - backend/src/main/java/.../ingestion/recover/IngestionStartupRecovery.java - backend/src/main/java/.../ingestion/audit/IngestionAuditEvents.java (event types + builders) - backend/src/main/java/.../ingestion/api/ImportCommitted.java (Spring application event) - Tests: happy / reject / duplicate / crash-recovery / feature-flag-closed / tenant-isolation / large-file

Acceptance criteria (binding for the story, derived from ING-US-01..03 cut down to v0.0.0): 1. Given a clean 5-row CSV in /<tid>/inbox/, when the cron tick runs, then within ≤15 s the rows are committed to customer + contract, ingestion_import.status=COMMITTED, ING_COMMITTED is in audit_log, and the file is in /<tid>/processed/. 2. Given a 5-row XLSX with one TELEPHONE_INVALIDE row, when the cron tick runs, then zero rows are committed, ingestion_import.status=PARSE_FAILED with parse_error_detail summarizing anomaly counts by code + first 5 row numbers, ING_PARSE_FAILED is in audit_log, the file is in /<tid>/rejected/ accompanied by a .errors.json sidecar. 3. Given a file already committed (SHA-256 matches an existing ingestion_import for the tenant), when re-dropped, then no business write occurs, a new ingestion_import row with status=DUPLICATE is inserted with originating_import_id set, ING_DUPLICATE_FILE is in audit_log, and the file is in /<tid>/duplicate/. 4. Given feature.artci.phone-auth.granted=false on prod profile, when the cron tick fires, then no MinIO LIST occurs and a [LEGAL-REVIEW] ingestion cron skipped WARN log line is emitted. Pre-prod profile (dev, test, staging) ignores the flag. 5. Given a PICKED_UP row older than 5 minutes (simulated by inserting one), when the application boots, then IngestionStartupRecovery rewrites it to PARSE_FAILED with reason="recovered_from_crash" and moves the inbox file (if still present) to /rejected/.

Estimated complexity: L. Split candidate — sub-slice boundaries proposed below.

Dependencies: ING-S0 merged; TENANT-001 merged (default agency + customer/contract base tables); AUDIT-001 merged (outbox + PiiHasher) and AUDIT catalog extended with ING_* event types (parallels OQ-X9 — folded into AUDIT-002 or a dedicated AUDIT pre-slice).

Split candidate — proposed sub-slices

Sub-slice Scope Complexity
ING-001-a IngestionScheduler + IngestionInboxClient (MinIO list/get/move) + IngestionService skeleton iterating tenants, no parse yet (just "found file, audit ING_UPLOAD_RECEIVED, move to processed"). S
ING-001-b CSV + XLSX parsers, RowValidator v0.0.0 subset, PhoneNormalizer. Pure unit-testable. No DB writes. M
ING-001-c IngestionCommitTransaction (single TX upsert + ingestion_import_row + audit + ImportCommitted). Reject path. SHA-256 dedup. M
ING-001-d IngestionStartupRecovery + feature-flag gate + structured logging. S
ING-001-e Cross-cutting integration tests: tenant-isolation, large-file (5 000 rows), magic-byte, crash-recovery, prod feature-flag closed. M

v0.0.1 → V1 forward-compat checkpoints (NOT v0.0.0 work)

Listed so PRODUCT_OWNER doesn't accidentally pull them into v0.0.0:

  • v0.0.2 ING-003: reintroduce DOB required + branche required validation + multi-agency code_agence resolution (depends on TENANT-002 agencies).
  • v0.0.3 ING-004: POST /api/ingestion/imports (multipart upload), ingestion_staged_row table, atelier endpoints (PATCH rows, commit, abandon), source-file download.
  • V1 ING-S1..S5: full V1 PRD validation set, configurable retention, history UI, template download.

12. Technical Risks

Risk Likelihood Impact Mitigation
A1. ARTCI feature-flag opens before code is ready (prod gate forgets to apply on staging-prod migration) Low High Profile-bound flag default. Smoke test (AC-4) blocks releases that don't gate correctly. Banner in startup log when flag is false.
A2. All-or-nothing reject means one bad row blocks the whole night's renewal pipeline Medium Medium Acceptable in v0.0.0 (V0 PRD §4 chose "simplified validation, manual re-drop"). v0.0.1 adds operator badge (NOTIF wireframe). v0.0.3 brings the atelier — the real fix.
A3. Cron crash leaves PICKED_UP row blocking a re-drop of the same file (SHA-256 collision with the orphaned row) Low Medium IngestionStartupRecovery (AC-5) clears stuck rows on boot. Recovery moves the file out of inbox so operator gets a re-drop signal.
A4. MinIO bucket misconfigured at TENANT onboarding — operator drops in wrong tenant's prefix Medium High MinIO ABAC scopes credentials per prefix. Operator structurally cannot LIST/PUT another tenant's prefix. Code review must reject any cron-side prefix derived from anything other than the iteration variable.
A5. Apache POI XLSX SAX mode harder to use; dev might fall back to DOM mode and OOM on a 5 000-row sheet Medium Medium Provide template XlsxIngestionParser skeleton in ING-S0 fixtures. CI test asserts heap usage stays under 256 MB on a 5 000-row file.
A6. Audit catalog extension lands after ING-001 dev-start; ING audit emissions fail validation Medium High AUDIT catalog extension is a blocker for ING-001 dev-start (parallels NOTIF OQ-X9). PRODUCT_OWNER must enforce ordering; see PROGRESS.md Open Questions registration below.
A7. OP listener races with ImportCommitted (consumes before customer/contract rows visible to its query) Low Medium @TransactionalEventListener(AFTER_COMMIT) (D4) guarantees the business commit is durable before the event is delivered. Default behaviour, but documented to OP ARCHITECT.
A8. Pilot tenant uploads a 5-million-row file (limit not enforced before parse) Low High HTTP-layer 5 MB cap is irrelevant (no HTTP upload). Cron-layer enforces: stream-count rows during parse; abort + PARSE_FAILED reason TROP_DE_LIGNES at row 5 001.
A9. Re-issued cron during DST? Africa/Abidjan is UTC+0 no-DST (D10). None Explicit zone in @Scheduled. D10 confirms.
A10. customer.source_customer_id clashes across tenants (someone forgets the UNIQUE includes tenant_id) Low High Critical rule #1 — code review. The UNIQUE is (tenant_id, source_customer_id) everywhere. A cross-tenant isolation test (Section 13) catches regressions.

13. Test Infrastructure

13.1 TestContainers stack

Inherited from TENANT-S0 + AUDIT-S0: - Postgres 16 ×2 logical DBs (platform, tenant). - MinIO with bucket papillon-ingestion-inbox pre-created and 2 tenant prefixes seeded. - Spring Boot test slice with @SpringBootTest(webEnvironment=NONE) since no HTTP. - spring-modulith-events-jdbc tables on platform DB (D4 outbox).

No new TestContainers.

13.2 Services to mock

  • Time: Clock Spring bean, fixed to 2026-05-21T02:00:00+00:00[Africa/Abidjan] in tests.
  • Scheduler: tests trigger IngestionService.tick() directly, bypassing @Scheduled.
  • No external HTTP — MinIO is real (TestContainers).
  • TenantConfigCache: real bean populated by inserting tenants in platform DB.

13.3 Sample tenant-isolation test (sketch)

@Test
@DisplayName("ING never commits rows of tenant A into tenant B even when SHA-256 collides")
void crossTenantIsolation() {
  // arrange: same file dropped by tenant A AND tenant B
  uploadFixture(tenantA, "clean-5-rows.csv");
  uploadFixture(tenantB, "clean-5-rows.csv"); // identical bytes ⇒ same SHA-256

  // act
  ingestionService.tick();

  // assert: both tenants got their own ingestion_import row (no DUPLICATE between tenants)
  assertEquals(1, countImports(tenantA, COMMITTED));
  assertEquals(1, countImports(tenantB, COMMITTED));
  assertEquals(0, countImports(tenantA, DUPLICATE));
  assertEquals(0, countImports(tenantB, DUPLICATE));

  // assert: customers landed in their respective tenants
  assertEquals(5, countCustomers(tenantA));
  assertEquals(5, countCustomers(tenantB));

  // assert: no customer row from A has tenant_id=B (the regression we're guarding)
  assertEquals(0, jdbcClient.sql("SELECT count(*) FROM customer WHERE tenant_id=? AND source_customer_id IN (SELECT source_customer_id FROM customer WHERE tenant_id=?)")
      .params(tenantB, tenantA).query(Long.class).single());
}

13.4 Sample resumability test (crash mid-parse)

@Test
@DisplayName("PICKED_UP row >5 min old is rewritten to PARSE_FAILED at boot")
void crashRecovery() {
  // arrange: insert a PICKED_UP row with picked_up_at = now - 10 min, file still in inbox
  UUID importId = insertImport(tenantA, "stale.csv", "PICKED_UP", Instant.now().minusSeconds(600));
  uploadFixture(tenantA, "stale.csv");

  // act: boot the recovery job
  startupRecovery.runOnce();

  // assert
  var row = findImport(importId);
  assertEquals("PARSE_FAILED", row.status());
  assertTrue(row.parseErrorDetail().contains("recovered_from_crash"));
  assertFileMoved(tenantA, "inbox/stale.csv", "rejected/");
  assertAuditEmitted("ING_PARSE_FAILED", importId);
}

13.5 Sample feature-flag-closed test

@Test
@DisplayName("prod profile + flag=false ⇒ tick is a no-op")
void productionFlagClosed() {
  System.setProperty("spring.profiles.active", "prod");
  System.setProperty("feature.artci.phone-auth.granted", "false");
  uploadFixture(tenantA, "clean-5-rows.csv");

  ingestionService.tick();

  assertEquals(0, countImports(tenantA, ANY));
  assertFileStillIn(tenantA, "inbox/clean-5-rows.csv");
  assertLogContains("[LEGAL-REVIEW] ingestion cron skipped");
}

14. Local Dev Environment

ING inherits the existing docker-compose.yml from TENANT-S0 + AUDIT-S0 + NOTIF-S0. No new container needed.

14.1 Services (relevant subset)

Service Image Purpose for ING
postgres-platform postgres:16-alpine Audit + spring-modulith outbox + ingestion_import (NO — ING tables on tenant DB). Used for outbox only.
postgres-tenant postgres:16-alpine ingestion_import, ingestion_import_run, customer, contract.
minio minio/minio:latest Bucket papillon-ingestion-inbox.
keycloak quay.io/keycloak:26.6 Not used by ING in v0.0.0 (cron is SYSTEM).

14.2 Env vars (added by ING; merge into .env.example)

MINIO_INGESTION_BUCKET=papillon-ingestion-inbox
INGESTION_CRON_EXPRESSION=0 0 2 * * *
INGESTION_CRON_ZONE=Africa/Abidjan
INGESTION_MAX_FILE_BYTES=5242880
INGESTION_MAX_ROWS=5000
INGESTION_STARTUP_RECOVERY_STALE_THRESHOLD_SECONDS=300
FEATURE_ARTCI_PHONE_AUTH_GRANTED=false   # production gate; dev/test/staging ignore

14.3 Smoke verification command

# 1. boot the stack
docker compose up -d postgres-platform postgres-tenant minio

# 2. run migrations + create bucket (handled by Spring Boot on first start)
./gradlew :backend:bootRun --args='--spring.profiles.active=local'

# 3. drop a fixture file
mc alias set local http://localhost:9000 minio minio12345
mc cp backend/src/test/resources/fixtures/ingestion/clean-5-rows.csv \
  local/papillon-ingestion-inbox/00000000-0000-0000-0000-000000000001/inbox/clean-5-rows.csv

# 4. trigger the cron (test endpoint exposed only in 'local' profile)
curl -X POST http://localhost:8080/internal/ingestion/tick

# 5. verify
psql -h localhost -p 5433 -U papillon_app -d papillon_tenant \
  -c "SELECT status, rows_created, finished_at FROM ingestion_import ORDER BY picked_up_at DESC LIMIT 1;"

mc ls local/papillon-ingestion-inbox/00000000-0000-0000-0000-000000000001/processed/
# expect: clean-5-rows.csv

Appendix A — V0 PRD §4 ING reconciliation matrix

V1 ING PRD § v0.0.0 status in this architecture
A.3 US-01 (upload) Cron pickup, not UI. Same acceptance criteria semantics (hash dedup, parse, staged or rejected).
A.3 US-02 (atelier) Deferred to v0.0.3. Replaced by all-or-nothing reject + sidecar .errors.json.
A.3 US-03 (commit) Auto-commit on clean parse (no manual "Confirmer" button — there's no UI).
A.3 US-04 (history) Deferred to v0.0.3 (no UI in v0.0.0). Data captured though (ingestion_import rows persisted).
A.3 US-05 (template DL) Deferred to v0.0.3.
A.4 NFR (parse latency, retention) Honored. Retention of source files = 12 months (MinIO lifecycle policy at bucket level — V1 detail).
B.1 ING-R-001..016 R-001 (formats), R-002 (limits), R-003 (dedup), R-004 (header model — relaxed: only core columns required), R-005 (phone normalize), R-008 (montant), R-010 (id/contract/name required), R-011 (intra-file dup), R-012 (inter-file dup key), R-014 (all-or-nothing), R-015/016 (upserts). Skipped in v0.0.0: R-006 (DOB required), R-007 (date_echeance future + ECHEANCE_DEPASSEE — partially: only required + parseable, not the >30-day-past rule), R-009 (agency resolution), R-013 (optional-field NON-BLOQUANT — every field optional except core), R-017 (operator role check), R-018 (staging expiration — no staging in v0.0.0), R-019 (badge — no UI).
B.5 schema ingestion_import minus committed_* extras (kept); ingestion_staged_row not created until v0.0.3; ingestion_import_row kept minimal (no anomalies_at_commit until v0.0.3).
B.6 multi-tenant Honored 1:1.
B.7 deps TENANT consumed for tenant list. AUDIT consumed for events. OP consumes ImportCommitted. No AUTH dep in v0.0.0 (cron is SYSTEM).
B.8 audit events Subset shipped: ING_UPLOAD_RECEIVED, ING_PARSE_FAILED, ING_DUPLICATE_FILE (new — v0.0.0 specific), ING_COMMITTED, ING_CUSTOMER_CREATED, ING_CUSTOMER_UPDATED, ING_CONTRACT_CREATED, ING_CONTRACT_UPDATED. Skipped: ING_STAGED, ING_ROW_CORRECTED, ING_ABANDONED, ING_STAGING_EXPIRED, ING_DOB_DRIFT_DETECTED.
C.1 conformité ARTCI B5 production gate via feature flag (feature.artci.phone-auth.granted=false) honored. CIMA Reg. 01-24 prestataire-technique stance unchanged.
C.3 sécurité TLS to MinIO, magic-byte check, no PII in error messages, no rate-limit needed (cron is SYSTEM not user-driven), per-tenant MinIO prefix scoping.
C.4 connectivity Operator-side / system-side only. Not low-bandwidth.
C.5 hors V1 Not in scope for v0.0.0.

Appendix B — ADR summary

ADR-ID Decision Linked PROGRESS.md decision
ING-ADR-01 v0.0.0 = all-or-nothing reject; no atelier. Operator feedback is the sidecar .errors.json in /rejected/. — (V0 PRD-derived)
ING-ADR-02 File source = platform MinIO bucket papillon-ingestion-inbox, per-tenant prefix; no customer-owned S3 in v0.0.0. reuses TENANT MinIO
ING-ADR-03 Cron = single Spring @Scheduled task iterating active tenants sequentially. No ShedLock yet (single-instance deploy in v0.0.0). D10
ING-ADR-04 ING tables on tenant DB (business data) per D1. Audit lives on platform DB per D12. D1 + D12
ING-ADR-05 All-or-nothing commit in a single DB transaction (V1 ING-R-014 semantics).
ING-ADR-06 Idempotency via SHA-256 dedup + per-file FSM. No outbox / queue beyond spring-modulith for cross-module events. D4
ING-ADR-07 XLSX parsing via Apache POI SAX (no DOM). CSV via Jackson CSV streaming.
ING-ADR-08 Production gate via feature.artci.phone-auth.granted (default false on prod profile, ignored elsewhere). — (V0 PRD §6.4)
ING-ADR-09 correlation_id for ING audit events = import_id (batch scope, NOT renewal chain). OP starts a fresh correlation_id when consuming ImportCommitted. D16 (does not extend chain to ING)
ING-ADR-10 Schema-stability: v0.0.0 tables are forward-compatible; v0.0.2 / v0.0.3 add columns + new tables (ingestion_staged_row) additively. AUDIT-ADR-03 pattern

Appendix C — Cross-module asks (must land before ING-001 dev-start)

Ask Owner module Tracking
Extend AuditEventType enum (or seed table) to admit ING_UPLOAD_RECEIVED, ING_PARSE_FAILED, ING_DUPLICATE_FILE, ING_COMMITTED, ING_CUSTOMER_CREATED/UPDATED, ING_CONTRACT_CREATED/UPDATED. AUDIT OQ-X10 (new)
Provision MinIO bucket papillon-ingestion-inbox + per-tenant prefix layout at tenant onboarding. Issue per-tenant scoped credentials. TENANT OQ-X11 (new)
Confirm customer and contract base table ownership and the additive ALTER strategy for source_customer_id/source_contract_id/last_import_id columns + unique indexes. TENANT OQ-X12 (new)
OP confirms consumption of ImportCommitted {importId, tenantId, rowsCreated, rowsUpdated, rowsSkipped, occurredAt} in v0.0.0 (listener may be a no-op until v0.0.1 UI lands). OP OQ-X13 (new)

End of document. ARCHITECT session 2026-05-21. v0.0.0 scope only.