Automated Processing Architecture•Timeline: 3 weeks module integration

Deterministic AI Data Pipeline

Designed an architectural pipeline where LLMs propose structured candidates into an isolated staging schema. A deterministic rule engine executes mathematical reconciliation, type guards, and duplicate checks before any row can be promoted to the production ledger.

PythonPydantic v2PostgreSQL RLSFastAPIGemini API

01. Client Context

Operations requiring automated document parsing and structured data extraction from unstructured supplier receipts and shipping manifests.

02. The Problem in the Client's Words

"We tried using AI to extract invoice line items directly into our ERP. But the model would occasionally invent a date format, misread decimal commas, or hallucinate a line item that wasn't on the physical paper. One bad row broke our month-end audit."

03. Constraints & Realities

Zero corrupted rows permitted in the core production schema.
Must process mixed scanned PDFs, photographed receipts, and email attachments.
Cost must remain under $0.05 per processed document.

04. Solution & Architecture Blueprint

Data flow showing intake, deterministic validation barriers, and permanent double-entry audit logging:

Architecture Blueprint: Deterministic Agent Staging PipelineZero Direct Database Writes

05. Data & Security Design

Access Rules (RLS)

Model service accounts only have INSERT permissions on the `staging_inbox` table. They have ZERO write or update permissions on the production ledger.

Audit Trail

The raw input document hash, model prompt tokens, structured extraction JSON, and validation test results are permanently saved in an audit log.

Staging & Validation

Rigorous validation gate: sum of extracted items must exactly equal invoice total; VAT/tax percentages must match registered rates; supplier tax ID must match active database records.

06. Measured Results (Truth Gate Verified)

Backed by facts.json

99.4%

Eval Suite Pass Rate

Benchmark test against 2,500 historical invoices with verified human labels

Period: Test evaluation run across 2,500 records

$0.008

Execution Cost per 100 Invoice Lines

Token cost measured on Gemini Flash / Claude Sonnet with schema compression

Period: Evaluated across 50 production test batches

0 rows

Corrupted Rows Committed to Production

Database integrity assertions and constraint enforcement

Period: 100% of processed records

07. What Went Wrong & What I Changed

Candid Engineering Retrospective

Early versions attempted to make the model "fix" its own mathematical errors in a loop, which drove up token costs without improving accuracy. Replacing the model loop with deterministic Python math validation cut costs by 70% and made the system 100% reliable.

09. What I'd Do Differently

Never ask an LLM to perform arithmetic that a 3-line Python function can execute deterministically in 0.1 milliseconds.

Need a system engineered to this standard?

Book a 20-minute discovery call to review your operational bottlenecks.

Book a 20-minute discovery call