Building Low-Latency AI Infra & Stateful Multi-Agent Backends

Commercial B2B system that reads CSV exports from Shopify, Amazon, Meta, and Google and rebuilds one consistent view of orders, inventory, ad spend, and customers. Every figure traces to its source file, row, and column via SHA-256 hashing and cell lineage. Nine leak detectors sit behind a common interface, and language-model answers must match engine-computed numbers within 0.5% or are replaced. Seven FastAPI routers, 200+ Pytest cases; within 2% of the answer key on a 9,578-error synthetic benchmark.

Runs each task in a disposable Docker container with all Linux capabilities dropped, no-new-privileges, and limits of 1 GB memory, 2 CPUs, and 256 processes. Redis/arq worker queue, WebSocket streaming of task output, and a per-task event log stored in the database. 40+ unit and integration tests.

LangGraph pipeline with intent routing, question condensing, retrieval evaluation, and query-rewrite and web-search fallbacks. State is checkpointed in PostgreSQL and answers stream over Server-Sent Events. Retrieval combines pgvector with BM25, reranks with a CrossEncoder, and filters by user and thread metadata. Traced with Langfuse and scored with RAGAS.

Single-binary Rust CLI with a multi-turn REPL and tool calling (document search, web search, file read/write, shell commands), gated by ReadOnly, Safe, and Yolo execution modes. Generates embeddings locally with ONNX models, so indexing and search need no network call, and falls back to a second Gemini model when the primary fails.