Writing
Notes on building reliable AI systems
Structured outputs, production LLM pipelines, and the MLOps plumbing in between — the unglamorous layer that decides whether a model ships.
Dead-Letter Queues for LLM Extraction Failures: Capture, Triage, and Replay Without Losing Trust
A validation failure is a record your system does not yet know how to trust. Dead-letter paths preserve evidence, support human review, and make replay safe.
Constrained Decoding vs Post-hoc Validation: Production LLM Extraction Needs Both
Constrained decoding reduces how often a model wanders off-shape. Post-hoc validation decides what your system is actually allowed to trust. Production extraction pipelines need both.
Zero Round-Trips: Cleaning Noisy LLM and OCR Output Without Calling Another Model
The common fix for a malformed LLM response is to send it back to the model to repair. That's a latency, cost, and reliability tax you don't need to pay. Here's the zero-round-trip approach behind confident-extract v0.1.0.
Synthetic Data Won't Save Your Eval Set
The hard part of automated prompt optimization was never the optimizer. It was building eval sets that don't lie to you — and synthetic data is very good at telling you exactly what you want to hear.
Schema Drift Is the Silent Killer of LLM Pipelines
Nothing throws. No alert fires. But the shape of your extractions has quietly changed, and downstream systems have been ingesting subtly wrong data for weeks. That's schema drift, and it's the failure mode nobody instruments for.
The 200ms You Can't See: Profiling Latency in LLM Document Pipelines
When an LLM pipeline feels slow, everyone blames the model. Usually the model is the fastest part. The latency is hiding in the orchestration layer — and it's measurable if you instrument the right seams.
msgspec, Pydantic, or dataclasses? Picking a Validation Layer for LLM Output
The validation layer in an LLM pipeline runs on every single record, so its overhead is a real budget line. Here's how I think about msgspec vs Pydantic v2 vs dataclasses for that job.
Put a Confidence Score on Every Extraction
A parse either succeeds or throws — which tells you nothing about the records that succeeded but shouldn't have. A confidence score per extraction turns 'trust everything vs review everything' into triage that scales.
Backpressure and Retry Budgets for LLM Services
LLM calls are slow, rate-limited, and fail in bursts — the exact conditions under which naive retries and unbounded queues turn a bad minute into an outage. Here's how to make an LLM service degrade gracefully.
AI Explains, Backend Computes: The Guardrail Architecture I'm Building Into SeerFlow
I'm building SeerFlow, a cash-flow intelligence platform for Indian D2C brands. Its strictest rule has nothing to do with models and everything to do with trust: AI explains, the backend computes, and a guardrail rejects any AI output that invents a number.
Beyond Prompt Engineering: The AI Systems Layer Production LLM Apps Need
Prompt engineering starts the prototype. AI systems engineering is the layer that turns a model capability into a reliable product through contracts, validation, observability, and bounded failure handling.
Deterministic Structured Outputs for Production LLM Pipelines
Most LLM demos fall apart in production because the output contract is an afterthought. Here's how I make structured output deterministic — schema-constrained generation, strict validation, and the operational discipline that separates a demo from a system.