Engineering in practice
Case Studies
Production engineering stories that follow a system question through evidence, implementation decisions, and tradeoffs.
Self-hosted AILocal LLMs · Agent workflows
Evaluating a Self-Hosted LLM for Real Development Work
Leadership wanted to know whether a model in our own data center could do coding work we'd otherwise pay a cloud AI for. It only finished once the task was cut down to a single unit test, and even then took 13 minutes to the cloud model's one.
Inside the case: each scope cut and why, a same-prompt comparison on time, quality, and cost, and the midnight-UTC flake a blind review caught
Selected investigations from my work on production software. System details are generalized where appropriate.