7 posts
Repeatable tasks, expected end states, adversarial cases and regression tests. We ran 180 agent runs: an answer-only judge missed 18 failures, and a better prompt made the agent less safe.
Approval gates for deleting files, changing access, deploying code, sending messages and admin commands. We tested 6 gate designs on 77 actions; letting the AI decide went badly.
We built an AI code reviewer with static checks, sandboxed tests and local models, planted 10 bugs, and measured who caught what. The AI found nothing the tools missed.
Classify IT tickets, search only approved knowledge, draft replies for agents and escalate anything uncertain. Built locally and tested on 60 labelled tickets.
Selection, retrieval, memory, freshness, permissions and context limits, tested on a real model. Five facts beat 120, and stale facts fooled it every time they came first.
We built a single agent, a four-agent team and a plain workflow for the same task, then measured accuracy, time, tokens, failures and complexity. More agents didn't win.
Beyond a basic RAG tutorial: parse PDF, Word and HTML, enforce who can read what, cite every answer, say 'not found', and measure it. Built and tested locally.
