Skip to content
Ben Draffin

Talk · Incrementally specified agents

Specify narrowly enough and your agents become code.

A human's job is the wrong abstraction for an agent. Decompose the work into function signatures, check the definitions into a repo, and keep narrowing until the agent is boring enough to delete.

Ben Draffin · based on the talk Incrementally Specified Agents

Try the walkthrough

01Wrong layer

“AI for security” is the wrong layer of abstraction.

It is tempting to build an agent scoped to a person. Make an agent for Ben. Ben updates tickets, opens pull requests, reviews designs, chats in channels, nudges people to acknowledge the security policy. All of it looks like one job from the outside.

It is not one job. Scope an agent to a human and you get an arbitrarily goal-seeking system that is uncomfortable to reason about and hard to monitor. You wanted a preference engine for a role. You got a snowball of responsibilities with no frozen boundary.

02Decompose the person

What is an agent scoped to, if not a job title?

Pull apart one person's work and you find a pile of narrow use cases. Each one is a candidate agent. Together they explain why “Ben-bot” never stays small.

  1. 01

    Ben as access-review agent

    Who still has access, who should not, and what the ticket says.

  2. 02

    Ben as system-design-review agent

    Threat model the design before it hardens into production debt.

  3. 03

    Ben as PR-review agent

    Read the diff, check the security-relevant edges, leave a useful comment.

  4. 04

    Ben as compliance-roadmap agent

    Turn control gaps into sequenced work someone can actually ship.

  5. 05

    Ben as finally-get-people-to-acknowledge-the-security-policy agent

    The unglamorous half of the job: follow-ups until the checkbox is real.

03Agent as interface

Model an agent as a function signature plus an instruction guide.

Narrow the snowball. Pick one goal you can monitor. Then define the specific inputs, outputs, targets, scopes, tools, and checks required to accomplish it. That package — the signature plus an agent operating procedure — is the agent.

Write it in a marked language that says what the agent does, why it does it, what it may touch, and what success looks like. The shape is approximately the same as a programming-language function signature. That is the point: you can reason about it the way you reason about code.

Agent definition

agent:    <name>
purpose:  <one sentence>
inputs:   (repo:HEAD:github, SEC-123:read:linear, …)
outputs:  (pr:code:github, SEC-123:write:linear, …)
target:   <success condition>
scope:    <boundary>
tools:    <capability:action>
checks:   <pre / post condition>

04Check them in

Permission changes for an agent become a diff.

Put the definition files in a repo. Then you can assess them, review them, and freeze permissions in time. When an agent needs a new tool or a wider scope, that is a pull request — not a silent expansion of autonomy.

UIs and other tooling can sit on top of the same repo structure. The source of truth stays inspectable.

05Progressive specification

Explore the domain until the work unit is specified enough to become code.

  1. 01

    Unspecified traffic

    Messy requests, broad goals, and agents that try to do the whole job.

  2. 02

    Specified agent

    A narrow signature, frozen tools, and an operating procedure you can review.

  3. 03

    Deterministic code

    When the same goal no longer needs a model, replace the agent and keep the interface.

You are progressively building tacit knowledge about your system's processes and problems — and freezing that knowledge into narrower permissions.

06Objections

Cost, dispatch, and the hard problems.

It's expensive

Maybe — but simpler agents carry less context. Run them in narrowly defined environments, on smaller models, even on endpoints. As you push work toward deterministic components, the expensive generalist shows up less often.

Dispatch is hard

Finding the right agent is a service-discovery problem, and that problem is already solved in other forms: registries, interface definitions, function signatures. A small intent model plus caching is often cheaper than one high-context agent that pretends to know every route.

Architecture is hard to reason about

That is the complaint about a swarm of goal-seeking agents. Narrow definitions, repo-checked permissions, and progressive replacement with code are how you keep the architecture inspectable.

07Escalation

Kick novel problems up. Decide, or ask a human.

When none of the narrow agents can solve the problem, escalate to a high-context agent on a stronger model — the expensive generalist that can do almost everything, with all the usual cost and risk. Use it for the novel cases. Let security and product review those escalations asynchronously where you can.

And sometimes the right next step is not another agent. Just decide. Or ask a human. You cannot review a goal. You can review a definition, a diff, and an outcome.

08We've done this before

The industry already walked this path.

  1. monolith
  2. services
  3. contracts
  4. agents

Monoliths became services. Services got contracts. Agents are next in that sequence — but only if you specify them the way you learned to specify everything else.

Thesis

A human's job is the wrong abstraction, and your agents are doing too much.

Specify narrowly enough and your agents become simpler — and eventually code. The best agent is the one you deleted.

Try the walkthrough

draffin.ai/incremental