Skip to content
, Home

AI Agent Quality & Evaluation

Process mapping
before anything is automated.

Process mapping writes down how a piece of work runs today: each step, the systems it touches, who does it and where it waits. It matters when you want to automate work that nobody has described from start to finish.

The real question

Whether the process you want to automate is the one your team actually runs.

Step 01 of 07 · AI Agent Quality & Evaluation

The path

From process map
to release decision.

Seven steps, each feeding the next: the first two automate the work, the other five verify it. This page is one of them.

  1. Process mapping (this page)
  2. Agent & automation build
  3. Agent evaluations
  4. Workflow and handoff testing
  5. Deterministic checks
  6. Evaluation harnesses
  7. Release criteria

Running exampleAn illustration, not a case study.

A support agent is asked to refund a duplicated charge. The map shows how a duplicated charge is handled today: who spots it, which systems they open, who can approve a refund and what the customer is told at each point.

01Questions

The questions
it answers.

  • What happens, step by step, from the moment the work arrives?

    Each step is written as who does it, in which system, with what input, and what it hands to the next step.

  • Where does the work wait?

    Queues, approvals and lookups are marked, because the hours usually go in the waiting, not in the doing.

  • Which steps follow a rule, and which need judgement?

    A step that follows a rule can be automated outright; a step that needs judgement is a candidate for an agent, with a person approving.

  • What would a mistake cost at each step?

    Each step is rated by what goes wrong if it runs without a person: a wrong message, money moved, a record changed or a promise made.

02Pitfalls

Where it
usually breaks.

  • Mapping the process as the procedure describes it

    The written procedure leaves out the workarounds people rely on. The automation follows the document and breaks on the first case the team has always handled by habit.

  • Automating the rare steps first

    The steps that are interesting to build get built, while the dull step the team repeats all day stays manual. The build is finished and the hours are still there.

  • No owner for the exceptions

    The map covers the normal path. When a case falls outside it, nobody has decided who picks it up, so it waits in a queue nobody watches.

  • No record of how long the work takes today

    Nobody wrote down the time each step takes before the change. Afterwards there is no way to say whether the automation saved anything.

03Approach

How I work
on it.

  1. Follow real cases, not the procedure

    With the people who do the work I follow real cases through each system and write down every step, wait and workaround as it happens.

  2. Rate each step

    For each step I note how often it happens, how long it takes and what a mistake costs, and whether it follows a rule or needs judgement.

  3. Agree what to automate first

    With you I pick the steps where the most time comes back and a mistake is cheapest to catch, and agree where a person keeps the decision.

What you leave with

  • A map of the process as it runs today: each step, the system it happens in, who does it and what it hands on.
  • A note for each step of how often it happens, roughly how long it takes and what a mistake costs.
  • A shortlist of steps to automate first, with the reason for each, and the steps that stay with a person.
  • A baseline of time and errors recorded before anything changes, so the effect of the automation can be measured.

04Measures

What I
would measure.

Signals to read, not targets to hit: what counts as good enough is your team’s decision.

  • Cases that leave the mapped path

    A high share means the map describes the ideal rather than the work; each such case is a step or an exception the map is missing.

  • Time spent waiting between steps

    Where it outweighs the work itself, an approval or a handoff is the first thing worth changing, before anything is automated.

  • Steps nobody can explain

    A step kept because it has always been done is either a control someone relies on or work that can go; the map should say which.

Where it sits in the assessment.

The Quality Maturity Assessment scores nine dimensions. These are the ones this work moves.

Open the assessment

A process to automate, or
an agent to check?

Describe the process or the workflow, what it should achieve and where it goes wrong today.