AI agents in practice: what actually changed for our customers

This file is an mdx file because of the added FaqItems. It is also available in Dutch.

4 min read708 words
An abstract illustration of connected automated workflows

This post has FaqItems and an HowTo that can be placed anywhere in the article.
It also contains an in-body image, which is stored in the blog folder. Astro handles the optimization of these, but we need to check if CloudFlare handles it well enough too.

  • Language selector
  • Language routing
  • SiteMap update
  • BreadCrumb
  • FaqItems
  • Howto
  • Wordcount
  • Readtime indication
  • scroll progress indication
  • Table of Contents in aside
  • CTA block in aside
  • Social sharing (& open graph in meta - still need to set user handle for X)
  • CTA banner
  • Blog Teaser (related content)
  • Tag routing
  • Author routing

Routing beat resolution

The pitch for agents is usually full resolution — the agent handles the request end to end, no human involved. That’s the part everybody wants and the part that works least often.

What worked immediately was routing. Deciding which team, which priority and what context to attach is a narrow, well-defined judgement, and it’s the step that was quietly costing the most time.

A request flowing through classification, enrichment and routing to the correct queue

One team measured it: average time from ticket creation to reaching the correct owner went from just over four hours to under ten minutes. Nothing was resolved automatically. It just stopped sitting in the wrong queue overnight.

The context problem is the whole problem

An agent with no context guesses. An agent with too much context is slow and expensive.

The teams that got good results spent most of their setup time on this and almost none on prompt wording:

  1. Decide what the agent genuinely needs to see
  2. Make that available in a form it can query, not a wall of text
  3. Log what it looked at, so a wrong answer is explainable

Step three is the one people skip and later regret. An automation you can’t audit is one you’ll eventually turn off, because the first time it’s confidently wrong nobody can explain why.

Confidence thresholds are a product decision

Every agent can tell you how sure it is. What you do with that number isn’t a technical question.

Set the bar high and the agent handles very little, but is almost never wrong. Set it low and coverage jumps along with the error rate. There is no correct threshold — there’s the one that matches how expensive a mistake is in your context.

The teams that got this right treated it as a dial they revisit monthly, not a setting they configure once.

Common questions

Do FAQ items count for the word count?

Unfortunately, it doesn't at the moment...

How long before we see anything useful?

For routing and classification, usually two to three weeks. Full resolution workflows take considerably longer, mostly because the context work described above is slower than the setup.

What happens when the agent gets it wrong?

It should fall back to a human with its reasoning attached, not fail silently. If you can't see what the agent looked at and why it decided what it did, that's the thing to fix before widening its scope.

Do we need to replace our existing tools?

No, and the teams that tried to do both at once had the hardest time. Add agents alongside what you already run, prove one workflow, then decide what else changes.

How to start

  1. Pick one decision, not a workflow

    Find the single judgement your team makes most often and agonises over least. Not the impressive one — the dull one that happens two hundred times a week.

  2. Give the agent only what it needs

    Work out what it genuinely has to see, and make that queryable rather than pasting in a wall of text. This is where the time actually goes.

    A request flowing through classification, enrichment and routing to the correct queue

  3. Log what it looked at

    Record the inputs, not a summary. The first time it’s confidently wrong, that log is the difference between a five-minute fix and a week of guessing.

  4. Set a threshold, then revisit it

    Decide how sure the agent has to be before it acts alone, and treat that as a dial you check monthly — not a setting you configure once.

Where we’d start again

If we were setting this up from scratch tomorrow: pick the single highest-volume, lowest-judgement decision your team makes. Automate only that. Instrument it properly. Then move.

The temptation is to start with the impressive workflow. The impressive workflow is where projects stall.

About the author

Sanne

Customer Success Lead

Works with the teams rolling Resolveo out day to day, which mostly means seeing what breaks between a demo and a real workflow. Writes about implementation, adoption and the unglamorous parts of automation that decide whether it sticks.

Put an agent on your busiest workflow

Start with the requests that arrive every day and go to the wrong place. We'll have it routing, enriched and measured inside a fortnight.

Book a demo

Free 7-day trial

Related posts

Keep reading