AI

What it costs to run an AI agent and how to keep that cost under control

delpra™ team6 min read

An AI agent looks cheap in testing: a few conversations, a few cents. The surprise comes with volume. Without measurement, the company finds out at the end of the month that it has a bill nobody can explain. With measurement, cost becomes a predictable number per conversation, one you can compare with the value the agent delivers.

How AI is billed

Language models charge by tokens, which are pieces of words. Everything the model reads counts as input, and everything it writes counts as output. Output usually costs more than input, and more capable models cost more per token than smaller ones.

The point that confuses people most: with every customer message, the model does not read just the new message. It rereads everything it needs to answer:

  • the agent's instructions (who it is, how it speaks, what it can and cannot do);
  • the relevant knowledge base;
  • the description of the tools it can use;
  • the conversation history up to that point.

In a well-built agent, this fixed context is far larger than the customer's message. It is what dominates the bill.

What goes into the cost besides the model

The model's cost is only one line. An agent's total cost includes:

  • Channel. On WhatsApp through the official API, the platform itself charges fees that vary by conversation type and by country.
  • Other AI services. Audio transcription, knowledge base search, classification.
  • Infrastructure. Server, database, monitoring.
  • Maintenance. Someone follows the conversations, adjusts the rules and updates the knowledge base. It is the most forgotten item.

Prices change often. Always check the vendor's official price list before doing the math, and be suspicious of any fixed number you read elsewhere, including in articles.

What makes the cost climb without anyone noticing

  1. Instructions that keep growing. Every adjustment becomes one more paragraph. In a few months, the agent rereads an enormous text with every message.
  2. The whole knowledge base sent every time. Sending all the documents in every conversation, instead of only the relevant passage.
  3. Long conversations. The history grows and is reread on every turn.
  4. A large model for a small task. Using the most expensive model to label a message as "question" or "complaint".
  5. Repeated calls. The agent queries three tools and the model is called again after each result.
  6. Messages that should never reach the model. Stickers, "ok", "thanks".

How to control it

Measure per conversation

Record, for each conversation, how many tokens went in and out, which model was used and what it cost. Without that measure, everything else is guesswork. With it, you find out which types of conversation cost more and why.

Use context caching

The main vendors offer caching: when the beginning of the context repeats from one call to the next, that part is billed at a discount. Since instructions and the knowledge base repeat in every message, the effect can be large.

To take advantage of it, order matters. The part that does not change comes first and must be identical on every call. What varies, such as the date, the customer's name and the new message, goes at the end. A small detail, like putting the current time at the top of the instructions, is enough to break the cache. The exact rules and the size of the discount vary by vendor and change over time, so it is worth reading the documentation.

Pick the model by task

Triage, classification and short summaries work well on smaller, cheaper models. The most capable model is kept for the conversation itself, where the quality of the answer is what the customer notices.

Send only what is needed

Instead of sending the whole knowledge base, retrieve the passages related to the question. Instead of carrying the full history of a long conversation, summarize what has already happened. We explain how to organize the knowledge base in How to train an AI agent on your company's knowledge base.

Filter before the model

Simple rules handle what does not need AI: an empty message, a repeated greeting, a request to stop receiving messages. Every call avoided costs zero.

Set limits

A daily spending cap, a limit on turns per conversation and an alert when the average cost leaves the expected range. A limit is not distrust of the agent. It is the same care you apply to any variable expense.

You do not control AI cost by switching vendors with every new release. You control it by designing the system to reread less and measure everything.

How to know whether it is worth it

Cost per conversation only makes sense next to what the conversation delivers. Compare it with:

  • the time a person would need to handle the same conversation;
  • the contacts that today go unanswered at night and on weekends;
  • the speed of the first response, which weighs on qualification. A study published in Harvard Business Review in 2011 (Oldroyd, McElheran and Elkington) showed that companies that tried to make contact within 1 hour were nearly 7 times more likely to qualify the lead than those that tried 1 hour later.

There is no universal number. A conversation that costs little and solves nothing is expensive. One that costs more and hands the salesperson a qualified contact can be cheap. The math has to be done with your own data.

An example to picture it

Imagine a language school that puts an agent in charge of answering prospective students. In the first month, the bill is alarming. Looking at the per-conversation log, the team sees that the agent rereads the entire course catalog with every message and that today's date sits in the first line of the instructions. By reorganizing the context so the fixed part comes first and retrieving only the course that was asked about, the cost per conversation drops, with no change of model and no change in the quality of the answers. How much it drops depends on each case. The path is the same.

Where to start

If you already have an agent, turn on token and cost-per-conversation logging today. Within a week, the three biggest culprits show up on their own. If you are still going to build one, put measurement in the scope from day one, along with the rules we describe in AI agents on WhatsApp.

AI agents with cost measured per conversation are part of delpra™'s services.

Written by the delpra™ teamProcesses, automation and AI so operations scale efficiently.

How mature are your operations?

The Operational Maturity Assessment takes 4 minutes and shows where to start.

Take the free assessment

Read next