Skip to main content
Simulation tests a chat agent against many conversations at once. Instead of typing every scenario into the Test Chat yourself, you describe a customer, and an AI plays that customer against your agent — as many times, and across as many versions, as you want. Each conversation is then graded against the success criteria you set.
Available for WhatsApp agents of type Basic or Advanced. Chat Center Workflows, widget flow agents and Instagram agents do not have the Simulation tab.

​
Where to find it

Open the agent and switch from Create to Simulation in the toggle at the top of the builder. The Simulation page has two tabs:
  • Test Cases — the scenarios you have written for this agent.
  • Batch Testing History — every batch you have run, with its results.

​
Create a test case

Click Add Test Case and fill in:
Click Generate with AI and describe the scenario in a sentence — e.g. a lead who is already registered asks for the payment link. It fills in the name, persona, prompt and success criteria from your agent’s configuration. On an existing test case the button becomes Edit with AI, for changes like make the lead push back on price twice.

​
Function mocks

Under Test Variables & Mocks, the Function Mocks tab lists the agent’s custom functions. Set the JSON each one should return during the simulation — no real API is called. A function left empty returns:
Mock the responses your scenario depends on. To test what the agent does when an order is not found, make the order-lookup function return that.

​
Run a batch

1

Pick the test cases

Click Test on one test case, or tick several and click Run Selected Test.
2

Name the batch

So you can find it again in the history, e.g. Pricing objections, v3.
3

Choose the versions

Defaults to the version open in the builder. Select more than one to compare versions on the same scenarios.
4

Set the epochs

How many times each test case runs on each version. The AI customer does not say the same thing twice, so several epochs show how consistent the agent is.
5

Start

A batch runs test cases × versions × epochs conversations — at most 100 epochs and 500 runs per batch. Nothing is sent and nothing is saved to your contacts. Simulations run in test mode, like the Test Chat: no WhatsApp message goes out and no CRM record is changed.

​
When a conversation ends

Each conversation runs until the agent ends it, the customer ends it, the agent and customer start repeating each other, or it reaches 20 turns.

​
Read the results

Batch Testing History lists your batches. Select one to see its pass rate, its passed, failed and grading-failed counts, and every run in it: Click a run to open the full result:
  • Agent Performance — how the agent handled the conversation.
  • Simulated User Performance — how closely the AI customer followed the persona and prompt. If this is poor, fix the test case before blaming the agent.
  • Success Flaws — what stopped the run from fully meeting the success criteria.
  • Transcript — the whole conversation, with the functions the agent called and, for Advanced agents, the flow step each reply came from.
The cost of each run, and of all batches together, is shown alongside the results.

​
Tips

A test case that checks pricing, rescheduling and opt-out at once fails without telling you which part broke. Split them.
“Shares the payment link and does not offer a discount” can be checked against a transcript. “Handles the customer well” cannot.
Run the same test cases on the published version and your new one. If the new one passes less often, you have found a regression before your customers did.
One run that passes can be luck. Five runs that all pass is a result.

Testing a chat agent

Try one conversation by hand in the Test Chat

Versions and publishing

Test on a version, publish when it is right