> ## Documentation Index
> Fetch the complete documentation index at: https://docs.callkaro.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Chat Simulation

**Simulation** tests a chat agent against many conversations at once. Instead of typing every scenario into the Test Chat yourself, you describe a customer, and an AI plays that customer against your agent — as many times, and across as many versions, as you want. Each conversation is then graded against the success criteria you set.

> [!NOTE]
>
> Available for **WhatsApp** agents of type **Basic** or **Advanced**. Chat Center Workflows, widget flow agents and Instagram agents do not have the Simulation tab.

## Where to find it

Open the agent and switch from **Create** to **Simulation** in the toggle at the top of the builder. The Simulation page has two tabs:

* **Test Cases** — the scenarios you have written for this agent.
* **Batch Testing History** — every batch you have run, with its results.

## Create a test case

Click **Add Test Case** and fill in:

| Field | What it is |
| - | - |
| **Name** | A short name for the scenario, e.g. *Asks for payment link*. |
| **Primary User Model** | The model that plays the customer. |
| **Secondary User Model** | Optional. Used if the primary model fails to reply. |
| **User Persona** | Who the customer is — background, tone and behaviour. |
| **User Prompt** | What the customer is trying to do, and how they go about it. |
| **Success Criteria** | What the agent has to achieve for the run to pass. |

> [!TIP]
>
> Click **Generate with AI** and describe the scenario in a sentence — e.g. *a lead who is already registered asks for the payment link*. It fills in the name, persona, prompt and success criteria from your agent's configuration. On an existing test case the button becomes **Edit with AI**, for changes like *make the lead push back on price twice*.

### Function mocks

Under **Test Variables & Mocks**, the **Function Mocks** tab lists the agent's custom functions. Set the JSON each one should return during the simulation — no real API is called. A function left empty returns:

```json theme={null}
{ "status": "success", "message": "Function executed successfully" }
```

Mock the responses your scenario depends on. To test what the agent does when an order is not found, make the order-lookup function return that.

## Run a batch

**1. Pick the test cases**

Click **Test** on one test case, or tick several and click **Run Selected Test**.

**2. Name the batch**

So you can find it again in the history, e.g. *Pricing objections, v3*.

**3. Choose the versions**

Defaults to the version open in the builder. Select more than one to compare versions on the same scenarios.

**4. Set the epochs**

How many times each test case runs on each version. The AI customer does not say the same thing twice, so several epochs show how consistent the agent is.

**5. Start**

A batch runs **test cases × versions × epochs** conversations — at most 100 epochs and 500 runs per batch.

**Nothing is sent and nothing is saved to your contacts.** Simulations run in test mode, like the Test Chat: no WhatsApp message goes out and no CRM record is changed.

### When a conversation ends

Each conversation runs until the agent ends it, the customer ends it, the agent and customer start repeating each other, or it reaches 20 turns.

## Read the results

**Batch Testing History** lists your batches. Select one to see its pass rate, its passed, failed and grading-failed counts, and every run in it:

| Result | Meaning |
| - | - |
| **Passed** | Rated 7/10 or higher against the success criteria. |
| **Failed** | Rated below 7/10. |
| **Grading failed** | The grader could not evaluate the run. Re-run it. |

Click a run to open the full result:

* **Agent Performance** — how the agent handled the conversation.
* **Simulated User Performance** — how closely the AI customer followed the persona and prompt. If this is poor, fix the test case before blaming the agent.
* **Success Flaws** — what stopped the run from fully meeting the success criteria.
* **Transcript** — the whole conversation, with the functions the agent called and, for Advanced agents, the flow step each reply came from.

The cost of each run, and of all batches together, is shown alongside the results.

## Tips

> **One behaviour per test case**
>
> A test case that checks pricing, rescheduling and opt-out at once fails without telling you which part broke. Split them.

> **Write success criteria the grader can check**
>
> *"Shares the payment link and does not offer a discount"* can be checked against a transcript. *"Handles the customer well"* cannot.

> **Compare versions before publishing**
>
> Run the same test cases on the published version and your new one. If the new one passes less often, you have found a regression before your customers did.

> **Use enough epochs**
>
> One run that passes can be luck. Five runs that all pass is a result.

## Related

> **[Testing a chat agent](/chat-agents/testing)**
>
> Try one conversation by hand in the Test Chat

> **[Versions and publishing](/chat-agents/versions-and-publishing)**
>
> Test on a version, publish when it is right


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.