Start by measuring whether the agent reliably completes the customer task you agreed on. For a spare part search, that means finding a compatible part with a current offer. A purchase adds approval, a clear order confirmation and service afterwards. The most useful metric follows the task you gave the agent.
Which steps should I measure separately?
| Step | Example of evidence |
|---|---|
| Discovery | The offer appears for a relevant request in the channel being tested. |
| Decision | The agent receives a valid offer for the right variant and delivery option. |
| Purchase | The merchant’s system confirms an order with a unique number. |
| Operations | The shop records clarifications, handovers, cancellations and returns. |
For each step, decide which attempts belong in the denominator. A completion rate based on all requests answers a different question from one based on already approved carts.
How do I compare the pilot with the current workflow?
Pick a task and compare similar periods or groups. Look at correct orders per 100 started tasks, for example, and the time service spends on them. Keep test purchases separate. A referral from an assistant can explain a visit. Attribute an order only when your data can connect that path to the confirmed purchase.