AI Solutions

    12 Chatbot KPIs to Track (and Vanity Metrics to Ignore)

    Which chatbot metrics actually matter? Twelve KPIs for support and sales chatbots, how to measure them, and the vanity numbers that mislead you.

    8SolveAI Team4 min read

    You launched a chatbot. A dashboard shows thousands of conversations. Is it working?

    Not necessarily. Many chatbot dashboards show big numbers that feel good but say little about results. This guide lists twelve KPIs that actually matter, how to measure each, and the vanity metrics that can mislead you.

    It applies to support, sales, and booking chatbots. For the bigger picture on building one, see our guide to AI customer support chatbots.

    Before you measure: decide the goal

    KPIs only make sense against a goal. Pick the main one:

    • Support: resolve questions faster and reduce the load on your team
    • Sales: capture and qualify leads, book meetings
    • Operations: complete tasks such as bookings and order lookups

    Then choose KPIs that show progress towards it, and record your baseline before the chatbot so you can see the change.

    The 12 KPIs that matter

    Quality and outcomes

    1. Resolution rate The share of conversations where the customer's issue is solved without a human. Count it only when the customer's problem is actually resolved, not merely when the chat ends.

    2. Escalation (handover) rate How often the bot passes to a human. Too low can mean customers are trapped; too high means the bot is not helping. Look at the reasons for escalation as well as the number.

    3. Customer satisfaction (CSAT) A quick rating at the end of the chat. Always read it alongside resolution rate. A bot that "resolves" many chats while customers are unhappy is a warning sign.

    4. Answer accuracy Sample conversations each week and score whether answers were correct and complete. This is how you catch wrong answers early. See our article on stopping hallucinations.

    5. Fallback / unanswered rate How often the bot says "I don't know" or fails to understand. The list of unanswered questions is a to-do list for your content.

    Speed and effort

    6. First response time Bots should answer in seconds. Track it, especially in busy periods.

    7. Time to resolution How long it takes from the first message to a solved problem, for both bot-resolved and human-resolved chats.

    8. Repeat contact rate How often the same customer returns about the same issue within a few days. A high rate suggests the answer did not really solve the problem.

    Business impact

    9. Ticket deflection / support volume Change in the number of tickets reaching your team for topics the bot covers. Compare with your baseline.

    10. Cost per conversation or per resolution Include AI usage, platform fees, and the cost of human review. Compare with the cost of a human-handled conversation. Our guides to LLM API costs and ROI of AI automation help with the maths.

    11. Lead conversion (for sales bots) Share of conversations that become qualified leads or booked meetings, and then customers. Track the full path, not only the first step.

    12. Goal completion rate For task-based bots, such as booking an appointment or checking an order, the share of started tasks that finish successfully.

    Vanity metrics to treat with caution

    MetricWhy it misleads
    Total conversationsMore chats can mean more confusion, not more value
    Total messagesLong conversations might mean the bot is struggling
    Time spent in chatLonger is not better for support
    "Containment" on its ownCan hide customers giving up or being blocked from a human
    Number of questions answeredSays nothing about whether the answers were right
    Uptime aloneA bot that is up but wrong is not working

    Vanity metrics are not useless. They show activity. But they are not outcomes.

    How to build a simple dashboard

    Keep it small. Start with five numbers:

    1. Resolution rate
    2. Escalation rate and top reasons
    3. CSAT
    4. Answer accuracy (from sampling)
    5. Cost per resolution

    Add the rest as you mature. Review weekly in the first months, then monthly.

    How to measure without heavy tools

    • Use your chatbot platform's analytics for volume, escalation, and fallback.
    • Add a one-click rating at the end of each chat.
    • Sample 20–50 conversations a week and score them in a spreadsheet.
    • Tag escalation reasons so you can see patterns.
    • Pull ticket volume by topic from your helpdesk.

    Set targets carefully

    Avoid copying numbers from vendor reports. Your questions, customers, and data are different. Use your own baseline, set a realistic target for the first 90 days, and improve step by step.

    Use the numbers to improve

    • Top unanswered questions become new content.
    • Frequent escalation reasons show where to improve flows or add handover rules.
    • Low-CSAT conversations show tone and accuracy problems.
    • High repeat contact shows answers that need clearer wording.

    Our chatbot testing checklist helps you turn each finding into a test so the problem does not return.

    We can set up measurement for you

    We build chatbots with analytics, review workflows, and improvement cycles included. See our chatbot development service, or book a free consultation.

    Have a project in mind?

    Tell us what you want to build and we will suggest the simplest path to get there.

    Keep reading

    All articles