Building an AI feature is one cost. Running it is another, and the second one surprises many businesses. Every time your chatbot, agent, or automation uses a language model, the provider charges you, and those small charges add up with volume.
This guide explains how LLM API costs work, how to estimate them before you build, and how to keep the bill under control. We avoid quoting prices, because they change often. Always check each provider's current pricing page.
How LLM usage is priced
Most providers charge by tokens. A token is a small chunk of text, roughly three-quarters of an English word on average. You pay for:
- Input tokens: everything you send to the model (instructions, the user's message, retrieved documents, conversation history)
- Output tokens: everything the model writes back
Output tokens are typically priced higher than input tokens, and larger, more capable models cost more per token than smaller ones. Some providers also charge for extra features such as image or audio processing, and some offer discounts for cached or batched requests.
What drives your bill
- Number of requests. More users and more automation runs mean more calls.
- Prompt length. Long instructions are re-sent with every request.
- Conversation history. In a chat, the whole history is often sent each turn, so cost grows as conversations get longer.
- Retrieved content. In a RAG chatbot, the passages you retrieve are added to the prompt. See our RAG guide.
- Answer length. Longer answers cost more.
- Model choice. The same task can cost many times more on a top-tier model than on a small one.
- Agent loops. Agents make several model calls per task, with tool results fed back each time.
- Retries and errors. Failed calls that you repeat still cost money.
A simple way to estimate before you build
Use a spreadsheet with these inputs:
- Expected requests per month
- Average input tokens per request
- Average output tokens per request
- Price per input and output token for the model (from the provider)
Monthly cost ≈ requests × (input tokens × input price + output tokens × output price)
Illustrative example with made-up numbers: Imagine a support chatbot handling 10,000 conversations a month, each needing three model calls. That is 30,000 requests. If each averages 2,000 input tokens and 300 output tokens, you can multiply by your chosen model's prices to get a monthly figure, then add platform and hosting costs.
Run the estimate for three scenarios: low, expected, and high volume. And test real conversations, because real prompts are usually longer than guesses.
Eight ways to reduce your AI bill
1. Use the smallest model that does the job
Test cheaper models on your real tasks. Many classification, extraction, and routing tasks work well on small models. Our guide to choosing between Claude, ChatGPT, and Gemini explains how to test fairly.
2. Route tasks to different models
Use a small model for simple steps and a stronger model only for hard ones. This is often the biggest saving.
3. Shorten your prompts
Remove repeated or unnecessary text. Clear, concise instructions are cheaper and often work better. See chatbot prompt design.
4. Control the context
- Send only the relevant retrieved passages, not everything.
- Summarise long conversation history instead of resending it all.
- Remove old turns that no longer matter.
5. Cache repeated work
If the same question or long document is processed repeatedly, caching can reduce cost. Some providers offer cheaper rates for repeated prompt prefixes.
6. Limit output length
Set sensible maximum lengths and instruct the model to be concise.
7. Batch non-urgent work
Tasks that can wait, such as overnight document processing, may qualify for cheaper batch options from some providers. Check current terms.
8. Avoid unnecessary calls
Use plain rules for what rules can do. Not every step needs an AI model. For example, validating a date format does not need a language model.
Add guardrails
- Per-user and per-day limits to stop runaway usage
- Maximum steps for agents, to prevent endless loops
- Budget alerts from the provider and in your own monitoring
- Logging of token use by feature, so you know where the money goes
Our article on AI agent security and guardrails covers limits and monitoring.
Costs beyond the model
Model usage is one part. Do not forget:
- Hosting and infrastructure
- Vector database or search for RAG
- Speech or messaging fees for voice or WhatsApp
- Monitoring and logging tools
- Maintenance and improvement time
- Human review of exceptions
Our guides to chatbot cost factors and measuring the ROI of AI automation cover the full picture.
Self-hosting models: does it save money?
Running open models yourself can reduce per-use charges at high volume and give more control over data, but it adds infrastructure and maintenance costs and needs technical skill. It usually makes sense only at larger scale or with strict data requirements. Compare the total cost, not only the per-token price.
Common mistakes
- Estimating with short test prompts and being surprised by real ones
- Using the top model for everything
- No usage limits or alerts
- Ignoring conversation length growth
- Forgetting that agents multiply calls
We can help you keep AI affordable
We design AI systems with cost in mind: right-sized models, efficient prompts, caching, and usage monitoring. See our AI solutions, or book a free consultation.
