Get a free Shopify store audit

How to Build an AI Chatbot: A Practical Guide

How to build an AI chatbot that's scoped to one job, grounded in your own data, and safe to let take real actions.

By Ahmed

Here's how to build an AI chatbot that actually works: get three decisions right, in order — scope one narrow job it will own, ground its answers in your own content so it stops guessing, and decide which of its actions can run on their own versus which need a person to confirm first. Skip any one of them and the other two won't save you.

How to Build an AI Chatbot: Decide the One Job It Will Do First

Before picking a platform or a model, name the single highest-volume, lowest-risk task you want the bot to own. Rasa's own guidance on this is blunt: a clear use case means something like "resolve order-status questions without a human," not a vague goal like "a chatbot for support." Order status, password resets, and store-hours questions work well as first targets because they're frequent, bounded, and cheap to get wrong while you're still learning what the bot does badly.

Write down the metric that tells you the bot is working before you build anything, because it changes the entire design. If you're measuring resolution — did the customer's actual problem get solved — you'll design confirmation steps that check in with the customer. If you're measuring deflection — did the conversation avoid a human handoff — you'll get a different bot, and not always a better one for the person using it. This same discipline, scoping a narrow first version and deciding up front how you'll know it worked, is exactly what we walk through in our guide to AI MVP development, if you haven't scoped a first AI build before.

Choose Your Build Path: No-Code Platform, Framework, or Direct API

Once you know the job, the next decision is how much of the plumbing you want to build yourself. A no-code platform needs no engineering and gets a bot live fast, but it limits how much custom logic you can wire in. Building directly against a model provider's API gives you full control over what the bot can do, at the cost of building and maintaining that integration yourself.

On the API side, the two major providers have converged on a similar shape. OpenAI's current recommendation for new projects is the Responses API, described as "our new API primitive" that is "recommended for all new projects," with Chat Completions still supported but not the forward path. Anthropic's tool-use documentation describes a parallel mechanism: "Tool use is a contract between your application and the model... Claude determines when and how to call them. The model never runs anything on its own." In both cases, the model doesn't execute anything directly — it emits a structured request, your code runs the actual operation, and the result goes back into the conversation.

Rasa's own framing of the no-code-versus-framework decision is a useful filter: if the bot only needs to answer questions, a no-code platform may be enough. If it needs to take actions — issuing a refund, booking an appointment, updating a record — you want a framework or a direct API build where the actions are explicit, testable pieces of logic rather than something the model is trusted to improvise.

Whichever path you take, the bot also needs a way to remember what was already said earlier in the conversation. OpenAI's documentation lays out three ways to manage that conversation state: passing the full history of prior messages back on every request, chaining responses together with a stored response ID, or creating a persistent conversation object that keeps working "across sessions, devices, or jobs." A no-code platform typically handles this for you; building directly against an API means picking one of these approaches yourself and making sure it survives a user closing the browser tab mid-conversation.

Ground It in Your Own Content So It Doesn't Invent Answers

A raw model doesn't know your return policy, your current pricing, or what's actually in stock. Retrieval-augmented generation, or RAG, is the standard fix: the bot retrieves relevant passages from your own documents, policies, or product data and answers from those instead of from its general training. Anthropic describes RAG as letting a model "leverage your internal knowledge bases or customer support documents, significantly enhancing its ability to answer domain-specific questions," and notes it's already used in production for customer support and Q&A over internal company documents.

It's worth being precise about what RAG actually does: it reduces how often the model invents an answer, by anchoring responses to retrieved content, rather than eliminating the problem outright. A bot grounded in a stale or incomplete knowledge base will still answer confidently and still be wrong — grounding only helps if what you're grounding it in is accurate and current.

This is the same principle behind simpler automations that read your store's own data before answering. Commiko, our own AI agent for Instagram, replies to comments and sends a DM with the answer by reading a store's actual Shopify product catalog rather than guessing at what's in stock or what something costs — see our breakdown of Instagram DM automation for what Meta actually allows there. If you want a chatbot built around your own data rather than a generic model's guesses, that grounding work is a core part of what we build in our AI automation agents service.

Give It Tools Carefully: Reads First, Writes With Guardrails

Once the bot can retrieve information, the next question is what it's allowed to do with that information. The useful distinction is between read actions and write actions. Looking up an order, checking a return policy, or pulling a tracking number are read actions — safe to let the model trigger on its own, because nothing changes if it gets the lookup slightly wrong. Cancelling an order, issuing a refund, or updating a customer's account are write actions, and they need confirmation, a permission check, and a record of what happened and why.

This maps directly onto how tool use actually works under the hood. Anthropic's documentation describes the mechanics plainly: the model emits a tool-use request, your application code executes it, and the result is returned as a tool result the model reads on its next turn — the model is never the thing actually touching your database or your payment processor. That gap between "the model decided" and "your code executed" is exactly where you put the guardrail: a dollar threshold under which a refund runs automatically, and above which it waits for a person.

If you want help scoping and building a chatbot that's grounded in your own data from day one, talk to our team.

Frequently asked questions

Do I need to know how to code to build an AI chatbot?

No, not for a first version. No-code platforms let you assemble and test a bot's question-answering behavior without writing code. Coding becomes necessary once the bot needs to take real actions — refunds, bookings, account changes — because those need to be built as explicit, testable integrations rather than left to the model to improvise.

What's the difference between a chatbot and an AI agent?

A chatbot, in the simplest sense, answers questions using retrieved content and a conversational interface. An agent goes further: it's given tools it can call — a tool-use contract, in Anthropic's terms — so it can take actions inside other systems, not just generate text. Many chatbots are really a thin conversational layer on top of an agent that has tools.

How do I stop my chatbot from making up answers?

Ground its answers in your own content with retrieval-augmented generation, so it answers from retrieved passages of your documentation, policies, or product data instead of its general training. That reduces how often it invents an answer, but it doesn't eliminate the risk entirely — a bot is only as accurate as the content it's grounded in, so keeping that content current matters as much as the grounding mechanism itself.

What happened to OpenAI's Assistants API?

OpenAI has moved its recommendation for new projects to the Responses API, which it describes as the current primitive for building agent-like applications. The Assistants API is not the path OpenAI recommends for new work going forward.

Should my chatbot be allowed to take actions like issuing refunds?

Only with guardrails matched to the consequence of being wrong. A read action, like checking an order's status, is safe to automate because nothing changes if it's slightly off. A write action like a refund should require confirmation, a permission check, and a record of what happened, with low-risk cases — a refund under a set dollar threshold, for instance — as the only candidates for running without a person in the loop.

Sources

Tell us what you're building.

Share a few details and we'll reply within 1–2 business days with questions, a scope and a quote.