AI Solutions
AI Chatbot Trained on Your Own Documents: SMB Guide
An AI chatbot "trained on your own documents" is almost never trained at all. It searches your files for the passages that match each question and has an AI model answer from those passages, citing them. This is called retrieval (RAG). A ChatGPT plugin (the replacement for Custom GPTs) works for small, public material; a custom build starts around $1,000 as of October 2026.
What does "trained on your documents" actually mean?
When a vendor says "trained on your data," they nearly always mean retrieval, not training. The difference matters for cost, accuracy and privacy.
Training (fine-tuning) changes the model's internal weights. It is slow, it does not reliably teach facts, and when a policy changes you have to do it again. Very few small businesses need it.
Retrieval leaves the model alone. Here is what happens:
- Your documents are split into short passages.
- Each passage is turned into a list of numbers (an "embedding") that captures its meaning, and stored in a database.
- When someone asks a question, the system finds the passages closest in meaning.
- Those passages go to the AI model with an instruction: answer from this, and say where it came from.
Think of it as an open-book exam. The model does not memorize your handbook. It reads the right pages every time. Change a page, and the next answer reflects it.
When is a ChatGPT plugin (formerly Custom GPT) enough?
A Custom GPT let you upload files into ChatGPT and give it instructions, with no code. OpenAI is retiring Custom GPTs (standard retirement date December 11, 2026) and replacing them with ChatGPT plugins, which bundle instructions, reference files and connected apps. A plugin is a fine starting point when:
- The documents are few and rarely change.
- Everyone who uses it is allowed to see everything in it.
- You are testing whether people will use it at all.
- Users already have ChatGPT accounts.
It gets weak when you need it on your website or WhatsApp, when different staff should see different documents, when content lives in many places, or when you need logs and reports of what was asked.
When do you need a custom build?
| Need | ChatGPT plugin | Custom build |
|---|---|---|
| Where it runs | Inside ChatGPT | Your website, app, WhatsApp, Slack, phone |
| Data sources | Uploaded files | Files, Google Drive, Notion, helpdesk, database, CRM |
| Keeping current | Re-upload manually | Syncs automatically when a source changes |
| Permissions | Everyone sees everything | Per-user or per-role access |
| Citations | Limited | Every answer links to the exact passage |
| Logs and analytics | Minimal | Every question, answer and source logged |
| Handover to a person | No | Yes, with the conversation attached |
| Cost | ChatGPT subscription | Build fee plus usage |
If two or more rows on the right matter to you, a custom build is worth it.
What documents work best?
The best sources are text that answers real questions:
- Policies, SOPs and staff handbooks
- Product catalogs, price lists and spec sheets
- Help-center articles and past support replies
- Service menus, opening hours and booking rules
- Contracts and templates (with care over who can see them)
Harder cases: scanned PDFs (need text extraction first), tables and spreadsheets (often better queried directly than searched as text), slide decks with little text, and documents that contradict each other.
That last one causes most bad answers. If two versions of your refund policy exist, the bot may quote the old one. Before any build, pick one current version of each document and archive the rest. Put a date and an owner on every document.
How do you stop wrong answers?
You cannot make a language model perfect, but you can make it honest about what it knows.
Answer only from sources. The instruction says: use only the retrieved passages. If they do not cover it, say so.
Show citations. Every answer links to the passage it used. Staff can check in one click, and wrong answers become easy to trace to a bad or outdated document.
Refuse when unsure. Set a relevance threshold. If nothing in your documents is close enough to the question, the bot says "I don't have that information" instead of guessing.
Hand over to a person. For refunds, medical questions, legal terms or angry customers, the bot passes the conversation to a person with the full history attached.
Test with real questions. Collect 30 to 50 real questions your team gets, with correct answers. Run them after every document change and check the results.
Where does your data live, and is it used for training?
Ask three questions of any vendor or builder.
Where are the documents and index stored? Ideally in your own cloud account or database, not a vendor's shared system.
Does the AI provider train on your data? Major providers such as OpenAI, Anthropic and Google say that by default they do not use business API data to train their models. Read each provider's current business data policy, since terms differ between consumer apps and API products.
Who can see what? Permissions must be enforced before retrieval. If a sales rep is not allowed to read HR files, those passages should never reach the model for that user's question.
What does a document chatbot cost, and how long does it take?
As of October 2026, Shorai Studios, a founder-led AI studio that builds AI agents and automations, starts AI Solutions projects from $1,000, with the final price set by scope.
| Cost driver | Lower cost | Higher cost |
|---|---|---|
| Sources | One folder of clean PDFs | Drive, helpdesk, CRM and database together |
| Document quality | Clean text, one version each | Scans, tables, conflicting versions |
| Channels | Website chat | Website, WhatsApp, phone, Slack |
| Permissions | Everyone sees everything | Role-based access |
| Actions | Answers only | Books, updates CRM, creates tickets |
| Languages | English | English, Urdu, Roman Urdu and more |
Running costs are mostly AI model usage per question, embedding costs when documents change, and database hosting. If you add documents to a voice agent platform instead, check its pricing. For example, Retell lists its Knowledge Base add-on at $0.005 per minute, with 10 knowledge bases included and $8 per knowledge base per month after that, as of October 2026.
On timing, Section AI's guide puts question-answering agents at one to two days when the information and access already exist, with action-taking agents taking a week or more. A custom build with several sources, permissions, citations and testing is usually live in four to eight weeks. Document cleanup is often the longest part.
How does Shorai build it?
We keep the stack simple and in your accounts:
- Storage and search in PostgreSQL with pgvector, so documents, embeddings and permissions sit in one database you own.
- Sync jobs that re-index a document when it changes, built with n8n or custom code. Our business automation work covers the same connectors.
- Answers that show their sources, with a refusal when nothing matches.
- A handover to a person, with the full conversation attached.
- A question log so you can see what people ask and which documents need work.
Your first version is on us: we build a working base on a sample of your documents, so you can ask it real questions before you pay. You own the code and the accounts at handover.
If the same knowledge should answer customers on WhatsApp, read our guide to WhatsApp AI agents for business.
FAQ
Can I train ChatGPT on my company documents? Not in the way most people mean. Business document chatbots almost never retrain the model. They store your documents in a searchable index, find the most relevant passages for each question and give those to the model to answer from. That is called retrieval. It is cheaper than training, updates the moment a document changes and lets every answer point to its source.
What is RAG in simple terms? RAG stands for retrieval-augmented generation. In plain terms: when someone asks a question, the system first looks up the most relevant paragraphs in your documents, then asks the AI model to answer using only those paragraphs. Think of it as an open-book exam. The model does not memorize your files. It reads the right pages each time and can tell you which pages it used.
Is my data safe if I build a chatbot on my documents? It can be, if the build is set up properly. Keep documents and the search index in your own cloud account, use AI providers whose business terms say API data is not used to train their models, and enforce user permissions before retrieval so people only see what they are allowed to. Check each provider's current data policy before you sign up.
How do I stop an AI chatbot from making things up? Use three controls. First, tell the model to answer only from the retrieved passages and show its sources. Second, set a relevance threshold so it says "I don't know" when nothing in your documents matches. Third, give it a handover path to a person for anything sensitive or unclear. Then test it with real questions every time your documents change.
How much does a custom document chatbot cost? A no-code ChatGPT plugin, OpenAI's replacement for Custom GPTs, costs only the ChatGPT subscription. A custom build with your own permissions, several data sources and citations starts from $1,000 at Shorai Studios as of October 2026, with the final price set by scope. Running costs are usually modest: AI model usage per question, embedding costs when documents change, and database hosting.
How long does it take to build an AI agent on my own data? A simple question-answering agent can be built in one to two days if the information and access already exist. Agents that take actions, like updating a CRM, take a week or more. Custom builds with several sources, permissions and testing typically go live in four to eight weeks. Cleaning up messy documents is often the longest step.
Put your documents to work
If your team answers the same questions every day from the same handbook, start with a free call. See our AI solutions, including assistants that answer from your own data, or look at the AI front desk we built for dental clinics to see how an agent hands over to staff.
Sources
More from the blog
AI Phone Ordering for Restaurants: How It Works in 2026
An AI phone agent can take a takeout order with modifiers, send it to your kitchen queue and text a payment link, for roughly 27 to 41 cents per three-minute call. Here is how the flow works and where it goes wrong.
AI Receptionist for Dental Clinics: What It Really Does
A dental AI receptionist answers every call and WhatsApp message, books and reschedules into your calendar, and hands anything clinical to a person. Here is what it says, what it must never do, and what it costs to run.
Related service: AI Solutions
Thinking about building this?
Your first version is on us. Tell us what you need.