RAG or fine-tuning: Which problem are you trying to solve?
“The AI should know our company” is not yet a technical requirement. Current documents may be missing, the answer format may be unusable, or the assistant may be able to access information restricted to another team. These problems need different solutions. RAG and fine-tuning are not interchangeable purchasing options.
Research checked: 2026-09-09 · Cover: AI-generated illustration
Make missing knowledge retrievable
Retrieval searches for relevant information and supplies it to the model as context. OpenAI documents semantic search that can identify similar meaning even without identical keywords. This is one foundation for retrieval-augmented generation, or RAG.
For an internal assistant, I would first establish the knowledge source: who maintains it, which version is authoritative and who may read it? Good search over outdated documents does not produce current knowledge. Semantic similarity does not replace an access check either.
Change behavior through examples
Supervised fine-tuning uses examples of desired answers to adapt model behavior. That addresses a different question from retrieving documents that change daily. It is not a convenient replacement for maintaining a knowledge base.
An operational detail matters: as checked on September 9, 2026, OpenAI says it is winding down its fine-tuning platform and no longer admits new users. Provider selection therefore needs an availability and migration check. The underlying technique exists independently of that particular commercial offering.
Separate three kinds of failure
If an assistant cannot find the correct passage, I would inspect the query, document segmentation and result ordering. If it retrieves the passage but answers incorrectly, the next question is how it uses context. If the answer and evidence are correct but unusably formatted, I would inspect output requirements and examples.
These distinctions avoid expensive detours. A larger model does not fix missing access controls. More documents do not correct neglected source maintenance. Another training run does not automatically establish which invoice was paid today.
The workflow at a glance
- Knowledge gapInspect sources and retrieval
- Behavior gapInspect examples and output
- Access gapCheck backend permissions
A pilot with verifiable answers
For a knowledge feature in a content platform, I would start with a bounded set of approved briefs. Test questions should include clear answers, conflicting document versions and questions the collection cannot answer. Each answer must expose the source it used.
Success includes explaining a knowledge gap clearly. Once the source of failure is understood, the next investment becomes easier to choose. The first deliverable should distinguish a data problem, a retrieval problem and a behavior problem.
