RAG vs Fine-Tuning vs Prompt Engineering: When You Need Which

Short answer: Start with prompt engineering. It is free, fast, and often enough. Add RAG (retrieval-augmented generation) when the model needs facts it doesn't have, like your documents, products or recent data. Use fine-tuning only when you need the model to behave a certain way every time and good prompts still fail, and you have at least dozens of good examples to train on.

A simple rule: prompting changes the request, RAG changes what the model can read, and fine-tuning changes the model itself.

---

Quick Comparison

	Prompt engineering	RAG	Fine-tuning
What changes	The instructions you send	The facts added to each request	The model's weights
Good for	Tasks, format, tone, rules	Private, large or changing information	Consistent style or format on a narrow task
Adds new facts?	Only what you paste in	Yes, from your documents	Poorly. Use RAG for facts
Time to try	Minutes	Days	Weeks
What you need	Clear writing	Documents, a search index	Labeled examples, a model you can tune
Works with any model?	Yes	Yes	No, only models that allow tuning
Upkeep	Edit the prompt	Keep the documents current	Retrain when needs change

---

Prompt Engineering: Change the Request

Prompt engineering means writing clear instructions: the task, the context, examples, rules and the output format. Nothing about the model changes. You change what you ask.

When it is enough

• The model already knows the subject (writing, summarizing, coding, general knowledge).
• The information it needs fits in the prompt. Today's models can read long inputs, so a few documents can often be pasted in directly.
• You need a format or tone. Models follow instructions like "Return JSON with these fields" or "Answer in two short paragraphs" well.
• You are still testing whether AI can do the task at all.

Vendors say the same. OpenAI's guide to improving model results says prompt engineering "may be all you need" for many use cases. Google's Vertex AI tuning guide says to start with prompting to find the best prompt before you tune.

Where it stops

• It can't add facts the model never saw, unless you paste them in every time.
• Very long prompts cost more per request and can slow answers down.
• Some behavior stays inconsistent no matter how you word it.

Example

You are a support assistant for a study-planner app.
Answer in 2 short paragraphs. Be friendly.
If you are not sure of the answer, say "I'll pass this to our team"
instead of guessing.

Question: How do I move a task to tomorrow?

---

RAG: Give the Model Your Facts

RAG searches your own documents for the parts that match a question, then adds those parts to the prompt. The model answers using that text. The model itself does not change.

How it works

1. Index. Split your documents into chunks and store them in a search index, often a vector database.
2. Retrieve. When a question comes in, find the most relevant chunks.
3. Generate. Send the question plus those chunks to the model, with an instruction to answer from them.

When to use it

• Your information is private: course notes, company docs, help articles, contracts.
• It changes often. Update a document and the next answer uses it. No retraining.
• There is too much to paste into one prompt.
• You need to show where an answer came from. RAG can cite the source chunk.

Where it stops

• Answers are only as good as the search. If the wrong chunks come back, the model gives a wrong answer that sounds right. Chunking and ranking matter, as covered in RAG retrieval optimization.
• It adds parts to build and maintain: a document pipeline, an index, and a search step.
• It gives the model information, not new habits. It won't change tone or reasoning style.

Want to build one? Our RAG chatbot tutorial with Next.js and Supabase walks through the full setup.

---

Fine-Tuning: Change the Model

Fine-tuning trains an existing model further on your own examples, usually pairs of input and ideal output. This changes the model's weights, so the new behavior shows up without long instructions.

When to use it

• You need the same format or style every time, and prompts get it right only most of the time.
• The task is narrow and well defined: sorting support tickets into categories, extracting fields from forms, writing in one fixed house style.
• You want a smaller, cheaper model to do one task as well as a bigger one.
• Your prompts have grown very long, and you want shorter requests.

What you need

• Examples. OpenAI's fine-tuning guide sets a minimum of 10 examples and suggests starting with about 50 good ones. Google's Vertex AI guide suggests around 100 or more. Quality matters more than count.
• A model you are allowed to tune. Not every model offers this. OpenAI's fine-tuning platform no longer accepts new users, and it lets existing customers create training jobs only until January 6, 2027. Google offers supervised tuning for several Gemini models on Vertex AI, including Gemini 2.5 Pro and Gemini 3.5 Flash. Open-weight models such as Llama, Mistral and Gemma can be tuned on your own hardware or a cloud GPU.
• A way to test it. Keep a set of test questions the model never trained on, and compare the tuned model to your best prompt.

Where it stops

• It is a weak way to add facts. The model may repeat training answers word for word but fail on the same question phrased differently. Use RAG for facts.
• The knowledge is frozen. When your needs change, you retrain.
• Tuning on a narrow task can make the model worse at other things.
• Bad examples teach bad habits. Cleaning data is often most of the work.

---

How to Choose

Go through these questions in order.

1. Does a well-written prompt do the job? Write one with clear instructions and two or three examples. Test it on 20 or more real inputs. If it works, stop here.
2. Is the problem missing facts? If the model gets things wrong because it doesn't know your information, add RAG.
3. Is the problem behavior, and do prompts keep failing? If the format or style is still wrong after real effort on the prompt, and you have good examples, consider fine-tuning.
4. Do you need both facts and behavior? Use RAG for the facts and fine-tuning for the behavior. They work together.

Your situation	Start with
"It needs to know our course material or product docs"	RAG
"Answers must cite a source"	RAG
"It keeps making things up about our policies"	RAG, plus a prompt rule to answer only from the sources
"It should always return valid JSON"	Prompt with a schema first, fine-tuning if it still fails
"Replies must match our brand voice"	Prompt with examples first, fine-tuning if it still drifts
"We want a small, cheap model for one task"	Fine-tuning
"We are just getting started"	Prompt engineering

---

Combining Them

Most real AI apps use more than one.

• Prompt engineering + RAG is the most common setup. The prompt sets the rules and format. RAG supplies the facts for each question.
• Fine-tuning + RAG suits specialized apps: the tuned model has the right style, and RAG keeps its facts current.
• All three shows up in larger systems: a tuned model, documents from RAG, and a prompt with per-request instructions on top.

Managing everything the model sees, across prompts, retrieved documents and conversation history, is called context engineering. Our post on context engineering vs prompt engineering covers it.

---

Common Mistakes

• Fine-tuning first. It is the slowest and most expensive option. Many problems that look like they need it are fixed by a better prompt.
• Fine-tuning to add knowledge. Use RAG for facts and fine-tuning for behavior.
• Building RAG for a few pages. If everything fits in the prompt and rarely changes, paste it in.
• Skipping tests. Write down what a good answer looks like and test on real inputs before and after every change.

---

Frequently Asked Questions

What is the difference between RAG and fine-tuning?

RAG finds relevant text in your documents and adds it to the prompt at the moment of each question. The model does not change. Fine-tuning trains the model further on your examples, which changes its weights. Use RAG to give the model facts. Use fine-tuning to change how it behaves.

Should I use RAG or fine-tuning first?

Start with prompt engineering. If the model lacks facts, add RAG next, because it is faster to build and works with any model. Move to fine-tuning only when prompts and RAG together still can't get the behavior you need.

Which is cheapest?

Prompt engineering. It costs only the normal model usage. RAG adds a search index and a document pipeline. Fine-tuning adds data preparation and training, though a tuned smaller model can be cheaper to run later.

How much data do I need to fine-tune a model?

OpenAI's guide sets a minimum of 10 examples and suggests starting with about 50 good ones. Google's Vertex AI guide suggests around 100 or more. Clean, varied examples matter more than a large count.

Can I fine-tune ChatGPT, Claude or Gemini?

Not all of them. OpenAI's fine-tuning platform no longer accepts new users. Google offers tuning for several Gemini models on Vertex AI. Open-weight models like Llama, Mistral and Gemma can be fine-tuned with your own setup. Check your provider's current docs before you plan a project around it.

Can I use RAG and fine-tuning together?

Yes. A common pattern is a fine-tuned model for the right style or format, with RAG supplying current facts for each question.

Does fine-tuning make a model smarter?

No. It makes the model better at the tasks in your examples. It does not add general intelligence, and it can make the model worse at unrelated tasks.

---

Learn More

• Context Engineering: control what the AI sees, from prompts to retrieved documents
• Local RAG for Beginners: build a small RAG system step by step
• Prompt Engineering: hands-on practice writing prompts

All three courses are free, with a certificate when you finish.

---

Facts checked on OpenAI's and Google Cloud's own documentation in November 2026. Model support for fine-tuning changes often, so check your provider's docs before you start.