When does fine-tuning make sense for code?

Fine-tuning helps when you want to change how a model writes or behaves, not what it knows: matching house style, emitting a proprietary DSL or internal API shape reliably, or cutting prompt overhead for a narrow, stable task. It is the wrong tool for 'answer questions about our repository' — the knowledge changes every commit.

Can I combine RAG and fine-tuning?

Yes, and it is the standard setup for serious systems: fine-tune (if needed) for style, format, and tool-use reliability; retrieve for facts, current code, and citations. The two solve different problems and compose — retrieval supplies the code, the tuned model supplies the manner.

How much does RAG cost compared to fine-tuning?

RAG costs an index build (minutes for most repos) plus a small amount of context per query; with local embeddings and SQLite like Open Context Engine, it is free and offline. Fine-tuning costs a curated training set, training runs, evaluation, and re-training whenever your code or the target model changes — a recurring engineering tax.

How do I run RAG over my own codebase?

With Open Context Engine: npm install -g @dhaxor/open-context, then oce index builds a local AST-aware hybrid index (semantic + keyword), and oce search queries it — or serve it to Claude Code and Cursor over MCP with claude mcp add oce -- oce mcp. Fully local by default; no code leaves the machine.

guides/rag-vs-fine-tuning 1.00

RAG vs fine-tuning for code

Q: Is RAG better than fine-tuning for code?

For questions about a specific codebase, usually yes. Code changes daily — retrieval re-indexes in seconds and always answers from current source, while a fine-tuned model freezes knowledge at training time and goes stale after the first merge. RAG also shows exactly which files an answer came from, so claims are verifiable; fine-tuning bakes knowledge into weights where it can't be cited or audited.

Should you retrieve from your codebase or train on it? What each approach actually does, where each fails — and why for code questions the answer is almost always RAG.

Two different questions

The confusion starts because both approaches get described as "teaching the model your codebase." They teach it different things:

RAG (retrieval-augmented generation) fetches the relevant code at question time and puts it in the model's context. The model reads your actual source and answers from it.
Fine-tuning adjusts the model's weights on training examples. The model internalizes patterns from your data — style, formats, and (imperfectly) facts — and reproduces them without seeing the source.

One supplies facts on demand; the other shapes behavior everywhere. Most "should we fine-tune on our repo?" questions are really asking for the first thing, which is what RAG does.

The case for RAG on code

Freshness. Code changes every commit. A retrieval index re-embeds the diff in seconds and always answers from current source. A fine-tuned model freezes knowledge at training time and starts going stale immediately — re-training on every merge is not a workflow.
Citations and trust. RAG answers come with the exact chunks the model saw: you can open the file, check the claim, review the edit. Fine-tuned knowledge lives in weights — unverifiable, unauditable, and confidently wrong on stale facts.
Hallucination resistance. Ask about a function invented last week: retrieval finds it or the system visibly says so. A fine-tuned model will happily generate a plausible API that doesn't exist.
Privacy. Local RAG (local embeddings, local SQLite) means nothing ever leaves the machine. Fine-tuning means shipping a curated corpus of your source to a training pipeline.
Cost. An index build takes minutes and querying adds modest context per call. Fine-tuning needs a curated dataset, training runs, evaluation, and a re-train cadence.

The case for fine-tuning

Fine-tuning is not useless — it's pointed at a different target. It's the right tool when you want to change how a model writes, not what it knows:

Consistent house style and conventions across generated code.
Reliably emitting a proprietary DSL, internal framework idioms, or a strict output format.
Cutting prompt overhead for a narrow, stable task done at volume.

Notice none of those are "answer questions about our repository." The moment the task is knowledge about code that changes, fine-tuning is fighting the wrong battle.

Side by side

	RAG	Fine-tuning
Knowledge freshness	Re-indexes on every save	Frozen at training time
Verifiability	Shows the exact source chunks	Weights only — no citations
Hallucination on unknowns	Visibly misses or admits gaps	Plausible fabrication
Data privacy	Can be fully local	Corpus shipped to training
Setup cost	Minutes (`oce index`)	Dataset + runs + eval loop
Best at	Questions about changing code	Style, format, narrow tasks

The standard answer: both, with retrieval first

Mature teams compose them: retrieval supplies facts and current code; fine-tuning (when justified) supplies manner and format. If you only do one for codebase work, do retrieval — it's where the correctness lives. See the code RAG guide for what a good retrieval pipeline needs (AST-aware chunking, hybrid fusion, measurement), or go straight to running it:

npm install -g @dhaxor/open-context
oce index                    # local hybrid index, minutes
claude mcp add oce -- oce mcp  # ground Claude Code in the real source

The MCP server guide covers Cursor and HTTP clients too; the benchmarks page shows what the retrieval actually scores before you trust it.

Frequently asked questions

Is RAG better than fine-tuning for code?

For questions about a specific, changing codebase — yes. Retrieval always answers from current source with citations; a fine-tuned model goes stale after the first merge and can't show its work.

When does fine-tuning make sense?

When the goal is behavior, not knowledge: house style, proprietary formats and DSLs, or a narrow stable task where trimmed prompts save real cost.

Can I combine both?

Yes — fine-tune for style and format, retrieve for facts and citations. They solve different problems and compose cleanly.

What does local RAG cost?

An index build in minutes plus modest context per query. With in-process embeddings and SQLite it's free and offline — nothing leaves the machine.

How do I start?

npm install -g @dhaxor/open-context, oce index, then oce search — or serve it over MCP. Setup guides are in the docs.