What 18 LLM papers taught me about the real bottleneck

If you’re a programmer who wants to understand how LLMs work under the hood, the research literature does not make it easy. Papers assume you already know which arguments are settled and which are tribal. They use the same words to mean different things. They skip the ablations that would tell you whether the method matters or the data does. And they almost never acknowledge that their headline number comes from GPT-4 grading itself.

I got tired of it. So I read 18 papers cover to cover, from the original Transformer through to Qwen3’s latest reasoning pipeline. Not skimming abstracts. Not watching YouTube summaries. Reading them side by side and asking what’s actually true when you hold the full stack in your head at once.

Here’s what I learned that I wish someone had told me when I started.

How an LLM actually works

The first paper in the stack — Jay Alammar’s Illustrated Transformer — is still the best explanation of the core mechanism, and it’s worth understanding before you touch anything else.

An LLM processes text as tokens: chunks of words or subwords that get converted into vectors (lists of numbers). These vectors live in a high-dimensional space where similar concepts sit near each other. “King” and “queen” end up close together. So do “Paris” and “France.” The model learns these relationships during training by predicting the next token in billions of sentences.

The clever part — the part that made everything since 2017 possible — is the attention mechanism. For every token in a sequence, the model computes three things: a query (what am I looking for?), a key (what do I contain?), and a value (what should I contribute if someone’s looking for me?). Every token attends to every other token by comparing its query against all the keys, building a weighted mixture of values. “The cat sat on the mat” — the word “sat” pays attention to “cat” (who’s doing the sitting?) and “mat” (where?). Multi-head attention runs several of these in parallel, each head learning to watch for different kinds of relationships.

This architecture scales shockingly well. If you feed it enough data and enough compute, it learns grammar, factual knowledge, reasoning patterns, and a surprising amount of world structure — all from the single objective of predicting the next token. Nobody explicitly teaches it subject-verb agreement or that Paris is the capital of France. It absorbs these patterns from the statistical structure of the text.

That’s pretraining. It’s the expensive part. It takes thousands of GPUs, months of compute, and terabytes of carefully filtered text. But once it’s done, you have a model that knows an enormous amount about language and the world. It just doesn’t know how to talk to you yet.

The training pipeline nobody diagrams

If pretraining is the expensive part, the rest of the pipeline is where most of the interesting engineering happens. Every paper after the Transformer is about one of these stages.

Stage 1: Pretraining. Next-token prediction on web-scale data. The output is a “base model” — it can complete text, but if you ask it a question it’ll either continue your sentence or ramble. The Flan Collection paper showed that what you train on during this stage matters enormously. Phi-1 proved that 7 billion tokens of textbook-quality code beats 100× more web-scraped code. SmolLM2 demonstrated that rebalancing your data mixture mid-training — more math when math is weak, more long-context later — produces better results than a fixed recipe. Pretraining is curriculum design whether you call it that or not.

Stage 2: Supervised fine-tuning. You take the base model and train it on examples of the behaviour you want: question-answer pairs, instruction-following dialogues, whatever format your application needs. This is where the model learns to say “Sure, I can help with that!” instead of continuing your prompt as if it were a half-finished sentence.

The surprising finding, from LIMA, is that you don’t need much. LIMA fine-tuned a 65B model on just 1,000 handpicked examples and reached 43% parity with GPT-4. The knowledge was already there from pretraining. SFT is mostly about teaching the model which format to use when surfacing it. The paper’s authors called this the “superficial alignment hypothesis,” and the evidence across multiple papers supports it: SFT changes output style, not the underlying knowledge.

Stage 3: Alignment. This is where the terminology gets messy. “Alignment” means different things to different papers, and the distinction actually matters if you’re going to run these methods yourself.

LIMA’s version of alignment is what I’ve started calling style alignment: teaching the model to behave like a helpful assistant. It’s cheap. A thousand examples, a few hours of GPU time, done.

DeepSeekMath’s version is reasoning alignment: teaching the model to think step by step, verify its own work, and explore multiple solution paths before committing. They used GRPO, an RL method that generates 64 candidate answers per question, scores each step with a process reward model, and iterates through multiple stages. This is expensive. It genuinely changes what the model can compute, not just how it presents the answer.

When you fire up Unsloth and run DPO on your dataset, you’re doing style alignment. When you run GRPO on math or code problems, you’re doing reasoning alignment. They look similar in a notebook. They’re not similar in what they cost or what kind of improvement you should expect.

DPO itself is worth understanding because it’s the method most people actually use. Traditional RLHF requires training a separate reward model, then using reinforcement learning to optimise the policy against it — two models, two training loops, lots of instability. DPO noticed something clever: the RLHF objective can be rewritten as a simple classification loss over preference pairs. Instead of “generate a response, score it, update the policy, repeat,” you just feed the model pairs of good and bad responses and it learns to prefer the good ones. Same outcome, one training loop, no separate reward model. ORPO took this further by merging SFT and preference learning into a single stage — the model learns format and quality simultaneously.

Four myths the papers kill

Myth 1: More parameters means better performance.

Phi-1 at 1.3B parameters matched or beat models 10× its size on Python coding tasks. The difference was data quality. Phi-1 was trained on 7 billion tokens of textbook-quality code with accompanying exercises — the equivalent of giving a student a well-written textbook instead of every GitHub repo ever created. SmolLM2 at 1.7B parameters, trained on 11 trillion tokens with careful data rebalancing across four stages, produced results competitive with much larger models. Parameter count is a marketing number. Data quality and training curriculum are what actually determine capability.

Myth 2: Fine-tuning teaches the model new knowledge.

It mostly doesn’t. LoRA showed that you can adapt a model to a new task by training a few thousand parameters at rank 4 or 8 while freezing the other billions. QLoRA quantized models to 4 bits — compressing them to a quarter of their size — and lost almost nothing in quality. LIMA aligned a 65B model with 1,000 examples. If fine-tuning were adding new knowledge, these numbers wouldn’t work. A rank-4 matrix doesn’t have enough capacity to store new factual knowledge. What fine-tuning does is rearrange the existing knowledge so it surfaces the right thing at the right time. The model already knows the answer. You’re teaching it to look in the right place.

There’s a tradeoff here worth knowing about. Every paper reports a 1–5% drop in factual accuracy after SFT. If you’re only teaching format, why does knowledge degrade? Because format-matching physically overwrites some of the pretraining signal. SFT shifts weights, and some of those weights were storing factual knowledge. DPO is gentler — it operates on output distributions rather than directly on weights — which is why ZEPHYR found DPO overfitting to be harmless while other papers found SFT overfitting to actively damage performance.

Myth 3: You need a datacenter to do anything useful.

TinyStories produced coherent, creative narratives with 10 million parameters. That’s smaller than a 1990s video game. Phi-3 runs a 3.8B model on an iPhone at over 12 tokens per second using 4-bit quantisation. QLoRA lets you fine-tune a 65B model on a single 48GB GPU. Unsloth fits 7B DPO training on a T4 with 16GB.

The practical frontier isn’t defined by what’s theoretically optimal. It’s defined by what fits in your VRAM. This constraint created the entire research direction around quantisation, parameter-efficient fine-tuning, and distillation. The papers document the results without always acknowledging that they’re products of hardware economics as much as algorithmic insight.

If you have a consumer GPU — even an RTX 2070 with 8GB — you can run a 4B model in INT4, fine-tune with LoRA at rank 16, and fit 32K context. That’s enough for real experiments. The ceiling is higher than most people assume.

Myth 4: Benchmark numbers mean what they say.

Almost every paper in the stack evaluates its results by asking GPT-4 to judge the output. QLoRA’s authors measured the correlation: Kendall τ = 0.43 between GPT-4’s judgments and human judgments. That’s moderate at best. Most papers don’t report this number at all.

The implications are uncomfortable. GPT-4 has known biases — it prefers verbose outputs, favours models distilled from itself, and responds to specific formatting patterns. When a paper claims their method improves performance by 3 points, there’s a real chance they measured GPT-4 preferring outputs that sound like GPT-4. The benchmark leaderboard might be tracking stylistic similarity rather than capability.

Worse, a recent paper on SFT-DPO interaction found that the choice between full fine-tuning and LoRA dominates the choice of training objective by 12 to 1. Papers that compare LoRA-DPO against LoRA-GRPO and claim one algorithm is better than the other might be measuring the ceiling of LoRA, not the algorithm. This wasn’t discovered by a new method. Someone ran careful ablations on both axes at once. Most papers don’t.

Where the field actually is

Read chronologically, the 18 papers tell a story that isn’t captured by any single headline.

Human annotation is being systematically eliminated. SELF-INSTRUCT bootstrapped instruction data from 175 seed examples. ZEPHYR replaced human feedback with GPT-4 scoring. Qwen3 distills from a 235B teacher model to train smaller students. The end state is a large model teaching a small model with no human in the loop. But there’s a hard ceiling: a student cannot exceed its teacher through distillation alone. No paper has demonstrated a path past this limit.

Small models fail differently from large ones, and the field doesn’t have good language for this yet. Below about 3 billion parameters, models struggle with exploration during RL — they overfit SFT patterns and can’t meaningfully search the space of possible solutions. Distillation beats online RL for small models. Above about 7B, GRPO with exploration starts to work. The strategies that work at one scale fail at another, and most papers only test at one scale.

Training is curriculum design whether the authors use that language or not. Phi-1 goes textbook → exercises. SmolLM2 rebalances data mixtures mid-flight. Qwen3 runs cold start → GRPO → fusion → GRPO again, with different data and different objectives at each stage. A paper on SFT-GRPO data overlap found that reusing the same data across stages costs 10 percentage points. GRPO needs novel prompts the model hasn’t seen — not repetition of previous material, but the next level of the curriculum.

Perhaps the most useful single insight across all 18 papers: what you exclude from training matters as much as what you include. Phi-1 removed 40% of its initial dataset and barely saw a performance drop. The SFT-GRPO overlap penalty is really about the cost of including stale data. Bad training data is net-negative. Curate by exclusion first — remove noise, repetition, and style inconsistency — then add volume.

What I’m doing with this

I have an RTX 2070 Super with 8GB of VRAM and an M3 MacBook Pro with 36GB. Between them I can run a 4B model in 4-bit, fine-tune with LoRA at rank 16, and fit 32K context on the RTX, or push to 7B–14B models on the Mac for larger experiments. GRPO with 64 samples per query is cheap on a 0.5B model and expensive on a 7B — start small, verify the approach works, then scale.