Context Engineering: The Real AI Bottleneck Explained
Stack Overflow says the AI bottleneck isn't model smarts, it's context. Here's what context engineering means, and why it's a free skill any Sri Lankan builder can practice.
Context engineering is the skill I now think separates people who get real work out of AI from people who get impressive-looking demos and nothing shippable. That framing comes from a piece I read this week: Stack Overflow's No Dumb Questions episode, "What is the AI bottleneck?", where their Director of Data Science, Michael Foree, explains the idea to a colleague named Phoebe.
The headline claim is the interesting bit. The model isn't the problem anymore. The plumbing around it is.
🔍 The bottleneck isn't intelligence, it's connection
Foree's definition of the bottleneck is blunt: AI is "competent and capable of doing most of the things that people want it to do," but it can't reach the actual context of your everyday work. The capability is sitting there. The wiring to your real files, threads, and tools is not.
That reframes a lot of the "AI can't do my job" complaints. Usually the model isn't failing at reasoning. It's failing because it never got the inputs a human would have gathered without thinking.
The article gives three grounded examples:
| Task | Where AI stalls |
|---|---|
| Replying to an email | You must manually gather the thread, related docs, Slack messages, and meeting context |
| Redesigning a room | AI picks paint and furniture, but can't iterate fast across disconnected tools |
| Grocery shopping | It builds the list fine, but ordering needs multiple store APIs with unclear incentives |
Key takeaway: In every example, the model was capable. What was missing was access to the right information and the ability to act on it. That gap is the bottleneck.
🛠️ What "context engineering" actually means
Foree defines context engineering as deliberately identifying and providing the AI with relevant information sources while filtering out distractions. His one-line version: "You have to stop and think about what you're doing."
That sounds soft until you notice it's the opposite of how most people use a chatbot. The default habit is to type a vague request and hope. Context engineering is the discipline of assembling the input on purpose.
I'd break the job into three moves:
- Include the sources that actually matter to the answer.
- Exclude the noise that will pull the model off track.
- Iterate on that selection and write down what worked.
The reason exclusion matters as much as inclusion: the article notes AI "gets distracted by irrelevant information." More context is not automatically better context. A bloated prompt full of unrelated dumps can produce a worse answer than a tight one.
📊 Why this is good news if you're on a small budget
Here's the part that matters for a student in Kandy or a two-person startup in Colombo. Context engineering is a skill, not a subscription.
You don't need a bigger model or an enterprise plan to practise it. You need to think harder about what you paste in. That levels the field in a way most AI advantages don't.
| Advantage | Costs money? | Who can practise it |
|---|---|---|
| Access to the biggest frontier model | Usually yes | Anyone with budget |
| Massive proprietary data warehouse | Yes | Enterprises |
| Context engineering | No | Anyone, including free-tier users |
There's one real constraint worth watching: the context window. Every model has a fixed budget for how much text it can hold at once, and long dumps get expensive on paid APIs and can crowd out the useful bits. If you're piecing together email threads, docs, and code into a single prompt, it helps to know roughly how much you're stuffing in. Our AI context window calculator and token counter are free ways to check that before you hit a limit or a bill.
Filtering isn't just about accuracy. On a metered API, every irrelevant paragraph you leave out is money you keep.
💡 How to actually get better at it
Foree's advice for becoming a better context engineer is to "observe and wonder." Pay attention to which information sources matter, what should be excluded, and what distractions keep creeping in. Then test, iterate, and document.
I'd turn that into a routine you can run this week:
- Before the prompt: ask "what would a competent colleague need to answer this?" Gather exactly that.
- During: watch where the model guesses or drifts. A guess usually means a missing input, not a dumb model.
- After: save the prompts that worked. Your reusable context beats a one-off clever question every time.
A small worked example. Say you're a freelancer drafting a reply to an overseas client about a late invoice. The lazy prompt is "write a polite follow-up about my payment." The engineered version includes the original invoice terms, the two prior emails, the agreed currency, and a note to keep it short and non-pushy. Same model, completely different output. The second one is usable; the first one is a form letter.
That's the whole discipline in one message: you did the gathering the model couldn't do for itself.
What this means for you
The Stack Overflow piece is really an argument about where the value is moving. If the models are already "competent," then the edge isn't owning the smartest one. It's feeding whatever model you have the right context and cutting the rest.
For a Sri Lankan builder, student, or small team, that's the encouraging read:
- You don't need the biggest model to get strong results.
- The scarce skill is judgement about inputs, and that's free to build.
- Documenting your good prompts compounds. Treat them like reusable code.
The people who win the next couple of years won't be the ones with the fanciest subscription. They'll be the ones who got disciplined about what they put in front of the model, and left the noise out. Start noticing your own context now, while it's still a rare habit.