Every week I get asked some version of the same question: “We want to add AI. Where do we start?”
The honest answer, until recently, was: “It depends.” And it still does. But now I have a cleaner framework for the conversation — one that centers on the Model Context Protocol, or MCP.
MCP is one of those things that’s easier to understand with a metaphor. So let’s start there.
The Library vs. the Librarian
Imagine your product is a library. It has shelves of content, data, functionality — all organized the way you organized it, for reasons that made sense at the time. Users come in knowing what they want (sometimes) and spend a variable amount of time finding it (always longer than it should take).
A traditional search bar is a card catalog. It finds things by name, by keyword, by tag. Precise, fast, literal.
An LLM connected to your product is a librarian. A librarian who has read everything in the building, understands what you’re actually trying to do when you ask a question, and can say: “That’s not in this section, but what you’re probably looking for is on shelf 4B, and by the way, have you seen the new collection that just arrived that’s very relevant to your project?”
MCP is how you tell that librarian which shelves they’re allowed to access, how to read the catalog, and when to escalate to a human.
What MCP Actually Is
The Model Context Protocol is an open standard — originally developed by Anthropic — that defines how a large language model accesses external tools and data sources. It’s the connective tissue between an LLM and your product’s actual capabilities.
Without MCP (or an equivalent), LLMs are trapped inside a conversation window. They know what you’ve told them in the chat. They can reason and generate, but they can’t do. They can’t query your live data. They can’t take actions in your system. They can’t remember what happened in the last session unless you feed it back manually.
With MCP, your LLM becomes an agent. It can read, call, and act on things that actually exist in your product. User preferences. Content libraries. Behavioral history. Product inventory. Session state.
The Five Layers of an AI-Enabled Product
When I was working on AI integration at Skillshare, I found it helpful to think in layers. Here’s how I frame it now:
Layer 1 — UI Layer: Where the user interacts with AI. Chat interfaces, inline suggestions, smart search bars, AI-generated recommendations. What the user sees.
Layer 2 — LLM Layer: The reasoning engine. Takes context, applies instruction and capability, generates output. This is where prompt engineering and model selection decisions live.
Layer 3 — Retrieval Layer: How the LLM gets relevant context from your data before answering. Usually built on vector search (semantic matching) rather than keyword search. This is where RAG (Retrieval-Augmented Generation) patterns live.
Layer 4 — Embedding/Index Layer: The transformation layer. Your content is converted from text into vectors — mathematical representations of meaning — and indexed so the retrieval layer can find the most semantically relevant material quickly.
Layer 5 — Data Layer: Your actual product data. Course catalog, user history, creator profiles, engagement signals. This is what you’re making intelligent.
MCP primarily operates between Layers 2 and 5 — defining what tools and data the LLM can reach, and how it should call them.
Designing an MCP Integration: The PM’s Checklist
Before you spec an MCP layer, answer these questions:
1. What decisions do you want the LLM to help users make? Not “what does AI do” — what is the user trying to decide? At Skillshare, the key decisions were: “What should I learn next?” and “Which teacher is right for my goal?” Everything flows from those decisions.
2. What data does the LLM need to make good recommendations? List the specific data sources. User profile. Completion history. Skills tag taxonomy. Content embeddings. This list becomes your MCP tool definition — each data source is a tool the LLM can call.
3. What actions, if any, should the LLM be able to take? Reading data is low risk. Taking actions — enrolling a user, sending a message, updating a preference — is higher risk. For most products starting out: read-only MCP tools first. Actions come after you trust the model’s reasoning.
4. How will you prevent hallucination for facts specific to your product? LLMs make things up. For general knowledge, that’s sometimes acceptable. For your specific product content (“Is there a course on watercolor for beginners with reviews?”), hallucination is unacceptable. Your retrieval layer is what prevents this — the model answers from retrieved facts, not from training data.
5. How will you handle context window limits? Every LLM conversation has a token budget. If your product has a large catalog, you can’t stuff it all into context. Your retrieval layer should return the N most relevant items (typically 5–20), not everything. Design for precision, not recall.
The Design Principle I Keep Coming Back To
AI augments, it doesn’t replace.
At Skillshare, the temptation was to build a “just ask the AI” interface that replaced the browse experience. We didn’t do that — and I think we were right not to. Browse is still faster for serendipitous discovery. Search is still better when you know exactly what you want. The AI layer is best when the user is mid-journey: they’ve started something, they’re not sure where to go next, and they need a knowledgeable guide, not a catalog.
The MCP layer should make that specific moment dramatically better, not compete with every mode of interaction.
Common Mistakes Product Teams Make
Treating MCP as a chat widget. MCP is infrastructure. The interface can be conversational, but the value comes from deep product integration, not a floating chat box.
Starting with actions before reads. Letting an LLM write to your system before you’ve validated its accuracy in read-only mode is asking for trouble. Walk before you run.
Ignoring latency. LLM + retrieval + reranking adds up. For inline suggestions or search completion, you’re aiming for sub-500ms. This usually means pre-computing embeddings, caching frequent queries, and carefully scoping what the model needs to evaluate.
Not logging reasoning. Log what context the LLM was given and what it returned. You’ll need this for debugging, for compliance, and for the inevitable moment when a user gets a bizarre recommendation and your engineers need to trace why.
The Question Worth Asking First
Before you build any of this: “What would a great human assistant know, and how would they use that knowledge?”
That question keeps AI product design grounded. It reminds you that LLMs are good at the things humans are good at — understanding intent, synthesizing across many inputs, explaining tradeoffs — and less good at things that require real-time precision or guaranteed correctness.
Design the MCP layer to give your AI the knowledge a great human assistant would have. Then get out of the way.
This framework draws from my work designing Skillshare’s AI product strategy and MCP integration. For the full technical and product context, see the Skillshare AI case study.