Short answer: the system prompt holds who your agent is and how it behaves, the knowledge base holds what it can look up, and project files hold the work in front of it right now. Get those three mixed up and you get an agent that is busy but not useful. I have watched founders retype the same instructions into the same chat box four times in one week, and that is always the tell.
What is the difference between a system prompt, a knowledge base, and project files?
Picture a new hire walking in on Monday.
The system prompt is your first conversation with them. Who you are, what you sell, how you talk to people, what you are never allowed to do.
The knowledge base is the shared drive you hand over on day two. Everything true about the business that will still be true next quarter.
Project files are the folder for the job they are doing this week. One client, one launch, one proposal, one set of numbers.
Before I file any piece of context, I ask three questions in order. If this changed, would it change how the agent behaves on every task? Then it goes in the system prompt. Would a smart new hire have to look it up because there is no way they would know? Then it goes in the knowledge base. Is it only true for this project, this client, or this month? Then it goes in project files.
That is most of context engineering for founders. Not prompt tricks. Filing.
What should go in the system prompt?
The system prompt is the constitution. It is not the case law.
Five things belong in it. Who you are and what the business does, in ten lines or less. How you sound, with two or three real sentences you have actually written as examples. Hard rules, things like never quote a price that is not in the knowledge base and never promise a delivery date. Output shape, meaning how long the answer should be and what format it takes. And an escape hatch: if you do not know, ask me one question before you answer.
That is it. Mine runs about 400 words. Every time I get tempted to add a paragraph, I ask whether a new hire would need it on their first day or their thirtieth. Thirtieth means knowledge base.
Two concrete places to put it: the instructions field of a Claude Project, or the system message node in n8n or Make if you are wiring up no code AI agents. Write the prompt in a Google Doc first, date it at the top, and paste it in. Reading it out loud catches the parts that sound like a legal disclaimer, which is usually the part the model ignores anyway.
What should go in the knowledge base?
Facts. The stuff that is true whether or not a client is on the phone.
Your offers and what they cost. What is included and what is not. The five objections you hear every week, with the answer you actually give. Your process, step by step. Onboarding. Three pieces of past work you are proud of. Transcripts of you talking about the business, because those teach voice better than any list of adjectives.
Tools: Notion or Google Drive as the source of truth. If the agent needs to search, Pinecone, Supabase, or Airtable as the retrieval layer. For most founders one to three years in, you can attach the folder to a Claude Project or a Custom GPT and skip the vector database completely.
Three steps that matter more than the tool. One doc per question, named the way a client would ask it, so how-much-does-the-membership-cost.md instead of pricingv3FINAL. Date stamp every doc at the top, because stale pricing is the number one cause of embarrassing agent output. And cut anything you would not say out loud to a client's face.
When I teach AI your business, the line that lands hardest is this: your knowledge base is not your website. Your website is marketing. Your knowledge base is the truth you would tell a friend who asked what you actually do all day.
What should go in project files?
The live working set. A client brief, this month's numbers, yesterday's call transcript, the draft you are editing right now.
Claude Projects and ChatGPT Projects are built for exactly this. A plain Markdown folder on your desktop synced to Dropbox works too, and so does Cursor if you like editing in one place. I keep a folder called _context and drop a file in it whenever a project starts.
Two rules. Put the deadline in the folder name so you can see what is live. Archive the whole thing when the project ships, otherwise your agent will cheerfully mention a client you fired in March to a prospect in September. That happened to me once. It was not a fun email to write.
How do I know when something is in the wrong place?
Three symptoms, three fixes.
The agent keeps restating rules you already gave it. That is prompt bloat. Move the detail down into the knowledge base.
The agent invents a price or a policy. That is a knowledge gap. Write the doc, name it like a question, attach it.
The agent brings up a dead client or last quarter's numbers. That is project file rot. Archive the folder and start a fresh one.
I run multiple AI companies, resorts, wedding venues, and a slew of other businesses, and each one gets its own three drawers. Treat them as one file system, not three separate tools you maintain on vibes. That is the whole point of an AI agent operating system. The context lives somewhere deliberate, so every chat does not start from zero.
FAQ
Can I put everything in the system prompt?
You can, and it works for about two weeks. Then the prompt is 4,000 words, the model starts ignoring the middle of it, and you cannot tell which rule broke what. Keep the prompt under 500 words and push the facts into the knowledge base.
Do I need a vector database for my knowledge base?
Not at your stage. If your knowledge fits in 30 or 40 documents, attach the folder to a Claude Project or a Custom GPT and let it read. Reach for Pinecone or Supabase when you have hundreds of documents or you need the agent searching on its own without you in the loop.
How often should I update each layer?
System prompt quarterly, or whenever your business model changes. Knowledge base whenever pricing, process, or offers change. Project files at the end of every project, archive and move on. I do a 20 minute pass on the first Monday of each month and it saves me hours of re-explaining.
I wrote the full playbook for this. How to Build Your Own AI Agent Operating System walks you through the exact architecture I use, step by step, including the prompt template, the knowledge base folder structure, and the project file rules. You can get it at a.mastermindshq.business/ai-os-book.