System vs user, and conversation hygiene
Artifact: your default system prompt vs user prompt — what goes where
1. System vs user, in one line
When you have both — most chat interfaces and all API access do — the rule is simple:
- System prompt = things that should be true across every turn.
- User prompt = the specific request for this turn.
System is the constant. User is the variable. Put role, format defaults, banlists, output style, behavior guarantees in system. Put the actual task in user.
The mistake is jamming everything into the user prompt every time. It works, but it's wasteful — you re-establish the same context every turn, and any drift in how you phrase it causes drift in the output.
2. What goes in system
A working set:
- Role and behavior. You are a code reviewer. You skip style nits.
- Format defaults. Always return JSON unless explicitly asked for prose. No preamble.
- Banlist. Never use "I'm sorry, but," "as an AI," "delve into."
- Output guarantees. If unsure, say "I don't know" instead of guessing.
- Voice. Direct. Plain words. Short sentences.
All of these are stable across tasks. Move them out of every user prompt and into the system.
3. Conversation hygiene
After a few turns in a conversation, two things happen:
- The context gets long. The model starts forgetting earlier turns.
- The context gets contaminated. Earlier wrong attempts color later ones.
Three rules for keeping conversations healthy:
One: reset when the topic changes. If you're done with the current task and starting a new one, open a new conversation. Carrying over context that isn't relevant just confuses the model.
Two: prune when you can. If your chat interface supports it, remove earlier turns that didn't go anywhere. Wrong attempts pollute the model's read of what you want.
Three: explicitly restate, mid-conversation, when needed. If you've changed direction or constraints, say so. Forget what we were trying earlier. New constraint: X. The model needs the bookmark.