Prompt Engineering for Devs

The difference between a mediocre AI feature and a great one is often just the prompt. This isn't about tricks - it's about clear communication.

---

The System Prompt

The system prompt sets the rules before any user input. It's your most powerful tool.

Bad System Prompt

messages = [
    {"role": "system", "content": "You are helpful."},
    {"role": "user", "content": "Summarize this article..."}
]

Good System Prompt

messages = [
    {
        "role": "system",
        "content": """You are a content summarizer for a tech news app.

TASK: Summarize articles for busy developers.

RULES:
- Maximum 3 bullet points
- Each bullet under 20 words
- Focus on practical implications, not hype
- If the article is not tech-related, respond with "NOT_RELEVANT"

OUTPUT FORMAT:
- Bullet 1
- Bullet 2
- Bullet 3"""
    },
    {"role": "user", "content": "Summarize this article..."}
]

---

Anatomy of a Good System Prompt

---

JSON Mode: Structured Outputs

When you need to parse the response in code, use JSON mode:

from openai import OpenAI
import json

client = OpenAI()

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {
            "role": "system",
            "content": """Extract product info from descriptions.
            Return JSON with: name, price, category, in_stock (boolean)"""
        },
        {
            "role": "user",
            "content": "The Nike Air Max 90 is available now for $150"
        }
    ],
    response_format={"type": "json_object"}  # Forces valid JSON
)

data = json.loads(response.choices[0].message.content)
print(data)
# {"name": "Nike Air Max 90", "price": 150, "category": "shoes", "in_stock": true}

---

Few-Shot Prompting

Show the model examples of what you want. This is the most effective technique for consistent outputs.

Zero-shot (No examples)

messages = [
    {
        "role": "system",
        "content": "Convert natural language to SQL."
    },
    {
        "role": "user",
        "content": "Show me all users who signed up last week"
    }
]
# Output might be inconsistent

Few-shot (With examples)

messages = [
    {
        "role": "system",
        "content": """Convert natural language to SQL for a users table.
        Table schema: users(id, email, name, created_at, plan)"""
    },
    # Example 1
    {
        "role": "user",
        "content": "Get all premium users"
    },
    {
        "role": "assistant",
        "content": "SELECT * FROM users WHERE plan = 'premium';"
    },
    # Example 2
    {
        "role": "user",
        "content": "Count users by plan"
    },
    {
        "role": "assistant",
        "content": "SELECT plan, COUNT(*) FROM users GROUP BY plan;"
    },
    # Actual query
    {
        "role": "user",
        "content": "Show me all users who signed up last week"
    }
]
# Output: SELECT * FROM users WHERE created_at >= NOW() - INTERVAL '7 days';

---

Chain-of-Thought Prompting

For complex reasoning, ask the model to think step-by-step:

messages = [
    {
        "role": "system",
        "content": """You are a code reviewer. When reviewing code:
        1. First, identify what the code is trying to do
        2. Then, list any bugs or issues
        3. Then, suggest improvements
        4. Finally, give an overall rating (1-5)

        Think through each step before giving your final answer."""
    },
    {
        "role": "user",
        "content": """Review this code:
        def get_user(id):
            user = db.query(f"SELECT * FROM users WHERE id = {id}")
            return user"""
    }
]

The model will now explain its reasoning, catching issues it might miss with a direct response (like the SQL injection vulnerability above).

---

Practical Template: API Endpoint Prompt

Here's a reusable template for backend tasks:

def create_system_prompt(task: str, output_schema: dict, rules: list[str]) -> str:
    return f"""You are a backend service component.

TASK: {task}

OUTPUT: Return valid JSON matching this schema:
{json.dumps(output_schema, indent=2)}

RULES:
{chr(10).join(f"- {rule}" for rule in rules)}

If you cannot complete the task, return:
{{"error": "reason"}}
"""

# Usage
prompt = create_system_prompt(
    task="Extract action items from meeting notes",
    output_schema={
        "action_items": [{"task": "string", "assignee": "string", "due": "string"}]
    },
    rules=[
        "Only include explicit action items, not general discussion",
        "If no assignee mentioned, use 'unassigned'",
        "If no due date mentioned, use 'no date'"
    ]
)

---

Common Mistakes

---

Prompt Testing Checklist

Before deploying a prompt to production:

Key Takeaways

Next up: Embeddings & Vector Search - how to search by meaning, not just keywords.