daBongo LMS AI Training Courses

Claude API in Practice – A Complete Developer Reference

Lesson 2: System Prompts and Conversation Management

Lesson Objectives

By the end of this lesson, students should be able to:

  • Design system prompts for consistent, reliable application behavior
  • Apply the system prompt structure that covers role, task, format, and constraints
  • Implement conversation history management within context window limits
  • Use system prompts to handle out-of-scope inputs gracefully

Lesson Content

System prompt design for applications.

The system prompt is the persistent instruction layer for every conversation in your application. In a production API integration, the system prompt does the work that a human moderator does in a manual workflow: it establishes the model's role, constrains its behavior, specifies output format, and handles edge cases.

The four-component system prompt structure.

“` [ROLE] You are a customer support assistant for Acme Software. You answer questions about Acme's products and help users troubleshoot issues.

[TASK SCOPE] Only answer questions about Acme's products. If a user asks about a competitor's product, acknowledge the question and redirect to Acme's equivalent feature.

[OUTPUT FORMAT] Respond in plain text. Keep responses under 150 words unless the issue requires longer explanation. Use numbered steps for troubleshooting instructions.

[EDGE CASES] If you do not know the answer to a product question, say "I don't have information on that – please contact support@acme.com." Do not guess or fabricate product details. “`

This four-component structure – role, task scope, output format, edge cases – covers the main failure modes of poorly-specified system prompts.

Conversation history management.

Context windows are finite. A long conversation consumes tokens from both ends: the system prompt consumes input tokens on every request; a long conversation history also consumes input tokens. At some point, the conversation history exceeds the available context.

Strategies for conversation history management:

  • Sliding window: Keep only the last N turns. Simple but loses early context.
  • Summarization: Periodically summarize earlier turns into a compressed context block and replace the full history with the summary.
  • Selective retention: Keep only turns that are explicitly relevant to the current request (requires application logic to determine relevance).

For most applications, a sliding window of the last 10-20 turns is sufficient. For long-session applications (document editing, extended research), a summarization strategy is necessary.

Handling out-of-scope inputs.

Every production application receives out-of-scope inputs – users will ask your AI assistant things it was not designed to answer. Handle this explicitly in the system prompt rather than leaving it to Claude's judgment:

If the user asks about topics outside [your defined scope], respond: "I'm here to help with [scope]. For [out-of-scope topic], I'd recommend [alternative resource]."

Explicit handling in the system prompt produces consistent, on-brand responses to out-of-scope inputs rather than variable behavior.

Practical Example

A developer's legal research assistant has no system prompt constraint on out-of-scope inputs.

Users start asking it for medical advice (adjacent domain, same "professional information" mental model).

It answers – inconsistently, sometimes accurately, sometimes not.

He adds a one-line system prompt instruction: "If asked for medical, financial, or other non-legal professional advice, respond: 'I specialize in legal research.

For [topic], please consult a qualified [professional].'" Out-of-scope handling becomes consistent and on-brand in one prompt change.

Safety Notes

System prompts for user-facing applications should include explicit content safety instructions relevant to your application's domain. Claude has default safety behaviors, but application-level system prompt instructions provide an additional layer specific to your context and user base. For applications where users may be vulnerable (minors, mental health contexts, crisis situations), domain-specific safety instructions in the system prompt are essential – not optional.

Log in and enroll to access lesson quizzes.

Scroll to Top