Claude Certified Architect - Foundations Practice Test

Claude Certified Architect - Foundations overview

  • 200 practice questions
  • 32 flashcards
  • 7 study guide topics
  • 46 key terms

What you'll study

  • Claude Fundamentals and Model Capabilities15%
  • Prompt Engineering and System Prompt Design20%
  • Tool Use and Agentic Workflows20%
  • Context Management and Retrieval-Augmented Generation15%
  • Safety, Guardrails, and Responsible AI10%
  • Evaluation, Testing, and Observability10%
  • Deployment, Integration, and Production Operations10%

The Claude Certified Architect - Foundations credential is aimed at engineers, solution architects, and technical leads who design and ship applications built on Claude. It validates that you can reason about model behaviour and limits, write system prompts and prompts that hold up under real inputs, design tool-use and agentic loops that fail safely, manage context and retrieval, apply guardrails, set up evaluation and observability, and operate a deployment in production. The exam is multiple choice and is weighted toward prompt engineering and agentic workflows, with smaller shares for fundamentals, context management, safety, evaluation, and operations. It tests judgement rather than memorised API trivia, so expect scenario questions where two answers are defensible and one is better because it handles failure, cost, or safety more cleanly. Preparation that works: build a small application that uses tools and retrieval end to end, keep notes on why you chose each design, then work through practice questions and justify every wrong option before moving on. This page is an independent study resource and is not affiliated with or endorsed by Anthropic; confirm all administrative details with the official certification body.

Common mistakes

Treating the model's context window as a fixed hard limit and assuming any conversation that fits is equally reliable, rather than recognizing degradation and 'lost in the middle' effects on long inputs.

Candidates memorize a single context-window number per model and assume recall is uniform across the window, so they miss questions about placing critical instructions or retrieved evidence near the start or end of long contexts.

What to do instead

Study how position affects recall in long contexts and the principle of putting the most important instructions and evidence at the beginning or end. Practice reasoning about context budget trade-offs between history, retrieved documents, and instructions.

Designing a tool-use loop that calls tools but never validates arguments, never handles tool errors, and never sets a termination condition, producing infinite or runaway agent loops.

Candidates focus on the happy path of 'model requests tool, tool returns result' and forget production concerns like schema validation, error feedback to the model, iteration caps, and stopping criteria.

What to do instead

For every agentic design question, explicitly include: strict input schema validation, structured error messages returned to the model, a maximum iteration/turn limit, and a defined success/termination condition.

Writing system prompts that pile on vague adjectives ('be helpful, accurate, and safe') instead of concrete behavioral rules, output format specs, and explicit refusal/edge-case instructions.

Candidates equate longer or more emphatic prompts with better prompts and underestimate how ambiguity in instructions produces inconsistent outputs that are hard to evaluate.

What to do instead

Practice rewriting prompts into explicit, testable requirements: role, task, constraints, output schema, examples, and handling for ambiguous or out-of-scope inputs. Evaluate whether each sentence is verifiable.

Confusing prompt engineering with fine-tuning or RAG, and recommending the wrong intervention (e.g., proposing fine-tuning to inject frequently changing factual knowledge).

Candidates learn the techniques in isolation and do not build a decision framework mapping problem types (knowledge freshness, format compliance, tone, latency, cost) to the right solution.

What to do instead

Build a comparison table: prompting for behavior/format, RAG for dynamic or proprietary knowledge with citations, fine-tuning for stable style/format at scale. Match each exam scenario to the cheapest adequate intervention.

Assuming adding a retrieval step automatically improves answers, without addressing chunking strategy, embedding quality, reranking, or grounding/citation of retrieved content.

Candidates treat RAG as a single checkbox and skip the pipeline details that determine whether retrieved context is relevant and whether the model actually uses it faithfully.

What to do instead

Study each RAG stage independently: chunk size/overlap trade-offs, embedding model choice, hybrid/keyword plus vector retrieval, reranking, and prompting the model to cite or abstain when evidence is insufficient.

Designing safety guardrails as a single input filter, ignoring output-side checks, tool-permission scoping, and defense against prompt injection from retrieved or tool-returned content.

Candidates picture safety as blocking bad user prompts and forget that untrusted content entering via RAG or tools can hijack the model, and that outputs can leak data or take unsafe actions.

What to do instead

Layer defenses: input validation, treating retrieved/tool content as untrusted data (not instructions), least-privilege tool scopes, human approval for high-impact actions, and output filtering. Articulate defense-in-depth in answers.

Proposing an evaluation plan based only on 'eyeballing outputs' or a single accuracy metric, with no golden dataset, no rubric, and no LLM-as-judge calibration.

Candidates underinvest in evaluation methodology and do not know how to make automated grading trustworthy, so they miss questions about judge bias, rubric design, and regression testing.

What to do instead

Learn to specify: a curated eval set with representative and adversarial cases, explicit rubrics, LLM-as-judge with human spot-checks and position-bias mitigation, and running evals as regression tests on every prompt/model change.

Ignoring operational realities in deployment answers: no handling of rate limits, retries with backoff, timeouts, streaming, token/cost budgeting, or latency trade-offs.

Candidates answer architecture questions at the whiteboard level and omit the failure handling and cost controls that distinguish a production system from a prototype.

What to do instead

For any deployment scenario, enumerate: idempotent retries with exponential backoff and jitter, timeout and fallback behavior, streaming for perceived latency, token accounting and caching, and monitoring/alerting on error rates and cost.

Choosing the largest or most capable model by default instead of matching model tier to task complexity, latency, and cost requirements.

Candidates equate 'best' with 'biggest' and do not consider that many tasks (classification, extraction, routing) are well served by smaller, faster, cheaper models with proper prompting.

What to do instead

Practice a model-selection framework: define quality bar, latency budget, and cost ceiling per task, then start with the smallest model that passes evals and escalate only when evaluations show it is needed.

Treating observability as basic logging of requests, without capturing prompts, model versions, tool calls, token usage, and trace context needed to debug and reproduce failures.

Candidates think of observability as generic application logging and miss the LLM-specific signals (prompt/response pairs, sampling, traces across multi-step agent runs) required for diagnosis.

What to do instead

Design traces that capture full prompt/response, model and prompt version, tool invocations and results, token counts, latency, and a correlation ID across agent steps, with PII redaction and sampling for cost control.

Exam requirements can change. Confirm current details with the exam provider.

Claude Certified Architect - Foundations practice

200 questions. Each one lists the choices, the correct answer, and an explanation. With JavaScript, this list opens one question at a time. The checker at the top of this page saves progress in this browser when it is available.

  1. Question 1Claude Fundamentals and Model Capabilities

    Which statement best describes how a large language model generates a response to a prompt?

    1. It searches the public internet in real time before every reply.
    2. It compiles the prompt into executable code and runs it.
    3. It predicts the next token repeatedly based on patterns learned during training. Correct answer.
    4. It retrieves a stored answer from a fixed database of prior conversations.

    Explanation

    Language models generate text autoregressively: at each step the model estimates a probability distribution over the next token given the preceding context, samples or selects a token, and repeats. This is a learned statistical process, not retrieval from a fixed answer store or code execution.

  2. Question 2Claude Fundamentals and Model Capabilities

    A prompt includes a long document followed by a question about it. What is the primary role of the model's context window in this scenario?

    1. It sets the maximum length of the model's final answer only.
    2. It controls how many documents the model can store permanently.
    3. It limits the number of users who can query the model concurrently.
    4. It determines the maximum number of tokens the model can attend to at once. Correct answer.

    Explanation

    The context window bounds the total tokens the model can consider in a single request, including system prompt, input, and generated output. It is a per-request attention limit, not persistent storage or a concurrency control.

  3. Question 3Claude Fundamentals and Model Capabilities

    Which technique gives a model access to current information beyond its training cutoff at inference time?

    1. Providing retrieved documents in the prompt Correct answer.
    2. Increasing the temperature setting
    3. Fine-tuning on historical text
    4. Reducing the max output tokens

    Explanation

    Supplying fresh information as retrieved context (retrieval-augmented generation) lets the model reason over data it never saw in training. Temperature, token limits, and fine-tuning on old data do not introduce new knowledge.

  4. Question 4Claude Fundamentals and Model Capabilities

    What does a model's 'temperature' parameter primarily control?

    1. The number of tokens generated per second
    2. The randomness of token sampling during generation Correct answer.
    3. The size of the context window
    4. The cost per input token

    Explanation

    Temperature scales the sampling distribution: lower values make output more deterministic and higher values increase diversity. It does not affect throughput, context size, or pricing.

  5. Question 5Claude Fundamentals and Model Capabilities

    Which of the following is a core capability that distinguishes modern instruction-following models from simple text-completion models?

    1. Guaranteeing factual correctness on every query
    2. Following multi-step natural-language instructions Correct answer.
    3. Encrypting the prompt before processing
    4. Executing arbitrary shell commands natively

    Explanation

    Instruction tuning trains models to interpret and carry out natural-language directives, including multi-step tasks. They do not encrypt prompts, guarantee factual accuracy, or natively run shell commands without tools.

  6. Question 6Claude Fundamentals and Model Capabilities

    Why does providing a few worked examples in a prompt (few-shot prompting) often improve output quality?

    1. It increases the model's context window size.
    2. It permanently updates the model's weights for that task.
    3. It demonstrates the desired format and reasoning pattern in context. Correct answer.
    4. It disables the model's safety filters.

    Explanation

    Few-shot examples act as in-context demonstrations that show the model the expected structure, tone, and reasoning, improving alignment with the task without changing weights. They do not resize the context window or alter safety behavior.

  7. Question 7Claude Fundamentals and Model Capabilities

    A developer wants deterministic, repeatable outputs for a classification task. Which setting best supports this goal?

    1. A high temperature with top-p at 1.0
    2. A long system prompt with many examples
    3. A large max output token limit
    4. A low temperature such as 0 Correct answer.

    Explanation

    Setting temperature near zero makes the model favor the most probable tokens, yielding near-deterministic output. High temperature and broad top-p increase randomness, and token limits or long prompts do not guarantee determinism.

  8. Question 8Claude Fundamentals and Model Capabilities

    Which statement about a model's knowledge cutoff is accurate?

    1. The cutoff can be extended by raising the temperature.
    2. The cutoff only affects the model's output length.
    3. The model's training data ends at a certain date, so later events may be unknown. Correct answer.
    4. The model automatically learns new facts after each conversation.

    Explanation

    The knowledge cutoff marks the end of the training data, so events after that date are not reliably known unless supplied in context. Models do not learn from conversations, and the cutoff is unrelated to output length or temperature.

  9. Question 9Claude Fundamentals and Model Capabilities

    A team needs the model to reliably return JSON matching a specific schema. Which approach most directly improves reliability?

    1. Increasing the max output tokens substantially
    2. Removing all formatting instructions to avoid bias
    3. Describing the schema explicitly and providing a valid example in the prompt Correct answer.
    4. Raising temperature to encourage creativity

    Explanation

    Clearly specifying the schema and showing a conforming example gives the model an explicit target format to imitate. Higher temperature, removed instructions, or larger token limits do not enforce structure.

  10. Question 10Claude Fundamentals and Model Capabilities

    What is a 'system prompt' used for in a Claude-based application?

    1. Storing the model's training weights
    2. Configuring the network endpoint for API calls
    3. Defining the model's tokenizer vocabulary
    4. Setting persistent role, tone, and behavioral guidance for the conversation Correct answer.

    Explanation

    The system prompt establishes durable instructions such as persona, constraints, and tone that apply across the conversation. It does not store weights, define tokenization, or configure networking.

  11. Question 11Claude Fundamentals and Model Capabilities

    A model gives a confident but incorrect answer about a niche fact. What is this behavior called?

    1. Tokenization
    2. Hallucination Correct answer.
    3. Quantization
    4. Overfitting

    Explanation

    Hallucination is when a model produces fluent but factually incorrect or unsupported content. Overfitting, tokenization, and quantization are training or processing concepts unrelated to this output behavior.

  12. Question 12Claude Fundamentals and Model Capabilities

    Which prompt design most reduces the chance of a model hallucinating when answering questions over a provided document?

    1. Asking the model to answer from memory first, then check the document
    2. Removing the document to keep the prompt short
    3. Increasing temperature to explore more possibilities
    4. Instructing the model to answer only from the provided document and to say when the answer is absent Correct answer.

    Explanation

    Grounding instructions that restrict answers to supplied context, plus an explicit fallback when information is missing, reduce fabrication. Answering from memory first, raising temperature, or removing the source all increase hallucination risk.

  13. Question 13Claude Fundamentals and Model Capabilities

    What is the main benefit of 'chain-of-thought' style prompting for complex reasoning tasks?

    1. It guarantees mathematically correct results.
    2. It eliminates the need for a system prompt.
    3. It reduces the number of tokens the model must generate.
    4. It encourages the model to work through intermediate steps before concluding. Correct answer.

    Explanation

    Chain-of-thought prompting elicits intermediate reasoning steps, which often improves accuracy on multi-step problems. It typically increases token usage, does not guarantee correctness, and is independent of system prompts.

  14. Question 14Claude Fundamentals and Model Capabilities

    A model's response is cut off mid-sentence. Which parameter most likely caused this?

    1. A max output tokens limit that is too low Correct answer.
    2. Top-p set to 0.9
    3. Temperature set to 0
    4. A very short system prompt

    Explanation

    The max output tokens parameter caps response length, so a low value truncates output. Temperature, top-p, and system prompt length do not impose a hard cap on response length.

  15. Question 15Claude Fundamentals and Model Capabilities

    Which scenario is the best fit for using extended reasoning effort on a model?

    1. Reformatting a short string to uppercase
    2. Echoing a fixed confirmation message
    3. A simple one-word classification of sentiment
    4. A multi-constraint planning problem requiring careful trade-offs Correct answer.

    Explanation

    Complex problems with multiple interacting constraints benefit most from additional reasoning effort. Trivial transformations and fixed messages gain little and waste latency and tokens.

  16. Question 16Claude Fundamentals and Model Capabilities

    What does 'streaming' a model response provide to an application?

    1. A guarantee of lower token cost
    2. Permanent storage of the conversation on the model server
    3. Automatic translation into multiple languages
    4. Incremental delivery of tokens as they are generated Correct answer.

    Explanation

    Streaming returns output tokens progressively so users see partial results before completion. It does not persist conversations, reduce token cost, or translate content.

  17. Question 17Claude Fundamentals and Model Capabilities

    A prompt asks the model to summarize a 200-page report that exceeds the context window. Which approach is most appropriate?

    1. Ask the model to ignore the length limit
    2. Raise temperature until the whole document fits
    3. Chunk the document, summarize each chunk, then combine the summaries Correct answer.
    4. Disable the system prompt to free space

    Explanation

    When input exceeds the context window, chunking with per-chunk summaries followed by a combining pass (map-reduce summarization) handles the document within limits. Temperature, system prompts, and instructions cannot expand the window.

  18. Question 18Claude Fundamentals and Model Capabilities

    Which practice best mitigates prompt injection when a model processes untrusted user content?

    1. Increasing the model's temperature
    2. Removing all system instructions
    3. Trusting all content in the prompt equally
    4. Separating untrusted content and instructing the model to treat it as data, not instructions Correct answer.

    Explanation

    Delimiting untrusted content and telling the model to treat it as data rather than commands reduces the risk that embedded text hijacks behavior. Trusting content, removing instructions, or raising temperature weakens defenses.

  19. Question 19Claude Fundamentals and Model Capabilities

    What is the purpose of the 'assistant' turn in a multi-turn conversation structure?

    1. It stores the model's configuration parameters.
    2. It represents prior model outputs that the model can reference for continuity. Correct answer.
    3. It encrypts the conversation history.
    4. It defines the tokenizer used for the request.

    Explanation

    Prior assistant messages are part of the conversation history, letting the model maintain context and continuity across turns. They are not configuration storage, encryption, or tokenizer definitions.

  20. Question 20Claude Fundamentals and Model Capabilities

    An application needs the model to extract fields from invoices with high consistency. Which combination of techniques is most effective?

    1. Multiple contradictory instructions to test the model
    2. A vague instruction and a long max token limit
    3. High temperature with no examples
    4. Clear field definitions, a schema example, and low temperature Correct answer.

    Explanation

    Precise field definitions plus a conforming example and low temperature maximize consistency for structured extraction. High temperature, vague instructions, contradictions, or large token limits do not improve reliability.

  21. Question 21Claude Fundamentals and Model Capabilities

    A model performs well on a benchmark but poorly on real user queries. Which explanation is most plausible?

    1. The model's context window shrank after deployment
    2. The tokenizer was replaced at runtime
    3. The benchmark queries differ in distribution from real usage, revealing a generalization gap Correct answer.
    4. The temperature was permanently fixed at zero

    Explanation

    A gap between benchmark and production performance usually reflects distribution shift: benchmark examples do not represent real inputs. Context windows, temperature, and tokenizers do not spontaneously change at deployment.

  22. Question 22Claude Fundamentals and Model Capabilities

    Why can simply adding more few-shot examples eventually degrade performance?

    1. The model stops tokenizing input beyond ten examples
    2. Examples permanently overwrite the model's weights
    3. Examples consume context and can introduce conflicting or noisy patterns Correct answer.
    4. Each example doubles the inference cost of the model itself

    Explanation

    More examples consume context budget and may add inconsistent patterns that confuse the model, sometimes hurting accuracy. There is no hard example limit, weights are unchanged, and per-example cost reflects tokens rather than doubling model cost.

  23. Question 23Claude Fundamentals and Model Capabilities

    A team observes that the model ignores a constraint buried in the middle of a very long prompt. Which factor best explains this?

    1. The model has no tokenizer for mid-prompt text
    2. The system prompt was placed first
    3. Attention can be weaker for information positioned in the middle of long contexts Correct answer.
    4. The temperature was set to zero

    Explanation

    Models can attend less reliably to details in the middle of long contexts, a known positional effect. Tokenizers cover all text, and system-prompt placement or zero temperature do not explain mid-context misses.

  24. Question 24Claude Fundamentals and Model Capabilities

    Which design decision best balances latency and quality when a task has both simple and complex queries?

    1. Always use maximum reasoning effort for every query
    2. Route simple queries to fast settings and complex ones to higher reasoning effort Correct answer.
    3. Always disable reasoning to minimize latency
    4. Randomly vary temperature per request

    Explanation

    Routing by complexity allocates reasoning effort where it is needed while keeping simple queries fast, balancing quality and latency. Uniform maximum effort wastes latency, uniform minimum sacrifices quality, and random temperature adds noise.

  25. Question 25Claude Fundamentals and Model Capabilities

    A model consistently produces plausible-sounding citations that do not exist. Which mitigation is most principled?

    1. Require the model to cite only from a provided source set and verify each citation Correct answer.
    2. Remove all citation instructions
    3. Increase temperature so citations vary
    4. Ask the model to be more confident in its answers

    Explanation

    Restricting citations to a supplied source set and verifying them against that set grounds the output and catches fabrications. Confidence requests, higher temperature, or removing instructions do not prevent invented references.

  26. Question 26Claude Fundamentals and Model Capabilities

    Which statement best describes how Claude's context window constrains a single API request?

    1. It bounds the combined length of the input and the generated output. Correct answer.
    2. It applies solely to the system prompt, not to user turns.
    3. It limits only the length of the input prompt.
    4. It limits only the length of the generated output.

    Explanation

    The context window is the total token budget shared by everything in a request: system prompt, conversation history, the current user input, and the model's generated output. Input tokens and output tokens both draw from the same budget, so a long prompt leaves less room for a long response.

  27. Question 27Claude Fundamentals and Model Capabilities

    A support application wants Claude to answer strictly from a supplied policy excerpt and to say when the excerpt does not contain the answer. Which prompt design most directly enforces this behavior?

    1. Increase the temperature so the model explores more possible answers.
    2. Ask the model to answer helpfully and rely on its general knowledge to fill gaps.
    3. Provide several example questions without any instructions about grounding.
    4. Instruct the model to answer only from the excerpt and to state that the information is unavailable when it is absent. Correct answer.

    Explanation

    Grounding instructions that explicitly limit the model to the provided source and define a fallback response give the model a clear rule for when to abstain. Without such a constraint, the model may blend in parametric knowledge and produce unsupported answers.

  28. Question 28Claude Fundamentals and Model Capabilities

    A developer sends the same prompt repeatedly with temperature set to 0 and still sees occasional wording differences in the output. Which explanation is most accurate?

    1. Temperature 0 disables the model and returns a cached response.
    2. Identical output is guaranteed only when streaming is enabled.
    3. Generation remains probabilistic, so temperature 0 reduces but does not guarantee identical text. Correct answer.
    4. Temperature 0 is invalid and is silently ignored by the API.

    Explanation

    Temperature 0 makes the sampling distribution sharply peaked, but it does not remove every source of nondeterminism in generation. Small differences can still appear across runs, so applications should not treat temperature 0 as a strict reproducibility guarantee.

  29. Question 29Claude Fundamentals and Model Capabilities

    Which capability is a direct consequence of Claude being a large language model trained on text?

    1. It can browse the live internet on every request without any external integration.
    2. It can perform many language tasks, such as summarizing and classifying, without task-specific retraining. Correct answer.
    3. It can execute arbitrary code in the caller's environment without any tool configuration.
    4. It maintains a persistent memory of past conversations across separate API calls by default.

    Explanation

    A core property of large language models is general-purpose language competence: one model can handle summarization, classification, extraction, and similar tasks through prompting rather than per-task training. Persistence, code execution, and live browsing are not inherent and require explicit configuration or external systems.

  30. Question 30Claude Fundamentals and Model Capabilities

    A prompt contains a long reference section followed by the actual question. The model's answers frequently ignore details from the reference. Which change is most likely to improve adherence?

    1. Remove all formatting and line breaks from the reference section.
    2. Place the question and key instructions after the reference material so they are closest to the generation point. Correct answer.
    3. Increase the temperature to make the model attend more broadly.
    4. Move the question to the very beginning and keep the reference after it.

    Explanation

    Models tend to weight content near the end of the prompt more heavily, so positioning the question and critical instructions after long reference material improves adherence. This is a placement effect, not a sampling or formatting issue.

  31. Question 31Prompt Engineering and System Prompt Design

    Which XML-style tag structure does Anthropic recommend for separating distinct sections of a system prompt?

    1. Wrapping each section in descriptive tags such as <instructions> and <context> Correct answer.
    2. Using only numbered lists with no headings
    3. Separating sections with triple backticks and no labels
    4. Concatenating all sections into one paragraph without delimiters

    Explanation

    Descriptive tags give the model unambiguous boundaries between sections, which improves instruction-following and reduces cross-contamination between roles, context, and examples. Unlabeled delimiters leave the model to infer section purpose from content alone.

  32. Question 32Prompt Engineering and System Prompt Design

    A support assistant keeps answering with marketing language instead of the terse troubleshooting steps the product team wants. Which change to the system prompt most directly addresses this?

    1. Lower the temperature to zero
    2. Add an explicit instruction describing the desired response format and tone Correct answer.
    3. Move the user question into the system prompt
    4. Raise the max_tokens parameter so the model has room to be precise

    Explanation

    Style and format are governed primarily by explicit instructions in the system prompt; stating the desired format and tone directly constrains the output. Sampling parameters affect variability, not which style the model chooses.

  33. Question 33Prompt Engineering and System Prompt Design

    What is the primary purpose of assigning a role (for example, 'You are a senior tax analyst') in a system prompt?

    1. It steers vocabulary, depth, and perspective in responses Correct answer.
    2. It reduces the cost of each API call
    3. It disables the model's safety guidelines for that domain
    4. It guarantees factual accuracy on domain questions

    Explanation

    Role framing is a steering technique: it conditions the register, assumed expertise, and level of detail. It does not add knowledge or change safety behavior.

  34. Question 34Prompt Engineering and System Prompt Design

    You want the model to return data your application can parse reliably. Which instruction is most effective?

    1. Ask the model to 'respond in a structured way'
    2. Append 'JSON please' to the user message each time
    3. Request that the answer be short
    4. Provide a concrete output schema and an example of a valid response Correct answer.

    Explanation

    Supplying an explicit schema plus a worked example anchors the exact keys, nesting, and types expected. Vague requests for structure leave the shape underspecified.

  35. Question 35Prompt Engineering and System Prompt Design

    Which of the following best describes a 'system prompt' in the Claude API?

    1. A per-turn message that replaces the user's input
    2. Persistent instructions that establish behavior across the conversation Correct answer.
    3. A parameter that sets the model's sampling temperature
    4. A log of previous assistant responses

    Explanation

    The system prompt carries durable instructions, persona, and constraints that apply to the whole exchange rather than a single turn. It is distinct from user turns and from sampling parameters.

  36. Question 36Prompt Engineering and System Prompt Design

    When including few-shot examples in a prompt, what is the most important property of the examples?

    1. They should be written in a different language than the target output
    2. They should all be easy cases so the model succeeds
    3. They should be as long as possible to maximize context use
    4. They should be representative of the inputs the model will actually see Correct answer.

    Explanation

    Examples teach by pattern; their value depends on matching the distribution, edge cases, and format of real inputs. Length or ease is not the criterion.

  37. Question 37Prompt Engineering and System Prompt Design

    What does 'chain-of-thought' prompting ask the model to do?

    1. Call a tool for every step of a task
    2. Retrieve documents from an external index before answering
    3. Shorten its answer to a single sentence
    4. Produce intermediate reasoning steps before the final answer Correct answer.

    Explanation

    Chain-of-thought prompting elicits step-by-step reasoning so the final answer is grounded in explicit intermediate work. It is a reasoning technique, not a retrieval or tooling mechanism.

  38. Question 38Prompt Engineering and System Prompt Design

    Which practice best protects a system prompt from being overridden by text inside user-supplied documents?

    1. Instruct the model to treat document content as data, not instructions, and delimit it clearly Correct answer.
    2. Remove all instructions from the system prompt
    3. Increase temperature so the model ignores conflicting text
    4. Place the document text before the system prompt

    Explanation

    Prompt-injection resistance comes from establishing that retrieved or user content is untrusted data and isolating it with clear delimiters, so embedded commands are not treated as instructions. Ordering and randomness do not create that boundary.

  39. Question 39Prompt Engineering and System Prompt Design

    A system prompt says: 'Always answer in under 50 words. Provide a full legal analysis with citations.' What is the main defect?

    1. The prompt lacks a role definition
    2. The prompt uses too few tokens
    3. The prompt does not specify an output language
    4. The instructions conflict with each other Correct answer.

    Explanation

    When two constraints cannot both be satisfied, the model must guess which to honor, producing inconsistent behavior. Conflicting directives should be resolved or prioritized explicitly.

  40. Question 40Prompt Engineering and System Prompt Design

    You need Claude to extract fields from invoices and return them as JSON. Which prompt design is most robust?

    1. Provide the JSON schema, field definitions, and one filled example, and instruct it to output only JSON Correct answer.
    2. Describe the desired fields in prose and ask for a 'clean' response
    3. Ask the model to explain its extraction process before outputting anything
    4. Request the output as a markdown table and parse it later

    Explanation

    An explicit schema plus definitions and a worked example constrains key names, types, and edge-case handling, and instructing JSON-only output avoids prose wrapping. Prose descriptions and tables require downstream normalization.

  41. Question 41Prompt Engineering and System Prompt Design

    A prompt includes five examples, all of which classify sentiment as 'positive'. At inference the model labels nearly everything 'positive'. What is the most likely cause?

    1. The examples are imbalanced and teach a prior toward one class Correct answer.
    2. The temperature is set too low
    3. The model cannot perform classification
    4. The system prompt is too short

    Explanation

    Few-shot examples act as training signal within the context; a skewed set of demonstrations biases predictions toward the overrepresented label. Balanced examples across classes correct this.

  42. Question 42Prompt Engineering and System Prompt Design

    Which technique best improves reliability when Claude must follow a multi-step procedure with conditional branches?

    1. Remove the procedure and rely on general knowledge
    2. Ask it to summarize the procedure before starting
    3. Have it restate the applicable branch and steps before executing them Correct answer.
    4. Ask for the final answer only, with no intermediate output

    Explanation

    Having the model explicitly identify the relevant branch and enumerate steps makes the reasoning auditable and reduces skipped conditions. Summarizing or suppressing intermediate output hides the branch selection.

  43. Question 43Prompt Engineering and System Prompt Design

    A team wants deterministic, repeatable classification output from Claude. Which combination is most appropriate?

    1. Top-p of 1.0 with free-form answers
    2. Temperature 0 with a tightly constrained output schema and explicit labels Correct answer.
    3. Multiple parallel calls with varied prompts
    4. High temperature with a long system prompt

    Explanation

    Low temperature reduces sampling variability, and a constrained schema with an enumerated label set removes ambiguity in the output space. Together they maximize reproducibility.

  44. Question 44Prompt Engineering and System Prompt Design

    What is the recommended way to handle a task where the model must answer only from a provided knowledge base and otherwise decline?

    1. Instruct it to answer only questions it finds interesting
    2. Instruct it to answer from the provided documents and to say it does not know when the answer is absent Correct answer.
    3. Instruct it to use its general knowledge to fill gaps
    4. Instruct it to guess the most likely answer

    Explanation

    Grounding instructions plus an explicit abstention path reduce hallucination by making 'not found' an acceptable, defined outcome. Filling gaps from parametric knowledge undermines the grounding requirement.

  45. Question 45Prompt Engineering and System Prompt Design

    A prompt instructs: 'Summarize the article. Do not use more than three bullets. Each bullet must contain a citation.' Which prompt-engineering principle does this best illustrate?

    1. Eliciting chain-of-thought
    2. Reducing token cost
    3. Using role prompting
    4. Providing explicit, checkable constraints Correct answer.

    Explanation

    The instructions specify countable, verifiable properties of the output, which makes compliance easy to check and failures easy to detect. Role prompting and reasoning elicitation are different techniques.

  46. Question 46Prompt Engineering and System Prompt Design

    Where in a long prompt is the most effective place to put the critical formatting requirement?

    1. Only in the first sentence, never repeated
    2. Implicitly, by demonstrating it once in an example
    3. Near the end, restated clearly after the context Correct answer.
    4. Buried in the middle of a long context block

    Explanation

    Instructions placed near the end of a long prompt, close to where generation begins, tend to be followed more reliably, and restating a key requirement reinforces it. Burying it mid-context or relying on a single implicit cue is weaker.

  47. Question 47Prompt Engineering and System Prompt Design

    A system prompt contains a large block of retrieved reference text. What is the best way to keep the model's behavioral instructions effective?

    1. Delete the behavioral instructions to save tokens
    2. Convert the instructions into the reference text's formatting
    3. Interleave instructions randomly within the reference text
    4. Keep behavioral instructions in a distinct tagged section separate from the reference data Correct answer.

    Explanation

    Separating durable instructions from variable data with clear tags preserves the model's ability to distinguish rules from content. Mixing them dilutes the instructions and invites confusion with injected text.

  48. Question 48Prompt Engineering and System Prompt Design

    Which is the best reason to include an explicit 'if the request is ambiguous, ask a clarifying question' instruction?

    1. It prevents the model from silently guessing at underspecified requirements Correct answer.
    2. It allows the model to skip the system prompt
    3. It reduces the number of tokens the model generates
    4. It guarantees the model will never make an error

    Explanation

    Underspecified requests invite the model to fill gaps with assumptions; authorizing clarification turns a hidden guess into a visible, correctable step. It reduces risk but cannot guarantee error-free output.

  49. Question 49Prompt Engineering and System Prompt Design

    A prompt asks Claude to 'evaluate the candidate fairly.' Outputs vary widely in what criteria they apply. What is the most effective fix?

    1. Increase the temperature to capture more perspectives
    2. Ask the model to be 'more objective'
    3. Add a numbered rubric with explicit criteria and scoring guidance Correct answer.
    4. Shorten the prompt to reduce noise

    Explanation

    Undefined evaluative terms leave the criteria to the model's discretion; a concrete rubric fixes the dimensions and scale, making outputs comparable. Appeals to objectivity or randomness do not specify criteria.

  50. Question 50Prompt Engineering and System Prompt Design

    A system prompt says: 'You are a helpful assistant. Be concise. Be thorough. Prioritize speed. Prioritize accuracy above all.' What should the author do first?

    1. Add more adjectives to reinforce each value
    2. Resolve the conflicting priorities into an explicit ordering Correct answer.
    3. Remove the role definition
    4. Increase max_tokens

    Explanation

    When directives pull in opposite directions, the model must choose arbitrarily; an explicit priority ordering tells it which constraint wins. Adding more adjectives or adjusting token limits does not resolve the conflict.

  51. Question 51Prompt Engineering and System Prompt Design

    A team's extraction prompt works on clean text but fails on scanned OCR output containing stray characters. Which change is most likely to improve robustness?

    1. Add few-shot examples that include noisy OCR text with correct extractions Correct answer.
    2. Shorten the system prompt
    3. Remove the output schema to give the model freedom
    4. Raise the temperature so the model adapts

    Explanation

    Demonstrations that match the failure distribution teach the model to tolerate noise and still map to the schema. Loosening the schema or adding randomness removes the constraints that make output usable.

  52. Question 52Prompt Engineering and System Prompt Design

    An agent prompt instructs: 'Use the search tool whenever helpful.' In production the model calls the tool on nearly every turn, including trivial ones. Which revision is most principled?

    1. Increase the temperature to vary tool usage
    2. Remove the tool entirely
    3. Add 'be efficient' to the prompt
    4. Specify concrete conditions for when to call the tool and when to answer directly Correct answer.

    Explanation

    Vague discretionary language yields over- or under-use; explicit trigger conditions define the decision boundary. Removing the tool or adding unquantified efficiency appeals does not establish when calls are warranted.

  53. Question 53Prompt Engineering and System Prompt Design

    A prompt includes a 40-page contract followed by 'Answer the question.' Responses cite irrelevant clauses. Which restructuring is most likely to improve accuracy?

    1. Increase max_tokens substantially
    2. Move the question to the top and repeat it immediately after the contract, and instruct the model to quote supporting clauses Correct answer.
    3. Delete the contract and rely on the model's knowledge
    4. Ask the model to summarize the contract before answering and use only the summary

    Explanation

    Placing the task near the generation point and requiring quoted support anchors attention on relevant passages and makes grounding checkable. Summarizing first can discard the specific clause needed.

  54. Question 54Prompt Engineering and System Prompt Design

    You are designing a system prompt for a high-stakes domain. Which design best supports auditing model behavior after deployment?

    1. Randomize the output format per request
    2. Rely on the model's general knowledge without citing sources
    3. Instruct the model to be as brief as possible with no explanation
    4. Require the model to output a structured rationale with cited evidence alongside each decision Correct answer.

    Explanation

    Structured rationales with citations create an inspectable record linking each decision to evidence, which is what post-hoc auditing requires. Brevity without explanation or randomized formats removes that trace.

  55. Question 55Prompt Engineering and System Prompt Design

    A prompt instructs the model to 'think step by step, then output only the final JSON.' On complex cases the JSON is sometimes malformed. Which design change best addresses this?

    1. Ask for reasoning and JSON in the same unstructured paragraph
    2. Increase temperature to explore more formats
    3. Separate a reasoning section from a strictly formatted final answer block, with an example of the exact final block Correct answer.
    4. Remove the JSON requirement and accept free text

    Explanation

    Giving reasoning its own section and isolating the final answer in a clearly delimited, exemplified block keeps the parseable output uncontaminated by prose. Mixing reasoning into the answer or dropping the schema removes that separation.

  56. Question 56Prompt Engineering and System Prompt Design

    Which element of a system prompt most directly defines the model's persona and the boundaries of its role?

    1. The list of few-shot examples placed at the end
    2. The role definition and scope statement at the top Correct answer.
    3. The maximum token limit configured for the response
    4. The temperature setting passed with the request

    Explanation

    A system prompt establishes behavior through an explicit role and scope statement, which frames how the model interprets everything that follows. Temperature, token limits, and examples influence output but do not define the persona or role boundaries.

  57. Question 57Prompt Engineering and System Prompt Design

    A support assistant must always reply in the customer's language. Where should this rule be placed to apply across every turn of a conversation?

    1. In each user message as a reminder
    2. In a tool description
    3. In the final assistant message
    4. In the system prompt Correct answer.

    Explanation

    Persistent behavioral rules belong in the system prompt because it is applied to every turn and outranks per-message instructions. Repeating the rule in user messages is fragile and easily omitted.

  58. Question 58Prompt Engineering and System Prompt Design

    Which prompt structure best separates durable behavioral rules from task-specific input?

    1. A system prompt for rules plus a user message for the task Correct answer.
    2. Two user messages, one for rules and one for data
    3. A single user message mixing rules and data
    4. A system prompt containing both rules and the full task data

    Explanation

    Separating stable rules in the system prompt from per-request task data in the user message keeps behavior consistent and makes the input easier to template and audit. Mixing them invites the data to be read as instructions.

  59. Question 59Prompt Engineering and System Prompt Design

    Which output specification gives the most consistent machine-parseable results from Claude?

    1. A prose description of the desired fields
    2. A request to 'respond in a structured way'
    3. An explicit schema with field names, types, and an example Correct answer.
    4. An instruction to 'be precise and complete'

    Explanation

    An explicit schema with named fields, types, and a concrete example removes ambiguity about shape and lets the model target an exact structure. Vague quality adjectives do not constrain the output format.

  60. Question 60Prompt Engineering and System Prompt Design

    A prompt must instruct Claude to handle three input categories with different processing rules. Which design reduces ambiguity most effectively?

    1. A single paragraph describing all three categories together
    2. A closing reminder to follow the rules carefully
    3. A numbered list with one labeled block per category Correct answer.
    4. A request to 'use judgment for each category'

    Explanation

    Labeled, numbered blocks give each category its own scoped instruction, so rules do not bleed across categories. Combined prose or vague judgment requests leave the model to infer boundaries.

  61. Question 61Prompt Engineering and System Prompt Design

    A team wants Claude to adopt a formal tone for legal summaries but a casual tone for marketing copy within the same application. What is the most maintainable design?

    1. A single prompt that says 'use an appropriate tone'
    2. Two separate system prompts, one per task type Correct answer.
    3. One system prompt that switches tone based on the detected content
    4. A user message that asks the model to pick a tone

    Explanation

    Separate system prompts per task type isolate tone rules and can be versioned and tested independently. A single prompt that must detect and switch tone couples unrelated behaviors and is harder to verify.

  62. Question 62Prompt Engineering and System Prompt Design

    An extraction prompt yields correct values but occasionally returns them under slightly different key names. Which change most directly fixes this?

    1. Adding a request to 'be consistent with keys'
    2. Fixing the exact key names in the schema and echoing them in an example Correct answer.
    3. Lowering temperature to zero without changing the schema
    4. Increasing the response length limit

    Explanation

    Naming the exact keys in the schema and showing them in a worked example anchors the model to a fixed vocabulary. Lowering temperature reduces randomness but does not tell the model which names to use.

  63. Question 63Prompt Engineering and System Prompt Design

    Which instruction ordering best supports a prompt that must first classify a message and then apply a category-specific response policy?

    1. State the response policies first, then the classification step
    2. Interleave classification and policy text in one paragraph
    3. State only the policies and let the model infer the categories
    4. State the classification step first, then the policies keyed to each category Correct answer.

    Explanation

    Ordering instructions to match the required execution sequence lets the model classify before it selects a policy. Policies stated first can be applied before the category is known, producing mismatches.

  64. Question 64Prompt Engineering and System Prompt Design

    A prompt asks Claude to write release notes that must mention every closed ticket. Which addition most improves completeness without changing the format?

    1. An instruction to list each ticket ID before drafting prose Correct answer.
    2. A higher temperature for creative phrasing
    3. A request to 'be thorough'
    4. A shorter maximum output length

    Explanation

    Requiring the model to enumerate ticket IDs first forces it to account for each item before writing, which improves coverage. Generic thoroughness requests and sampling changes do not enforce enumeration.

  65. Question 65Prompt Engineering and System Prompt Design

    A system prompt must tell Claude to refuse requests outside a defined scope. Which phrasing is most testable?

    1. 'Use good judgment about relevance.'
    2. 'Avoid answering things that seem off-topic.'
    3. 'Decline requests that fall outside the listed topics and reply with the standard refusal message.' Correct answer.
    4. 'Try to stay on subject when possible.'

    Explanation

    A testable rule names the condition for refusal and specifies the exact response, so behavior can be verified against cases. Hedged or judgment-based phrasing produces inconsistent outcomes that cannot be asserted in tests.

  66. Question 66Prompt Engineering and System Prompt Design

    A prompt includes a long list of banned phrases. What design change best keeps the model from using them?

    1. Stating the ban once in a single dense paragraph
    2. Adding more banned phrases to the same paragraph
    3. Repeating the list at the start and end of the prompt Correct answer.
    4. Replacing the list with a general request for 'clean writing'

    Explanation

    Placing critical constraints both before and after long content improves adherence because instructions near the end are not diluted by intervening text. A single dense paragraph or a vague request weakens the constraint.

  67. Question 67Prompt Engineering and System Prompt Design

    A prompt tells Claude to 'return a JSON object' but the response sometimes includes a conversational preamble. Which combination most reliably yields pure JSON?

    1. Asking politely for JSON only
    2. Specifying 'output only the JSON object, with no other text' and providing a JSON example Correct answer.
    3. Requesting JSON and increasing the temperature
    4. Adding 'JSON' in capital letters to the request

    Explanation

    Explicitly prohibiting any text outside the object, combined with a worked JSON example, constrains both the format and the surrounding prose. Politeness or capitalization alone does not remove the preamble.

  68. Question 68Prompt Engineering and System Prompt Design

    A prompt instructs Claude to 'review the code and mention any problems.' Outputs vary in depth and often miss security issues. Which redesign most improves consistency?

    1. Asking for a 'thorough review'
    2. Requesting the review in a friendlier tone
    3. Defining a fixed checklist of review dimensions, including security, to address in order Correct answer.
    4. Removing all structure so the model can explore freely

    Explanation

    A fixed checklist enumerates the dimensions to cover and their order, making coverage consistent and comparable across runs. Open-ended requests let the model choose which concerns to raise.

  69. Question 69Prompt Engineering and System Prompt Design

    Two prompt versions perform equally on your test set, but one relies on implicit cues and the other states every rule explicitly. Which is the better production choice and why?

    1. The explicit version, because stated rules are easier to audit and regression-test Correct answer.
    2. The implicit version, because models infer intent better without rules
    3. The implicit version, because shorter prompts cost less
    4. Either version, because equal test scores imply equal robustness

    Explanation

    Explicit rules create an auditable contract: each behavior can be traced to an instruction and covered by a regression test. Equal scores on one test set do not show which prompt will hold up as inputs drift.

  70. Question 70Prompt Engineering and System Prompt Design

    A system prompt has grown to include persona, tone rules, safety rules, formatting rules, and tool guidance. What refactoring most improves clarity without losing behavior?

    1. Grouping rules under labeled sections with the highest-priority rules first Correct answer.
    2. Moving safety rules to the end of the user message
    3. Removing section labels to shorten the prompt
    4. Merging all sections into one flowing paragraph

    Explanation

    Labeled sections let readers and the model locate related rules, and priority ordering signals which rules win when they compete. Merging or stripping labels hides the structure, and moving safety rules to user content weakens their authority.

  71. Question 71Tool Use and Agentic Workflows

    In a tool-use loop, which component decides whether the model should call a tool or return a final answer to the user?

    1. The client, based on a fixed call-count limit
    2. The model itself, based on the conversation and tool definitions Correct answer.
    3. The transport layer, based on the HTTP status code
    4. The tool executor, based on the tool's output schema

    Explanation

    The model drives control flow in an agentic loop: it reads the conversation and available tool definitions and emits either a tool-use request or a final response. The executor only runs what the model requests and returns results.

  72. Question 72Tool Use and Agentic Workflows

    What is the primary purpose of a tool's input schema in a tool definition?

    1. To cache previous tool results across sessions
    2. To constrain and validate the arguments the model may supply Correct answer.
    3. To encrypt the tool's response before returning it
    4. To document the tool for human maintainers only

    Explanation

    The input schema declares the expected parameter names, types, and required fields, which guides the model's generation and lets the runtime validate arguments before execution. It is machine-consumed, not merely documentation.

  73. Question 73Tool Use and Agentic Workflows

    An agent must call a tool that returns a large list of records, but only a few fields matter for the next step. Which practice best keeps the loop efficient and accurate?

    1. Truncate the tool output to the first few characters
    2. Store the full output and pass a record count instead
    3. Have the tool return only the relevant fields and rows Correct answer.
    4. Return all records and let the model summarize them later

    Explanation

    Shaping tool output to the fields and rows the next step needs reduces context bloat and keeps the model focused on relevant data. Passing counts or truncated strings strips away information the model may require.

  74. Question 74Tool Use and Agentic Workflows

    Why should tool results be returned to the model as a structured tool-result message rather than as plain user text?

    1. It reduces the token cost of the result
    2. It automatically retries failed tool executions
    3. It preserves the association between the result and the specific tool call Correct answer.
    4. It prevents the model from calling the same tool twice

    Explanation

    A structured tool-result message carries an identifier linking it to the originating tool call, so the model can correctly attribute and use the output. Plain text loses that linkage and can be misread as user input.

  75. Question 75Tool Use and Agentic Workflows

    What does the stop reason or finish reason of a model response typically indicate in a tool-use workflow?

    1. The billing tier applied to the request
    2. Whether the model wants to invoke a tool or has finished its answer Correct answer.
    3. The number of tokens remaining in the context window
    4. The latency of the previous tool execution

    Explanation

    The stop reason signals why generation ended, for example a tool-use request versus a natural completion. The orchestrator uses it to decide whether to execute tools and continue the loop.

  76. Question 76Tool Use and Agentic Workflows

    Which design choice most improves a model's ability to select the right tool among several similar ones?

    1. Assign identical descriptions to all tools to keep them consistent
    2. Omit descriptions and rely on parameter names alone
    3. Merge all similar tools into one tool with no parameters
    4. Give each tool a distinct, specific name and description of when to use it Correct answer.

    Explanation

    Distinct, specific names and descriptions that state when each tool applies help the model disambiguate similar options. Identical or missing descriptions remove the signal the model needs to choose correctly.

  77. Question 77Tool Use and Agentic Workflows

    What is the purpose of setting a maximum iteration or step limit in an agentic loop?

    1. To eliminate the need for tool input validation
    2. To raise the model's reasoning quality per step
    3. To guarantee the model produces a correct answer
    4. To bound runaway loops and control cost and latency Correct answer.

    Explanation

    A step limit is a safety bound that stops loops that fail to converge, protecting cost and latency. It does not guarantee correctness or replace validation.

  78. Question 78Tool Use and Agentic Workflows

    Which situation best justifies giving an agent a tool rather than relying on the model's parametric knowledge alone?

    1. The task requires rephrasing a provided paragraph
    2. The question asks for a creative slogan
    3. The answer depends on live inventory data that changes hourly Correct answer.
    4. The user wants a general definition of a common term

    Explanation

    Tools are warranted when the answer depends on fresh or external state the model cannot know, such as live inventory. Static or generative tasks can often be handled without a tool.

  79. Question 79Tool Use and Agentic Workflows

    A tool call fails because the model supplied a string where the schema requires an integer. What is the most appropriate first response by the orchestration layer?

    1. Silently coerce the value and continue without informing the model
    2. Return a structured validation error to the model so it can correct the arguments Correct answer.
    3. Disable the tool for the remainder of the conversation
    4. Terminate the session and require the user to restart

    Explanation

    Returning a structured validation error lets the model see what was wrong and retry with corrected arguments, which is the core self-correction pattern. Silent coercion can mask bugs, and terminating or disabling is overly drastic for a recoverable error.

  80. Question 80Tool Use and Agentic Workflows

    An agent repeatedly calls the same search tool with identical arguments and gets identical results. Which change best addresses this loop?

    1. Remove the tool from the toolset permanently
    2. Increase the maximum iteration limit
    3. Add loop detection that flags repeated identical calls and injects guidance Correct answer.
    4. Increase the model temperature on every subsequent call

    Explanation

    Detecting repeated identical calls and injecting guidance, such as a note that the result is unchanged, breaks the cycle while preserving the tool. Raising limits or randomness does not address the root cause.

  81. Question 81Tool Use and Agentic Workflows

    Which is the best reason to keep tool definitions minimal and narrowly scoped?

    1. Minimal tools remove the need for error handling
    2. Broad tools cannot be validated by a runtime
    3. Narrow tools reduce ambiguity and the chance of misapplied arguments Correct answer.
    4. Fewer tools always reduce the model's total token usage to zero

    Explanation

    Narrowly scoped tools with clear parameters reduce ambiguity in selection and argument construction. They do not eliminate token usage or the need for error handling.

  82. Question 82Tool Use and Agentic Workflows

    An agent must complete a multi-step task: fetch a customer record, then fetch that customer's orders, then summarize. How should the tool calls be sequenced?

    1. Fetch the record first, use its identifier to fetch orders, then summarize Correct answer.
    2. Issue all three operations in parallel in a single turn
    3. Fetch orders first and infer the customer from the orders
    4. Summarize first and fetch data only if the summary is challenged

    Explanation

    Dependent steps must run in order because later calls require identifiers produced by earlier ones. Parallelizing dependent calls or reversing the order breaks the data dependency.

  83. Question 83Tool Use and Agentic Workflows

    Which scenario is a legitimate use of parallel tool calls?

    1. Writing a file and then reading the same file to verify contents
    2. Charging a payment and then issuing a refund for it
    3. Fetching weather for three independent cities in one turn Correct answer.
    4. Looking up a user ID and then using it to fetch that user's profile

    Explanation

    Parallel calls are appropriate when operations are independent and their results do not depend on each other, such as three separate city lookups. The other scenarios have ordering dependencies.

  84. Question 84Tool Use and Agentic Workflows

    What is the main risk of passing an entire raw tool response, including metadata and pagination tokens, directly into the model context?

    1. It consumes context with irrelevant data and can distract from key fields Correct answer.
    2. It guarantees the model will hallucinate field values
    3. It prevents the tool from being called again
    4. It changes the tool's input schema

    Explanation

    Irrelevant metadata and pagination details consume limited context and can dilute attention away from the fields that matter. It does not guarantee hallucination or alter schemas.

  85. Question 85Tool Use and Agentic Workflows

    A tool returns a success status but its payload indicates a business-level failure, such as 'account not found'. How should the orchestration layer treat this?

    1. As a reason to retry the identical call indefinitely
    2. As a failure condition the model should see and reason about Correct answer.
    3. As grounds to discard the entire conversation history
    4. As a transport success that requires no further action

    Explanation

    Business-level failures must be surfaced to the model so it can adapt, for example by asking the user or trying a different path. Treating them as success ignores the actual outcome.

  86. Question 86Tool Use and Agentic Workflows

    Which approach best handles a tool that intermittently times out due to transient network issues?

    1. Treat the first timeout as a permanent tool removal
    2. Retry forever until the tool eventually responds
    3. Retry with bounded attempts and backoff, then surface failure to the model Correct answer.
    4. Return an empty successful result to keep the loop moving

    Explanation

    Transient failures call for bounded retries with backoff, after which the failure is reported so the model can choose an alternative. Infinite retries risk hangs, and empty success results hide the problem.

  87. Question 87Tool Use and Agentic Workflows

    What distinguishes a tool-use request from a final answer in the model's output stream?

    1. The presence of a structured tool-call block with a name and arguments Correct answer.
    2. The HTTP status code of the API response
    3. The length of the generated text
    4. The number of prior conversation turns

    Explanation

    A tool-use request is expressed as a structured block naming the tool and supplying arguments, which the orchestrator parses and dispatches. Text length, status codes, and turn counts do not distinguish the two.

  88. Question 88Tool Use and Agentic Workflows

    An agent has access to a destructive 'delete_record' tool. Which control best limits unintended data loss?

    1. Remove all logging so deletions are not recorded
    2. Require explicit confirmation or a dry-run step before destructive execution Correct answer.
    3. Rely on the model's judgment to avoid deleting records
    4. Increase the model temperature for more cautious behavior

    Explanation

    Destructive operations should be gated by confirmation or a dry-run so a mistaken call cannot cause irreversible harm. Model judgment alone is insufficient, and removing logging reduces accountability.

  89. Question 89Tool Use and Agentic Workflows

    Which design best supports a human-in-the-loop approval step within an agentic workflow?

    1. Batch all sensitive calls and execute them without review
    2. Pause before the sensitive tool call and resume after explicit approval Correct answer.
    3. Run the sensitive call twice to compare results
    4. Log the call after execution and notify the user later

    Explanation

    Human-in-the-loop control requires pausing before the sensitive action and resuming only after approval, so the human decision actually gates execution. Post-hoc logging does not prevent harm.

  90. Question 90Tool Use and Agentic Workflows

    A model keeps choosing a general-purpose 'run_command' tool when a safer, specific tool exists for the task. Which change most directly improves selection?

    1. Remove the specific tool to reduce confusion
    2. Lower the maximum iteration limit
    3. Increase the general tool's parameter count
    4. Strengthen the specific tool's description and note when to prefer it over general tools Correct answer.

    Explanation

    Improving the specific tool's description and stating when it should be preferred gives the model the signal needed to choose it over a general alternative. Removing the safer tool or altering limits does not guide selection.

  91. Question 91Tool Use and Agentic Workflows

    Which characteristic most clearly indicates that a task should be decomposed across multiple specialized tools rather than one monolithic tool?

    1. The task involves several distinct capabilities with different input requirements Correct answer.
    2. The task output is a short string
    3. The task can be completed in a single sentence
    4. The task requires no external data

    Explanation

    Distinct capabilities with differing inputs map naturally to specialized tools, improving clarity and validation. Short, self-contained, or data-free tasks do not by themselves justify decomposition.

  92. Question 92Tool Use and Agentic Workflows

    An agent's tool results are correct, but its final answer omits a key value that the tool returned. Which is the most likely contributing cause?

    1. The tool returned too few tokens to be read
    2. The input schema had an optional field
    3. The relevant tool result was truncated or not carried forward into later context Correct answer.
    4. The stop reason was set to tool_use

    Explanation

    When correct data exists in a tool result but is missing from the final answer, the usual cause is loss of that result through truncation or failure to include it in subsequent context. Stop reasons and optional fields do not remove data from context.

  93. Question 93Tool Use and Agentic Workflows

    A workflow must be resilient when a tool returns partial results alongside an error indicator. Which handling strategy is most robust?

    1. Treat the call as fully successful and ignore the error indicator
    2. Retry until a call returns with no error indicator at all
    3. Preserve the partial results, surface the error, and let the model decide how to proceed Correct answer.
    4. Discard partial results and treat the call as a total failure

    Explanation

    Partial success requires preserving usable data while surfacing the error so the model can reason about gaps and next steps. Discarding everything or ignoring the error both lose information or hide risk.

  94. Question 94Tool Use and Agentic Workflows

    Why is it important that tool error messages returned to the model be specific, for example naming the offending field, rather than generic?

    1. Generic errors make the model more creative in recovery
    2. Generic errors reduce token usage enough to matter
    3. Specific errors eliminate the need for input schemas
    4. Specific errors let the model correct the exact problem instead of guessing Correct answer.

    Explanation

    Specific error messages give the model actionable information to fix the precise defect, enabling reliable self-correction. Generic messages force guessing, and schemas remain necessary regardless.

  95. Question 95Tool Use and Agentic Workflows

    An agent is given overlapping tools whose descriptions contradict each other about which handles refunds. What is the most principled fix?

    1. Increase the number of retries for refund calls
    2. Reconcile the descriptions so each tool's responsibility is unambiguous Correct answer.
    3. Rely on the model to resolve contradictions at runtime
    4. Add more tools so the model has additional choices

    Explanation

    Contradictory descriptions create ambiguity that no amount of retrying or added choice resolves; the descriptions must be reconciled so responsibilities are clear. Relying on runtime resolution is unreliable.

  96. Question 96Tool Use and Agentic Workflows

    A tool's input schema declares a field as an enumeration of three allowed values. What is the primary benefit of this constraint for an agentic workflow?

    1. It narrows the space of valid inputs so the model is far less likely to emit an unusable value. Correct answer.
    2. It guarantees the tool's backend will never return an error.
    3. It removes the need to describe the field in the tool description.
    4. It allows the model to skip validating its own arguments.

    Explanation

    Enumerations constrain the value space to a known set, which sharply reduces malformed arguments and makes behavior predictable. Schema constraints shape generation but do not guarantee backend success or replace descriptive text.

  97. Question 97Tool Use and Agentic Workflows

    An agent is given a tool whose description is a single vague sentence. What is the most likely consequence during tool selection?

    1. The model will call the tool more often because fewer constraints apply.
    2. The tool will be automatically excluded from the tool list.
    3. The model may misjudge when the tool applies and select it for the wrong tasks. Correct answer.
    4. The model will always ask the user for clarification before calling it.

    Explanation

    Tool descriptions are the model's only guide for matching intent to capability, so vague text increases misrouting. The model does not automatically drop or always clarify ambiguous tools.

  98. Question 98Tool Use and Agentic Workflows

    An agentic workflow must summarize a long document and then answer questions about the summary. Which structure best reflects a sound agent design?

    1. Have the model answer from memory of the document without producing a summary artifact.
    2. Ask the user to re-supply the document each time a question is asked.
    3. Store the summary as an explicit intermediate artifact that later steps reference. Correct answer.
    4. Pass the entire document to the model on every subsequent turn to avoid losing detail.

    Explanation

    Persisting intermediate results as explicit artifacts gives later steps a stable, inspectable input and avoids redundant work. Re-sending the full document or relying on unstated memory is wasteful or unreliable.

  99. Question 99Tool Use and Agentic Workflows

    What is the primary role of a tool result message in a tool-use conversation?

    1. It records the model's reasoning for auditing purposes.
    2. It returns the tool's output to the model so it can continue reasoning. Correct answer.
    3. It replaces the system prompt for the remainder of the session.
    4. It signals that the agent has finished its task.

    Explanation

    Tool results feed execution output back into the conversation so the model can incorporate it into subsequent reasoning or answers. They do not terminate the task or substitute for other message roles.

  100. Question 100Tool Use and Agentic Workflows

    An agent must look up three independent product prices before comparing them. Which execution pattern is most appropriate?

    1. Skip the lookups and estimate prices from the model's prior knowledge.
    2. Chain the lookups so each depends on the previous result.
    3. Issue the three lookups as parallel tool calls in a single turn. Correct answer.
    4. Perform one lookup per user turn and wait for confirmation between each.

    Explanation

    Independent lookups with no data dependency can be issued together, reducing latency and turns. Chaining or serializing them adds unnecessary dependency and delay.

  101. Question 101Tool Use and Agentic Workflows

    A tool's response includes a nested object with many fields, but downstream reasoning needs only two of them. What is the best practice?

    1. Have the tool return nothing and let the model infer the values.
    2. Flatten every field into the prompt so nothing is lost.
    3. Extract and pass only the two needed fields to the next step. Correct answer.
    4. Pass the full nested object through unchanged to preserve all context.

    Explanation

    Projecting tool output down to the fields the next step actually needs keeps context focused and reduces noise. Passing everything or flattening all fields bloats context without benefit.

  102. Question 102Tool Use and Agentic Workflows

    A tool call is rejected because a required field was omitted from the arguments. What should the agent do next?

    1. Switch to an unrelated tool that has fewer required fields.
    2. Abandon the task and report failure to the user immediately.
    3. Re-issue the call with the missing field supplied and continue. Correct answer.
    4. Retry the identical call unchanged several times.

    Explanation

    A missing-required-field error is a recoverable validation failure, so the agent should correct the arguments and retry. Blind retries or task abandonment are unwarranted for a fixable input error.

  103. Question 103Tool Use and Agentic Workflows

    An agent has a read-only 'get_invoice' tool and a separate 'void_invoice' tool. Which design principle does this separation best illustrate?

    1. Separating safe reads from consequential writes so each can be governed differently. Correct answer.
    2. Giving every tool the broadest possible permissions for flexibility.
    3. Combining read and write operations into one tool to reduce tool count.
    4. Requiring the model to choose tools based on alphabetical order.

    Explanation

    Distinct read and write tools let you apply different safeguards, such as auto-approving reads while gating destructive writes. Merging them or broadening permissions undermines that control.

  104. Question 104Tool Use and Agentic Workflows

    An agent's plan calls for a step that depends on the output of a prior step that has not yet run. What is the correct handling?

    1. Execute both steps in parallel to reduce latency.
    2. Execute the dependent step first to save time.
    3. Skip the dependent step and proceed with the rest of the plan.
    4. Defer the dependent step until the required output is available. Correct answer.

    Explanation

    A true data dependency means the later step cannot run until the earlier output exists, so execution must be ordered accordingly. Parallel or reversed execution would use missing inputs.

  105. Question 105Tool Use and Agentic Workflows

    A tool returns a large paginated list, and the agent needs only the first page to answer the user. What is the most efficient behavior?

    1. Ask the user to paginate manually on the agent's behalf.
    2. Fetch all pages before answering to ensure completeness.
    3. Use the first page and stop paging once the needed information is present. Correct answer.
    4. Ignore pagination tokens and request page one repeatedly.

    Explanation

    Fetching only what the task requires avoids unnecessary calls and context growth. Exhausting all pages or looping on the same page wastes resources.

  106. Question 106Tool Use and Agentic Workflows

    Which practice best prevents an agent from acting on a stale tool result that no longer reflects current state?

    1. Assuming tool results are always current once returned.
    2. Re-validating or re-fetching state at the point where a consequential action depends on it. Correct answer.
    3. Increasing the model's temperature so it reconsiders the data.
    4. Caching the first result and reusing it for all later steps.

    Explanation

    State can change between when a result was fetched and when it is used, so re-checking at the point of a consequential action guards against stale data. Caching indefinitely or assuming freshness increases risk.

  107. Question 107Tool Use and Agentic Workflows

    An agent must produce a final report that cites values gathered from several tools. Which design most reliably keeps the report consistent with the tool outputs?

    1. Assemble the report from the stored tool outputs rather than from recollection. Correct answer.
    2. Have the model regenerate all values from memory at the end.
    3. Instruct the model to round all values to avoid mismatches.
    4. Rely on the model to recall values without referencing tool outputs.

    Explanation

    Grounding the report in the actual stored tool outputs prevents drift between what tools returned and what the report claims. Regenerating from memory invites inconsistency.

  108. Question 108Tool Use and Agentic Workflows

    An agent is asked to complete a task that requires both a database write and a notification email, and the email must not be sent if the write fails. Which coordination approach is correct?

    1. Send the email and perform the write in parallel.
    2. Send the email first so the user is informed early.
    3. Skip checking the write result and send the email regardless.
    4. Perform the write, check its result, and only then send the email. Correct answer.

    Explanation

    When a later action is conditional on an earlier one succeeding, the earlier action must complete and its result inspected first. Parallel or reversed ordering can send notifications for failed writes.

  109. Question 109Tool Use and Agentic Workflows

    An agent operating in production has a tool that can modify records. Which combination of controls best limits the impact of a mistaken call?

    1. A higher model temperature and a longer system prompt.
    2. Dry-run validation, scoped permissions, and an audit log of each invocation. Correct answer.
    3. Allowing unlimited retries until the call succeeds.
    4. Removing the tool description so the model calls it less often.

    Explanation

    Layered controls such as pre-execution validation, least-privilege permissions, and audit logging bound and record the effects of mistakes. Prompt tweaks or retries do not constrain impact.

  110. Question 110Tool Use and Agentic Workflows

    A multi-step agent occasionally reaches a state where its plan and the actual tool results diverge. Which mechanism most directly detects this divergence before it compounds?

    1. Relying on the model to notice inconsistencies on its own.
    2. Reducing the number of available tools to one.
    3. Explicit checks that compare expected step outcomes against observed results. Correct answer.
    4. Increasing the maximum token budget for each response.

    Explanation

    Comparing expected versus observed outcomes at each step surfaces divergence early so the agent can replan. Relying on spontaneous model detection or unrelated tuning does not reliably catch it.

  111. Question 111Context Management and Retrieval-Augmented Generation

    What is the primary purpose of a context window in a large language model application?

    1. To encrypt retrieved documents before transmission
    2. To cache embeddings for the vector database
    3. To bound the amount of text the model can attend to in a single request Correct answer.
    4. To store the model's permanent weights between sessions

    Explanation

    The context window is the fixed token budget the model can process in one call, covering the system prompt, conversation history, retrieved passages, and the response. It is a hard limit, not persistent storage.

  112. Question 112Context Management and Retrieval-Augmented Generation

    In a RAG pipeline, what does the retriever component produce for a given user query?

    1. A rewritten system prompt
    2. A fine-tuned model checkpoint
    3. A ranked set of candidate passages Correct answer.
    4. A tokenized version of the final answer

    Explanation

    The retriever searches an index and returns candidate passages ranked by relevance to the query. Those passages are then optionally reranked and inserted into the prompt.

  113. Question 113Context Management and Retrieval-Augmented Generation

    An application must answer questions over a 400-page policy manual that far exceeds the context window. Which approach best fits?

    1. Paste the entire manual into every request
    2. Increase the temperature to fit more text
    3. Retrieve relevant chunks and include only those Correct answer.
    4. Drop the system prompt to save tokens

    Explanation

    Retrieval-augmented generation selects only the passages relevant to the query, keeping the prompt within the token budget while grounding the answer in source text.

  114. Question 114Context Management and Retrieval-Augmented Generation

    Which embedding property makes cosine similarity a reasonable relevance measure?

    1. Embeddings are sparse binary vectors
    2. Embeddings encode token counts directly
    3. Embeddings place semantically similar text near each other Correct answer.
    4. Embeddings are sorted alphabetically

    Explanation

    Embedding models map text into a vector space where semantically related inputs have high cosine similarity. This geometric proximity is what makes nearest-neighbor search meaningful.

  115. Question 115Context Management and Retrieval-Augmented Generation

    What is the main reason to chunk documents before indexing them for RAG?

    1. To reduce the size of the embedding model
    2. To eliminate the need for a vector store
    3. To guarantee that every chunk is factually correct
    4. To allow retrieval at a granularity that fits the prompt budget Correct answer.

    Explanation

    Chunking splits documents into units small enough to embed and to fit several into the prompt, while improving retrieval precision by matching on focused passages.

  116. Question 116Context Management and Retrieval-Augmented Generation

    Which technique keeps a long multi-turn conversation within the context budget while preserving recent detail?

    1. Deleting the system prompt after the first turn
    2. Lowering the max_tokens of the model
    3. Summarizing older turns and retaining recent turns verbatim Correct answer.
    4. Repeating the full history twice for emphasis

    Explanation

    Rolling summarization compresses older history into a synopsis while recent turns stay verbatim, balancing token savings against recency. This is a standard context-management pattern.

  117. Question 117Context Management and Retrieval-Augmented Generation

    What does a reranker typically do after initial retrieval?

    1. Reorders candidate passages by a more precise relevance score Correct answer.
    2. Generates the final answer text
    3. Translates passages into the user's language
    4. Compresses the vector index on disk

    Explanation

    A reranker scores each query-passage pair with a heavier model and reorders the candidates so the most relevant ones are placed first in the prompt. This improves precision over embedding similarity alone.

  118. Question 118Context Management and Retrieval-Augmented Generation

    Which statement best describes grounding in a RAG system?

    1. The prompt is sent without any external text
    2. The model's answers are conditioned on retrieved source text Correct answer.
    3. The model is retrained on every user query
    4. The vector database stores the model weights

    Explanation

    Grounding means the response is generated with reference to retrieved evidence, which reduces unsupported claims and lets answers be traced to sources.

  119. Question 119Context Management and Retrieval-Augmented Generation

    A RAG system returns relevant passages but the model's answer ignores them and relies on parametric memory. Which prompt change most directly improves grounding?

    1. Remove the retrieved passages to reduce noise
    2. Instruct the model to cite and rely on the provided passages Correct answer.
    3. Shorten the user question
    4. Increase the temperature

    Explanation

    Explicit instructions to base the answer on the supplied passages and cite them steer the model toward the retrieved evidence. This is a prompt-level grounding control.

  120. Question 120Context Management and Retrieval-Augmented Generation

    When several retrieved chunks overlap heavily, what is the most direct effect on the prompt?

    1. It wastes tokens on redundant content and crowds out other evidence Correct answer.
    2. It reduces the embedding dimensionality
    3. It disables the reranker
    4. It automatically improves answer accuracy

    Explanation

    Overlapping or duplicate chunks consume context budget without adding new information, potentially displacing more diverse and useful passages. Deduplication or overlap-aware chunking mitigates this.

  121. Question 121Context Management and Retrieval-Augmented Generation

    A team wants to answer questions that require combining facts from two different documents. Which retrieval strategy best supports this?

    1. Retrieve only the single top-scoring chunk
    2. Retrieve multiple diverse chunks and include them together Correct answer.
    3. Lower the similarity threshold to return nothing
    4. Rely solely on the model's pretrained knowledge

    Explanation

    Multi-hop questions need evidence from more than one source, so the prompt must include several relevant passages. Diversity-aware retrieval increases the chance both needed facts appear.

  122. Question 122Context Management and Retrieval-Augmented Generation

    Which is the most appropriate way to handle a user query that is too vague for good retrieval?

    1. Return an empty answer
    2. Retrieve on the raw query and hope for the best
    3. Disable retrieval for that request
    4. Rewrite or expand the query before retrieval Correct answer.

    Explanation

    Query rewriting or expansion turns a vague question into a clearer search expression, improving recall. This is a common pre-retrieval transformation step.

  123. Question 123Context Management and Retrieval-Augmented Generation

    What is the role of metadata filtering in a vector search system?

    1. It converts the query into SQL automatically
    2. It increases the model's context window size
    3. It narrows candidates by attributes such as date or source before similarity ranking Correct answer.
    4. It replaces the need for embeddings

    Explanation

    Metadata filters restrict the candidate set using structured attributes, so similarity search runs only over eligible documents. This improves precision and can enforce access or freshness constraints.

  124. Question 124Context Management and Retrieval-Augmented Generation

    An index was built with an embedding model that has since been replaced. What must happen for retrieval to remain valid?

    1. Only the query encoder needs updating
    2. The vector store must be switched to keyword search
    3. Nothing, because embeddings are model-independent
    4. The stored document embeddings must be re-embedded with the new model Correct answer.

    Explanation

    Query and document vectors must come from the same embedding space to be comparable. Changing the model requires re-embedding the corpus so similarity scores remain meaningful.

  125. Question 125Context Management and Retrieval-Augmented Generation

    Which pattern best limits hallucination when the retrieved context does not contain the answer?

    1. Raise the temperature so the model tries harder
    2. Always return the top chunk as the answer
    3. Instruct the model to say it lacks information rather than guess Correct answer.
    4. Remove all instructions about uncertainty

    Explanation

    An explicit fallback instruction lets the model decline when evidence is absent, which reduces fabricated answers. This is a standard guardrail in grounded systems.

  126. Question 126Context Management and Retrieval-Augmented Generation

    What is the main benefit of including source identifiers alongside retrieved passages in the prompt?

    1. It removes the need for a reranker
    2. It reduces the number of tokens used
    3. It lets the response cite sources for verification Correct answer.
    4. It increases the embedding similarity score

    Explanation

    Source identifiers enable citations, so users and downstream checks can verify claims against the original documents. This supports trust and auditability in RAG outputs.

  127. Question 127Context Management and Retrieval-Augmented Generation

    A retrieval pipeline returns many low-relevance chunks, flooding the prompt. Which change most directly improves precision?

    1. Remove the system prompt
    2. Increase the number of retrieved chunks
    3. Apply a relevance threshold or reranker before inclusion Correct answer.
    4. Use a larger output token limit

    Explanation

    Filtering by a relevance threshold or reranking keeps only high-quality passages, reducing noise in the prompt. Precision improves when weak matches are excluded.

  128. Question 128Context Management and Retrieval-Augmented Generation

    What is the purpose of a sliding-window overlap when chunking long documents?

    1. To eliminate the need for embeddings
    2. To reduce the total number of chunks to one
    3. To make every chunk identical in content
    4. To preserve context that would otherwise be split across chunk boundaries Correct answer.

    Explanation

    Overlap carries a small amount of text across adjacent chunks so that sentences or ideas spanning a boundary remain retrievable. It trades some redundancy for better context continuity.

  129. Question 129Context Management and Retrieval-Augmented Generation

    Which approach best handles a query that mixes a structured filter (price under 50) with a semantic need (comfortable shoes)?

    1. Ask the user to choose one criterion
    2. Keyword search only on the word comfortable
    3. Metadata filter on price combined with vector similarity search Correct answer.
    4. Semantic search only, ignoring the price

    Explanation

    Hybrid handling applies the structured constraint as a metadata filter and uses vector similarity for the semantic part. This satisfies both the hard constraint and the meaning-based need.

  130. Question 130Context Management and Retrieval-Augmented Generation

    What is the primary trade-off when increasing the number of chunks retrieved per query?

    1. Better encryption versus weaker access control
    2. Smaller index size versus larger embeddings
    3. Higher recall of relevant evidence versus more tokens and potential noise Correct answer.
    4. Faster inference versus lower accuracy

    Explanation

    Retrieving more chunks raises the chance the needed evidence is present (recall) but consumes context budget and may introduce distracting or redundant content. Tuning this balance is central to RAG quality.

  131. Question 131Context Management and Retrieval-Augmented Generation

    A system must answer follow-up questions like 'and what about the second one?' reliably. Which context-management choice most helps?

    1. Store the conversation only in the vector index
    2. Carry forward recent turns so pronouns and references resolve Correct answer.
    3. Increase the embedding dimensions
    4. Discard all prior turns on each request

    Explanation

    Follow-up questions depend on prior context to resolve references, so recent turns must remain in the prompt or be condensed into a resolved standalone query. Dropping history breaks coreference.

  132. Question 132Context Management and Retrieval-Augmented Generation

    Retrieval quality is high, but answers are truncated mid-sentence. Which context-related setting most likely explains this?

    1. The vector index is too small
    2. The metadata filter is too strict
    3. The embedding model is outdated
    4. The output token limit is set too low Correct answer.

    Explanation

    Truncated responses usually indicate the max output tokens is too small relative to the answer length. This is an output budget issue, distinct from input context size.

  133. Question 133Context Management and Retrieval-Augmented Generation

    An evaluation shows the retriever surfaces the right document but the model still answers incorrectly. Which is the most likely context-management cause?

    1. The relevant passage was truncated or buried among many low-value chunks Correct answer.
    2. The vector store used cosine similarity
    3. The embedding model was trained on too much data
    4. The user asked a question at all

    Explanation

    Even with correct retrieval, the evidence must actually reach the model intact and be salient within the prompt. Truncation or dilution by noisy chunks can cause the model to miss or misread the key passage.

  134. Question 134Context Management and Retrieval-Augmented Generation

    Which design most reduces the risk of stale answers in a RAG system over frequently updated documents?

    1. Rebuild the index on a schedule and filter by document freshness Correct answer.
    2. Increase the temperature on each request
    3. Freeze the index permanently after the first build
    4. Rely on the model's pretrained knowledge for recent facts

    Explanation

    Periodic reindexing plus freshness metadata filters ensures retrieval draws on current documents. Without updates, the index drifts out of date relative to the source corpus.

  135. Question 135Context Management and Retrieval-Augmented Generation

    A team wants to compare two chunking strategies for answer quality. Which evaluation approach is most sound?

    1. Choose the strategy with the largest average chunk size
    2. Pick the strategy that produces the most chunks
    3. Ask a single user which feels better
    4. Compare average answer correctness on a labeled question set while holding other pipeline stages fixed Correct answer.

    Explanation

    A controlled comparison changes only the chunking variable and measures answer correctness against labeled ground truth. Holding other stages fixed isolates the effect of chunking.

  136. Question 136Context Management and Retrieval-Augmented Generation

    Which component of a retrieval-augmented generation (RAG) system is responsible for converting a user's query into a vector for similarity search?

    1. The document chunker
    2. The vector database index
    3. The large language model
    4. The embedding model Correct answer.

    Explanation

    In a RAG pipeline, the embedding model encodes both documents and queries into dense vectors so that semantically similar items are close in vector space. The LLM generates the final answer, the vector index stores and searches embeddings, and the chunker prepares documents before embedding.

  137. Question 137Context Management and Retrieval-Augmented Generation

    A RAG system must answer questions about a product catalog where each item has a category, price, and description. Users often ask things like 'show me affordable running shoes under $80'. Which retrieval approach best handles this type of query?

    1. Random sampling of products followed by LLM filtering
    2. Pure semantic search over description embeddings
    3. Keyword search on the product name field only
    4. Hybrid search combining metadata filters with semantic similarity Correct answer.

    Explanation

    Queries that mix structured constraints (price, category) with semantic intent (running shoes) are best served by applying metadata filters to narrow the candidate set and then using semantic similarity to rank the remaining items. This ensures hard constraints are respected while still capturing meaning.

  138. Question 138Context Management and Retrieval-Augmented Generation

    In a RAG system, what is the primary purpose of including the retrieved document's source identifier (e.g., URL or document ID) in the prompt alongside the passage text?

    1. It allows the model to cite sources and enables users to verify claims. Correct answer.
    2. It guarantees the model will not hallucinate.
    3. It reduces the token count of the prompt.
    4. It speeds up the embedding step during retrieval.

    Explanation

    Source identifiers let the model attribute statements to specific documents, which supports verification and trust. They do not affect token count, embedding speed, or eliminate hallucination entirely.

  139. Question 139Context Management and Retrieval-Augmented Generation

    A team is building a RAG system over a knowledge base that is updated daily. They want to ensure that answers reflect the most current information and do not become stale. Which design choice most directly addresses this requirement?

    1. Increasing the number of retrieved chunks per query
    2. Using a larger language model for generation
    3. Scheduling regular re-indexing of updated documents Correct answer.
    4. Adding a reranker after initial retrieval

    Explanation

    Staleness in RAG arises when the vector index contains outdated embeddings of documents that have since changed. Regularly re-indexing updated documents ensures the retrieval corpus stays in sync with the source of truth.

  140. Question 140Context Management and Retrieval-Augmented Generation

    A RAG system answers questions by retrieving passages from a large corpus. Evaluation shows that for multi-hop questions, the retriever often returns only one of the two necessary documents. Which change to the retrieval strategy is most likely to improve performance on such questions?

    1. Replacing the embedding model with one that produces larger vectors
    2. Decreasing the chunk size to increase retrieval granularity
    3. Increasing the number of retrieved passages and using a reranker to select the most relevant ones Correct answer.
    4. Switching from dense retrieval to sparse keyword-based retrieval

    Explanation

    Multi-hop questions require evidence from multiple documents. Retrieving a larger candidate set increases the chance that all necessary documents are included, and a reranker then prioritizes the most relevant passages to fit the context budget.

  141. Question 141Safety, Guardrails, and Responsible AI

    Which practice most directly reduces the risk of a model producing harmful content in a customer-facing application?

    1. Removing response logging to protect user privacy
    2. Shortening the context window to limit conversation history
    3. Adding a system prompt that defines allowed behavior and refusal criteria Correct answer.
    4. Increasing the model's temperature setting for more varied responses

    Explanation

    A system prompt establishes behavioral boundaries and refusal criteria that shape how the model responds to user input. Other sampling or context settings affect style and memory, not safety policy.

  142. Question 142Safety, Guardrails, and Responsible AI

    A team wants defense in depth for an LLM feature. Which layer complements input filtering and output filtering?

    1. Compressing prompts before sending
    2. Human review for high-risk actions Correct answer.
    3. Caching model responses at the edge
    4. Raising the max_tokens limit

    Explanation

    Defense in depth combines automated controls with human oversight where consequences are high. Input and output filters catch many issues, but human review adds judgment for consequential actions.

  143. Question 143Safety, Guardrails, and Responsible AI

    What is the primary purpose of a content moderation classifier placed before the main model in a pipeline?

    1. To reduce token costs by shortening prompts
    2. To translate user input into the model's preferred language
    3. To improve factual accuracy of model answers
    4. To screen requests and block or route disallowed inputs Correct answer.

    Explanation

    An input moderation classifier evaluates requests against policy and can block or reroute them before the main model is invoked. It is a gatekeeping control rather than an accuracy or cost optimization tool.

  144. Question 144Safety, Guardrails, and Responsible AI

    Which is the most appropriate way to handle a user request that falls outside the assistant's permitted scope?

    1. Answer partially and omit the restricted portion
    2. Silently return an empty response
    3. Return a clear refusal and offer an allowed alternative Correct answer.
    4. Repeat the user's request back verbatim

    Explanation

    Clear refusals that explain the boundary and suggest a permitted alternative keep the interaction useful while respecting scope. Silent failures or partial compliance create confusion and can leak restricted content.

  145. Question 145Safety, Guardrails, and Responsible AI

    What does prompt injection primarily attempt to do?

    1. Overflow the model's token budget
    2. Force the model to respond in a specific language
    3. Extract the model's weight values
    4. Override the developer's instructions using untrusted input Correct answer.

    Explanation

    Prompt injection embeds adversarial instructions in untrusted content so the model treats them as authoritative and departs from developer intent. It targets instruction hierarchy rather than resources or weights.

  146. Question 146Safety, Guardrails, and Responsible AI

    Why should untrusted retrieved documents be treated differently from developer instructions in a RAG system?

    1. Retrieved text may contain adversarial instructions and must not outrank system policy Correct answer.
    2. Retrieved text is encrypted at rest
    3. Retrieved text cannot be processed by the model
    4. Retrieved text is always shorter than instructions

    Explanation

    Content from external sources is untrusted and can carry injected instructions, so the system must enforce an instruction hierarchy where developer and system policy outrank retrieved text. Treating all text as equally authoritative enables injection.

  147. Question 147Safety, Guardrails, and Responsible AI

    A model output must be rendered on a web page. Which control most directly prevents stored cross-site scripting from model-generated text?

    1. Escaping or sanitizing the output before rendering Correct answer.
    2. Increasing the model's temperature
    3. Logging the raw output to a database
    4. Requesting JSON mode from the model

    Explanation

    Output encoding and sanitization neutralize executable markup so model text cannot run as script in the browser. Model-side formatting choices do not substitute for safe rendering.

  148. Question 148Safety, Guardrails, and Responsible AI

    Which metric best captures whether a guardrail is failing to block harmful inputs?

    1. False negative rate on a labeled harmful set Correct answer.
    2. Average latency of the classifier
    3. Cache hit ratio
    4. Token count per request

    Explanation

    False negatives measure harmful cases the guardrail let through, which is the core failure mode for blocking controls. Latency and cost metrics describe performance, not protective coverage.

  149. Question 149Safety, Guardrails, and Responsible AI

    What is the main risk of granting an agent broad tool permissions by default?

    1. Tool calls become slower to execute
    2. The model loses access to its context window
    3. A compromised or manipulated agent could take damaging actions Correct answer.
    4. The model may generate longer responses

    Explanation

    Least privilege limits the blast radius if the agent is manipulated through injection or errs. Broad default permissions expand what a single bad decision can do.

  150. Question 150Safety, Guardrails, and Responsible AI

    A team must decide whether a model response should be logged for safety review. Which consideration most strongly supports logging with appropriate controls?

    1. Logs guarantee the model cannot be misused
    2. Logs eliminate the need for output filters
    3. Logs reduce the model's inference cost
    4. Logs allow retrospective detection and investigation of harmful behavior Correct answer.

    Explanation

    Audit logs enable detection, investigation, and accountability for incidents after the fact. They support but do not replace preventive controls such as filters.

  151. Question 151Safety, Guardrails, and Responsible AI

    Which approach best mitigates hallucinated citations in a research assistant?

    1. Increasing the model's creativity setting
    2. Disabling retrieval to reduce latency
    3. Shortening the system prompt
    4. Requiring answers to cite retrieved passages and verifying the citations Correct answer.

    Explanation

    Grounding answers in retrieved passages and programmatically verifying that cited sources exist and support the claim reduces fabricated references. Creativity settings and prompt length do not establish factual support.

  152. Question 152Safety, Guardrails, and Responsible AI

    What does a red-team exercise for an LLM application primarily aim to discover?

    1. Weaknesses in safety controls under adversarial input Correct answer.
    2. The optimal batch size for inference
    3. The cost of fine-tuning the model
    4. The model's preferred response language

    Explanation

    Red teaming probes the system adversarially to surface control gaps such as bypassable filters or instruction overrides. It is a security and safety assessment, not a performance tuning exercise.

  153. Question 153Safety, Guardrails, and Responsible AI

    A system prompt says to never reveal internal instructions. A user asks the model to repeat its system prompt. What is the most robust mitigation?

    1. Rely solely on the instruction not to reveal the prompt
    2. Increase temperature so the answer varies
    3. Add output filtering that detects and redacts system prompt content Correct answer.
    4. Store the system prompt in the user message

    Explanation

    Instruction-only defenses can be bypassed, so a detection layer that recognizes and redacts sensitive prompt content provides a stronger control. Defense in depth does not rely on a single instruction.

  154. Question 154Safety, Guardrails, and Responsible AI

    Which design best limits the harm of an incorrect high-stakes recommendation from an AI assistant?

    1. Requiring human confirmation before the action is executed Correct answer.
    2. Presenting the recommendation as definitive to build trust
    3. Removing uncertainty disclosures to reduce confusion
    4. Hiding the model's reasoning from the operator

    Explanation

    Human-in-the-loop confirmation places a judgment step before consequential actions, containing the impact of model error. Transparency and uncertainty disclosure further support informed review.

  155. Question 155Safety, Guardrails, and Responsible AI

    A classifier flags 30 percent of benign inputs as harmful. Which property is most clearly deficient?

    1. Recall on harmful inputs
    2. Throughput of the guardrail
    3. Latency of the guardrail
    4. Precision of the guardrail Correct answer.

    Explanation

    Precision measures how many flagged items are truly harmful; a high false positive rate indicates low precision. Recall concerns missed harmful cases, while latency and throughput are performance characteristics.

  156. Question 156Safety, Guardrails, and Responsible AI

    Which is the strongest reason to document a model's intended use and known limitations for downstream developers?

    1. It guarantees regulatory compliance in all jurisdictions
    2. It helps integrators apply the system within safe operating bounds Correct answer.
    3. It eliminates the need for testing in the integrating application
    4. It reduces the number of API calls

    Explanation

    Intended-use documentation communicates where the system is appropriate and where it may fail, enabling integrators to design safeguards and avoid misuse. It informs but does not replace testing or legal review.

  157. Question 157Safety, Guardrails, and Responsible AI

    An agent can read email and send email. A retrieved email contains text instructing the agent to forward the inbox to an external address. What is the most appropriate architectural response?

    1. Disable all logging to avoid storing the malicious email
    2. Trust the email because it came from the user's inbox
    3. Treat email content as untrusted data and require confirmation for outbound sends Correct answer.
    4. Increase the model's context window to capture the full email

    Explanation

    Email bodies are untrusted content that can carry injected instructions, so the agent must not treat them as commands. Requiring explicit confirmation for outbound actions contains the impact of manipulation.

  158. Question 158Safety, Guardrails, and Responsible AI

    A team wants to enforce a policy that certain medical topics must always include a disclaimer. Where should this be enforced most reliably?

    1. As a suggestion in the model's temperature setting
    2. As a deterministic post-processing check on model output Correct answer.
    3. In the model's training data only
    4. In the user-facing help documentation only

    Explanation

    Deterministic post-processing verifies the required element is present and can add or block output if it is missing, giving reliable enforcement. Documentation and training data influence but do not guarantee per-response compliance.

  159. Question 159Safety, Guardrails, and Responsible AI

    A safety review finds that a guardrail blocks harmful content in English but not in a lower-resource language. Which conclusion is best supported?

    1. The guardrail is robust because it works in the primary language
    2. Language coverage is irrelevant to safety outcomes
    3. Coverage is uneven across languages and requires multilingual evaluation Correct answer.
    4. The model should be restricted to a single language permanently

    Explanation

    Uneven performance across languages shows the evaluation set did not represent those languages and the control generalizes poorly. The appropriate response is multilingual testing and mitigation.

  160. Question 160Safety, Guardrails, and Responsible AI

    Which combination best reflects a layered approach to preventing sensitive data disclosure in model responses?

    1. Prompt instructions alone
    2. Caching responses to reduce repeated exposure
    3. Higher temperature and longer max_tokens
    4. Input redaction, output scanning, and access controls on retrieved data Correct answer.

    Explanation

    Layered controls address disclosure at multiple points: redacting sensitive input, scanning output, and restricting what data retrieval can access. A single instruction layer is insufficient on its own.

  161. Question 161Evaluation, Testing, and Observability

    A team wants a reliable regression signal from an LLM-based support assistant whose outputs vary between runs. Which evaluation approach best supports stable pass/fail gating in CI?

    1. Curated test cases with deterministic assertions or graded rubrics Correct answer.
    2. Relying on the model's self-reported confidence score
    3. Comparing only aggregate latency metrics across releases
    4. Manually reviewing a random transcript each release

    Explanation

    Stable gating requires a fixed evaluation set with explicit expected outcomes, either exact checks or a rubric scored consistently. Self-reported confidence and latency do not measure output quality. Manual spot checks are not repeatable enough for automated gates.

  162. Question 162Evaluation, Testing, and Observability

    What is the primary purpose of a held-out evaluation set in an LLM application pipeline?

    1. To serve as the fallback response source when the model fails
    2. To store the prompts that produced the best demo results
    3. To measure quality on data not used for prompt or model tuning Correct answer.
    4. To reduce token costs during production inference

    Explanation

    A held-out set estimates generalization by testing on examples the team did not optimize against. If the same examples drive prompt iteration, scores become optimistic and stop predicting real-world performance.

  163. Question 163Evaluation, Testing, and Observability

    An LLM judge is used to grade open-ended answers. Which practice most improves the reliability of its scores?

    1. Asking the judge to grade many answers in one long response
    2. Providing a detailed rubric with graded criteria and examples Correct answer.
    3. Removing the reference answer to avoid biasing the judge
    4. Increasing the judge's temperature so it explores more

    Explanation

    Explicit rubrics with anchor examples constrain the judge to consistent criteria and reduce drift. Higher temperature increases variance, and omitting references removes grounding the judge needs.

  164. Question 164Evaluation, Testing, and Observability

    A judge model agrees with human reviewers only 60% of the time on a binary quality label. What is the most appropriate next step?

    1. Switch the judge to a larger model without changing the prompt
    2. Replace the judge with human review for all future runs
    3. Ship the judge as-is since 60% exceeds chance
    4. Calibrate the judge against a human-labeled set and revise its rubric Correct answer.

    Explanation

    Low agreement signals that the judge's criteria do not match human standards, so the rubric and prompt must be aligned using labeled examples. Only after calibration should the judge be trusted for scale.

  165. Question 165Evaluation, Testing, and Observability

    Which metric is most appropriate for evaluating a retrieval step that must return all relevant documents?

    1. Token count of the generated answer
    2. Average response latency
    3. Perplexity of the query embedding
    4. Recall of the retrieved document set Correct answer.

    Explanation

    Recall measures the fraction of relevant documents that were retrieved, which directly reflects whether anything important was missed. Latency and token counts describe cost, not retrieval completeness.

  166. Question 166Evaluation, Testing, and Observability

    A RAG system returns correct answers but cites irrelevant sources. Which evaluation dimension does this failure primarily concern?

    1. Prompt token efficiency
    2. Faithfulness of the answer to the retrieved context
    3. Retrieval latency
    4. Attribution or citation accuracy Correct answer.

    Explanation

    Attribution accuracy checks whether cited sources actually support the claims made. The answer can be faithful to context while still pointing to the wrong documents. Faithfulness concerns grounding, not citation correctness.

  167. Question 167Evaluation, Testing, and Observability

    What does tracing add to an LLM application that plain logging of final outputs does not?

    1. It replaces the need for evaluation datasets
    2. It guarantees deterministic model responses
    3. It captures the sequence of intermediate steps and their inputs and outputs Correct answer.
    4. It encrypts sensitive data in transit

    Explanation

    Tracing records each span of a request, such as retrieval, tool calls, and model invocations, so failures can be localized. Plain output logs show only the end result and hide where the pipeline went wrong.

  168. Question 168Evaluation, Testing, and Observability

    A production alert fires whenever average latency exceeds a threshold, but it triggers constantly during normal traffic spikes. What is the best improvement?

    1. Alert on total request count instead of latency
    2. Remove the alert and rely on dashboards
    3. Alert on sustained percentile latency rather than a raw average Correct answer.
    4. Lower the threshold so spikes are caught earlier

    Explanation

    Percentile-based thresholds over a sustained window distinguish real degradation from brief spikes and are robust to outliers. Averages are easily skewed by a few slow requests.

  169. Question 169Evaluation, Testing, and Observability

    Which sampling strategy best balances cost and coverage when reviewing production LLM traffic for quality issues?

    1. Review only requests users explicitly complained about
    2. Uniform random sampling plus targeted sampling of high-risk categories Correct answer.
    3. Review the single longest request each day
    4. Review only requests that returned an error code

    Explanation

    Combining uniform random sampling with targeted sampling of risky categories captures both baseline quality and known failure modes. Complaint-only or error-only review misses silent quality failures.

  170. Question 170Evaluation, Testing, and Observability

    A test suite of 40 examples shows a 5-point accuracy drop after a prompt change. What should the team check before concluding the change caused a regression?

    1. Whether the model provider changed its pricing
    2. Whether the new prompt is shorter than the old one
    3. Whether the test examples were written by the same author
    4. Whether the drop exceeds the confidence interval for that sample size Correct answer.

    Explanation

    With small samples, observed differences can fall within statistical noise, so the team should assess significance before attributing the drop to the change. Prompt length or authorship does not determine whether the difference is real.

  171. Question 171Evaluation, Testing, and Observability

    What is the main risk of repeatedly tuning prompts against the same evaluation set?

    1. The model will refuse to answer those examples
    2. The evaluation set will automatically shrink
    3. Token usage will grow with each iteration
    4. Evaluation scores become optimistic and no longer reflect generalization Correct answer.

    Explanation

    Iterating against a fixed set causes overfitting to those examples, inflating scores relative to unseen inputs. A separate held-out or rotating set preserves an honest estimate of generalization.

  172. Question 172Evaluation, Testing, and Observability

    Which is the most direct indicator that an LLM application is hallucinating in a RAG pipeline?

    1. The model used a high temperature setting
    2. The answer contains claims unsupported by any retrieved passage Correct answer.
    3. The answer is longer than the retrieved context
    4. The retrieval step returned more than five documents

    Explanation

    Hallucination is defined by claims that lack grounding in the provided context. Length, document count, and temperature are contributing factors but not direct evidence of unsupported claims.

  173. Question 173Evaluation, Testing, and Observability

    An agentic workflow fails intermittently at a tool-calling step. Which observability artifact is most useful for diagnosing the failure?

    1. The aggregate success rate over the past month
    2. The average token count per request
    3. The trace showing the tool input, output, and error for the failing span Correct answer.
    4. The system prompt version history

    Explanation

    Span-level traces expose the exact inputs and errors at the failing step, allowing root-cause analysis. Aggregate metrics and prompt history lack the per-request detail needed to isolate the fault.

  174. Question 174Evaluation, Testing, and Observability

    Which evaluation design best tests whether a model follows negative constraints such as 'do not mention competitors'?

    1. Measuring the average response length
    2. Comparing outputs to a single golden answer
    3. Including adversarial prompts that tempt the model to violate the constraint Correct answer.
    4. Asking the model to state whether it followed the constraint

    Explanation

    Constraint compliance is best tested with inputs designed to provoke violations, then checked programmatically or by rubric. Self-report is unreliable, and a single golden answer cannot cover the space of possible violations.

  175. Question 175Evaluation, Testing, and Observability

    A team reports that their evaluation shows 92% accuracy, but the dataset contains 90% examples of one class. What does this reveal?

    1. The evaluation should switch to measuring latency instead
    2. The dataset is large enough to trust the accuracy figure
    3. Accuracy is inflated by class imbalance and should be paired with per-class metrics Correct answer.
    4. The model is well calibrated across all classes

    Explanation

    With severe class imbalance, a trivial majority-class predictor can score near 90%, so overall accuracy hides poor minority-class performance. Per-class precision and recall reveal the true picture.

  176. Question 176Evaluation, Testing, and Observability

    What is the role of a golden dataset in a regression testing workflow?

    1. To provide stable expected outputs or judgments for comparison across versions Correct answer.
    2. To train the base model on domain-specific data
    3. To store production secrets used by the test harness
    4. To replace human review entirely in all cases

    Explanation

    A golden dataset offers a fixed reference of inputs and expected outcomes so successive versions can be compared on equal footing. It supports regression detection but does not train the model or eliminate human oversight.

  177. Question 177Evaluation, Testing, and Observability

    Which approach best detects a gradual drift in answer quality over weeks when no explicit failures are reported?

    1. Tracking a rolling quality score from sampled production traffic over time Correct answer.
    2. Reviewing only the first request after each deployment
    3. Relying on user complaints as the sole signal
    4. Checking the model provider's status page daily

    Explanation

    Continuous sampling with a rolling quality score surfaces slow degradation that individual requests do not reveal. Complaints and status pages miss silent quality decline.

  178. Question 178Evaluation, Testing, and Observability

    A team wants to compare two prompt variants fairly. Which practice best controls for confounding factors?

    1. Change the model, temperature, and prompt at the same time
    2. Run variant A on Monday and variant B on Friday
    3. Evaluate each variant on a different dataset
    4. Hold model and settings constant and vary only the prompt Correct answer.

    Explanation

    Fair comparison requires changing one factor at a time while keeping model, parameters, and data constant. Varying multiple factors or datasets makes it impossible to attribute differences to the prompt.

  179. Question 179Evaluation, Testing, and Observability

    Which failure mode is most likely when an LLM judge is asked to score responses without being told the scoring scale?

    1. It will automatically generate a new rubric
    2. It will refuse to produce any score
    3. It will apply inconsistent scales across items, reducing comparability Correct answer.
    4. It will always return the maximum score

    Explanation

    Without a defined scale, the judge improvises criteria per item, producing scores that cannot be compared or aggregated reliably. Explicit scales and anchors are needed for consistent grading.

  180. Question 180Evaluation, Testing, and Observability

    Which signal most directly indicates that an LLM feature's cost is growing without a corresponding quality gain?

    1. An increase in the number of deployed prompt versions
    2. A rise in the count of unit tests in the repository
    3. Rising average tokens per request while quality scores stay flat Correct answer.
    4. A decrease in the number of evaluation examples

    Explanation

    Cost per request scales with token usage, so rising tokens alongside flat quality scores indicates inefficiency. The other signals describe development activity, not production cost efficiency.

  181. Question 181Deployment, Integration, and Production Operations

    Which HTTP response header should a production service return to let clients know how long they may cache a response without revalidating it?

    1. Expires-After: 300
    2. X-Cache-Duration: 300
    3. Retry-After: 300
    4. Cache-Control: max-age=300 Correct answer.

    Explanation

    Cache-Control with max-age is the standard directive that sets a freshness lifetime for a response. Clients and intermediary caches use it to decide when to revalidate.

  182. Question 182Deployment, Integration, and Production Operations

    A team wants to detect regressions before users see them by routing a small percentage of live traffic to a new model version. Which deployment strategy does this describe?

    1. Blue-green deployment
    2. Canary release Correct answer.
    3. Shadow deployment
    4. Rolling restart

    Explanation

    A canary release sends a small slice of real traffic to the new version so its behavior can be observed before full rollout. Blue-green swaps entire environments, and shadow deployments mirror traffic without serving responses from the new version.

  183. Question 183Deployment, Integration, and Production Operations

    An application calls the Claude API and receives HTTP 429. Which client behavior is the most appropriate production response?

    1. Log the error and fail the user request permanently
    2. Reduce the request payload size and resend
    3. Immediately retry in a tight loop until it succeeds
    4. Back off with jitter and retry after the indicated delay Correct answer.

    Explanation

    HTTP 429 indicates rate limiting, so the client should wait and retry with exponential backoff plus jitter to avoid synchronized retry storms. Honoring any Retry-After hint further reduces pressure.

  184. Question 184Deployment, Integration, and Production Operations

    What is the primary purpose of running an automated evaluation suite in a CI pipeline before deploying a prompt change?

    1. To reduce the token cost of production requests
    2. To replace the need for production monitoring
    3. To catch quality regressions before they reach users Correct answer.
    4. To guarantee deterministic model outputs

    Explanation

    Evaluation suites act as regression tests for model behavior, catching quality drops introduced by prompt or model changes before release. They complement, rather than replace, production monitoring.

  185. Question 185Deployment, Integration, and Production Operations

    Which practice best protects an API key used by a backend service that calls the Claude API?

    1. Embed the key in the client-side JavaScript bundle
    2. Load the key from a secrets manager at runtime Correct answer.
    3. Store the key in a version-controlled config file
    4. Commit the key to a private repository branch

    Explanation

    Secrets should be injected at runtime from a managed secret store so they are not exposed in code, bundles, or version history. This also enables rotation without redeploying code.

  186. Question 186Deployment, Integration, and Production Operations

    Which signal is the most direct indicator of end-to-end latency experienced by users of an LLM-backed endpoint?

    1. CPU utilization of the application server
    2. Number of open database connections
    3. Server-side request duration percentiles Correct answer.
    4. Count of deployed model versions

    Explanation

    Request duration percentiles (such as p95 and p99) measured at the serving boundary reflect the latency users actually experience. Resource metrics are causes rather than the user-facing effect.

  187. Question 187Deployment, Integration, and Production Operations

    A service must remain functional when the Claude API is temporarily unavailable. Which design pattern addresses this requirement?

    1. Circuit breaker with a fallback response Correct answer.
    2. Synchronous blocking calls with no timeout
    3. Increasing the request batch size
    4. Retrying indefinitely until the API recovers

    Explanation

    A circuit breaker stops sending requests after repeated failures and can route to a fallback, preserving service availability during outages. This prevents cascading failures and thread exhaustion.

  188. Question 188Deployment, Integration, and Production Operations

    Why is idempotency important when retrying a request that triggers a side effect such as creating a record?

    1. It encrypts the request in transit
    2. It prevents duplicate side effects from repeated attempts Correct answer.
    3. It compresses the request payload
    4. It guarantees lower latency on retries

    Explanation

    An idempotent operation produces the same effect whether executed once or multiple times, so a retry after an ambiguous failure does not create duplicates. Idempotency keys let the server detect and deduplicate retried requests.

  189. Question 189Deployment, Integration, and Production Operations

    A team wants to roll out a new prompt to 10 percent of users, monitor quality metrics, and automatically revert if error rates rise. Which combination of capabilities is required?

    1. Static prompt templating with manual rollback
    2. Database migrations with schema versioning
    3. Feature flags with metric-based automated rollback Correct answer.
    4. Load balancing across identical instances

    Explanation

    Feature flags control exposure of the new prompt to a subset of users, and automated rollback tied to metrics restores the prior behavior when thresholds are breached. Together they enable controlled, observable rollout.

  190. Question 190Deployment, Integration, and Production Operations

    Which logging practice is most important when capturing LLM request and response data in production?

    1. Storing logs only in application memory
    2. Redacting sensitive fields and applying retention limits Correct answer.
    3. Logging full prompts and responses verbatim to a shared bucket
    4. Disabling logging to minimize storage cost

    Explanation

    LLM traffic can contain personal or confidential data, so logs should redact sensitive fields and follow defined retention policies. This balances observability with privacy and compliance obligations.

  191. Question 191Deployment, Integration, and Production Operations

    An application needs to stream model output to users. Which operational consideration is most specific to streaming responses?

    1. The total response size must be known before the request
    2. Time-to-first-token becomes a key latency metric Correct answer.
    3. Responses cannot be cached at any layer
    4. The model cannot be changed without downtime

    Explanation

    With streaming, users perceive responsiveness through how quickly the first token arrives, so time-to-first-token is a primary latency indicator alongside total duration. Total size is not required in advance.

  192. Question 192Deployment, Integration, and Production Operations

    What is the main benefit of pinning a specific model version in a production configuration?

    1. It makes responses deterministic across runs
    2. It reduces the per-token price of requests
    3. It increases the maximum context window
    4. It prevents silent behavior changes from automatic upgrades Correct answer.

    Explanation

    Pinning a model version keeps production behavior stable because upgrades happen only when the team deliberately changes the pin. This makes evaluation and rollback predictable.

  193. Question 193Deployment, Integration, and Production Operations

    A batch summarization job processes thousands of documents nightly. Which design choice most improves cost efficiency for this workload?

    1. Increasing max_tokens far beyond the expected summary length
    2. Re-sending failed documents without recording their identifiers
    3. Using a batch processing interface that accepts asynchronous jobs Correct answer.
    4. Sending each document as a separate synchronous request

    Explanation

    Batch interfaces process large asynchronous workloads at lower cost than one-by-one synchronous calls and return results when complete. This suits non-interactive nightly jobs well.

  194. Question 194Deployment, Integration, and Production Operations

    Which metric best detects that a model-backed feature is producing lower-quality answers after a deployment, independent of infrastructure health?

    1. Number of HTTP 200 responses returned
    2. Count of active feature flags
    3. Rate of user thumbs-down or task-abandonment signals Correct answer.
    4. Average CPU load on inference hosts

    Explanation

    Quality regressions show up in user feedback and behavioral signals such as thumbs-down rates or task abandonment, which infrastructure metrics cannot reveal. These signals should be tracked alongside latency and error rates.

  195. Question 195Deployment, Integration, and Production Operations

    A production system experiences a spike in timeouts when prompts grow longer. Which mitigation most directly addresses the root cause?

    1. Disabling request logging
    2. Adding more retries on timeout
    3. Lowering the client timeout threshold
    4. Reducing prompt size and setting an appropriate max_tokens limit Correct answer.

    Explanation

    Longer prompts increase processing time, so trimming prompt content and capping output length reduces work per request and brings latency back within budget. Retries and shorter client timeouts do not reduce server-side processing time.

  196. Question 196Deployment, Integration, and Production Operations

    Why should a health check endpoint for an LLM-backed service verify dependency reachability rather than only confirming the process is running?

    1. It reduces the number of model versions in production
    2. It allows the load balancer to route traffic away from instances that cannot serve requests Correct answer.
    3. It eliminates the need for retries in the client
    4. It lowers the token cost of health checks

    Explanation

    A readiness check that includes dependency reachability lets the load balancer stop routing traffic to instances that cannot complete requests. This prevents avoidable user-facing errors during partial outages.

  197. Question 197Deployment, Integration, and Production Operations

    A team observes that p99 latency is high while p50 is normal. Which interpretation is most reasonable?

    1. A small subset of requests is hitting slow paths such as long prompts or retries Correct answer.
    2. The deployment has too many model versions
    3. The service is returning incorrect answers
    4. The model is uniformly slow for all requests

    Explanation

    A normal median with an elevated p99 indicates that most requests are fast while a minority take unusually long, often due to long inputs, retries, or dependency hiccups. Investigating the tail cases targets the actual cause.

  198. Question 198Deployment, Integration, and Production Operations

    An organization must ensure that prompts and responses containing regulated data are not sent to an external model provider without safeguards. Which control set is most appropriate?

    1. Increasing retry counts and timeouts
    2. Pre-send redaction with policy checks and audited data-handling agreements Correct answer.
    3. Enabling verbose logging of raw payloads
    4. Rotating API keys on every request

    Explanation

    Regulated data requires controls that remove or mask sensitive content before transmission and that document provider obligations through audited agreements. These measures address both exposure risk and compliance evidence.

  199. Question 199Deployment, Integration, and Production Operations

    A team wants reproducible evaluation results across CI runs for a prompt change. Which step is most important?

    1. Running evaluations only against live production traffic
    2. Varying the temperature on each run to sample broadly
    3. Using a fixed evaluation dataset and recorded model settings Correct answer.
    4. Allowing evaluators to change the rubric between runs

    Explanation

    Reproducibility requires holding inputs and configuration constant so that differences in results can be attributed to the change under test. A fixed dataset with recorded settings provides that baseline.

  200. Question 200Deployment, Integration, and Production Operations

    Which approach best handles a downstream dependency that fails intermittently for a small fraction of requests?

    1. Disabling monitoring to reduce noise
    2. Failing the entire batch when any single call errors
    3. Removing all timeouts to let slow calls finish
    4. Retrying only transient errors with bounded backoff Correct answer.

    Explanation

    Intermittent failures are typically transient, so bounded retries with backoff recover most of them without unbounded resource use. Retrying only transient error classes avoids wasting attempts on permanent failures.

Claude Certified Architect - Foundations flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Context window
The maximum number of tokens a single request may contain across system prompt, history, tool definitions, retrieved documents, and generated output. Exceeding it forces truncation or summarization.
Temperature parameter
A sampling setting that scales randomness. Values near zero make output nearly deterministic and repeatable; higher values broaden the distribution and increase variety and variance.
Extended thinking
A mode where the model produces additional internal reasoning tokens before its final answer. It raises latency and cost but improves accuracy on multi-step problems.
Training cutoff
The date after which the model has no native knowledge. Facts past this point must be supplied through retrieval or directly in the prompt.
Stop sequence
A string that halts generation as soon as the model produces it. It truncates output at a chosen boundary rather than summarizing or restructuring it.
System prompt role
It establishes persistent persona, constraints, and output format for the whole conversation. Task-specific user input should not be trusted to override it.
Few-shot prompting
Including several worked input-output examples in the prompt. Examples steer format and reasoning style more reliably than descriptive adjectives, and precede the target input.
Chain-of-thought prompting
Asking the model to produce intermediate reasoning steps before the final answer. It measurably improves accuracy on multi-step arithmetic, logic, and planning tasks.
Assistant prefill
Supplying the opening text of the model's reply so generation continues from that prefix. It constrains output format without adding extra instructions.
Delimiter isolation
Wrapping untrusted content in tags or quotes so it is clearly separated from instructions. This reduces instruction bleed and prompt-injection spillover.
Tool use loop
The cycle where the model emits a tool call, the client executes it and returns the result, and the model continues until it stops requesting tools.
Tool input schema
A JSON Schema describing a tool's accepted arguments. The model uses it to construct calls; validation failures should be returned as error results for correction.
Iteration budget
A cap on the number of agent turns or tool calls per run. It bounds cost and latency and prevents runaway loops from a misbehaving agent.
Parallel tool calls
Requesting multiple tool invocations in one turn. Valid only when calls are independent; dependent calls must be sequenced so later arguments can use earlier results.
Human-in-the-loop gate
A required human approval before executing an irreversible or high-impact action such as a payment, deletion, or external write. Read-only lookups usually skip it.
RAG pipeline stages
Chunk documents, embed them, index vectors, retrieve top-k by similarity, optionally rerank, then place selected passages in the prompt for grounded generation.
Chunk size tradeoff
Large chunks dilute embeddings and waste tokens; small chunks lose surrounding context. Chunk boundaries should follow semantic units, not fixed character counts alone.
Hybrid retrieval
Combining dense vector similarity with lexical matching such as BM25. Dense handles paraphrase and semantics; lexical handles exact names, codes, and rare terms.
Reranking
A second-stage model that scores a larger candidate set and keeps only the best few. It improves precision before tokens are spent on prompt context.
Lost in the middle
The tendency for models to attend less reliably to content buried in the middle of a long context. Place the most relevant passages near the beginning or end.
Grounded citations
Tying answer claims to specific retrieved chunk identifiers. Citations make outputs auditable and expose fabricated references that do not map to any source.
Prompt injection
An attack where instructions hidden in untrusted content, such as retrieved documents or user input, attempt to override system rules or hijack tool use.
Defense in depth
Layering input filtering, system-prompt constraints, output validation, and monitoring. Each layer catches failure classes the others miss, so no single control is trusted alone.
Least-privilege tools
Granting an agent only the credentials and permissions its specific task needs. This limits the blast radius if the agent is manipulated or errs.
PII redaction
Removing or masking personal identifiers before content is logged, stored, or sent to third parties. Redacting only at display time leaves data exposed downstream.
Output guardrail classifier
A separate model or rule set that screens generated responses against policy before delivery. It catches violations that prompt instructions alone fail to prevent.
Golden evaluation set
A labeled collection of representative and adversarial cases with expected outcomes. It is the reference for measuring accuracy per category across prompt or model changes.
LLM-as-judge
Using a model to score outputs against a written rubric. It requires calibration against human labels, since uncalibrated judges drift and inflate scores.
End-to-end tracing
Recording prompt version, retrieved chunk IDs, tool calls, token counts, and final output per request. Traces make production failures reproducible and debuggable.
Canary comparison
Routing a small traffic slice to a changed prompt or model and comparing metrics against the baseline. It attributes quality shifts to a specific modification.
Exponential backoff with jitter
A retry strategy that increases wait time between attempts and randomizes it to avoid synchronized retries. Apply only to retryable errors such as rate limits and timeouts.
Model version pinning
Referencing an explicit model version rather than a floating alias. Behavior then changes only on a deliberate, reviewed upgrade rather than silently.

Claude Certified Architect - Foundations study guide

Claude Fundamentals and Model Capabilities

What to know

Understand the Claude model family as a set of capability tiers rather than a single product. Candidates should be able to explain the practical trade-offs among model sizes: the largest models favor complex reasoning, long-form synthesis, and nuanced instruction-following, while smaller/faster models favor latency, throughput, and cost. Know that all current Claude models are conversational text models accessed through a Messages-style API where the request carries a system prompt plus an ordered list of user and assistant turns, and the response contains content blocks (text blocks, and tool-use blocks when tools are supplied).

Key concepts: the distinction between the system prompt and user turns; that the model is stateless, so conversation history must be resent each call; max_tokens as an output cap (not a target); stop sequences; temperature/top-p as sampling controls; and the idea that the model returns structured content blocks rather than a bare string. You should also know that context window size is a hard limit and that output counts toward your budget.

Traps

  • Assuming the model remembers prior calls. It does not; state lives in your application.
  • Treating max_tokens as "how long the answer will be." It is an upper bound; truncation shows up as a stop_reason of max_tokens.
  • Believing a bigger model is always the right choice. Questions often reward matching model tier to task complexity and latency/cost constraints.
  • Confusing "the model refused" with "the model was truncated" or "a tool call was requested."

Rule of thumb

Read stop_reason first: end_turn means a natural finish, max_tokens means you were cut off, tool_use means the model wants you to act. Diagnose before you change the prompt.

Prompt Engineering and System Prompt Design

What to know

Strong prompts are explicit about role, task, constraints, input, and output format. The system prompt sets durable behavior (persona, policies, tone, tool-use rules); user turns carry the task and data. Candidates should understand techniques: giving clear instructions and context, using examples (few-shot) to demonstrate format, asking the model to think step by step before answering for reasoning-heavy tasks, and specifying output structure (e.g., "return JSON with keys a and b").

Know how to place long documents: put large reference material in the prompt with clear delimiters (XML-style tags work well) and put the actual question after the documents. Understand that instructions and data should be separated so untrusted content cannot be mistaken for directives.

Traps

  • Vague instructions ("be helpful") instead of concrete constraints and formats.
  • Negative-only instructions ("don't be verbose") without saying what to do instead.
  • Putting the question before a huge document, which buries the task.
  • Over-stuffing examples until they crowd out the actual input, or examples that contradict the stated rules.
  • Assuming temperature fixes factual errors; it changes variability, not knowledge.
  • Expecting the system prompt to override clearly conflicting user instructions without reinforcement.

Worked example

For extraction: system prompt states role and rules ("extract only fields present; return JSON; use null when absent"); user turn contains the document in <doc> tags followed by the instruction. This ordering plus a schema plus an explicit null rule reduces hallucinated fields.

Rule of thumb

If a colleague could not follow your prompt unambiguously, neither will the model. Specify role, task, constraints, and output shape, and show one example of "good."

Tool Use and Agentic Workflows

What to know

Tool use turns Claude into an actor: you declare tools with a name, description, and JSON-schema input; the model may respond with a tool_use block containing a tool name and arguments; your code executes it and returns a tool_result block in a subsequent user turn; the loop repeats until the model returns a final text answer. This is a client-executed loop - the model never runs code itself.

Understand tool descriptions as prompts: precise names, clear purpose, and input schemas with required fields and enums dramatically improve reliability. Know how to handle errors: return an error message in the tool result so the model can adapt, rather than silently failing. Understand tool_choice to force or forbid tool use, and parallel tool calls where multiple independent calls arrive together.

Traps

  • Forgetting to send the tool_result back, leaving the loop hanging.
  • Mismatched tool_use_id between the call and its result.
  • Assuming the model validates arguments against your schema; your code must validate and sanitize.
  • Overly terse tool descriptions causing wrong tool selection.
  • Letting an agent loop run unbounded without a max-iterations guard.
  • Trusting tool output as instructions (injection via tool results).

Rule of thumb

The agent loop is: model proposes → your code authorizes and executes → you return results → model continues. Every step must be bounded, validated, and logged.

Context Management and Retrieval-Augmented Generation

What to know

RAG grounds answers in retrieved documents. The pipeline is: chunk documents, embed them, store vectors, embed the query, retrieve top-k candidates, optionally rerank, then assemble a prompt with the retrieved passages and the question. Candidates must understand that retrieval quality bounds answer quality - no prompt fixes missing context.

Key concepts: chunk size and overlap trade-offs (small chunks improve precision but lose surrounding meaning; large chunks preserve context but dilute relevance and consume tokens); top-k selection; reranking to improve ordering; metadata filtering; and citation/attribution so answers can be traced to sources. Also understand context-window management: summarizing or trimming old turns, keeping the most relevant passages, and placing the question after documents.

Traps

  • Assuming a larger top-k is always better; irrelevant passages add noise and cost.
  • Ignoring chunking strategy and blaming the model for poor answers.
  • Forgetting to instruct the model to answer only from provided context and to say when the answer is absent.
  • Losing citations by not asking for them or not preserving source IDs.
  • Stuffing the whole corpus into the prompt instead of retrieving.
  • Over-summarizing history until critical constraints vanish.

Rule of thumb

Design retrieval and prompt together: retrieve a focused set, label each passage with an ID, instruct "answer only from the passages; if not present, say so," and require the ID in the answer.

Safety, Guardrails, and Responsible AI

What to know

Safety is layered: model-level behavior (training and system-prompt policies), input filtering, output validation, and human oversight. Candidates should understand prompt injection - untrusted text in documents, tool results, or user input attempting to override instructions - and mitigations: separating data from instructions, delimiting untrusted content, least-privilege tools, and validating model actions before execution.

Know that guardrails should be enforceable in code, not just requested in prose: schema validation, allow-lists, rate limits, PII redaction, content classifiers, and human-in-the-loop approval for high-impact actions. Understand the principle of least privilege for tools (read-only where possible) and that the model's confidence is not a safety control.

Traps

  • Believing a system prompt alone prevents injection; untrusted content can still influence output.
  • Granting broad tool permissions "just in case."
  • Trusting model-generated JSON without schema validation.
  • Skipping human review for irreversible or high-stakes actions (payments, deletions, external messages).
  • Logging raw prompts containing secrets or PII.
  • Assuming refusals are always correct or always wrong; evaluate both over- and under-refusal.

Rule of thumb

Defense in depth: constrain the model, validate every input and output in code, minimize tool privilege, and require human approval before irreversible actions.

Evaluation, Testing, and Observability

What to know

Evaluation answers "is it good enough, and did my change help?" Build a labeled test set that reflects real inputs, including edge cases and adversarial cases. Use a mix of automated checks (exact match, schema validation, keyword/rule checks), model-graded rubrics (LLM-as-judge with explicit criteria), and human review for subjective quality. Track metrics per task: accuracy, groundedness, format compliance, latency, and cost.

Observability means instrumenting the pipeline: log prompts, responses, tool calls, token counts, latencies, and errors with correlation IDs, while redacting sensitive data. Understand regression testing: any prompt or model change should be re-run against the eval set before deployment. Understand that LLM judges have biases (verbosity, position) and should be calibrated against human labels.

Traps

  • Testing only happy-path inputs.
  • Changing a prompt and shipping without re-running evals.
  • Trusting a single LLM-judge score without calibration or spot checks.
  • Measuring only accuracy while ignoring latency, cost, and refusal behavior.
  • Logging without redaction, or logging without correlation IDs so failures can't be traced.
  • Overfitting prompts to a tiny eval set.

Rule of thumb

No change ships without an eval run. Keep a frozen regression set, add a new case for every production failure, and calibrate any automated judge against human ratings before trusting it.

Deployment, Integration, and Production Operations

What to know

Productionizing Claude means treating the API as an unreliable dependency. Design for retries with exponential backoff and jitter on rate limits and transient errors, idempotency keys for side-effecting operations, timeouts, and circuit breakers. Understand rate limiting (requests and tokens per minute) and how to smooth traffic with queues and batching.

Key operational concepts: streaming responses to reduce perceived latency; caching where appropriate; version-pinning model identifiers and having a rollback plan when model versions change; secret management (never in client code); and observability with dashboards and alerts on error rate, latency, and cost. Understand graceful degradation: fall back to a smaller model or a cached answer rather than failing hard.

Traps

  • Retrying non-idempotent operations without idempotency keys, causing duplicate side effects.
  • Hard-coding API keys in client applications.
  • No timeout, so a hung request blocks a worker.
  • Retrying immediately without backoff, amplifying an outage.
  • Assuming a model version is permanent; unpinned versions can shift behavior.
  • Ignoring token-based rate limits when only counting requests.

Rule of thumb

Every external call needs a timeout, a bounded retry with backoff, and a fallback. Pin versions, keep secrets server-side, and alert on error rate, latency, and spend.

Claude Certified Architect - Foundations cheat sheet

Quick reference

Claude Fundamentals and Model Capabilities

  • Model tiers trade capability against latency and cost: larger models favor complex reasoning, smaller models favor throughput and low-latency tasks.

  • The context window bounds the total tokens of system prompt, conversation history, tool definitions, retrieved documents, and the response combined.

  • Output tokens are generated one at a time and each token conditions on all prior tokens, so response cost scales with output length, not input length alone.

  • Temperature near 0 yields near-deterministic sampling; higher values widen the sampling distribution and increase output variance.

  • Extended thinking allocates additional reasoning tokens before the final answer and is appropriate when correctness on multi-step problems outweighs added latency.

  • Multimodal input lets a single request carry text plus images or documents; token cost of an image depends on its resolution and dimensions.

  • A model's training cutoff bounds what it knows natively; anything after that cutoff requires retrieval or inclusion in the prompt.

  • Stop sequences and max output tokens are hard controls on generation length; they truncate rather than summarize.

Quick reference

Prompt Engineering and System Prompt Design

  • The system prompt sets persistent role, constraints, and output format; user turns carry task-specific input and should not be relied on to override system rules.

  • Few-shot examples steer format and reasoning style more reliably than adjectives; place examples before the target input.

  • Instruction-following degrades when a prompt contains contradictory rules; resolve conflicts explicitly rather than stacking both.

  • Chain-of-thought prompting improves multi-step accuracy by eliciting intermediate reasoning before the final answer.

  • Structured output is more reliable when the prompt states the exact schema and includes a filled example of that schema.

  • Prefilling the assistant turn forces the response to begin with a chosen prefix, which constrains format without extra instructions.

  • Delimiters (XML-style tags, triple quotes) separate untrusted content from instructions and reduce prompt-injection spillover.

  • Positive, concrete instructions ('respond in under 100 words') outperform negative abstractions ('do not be verbose').

Quick reference

Tool Use and Agentic Workflows

  • Tool use is a loop: the model emits a tool_use block, the client executes it and returns a tool_result block, and the model continues until it stops requesting tools.

  • Tool descriptions are part of the prompt; ambiguous or overlapping descriptions cause wrong tool selection and wrong arguments.

  • Input schemas constrain arguments via JSON Schema; validation failures should be returned to the model as error results so it can correct.

  • Set a maximum iteration or turn budget per agent run to bound cost and prevent runaway loops.

  • Parallel tool calls are appropriate only for independent operations; dependent calls must be sequenced because later arguments depend on earlier results.

  • Tool errors should be surfaced verbatim to the model rather than silently swallowed, since the model can often recover from a descriptive error.

  • Agentic workflows should be idempotent at the tool layer, because retries after transient failures may re-execute a call.

  • Human-in-the-loop approval gates belong on irreversible or high-blast-radius tools (writes, payments, deletions), not on read-only lookups.

Quick reference

Context Management and Retrieval-Augmented Generation

  • RAG pipeline stages: chunk, embed, index, retrieve top-k by similarity, rerank, then assemble retrieved passages into the prompt.

  • Retrieval quality is bounded by chunking: chunks too large dilute the embedding, chunks too small lose the surrounding context needed to answer.

  • Hybrid retrieval combines dense vector similarity with lexical (BM25-style) matching to cover both semantic and exact-term queries.

  • Reranking a larger candidate set down to a small top-k improves precision before tokens are spent on the prompt.

  • Lost-in-the-middle behavior means models attend less reliably to content buried mid-context; place the most relevant passages near the start or end.

  • Summarization or truncation of older turns is the standard remedy when conversation history approaches the context limit.

  • Citations should be grounded in retrieved chunk identifiers so answers can be traced and hallucinated references detected.

  • Retrieved content is untrusted input; instructions embedded inside documents must not be treated as system-level commands.

Quick reference

Safety, Guardrails, and Responsible AI

  • Defense in depth: input filtering, system-prompt constraints, output validation, and monitoring each catch different failure classes.

  • Prompt injection is mitigated by separating instructions from data and by never granting untrusted content authority over tool calls.

  • Constitutional or policy-based self-critique passes let the model check its draft against stated rules before returning it.

  • Guardrail classifiers on both input and output catch policy violations that prompt instructions alone may miss.

  • Least-privilege tool scoping limits blast radius: an agent should hold only the credentials and permissions its task requires.

  • PII redaction belongs before logging and before sending content to third-party services, not only at display time.

  • Refusals should be specific about the boundary crossed; over-broad refusals are a quality defect measured like any other error.

  • High-stakes outputs (medical, legal, financial) require human review and explicit uncertainty signaling rather than silent confidence.

Quick reference

Evaluation, Testing, and Observability

  • Build a labeled golden set with representative and adversarial cases; measure accuracy per category rather than as one aggregate number.

  • LLM-as-judge scoring requires a rubric and calibration against human labels; uncalibrated judges drift and inflate scores.

  • Regression suites should run on every prompt or model change, since prompt edits are code changes with unbounded blast radius.

  • Track latency percentiles (p50/p95/p99), token counts, cost per request, and tool-call error rates as first-class production metrics.

  • Trace each request end to end: prompt version, retrieved chunk IDs, tool calls, and final output, so failures are reproducible.

  • A/B or canary comparison against a baseline is the reliable way to attribute quality changes to a specific modification.

  • Deterministic assertions (schema validity, required fields, forbidden strings) complement subjective quality scoring.

  • Online evaluation on sampled production traffic catches distribution shift that a static offline set cannot.

Quick reference

Deployment, Integration, and Production Operations

  • Retries should use exponential backoff with jitter and apply only to retryable errors (rate limits, 5xx, timeouts), never to validation errors.

  • Rate limits are enforced on requests per minute and tokens per minute; batch or queue work rather than retrying immediately.

  • Streaming reduces perceived latency and enables early termination but complicates downstream validation, which must buffer before checking.

  • Pin model versions in production and roll forward deliberately; unpinned aliases can change behavior without a code deploy.

  • Prompt and configuration changes belong in version control with the same review and rollback path as application code.

  • Idempotency keys and request timeouts prevent duplicate side effects when a client retries after an ambiguous failure.

  • Caching stable prefixes (system prompt, tool definitions) reduces cost and latency where prompt caching is supported.

  • Health checks, circuit breakers, and graceful degradation (fallback model or cached answer) keep the integration available during upstream incidents.

Claude Certified Architect - Foundations key terms

Context window

The maximum number of tokens a model can consider at once, covering system prompt, conversation history, retrieved documents, and the response. Inputs beyond this limit must be truncated, summarized, or split across calls.

Related: Token, Prompt caching
Token

The basic unit of text a model reads and generates, roughly a word fragment. Token counts drive cost, latency, and context limits, so estimating them accurately matters for capacity planning.

Related: Context window, Max output tokens
Max output tokens

A per-request cap on how many tokens the model may generate. Hitting the cap truncates the response, so long-form tasks need either a higher limit or chunked generation.

Related: Token, Stop sequence
Temperature

A sampling parameter controlling randomness. Lower values make output more deterministic and focused; higher values increase variety. Extraction and classification tasks usually favor low values.

Related: Top-p sampling, Determinism
Top-p sampling

Nucleus sampling that restricts generation to the smallest set of tokens whose cumulative probability exceeds a threshold. Adjusting it together with temperature changes output diversity.

Related: Temperature
Stop sequence

A string that halts generation when produced. Useful for delimiting structured output or ending a turn early, but the stop text itself is typically excluded from the returned content.

Related: Max output tokens
Prefill

The assistant turn partially provided by the caller so the model continues from it. It constrains output format or role, and it cannot be used to force content the model would otherwise refuse.

Related: System prompt, Stop sequence
System prompt

Instructions supplied separately from user messages that set role, tone, constraints, and available tools. It carries higher priority than user turns and should be explicit about refusals and output format.

Related: Role hierarchy, Prompt injection
Role hierarchy

The ordering in which system instructions outrank user messages, which outrank content inside tool results or retrieved documents. Attacks succeed when untrusted text is treated as instructions rather than data.

Related: Prompt injection, Tool result
Few-shot prompting

Including several worked examples in the prompt so the model can imitate the desired pattern. Examples should be varied, correctly labeled, and representative of edge cases you expect.

Related: Prompt template, Chain-of-thought prompting
Chain-of-thought prompting

Asking the model to reason step by step before answering, which often improves multi-step accuracy. The reasoning can be hidden from end users while the final answer is surfaced.

Related: Few-shot prompting, Structured output
Structured output

Constraining responses to a schema such as JSON so downstream code can parse them reliably. It usually combines explicit format instructions, examples, and validation with retry on failure.

Related: Schema validation, Tool use
Prompt template

A reusable prompt with placeholders filled at runtime. Good templates separate static instructions from variable data and guard against untrusted values altering the instructions.

Related: Few-shot prompting, Prompt injection
Prompt injection

An attack where text in user input, documents, or tool output tries to override the developer's instructions. Defenses include instruction hierarchy, delimiting untrusted content, and least-privilege tools.

Related: Role hierarchy, Indirect prompt injection, Guardrail
Indirect prompt injection

Injection delivered through content the model processes rather than the user's direct message, such as a retrieved web page or file. It is dangerous because the attacker never talks to the app directly.

Related: Prompt injection, Retrieval-augmented generation
Tool use

Giving the model a declared set of functions it can request, with the application executing them and returning results. The model chooses arguments; your code decides whether and how to run them.

Related: Tool schema, Tool result, Agentic loop
Tool schema

The name, description, and JSON input definition that tells the model when and how to call a tool. Clear descriptions and tight parameter types reduce malformed or unnecessary calls.

Related: Tool use, Schema validation
Tool result

The output returned to the model after a tool executes, typically as a user-role content block. It is untrusted data and should never be interpreted as higher-priority instructions.

Related: Role hierarchy, Tool use
Agentic loop

The cycle of model reasoning, tool call, result observation, and repeat until a stopping condition. Production loops need iteration caps, timeouts, and cost limits to prevent runaway behavior.

Related: Tool use, Orchestration
Orchestration

The application layer that manages model calls, routes tool requests, tracks state, and enforces limits. It, not the model, owns control flow and side effects.

Related: Agentic loop, Human-in-the-loop
Human-in-the-loop

Requiring human approval before high-impact actions such as payments, deletions, or external messages. It is a primary control for agent autonomy where errors are costly or irreversible.

Related: Least privilege, Guardrail
Least privilege

Granting a tool or agent only the permissions needed for its task. Scoping credentials and restricting filesystem or network access limits damage from mistakes or injected instructions.

Related: Human-in-the-loop, Sandboxing
Sandboxing

Running generated code or tool actions in an isolated environment with restricted resources and no sensitive network access. It contains failures and malicious behavior during execution.

Related: Least privilege, Guardrail
Retrieval-augmented generation

Fetching relevant documents at query time and placing them in the prompt so answers are grounded in current, private knowledge rather than only training data.

Related: Embedding, Chunking, Reranking
Embedding

A numeric vector representing text meaning, used to find semantically similar passages. Similarity is usually measured with cosine distance between query and document vectors.

Related: Vector database, Semantic search
Chunking

Splitting source documents into passages small enough to embed and fit in context. Chunk size and overlap trade retrieval precision against losing surrounding meaning.

Related: Retrieval-augmented generation, Reranking
Vector database

A store that indexes embeddings and returns nearest neighbors for a query vector. It enables semantic search over large corpora without scanning every document.

Related: Embedding, Semantic search
Semantic search

Retrieval ranked by meaning similarity rather than exact keyword match. It handles paraphrase well but can miss rare identifiers, so hybrid approaches with keyword search are common.

Related: Embedding, Hybrid search
Hybrid search

Combining lexical keyword search with semantic vector search and merging the ranked results. It improves recall for exact terms such as product codes while keeping meaning-based matching.

Related: Semantic search, Reranking
Reranking

A second-stage model that reorders initially retrieved passages by relevance to the query. It improves precision when the first-stage recall set is broad.

Related: Hybrid search, Retrieval-augmented generation
Grounding

Tying model answers to supplied sources so claims can be traced and verified. Citations and explicit instructions to answer only from provided context reduce fabrication.

Related: Hallucination, Retrieval-augmented generation
Hallucination

Confidently stated output that is unsupported or false. Mitigations include grounding in retrieved sources, asking the model to abstain when evidence is missing, and automated verification.

Related: Grounding, Evaluation harness
Guardrail

A control that constrains model behavior, such as input filtering, output validation, topic restrictions, or refusal rules. Guardrails complement, but do not replace, careful prompt design.

Related: Prompt injection, Schema validation
Red teaming

Deliberately probing a system with adversarial inputs to find harmful, unsafe, or off-policy behavior before release. Findings feed back into prompts, guardrails, and evaluations.

Related: Evaluation harness, Guardrail
Evaluation harness

A repeatable setup that runs a fixed dataset of inputs through the system and scores outputs against criteria. It enables regression detection when prompts or models change.

Related: Golden dataset, LLM-as-judge
Golden dataset

A curated set of representative inputs with expected outputs or rubric labels, used as the reference for measuring quality. It should cover common cases, edge cases, and known failure modes.

Related: Evaluation harness, LLM-as-judge
LLM-as-judge

Using a model to score another model's output against a rubric. It scales subjective evaluation but needs calibration against human labels to avoid systematic bias.

Related: Evaluation harness, Golden dataset
Observability

Instrumenting a system to capture traces, token usage, latency, errors, and tool calls. It makes production failures diagnosable and supports cost and quality monitoring.

Related: Tracing, Latency
Prompt caching

Reusing computed representations of unchanged prompt prefixes to cut cost and latency on repeated long contexts. It works best when static instructions precede variable content.

Related: Context window, Latency
Rate limit

A cap on requests or tokens per unit time enforced by the provider. Production clients need backoff, retries, and queuing to handle throttling gracefully.

Related: Retry with backoff, Latency
Retry with backoff

Re-sending failed requests after increasing delays, often with jitter, to recover from transient errors without overwhelming a struggling service. Non-retryable errors should fail fast.

Related: Rate limit, Idempotency
Idempotency

Designing operations so repeating the same request produces the same result. It matters when retries or agent loops might execute a side effect more than once.

Related: Retry with backoff, Agentic loop
Streaming

Delivering model output incrementally as it is generated, improving perceived responsiveness. It complicates validation and error handling because content arrives in pieces.

Related: Latency, Structured output
Latency

The time between a request and a usable response. It is shaped by model choice, prompt size, output length, and tool round-trips, and is often managed with streaming.

Related: Streaming, Prompt caching
Model routing

Sending each request to the cheapest or fastest model that meets its quality bar. It balances cost against capability, with harder tasks escalated to stronger models.

Related: Latency, Evaluation harness
Fallback

A predefined alternative path when a primary model call fails, such as a secondary model, cached answer, or graceful error message. It keeps user-facing features available during outages.

Related: Retry with backoff, Model routing

Claude Certified Architect - Foundations study plan

8 weeks · About 7 hours per week

Week 1

Prompt Engineering and System Prompt Design

  • Write 10 system prompts that each isolate a distinct behavior (role, tone, output format, refusal policy, tool routing)

  • Rewrite 5 weak prompts to add explicit constraints, examples, and output schemas, then diff before/after responses

  • Classify prompt failures into ambiguity, missing constraint, conflicting instruction, and context-starvation buckets

  • Draft a reusable system-prompt template with sections for role, task, constraints, format, and escalation

Your checkpoint

A 10-prompt portfolio with a one-page failure-taxonomy table mapping each prompt to its dominant failure mode

Week 2

Prompt Engineering and System Prompt Design & Claude Fundamentals and Model Capabilities

  • Compare model tiers on the same prompt set and record latency, cost, and quality tradeoffs

  • Build few-shot examples for 3 tasks and measure accuracy change versus zero-shot

  • Test chain-of-thought versus direct-answer prompting on 5 reasoning tasks and note where each wins

  • Document how context window size and output token limits constrain prompt design

Your checkpoint

A benchmarking spreadsheet with 5 prompts x 3 model configs, including pass/fail, latency, and token cost

Week 3

Tool Use and Agentic Workflows

  • Define 4 tool schemas with precise descriptions, parameter types, and required fields

  • Implement a single-turn tool-use loop that parses tool calls and returns results to the model

  • Add error handling for malformed arguments, unknown tools, and empty results

  • Trace one multi-step agent run and identify where the loop could stall or repeat

Your checkpoint

A working tool-use demo with 4 tools and a logged trace of a successful multi-step run plus 3 handled error cases

Week 4

Tool Use and Agentic Workflows

  • Design an agent with explicit stop conditions, max iterations, and a fallback path

  • Separate deterministic orchestration logic from model-decided routing

  • Write tool descriptions that disambiguate near-overlapping tools

  • Add structured logging for every tool call with inputs, outputs, and latency

Your checkpoint

An agent design doc plus runnable prototype showing stop conditions triggered in at least 2 test scenarios

Week 5

Context Management and Retrieval-Augmented Generation

  • Chunk a document set three ways and compare retrieval relevance on 10 queries

  • Implement a retrieval pipeline that injects only relevant chunks and cites sources

  • Test how the system behaves when retrieval returns nothing relevant

  • Measure how context position affects answer quality by placing key chunks first versus last

Your checkpoint

A RAG prototype with a 10-query relevance scorecard comparing at least 2 chunking strategies

Week 6

Safety, Guardrails, and Responsible AI & Context Management and Retrieval-Augmented Generation

  • Write guardrails for prompt injection, unsafe output, and data leakage, then attempt to bypass each

  • Add input validation and output filtering layers around model calls

  • Document a responsible-AI checklist covering transparency, refusal behavior, and escalation

  • Test retrieval poisoning by inserting a misleading document and verifying the system flags it

Your checkpoint

A red-team report listing 8 attack attempts, which guardrail caught each, and residual risks

Week 7

Evaluation, Testing, and Observability

  • Build an eval set of 25 labeled cases spanning normal, edge, and adversarial inputs

  • Automate scoring with both exact-match and rubric-based grading

  • Instrument logging and tracing to capture inputs, outputs, latency, and token usage

  • Define regression thresholds that would block a prompt or model change

Your checkpoint

An automated eval harness with 25 cases, a scoring report, and documented regression thresholds

Week 8

Deployment, Integration, and Production Operations & Evaluation, Testing, and Observability

  • Configure environment-based secrets, retries with backoff, and timeout handling

  • Design a deployment checklist covering versioning, rollback, and canary rollout

  • Run the full eval suite as a pre-deploy gate and record results

  • Review all 7 domains and close gaps using missed eval cases as study targets

Your checkpoint

A production readiness checklist plus a full eval run report used as the deploy gate

Milestones

Week 2: Prompt portfolio complete and model-tier benchmark recorded with cost and latency data

Week 4: Tool-use demo and agent prototype both run end to end with logged traces and handled errors

Week 6: RAG prototype scores at least 7/10 relevant on the query scorecard and red-team report is complete

Week 8: Eval harness passes all 25 cases at or above the defined thresholds and deployment checklist is signed off

Claude Certified Architect - Foundations exam day

Your checklist

Government-issued photo ID

Bring the exact ID type and name match your registration requires. Confirm acceptable forms and name-match rules with the provider in advance; if your ID name differs from your registration, resolve it before test day.

Appointment confirmation and login details

Have your confirmation email, registration number, and account credentials accessible. For remote proctoring, verify the login link and system check beforehand.

Testing environment requirements confirmed

For remote delivery, confirm the provider's rules on webcam, microphone, room privacy, desk clearance, and permitted items. Requirements vary by provider, so confirm specifics with them.

Permitted materials list reviewed

Check whether any notes, scratch paper, or calculator are allowed. Assume nothing is permitted unless the provider states it explicitly.

Scratch materials per provider policy

If scratch paper or an on-screen notepad is allowed, confirm how it is supplied and whether it must be destroyed or returned afterward.

Water and a light snack for before/after

Keep these outside the testing area unless the provider permits them at your desk.

Charged device and stable internet (remote)

Use a wired or reliable connection, close background apps, and have a backup plan such as a phone hotspot if the provider allows it.

Quiet, well-lit, private room

Remove papers, phones, watches, and second screens from the desk and surrounding area per provider rules.

Restroom and logistics plan

Arrive or log in early. Know the check-in window and late-arrival policy; confirm with the provider rather than assuming a grace period.

Scoring and results expectations understood

Know the passing standard, whether results are immediate or delayed, and how a retake would work. Confirm these with the provider.

Emergency contact / provider support number

Save the provider's support contact so you can resolve technical or check-in issues quickly.

Backup time buffer

Block extra time around the appointment for check-in, technical setup, and a decompression period afterward.

Plan the day

72 hours before

Confirm appointment details, ID requirements, delivery mode (in person or remote), and permitted-items rules with the provider. Run any required system or equipment check. Resolve name or scheduling discrepancies now, not on test day.

72 hours before

Do a final review pass focused on weak areas and architecture decision patterns rather than new topics. Stop heavy study at least a day before.

48 hours before

Do one timed practice set under realistic conditions, then review only the reasoning behind missed items. Avoid cramming new material.

24 hours before

Prepare ID, confirmation, and any permitted materials in one place. Confirm the start time in your local time zone. Charge devices and test your connection if remote. Get a full night's sleep.

Morning of

Eat a normal meal, avoid last-minute cramming, and log in or arrive with generous buffer. Close unnecessary apps and silence notifications. Re-read the provider's check-in instructions once.

During the test

Read each question fully and note what is actually being asked (best, most likely, first step). Answer confident items first, flag uncertain ones, and return to them. Watch the clock but avoid panic-guessing near the end. Use only permitted materials and follow proctor instructions.

After the test

Note which topics felt weak while memory is fresh, in case a retake is needed. Confirm with the provider how and when results are released.

Rules to check

  • Present valid, unexpired government-issued photo ID matching your registration; confirm accepted ID types and name-match requirements with the provider before test day.

  • Follow all proctor instructions. Do not communicate with anyone, use unauthorized devices or materials, or leave the testing area without permission; violations can void your result.

  • For remote testing, keep your face and workspace visible per provider rules and comply with any room scan or desk-clearance requirement.

  • Breaks, if any, are governed by provider policy; confirm in advance whether scheduled or unscheduled breaks are allowed and whether the clock stops.

  • Scoring, passing standards, and result timing are set by the provider; confirm these rather than relying on secondhand information.

  • Retake and rescheduling policies vary; confirm eligibility windows, fees, and deadlines with the provider before you need them.

  • Report technical problems to the proctor or support immediately and ask for them to be documented; do not attempt to fix issues in ways that breach testing rules.

  • Keep all exam content confidential. Do not reproduce, share, or discuss specific questions after the test.

Claude Certified Architect - Foundations FAQ

How many Claude Certified Architect - Foundations practice questions are available?

The practice set has 200 Claude Certified Architect - Foundations questions. Each one includes the choices, the correct answer, and an explanation, and the set is free to read with no signup.

How many Claude Certified Architect - Foundations flashcards are available?

The flashcard set has 32 Claude Certified Architect - Foundations cards. Each card shows the term and its definition.

Which topics does the Claude Certified Architect - Foundations study material cover?

The Claude Certified Architect - Foundations study material has 7 sections, including Claude Fundamentals and Model Capabilities, Prompt Engineering and System Prompt Design, and Tool Use and Agentic Workflows.

What format facts does the catalog list for the Claude Certified Architect - Foundations exam?

The catalog lists these item types: multiple choice for the Claude Certified Architect - Foundations exam.