Databricks GenAI Engineer Practice Test

Databricks Certified Generative AI Engineer Associate

Databricks GenAI Engineer overview

  • 200 practice questions
  • 32 flashcards
  • 5 study guide topics
  • 40 key terms

What you'll study

  • Design and build generative AI applications20%
  • Prepare data for retrieval augmented generation (RAG)20%
  • Develop, tune and evaluate generative AI applications20%
  • Deploy and monitor generative AI applications20%
  • Governance, security and responsible AI for generative AI applications20%

The Databricks Certified Generative AI Engineer Associate is a professional credential from Databricks that validates a practitioner's ability to design, build, evaluate, deploy, and govern generative AI applications on the Databricks Data + AI Platform. It is aimed at data engineers, ML engineers, solution architects, and application developers who already work with Spark, Delta Lake, and Unity Catalog and who now need to ship LLM-backed systems - retrieval-augmented generation pipelines, agentic workflows, and evaluation harnesses - rather than train models from scratch. Coverage is generally understood to span five areas: designing and building GenAI applications, preparing data for RAG, developing/tuning/evaluating applications, deployment and monitoring, and governance, security and responsible AI. Databricks periodically revises its blueprints, so treat any figure you see for question count, time limit, passing score, fee, or domain weighting as provisional until confirmed against the official exam guide. Preparation should pair hands-on platform work with the Databricks Academy learning path: build an end-to-end RAG app using Vector Search and Foundation Model APIs, instrument it with MLflow evaluation and tracing, serve it behind an endpoint, and lock it down with Unity Catalog permissions. The GenAI partner accelerators described in Databricks' blog are ecosystem products, not blueprint material. This page is independent study content and is not affiliated with or endorsed by Databricks.

Common mistakes

Selects a serving option by name recognition alone: points the app at a pay-per-token Foundation Model API endpoint, a provisioned throughput endpoint, or an external model endpoint without verifying which one supports the chosen model, request schema, context length, or throughput - then ships code with a payload format copied from a different provider and no rate-limit handling.

Candidates memorise 'use Foundation Model APIs' as a slogan and never learn that endpoint type determines supported models (e.g. whether a fine-tuned or non-Databricks-hosted model is served), quota behaviour, and JSON contract. The first 429 or 400 schema error appears only at demo time.

What to do instead

For each design question, write down the requirement first (latency SLA, throughput, custom/fine-tuned weights, data residency), then map it to pay-per-token, provisioned throughput, or external model serving. Practice the actual request/response JSON for each, including retry-with-backoff and fallback-endpoint logic, and check the supported-model list before committing to an endpoint name.

Stuffs a fixed top-k of retrieved chunks (or the full conversation history) into every prompt with no token budgeting, no chunk-size cap, and no structured output contract, so prompts silently exceed the model context window, get truncated, or return free text the downstream code cannot parse.

Prototyping with one small document makes context limits invisible, and candidates treat 'more context = better answers'. Truncation and parse failures look like model quality problems rather than engineering ones.

What to do instead

Size the prompt explicitly: reserve tokens for system prompt, output, and history; rank and truncate chunks to fit; summarise or window long conversations. Define the output contract (JSON schema / function-calling tool definition) and validate the response before downstream use, with a repair or retry path.

Chunks documents by fixed character or paragraph counts with no overlap, no token awareness, no respect for structural boundaries (tables, headers, code blocks), and throws away source metadata such as file path, section, page, and document ID.

Character counts are easy to compute and 'feels like chunking', but embedding models have token limits and answers frequently span a sentence or table that got cut in half. Metadata loss is invisible until you need citations or filtering.

What to do instead

Parse first (proper document parsing for PDFs/tables), chunk on structural boundaries with token-based sizing and a 10-20% overlap, and store metadata (source, section, page, timestamps, access-control tags) alongside each chunk so retrieval can filter and answers can cite.

Builds the Vector Search index once and never manages its lifecycle: no sync triggered after the source Delta table changes, confusion between Delta Sync and Direct Vector Access modes, and re-embedding with a different or upgraded embedding model while queries still use the old query-side model or dimension.

The index looks correct on the demo dataset. Staleness, sync-mode mismatch, and embedding-model drift produce silently wrong retrieval rather than errors, so nobody notices until a user complains about outdated answers.

What to do instead

Decide deliberately between Delta Sync (managed sync of a source table, including continuous or triggered sync) and Direct Vector Access (you push vectors), and set up the sync trigger as part of the pipeline. Treat the embedding model as a versioned contract: pin model and dimensions, record them in the index, and rebuild fully whenever either the model or the chunking strategy changes.

Reaches for fine-tuning as the default fix for 'the model does not know our content', fine-tuning on internal documents instead of retrieving them, and then re-fine-tuning on every document update.

Fine-tuning is the more familiar ML workflow, so it feels like the rigorous answer. The failure surfaces later as stale or hallucinated facts, inability to cite sources, and an unmaintainable retraining cadence.

What to do instead

Route by intent: RAG for dynamic, citable, permission-scoped knowledge; fine-tuning (or a smaller tuned model) for style, format, tone, tool-calling behaviour, or cost/latency reduction on a stable task. State the tradeoff explicitly - update cadence, citation needs, data volume, and per-user access control - when answering design questions.

Declares the application 'evaluated' after reading a handful of outputs, or runs a single LLM-judge with the same model that generated the answer and no rubric, no labelled ground truth for retrieval, and no tracking of prompt or model version between runs.

Manual review is fast for ten examples and judges feel objective. Without ground truth and versioned experiments, quality changes are indistinguishable from noise and regressions ship silently.

What to do instead

Build an evaluation set with expected answers and expected source documents; measure retrieval separately (recall/precision of retrieved chunks) from generation (groundedness, correctness, relevance) using rubric-based judges, including a different judge model from the generator. Run it through the evaluation tooling you would use in production, keep the results in the experiment, and re-run the same set for every prompt, model, or retrieval change.

Deploys a working notebook chain to a model serving endpoint without packaging requirements: relies on notebook-scoped %pip installs or local DBFS paths, hardcodes secrets or API keys, and assumes the endpoint identity can read the backing tables, secret scope, or external model credentials.

The notebook environment already has the libraries, mounted paths, and the developer's permissions, so nothing fails in development. Serving runs in a different container with a different identity and no notebook state.

What to do instead

Declare dependencies and versions in the logged model/chain, use relative or governed paths (Unity Catalog volumes rather than local paths), fetch credentials from a secret scope the endpoint's identity is granted, and test in a staging endpoint with the same permissions as production before promoting.

Monitors only endpoint latency and error rate after launch: no inference table capturing requests and responses, no capture or storage of user feedback, and no quality or drift checks on retrieval and answers.

Latency is the easy metric with a ready dashboard, and quality regressions produce successful HTTP 200s. Without logged payloads and feedback there is no data to detect or debug degradation.

What to do instead

Enable inference tables and payload logging on the endpoint, capture user feedback (thumbs, edits, corrections) tied to a request ID, and monitor both operational metrics and quality metrics - groundedness, refusal rate, retrieval hit rate, and input drift - with alerts routed to an owner and a review loop that feeds failures back into the evaluation set.

Assumes Unity Catalog table permissions automatically govern what the RAG application retrieves: the retriever queries all indexed chunks with one service identity, so any user of the app can receive content from documents they are not entitled to see.

Permissions are set on tables and the index looks 'governed', but the application is the only principal touching the data. Candidates conflate data governance of the store with enforcement inside the retrieval path.

What to do instead

Enforce access control at retrieval time: store document-level ACL/group tags in chunk metadata, resolve the end user's identity and groups, and filter (or pre-filter) the vector search query accordingly before the chunks reach the prompt. Test with two users who have different entitlements.

Logs raw prompts, retrieved chunks, and model responses - including customer PII or regulated data - to inference tables and trace stores without redaction, and ships with no input/output guardrails against prompt injection, jailbreaks, or unsafe content.

Logging everything is the default for debugging and tracing is required for evaluation, so sensitive data flows into tables with broader read access than the source. Guardrails are treated as a later hardening step rather than part of the design.

What to do instead

Classify data before it enters the prompt, mask or redact PII in logged payloads, restrict access to inference and trace tables to the minimum group, set retention, and place guardrails at both input and output - including instruction-injection detection and filtering of retrieved content treated as untrusted input. Document the responsible-AI controls alongside the architecture.

Exam requirements can change. Confirm current details with the exam provider.

Databricks GenAI Engineer practice

200 questions. Each one lists the choices, the correct answer, and an explanation. With JavaScript, this list opens one question at a time. The checker at the top of this page saves progress in this browser when it is available.

  1. Question 1Design and build generative AI applications

    What is the primary purpose of Databricks Foundation Model APIs?

    1. To train new foundation models from scratch on a Databricks cluster
    2. To store embeddings in Unity Catalog as Delta tables
    3. To call hosted foundation models without managing infrastructure Correct answer.
    4. To convert notebooks into scheduled workflows automatically

    Explanation

    Foundation Model APIs expose hosted foundation models as serving endpoints that applications can query through an API. The platform manages the model infrastructure, so teams do not provision or patch model servers themselves. These APIs do not train models from scratch or store embeddings.

  2. Question 2Design and build generative AI applications

    In an OpenAI-compatible chat completions request, which message role supplies instructions that guide the model's behavior across the conversation?

    1. assistant
    2. user
    3. tool
    4. system Correct answer.

    Explanation

    The system role carries high-level instructions such as persona, constraints, and response format. The user role supplies requests, the assistant role represents prior model responses, and the tool role carries tool results.

  3. Question 3Design and build generative AI applications

    What is an embedding primarily used for in a generative AI application?

    1. Translating a SQL query into natural language
    2. Compressing a prompt to reduce token count before generation
    3. Encrypting sensitive text before storing it in Delta Lake
    4. Representing text as a numeric vector for similarity search Correct answer.

    Explanation

    An embedding maps text to a numeric vector so that semantically similar content has similar vector representations. Applications use these vectors for retrieval, clustering, and similarity comparison. Embeddings are not an encryption or prompt-compression format.

  4. Question 4Design and build generative AI applications

    In a retrieval-augmented generation pipeline, what is the correct high-level order of operations?

    1. Embed the query, retrieve documents, then generate an answer Correct answer.
    2. Embed the query, generate an answer, then retrieve documents
    3. Retrieve documents, generate an answer, then embed the query
    4. Generate an answer, retrieve documents, then construct a prompt

    Explanation

    A RAG application first embeds the user query, uses that vector to retrieve relevant documents, and then sends the retrieved context with the query to the model. Generation must happen after retrieval so the model can condition on the retrieved content.

  5. Question 5Design and build generative AI applications

    Which setting most directly reduces randomness in a chat model's responses for a factual question-answering application?

    1. A high top-p value
    2. A larger max_tokens value
    3. A low temperature value Correct answer.
    4. A longer system prompt

    Explanation

    Temperature controls sampling randomness, and lower values make token selection more deterministic. Top-p controls the cumulative probability mass considered during sampling, while max_tokens controls response length. Prompt length may influence behavior but does not directly control sampling randomness.

  6. Question 6Design and build generative AI applications

    What does the max_tokens parameter control in a chat completions request?

    1. The maximum number of input tokens accepted by the model
    2. The maximum number of tokens generated in the response Correct answer.
    3. The maximum number of documents retrieved for RAG
    4. The maximum number of concurrent requests to an endpoint

    Explanation

    max_tokens limits the number of tokens the model may generate in its response. It is an output-length control and can affect cost and latency, but it does not set the input limit or retrieval depth.

  7. Question 7Design and build generative AI applications

    Which Databricks mechanism should an application use to retrieve an API key stored securely for calling an external model?

    1. Write the key to a Delta table in a public schema
    2. Reference the key through a Databricks secret scope Correct answer.
    3. Include the key in the model's system prompt
    4. Hard-code the key in a notebook widget default value

    Explanation

    Databricks secret scopes store credentials and expose them to authorized code at runtime without embedding them in source or prompts. Applications should retrieve secrets through the secrets API or secret-backed environment configuration. Hard-coding or prompting exposes credentials.

  8. Question 8Design and build generative AI applications

    What is the Databricks SQL ai_query function commonly used for in a generative AI application?

    1. Creating a vector search index from a Delta table
    2. Scheduling a notebook to retrain a custom model
    3. Encrypting columns that contain personally identifiable information
    4. Calling a model endpoint from SQL for each row Correct answer.

    Explanation

    ai_query invokes a supported model serving endpoint from SQL and returns the model response for each row. This is useful for batch inference and large-scale enrichment directly in a query. It does not create indexes or encrypt data.

  9. Question 9Design and build generative AI applications

    A Delta table contains document text and an embedding column. Which Databricks Vector Search capability keeps a vector index updated as new rows are inserted into the table?

    1. A Parquet-backed index that reads files on every query
    2. A manually rebuilt index triggered by a SQL alert
    3. A Delta Sync index that syncs from the source table Correct answer.
    4. A temporary view that recomputes embeddings at query time

    Explanation

    A Delta Sync vector index is connected to a source Delta table and synchronizes changes so index content stays current. It avoids manual rebuilds for routine inserts and updates. Other storage or view constructs do not provide the managed Vector Search synchronization behavior.

  10. Question 10Design and build generative AI applications

    When splitting source documents into chunks for a RAG application, why is overlap between adjacent chunks often used?

    1. It reduces the number of embeddings that must be stored
    2. It guarantees that every query retrieves the same chunks
    3. It helps preserve context that falls near a chunk boundary Correct answer.
    4. It removes the need for a vector search index

    Explanation

    Overlap repeats some text across neighboring chunks so that sentences or ideas crossing a boundary are not lost from retrieval. This improves the chance that a relevant passage is fully represented in at least one chunk. Overlap increases rather than reduces stored content.

  11. Question 11Design and build generative AI applications

    In a LangChain application running on Databricks, which component is designed to call a Databricks Foundation Model API chat endpoint?

    1. The ChatDatabricks chat model class Correct answer.
    2. A Unity Catalog volume
    3. A Spark MLlib transformer
    4. A Delta Live Tables pipeline

    Explanation

    ChatDatabricks is the LangChain integration for invoking Databricks chat model serving endpoints. It lets LangChain chains and agents use a Databricks-hosted chat model through the standard chat model interface. Delta Live Tables, volumes, and MLlib transformers serve different purposes.

  12. Question 12Design and build generative AI applications

    What is the main benefit of enabling MLflow tracing for a retrieval-augmented generation chain?

    1. It increases the model endpoint's provisioned throughput
    2. It records retrieval and model-call steps for debugging Correct answer.
    3. It automatically fine-tunes the embedding model on user queries
    4. It replaces the need to evaluate answer quality

    Explanation

    MLflow tracing captures spans for steps such as document retrieval, prompt construction, and model invocation, including inputs, outputs, and timing. These traces help diagnose failures and analyze latency or quality issues. Tracing does not tune models or replace evaluation.

  13. Question 13Design and build generative AI applications

    An MLflow evaluation uses an LLM judge to score answer correctness. Which input is needed to compare generated answers with known correct answers?

    1. A vector search endpoint with at least two indexes
    2. An evaluation dataset that includes ground-truth responses Correct answer.
    3. A custom Docker image for the model serving endpoint
    4. A larger max_tokens value for the judge model

    Explanation

    Correctness comparisons require reference answers so the judge or metric has a target to compare against. The evaluation dataset supplies inputs and expected outputs. Larger output limits or additional indexes do not provide ground truth.

  14. Question 14Design and build generative AI applications

    You have a Python chain that calls a foundation model and post-processes the response. Which approach deploys it as a Databricks model serving endpoint?

    1. Wrap the chain in an MLflow pyfunc model and serve that model Correct answer.
    2. Register the chain as a Unity Catalog table and query it with SQL
    3. Attach the chain to a notebook and share the notebook URL
    4. Convert the chain into a Delta Live Tables expectation

    Explanation

    An MLflow pyfunc model packages Python inference logic with its dependencies so Databricks model serving can deploy it behind an endpoint. The endpoint handles request routing and scaling. Tables, pipeline expectations, and notebook sharing do not create model serving endpoints.

  15. Question 15Design and build generative AI applications

    A team needs predictable latency and reserved capacity for a high-volume production generative AI application. Which endpoint configuration is most appropriate?

    1. Provisioned throughput on a serving endpoint Correct answer.
    2. A SQL warehouse set to serverless
    3. Pay-per-token Foundation Model APIs
    4. A single-node interactive cluster

    Explanation

    Provisioned throughput reserves model serving capacity for an endpoint, which is appropriate when latency and volume must be predictable. Pay-per-token access is convenient but can be subject to shared capacity and rate limits. SQL warehouses and interactive clusters are not model serving endpoint capacity options.

  16. Question 16Design and build generative AI applications

    What role does AI Gateway play for generative AI applications on Databricks?

    1. It converts vector indexes into relational tables
    2. It trains custom foundation models from Delta tables
    3. It replaces MLflow for experiment tracking
    4. It adds rate limits and usage tracking to endpoints Correct answer.

    Explanation

    AI Gateway provides governance and monitoring features for model endpoints, including rate limiting and usage tracking. It helps teams control access and observe generative AI traffic. It does not train models, convert indexes, or replace experiment tracking.

  17. Question 17Design and build generative AI applications

    An application queries a Databricks Vector Search index. Which Unity Catalog object typically controls whether a user can query the index?

    1. The vector search index's Unity Catalog privileges Correct answer.
    2. The cluster's Spark configuration
    3. The Delta table's retention period
    4. The notebook's default language

    Explanation

    Vector Search indexes are Unity Catalog securables, so access is governed through privileges on the index and related objects. A user typically needs permission to query the index and may also need access to the underlying source data. Cluster configuration and notebook settings do not grant index access.

  18. Question 18Design and build generative AI applications

    What is a Databricks external model endpoint used for?

    1. Serving a model that has no registered MLflow artifact
    2. Running SQL queries against a serverless warehouse
    3. Calling a model hosted outside Databricks through a governed endpoint Correct answer.
    4. Storing embeddings inside a Unity Catalog volume

    Explanation

    External model endpoints let Databricks applications call supported third-party or externally hosted models while applying Unity Catalog governance and endpoint controls. They proxy requests to the external provider rather than hosting the model weights. They are not a storage or SQL execution feature.

  19. Question 19Design and build generative AI applications

    In a tool-calling workflow, what does the application do after the model returns a tool call request?

    1. It retrains the foundation model with the tool definition
    2. It executes the requested tool and sends the result back to the model Correct answer.
    3. It converts the tool call into a vector search index
    4. It ignores the request and returns the raw tool name to the user

    Explanation

    The application is responsible for executing the requested tool and appending the tool result to the conversation so the model can produce a final answer. The model selects a tool and arguments but does not execute external actions itself. This separation lets the application validate and control tool use.

  20. Question 20Design and build generative AI applications

    Which technique best enforces a specific JSON schema for a model's response?

    1. Adding a random suffix to the prompt for each request
    2. Increasing the temperature to encourage format variation
    3. Removing the system message from the conversation
    4. Using response-format constraints with schema validation Correct answer.

    Explanation

    Response-format or structured-output constraints instruct the model to produce JSON that matches a schema, and application-side validation catches any remaining mismatch. This combination is more reliable than prompt-only formatting instructions. Random variation and removing instructions make format compliance less likely.

  21. Question 21Design and build generative AI applications

    A RAG application indexes documents with one embedding model but embeds user queries with a different embedding model. What is the most likely effect?

    1. Similarity scores become unreliable because the vectors are incompatible Correct answer.
    2. The foundation model increases its context window to compensate
    3. Retrieval improves because the two models provide complementary views
    4. The vector index automatically retrains the query model

    Explanation

    Embedding models define their own vector spaces, so vectors from different models are not directly comparable. Query and document embeddings should come from the same model and compatible version. Mixing models usually degrades retrieval rather than improving it.

  22. Question 22Design and build generative AI applications

    An application places untrusted user text directly into the system prompt of a RAG chatbot. Which mitigation most directly reduces prompt injection risk?

    1. Keeping user content in a separate message and validating model/tool outputs Correct answer.
    2. Storing the prompt in a Delta table instead of a notebook
    3. Increasing the model's temperature for creative responses
    4. Using a larger embedding dimension for document chunks

    Explanation

    Untrusted content should be separated from trusted system instructions and clearly delimited as data. Applications should also validate model outputs and tool calls before acting on them. Prompt injection defenses combine instruction isolation, least-privilege tools, and output checks.

  23. Question 23Design and build generative AI applications

    A multi-turn chat application is approaching the model's context window limit. Which approach best preserves useful conversation history while controlling token usage?

    1. Delete all prior turns whenever the limit is reached
    2. Summarize older turns and keep recent turns plus retrieved context Correct answer.
    3. Append every prior turn without modification until the request fails
    4. Increase max_tokens so that the input history is stored in the response

    Explanation

    Summarizing older conversation history retains key facts while reducing token count, and keeping recent turns preserves local coherence. Relevant retrieved context can be included separately for the current request. Simply appending all history or deleting everything sacrifices reliability or continuity.

  24. Question 24Design and build generative AI applications

    A vector-only retriever misses documents containing exact product codes that are semantically similar to many other descriptions. Which enhancement is most appropriate?

    1. Combine vector and keyword search, then rerank results Correct answer.
    2. Remove metadata filters so all documents are always considered
    3. Increase the chunk size until each chunk contains the entire corpus
    4. Lower the embedding dimension to make codes easier to match

    Explanation

    Hybrid search adds keyword matching for exact terms such as product codes while retaining semantic vector retrieval. Reranking the merged candidates improves the final ordering. Neither reducing dimensions nor removing filters addresses exact-token matching.

  25. Question 25Design and build generative AI applications

    When evaluating a RAG retriever, what does recall@k measure?

    1. The share of relevant documents found in the top k results Correct answer.
    2. The model's ability to generate the exact reference answer
    3. The fraction of retrieved documents that are relevant to the query
    4. The average number of tokens in the top k retrieved chunks

    Explanation

    Recall@k measures how many of all relevant documents are included in the top k retrieved results. Precision, by contrast, measures how many retrieved documents are relevant. Token counts and answer generation quality are separate metrics.

  26. Question 26Design and build generative AI applications

    In a chat completions request, what does the top_p (nucleus sampling) parameter control?

    1. It sets the maximum number of tokens the model is allowed to generate.
    2. It controls how many alternative completions are returned for each request.
    3. It limits sampling to the smallest token set whose cumulative probability reaches p. Correct answer.
    4. It scales a penalty applied to tokens that already appear in the text.

    Explanation

    top_p is a sampling control: the model sorts next-token probabilities and keeps only the smallest group whose cumulative probability reaches p, then samples from that group. Lowering p narrows the candidate pool and makes output more focused. It is independent of length limits and of how many completions are requested.

  27. Question 27Design and build generative AI applications

    Why would you include a few input-output examples in a prompt before the real user request?

    1. To remove the need for a system message in the request.
    2. To show the model the desired input-output pattern so it imitates it. Correct answer.
    3. To permanently update the model weights with task-specific knowledge.
    4. To lower the number of billed tokens by pre-filling part of the response.

    Explanation

    Examples supplied inside the prompt act as in-context demonstrations: the model infers the expected format, tone, and reasoning pattern from them and applies it to the new input. This conditioning happens only for that request. Nothing about the model's parameters changes.

  28. Question 28Design and build generative AI applications

    What is the main benefit of requesting a streamed response from a chat model in an interactive application?

    1. It lowers the total cost because partially streamed responses are billed at a reduced rate.
    2. It lets the client render partial output as tokens arrive rather than waiting. Correct answer.
    3. It extends the context window by sending the prompt incrementally.
    4. It guarantees the model returns a response that matches the requested schema.

    Explanation

    With streaming, the endpoint emits incremental chunks as tokens are produced, so time-to-first-token becomes the user-visible latency instead of the full generation time. Total tokens, total cost, and the context window are unaffected. Schema conformance must still be enforced by prompt design or structured-output settings.

  29. Question 29Design and build generative AI applications

    In a retrieval-augmented generation application, why are retrieved passages inserted into the prompt?

    1. They are used to fine-tune the model's weights at request time.
    2. They are recorded for billing reconciliation only.
    3. They remove the need for an embedding model during retrieval.
    4. They give the model source text to condition its answer on. Correct answer.

    Explanation

    RAG supplies grounding context: the retrieved passages appear in the prompt so the answer can be derived from that source content instead of the model's parametric memory. This is what allows answers to reflect current or private documents. The model itself is unchanged by the request.

  30. Question 30Design and build generative AI applications

    A knowledge base consists of long Markdown documents with clear headings, tables, and code blocks. Which chunking approach best preserves meaning for retrieval?

    1. Keep every document as one chunk unless it exceeds the model context window.
    2. Split at every line break so each chunk is as small as possible.
    3. Split each document into fixed-size token windows with no awareness of headings.
    4. Split on structural boundaries such as headings and keep related blocks together. Correct answer.

    Explanation

    Structure-aware splitting keeps a heading with the content it governs, so each embedded chunk is self-contained and topically coherent. That improves both the embedding's representativeness and the usefulness of the text handed to the model. Fixed windows and per-line splits cut across semantic units, and whole-document chunks dilute the embedding.

  31. Question 31Design and build generative AI applications

    A retriever returns 50 candidate passages for each question. Why add a cross-encoder reranking step before calling the model?

    1. It replaces the embedding model with a keyword-only index.
    2. It permanently retrains the embedding model using the user query.
    3. It expands the number of documents stored in the vector index.
    4. It re-scores candidate passages jointly with the query to raise precision. Correct answer.

    Explanation

    A cross-encoder reads the query and each candidate together, which captures relevance signals that independent embeddings miss, and it reorders the shortlist so only the strongest passages fill the prompt. It runs on a small candidate set, so the extra cost is bounded. It is a scoring step, not an indexing or training step.

  32. Question 32Design and build generative AI applications

    A generative AI application intermittently receives HTTP 429 responses from its model endpoint during traffic spikes. Which client-side handling is most appropriate?

    1. Retry with exponential backoff and jitter, honoring any retry-after hint. Correct answer.
    2. Swallow the failures and return an empty answer to the user.
    3. Lower the temperature to zero so that fewer requests need to be sent.
    4. Resend the identical request in a tight loop until one of the attempts finally succeeds.

    Explanation

    HTTP 429 signals rate limiting rather than a malformed request, so the correct response is to slow down and retry. Exponential backoff with jitter spreads retries out and avoids synchronized retry storms, while honoring retry-after respects the server's stated wait. Persistent failures should eventually surface rather than be hidden.

  33. Question 33Design and build generative AI applications

    A request handler issues 30 independent model calls and total latency is too high because they run one after another. Which change most directly reduces wall-clock latency?

    1. Issue the calls concurrently with an async client and gather the results. Correct answer.
    2. Cache the response for one prompt and reuse it for the others.
    3. Increase max_tokens on every call so that each individual request finishes sooner.
    4. Merge all thirty prompts into a single unstructured prompt.

    Explanation

    Independent calls that do not depend on each other's output can overlap in time, so the total duration approaches the slowest single call rather than the sum. An async client with a gather-style await does exactly that, subject to endpoint rate limits. Combining unrelated prompts degrades output quality and does not preserve per-item structure.

  34. Question 34Design and build generative AI applications

    Users ask the same support questions with slightly different wording, and the application repeatedly pays to regenerate near-identical answers. Which approach reduces cost and latency while keeping answers accurate?

    1. Disable retrieval for repeat users so that responses become shorter and cheaper to produce.
    2. Cache answers by embedding similarity so near-duplicate questions reuse a prior response. Correct answer.
    3. Pin the model weights in the driver's memory between requests.
    4. Raise temperature to 1.0 so the model produces more diverse answers.

    Explanation

    Semantic caching stores responses keyed by an embedding of the question, so a new question that is close in vector space to a cached one can reuse that answer after a similarity threshold is met. This converts repeated generation into a lookup. Accuracy is preserved only if the threshold is tuned and cached entries are invalidated when the underlying source data changes.

  35. Question 35Design and build generative AI applications

    A Vector Search index stores a department field alongside each document's embedding. An application must never surface content from departments a user cannot access. Which mechanism enforces this at query time?

    1. Pass a metadata filter on the department field with the vector search query. Correct answer.
    2. Increase k and let the model discard any documents the user is not permitted to see.
    3. Set top_k to one so only a single document can ever be returned.
    4. Rebuild the index nightly with an embedding model per department.

    Explanation

    Metadata (pre-filtering) restricts the candidate set inside the vector search query, so disallowed documents are never retrieved and cannot leak into the prompt. Relying on the model to discard them exposes restricted text to the model and to logs. Separate indexes per department add cost without being necessary.

  36. Question 36Design and build generative AI applications

    A 300-page report must be summarized, but it exceeds the model's context window. Which approach is most appropriate?

    1. Truncate the report to its first N tokens and summarize only that portion.
    2. Summarize each section, then summarize the combined section summaries. Correct answer.
    3. Raise the temperature until the model accepts the longer input.
    4. Ask the model to internally expand its context window for this request.

    Explanation

    A map-reduce pattern summarizes chunks independently and then summarizes those intermediate summaries, so the entire document is covered using requests that each fit the window. Truncation silently drops content, and sampling settings cannot change the accepted input length. The context window is a fixed model property.

  37. Question 37Design and build generative AI applications

    A support assistant must answer from a product catalog whose prices and availability change every day. Which design best keeps the answers current?

    1. Fine-tune the model nightly on the full product catalog.
    2. Retrieve current catalog entries at query time and put them in the prompt. Correct answer.
    3. Raise the temperature so the model recalls recently added items.
    4. Increase max_tokens substantially so the whole catalog fits inside the response.

    Explanation

    Retrieval keeps volatile facts in an external store that is updated as the catalog changes, and only the relevant entries are injected into each prompt. Fine-tuning bakes facts into weights that immediately begin to age and cannot be updated cheaply. Generation parameters do not affect factual currency.

  38. Question 38Design and build generative AI applications

    Several engineers edit the prompts used by a production generative AI application, and regressions appear with no way to tell which prompt version produced a given answer. Which change best addresses this?

    1. Store prompts as versioned prompt objects in a prompt registry that the app loads at runtime. Correct answer.
    2. Keep the prompts in an untracked text file on each developer's laptop.
    3. Bake the prompts into the container image and rebuild for each wording tweak.
    4. Keep the prompts in a shared notebook cell that engineers edit in place.

    Explanation

    A prompt registry gives each prompt a versioned, tracked object in a governed store, so an application can resolve a specific version at runtime and link that version to evaluation results and logged traces. That traceability is what makes regressions attributable. Editing prompts in place or on local disks destroys the link between prompt version and output.

  39. Question 39Design and build generative AI applications

    A routing component must always invoke one of the declared tools instead of replying in plain text. Which request setting enforces that?

    1. Set stream to true so tool calls are delivered as they are produced.
    2. Set temperature to zero so the model always picks the same tool.
    3. Set response_format to json_object so the model can only emit valid JSON text.
    4. Set tool_choice to require, or to the specific function that must run. Correct answer.

    Explanation

    tool_choice controls whether the model may answer in text or must emit a tool call; the require value (or naming a specific function) removes the plain-text option. response_format constrains the shape of emitted text but does not force a tool invocation. Deterministic sampling and streaming change behavior at the margins, not the tool contract.

  40. Question 40Design and build generative AI applications

    A team wants every request and response payload handled by a served generative AI endpoint captured in Delta tables for monitoring and later quality analysis. Which capability provides this?

    1. Unity Catalog grants applied to the serving endpoint.
    2. Inference tables enabled on the model serving endpoint. Correct answer.
    3. Autoscaling with scale-to-zero on the serving endpoint.
    4. Provisioned throughput on the serving endpoint.

    Explanation

    Inference tables capture the request and response payloads of a served model endpoint into a Delta table, which can then be queried for monitoring, drift checks, and offline evaluation. Provisioned throughput and autoscaling change capacity behavior, and grants control who may invoke the endpoint. Neither of those records payload content.

  41. Question 41Prepare data for retrieval augmented generation (RAG)

    Why is a long document split into chunks before embeddings are generated for a RAG application?

    1. To reduce the storage cost of the underlying Delta table
    2. To avoid the need for a vector index
    3. To respect token limits and sharpen retrieval precision Correct answer.
    4. To shorten the embedding vector dimension

    Explanation

    Embedding models accept a bounded number of tokens, so long documents must be split to fit. Smaller chunks also let retrieval return narrowly relevant passages rather than an entire document. Chunking does not change the vector dimension or remove the need for an index.

  42. Question 42Prepare data for retrieval augmented generation (RAG)

    Which Databricks component stores embeddings and serves similarity search queries for a RAG application?

    1. A Mosaic AI Vector Search index Correct answer.
    2. A Delta Live Tables pipeline
    3. A Unity Catalog volume
    4. A Model Serving endpoint for the embedding model

    Explanation

    Mosaic AI Vector Search builds a vector index over embeddings and answers nearest-neighbor queries; the source Delta table supplies the rows. Delta Live Tables and volumes handle transformation and file storage, and Model Serving hosts models rather than vector indexes.

  43. Question 43Prepare data for retrieval augmented generation (RAG)

    For a RAG application that must cite sources, what should be stored in the Delta table alongside each chunk's embedding?

    1. Only the source document identifier, with no chunk text stored
    2. Only the embedding vector and its declared dimension
    3. The chunk text plus source metadata such as file path and page number Correct answer.
    4. The weights of the embedding model so it can be retrained at query time

    Explanation

    Returning the chunk text together with provenance metadata lets the application cite and display the passage the model used. A vector is not human-readable on its own, and a bare document identifier cannot pinpoint which passage was retrieved.

  44. Question 44Prepare data for retrieval augmented generation (RAG)

    Which Delta Lake feature lets a pipeline process only rows inserted or updated since the previous run when refreshing embeddings?

    1. Time travel to previous table versions
    2. OPTIMIZE with file compaction
    3. Change Data Feed Correct answer.
    4. Z-ordering on the ingestion timestamp column

    Explanation

    Change Data Feed exposes row-level inserts, updates, and deletes so a downstream job can read only what changed since the last processed version. That avoids paying to re-embed the entire corpus on every run.

  45. Question 45Prepare data for retrieval augmented generation (RAG)

    What does a text embedding model output for an input chunk?

    1. A fixed-length dense vector of floating-point numbers Correct answer.
    2. A ranked list of extracted keywords
    3. A tokenized string that retains the original vocabulary and casing
    4. A compressed representation of the input text

    Explanation

    Text embedding models map input text to a fixed-length dense vector positioned in a learned semantic space. Texts with similar meaning land close together, which is what makes similarity search possible.

  46. Question 46Prepare data for retrieval augmented generation (RAG)

    Which approach extracts text from PDFs most reliably when building a RAG knowledge base on Databricks?

    1. Store only the PDF file path and embed the file name
    2. Convert each PDF page to an image and embed it with a text embedding model
    3. Extract text with a document parser and persist it to a Delta table Correct answer.
    4. Read the PDF bytes directly into the embedding model

    Explanation

    PDFs generally require a parsing step to recover text, tables, and layout, and the parsed text must be persisted before chunking and embedding. Databricks provides document parsing functions for exactly this purpose.

  47. Question 47Prepare data for retrieval augmented generation (RAG)

    What is the effect of configuring chunk overlap so the stride is smaller than the chunk size?

    1. It preserves context that would otherwise be cut at chunk boundaries Correct answer.
    2. It reduces the storage footprint of the embeddings table by deduplicating rows
    3. It guarantees adjacent chunks receive identical embeddings
    4. It removes the need to store chunk metadata

    Explanation

    Overlap repeats a small span of text across neighboring chunks so ideas that straddle a boundary are not lost to retrieval. The cost is duplicated text, which increases storage rather than reducing it.

  48. Question 48Prepare data for retrieval augmented generation (RAG)

    Where should the Delta table holding chunk text and embeddings be registered so that Unity Catalog permissions and lineage apply to it?

    1. In a notebook-scoped temporary view that lasts for the session
    2. In the DBFS root directory
    3. As a Unity Catalog table in a catalog and schema Correct answer.
    4. In a local file system on the driver node

    Explanation

    Registering the table in a Unity Catalog catalog and schema gives fine-grained grants, column-level controls, lineage, and cross-workspace discoverability. Local paths, the DBFS root, and session-scoped views lack that governance.

  49. Question 49Prepare data for retrieval augmented generation (RAG)

    A team must embed 200 million chunks and re-embed them quarterly. Which consideration most strongly favors a batch or self-hosted embedding approach over a per-request foundation model endpoint?

    1. Throughput and cost per token at very large batch volumes Correct answer.
    2. The choice of similarity metric at query time
    3. The need to return results as a JSON document
    4. The number of columns in the Delta table and their data types

    Explanation

    At hundreds of millions of chunks, sustained throughput and per-token price dominate the design decision, and batch or self-hosted jobs amortize that cost better. Response format, table shape, and metric choice do not change embedding economics.

  50. Question 50Prepare data for retrieval augmented generation (RAG)

    What is the typical trade-off when chunk size is reduced from 2000 tokens to 200 tokens?

    1. Retrieval precision improves but each chunk carries less surrounding context Correct answer.
    2. Retrieval precision decreases while context completeness increases
    3. Embedding quality is unaffected and the per-document ingestion cost stays unchanged
    4. The embedding vector dimension shrinks proportionally

    Explanation

    Smaller chunks match specific queries more precisely, but each retrieved chunk carries less surrounding context, which can leave the LLM without enough information. Larger chunks preserve context but dilute the embedding and reduce precision.

  51. Question 51Prepare data for retrieval augmented generation (RAG)

    After new rows are appended to the source Delta table, what must happen for a Vector Search index to return the new chunks?

    1. Nothing, the index reads the source table at query time
    2. The source table must be converted to a view
    3. A sync must be triggered so the index ingests the new rows Correct answer.
    4. The index must be dropped and fully recreated each time

    Explanation

    A Vector Search index maintains its own copy of the data, so appends are invisible until a sync runs. A triggered sync can be scheduled or launched after the write completes.

  52. Question 52Prepare data for retrieval augmented generation (RAG)

    Which Vector Search index configuration keeps up with changes to its source Delta table?

    1. A Delta Sync index, either continuous or triggered Correct answer.
    2. A manually synced snapshot index refreshed on demand
    3. A Parquet-based index built once at creation
    4. A CSV sync index rebuilt by a nightly job

    Explanation

    A Delta Sync index follows the source Delta table's change feed and can update continuously or on a trigger. Snapshot-style builds stay at the state they were created from until someone refreshes them.

  53. Question 53Prepare data for retrieval augmented generation (RAG)

    A RAG application must restrict retrieval to documents the requesting user is allowed to see. Which design supports this?

    1. Encode the requesting user's id into the embedding vector
    2. Increase top_k so that authorized documents are likely to appear in the results
    3. Retrieve every chunk and drop unauthorized ones in the prompt
    4. Store an ACL column per chunk and filter at query time Correct answer.

    Explanation

    Storing row-level access metadata beside each embedding lets the query filter candidates before they are returned, which is both correct and efficient. Unity Catalog grants on the underlying tables then protect the data at rest.

  54. Question 54Prepare data for retrieval augmented generation (RAG)

    When does hybrid search combining vector similarity with keyword matching outperform pure vector search?

    1. Queries that rely on synonyms and paraphrases
    2. Queries phrased as long natural-language questions
    3. Queries containing exact identifiers, product codes, or rare terms Correct answer.
    4. Queries where every document in the corpus has exactly the same length

    Explanation

    Dense vectors blur exact strings such as SKUs, codes, or rare jargon, while lexical matching catches them precisely. Combining both recovers exact matches without sacrificing semantic recall.

  55. Question 55Prepare data for retrieval augmented generation (RAG)

    An embedding model was trained for cosine similarity. Which index configuration is consistent with that?

    1. Truncate every vector to three dimensions before indexing
    2. Normalize vectors to unit length and use cosine similarity Correct answer.
    3. Use Euclidean distance for every index regardless of training
    4. Use Hamming distance on the raw float vectors

    Explanation

    Cosine similarity compares direction, so unit-normalized vectors produce rankings consistent with the training objective, and inner product on normalized vectors is equivalent. The index metric should match what the model was trained with.

  56. Question 56Prepare data for retrieval augmented generation (RAG)

    Why should chunking be based on tokens rather than characters when the embedding model has a hard token limit?

    1. Tokens and characters are always equivalent for plain English text
    2. The token-to-character ratio varies, so character-based sizes can exceed the limit Correct answer.
    3. Character counts always overestimate the token count
    4. Delta tables only store token counts in their metadata

    Explanation

    Token counts depend on the tokenizer and on language and content, so a fixed character budget can silently exceed the model's limit. Counting with the model's own tokenizer keeps each chunk safely inside the window.

  57. Question 57Prepare data for retrieval augmented generation (RAG)

    What distinguishes semantic or structure-aware chunking from fixed-size chunking?

    1. It embeds every sentence separately with no overlap at all
    2. It always produces chunks of exactly 512 tokens
    3. It splits on natural boundaries such as headings, paragraphs, or sentences Correct answer.
    4. It strips all metadata from each chunk before the embedding call

    Explanation

    Structure-aware chunking follows the document's own boundaries so each chunk remains coherent and self-contained. Fixed-size chunking ignores headings and can cut a sentence or table in the middle.

  58. Question 58Prepare data for retrieval augmented generation (RAG)

    Duplicate and near-duplicate chunks in the corpus cause which problem when evaluating retrieval quality?

    1. They reduce the embedding vector dimension at index build time
    2. They make the index read-only after the first sync
    3. They break Unity Catalog lineage tracking for the table
    4. They crowd top-k with near-identical passages and inflate apparent recall Correct answer.

    Explanation

    Near-identical passages occupy several top-k slots, so a retriever looks better at recall than it really is and diverse evidence is pushed out. Deduplicating before indexing keeps both answers and evaluation honest.

  59. Question 59Prepare data for retrieval augmented generation (RAG)

    A pipeline must embed only newly arrived files each night. Which pattern is appropriate?

    1. Fully re-embed the entire corpus every night regardless of changes
    2. Drop and recreate the Vector Search endpoint before each run
    3. Use Auto Loader to land new files, embed only the new chunks, and merge into the embeddings table Correct answer.
    4. Increase the chunk size so fewer API calls are needed

    Explanation

    Auto Loader incrementally discovers newly arrived files, and merging only the new chunk rows avoids paying to re-embed unchanged content. The embeddings table then stays consistent with the corpus without full recomputation.

  60. Question 60Prepare data for retrieval augmented generation (RAG)

    Before embedding, which preprocessing step most improves consistency across a heterogeneous document corpus?

    1. Prepending boilerplate headers to every chunk
    2. Randomly shuffling the word order of each chunk before embedding it
    3. Normalizing whitespace, removing repeated headers and footers, and fixing encoding Correct answer.
    4. Raising the chunk overlap to ninety percent of the chunk size for every document

    Explanation

    Repeated headers, footers, and inconsistent whitespace or encoding add noise that dilutes the embedding and makes unrelated chunks look similar. Cleaning and normalizing first improves retrieval quality without extra cost.

  61. Question 61Prepare data for retrieval augmented generation (RAG)

    A team wants precise matching on small passages but full-section context for the LLM. Which retrieval design achieves both?

    1. Embed each entire document as a single chunk
    2. Embed small child chunks but return the parent section to the LLM Correct answer.
    3. Embed only the section headings and titles
    4. Use a single chunk size of 128 tokens with no overlap between chunks

    Explanation

    Matching on small child chunks keeps the embedding focused, while returning the parent section restores the surrounding context the model needs to answer. This small-to-big pattern separates the unit of search from the unit of generation.

  62. Question 62Prepare data for retrieval augmented generation (RAG)

    An embedding model is upgraded to a version that produces vectors of a different dimension. What must be done?

    1. Re-embed the corpus and rebuild or re-sync an index for the new dimension Correct answer.
    2. Pad the shorter vectors with zeros to match the previous length
    3. Switch the similarity metric from cosine to Euclidean
    4. Nothing; dimensions are reconciled automatically at query time

    Explanation

    A vector index is built for one fixed dimension, so a new model requires re-embedding the corpus and creating a new index or column. Vectors from different models are not comparable even if their lengths were forced to match.

  63. Question 63Prepare data for retrieval augmented generation (RAG)

    A corpus contains scanned PDFs, complex tables, and figures. Which ingestion plan best preserves retrievable content?

    1. Skip scanned pages and embed only the existing text layer
    2. Embed the raw PDF bytes directly with the text embedding model
    3. OCR scanned pages, extract tables as structured text, and caption figures before chunking Correct answer.
    4. Convert every figure to base64 and embed that string as text

    Explanation

    Scanned pages need OCR, tables need structure-preserving extraction, and figures need descriptive text before they can be retrieved. Without these steps large parts of the corpus stay invisible to similarity search.

  64. Question 64Prepare data for retrieval augmented generation (RAG)

    Which set of metrics best evaluates the retrieval stage of a RAG pipeline separately from the generation stage?

    1. Recall@k, precision@k, and mean reciprocal rank against labeled relevant chunks Correct answer.
    2. End-to-end latency and cost per query
    3. The total number of documents in the corpus and their average length
    4. BLEU and ROUGE scores computed on the final generated answer

    Explanation

    Retrieval quality is measured against labeled relevant chunks using rank-aware metrics, which isolates the retriever from the generator. Generation metrics and cost figures cannot tell you whether the right context was found.

  65. Question 65Prepare data for retrieval augmented generation (RAG)

    Which prerequisite must the source table satisfy for a Vector Search Delta Sync index to reflect inserts, updates, and deletes?

    1. The source must be exposed as a view over the Delta table
    2. The table must contain fewer than one million rows
    3. Change Data Feed must be enabled on the Delta table Correct answer.
    4. The table must be partitioned by ingestion date

    Explanation

    A Delta Sync index consumes the source table's change data, so Change Data Feed must be enabled for it to see inserts, updates, and deletes. Without that feed the index can only reflect the state at build time.

  66. Question 66Prepare data for retrieval augmented generation (RAG)

    In a RAG ingestion pipeline, what does an embedding of a text chunk represent?

    1. An encrypted form of the chunk used for access control
    2. A compressed file that stores the chunk's original bytes
    3. A human-readable summary that replaces the original chunk
    4. A numeric vector that captures semantic features of the text Correct answer.

    Explanation

    An embedding model maps text to a fixed-dimensional numeric vector whose geometry reflects semantic similarity. RAG retrieval compares query and chunk vectors to find passages with related meaning. Embeddings are not summaries, encryption, or compression.

  67. Question 67Prepare data for retrieval augmented generation (RAG)

    Why should a chunk table include source metadata such as the document path and page number?

    1. To increase the dimensionality of the embedding vector
    2. To allow Vector Search to skip embedding at query time
    3. To let Delta Lake enforce uniqueness on every metadata column
    4. To support citations and trace retrieved evidence to its source Correct answer.

    Explanation

    Metadata travels with the chunk row and can be returned alongside retrieved text, enabling citations, filtering, and traceability. Embeddings encode semantic content but do not inherently identify the source location. Metadata should therefore be stored as separate columns.

  68. Question 68Prepare data for retrieval augmented generation (RAG)

    When preparing web pages for RAG, what should be done with repeated navigation menus, ads, and footer boilerplate?

    1. Convert them into primary keys for the chunk table
    2. Keep them so the retriever can answer questions about page layout
    3. Embed them separately with a higher-dimensional model
    4. Remove them during parsing so chunks focus on primary content Correct answer.

    Explanation

    Boilerplate is repeated across many pages and pollutes chunks with text unrelated to the page's main topic. Removing it during parsing improves retrieval precision and reduces wasted index storage. The parser should isolate the primary content before chunking.

  69. Question 69Prepare data for retrieval augmented generation (RAG)

    Which Delta table column type is appropriate for storing embeddings produced by Databricks Foundation Model embedding endpoints for Vector Search?

    1. STRING containing a JSON object
    2. MAP<STRING, FLOAT>
    3. BINARY
    4. ARRAY<FLOAT> Correct answer.

    Explanation

    Embedding endpoints return a list of floating-point values, and Vector Search expects the embedding column to be an array of floats. An ARRAY<FLOAT> column preserves the vector values for indexing and similarity computation. Strings, maps, or binary blobs do not provide the expected vector representation.

  70. Question 70Prepare data for retrieval augmented generation (RAG)

    A pipeline assigns a new random UUID to a chunk every time it reprocesses an unchanged document. What problem does this most directly cause?

    1. Lower-quality vectors because UUIDs are random
    2. Failure of approximate nearest-neighbor search
    3. Duplicate chunk rows and unstable references for updates Correct answer.
    4. Rejection by Delta Lake because UUIDs are not sortable

    Explanation

    Chunk IDs should be stable and deterministic so reprocessing can upsert or replace the same logical chunk. Random IDs make unchanged chunks appear new, causing duplicate rows and breaking citations or update logic. The embedding quality and Vector Search algorithm are not affected by ID randomness.

  71. Question 71Prepare data for retrieval augmented generation (RAG)

    What is the most appropriate way to avoid re-embedding unchanged chunks when a source document is edited slightly?

    1. Hash chunk content and re-embed only changed chunks Correct answer.
    2. Increase embedding dimensions until changes are detected
    3. Re-embed every chunk in the corpus on each edit
    4. Delete and recreate the Vector Search index after every edit

    Explanation

    A content hash lets the pipeline compare the current chunk text with the previously embedded text and skip unchanged chunks. Only new or modified chunks require calls to the embedding model. Re-embedding everything or rebuilding the index wastes compute and time.

  72. Question 72Prepare data for retrieval augmented generation (RAG)

    Before embedding documents that contain customer personal data, which step best reduces privacy risk while preserving useful retrieval?

    1. Use a larger chunk size so personal data is diluted
    2. Store raw text only inside the embedding vector
    3. Redact sensitive fields before embedding the text Correct answer.
    4. Disable metadata filtering on the Vector Search index

    Explanation

    Sensitive values should be removed or masked before text is sent to an embedding model or stored in the chunk table. The sanitized text can still support retrieval for non-sensitive content. Access controls and metadata filters remain important but do not replace redaction.

  73. Question 73Prepare data for retrieval augmented generation (RAG)

    A source contains complex tables. Which preprocessing approach makes table content most retrievable by a RAG system?

    1. Serialize each row with column headers repeated Correct answer.
    2. Flatten the table into one string without row or column labels
    3. Drop tables because embeddings cannot represent numeric data
    4. Replace each table with a single low-resolution screenshot description

    Explanation

    Serializing each row with its column headers makes every row self-contained, so a retrieved chunk preserves the relationship between values and their labels. This helps the model answer questions that depend on specific cells or row-level facts. Unlabeled flattened text loses that structure.

  74. Question 74Prepare data for retrieval augmented generation (RAG)

    A corpus includes English, Spanish, and Japanese documents, and queries may arrive in any of these languages. Which embedding model choice is most appropriate?

    1. An English-only model with translated metadata
    2. A multilingual model trained on the same languages Correct answer.
    3. Any model, as long as it has the largest token limit
    4. A separate model per document selected at random

    Explanation

    A multilingual embedding model places semantically equivalent text from different languages into a shared vector space. That allows a query in one language to retrieve relevant passages in another. Token limit alone does not provide cross-lingual matching.

  75. Question 75Prepare data for retrieval augmented generation (RAG)

    When calling an embedding endpoint for a large set of chunks, why is batching requests usually preferred over one request per chunk?

    1. It lowers request overhead and raises throughput Correct answer.
    2. Batching changes cosine similarity into Euclidean distance
    3. Batching removes the need to store chunk metadata
    4. Batching guarantees vectors with lower dimensionality

    Explanation

    Embedding endpoints can process multiple inputs in one call, which amortizes network and authentication overhead across many chunks. This improves throughput for large ingestion jobs. Batching does not change the similarity metric or eliminate metadata requirements.

  76. Question 76Prepare data for retrieval augmented generation (RAG)

    To evaluate the retrieval stage of a RAG pipeline, what dataset is most directly useful?

    1. A log of LLM response times from production traffic
    2. A list of embedding dimensions used by the model
    3. A count of documents ingested each night
    4. Queries paired with known relevant chunk IDs Correct answer.

    Explanation

    Retrieval evaluation needs labeled judgments about which chunks are relevant to each query. With those labels, metrics such as recall or precision can measure whether the retriever surfaces the right passages. Generation latency and ingestion counts do not measure retrieval relevance.

  77. Question 77Prepare data for retrieval augmented generation (RAG)

    Why attach a department field to each chunk and define it as a filterable Vector Search column?

    1. It lets queries filter retrieval by department Correct answer.
    2. It automatically encrypts chunks for each department
    3. It forces Delta Lake to repartition the table by department
    4. It reduces embedding dimension to the department count

    Explanation

    A filterable metadata column lets the application restrict similarity search to chunks that satisfy a condition such as department = 'finance'. This narrows the candidate set and can support access or relevance constraints. It does not change the embedding model or encrypt data.

  78. Question 78Prepare data for retrieval augmented generation (RAG)

    A document is updated and its text changes substantially. The pipeline inserts the new chunks but leaves the old chunks in the Delta table. What is the most likely effect on retrieval?

    1. Stale and current passages may both be retrieved Correct answer.
    2. The embedding model rejects new chunks as duplicates
    3. Delta Lake merges old and new chunks into an averaged vector
    4. Vector Search automatically removes old chunks by source path

    Explanation

    Leaving old chunks in place means both outdated and current passages remain candidates for retrieval, which can mislead the generator. The pipeline should delete or replace chunks associated with the prior document version before or during the update. Vector Search does not infer semantic replacement from a shared path.

  79. Question 79Prepare data for retrieval augmented generation (RAG)

    Sources provide different metadata: some have author and date, others only a path. Which schema design best supports consistent retrieval filters?

    1. A common schema with nullable metadata columns Correct answer.
    2. Encode all metadata into the embedding vector and omit filter columns
    3. Create a separate table per metadata shape and query all tables at runtime
    4. Discard chunks from sources that lack complete metadata

    Explanation

    A single chunk schema with nullable metadata columns keeps all chunks queryable through one index and allows filters to handle missing values consistently. Separate tables or discarding incomplete sources complicate retrieval and reduce coverage. Metadata should remain in columns rather than being hidden inside embeddings.

  80. Question 80Prepare data for retrieval augmented generation (RAG)

    A chunking job must stay under an embedding model's maximum token limit, and each chunk will later be prefixed with a short section title. What is the safest chunk-size rule?

    1. Set chunk size equal to the model limit in characters, ignoring the title prefix
    2. Choose chunk size by file count rather than by tokens
    3. Use the model limit exactly because special tokens are never counted
    4. Reserve tokens for the title prefix and set chunk size below the model limit Correct answer.

    Explanation

    The embedding input includes the chunk text, the title prefix, and any special tokens added by the model. The chunk size should therefore leave a safety margin below the model's maximum token limit. Measuring in characters or ignoring the prefix can cause silent truncation.

  81. Question 81Develop, tune and evaluate generative AI applications

    A team expects sustained high request volume for a foundation model and needs predictable latency and cost. Which Databricks Foundation Model API deployment option should they choose?

    1. Single-node CPU cluster
    2. Serverless SQL warehouse
    3. Provisioned throughput Correct answer.
    4. Pay-per-token

    Explanation

    Provisioned throughput reserves dedicated model capacity, which suits sustained production traffic with predictable performance and cost. Pay-per-token is better for variable or lower-volume workloads and can encounter rate limits.

  82. Question 82Develop, tune and evaluate generative AI applications

    Which Databricks Vector Search index type automatically keeps its index synchronized with a source Delta table?

    1. Direct Vector Access index
    2. HNSW-only index
    3. Parquet sync index
    4. Delta Sync index Correct answer.

    Explanation

    A Delta Sync index is created from a source Delta table and uses a sync pipeline to reflect inserts, updates, and deletes. Direct Vector Access requires the application to write vectors through the API.

  83. Question 83Develop, tune and evaluate generative AI applications

    What is the primary purpose of MLflow tracing in a generative AI application?

    1. To fine-tune the foundation model
    2. To provision throughput for model serving
    3. To trace agent and LLM calls Correct answer.
    4. To encrypt endpoint credentials

    Explanation

    MLflow tracing records nested spans for agent, retrieval, and LLM calls, including inputs, outputs, and latency, so developers can debug execution. It is an observability feature rather than a training or deployment mechanism.

  84. Question 84Develop, tune and evaluate generative AI applications

    What does an embedding model return for an input text?

    1. A SQL query plan
    2. A semantic-meaning vector Correct answer.
    3. A tokenized prompt template
    4. A human-readable summary

    Explanation

    Embedding models map text to numeric vectors whose distances represent semantic similarity, and Vector Search uses those vectors for retrieval. Summaries, query plans, and prompt templates are different artifacts.

  85. Question 85Develop, tune and evaluate generative AI applications

    In a chat-style foundation model API request, which message role is used to set persistent instructions and behavior for the assistant?

    1. system Correct answer.
    2. tool
    3. assistant
    4. user

    Explanation

    The system role carries high-level instructions and behavioral constraints that apply across the conversation. User messages express requests, assistant messages are prior model turns, and tool messages carry tool results.

  86. Question 86Develop, tune and evaluate generative AI applications

    What do inference tables capture for a Databricks model serving endpoint?

    1. The endpoint's IAM roles
    2. The training data used to fine-tune the model
    3. The vector index's embedding dimensions
    4. Request and response payloads Correct answer.

    Explanation

    When enabled, inference tables log endpoint request and response payloads to a Delta table for monitoring, auditing, and debugging. They do not contain IAM roles, training data, or index metadata.

  87. Question 87Develop, tune and evaluate generative AI applications

    Which Databricks AI Gateway capability helps prevent sensitive personal data from being sent to or returned by a model endpoint?

    1. Provisioned throughput
    2. Response caching
    3. PII guardrails Correct answer.
    4. Traffic routing

    Explanation

    AI Gateway guardrails can detect and block or redact PII in requests and responses. Traffic routing and caching control delivery, while provisioned throughput controls serving capacity.

  88. Question 88Develop, tune and evaluate generative AI applications

    In a retrieval-augmented generation application, what is the retrieval component responsible for?

    1. Fine-tuning the model weights
    2. Retrieving relevant context Correct answer.
    3. Compressing the final answer
    4. Assigning endpoint permissions

    Explanation

    Retrieval searches an index for documents or chunks relevant to the user query and supplies them as context to the model. Fine-tuning, answer compression, and permissions are separate concerns.

  89. Question 89Develop, tune and evaluate generative AI applications

    Why is overlap often added between adjacent text chunks during RAG ingestion?

    1. To eliminate the need for a vector index
    2. To preserve context split by chunk boundaries Correct answer.
    3. To guarantee that every chunk has the same token count
    4. To reduce the embedding model's vector dimensions

    Explanation

    Overlap duplicates text near chunk boundaries so a fact spanning two chunks remains retrievable in at least one chunk. It does not change vector dimensions, equalize token counts, or remove the need for an index.

  90. Question 90Develop, tune and evaluate generative AI applications

    Which agent evaluation metric measures whether the response is supported by the retrieved context?

    1. Token count
    2. Throughput
    3. Groundedness Correct answer.
    4. Latency

    Explanation

    Groundedness, also called faithfulness, checks whether claims in an answer are supported by the retrieved context and helps detect hallucination. Latency, token count, and throughput are operational measurements.

  91. Question 91Develop, tune and evaluate generative AI applications

    In Mosaic AI Agent Evaluation, what is the role of an LLM judge?

    1. It deploys the agent to a serving endpoint
    2. It generates embeddings for the vector index
    3. It scores responses against rubric criteria Correct answer.
    4. It stores the evaluation dataset in Unity Catalog

    Explanation

    An LLM judge uses a language model to score agent outputs against rubric-based criteria such as relevance, correctness, or groundedness. It does not store datasets, deploy endpoints, or create embeddings.

  92. Question 92Develop, tune and evaluate generative AI applications

    A team has product documentation but no labeled evaluation questions. What Databricks capability can generate an evaluation dataset from the documentation?

    1. Vector Search direct upsert
    2. Model serving autoscaling
    3. Synthetic data generation Correct answer.
    4. Inference table compaction

    Explanation

    Synthetic evaluation data generation uses source documents to create questions and reference answers for bootstrapping agent evaluation. Compaction, autoscaling, and direct upsert do not create evaluation datasets.

  93. Question 93Develop, tune and evaluate generative AI applications

    Which training data format is appropriate for fine-tuning a chat-style foundation model with Mosaic AI Model Training?

    1. Conversations with role-tagged messages Correct answer.
    2. A ZIP archive of PDF manuals
    3. A single unlabeled CSV of numeric features
    4. SQL DDL statements for the target table

    Explanation

    Chat fine-tuning expects conversation records with roles such as system, user, and assistant, and their content. Instruction-response pairs can also be formatted for training, but raw numeric features, PDFs, or DDL are not training examples.

  94. Question 94Develop, tune and evaluate generative AI applications

    After fine-tuning a foundation model with Mosaic AI Model Training, how is the resulting model typically made available for low-latency inference?

    1. By exporting it as a dashboard
    2. By deploying it to a model serving endpoint Correct answer.
    3. By querying the training Delta table directly
    4. By attaching it to a SQL warehouse

    Explanation

    Fine-tuned model artifacts can be registered and deployed to a Databricks model serving endpoint for low-latency inference. A Delta table, SQL warehouse, or dashboard does not serve the model itself.

  95. Question 95Develop, tune and evaluate generative AI applications

    Which metric evaluates how many of the retrieved chunks are actually relevant to the query?

    1. Retrieval precision Correct answer.
    2. Endpoint uptime
    3. Model perplexity
    4. Answer length

    Explanation

    Retrieval precision measures the proportion of retrieved chunks that are relevant, while recall measures coverage of all relevant chunks. Length, perplexity, and uptime do not measure retrieval relevance.

  96. Question 96Develop, tune and evaluate generative AI applications

    What does hybrid search in Databricks Vector Search combine?

    1. Provisioned throughput and pay-per-token
    2. Keyword-based and vector similarity search Correct answer.
    3. Fine-tuning and prompt engineering
    4. Batch inference and streaming

    Explanation

    Hybrid search combines lexical keyword matching with semantic vector similarity, which can improve recall for exact terms and names. The other pairs are unrelated to search ranking.

  97. Question 97Develop, tune and evaluate generative AI applications

    What is the purpose of a reranker in a RAG pipeline?

    1. To rerank retrieved candidates by relevance Correct answer.
    2. To reduce the number of embedding dimensions
    3. To train the foundation model on new documents
    4. To cache endpoint responses

    Explanation

    A reranker takes an initial candidate set from retrieval and scores each candidate more precisely, improving the context passed to the generator. It does not change embedding dimensions, train the model, or cache responses.

  98. Question 98Develop, tune and evaluate generative AI applications

    Which mechanism is appropriate for a production service to authenticate to a Databricks model serving endpoint without embedding a user's personal access token?

    1. OAuth M2M for a service principal Correct answer.
    2. Anonymous public access
    3. A browser session cookie
    4. A shared notebook username and password

    Explanation

    A service principal with OAuth machine-to-machine credentials provides a non-human identity and short-lived tokens suitable for production services. Shared user credentials, anonymous access, and browser cookies are not appropriate for service authentication.

  99. Question 99Develop, tune and evaluate generative AI applications

    How does AI Gateway fallback improve reliability for a model serving endpoint?

    1. It deduplicates training data
    2. It fine-tunes the model automatically
    3. It increases the model's context window
    4. It routes traffic to a backup endpoint on failure Correct answer.

    Explanation

    Fallback configuration sends requests to a secondary endpoint when the primary endpoint errors or times out, improving availability. It does not alter the model context window, train models, or clean training data.

  100. Question 100Develop, tune and evaluate generative AI applications

    Which MLflow API is used to run evaluation of a generative AI application against a dataset and produce metrics?

    1. mlflow.register_model
    2. mlflow.start_run only
    3. mlflow.evaluate Correct answer.
    4. mlflow.log_artifact

    Explanation

    mlflow.evaluate runs evaluators over a dataset and records metrics and per-row results for the evaluated model or agent. log_artifact stores files, register_model manages model versions, and start_run only opens a run context.

  101. Question 101Develop, tune and evaluate generative AI applications

    An agent has no ground-truth answers for a new document set. Which evaluation approach is most appropriate?

    1. Compute exact-match accuracy against empty strings
    2. Use reference-free LLM judges for relevance and groundedness Correct answer.
    3. Use training loss as the only quality metric
    4. Skip evaluation until labels are manually created for every query

    Explanation

    Reference-free LLM judges can assess relevance and groundedness using the query and retrieved context without gold answers. Waiting for exhaustive labels or using exact match against empty strings does not produce meaningful quality signals.

  102. Question 102Develop, tune and evaluate generative AI applications

    A developer needs more deterministic, repeatable outputs from a chat model for an extraction task. Which parameter change is most directly appropriate?

    1. Decrease temperature Correct answer.
    2. Increase top_p to 1.0
    3. Increase temperature
    4. Increase max_tokens

    Explanation

    Lowering temperature sharpens the sampling distribution, reducing randomness and making outputs more repeatable. Higher temperature increases diversity, while top_p and max_tokens control sampling breadth and length rather than determinism directly.

  103. Question 103Develop, tune and evaluate generative AI applications

    A company needs an assistant to answer from a knowledge base that changes daily and must cite current documents. Which approach should be prioritized?

    1. Rely only on the model's pretrained knowledge
    2. Increase the model's temperature
    3. Fine-tune the model on the entire knowledge base each day
    4. Use RAG over a regularly refreshed vector index Correct answer.

    Explanation

    RAG retrieves current documents at query time and can return citations without retraining the model. Daily full fine-tuning is costly and slow, temperature does not add knowledge, and pretrained knowledge may be stale.

  104. Question 104Develop, tune and evaluate generative AI applications

    A team wants to detect changes in the distribution of requests sent to a production RAG endpoint over time. Which Databricks setup is most appropriate?

    1. Inference tables plus Lakehouse Monitoring Correct answer.
    2. Enable auto-scaling on the endpoint
    3. Run mlflow.log_artifact after each request
    4. Increase the endpoint's provisioned throughput

    Explanation

    Inference tables capture request and response payloads in Delta tables, and Lakehouse Monitoring can profile those tables and detect drift over time. Autoscaling and throughput changes address capacity, not distribution monitoring.

  105. Question 105Develop, tune and evaluate generative AI applications

    An agent's answers are sometimes wrong. Trace inspection shows the correct document is retrieved but the final answer contradicts it. Which component should be investigated first?

    1. The embedding model's vector dimensions
    2. The endpoint's autoscaling configuration
    3. The vector index sync schedule
    4. The generation prompt and LLM synthesis Correct answer.

    Explanation

    If the correct document is already retrieved, the retrieval layer is working for that case, so the next suspect is how the prompt presents context and how the LLM synthesizes the answer. Embedding dimensions, index sync, and autoscaling would primarily affect retrieval or capacity.

  106. Question 106Develop, tune and evaluate generative AI applications

    A team wants a chat model to always return a JSON object with a fixed set of keys for a ticket-classification task. Which prompt design choice most reliably improves format compliance without additional training?

    1. Include several input/output examples in the prompt that show the exact JSON shape. Correct answer.
    2. Add a sentence at the end of the prompt asking the model to be careful and accurate.
    3. Increase the maximum output tokens so the JSON has room to complete.
    4. Raise the temperature so the model can explore more output formats.

    Explanation

    Few-shot prompting supplies concrete demonstrations of the desired output schema, which conditions the model on the exact structure far more reliably than an instruction alone. Decoding parameters change randomness or length, but neither teaches the model a required shape.

  107. Question 107Develop, tune and evaluate generative AI applications

    In a RAG application, what is the main trade-off when choosing a small chunk size such as 200 tokens?

    1. Smaller chunks make the embedding model produce vectors with fewer dimensions.
    2. Smaller chunks improve retrieval precision but may omit surrounding context the model needs to answer. Correct answer.
    3. Smaller chunks reduce index storage, so retrieval quality always improves.
    4. Smaller chunks prevent the retriever from applying metadata filters at query time.

    Explanation

    Chunk size controls how much context travels with each retrieved passage. Small chunks match query terms closely but can cut sentences that carry the answer, while large chunks carry more context but dilute relevance.

  108. Question 108Develop, tune and evaluate generative AI applications

    What is the purpose of implementing predict_stream in a Mosaic AI Agent Framework agent?

    1. It automatically evaluates each response with an LLM judge during serving.
    2. It returns response tokens incrementally so users see output before generation finishes. Correct answer.
    3. It lets the endpoint batch multiple requests into a single model call.
    4. It stores each agent response in a Delta table for later review.

    Explanation

    Streaming yields partial output as the model generates it, which lowers perceived latency in chat experiences. Batching, logging and evaluation are separate concerns handled by other components.

  109. Question 109Develop, tune and evaluate generative AI applications

    Which property of a fine-tuning dataset most directly determines how well a tuned model behaves at inference time?

    1. The size of the compute cluster used for the training run.
    2. How closely the training prompts and responses match real production traffic. Correct answer.
    3. The number of columns in the Delta table holding the examples.
    4. The number of evaluation metrics registered for the run.

    Explanation

    A tuned model learns the patterns present in its examples, so training pairs that mirror production inputs and desired outputs teach the behavior actually needed in the application. Cluster size and metric registration change cost and reporting, not learned behavior.

  110. Question 110Develop, tune and evaluate generative AI applications

    A RAG assistant serves multiple tenants from a single Vector Search index, and each tenant must only see its own documents. Which approach enforces this at query time?

    1. Set the model temperature to zero so the retriever cannot mix tenant content.
    2. Increase the number of retrieved chunks and rely on the model to ignore other tenants' text.
    3. Apply a metadata filter on a tenant field in the query to the index. Correct answer.
    4. Store documents for each tenant in a separate embedding dimension.

    Explanation

    Vector Search supports filtering on metadata fields alongside the similarity search, so a tenant filter restricts candidates before they ever reach the prompt. Retrieval cannot rely on the model to disregard retrieved text, and temperature has no bearing on which documents are returned.

  111. Question 111Develop, tune and evaluate generative AI applications

    An agent calls an internal pricing API before answering, and the team needs that call's latency visible in the agent's MLflow trace. What should the developer do?

    1. Log the pricing API response as a model metric with mlflow.log_metric.
    2. Wrap the pricing API call in a function decorated with mlflow.trace. Correct answer.
    3. Add the pricing API output to the system prompt of the agent.
    4. Enable inference tables to capture the pricing call.

    Explanation

    The trace decorator creates a span around any Python function, recording its inputs, outputs and duration inside the trace of the enclosing request. Metrics and inference tables record aggregates or payloads but do not add a timed step to the trace.

  112. Question 112Develop, tune and evaluate generative AI applications

    A team builds a Vector Search index with a self-managed embedding model and later swaps the query-time embedding model for a newer one. What is the most likely consequence?

    1. Query vectors no longer align with the stored vectors, so retrieval quality degrades. Correct answer.
    2. Vector Search rejects the query because the model name is not in the index metadata.
    3. Retrieval becomes slower but returns the same results as before.
    4. The index rebuilds itself automatically with the new model's vectors.

    Explanation

    Similarity search only makes sense when queries and documents live in the same embedding space; changing only one side makes the computed distances meaningless. Databricks-hosted embedding models avoid this by embedding queries with the same model used to build the index.

  113. Question 113Develop, tune and evaluate generative AI applications

    Mosaic AI Agent Evaluation must score retrieval quality for an agent. Which trace content does it need for those retrieval metrics?

    1. The final chat completion request sent to the model.
    2. The agent's system prompt and its version number.
    3. Spans of type RETRIEVER recording the retrieved documents. Correct answer.
    4. The token counts reported by the serving endpoint.

    Explanation

    Retrieval metrics compare the documents the agent actually retrieved against the expected documents, so the trace must contain retriever spans that carry the retrieved chunks. Prompt text and token counts describe generation, not what was retrieved.

  114. Question 114Develop, tune and evaluate generative AI applications

    Which retrieval metric measures the share of the ground-truth relevant chunks that the retriever actually returned?

    1. Answer correctness
    2. Context recall Correct answer.
    3. Groundedness
    4. Context precision

    Explanation

    Context recall compares retrieved chunks with the known relevant set and counts how much of that set was found. Context precision measures the opposite direction: how many of the retrieved chunks were relevant.

  115. Question 115Develop, tune and evaluate generative AI applications

    A team versions its agent's prompt in the MLflow Prompt Registry and wants production traffic to pick up a newly promoted version without a code change. Which mechanism should they use?

    1. Load the prompt by a mutable alias such as production. Correct answer.
    2. Export the prompt to a JSON file stored in the workspace.
    3. Copy the prompt text into a notebook widget each release.
    4. Reference the prompt version number directly in the code.

    Explanation

    Prompt versions are immutable and aliases are movable pointers to a version, so loading by alias lets a promotion change what production resolves without editing code. Pinning a version or copying text into code or files requires a redeploy for every update.

  116. Question 116Develop, tune and evaluate generative AI applications

    A team needs to score whether agent responses match their brand voice, a requirement no built-in metric covers. What is the appropriate approach in Mosaic AI Agent Evaluation?

    1. Lower the model temperature until the responses become on-brand.
    2. Rely on the groundedness metric as a proxy for tone.
    3. Ask human reviewers to score every response in production.
    4. Define a custom LLM judge metric that includes a rubric describing the desired voice. Correct answer.

    Explanation

    Custom judge metrics let a team encode domain-specific criteria in a rubric that an LLM applies consistently across the evaluation set. Built-in metrics measure correctness or grounding, and decoding parameters do not encode style requirements.

  117. Question 117Develop, tune and evaluate generative AI applications

    Which situation makes fine-tuning a better choice than few-shot prompting?

    1. Only a dozen example conversations are available.
    2. The desired behavior must change weekly as new policies are published.
    3. Many consistent examples exist and style must hold at high volume. Correct answer.
    4. The task requires citing newly uploaded documents at answer time.

    Explanation

    Fine-tuning excels when many demonstrations share a stable target behavior, because the behavior is baked into the weights and paid for once rather than on every prompt. Rapidly changing content is better handled by retrieval or prompt updates.

  118. Question 118Develop, tune and evaluate generative AI applications

    Users report that an agent forgets the previous turn of a conversation. What is the most likely cause in the agent's request handling?

    1. The retrieval index is too small to hold conversation history.
    2. The serving endpoint has autoscaling disabled during peak hours.
    3. The judge model in evaluation was configured with a low threshold.
    4. Only the newest user message is sent to the model on each call. Correct answer.

    Explanation

    Chat models are stateless, so continuity depends on the application resending prior turns as part of the messages list on each request. Index size, judge thresholds and autoscaling affect retrieval, scoring and capacity, not conversational memory.

  119. Question 119Develop, tune and evaluate generative AI applications

    Evaluation shows context precision is high but context recall is low for a RAG agent. Which change most directly addresses the problem?

    1. Replace the LLM judge with a custom rubric metric.
    2. Raise the temperature of the generator model.
    3. Increase the number of retrieved chunks and review chunking for split answers. Correct answer.
    4. Add a system prompt instructing the model to answer only from context.

    Explanation

    Low recall means relevant passages never reach the prompt, so the fix is to widen candidate retrieval or repair chunking that splits answers across boundaries. Reranking and prompting operate only on chunks that were already retrieved and cannot recover missing evidence.

  120. Question 120Develop, tune and evaluate generative AI applications

    A team compares two agent versions by having an LLM judge pick the better response, and they suspect the judge favors whichever response appears first. Which evaluation design reduces this bias?

    1. Increase the judge's temperature so its choices vary between runs.
    2. Score each pair twice with the response order swapped and average the results. Correct answer.
    3. Ask the judge to output a confidence score alongside its choice.
    4. Use a single judge call per pair but with a longer rubric.

    Explanation

    Position bias is a known effect in pairwise LLM judging; running each comparison in both orders and aggregating cancels the systematic preference for one slot. Higher temperature adds noise rather than removing bias, and longer rubrics do not address ordering.

  121. Question 121Deploy and monitor generative AI applications

    An engineer enables inference tables on a Model Serving endpoint. What does this capability provide?

    1. A cached copy of the model weights stored alongside the endpoint for faster loading
    2. A dashboard of GPU utilization for the compute that hosts the endpoint
    3. Automatic retraining of the served model whenever response quality degrades
    4. Request and response payloads logged to a Unity Catalog Delta table Correct answer.

    Explanation

    Inference tables capture the request and response payloads an endpoint handles and write them to a Delta table in Unity Catalog, given a catalog, schema, and table prefix in the endpoint configuration. The resulting table can then be queried, audited, or monitored like any other Delta table.

  122. Question 122Deploy and monitor generative AI applications

    Which serving option reserves dedicated model capacity and is billed by the hour rather than per token?

    1. CPU model serving with scale to zero enabled
    2. Provisioned throughput Correct answer.
    3. Pay-per-token Foundation Model APIs
    4. A serverless SQL warehouse running a Photon cluster

    Explanation

    Provisioned throughput reserves dedicated capacity for a foundation model and is billed for that reserved capacity, which gives predictable performance and protection from shared-pool rate limits. Pay-per-token endpoints instead bill by tokens consumed and are subject to shared capacity.

  123. Question 123Deploy and monitor generative AI applications

    A Databricks App calls a serving endpoint on behalf of its own service principal. Which grant makes those calls succeed?

    1. CAN QUERY on the serving endpoint Correct answer.
    2. USE CATALOG on the catalog that stores the model
    3. SELECT on the endpoint's inference table
    4. CAN MANAGE on the workspace folder that holds the app

    Explanation

    Databricks Apps run under a service principal, and that principal must hold the CAN QUERY privilege on the serving endpoint to invoke it. Granting privileges on the catalog or the app folder does not authorize endpoint invocation.

  124. Question 124Deploy and monitor generative AI applications

    Which HTTP request invokes a Databricks serving endpoint named chat-endpoint?

    1. POST /api/2.0/serving-endpoints/chat-endpoint/invocations Correct answer.
    2. POST /api/2.0/serving-endpoints/chat-endpoint/predict
    3. GET /api/2.0/serving-endpoints/chat-endpoint
    4. PUT /api/2.0/serving/chat-endpoint/score

    Explanation

    Serving endpoints accept scored requests as a POST to the invocations path under serving-endpoints, with the payload sent as JSON in the request body. The GET form of the same path returns endpoint metadata rather than a model response.

  125. Question 125Deploy and monitor generative AI applications

    Which base_url lets the OpenAI Python client call Databricks foundation model endpoints?

    1. https://<workspace-url>/api/2.0/mlflow
    2. https://<workspace-url>/model-serving/v1/chat
    3. https://<workspace-url>/serving-endpoints Correct answer.
    4. https://<workspace-url>/api/2.0/llm/v1

    Explanation

    Databricks exposes an OpenAI-compatible surface under the workspace's serving-endpoints path, so the OpenAI client is configured with that base_url plus a Databricks token as the API key. The model argument is then the name of the serving endpoint.

  126. Question 126Deploy and monitor generative AI applications

    An endpoint shows the state READY in the Serving UI. What does that indicate?

    1. The endpoint is scaled to zero and will cold start on the next request
    2. The endpoint served at least one request in the past 24 hours
    3. The endpoint configuration is applied and the endpoint accepts requests Correct answer.
    4. The endpoint passed all quality evaluations on its inference table

    Explanation

    READY means the endpoint has finished deploying its current configuration and is available to serve traffic. A scaled-to-zero endpoint can still be READY; readiness does not imply that requests are currently warm or that quality checks passed.

  127. Question 127Deploy and monitor generative AI applications

    An engineer decorates a retrieval function with @mlflow.trace. What is the effect on the deployed application?

    1. The function is registered as a Unity Catalog function
    2. The function is automatically deployed behind a serving endpoint
    3. The function's outputs are cached to avoid repeated model calls
    4. Each call is recorded as a span inside an MLflow trace Correct answer.

    Explanation

    The trace decorator instruments the function so that every invocation becomes a span within a trace, capturing inputs, outputs, timing, and nesting relative to parent calls. This produces the execution detail needed to debug and monitor multi-step GenAI chains.

  128. Question 128Deploy and monitor generative AI applications

    Which prerequisite must be met before a custom MLflow model can be deployed to a Model Serving endpoint?

    1. The model must be packaged into a Docker image by the user
    2. The model must carry a champion alias in the workspace registry
    3. The model must be exported to an ONNX file
    4. The model must be registered in Unity Catalog Correct answer.

    Explanation

    Model Serving deploys custom models from Unity Catalog, so the MLflow model must be registered there with a catalog, schema, and model name. Databricks builds and hosts the serving container, so users do not supply container images.

  129. Question 129Deploy and monitor generative AI applications

    An engineer wants 90 percent of endpoint traffic on the current model and 10 percent on a challenger. Which action accomplishes this?

    1. Deploy the challenger to a separate workspace and compare usage dashboards
    2. Set the challenger's model version number to 10 in the endpoint configuration
    3. Add both models as served entities with a traffic split of 90 and 10 Correct answer.
    4. Create two separate endpoints and give each user a different endpoint URL

    Explanation

    A single endpoint can host multiple served entities, and a traffic configuration assigns a percentage of requests to each route, with percentages summing to 100. This is the mechanism for A/B tests and canary rollouts.

  130. Question 130Deploy and monitor generative AI applications

    A challenger route has received 10 percent of traffic without errors for a week. What is the standard way to complete the rollout?

    1. Keep the 90/10 split permanently so both models stay warm
    2. Delete the original served entity first, then resend all traffic to the challenger
    3. Recreate the endpoint from scratch with only the challenger model attached
    4. Raise the challenger's traffic to 100 percent and then remove the old served entity Correct answer.

    Explanation

    A canary rollout is completed by shifting the traffic configuration fully onto the new route and then removing the superseded served entity, which keeps a rollback path open until the shift is done. Deleting the old route first removes the fallback if the new version misbehaves.

  131. Question 131Deploy and monitor generative AI applications

    A new model version was promoted to 100 percent of endpoint traffic and error rates spiked. What is the fastest way to restore service?

    1. Enable inference tables so the failures can be inspected later
    2. Retrain the model and redeploy it as a new version
    3. Lower the endpoint's rate limit to shed the failing requests
    4. Point the served entity back to the previous model version Correct answer.

    Explanation

    Because the endpoint references a specific model version per served entity, reverting that version reference is a configuration change that quickly restores the last known good behavior. Diagnosis of the failing version can proceed afterwards using the captured payloads.

  132. Question 132Deploy and monitor generative AI applications

    Compliance requires an auditable record of prompts and completions for an endpoint. Which AI Gateway feature provides it?

    1. Fallback routing
    2. Payload logging Correct answer.
    3. Usage tracking
    4. Rate limiting

    Explanation

    AI Gateway payload logging writes request and response bodies to a configured Delta table, creating the auditable record of what was sent and returned. The other gateway features govern cost visibility, request volume, and availability rather than content capture.

  133. Question 133Deploy and monitor generative AI applications

    An application must keep personal data out of model completions and downstream logs. Which configuration fits?

    1. Enable AI Gateway guardrails with PII detection and redaction Correct answer.
    2. Enable inference tables and delete the payload table every night
    3. Increase provisioned throughput so that filtering runs faster
    4. Add a system prompt asking the model to be careful with personal data

    Explanation

    AI Gateway guardrails can detect and redact PII in requests and responses before the content is returned or persisted. Interactive filtering is deterministic policy enforcement, whereas prompt instructions are advisory and cannot be relied on for compliance.

  134. Question 134Deploy and monitor generative AI applications

    One client is consuming most requests to a shared endpoint and starving other users. Which control addresses this directly?

    1. Scale the endpoint to zero when it is idle
    2. Enable streaming responses for all clients
    3. Configure per-user rate limits in AI Gateway Correct answer.
    4. Turn on inference tables for the endpoint

    Explanation

    AI Gateway rate limits cap how many requests a given caller can issue in a time window, preventing a single consumer from monopolizing shared capacity. Other endpoint settings affect capacity, debugging, or latency, but not fair allocation across callers.

  135. Question 135Deploy and monitor generative AI applications

    A primary model intermittently returns 429 errors during peak hours. Which AI Gateway configuration keeps responses flowing?

    1. Raise the model's max_tokens so fewer requests are needed
    2. Set the endpoint's scale-to-zero timeout to zero
    3. Log payloads to a Delta table and replay the failed requests later
    4. Add a fallback route to a secondary served model Correct answer.

    Explanation

    AI Gateway can route to a fallback served entity when the primary route fails or is throttled, so callers still get a response during capacity pressure. This provides availability continuity without changing the client.

  136. Question 136Deploy and monitor generative AI applications

    An engineer wants drift and quality metrics on the traffic a deployed model actually receives. Where should a monitor be created?

    1. On the MLflow run that trained the model
    2. On the Git folder that holds the deployment notebook
    3. On the serving endpoint's GPU metrics dashboard
    4. On the inference table that stores the endpoint's payloads Correct answer.

    Explanation

    Lakehouse Monitoring profiles Delta tables, so a monitor on the inference table analyzes the deployed model's real inputs and outputs and compares them against a baseline. This yields drift and quality statistics computed over production traffic.

  137. Question 137Deploy and monitor generative AI applications

    Server-side errors from a custom model endpoint must be inspected after the fact. Which capability writes them to Unity Catalog?

    1. Trace spans emitted by MLflow tracing
    2. Endpoint logs delivered to a Delta table Correct answer.
    3. Model signatures recorded in the model registry
    4. Query history stored in the system tables

    Explanation

    Endpoint logging can deliver the container's server and build logs to a Delta table in Unity Catalog, preserving the error detail needed to debug failed or slow requests. This is separate from request payload capture and from application-level tracing.

  138. Question 138Deploy and monitor generative AI applications

    An interactive chat application feels slow because users wait for the entire completion before seeing anything. Which change improves perceived latency?

    1. Enable payload logging on the endpoint
    2. Increase the maximum number of output tokens
    3. Reduce the endpoint's provisioned concurrency
    4. Stream tokens to the client as they are generated Correct answer.

    Explanation

    Streaming returns incremental tokens so the user sees output almost immediately, which lowers perceived latency even when total generation time is unchanged. Endpoint-side logging or capacity changes do not improve time to first visible token.

  139. Question 139Deploy and monitor generative AI applications

    A custom model needs a third-party API key at inference time without embedding it in the model artifact. How should the key be supplied?

    1. As a comment in the MLflow model's conda.yaml
    2. As a widget default in the deployment notebook
    3. As an environment variable that references a secret scope key Correct answer.
    4. As a column in the inference table's payload schema

    Explanation

    Serving endpoint configuration supports environment variables whose values can reference Databricks secret scope keys, so credentials are injected at runtime and never stored in the model artifact. This keeps the secret out of versioned code and logged payloads.

  140. Question 140Deploy and monitor generative AI applications

    A RAG application must reflect newly added documents in its Vector Search index within minutes. Which index option should be used?

    1. A blockwise index built once from a Parquet snapshot
    2. A Delta Sync index with continuous sync enabled Correct answer.
    3. A triggered sync index refreshed nightly by a job
    4. A managed embeddings index over a static CSV file

    Explanation

    Continuous Delta Sync keeps the index updated as the source Delta table changes, so new documents become searchable shortly after they land. Triggered sync only refreshes when explicitly invoked, which introduces staleness.

  141. Question 141Deploy and monitor generative AI applications

    Latency and error metrics look healthy, but users report that many answers are low quality. Which monitoring addition best detects this?

    1. Enable a second fallback model behind the endpoint
    2. Collect user feedback and run quality evaluations on sampled traces Correct answer.
    3. Add more provisioned concurrency to the endpoint
    4. Increase the payload logging retention window

    Explanation

    Operational metrics cannot reveal relevance or correctness, so quality must be measured with human feedback and evaluator scores attached to logged traces. Reviewing sampled traces links those scores back to the exact prompts, retrievals, and responses that produced them.

  142. Question 142Deploy and monitor generative AI applications

    Requests fail with a 400 error stating that input tokens plus max_tokens exceed the model's context length. Which fix applies?

    1. Switch the endpoint to scale to zero to free capacity
    2. Enable payload logging and resend the identical request
    3. Raise the endpoint's rate limit for the affected client
    4. Shorten the prompt or reduce the max_tokens value Correct answer.

    Explanation

    The context window bounds input plus reserved output tokens, so the request must be brought under that limit by trimming the prompt, reducing retrieved context, or lowering max_tokens. Capacity and throttling settings cannot enlarge the model's context window.

  143. Question 143Deploy and monitor generative AI applications

    Finance needs token cost attributed to each team that shares a production endpoint. Which approach supports that?

    1. Count requests in the serving endpoint access logs
    2. Read the endpoint's GPU utilization metrics per hour
    3. Compare model versions in the MLflow experiment runs
    4. Use AI Gateway usage tracking records with per-request token counts Correct answer.

    Explanation

    Cost is driven by tokens rather than request count, so usage tracking records that include per-request prompt and completion token counts can be attributed to consumers and priced. Utilization metrics describe hardware and cannot be converted into per-team token cost.

  144. Question 144Deploy and monitor generative AI applications

    A provisioned throughput endpoint drops requests with timeouts during sharp traffic bursts. What addresses the root cause?

    1. Increase the endpoint's provisioned capacity to cover the peak Correct answer.
    2. Increase the client-side retry count and retry immediately
    3. Reduce max_tokens so that responses complete faster
    4. Disable inference tables to reduce write overhead

    Explanation

    Timeouts during bursts indicate that demand exceeds the reserved serving capacity, so requests queue past the timeout threshold. Raising provisioned capacity sizes the endpoint for the peak; retries and token pruning treat symptoms or assume a different bottleneck.

  145. Question 145Deploy and monitor generative AI applications

    Which practice best gates a new model version before it takes full production traffic?

    1. Evaluate it offline on a labeled set, then canary it at a small traffic share Correct answer.
    2. Skip evaluation because the endpoint's health check passes
    3. Deploy the version to 100 percent of traffic and watch the error rate
    4. Rely on the training loss reported in the MLflow run

    Explanation

    Offline evaluation on a labeled set estimates quality before any user is exposed, and a small canary share then validates behavior on real traffic with limited blast radius. Promotion proceeds only when both stages meet the agreed thresholds.

  146. Question 146Deploy and monitor generative AI applications

    A Model Serving endpoint has payload logging enabled with a Unity Catalog destination. What does the resulting inference table contain?

    1. Request and response payloads with timestamps and status codes Correct answer.
    2. A snapshot of the model artifacts and their library dependencies
    3. Rolled-up latency and error counts per endpoint, refreshed hourly
    4. The endpoint's traffic-split configuration history over time

    Explanation

    With payload logging enabled, the endpoint writes one row per request to a Unity Catalog Delta table containing the request body, the response body, and metadata such as timestamps and status codes. Teams query this table with SQL to debug and audit individual interactions. It is neither an aggregate metrics store nor a copy of model files.

  147. Question 147Deploy and monitor generative AI applications

    A team wants domain experts to score answers produced by a deployed RAG application so the scores can be reused as an evaluation dataset. Which Databricks capability is designed for this?

    1. The Review App, which collects expert labels on app responses Correct answer.
    2. Lakehouse Monitoring, which profiles the inference table on a schedule
    3. Model Serving traffic splitting, which compares two model versions
    4. AI Gateway payload logging, which stores requests for later auditing

    Explanation

    The Review App presents application responses to human reviewers who record judgments, and those labels can be exported as an evaluation dataset for MLflow evaluation. It is the human-in-the-loop complement to automated scoring.

  148. Question 148Deploy and monitor generative AI applications

    An application already uses the OpenAI Python client and must call a Databricks chat endpoint without changing its SDK. What must the application configure?

    1. Base URL pointing at the workspace's serving-endpoints path, endpoint name as model Correct answer.
    2. The base URL set to the MLflow tracking server, with the run ID used as the model name
    3. The workspace URL passed as the api_key, with a personal access token as the model name
    4. A local model artifact path, with the cluster driver's address used as the base URL

    Explanation

    Databricks serving endpoints expose an OpenAI-compatible route. The client sets base_url to the workspace URL followed by /serving-endpoints, passes the endpoint name as the model parameter, and uses a Databricks token as the API key.

  149. Question 149Deploy and monitor generative AI applications

    Two candidate model versions must be compared on the same dataset before either receives production traffic. Which approach fits this requirement?

    1. Deploy the newer version to production and watch the inference table for regressions
    2. Route live production traffic to both versions and compare error rates after a week
    3. Enable payload logging on the endpoint and inspect the raw responses manually later
    4. Score both with MLflow evaluation on the shared dataset, then compare metrics Correct answer.

    Explanation

    mlflow.evaluate scores a candidate model or predict function against a dataset and produces named metrics, so two versions can be compared on identical inputs before any promotion. This keeps unvalidated behavior away from real users.

  150. Question 150Deploy and monitor generative AI applications

    A team must block requests and responses that contain violent or hateful content before they reach users. Which AI Gateway configuration should they enable?

    1. A fallback route that sends failing requests to a second served model
    2. The safety guardrail, which classifies content and blocks unsafe results Correct answer.
    3. Payload logging that writes every request and response to a Unity Catalog table
    4. A per-endpoint rate limit that caps requests per minute for each caller

    Explanation

    AI Gateway guardrails run inside the request path, so enabling the safety guardrail classifies prompts and completions and blocks or flags unsafe content before it is returned. Rate limits, logging, and fallbacks address volume, audit, and availability instead.

  151. Question 151Deploy and monitor generative AI applications

    An external application must call a production serving endpoint using credentials that rotate automatically, without storing a long-lived personal access token. Which approach is recommended?

    1. A service principal with endpoint permission, using its OAuth client credentials Correct answer.
    2. Disable authentication on the serving endpoint so the application can call it anonymously
    3. Embed a long-lived personal access token directly in the container image at build time
    4. Use a workspace admin account and share the account password with the application team

    Explanation

    Service principals authenticate with OAuth machine-to-machine client credentials, and Databricks issues short-lived tokens that refresh without code changes; the client secret belongs in a secret scope. This follows least privilege because the principal is granted only query permission on the endpoint.

  152. Question 152Deploy and monitor generative AI applications

    A pay-per-token serving endpoint with autoscaling enabled adds replicas when demand rises. Which signal primarily drives the scale-out decision?

    1. The on-disk size of the registered model artifact stored in Unity Catalog
    2. Concurrent requests compared with the endpoint's concurrency target Correct answer.
    3. The number of notebooks that are attached to the serving compute cluster
    4. The number of rows written to the endpoint's inference table each hour

    Explanation

    Model Serving scales on concurrency, meaning the number of requests in flight relative to the configured concurrency target for a replica. Stored data size and artifact size affect load time, not the runtime scaling decision.

  153. Question 153Deploy and monitor generative AI applications

    A team must score 5 million stored documents with a deployed model overnight. Which approach is appropriate?

    1. Raise the endpoint timeout and send all documents in a single request
    2. Loop over the documents and send one synchronous REST request per document
    3. Enable scale-to-zero on the endpoint so idle capacity is released
    4. Use ai_query or a Spark batch job that applies the model to a DataFrame Correct answer.

    Explanation

    Large offline workloads are handled with ai_query or a Spark job that distributes batched requests across the cluster, which provides throughput and retry handling. Synchronous per-row calls do not scale to millions of records, and a single mega-request exceeds payload and context limits.

  154. Question 154Deploy and monitor generative AI applications

    An engineer wants every request and response from a serving endpoint written to a Delta table in Unity Catalog for later SQL analysis. What must be configured on the endpoint?

    1. An MLflow experiment where each individual request is logged as a run
    2. A system table prefix so that requests are written to audit logs
    3. A cluster log delivery path that points at a DBFS mount
    4. A destination catalog and schema for the inference table Correct answer.

    Explanation

    Payload logging is enabled in the endpoint configuration by naming the Unity Catalog catalog and schema where the inference table is created. The endpoint then writes one row per request to that table.

  155. Question 155Deploy and monitor generative AI applications

    A RAG application's answers must be checked for claims that are not supported by the retrieved documents. Which evaluation metric measures this?

    1. Token count, which measures how many completion tokens the request consumed overall
    2. Latency, which measures how long the retrieval and generation steps take to complete
    3. Chunk size, which measures how many tokens each retrieved document passage contains
    4. Groundedness, which judges whether the answer follows the retrieved context Correct answer.

    Explanation

    Groundedness judges whether the statements in an answer are supported by the retrieved context, which is how unsupported claims and hallucination relative to the corpus are detected. Latency, chunk size, and token count measure operational properties rather than factual support.

  156. Question 156Deploy and monitor generative AI applications

    A chat endpoint with scale-to-zero enabled responds in about 300 ms normally, but the first request after hours of inactivity takes several seconds. What explains this?

    1. The endpoint scaled to zero and must provision a replica before serving Correct answer.
    2. The AI Gateway rate limit delayed the request until the next minute
    3. The request exceeded the model's maximum context length
    4. The inference table was compacted and briefly locked the endpoint

    Explanation

    With scale-to-zero the endpoint releases all replicas when idle, so the first request after a quiet period pays model loading and provisioning time before it can be served. Keeping a minimum number of replicas avoids that cold start for latency-sensitive applications.

  157. Question 157Deploy and monitor generative AI applications

    A model version is already serving production traffic and the team must change the endpoint's workload size without interrupting requests. What happens when the endpoint configuration is updated?

    1. All in-flight requests fail and clients must retry after the update
    2. The change takes effect only after the next scheduled nightly restart
    3. The endpoint is deleted and must be recreated with the new configuration
    4. The update is applied as a rolling deployment while the endpoint keeps serving Correct answer.

    Explanation

    Endpoint configuration changes are applied as rolling updates: new capacity is started with the new settings and traffic moves over while existing replicas continue to serve. The endpoint URL stays stable and requests are not dropped because of the change.

  158. Question 158Deploy and monitor generative AI applications

    Median latency on a serving endpoint stays near 400 ms, but p99 latency climbs steadily during peak hours and requests wait before execution. Which explanation fits?

    1. The AI Gateway guardrails are rejecting a growing share of the incoming traffic
    2. The endpoint has hit its maximum replicas at the configured concurrency Correct answer.
    3. The inference table has grown too large for the endpoint to query efficiently
    4. The model artifacts are being downloaded again for each individual request

    Explanation

    A rising p99 with a stable median, together with requests queuing before execution, indicates saturation: concurrency has reached the target but the autoscaler cannot add replicas past its maximum, so excess requests wait. The fix is to raise the max replica count or adjust the workload size and concurrency settings.

  159. Question 159Deploy and monitor generative AI applications

    A trace of one RAG request shows the retrieval span at 400 ms, the LLM span at 3.5 s, and post-processing at 100 ms. Where should optimization effort go first?

    1. The retrieval span, since it is the first step in the pipeline
    2. The trace exporter, since it writes span data to the experiment
    3. The LLM span, since it dominates the request's total duration Correct answer.
    4. Post-processing, since it runs after the model responds

    Explanation

    Span durations in the trace attribute time to each stage; here the LLM call accounts for most of the roughly four-second request. Reducing generation time, for example with fewer output tokens, a faster model, or streaming, yields the largest improvement.

  160. Question 160Deploy and monitor generative AI applications

    A team plans to score production answers with an LLM judge and alert when the score drops. What should be done before the score is trusted for alerting?

    1. Alert on every score change and let responders decide whether it matters
    2. Reuse the same model as both the generator and the judge to save cost
    3. Raise the judge's temperature so that its scores vary more widely
    4. Compare judge scores with human-labeled examples and measure their agreement Correct answer.

    Explanation

    An LLM judge is itself a model with biases and blind spots, so its scores must be validated against a human-labeled gold set before they drive alerts. Measuring agreement shows whether the score tracks real quality and helps set a meaningful threshold.

  161. Question 161Governance, security and responsible AI for generative AI applications

    Which Unity Catalog privilege allows a user to invoke a Databricks model serving endpoint?

    1. CAN QUERY Correct answer.
    2. USE CATALOG
    3. CAN MANAGE
    4. CREATE MODEL

    Explanation

    Serving endpoints are securable objects in Unity Catalog, and the CAN QUERY privilege grants the ability to send requests to an endpoint. CAN MANAGE controls administration of the endpoint rather than invocation. Catalog-level and model-creation privileges do not by themselves permit querying an endpoint.

  162. Question 162Governance, security and responsible AI for generative AI applications

    An auditor asks which users invoked a GenAI serving endpoint and when. Which Databricks source records the caller identity for these API requests?

    1. The system.serving.endpoint_usage table, which records token counts for billing
    2. The driver logs of the cluster that created the endpoint
    3. The system.access.audit table, which records API calls and the identity of the caller Correct answer.
    4. The inference table for the endpoint, which stores request and response payloads

    Explanation

    Audit logs capture control-plane and data-plane API activity, including the identity of the principal making a serving request. Usage tables focus on consumption metrics such as token counts, while inference tables store payload content. Driver logs are not the authoritative record of endpoint invocations.

  163. Question 163Governance, security and responsible AI for generative AI applications

    Which AI Gateway guardrail evaluates model inputs and outputs against harmful-content categories?

    1. Safety filter Correct answer.
    2. Caching
    3. Rate limiting
    4. Fallback routing

    Explanation

    The safety guardrail inspects requests and responses for harmful content and can block them according to policy. Rate limiting controls request volume, caching reuses prior responses, and fallback routing provides availability when a model fails.

  164. Question 164Governance, security and responsible AI for generative AI applications

    What does the PII guardrail in AI Gateway do?

    1. It detects personally identifiable information in requests and responses and can block the request. Correct answer.
    2. It enforces row-level filters on the underlying training table.
    3. It encrypts request payloads before they reach the model.
    4. It rotates the endpoint's API credentials on a schedule.

    Explanation

    The PII guardrail inspects prompts and completions for personally identifiable information and can stop a request when the configured policy is violated. Encryption, row-level filtering, and credential rotation are separate security controls. The guardrail is a detection and enforcement mechanism at the gateway layer.

  165. Question 165Governance, security and responsible AI for generative AI applications

    A team wants to cap how many requests a single user can send to an endpoint per minute. Which AI Gateway feature should they configure?

    1. Guardrails
    2. Payload logging
    3. Rate limits Correct answer.
    4. Fallback

    Explanation

    AI Gateway rate limits constrain request volume and can be applied at the endpoint and per-user levels. Guardrails evaluate content, payload logging stores request and response data, and fallback provides alternate routing. Rate limiting is the control designed to cap request frequency.

  166. Question 166Governance, security and responsible AI for generative AI applications

    Within AI Gateway configuration, which feature writes full request and response payloads to a Unity Catalog Delta table for later inspection?

    1. Usage tracking
    2. Safety guardrail
    3. Caching
    4. Payload logging Correct answer.

    Explanation

    Payload logging captures the request and response bodies and stores them in a Delta table that can be governed with Unity Catalog. Usage tracking records metrics such as token counts, caching reuses prior responses, and the safety guardrail evaluates content. Payload logging is the feature intended for detailed request and response inspection.

  167. Question 167Governance, security and responsible AI for generative AI applications

    What is the primary purpose of inference tables for a model serving endpoint?

    1. To store the model weights in Unity Catalog
    2. To capture request and response data in a Delta table for monitoring and debugging Correct answer.
    3. To rotate service principal credentials
    4. To enforce column masks on the endpoint's input

    Explanation

    Inference tables automatically capture request and response payloads from a serving endpoint into a Delta table. This supports monitoring, debugging, and quality analysis of deployed models. They do not store model weights, enforce column masks, or manage credentials.

  168. Question 168Governance, security and responsible AI for generative AI applications

    How should a GenAI application store a third-party LLM API key that it uses from a Databricks notebook?

    1. Hardcode the key in the notebook so all users can run it.
    2. Store the key in a Databricks secret scope and reference it at runtime. Correct answer.
    3. Set the key as a widget's default value in the notebook.
    4. Commit the key to the workspace Git repository with restricted access.

    Explanation

    Databricks secret scopes store credentials securely and allow notebooks and jobs to reference them at runtime without exposing the value. Hardcoding, widget defaults, and repository commits expose the key in code or configuration history. Secret scopes integrate with access controls and auditability.

  169. Question 169Governance, security and responsible AI for generative AI applications

    An agent calls a Unity Catalog function as a tool. Which privilege must the agent's service principal have on the function?

    1. MODIFY
    2. EXECUTE Correct answer.
    3. SELECT
    4. CREATE FUNCTION

    Explanation

    Unity Catalog functions require the EXECUTE privilege for a principal to invoke them. SELECT applies to tables and views, MODIFY controls alteration of data objects, and CREATE FUNCTION allows creation of new functions. The agent needs EXECUTE on each function it uses as a tool.

  170. Question 170Governance, security and responsible AI for generative AI applications

    A RAG application calls a Unity Catalog function and queries Unity Catalog tables on behalf of the signed-in user rather than a shared service principal. What is the main governance benefit?

    1. The model can see all rows regardless of the user's grants.
    2. The application can bypass row filters for faster retrieval.
    3. Audit logs no longer need to record data access.
    4. The user's existing Unity Catalog privileges determine which data the application can retrieve. Correct answer.

    Explanation

    When an application acts on behalf of the end user, Unity Catalog enforces that user's existing privileges, row filters, and column masks. This prevents the application from becoming a privilege-escalation path to data the user cannot otherwise access. Audit logs still record access, and row filters are not bypassed.

  171. Question 171Governance, security and responsible AI for generative AI applications

    Which statement about Vector Search indexes in Unity Catalog is correct?

    1. They are securable objects whose access is controlled by Unity Catalog privileges. Correct answer.
    2. They inherit no permissions and are public by default.
    3. They can be queried by any workspace user once the endpoint is created.
    4. They are governed only by cluster-level access controls.

    Explanation

    Vector Search indexes are Unity Catalog securable objects, so access is granted through Unity Catalog privileges rather than being open to all workspace users. Creating an endpoint does not make every index queryable by everyone. Governance is centralized in Unity Catalog rather than in cluster-level settings.

  172. Question 172Governance, security and responsible AI for generative AI applications

    Databricks Assistant in a notebook generates SQL for a user. Whose permissions govern the data that the Assistant can access?

    1. A system-owned service principal's permissions
    2. The current user's permissions Correct answer.
    3. The metastore owner's permissions
    4. The workspace administrator's permissions

    Explanation

    Databricks Assistant operates within the context of the signed-in user, so its generated queries and data access are bounded by that user's Unity Catalog privileges. It does not elevate the user to administrator or metastore-owner access. This keeps AI-assisted authoring consistent with existing access controls.

  173. Question 173Governance, security and responsible AI for generative AI applications

    An administrator wants to prevent users from sending notebook code to partner-powered AI features. Which action should the administrator take?

    1. Revoke CAN QUERY on all serving endpoints.
    2. Delete the workspace's secret scopes.
    3. Disable partner-powered AI features in the workspace admin settings. Correct answer.
    4. Turn off audit log delivery to the metastore.

    Explanation

    Partner-powered AI features can be controlled through workspace admin settings, allowing an administrator to disable the data-sharing behavior. Revoking endpoint privileges affects serving access but does not address partner-powered features. Secrets and audit logs serve different security purposes.

  174. Question 174Governance, security and responsible AI for generative AI applications

    When configuring an external model in Unity Catalog, where should the model provider's API key be stored?

    1. In a notebook widget
    2. In the model version's description field
    3. In a Unity Catalog connection's credentials Correct answer.
    4. In the serving endpoint's URL path

    Explanation

    External models use Unity Catalog connections to store provider credentials securely, and access to the connection is governed by Unity Catalog privileges. The API key should not appear in descriptions, URLs, or notebook widgets. This design keeps secrets out of model metadata and application code.

  175. Question 175Governance, security and responsible AI for generative AI applications

    Which consideration applies when using an external model instead of a Databricks Foundation Model API?

    1. Data is sent to the external provider and is subject to that provider's data-handling terms. Correct answer.
    2. External models do not require any credentials.
    3. External models cannot be governed by Unity Catalog.
    4. External models do not support rate limiting or usage tracking.

    Explanation

    An external model routes prompts and responses to a third-party provider, so the organization must evaluate the provider's data retention and usage policies. Unity Catalog can still govern the external model object and its connection. Credentials are required, and gateway controls such as rate limits and usage tracking can still apply.

  176. Question 176Governance, security and responsible AI for generative AI applications

    Which system table provides token usage and request counts for Databricks serving endpoints?

    1. system.serving.endpoint_usage Correct answer.
    2. system.information_schema.tables
    3. system.billing.usage
    4. system.access.audit

    Explanation

    The system.serving.endpoint_usage table records serving endpoint consumption metrics such as request counts and token usage. The audit table records activity and identity, billing usage records billable consumption, and information_schema describes metadata objects. Usage monitoring for endpoints should query the serving usage table.

  177. Question 177Governance, security and responsible AI for generative AI applications

    A production GenAI application authenticates to Databricks with a service principal. Which access strategy follows least privilege?

    1. Share one developer's personal access token with the service principal.
    2. Grant only the Unity Catalog privileges and endpoint CAN QUERY that the application needs. Correct answer.
    3. Embed the service principal's secret in the application code.
    4. Grant the service principal account admin so it can access all data.

    Explanation

    Least privilege means granting the service principal only the specific Unity Catalog privileges and endpoint permissions required for its function. Account admin access is far broader than necessary and increases blast radius. Shared personal tokens and embedded secrets also weaken accountability and security.

  178. Question 178Governance, security and responsible AI for generative AI applications

    For server-to-server calls to a serving endpoint, which authentication method is preferred over long-lived personal access tokens?

    1. OAuth machine-to-machine authentication with a service principal Correct answer.
    2. Anonymous access to the endpoint
    3. HTTP basic authentication with a shared username and password
    4. A long-lived personal access token embedded in the client

    Explanation

    OAuth machine-to-machine authentication lets a service principal obtain short-lived tokens and is the recommended pattern for automated workloads. Long-lived personal access tokens are tied to a user and are harder to rotate safely. Basic authentication and anonymous access are not appropriate for governed serving endpoints.

  179. Question 179Governance, security and responsible AI for generative AI applications

    A GenAI application drafts prior-authorization letters for a healthcare payer. Which responsible AI control is most appropriate before a letter is sent?

    1. Automatically send the letter without review to reduce latency.
    2. Disable payload logging so drafts are not stored.
    3. Require a qualified human to review and approve each letter before sending. Correct answer.
    4. Use the largest available model for every request.

    Explanation

    High-stakes healthcare communications warrant human oversight so that errors, omissions, or inappropriate content are caught before external impact. Human-in-the-loop review is a core responsible AI control for consequential decisions and communications. Model size and logging choices do not substitute for accountable review.

  180. Question 180Governance, security and responsible AI for generative AI applications

    When the AI Gateway safety guardrail detects unsafe content in a request or response, what does it do?

    1. It escalates the request to a workspace administrator.
    2. It retries the request against a fallback model.
    3. It caches the request for later reuse.
    4. It blocks the request or response and returns a guardrail message. Correct answer.

    Explanation

    The safety guardrail enforces content policy by blocking the request or response when unsafe content is detected and returning a guardrail indication. It does not cache content, create an administrator escalation ticket, or automatically retry against another model. Blocking is the enforcement action that prevents unsafe content from proceeding.

  181. Question 181Governance, security and responsible AI for generative AI applications

    Which AI Gateway feature routes traffic to a secondary model if the primary model is unavailable?

    1. Payload logging
    2. Rate limiting
    3. Fallback routing Correct answer.
    4. Caching

    Explanation

    Fallback routing defines an alternate model or route so requests can still be served when the primary model fails or is unavailable. Caching reuses prior responses, rate limiting constrains request volume, and payload logging records request and response data. Fallback is the availability feature for model failure scenarios.

  182. Question 182Governance, security and responsible AI for generative AI applications

    A team enables AI Gateway caching on an endpoint that handles customer PII. What risk should they evaluate?

    1. Caching requires the endpoint to be publicly accessible.
    2. Caching prevents audit logs from recording endpoint calls.
    3. Cached responses may be served to a different user and expose another customer's data. Correct answer.
    4. Caching disables the endpoint's safety guardrail.

    Explanation

    Caching can return a previously generated response for a matching request, so prompts containing sensitive or user-specific data may expose one user's information to another if cache scope is not carefully controlled. Caching does not disable guardrails, suppress audit logs, or require public accessibility. The key governance question is whether cached content can cross user or tenant boundaries.

  183. Question 183Governance, security and responsible AI for generative AI applications

    In Unity Catalog, what does lineage for a registered model version primarily show?

    1. The endpoint's configured rate limit
    2. The browser session of the user who deployed the model
    3. The API key used to call the model
    4. The upstream data and notebooks used to produce the model Correct answer.

    Explanation

    Unity Catalog lineage tracks the data and code artifacts that contributed to a model version, helping teams trace how models were produced and what data they depend on. It does not expose API keys, endpoint rate limits, or browser sessions. Lineage supports impact analysis and governance reviews for GenAI assets.

  184. Question 184Governance, security and responsible AI for generative AI applications

    Payload logging is enabled for a customer-support endpoint. What governance control should be applied to the resulting log table?

    1. Grant public SELECT so support agents can inspect it.
    2. Disable audit logging for the endpoint to reduce noise.
    3. Store the table in the DBFS root for easy access.
    4. Apply Unity Catalog privileges, masking where needed, and a retention policy. Correct answer.

    Explanation

    Payload logs can contain sensitive customer content, so the resulting Delta table should be protected with Unity Catalog privileges, column masks or row filters where appropriate, and a defined retention policy. Public grants and broad DBFS locations would expose sensitive data. Audit logging should remain enabled for accountability.

  185. Question 185Governance, security and responsible AI for generative AI applications

    A Unity Catalog table used by a RAG pipeline has a column mask on the email column. The application queries the table using the end user's credentials. What does the retrieval step return for that column?

    1. The unmasked email because AI functions bypass column masks.
    2. The mask is ignored unless the user is a workspace admin.
    3. The masked value that the user's policy allows. Correct answer.
    4. An error because column masks are incompatible with RAG.

    Explanation

    Unity Catalog column masks are enforced at query time based on the querying principal's identity, so retrieval under the end user's credentials returns only the masked value that user is permitted to see. AI functions and RAG pipelines do not automatically bypass Unity Catalog masks. This prevents sensitive values from being embedded or sent to a model.

  186. Question 186Governance, security and responsible AI for generative AI applications

    A GenAI application must call a production model serving endpoint. Which permission on the endpoint lets the application's service principal invoke it without granting management rights?

    1. CAN MANAGE
    2. SELECT
    3. CAN QUERY Correct answer.
    4. EXECUTE

    Explanation

    Serving endpoints use endpoint permissions, and CAN QUERY allows a principal to invoke the endpoint. CAN MANAGE is broader because it also allows editing and deleting the endpoint. EXECUTE and SELECT apply to other Unity Catalog securables, not to invoking a serving endpoint.

  187. Question 187Governance, security and responsible AI for generative AI applications

    A company deploys a customer-facing chatbot powered by a large language model. Which practice best supports transparency for responsible AI?

    1. Disable request logging to protect user privacy.
    2. Increase the model temperature to make answers seem more human.
    3. Hide the model provider name to prevent external scrutiny.
    4. Disclose to users that they are interacting with an AI system. Correct answer.

    Explanation

    Transparency means users should understand when they are interacting with an AI system rather than a human. Clear disclosure supports informed use and aligns with responsible AI principles. Hiding provider details, disabling logging, or changing temperature does not provide meaningful transparency.

  188. Question 188Governance, security and responsible AI for generative AI applications

    A production GenAI job runs as a service principal and must read an API key from a Databricks secret scope. Which access should be granted for least privilege?

    1. Embed the API key directly in the job's notebook.
    2. Grant READ on the secret scope to all workspace users.
    3. Grant READ on the secret scope to the service principal. Correct answer.
    4. Grant MANAGE on the secret scope to the service principal.

    Explanation

    Secret scope ACLs distinguish READ, WRITE, and MANAGE. READ allows a principal to read secrets without administering the scope, which is the minimum needed by the job. MANAGE and broad workspace access exceed least privilege, and embedding secrets in code is insecure.

  189. Question 189Governance, security and responsible AI for generative AI applications

    Which Databricks system table should an auditor query to review who accessed a model serving endpoint?

    1. The audit log system table Correct answer.
    2. The lineage system table
    3. The compute system table
    4. The billing usage system table

    Explanation

    The audit log system table records governance-relevant events, including access to workspace and account resources. Auditors use it to review who performed actions such as querying serving endpoints. Billing, lineage, and compute system tables answer different questions.

  190. Question 190Governance, security and responsible AI for generative AI applications

    An externally exposed GenAI application calls a Foundation Model API endpoint. Which AI Gateway control limits the number of requests a single user can send in a time window?

    1. Payload logging
    2. Caching
    3. Rate limiting Correct answer.
    4. Fallback

    Explanation

    AI Gateway rate limiting restricts request volume over a time window and can help protect endpoints from abuse or accidental overuse. Caching reuses responses, fallback redirects traffic when a model is unavailable, and payload logging records request and response data.

  191. Question 191Governance, security and responsible AI for generative AI applications

    AI Gateway PII detection is enabled on a customer-support endpoint. What governance outcome does this feature primarily provide?

    1. It flags sensitive data in prompts or responses for filtering or redaction. Correct answer.
    2. It prevents all prompt injection attacks by automatically rewriting the system prompt before each call.
    3. It encrypts every Delta table that stores the endpoint's inference logs.
    4. It automatically deletes all historical inference tables after a fixed retention period.

    Explanation

    PII detection identifies sensitive information in request or response content so governance controls can filter, redact, or monitor it. It is not an encryption mechanism, a complete prompt-injection defense, or a retention deletion policy. Those outcomes require separate controls.

  192. Question 192Governance, security and responsible AI for generative AI applications

    What is the primary purpose of a model card for a GenAI application?

    1. To store the provider API key so the endpoint can authenticate to the model.
    2. To grant CAN QUERY permission to application users calling the endpoint.
    3. To define autoscaling limits and concurrency settings for the serving endpoint.
    4. To document intended use, limitations, and evaluation results. Correct answer.

    Explanation

    A model card documents key governance information such as intended use, known limitations, evaluation results, and ethical considerations. It helps consumers understand appropriate and inappropriate use of the model. It is not a credential store, an autoscaling configuration, or an access-control mechanism.

  193. Question 193Governance, security and responsible AI for generative AI applications

    Before launch, a team wants to measure whether a RAG assistant's answers are factually supported. Which evaluation practice is most appropriate?

    1. Disable guardrails so the raw model behavior can be observed.
    2. Compare answers against a labeled ground-truth evaluation set. Correct answer.
    3. Increase the number of retrieved chunks for every user query.
    4. Enable response caching to reduce latency for repeated questions.

    Explanation

    Factual-support evaluation requires a reference standard, such as a labeled ground-truth set or expert-reviewed answers, against which generated answers can be compared. Retrieval changes, guardrail changes, and caching affect behavior or performance but do not by themselves measure factual support.

  194. Question 194Governance, security and responsible AI for generative AI applications

    A RAG chatbot retrieves text from user-uploaded documents. Which design choice best reduces the risk of prompt injection from retrieved content?

    1. Remove citations so users cannot see the retrieved source.
    2. Raise the model temperature to make injected instructions less predictable.
    3. Treat retrieved text as untrusted data, separate from system instructions. Correct answer.
    4. Concatenate retrieved text into the system prompt so the model follows it.

    Explanation

    Retrieved content can contain adversarial instructions, so it should be treated as untrusted data rather than as trusted system-level instructions. Keeping it separate from system instructions reduces the chance that document text overrides application policy. Raising temperature or removing citations does not address the injection risk.

  195. Question 195Governance, security and responsible AI for generative AI applications

    A governance team wants to mark a Unity Catalog table used by a RAG pipeline as containing sensitive personal data. Which Unity Catalog feature is designed for this classification?

    1. A model version alias
    2. A serving endpoint rate limit
    3. A tag on the table or column Correct answer.
    4. A notebook workspace path

    Explanation

    Unity Catalog tags can be applied to securables such as tables and columns to classify data, including marking sensitive or personal data. Tags support governance, discovery, and policy workflows. Model aliases, rate limits, and workspace paths do not classify table data.

  196. Question 196Governance, security and responsible AI for generative AI applications

    A developer accidentally prints a Databricks secret value in notebook output. Which Databricks capability reduces the chance that the secret is displayed in clear text?

    1. Inference tables
    2. Secret redaction Correct answer.
    3. Model registry stages
    4. AI Gateway caching

    Explanation

    Databricks secret redaction helps prevent secret values from appearing in notebook output and logs. This reduces accidental exposure when secrets are printed or included in command results. Inference tables, model stages, and caching serve different purposes.

  197. Question 197Governance, security and responsible AI for generative AI applications

    A team wants deployments to reference an approved model version through a stable name rather than a version number. Which Unity Catalog model registry feature should they use?

    1. Inference table schema evolution
    2. Vector Search index sync
    3. Endpoint autoscaling
    4. Model version aliases Correct answer.

    Explanation

    Model version aliases provide stable names, such as a production alias, that can point to a specific model version. After governance approval, the alias can be moved to the approved version so deployments do not need to change version numbers. Autoscaling, inference table schema evolution, and vector index sync are unrelated to model version promotion.

  198. Question 198Governance, security and responsible AI for generative AI applications

    What is the primary purpose of red teaming a GenAI application before production release?

    1. To configure OAuth machine-to-machine credentials for the serving endpoint.
    2. To reduce serving endpoint cost by lowering the deployed model size.
    3. To probe adversarially for harmful behavior and inform mitigations. Correct answer.
    4. To generate embeddings for the Vector Search index used by the application.

    Explanation

    Red teaming uses adversarial testing to uncover harmful, biased, unsafe, or otherwise undesirable model behavior before launch. Findings are used to improve guardrails, prompts, evaluation, and monitoring. It is not a cost-optimization, embedding-generation, or authentication configuration activity.

  199. Question 199Governance, security and responsible AI for generative AI applications

    A GenAI model is used to screen job applicants. Which evaluation approach best tests for fairness across demographic groups?

    1. Measure average latency for all users of the application.
    2. Compare metrics on labeled data disaggregated by demographic group. Correct answer.
    3. Cache responses to reduce the number of repeated evaluations.
    4. Increase the model temperature until outputs vary across repeated runs.

    Explanation

    Fairness evaluation requires comparing outcome metrics across relevant groups using labeled or audited data. Disaggregating metrics by demographic group can reveal disparate performance or selection rates. Latency, temperature, and caching do not measure fairness.

  200. Question 200Governance, security and responsible AI for generative AI applications

    A team is designing prompts for a GenAI application that handles customer records. Which practice best follows data minimization for responsible AI?

    1. Disable all logging and monitoring to avoid any data exposure.
    2. Store all prompts indefinitely so they can be used for future model training.
    3. Send only the fields needed for the task and redact unnecessary PII. Correct answer.
    4. Include the entire customer record in every prompt to maximize context.

    Explanation

    Data minimization means limiting the personal data sent to a model to what is necessary for the task and removing unnecessary sensitive fields. This reduces privacy and compliance risk. Sending entire records or retaining prompts indefinitely increases exposure, while disabling all monitoring weakens accountability.

Databricks GenAI Engineer flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Foundation Model APIs
Databricks-hosted endpoints providing pay-per-token access to models such as Llama, DBRX, and MPT, callable through an OpenAI-compatible API within the workspace.
ChatDatabricks LangChain class
A LangChain chat model wrapper from the databricks-langchain package that sends requests to a Databricks Foundation Model or serving endpoint by name.
Model Serving endpoint
A Databricks resource that hosts models or foundation model APIs behind a REST API, supporting autoscaling, concurrency, and optional GPU hardware.
Provisioned throughput
A serving mode that reserves model capacity for consistent latency and throughput, billed per hour rather than per token.
RAG pipeline
Retrieval-augmented generation: embed and index documents, retrieve relevant chunks for a query, then inject them into the prompt sent to an LLM.
Agent Framework
Mosaic AI Agent Framework for building, deploying, and evaluating LLM agents with tool-calling and retrieval, integrated with MLflow tracing.
OpenAI-compatible client
Databricks serving endpoints expose /chat/completions; point the OpenAI SDK base_url to the workspace serving endpoint and authenticate with a Databricks token.
Delta Sync index
A Vector Search index that automatically syncs embeddings from a Delta table, keeping the index current as source data changes.
Direct Vector Access
A Vector Search index type where vectors are inserted and updated manually via API rather than synced from Delta.
Chunking
Splitting documents into smaller passages before embedding; chunk size and overlap trade retrieval precision against context completeness.
Embedding model
A model converting text to dense vectors; the same model must be used for indexing and query time to keep vector spaces aligned.
ai_parse_document
Databricks SQL AI function that extracts text and structure from PDFs, images, and documents for downstream RAG ingestion.
Change Data Capture
Reading Delta table change feeds to incrementally update embeddings instead of reprocessing the entire corpus.
Vector Search index metadata
Additional Delta columns stored alongside vectors, filterable at query time to restrict retrieval by attributes like source or date.
MLflow evaluate
An MLflow API that runs an evaluation dataset through a model or agent and computes scorers such as correctness, relevance, and groundedness.
LLM-as-judge
An evaluation technique where a language model scores outputs against criteria or reference answers using a rubric prompt.
Groundedness scorer
A metric measuring whether an answer is supported by retrieved context, flagging hallucinations in RAG responses.
Foundation Model Fine-tuning
Databricks supervised fine-tuning of supported models using instruction-response pairs, producing a new model version registered in Unity Catalog.
Evaluation dataset
A structured collection of inputs and optional expectations used to reproducibly score model or agent behavior over time.
Review App
A Databricks labeling interface where domain experts review traces and provide feedback used to improve and evaluate agents.
Inference tables
Delta tables that log requests and responses for a serving endpoint, enabling monitoring, debugging, and quality analysis.
Scale to zero
A serving endpoint setting that scales replicas down to zero when idle, reducing cost at the expense of cold-start latency.
Traffic splitting
Routing a percentage of endpoint requests to different served model versions to support canary releases and A/B tests.
Lakehouse Monitoring
Databricks profiling and drift detection for tables, including inference tables, with configurable metrics and refresh schedules.
Databricks Asset Bundles
A declarative packaging format for jobs, pipelines, and serving endpoints that supports versioned deployment across environments.
System tables
Account-level Delta tables such as system.serving and system.billing.usage that expose endpoint logs, usage, and cost for monitoring.
AI Gateway
A governance layer on serving endpoints providing usage tracking, rate limits, payload logging, and guardrails such as PII and content safety filters.
Unity Catalog
A unified governance layer for data, models, and functions providing fine-grained grants, lineage, and audit across the lakehouse.
PII redaction
Detecting and masking personal identifiers in prompts or logs using functions like ai_mask or presidio before storage or model calls.
Audit logs
Audit logs in system.access.audit record who accessed or modified data, models, and endpoints, supporting compliance investigations and incident response.
Content safety filter
A guardrail that screens model inputs and outputs for harmful categories such as hate, violence, or self-harm and blocks or flags them.
Service principal
A non-human identity used by applications to authenticate to Databricks, granted least-privilege access to data and endpoints.

Databricks GenAI Engineer study guide

Design and build generative AI applications

Start with the interface and the data path

A Databricks generative AI application is an endpoint-consuming client plus governed data and models. Know the building blocks: Foundation Model APIs (pay-per-token or provisioned throughput), custom Model Serving endpoints for fine-tuned or packaged models, Vector Search endpoints, and AI Functions such as ai_query for SQL inference. Applications call endpoints through OpenAI-compatible clients, LangChain, or Databricks SDKs. Prompts are templates with system instructions, few-shot examples, and retrieved context.

Design choices

Decide synchronous request/response versus batch inference over a Delta table, and single-turn versus agentic tool use. For agents, define tools, state, and a stopping condition. Separate retrieval from generation so each can be evaluated and tuned independently. Keep models, indexes, and tables in Unity Catalog so permissions and lineage follow the application.

Traps

  • A serving endpoint is not a cluster; you do not attach a notebook to it.
  • Pay-per-token Foundation Model APIs are good for prototyping but have rate limits and no reserved capacity. Provisioned throughput reserves capacity for predictable latency and cost at volume.
  • Endpoint names are not model names; verify the exact endpoint you call.
  • Do not hard-code tokens; use secrets or service credentials.
  • Tool-using agents fail silently when tool schemas are ambiguous.

Rule of thumb

Prototype with pay-per-token APIs; move to provisioned throughput when traffic is steady or latency SLOs matter. Example: a support assistant at 50 requests/minute with a 2-second SLO should use a provisioned-throughput endpoint and cache frequent answers, not a pay-per-token endpoint.

Prepare data for retrieval augmented generation (RAG)

From documents to retrievable chunks

RAG quality is mostly a data-engineering problem. Ingestion parses files (PDF, HTML, Office), cleans text, and writes to a Delta table with metadata such as source, title, section, timestamp, and access tags. Chunking splits documents into passages; embeddings convert each chunk into a vector stored alongside the text. Databricks Vector Search builds an index from a Delta table (Delta Sync) or directly from vectors, and refreshes continuously or on a trigger.

What to get right

  • Chunking: respect document structure such as headings, tables, and code. Typical chunks are 200-800 tokens with 10-20% overlap; smaller chunks improve precision, larger chunks improve context.
  • Embedding consistency: embed queries with the same model and version used for chunks; changing models requires re-embedding and re-indexing.
  • Metadata: keep source and access fields for filtering and citations.
  • Hybrid search: combine vector similarity with keyword search for exact terms, IDs, and acronyms.
  • Permissions: index permissions and underlying table permissions are separate; an app still needs data access governed in Unity Catalog.

Traps

Assuming an index is fresh after a table update without checking refresh mode; storing chunks without parent document IDs, which breaks citations; fixed character chunking that splits tables; embedding queries with a different model than the index.

Worked example

For a 500-page policy manual: parse to text, split by heading into sections, then into about 500-token chunks with 50-token overlap; store doc_id, section, and effective_date; embed with the same endpoint used at query time; create a Delta Sync index with continuous refresh and a metadata filter on effective_date. At query time, retrieve top-k, optionally rerank, and pass chunks with citations to the LLM.

Develop, tune and evaluate generative AI applications

Build the evaluation set before tuning

Generative AI development is iterative: prompt, evaluate, inspect failures, then change data, prompt, retrieval, or model. Create a small, representative evaluation set with expected answers or rubrics plus edge cases. Use MLflow evaluation to score retrieval and generation separately: retrieval recall and precision, answer correctness, groundedness or faithfulness, relevance, and safety. LLM-as-a-judge scales well but must be calibrated with human labels; keep a human-reviewed subset.

Tuning options

Prompt engineering (instructions, few-shot examples, output format), retrieval tuning (chunk size, k, filters, reranking), and model tuning (fine-tuning or LoRA) are progressively more expensive. Fine-tuning teaches style, format, and domain behavior; it does not reliably add fresh facts, so use RAG for knowledge. For agents, use agent evaluation with traces to inspect tool calls, retrieval steps, and failure points.

Traps

  • Judging with the same model and prompt that produced the answer without a rubric, creating self-preference bias.
  • Tuning before establishing a baseline, so improvement cannot be attributed.
  • Leaking evaluation questions into training or few-shot examples.
  • Optimizing fluency while groundedness drops.
  • Reporting one aggregate score; segment results by question type.

Rule of thumb

Baseline first, change one variable at a time, and require both a quality gain and no safety regression before shipping. Example: if groundedness is low but retrieval recall is high, the prompt is likely ignoring context; if recall is low, fix chunking or increase k.

Deploy and monitor generative AI applications

Serving and observing

Deploy the application's model or chain to a Model Serving endpoint with autoscaling and concurrency limits. Enable inference tables to capture request and response payloads into a Unity Catalog Delta table; this is the foundation for monitoring, debugging, and quality analysis. Payload logging can be large and may contain sensitive data, so sample and govern it. Use Lakehouse Monitoring for drift and statistical profiles, and usage tracking for token counts and cost.

What to monitor

Track latency (p50 and p95), error rate, throughput, throttling and rate-limit events, token usage and cost, retrieval hit rate, and quality signals from user feedback and periodic judge scoring on sampled traffic. Close the loop: collect feedback, label a sample, and feed confirmed failures back into the evaluation set.

Traps

  • Treating deployment as finished when the endpoint is green; without payload logs you cannot diagnose quality regressions.
  • Monitoring only infrastructure: a healthy endpoint can return confidently wrong answers.
  • Forgetting that inference tables must be enabled and that schema changes can break downstream monitors.
  • Autoscaling cold starts causing latency spikes; keep minimum capacity for SLOs.

Worked example

For a RAG assistant, enable inference tables and log question, retrieved document IDs, answer, and latency. Sample 5% of traffic for nightly judge scoring on groundedness. Alert when p95 latency exceeds 3 seconds or groundedness drops below baseline. Route thumbs-down feedback to a review app and add confirmed failures to the evaluation set.

Governance, security and responsible AI for generative AI applications

Governance is part of the architecture

Every component, including source documents, Delta tables, Vector Search indexes, models, endpoints, and inference logs, should live in Unity Catalog with owners, tags, and lineage. Grant least privilege: consumers need SELECT or EXECUTE on what they use, not ownership. Use service credentials and secrets for external access; never embed tokens in notebooks. Classify data as PII or confidential and apply row filters, column masks, and dynamic views before data reaches prompts.

Responsible AI controls

Use AI Gateway guardrails on serving endpoints for PII detection and redaction, safety and content filters, rate limits, and payload logging. Add application-level controls: input validation, prompt-injection defenses, output filtering, citation requirements, and human review for high-impact actions. Keep audit logs and document model cards, evaluation results, and intended use. For agents with write actions, require approval steps or narrowly scoped tools.

Traps

  • Assuming index access equals table access; both must be granted.
  • Guardrails that add latency or block legitimate content; test thresholds.
  • Logging raw prompts containing PII into broadly readable tables.
  • Confusing endpoint permissions with model or data permissions.
  • No retention or deletion policy for logged payloads.

Rule of thumb

Default deny, then grant the narrowest object and action; redact PII before the model, not after. Example: a RAG application over HR data should query a filtered view, use a service credential scoped to that view, and enable PII filters on the endpoint.

Databricks GenAI Engineer cheat sheet

Quick reference

Design and build generative AI applications

  • Foundation Model APIs bill per token; provisioned throughput bills per hour and reserves capacity for latency-sensitive workloads.

  • Serving endpoints expose an OpenAI-compatible /chat/completions route; set base_url to https:///serving-endpoints and authenticate with a Databricks token.

  • WorkspaceClient().serving_endpoints.get_open_ai_client() returns an OpenAI client preconfigured for the workspace.

  • LangChain integration uses ChatDatabricks from databricks-langchain, passing the endpoint name and optional temperature and max_tokens.

  • Custom models deploy via MLflow pyfunc or logged transformers models; GPU endpoints are needed for large models, CPU for small classifiers.

  • RAG decision rule: retrieve top-k chunks (commonly 3-10), insert them into the prompt, and instruct the model to answer only from the provided context to reduce hallucination.

  • Streaming responses use stream=True; tokens arrive as server-sent events, reducing perceived latency for chat interfaces.

  • Agent tools are Python functions described with docstrings; the LLM selects tools by name and arguments, so precise descriptions improve tool selection.

  • Model Serving supports scale-to-zero, concurrency, and autoscaling; choose provisioned throughput when pay-per-token latency is too variable.

Quick reference

Prepare data for retrieval augmented generation (RAG)

  • A Delta Sync index automatically updates from a Delta source table; Direct Vector Access requires manual upserts and deletes.

  • Vector Search requires a primary key column, an embedding column of type array, and optional metadata columns for filtering.

  • The embedding model used at index time must match the model used at query time; changing models requires re-embedding and reindexing.

  • Chunk size is commonly 200-1000 tokens with 10-20% overlap; smaller chunks improve retrieval precision, larger chunks preserve context.

  • Use delta readChangeFeed or a streaming table to incrementally embed only new or changed rows rather than recomputing the full corpus.

  • ai_parse_document extracts text from PDFs and images; normalize, deduplicate, and remove boilerplate before chunking.

  • Batch embedding runs with ai_query or pandas UDFs; persist embeddings to Delta so downstream indexes read a single source of truth.

  • Filter metadata columns at query time to constrain retrieval by source, department, or date, improving relevance and supporting access control.

  • Built-in embedding endpoints include databricks-bge-large-en; dimension mismatch with the index configuration causes upsert failures.

Quick reference

Develop, tune and evaluate generative AI applications

  • mlflow.evaluate with model_type='databricks-agent' runs built-in scorers including relevance, groundedness, safety, and retrieval metrics.

  • LLM-as-judge metrics are defined with make_genai_metric, specifying a judge model, a rubric, and examples.

  • Evaluation datasets pair inputs with optional expectations; store them in Delta for versioned, reproducible comparisons.

  • Fine-tuning on Foundation Model APIs uses instruction/response JSONL; supported families include Llama and MPT, and LoRA reduces trainable parameters.

  • MLflow tracing records each agent step (retrieval, tool call, LLM call) so failures can be attributed to a specific stage.

  • Prompt patterns: a system prompt sets role and constraints; few-shot examples improve format adherence; chain-of-thought improves multi-step reasoning.

  • Compare runs in MLflow only when the same metric names and dataset version are logged, because scores are otherwise not comparable.

  • Human feedback collected in the Review App becomes labeled data for evaluation or fine-tuning.

Quick reference

Deploy and monitor generative AI applications

  • Enable inference tables on a serving endpoint to log request and response payloads to a Delta table.

  • Traffic splitting routes a percentage of requests to each served model version; use it for canary releases and A/B tests.

  • Provisioned throughput endpoints reserve capacity; use them when p95 latency or throughput must be guaranteed.

  • Lakehouse Monitoring profiles inference tables for drift, data quality, and custom metrics; the refresh schedule controls cost.

  • system.serving and system.billing.usage system tables expose endpoint usage and cost for chargeback and alerting.

  • Databricks Asset Bundles deploy jobs, pipelines, and endpoints via databricks bundle deploy with environment-specific targets.

  • Scale-to-zero reduces idle cost but adds cold-start latency; keep min replicas greater than zero for latency-sensitive endpoints.

  • Alert on monitoring metrics via SQL alerts or scheduled jobs that query the monitoring output tables.

Quick reference

Governance, security and responsible AI for generative AI applications

  • Unity Catalog governs models, functions, and endpoints; GRANT EXECUTE ON FUNCTION and GRANT SELECT on tables enforce least privilege.

  • AI Gateway provides rate limits, payload logging, usage tracking, and guardrails such as PII detection and content safety on serving endpoints.

  • Store API keys and tokens in Databricks secrets and reference them with dbutils.secrets.get rather than hardcoding values.

  • system.access.audit records access and changes; join it with system.billing.usage for cost attribution by user.

  • PII redaction can use ai_mask or presidio before logging prompts and responses to inference tables.

  • Responsible AI evaluation includes safety, toxicity, and bias scorers in mlflow.evaluate alongside quality metrics.

  • Service principals with OAuth authenticate applications; grant only the catalogs, tables, and endpoints the application needs.

  • Model lineage in Unity Catalog tracks which data and notebooks produced a model version, supporting audits and reproducibility.

Databricks GenAI Engineer key terms

Retrieval Augmented Generation (RAG)

A pattern where an application searches a knowledge base for relevant passages and inserts them into the prompt, so the model answers from grounded, current evidence instead of relying only on training data.

Related: Chunking, Embedding, Vector Search Index
Foundation Model

A large pretrained model, such as a Llama or DBRX variant, that can be adapted to many tasks through prompting or fine-tuning instead of being trained from scratch for one purpose.

Related: Fine-Tuning, Prompt Template
Foundation Model APIs

Databricks-hosted endpoints exposing pretrained chat, completion, and embedding models, billed per token or through reserved capacity, letting teams prototype without managing GPU infrastructure themselves.

Related: Provisioned Throughput, Serving Endpoint
Context Window

The maximum number of tokens a model can process in one request, covering the system prompt, user input, retrieved passages, and the generated answer combined.

Related: System Prompt, Temperature
System Prompt

Instructions placed ahead of user input that set the model's role, tone, constraints, and output format, persisting across turns and steering behavior for the whole conversation.

Related: Prompt Template, Guardrails
Prompt Template

A reusable parameterized string that combines fixed instructions with runtime variables such as retrieved context and the user question, keeping prompts consistent and testable across an application.

Related: System Prompt, Context Window
Function Calling

A model capability where the LLM returns a structured request naming a tool and its arguments, letting the application run code, query data, or call APIs before continuing the conversation.

Related: Agent, MLflow Tracing
Agent

An LLM-driven program that plans, selects tools, executes steps, and observes results in a loop until a user request is satisfied, rather than producing a single response.

Related: Function Calling, MLflow Tracing
Embedding Model

A model that converts text into vectors; the same model must encode both indexed documents and incoming queries, otherwise similarity scores become meaningless.

Related: Embedding, Vector Search Index
Vector Search Index

A Databricks-managed similarity index that syncs automatically from a Delta table with change data feed, embedding source text and serving nearest-neighbor queries for retrieval.

Related: Embedding, Delta Lake, Metadata Filtering
Embedding

A dense numeric vector representing text meaning, so semantically similar passages land close together and can be compared using distance or similarity metrics.

Related: Embedding Model, Vector Search Index
Chunking

Splitting source documents into smaller passages before embedding, balancing chunk size against retrieval precision and the amount of context that will fit in the prompt.

Related: Document Parsing, Embedding
Document Parsing

Converting PDFs, HTML, and other files into clean structured text, preserving headings and tables so each chunk carries accurate meaning before it is embedded.

Related: Chunking, Delta Lake
Hybrid Search

Combining vector similarity with keyword matching such as BM25 and merging the ranked results, which improves recall for exact identifiers, product codes, and rare terms.

Related: Vector Search Index, Reranking
Reranking

A second-stage model that rescores a shortlist of retrieved passages for relevance to the query, improving the ordering of context sent into the prompt.

Related: Hybrid Search, Groundedness
Metadata Filtering

Restricting a vector search to rows matching conditions on fields such as department, date, or access level, keeping results both relevant and permitted for the requesting user.

Related: Vector Search Index, Access Control
Delta Lake

An open table format on cloud storage adding transactions, schema enforcement, and time travel, serving as the governed source of truth that RAG indexes sync from.

Related: Vector Search Index, Unity Catalog
Fine-Tuning

Further training a pretrained model on your own labeled examples to change its behavior, style, or domain accuracy; it teaches skills and formats rather than adding fresh factual knowledge.

Related: LoRA, Foundation Model
LoRA

A parameter-efficient technique that freezes the base model and trains small low-rank matrices, cutting GPU memory and cost while still adapting style and task behavior.

Related: Fine-Tuning, Model Registry
MLflow

An open-source platform for tracking experiments, logging parameters and metrics, versioning models in a registry, and packaging them for deployment across the lifecycle.

Related: MLflow Tracing, Model Registry
MLflow Tracing

Structured logging of every step in a generative AI request, including retrieval, prompt assembly, model calls, and tool use, so developers can debug quality issues and inspect traffic.

Related: Agent, Evaluation Dataset
LLM-as-a-Judge

An evaluation method where a capable model scores another model's responses against a rubric, scaling subjective quality checks that would be costly to run with human reviewers.

Related: Evaluation Dataset, Groundedness
Groundedness

The degree to which an answer's claims are supported by the retrieved context, a key quality metric for detecting unsupported statements in retrieval augmented systems.

Related: Hallucination, Reranking
Hallucination

A confident model output that is factually wrong or unsupported by the supplied context, often caused by retrieval gaps or prompts that invite guessing.

Related: Groundedness, Retrieval Augmented Generation (RAG)
Evaluation Dataset

A curated set of questions paired with expected answers or reference facts, used to score retrieval and generation quality before and after changes.

Related: LLM-as-a-Judge, MLflow
Temperature

A sampling parameter that scales randomness in token selection: low values make output focused and repeatable, while higher values increase variety at the cost of consistency.

Related: Top-p, Context Window
Top-p

A decoding setting that restricts choices to the smallest set of tokens whose cumulative probability reaches p, another lever for controlling creativity alongside temperature.

Related: Temperature
Model Registry

A catalog for versioning trained and fine-tuned models, recording lineage and approval status, and promoting a chosen version to a serving endpoint.

Related: MLflow, Traffic Splitting
Serving Endpoint

A Databricks-managed REST API that hosts a model or chain, handling scaling, authentication, and rollout so applications can call it over HTTPS.

Related: Provisioned Throughput, Inference Tables
Provisioned Throughput

Reserving dedicated model capacity for predictable latency and higher rate limits, billed by the hour rather than per token, typically for production workloads.

Related: Foundation Model APIs, Serving Endpoint
Inference Tables

Delta tables that automatically capture requests and responses sent to a serving endpoint, enabling monitoring, debugging, and quality evaluation of live traffic.

Related: Lakehouse Monitoring, PII Redaction
Lakehouse Monitoring

A Databricks feature that builds profile and drift metrics over Delta tables, including inference tables, and raises alerts when data or model behavior shifts.

Related: Inference Tables, MLflow Tracing
Traffic Splitting

Routing a percentage of requests to a new model version while the remainder continue to the current one, supporting canary releases and A/B comparisons before full cutover.

Related: Serving Endpoint, Model Registry
Unity Catalog

A unified governance layer for data and models that manages permissions, lineage, auditing, and discovery across workspaces and cloud storage.

Related: Access Control, Audit Logs
AI Gateway

A governance layer in front of model endpoints that logs payloads, enforces rate limits, filters unsafe content, and applies usage policies consistently across applications.

Related: Guardrails, Inference Tables
Guardrails

Input and output filters that block prompt injection, unsafe content, or disallowed topics, typically implemented as policy checks before and after the model call.

Related: AI Gateway, System Prompt
PII Redaction

Detecting and masking personal identifiers in prompts, source documents, or request logs so sensitive data is not sent to a model or retained in monitoring tables.

Related: Guardrails, Inference Tables
Audit Logs

Records of who accessed or changed data, models, and endpoints, used to investigate incidents and demonstrate compliance during security and privacy reviews.

Related: Unity Catalog, Access Control
Access Control

Granting or denying permissions on tables, models, and endpoints through catalog privileges, so users and service principals see only the data their role allows.

Related: Unity Catalog, Metadata Filtering
Prompt Injection

An attack where untrusted text in a document or user message tries to override system instructions, causing the model to leak data or take unintended actions.

Related: Guardrails, System Prompt, PII Redaction

Databricks GenAI Engineer study plan

8 weeks · About 7 hours per week

Week 1

Prepare data for retrieval augmented generation (RAG)

  • Ingest a document corpus (PDFs/markdown/HTML) into a Bronze→Silver Delta pipeline and strip boilerplate, headers, and navigation text

  • Split the same corpus three ways (fixed-token, recursive-structure, semantic) and record chunk count, mean chunk length, and overlap for each

  • Generate embeddings with a Databricks Foundation Model embedding endpoint and persist vectors plus chunk metadata to a Delta table

  • Create a Vector Search index over the embedding table using Delta Sync and confirm it refreshes on new writes

Your checkpoint

A committed notebook/DLT pipeline plus a chunking comparison table (3 strategies × chunk count, avg tokens, overlap) and a queryable Vector Search index over ≥1,000 chunks

Week 2

Prepare data for retrieval augmented generation (RAG) & Design and build generative AI applications

  • Add metadata columns (source, section, last_updated, access_level) to the index and demonstrate filtered retrieval queries

  • Hand-label 50 questions with their correct source chunks and compute recall@5 and recall@10 for the baseline retriever

  • Build a minimal retrieve→prompt→Foundation Model API chain and capture each step with MLflow tracing

  • Test one hybrid or re-ranked retrieval variant and record the recall delta against baseline

Your checkpoint

A retrieval evaluation report showing recall@5 and recall@10 for baseline vs. one improved retriever, with the 50-question labeled set committed alongside it

Week 3

Develop, tune and evaluate generative AI applications

  • Author a 50-item evaluation dataset with question, reference answer, and ground-truth context

  • Implement an LLM-as-judge scorer with an explicit rubric and few-shot calibration examples, then check judge agreement against 15 human-scored items

  • Score the RAG chain on correctness, groundedness, and context sufficiency using MLflow evaluation

  • Tune the system prompt and compare two model endpoints or parameter settings against the same eval set

Your checkpoint

An MLflow evaluation run comparing three prompt/model variants on correctness and groundedness, with judge-vs-human agreement reported

Week 4

Develop, tune and evaluate generative AI applications & Design and build generative AI applications

  • Add a query-rewriting step for follow-up and ambiguous questions and measure its effect on the eval set

  • Add an input/output validation layer that rejects off-topic or unsafe responses before returning them

  • Manually review 20 failing eval cases and group them into an error taxonomy (retrieval miss, stale data, hallucination, refusal, formatting)

  • Refactor the chain into a parameterized, test-covered module callable from notebooks and serving code

Your checkpoint

An error taxonomy covering 20 analyzed failures plus a chain version that improves the week-3 groundedness score, committed as a versioned module with unit tests

Week 5

Design and build generative AI applications

  • Build a chat front end as a Databricks App that calls the chain and streams tokens to the user

  • Externalize model name, index name, prompt template, and top-k into config stored in Unity Catalog rather than hardcoded values

  • Add source citations with chunk links to every response and show a retrieval score beside each citation

  • Handle empty-retrieval and timeout cases with a graceful fallback message and verify via three simulated failure tests

Your checkpoint

A running dev-stage app that streams cited answers, reads its configuration from Unity Catalog, and passes the three failure-mode tests

Week 6

Deploy and monitor generative AI applications

  • Package the chain as an MLflow logged model and deploy it to a Model Serving endpoint with a defined concurrency and scale-to-zero policy

  • Enable inference tables on the endpoint and query logged requests and responses to confirm payload capture

  • Configure Lakehouse Monitoring on the inference table for latency, token count, refusal rate, and retrieval-score drift

  • Create alerting on error rate and p95 latency thresholds, and run a load test at expected peak QPS

  • Route 10% of traffic to a second endpoint variant and compare quality and latency against the primary

Your checkpoint

A monitoring dashboard plus firing alert configuration and a load-test report showing p50/p95 latency and error rate at target QPS

Week 7

Governance, security and responsible AI for generative AI applications

  • Apply Unity Catalog catalog/schema/table grants and one column mask or row filter so retrieval cannot surface data a user is not entitled to

  • Detect and redact PII in source documents before embedding, and re-index the corpus to confirm no PII remains in the vector store

  • Enable AI Gateway guardrails on the serving endpoint for PII, safety filtering, and rate limits, then verify each blocks an adversarial request

  • Write a prompt-injection and jailbreak test suite of 15 cases and record pass/fail for each

  • Draft a model card documenting intended use, data lineage, evaluation results, and known limitations

Your checkpoint

A governance and responsible-AI checklist marked complete with supporting evidence, plus a 15-case prompt-injection test suite with pass/fail results

Week 8

Governance, security and responsible AI for generative AI applications & Deploy and monitor generative AI applications & Design and build generative AI applications

  • Take a full-length timed mock exam across all five domains and log every missed item by domain

  • Re-study the two weakest domains using the missed-item log and re-test those items until correct

  • Validate end-to-end access control by querying the app as a low-privilege user and confirming restricted content is excluded

  • Finalize an operations runbook covering rollback to a previous model version, endpoint scaling changes, and re-indexing procedure

Your checkpoint

An operations runbook plus a mock exam score of at least 80% with a written remediation log for every missed item

Milestones

Week 2: A populated Vector Search index exists and baseline recall@5 is measured on 50 labeled questions

Week 4: An end-to-end RAG chain passes its unit tests and holds a measured correctness/groundedness baseline on a 50-item eval set

Week 6: A serving endpoint with inference tables and working alerts has been load-tested at target QPS with p95 latency reported

Week 8: All governance evidence is filed, the runbook is complete, and the timed mock exam scores at least 80%

Databricks GenAI Engineer exam day

Your checklist

Government-issued photo ID

Name on the ID must match your registration exactly. Confirm which ID types the provider accepts (passport vs. driver's license vs. national ID) before test day.

Appointment confirmation

Save the confirmation email or portal link with your date, start time, time zone, and session/launch link. Double-check the time zone conversion the day before.

Working webcam, microphone, and speakers/headset

If you are testing online with a proctor, run the provider's system/equipment check at least 24 hours ahead so you have time to fix failures.

Stable internet connection

Prefer a wired connection or a spot close to the router. Have a phone hotspot as backup only if the provider permits a reconnect, and confirm that with the provider.

Charger and power cable

Fully charge the laptop and keep it plugged in for the whole session; a drained battery mid-exam is the most avoidable failure mode.

Login credentials tested in advance

Confirm you can sign in to the testing portal with the exact account you registered under. Password resets on exam morning are a common source of lost time.

Quiet, private, well-lit room

No other people, no interruptions, door closed. Clear the desk of papers, books, and devices that are not explicitly allowed.

Scratch-paper / whiteboard plan

Provider rules differ: some allow a physical whiteboard or paper, some require an on-screen notepad, some forbid both. Confirm the permitted method with the provider and prepare that specific tool.

Known exam format and timing

Confirm the current number of questions, time limit, question types, and whether any questions are unscored. Set your pacing target only after you know the real numbers.

Official exam guide / skills outline printed or bookmarked

Use the official domain breakdown as your final-week study map, so your last reviews hit the weighted areas rather than low-value trivia.

Hands-on familiarity with the core toolchain

Be ready to reason about RAG pipelines, embeddings and vector search, prompt engineering, model serving/deployment, evaluation, and chain orchestration in the Databricks ecosystem. Conceptual command beats memorized syntax.

Water and a restroom trip beforehand

Drink modestly before the start so you are not distracted, and use the restroom immediately before check-in since unscheduled breaks may be restricted.

Plan the day

72 hours out

Reread the official exam guide and score each domain red/yellow/green based on your own confidence. Spend the remaining time only on red and yellow domains.

72 hours out

Do one full-length timed practice set under exam-like conditions (no pausing, no notes) to calibrate your pace and stamina.

72 hours out

Spend 60-90 minutes hands-on in a workspace: build or debug a small RAG flow, call a foundation model endpoint, create/query a vector index, and run an evaluation. Muscle memory beats rereading docs.

72 hours out

Shift your sleep schedule so you are naturally alert at your actual exam start time.

24 hours out

Light review only: your own notes, error log, and one-page summaries of weak domains. Do not start a new topic.

24 hours out

Complete the provider's system check on the exact machine and network you will use, and confirm your appointment time in your local time zone.

24 hours out

Read the provider's rules page on ID, permitted items, room scan, and breaks once, carefully, and note anything you need to prepare.

24 hours out

Lay out ID, charger, and any permitted items by the desk. Set two alarms. Get a full night's sleep - this is worth more than an extra hour of cramming.

Morning of

Eat a normal meal, hydrate moderately, and avoid last-minute cramming beyond a single page of reminders.

Morning of

Close all background apps, notifications, and extra browser tabs. Silence your phone and put it out of reach per the provider's rules.

Morning of

Join the check-in 15-30 minutes early to absorb ID verification, room scan, and proctor setup without eating into your exam time.

During the test

Do a first pass at a steady pace. If a question stalls you, flag it and move on - do not let one item cost you three.

During the test

Read the full question stem and every option before answering; several options are usually plausible but only one satisfies all stated constraints.

During the test

Eliminate clearly wrong options first, then choose between the survivors. Prefer the simplest design that meets the stated requirements over the most elaborate one.

During the test

Check the clock at fixed checkpoints, not constantly, and leave a buffer to revisit flagged questions. If there is no penalty for wrong answers, never submit with blanks.

During the test

If anything breaks - freeze, disconnect, or a proctor message - stop and contact the proctor or support channel immediately instead of troubleshooting alone; the timestamp matters for any later dispute.

Rules to check

  • Carry valid government-issued photo ID whose name matches your registration exactly; confirm the accepted ID types with the provider before test day.

  • Keep the workspace clear: no notes, books, phones, smartwatches, or additional monitors unless the provider explicitly permits them.

  • For proctored online delivery, expect an environment and room check. Comply with the proctor's instructions and confirm the exact scan procedure with the provider in advance.

  • Stay visible and audible in frame; do not cover the camera or microphone, leave the view, or speak to anyone, as this can invalidate the attempt.

  • Ask the proctor before taking any break. Confirm with the provider whether breaks are scheduled, whether they pause the clock, and whether they are permitted at all.

  • Treat exam content as confidential: do not screenshot, record, copy, or discuss questions afterward. You will typically accept a confidentiality term at check-in.

  • For technical failures, freeze and contact the proctor or support channel immediately, and confirm the provider's technical-issue and reschedule policy beforehand rather than troubleshooting alone.

  • Scoring and reporting vary by provider: confirm how the exam is scored, what the passing threshold is, and when results appear. Do not assume an instant pass/fail.

Databricks GenAI Engineer FAQ

How many Databricks GenAI Engineer practice questions are available?

The practice set has 200 Databricks GenAI Engineer questions. Each one includes the choices, the correct answer, and an explanation, and the set is free to read with no signup.

How many Databricks GenAI Engineer flashcards are available?

The flashcard set has 32 Databricks GenAI Engineer cards. Each card shows the term and its definition.

Which topics does the Databricks GenAI Engineer study material cover?

The Databricks GenAI Engineer study material has 5 sections, including Design and build generative AI applications, Prepare data for retrieval augmented generation (RAG), and Develop, tune and evaluate generative AI applications.

What format facts does the catalog list for the Databricks GenAI Engineer exam?

The catalog lists these item types: multiple choice for the Databricks GenAI Engineer exam.