Master Assessment – Designing Modern AI Systems
A comprehensive capstone that tests your command of the full AI engineering stack — from system design and RAG pipelines to agent architecture, prompt engineering, and production deployment. 40 questions across concept, application, and real-world scenario formats. Results revealed at the end.
Match each AI behavior or architecture to the system it most closely reflects.
Which behaviors are best classified as primarily System 2? Select all that apply.
- Generating an immediate reply from surface linguistic cues
- Exploring several candidate solution paths before committing
- Using a scratchpad to carry intermediate results across steps
- Retrieving a memorized phrase pattern for autocomplete
Assertion: An AI system that performs explicit step-by-step verification during inference is primarily exhibiting System 2 behavior. Reason: System 2 is characterized by slower, more deliberate reasoning that uses intermediate checks before a final answer.
Arrange the parts of a Chain-of-Thought style solution in the most appropriate order.
In a Chain-of-Thought style solution, the explicitly written intermediate reasoning steps are often called a ___ .
Which response best demonstrates a Chain-of-Thought style approach to solving a multi-step word problem?
A model must solve a puzzle with several plausible branches, where early choices can lead to dead ends. Which architecture component is most appropriate?
Which scenarios are best served by self-reflection or deliberative inference rather than a single direct response? Select all that apply.
- A task where the model must check for contradictions in its own draft before submitting
- A one-turn factual lookup with a clearly known answer
- A safety-sensitive recommendation that should be reviewed for reasoning errors
- A short greeting response that requires no verification
For a long-horizon task like booking travel with several dependencies and ordered subtasks, the most suitable component is ___ .
Match each workflow stage to its primary purpose in a browser-based agent loop.
A user wants to buy a blue size-9 shoe. The product page shows color swatches, a size dropdown, an inactive Add to Cart button, and a note saying 'Select size to continue.' Which actions belong in a good Observe-Plan-Act response? Select all that apply.
- Observe that size must be selected before Add to Cart becomes usable
- Plan to choose the size-9 option from the dropdown next
- Act by clicking Add to Cart immediately despite the inactive state
- Observe that the blue color control is relevant if the current color is not blue
In the Observe-Plan-Act workflow, it is appropriate to choose an action before checking whether the target element is visible and enabled.
Match each web-page cue to the most likely intended interactive target.
A form shows the text 'Search products' above an input. In the DOM, a label with for='q' is visible, and the input has id='q'. Which element should be targeted for text entry?
If two elements have the same accessible name, a hidden element is usually a worse interaction target than the visible enabled one.
In which task descriptions is an AI agent more appropriate than a chatbot? Select all that apply.
- Track a package across several carrier pages and notify the user if delivery is delayed
- Answer a one-off question about the definition of reinforcement learning
- Monitor calendar conflicts, reschedule meetings when needed, and send updates automatically
- Generate a short joke on request
A chatbot is usually sufficient when the task mainly involves a single conversational response and does not require tool use or persistent goal tracking.
Assertion: An AI agent is more appropriate than a chatbot for a task that requires repeated tool use, state tracking, and adaptation over time. Reason: Agents are designed for sustained goal pursuit with planning and memory across multiple actions.
Which description best fits System 1 behavior in AI?
Chain-of-Thought prompting is especially useful for problems that require several intermediate reasoning steps.
A task requires the model to generate an answer and then review it for possible mistakes before finalizing. Which component is most appropriate?
Which actions are appropriate during the Observe stage of a browser agent loop? Select all that apply.
- Checking whether a button is visible and enabled
- Reading the current page text and relevant form labels
- Clicking the first control without inspection
- Noting any warning messages or constraints shown on the page
When a visible label uses the attribute for='email', it usually points to the input whose ___ matches that value.
Which scenario is best handled by a chatbot rather than an AI agent?
Assertion: An AI agent often needs memory of prior steps while completing a long-running task. Reason: Long-running tasks may involve intermediate state, prior actions, and changing conditions that affect the next decision.
Which scenario most clearly reflects System 2 processing in AI?
A scratchpad in reasoning is used to store intermediate steps before reaching the final answer.
Which tasks are best suited for tree-of-thought reasoning? Select all that apply.
- A maze-like problem with many branching decisions
- A puzzle where several candidate paths should be compared
- A simple factual question with one obvious answer
- A task that benefits from exploring alternatives before choosing one
A form shows a disabled Submit button and a required checkbox labeled 'I agree to the terms.' What should the agent plan to do next?
If a visible label is connected to an input through the for attribute, the target control is usually identified by its ___ attribute.
Assertion: An AI agent is better than a chatbot for tasks that unfold over time and require reacting to changing conditions. Reason: Agents can maintain state, use tools, and revise actions as new information appears.
Match each task description to the more appropriate system.
Which feature most strongly suggests System 2 reasoning in an AI model?
Tree-of-thought reasoning is especially useful when a problem contains several branching paths that should be explored.
Which observations would help an agent decide the next action on a checkout page? Select all that apply.
- Whether required fields are empty
- Whether the Pay button is enabled
- Clicking a random link before inspecting the form
- Any visible error messages near payment fields
An element hidden with display:none is generally not the best ___ target for a user-facing action.
Assertion: A chatbot is usually less suitable than an AI agent for managing a task that requires repeated monitoring and follow-up actions. Reason: Such tasks often require memory, planning, and adaptation across multiple steps over time.
Match each reasoning element to its role in a Chain-of-Thought style solution.
A task involves a long sequence of dependent actions, where each step affects what can happen next. Which component is most appropriate?
A login page shows fields for username and password, plus a disabled Sign In button and a message saying 'Enter all required fields.' What should the agent do next?