AI Certificates

Domain deep dive

Model Selection and Optimization on the CCDV-F exam

Matthew Hartman · CCAR-P certified, scored 965/1000 · 3 min read

Model Selection and Optimization is 16.8% of the exam, roughly nine questions, and it is the domain that checks whether you understand what is happening underneath the SDK.

Four sub-objectives: LLM fundamentals, technical fundamentals, model selection and tradeoffs, and cost and token management.

Tokens and the context window

Claude reads tokens, not words or characters, and the characters-per-token ratio depends on the tokenizer of the specific model. Any rule of thumb you have memorized is model-dependent and worth re-confirming rather than trusting.

The context window is a fixed budget covering everything at once: system prompt, full conversation, injected documents, tool definitions, tool results, and the generated output. It has two distinct failure modes that questions like to separate. A request whose input already exceeds the window is rejected before generation begins. A request that fits on input can still hit the ceiling during generation, in which case current models stop and return what they produced with a context-window stop reason rather than raising an error.

Sampling and non-determinism

The model does not pick a next token, it samples one from a probability distribution. That is the root of non-determinism, and it has a direct testing consequence the exam cares about: asserting on exact response text produces flaky tests, so assert on the property that must hold instead. Required field present, value in range, structure parses.

From the free practice form

A developer runs the same extraction prompt nightly and wants outputs that vary as little as possible between runs. Which sampling change most directly reduces run-to-run variation?

Choosing a model versus turning on reasoning

A distinction worth holding precisely: selecting a capability tier and enabling a reasoning mode such as extended thinking are different decisions with different costs. Questions exploit the conflation in both directions, offering a bigger model where the real problem is prompt shape, or extended thinking where the real problem is that the model has never been shown the target format.

From the free practice form

Despite detailed formatting instructions, Claude keeps drifting from a team's unusual in-house markup when summarizing tickets. What change is most likely to make outputs conform?

Cost and token management

Cost questions are arithmetic wearing a scenario. Tokens are the billing unit, so estimating what a feature costs means counting tokens across the prompt, the history, the tool definitions and the output. The levers worth knowing cold are prompt caching, batch processing for non-urgent volume, and choosing a cheaper tier for work that does not need the expensive one.

Prompt caching rewards a specific structural habit: static content first, dynamic values after, because matching is prefix-based. Getting that order backwards defeats the cache entirely, which is exactly the mistake a question will describe without naming.

Practice this

Form 1 has 9 Model Selection and Optimization questions. Free, no signup.

Start form 1 · study mode

Sets 2–6 add 27 more Model Selection and Optimization questions — on Udemy.