Question 100
A Snowflake engineering lead is evaluating this business requirement: A customer-support draft tool must respond in under two seconds at predictable cost. What decision most accurately addresses the capability boundary in the scenario?
Explanation
Correct Answer
“Benchmark candidate supported models against task accuracy, end-to-end latency, token cost and required context before selecting one” is supported by this requirement: A customer-support draft tool must respond in under two seconds at predictable cost. Model choice is an empirical multi-objective tradeoff, not a universal ranking. Verify it using task-specific evaluation pass rate, measured p95 latency, token expense and stability on realistic production samples.
Incorrect Answers
“Use one synthetic prompt to conclude the model always meets production quality targets” would not meet the requirement that a customer-support draft tool must respond in under two seconds at predictable cost. Model choice is an empirical multi-objective tradeoff, not a universal ranking. The more reliable approach is to benchmark candidate supported models against task accuracy, end-to-end latency, token cost and required context before selecting one.
“Choose solely by maximum context window while ignoring security and throughput” would not meet the requirement that a customer-support draft tool must respond in under two seconds at predictable cost. Model choice is an empirical multi-objective tradeoff, not a universal ranking. The more reliable approach is to benchmark candidate supported models against task accuracy, end-to-end latency, token cost and required context before selecting one.
“Select the most expensive model by default without measuring task performance” would not meet the requirement that a customer-support draft tool must respond in under two seconds at predictable cost. Model choice is an empirical multi-objective tradeoff, not a universal ranking. The more reliable approach is to benchmark candidate supported models against task accuracy, end-to-end latency, token cost and required context before selecting one.