Question 240
During a multi-team AI platform review at an applied research institute, engineers report the following issue: An inference service repeatedly copies the same weights from host memory for every request. The affected work supports training and inference experiments; jobs must produce reproducible, defensible technical results. Select TWO: the most appropriate action and the strongest verification of that action. (Select TWO.)
Explanation
Correct Answers
Correct corrective action. Account for GPU memory capacity and the cost of moving inputs between host memory and device memory when assessing acceleration and batch size. This fits the specific reported issue: An inference service repeatedly copies the same weights from host memory for every request. The corresponding failure being reduced is: Transfer overhead and out-of-memory conditions can erase expected GPU gains even when peak GPU arithmetic throughput is high.
Correct independent evidence. A representative run measures host-to-device transfer, allocated GPU memory and kernel execution without exceeding supported memory limits. This verifies the corrective approach for the affected training and inference experiments instead of treating an architecture diagram alone as success.
Incorrect Answers
Not the most direct the action-and-evidence pair for these facts. Test questions requiring recent internal data return answers supported by the approved retrieved documents and respect access rules. This choice concerns natural language assistants and retrieval use cases and may be valid in another design. But this case reports 'An inference service repeatedly copies the same weights from host memory for every request', which calls for reasoning about gpu device memory and transfer considerations. The targeted response is: Account for GPU memory capacity and the cost of moving inputs between host memory and device memory when assessing acceleration and batch size.
Not the most direct the action-and-evidence pair for these facts. Evaluate request latency, concurrency, batching and model size together when selecting inference resources, with measurements under a representative service-level target. This choice concerns inference latency throughput and batching tradeoffs and may be valid in another design. But this case reports 'An inference service repeatedly copies the same weights from host memory for every request', which calls for reasoning about gpu device memory and transfer considerations. The targeted response is: Account for GPU memory capacity and the cost of moving inputs between host memory and device memory when assessing acceleration and batch size.
Not the most direct the action-and-evidence pair for these facts. Select a GPU for many data-parallel numerical operations and a CPU for branching control or low-parallelism tasks, considering end-to-end bottlenecks. This choice concerns gpu parallel throughput versus cpu serial control and may be valid in another design. But this case reports 'An inference service repeatedly copies the same weights from host memory for every request', which calls for reasoning about gpu device memory and transfer considerations. The targeted response is: Account for GPU memory capacity and the cost of moving inputs between host memory and device memory when assessing acceleration and batch size.