Question 90
During an NVIDIA AI factory acceptance, engineers observe: GPU utilization stalls during checkpoint writes while parallel storage servers show high metadata latency rather than low network throughput. Which repair addresses the principal defect?
Explanation
Correct Answer
The appropriate intervention is Profile checkpoint file counts, metadata operations and storage layout, then tune with the storage vendor's supported method. The defining technical dependency is that Checkpoint bottlenecks can arise from metadata overhead independently of bulk sequential bandwidth. Validate the correction by confirming that A representative checkpoint workload completes faster without integrity errors after metadata tuning.
Incorrect Answers
This response is applicable to a different failure: A training job streams datasets from a shared NFS path and repeatedly rereads the same small files over an overloaded link. Here the central dependency is that Checkpoint bottlenecks can arise from metadata overhead independently of bulk sequential bandwidth, so the specific response is Profile checkpoint file counts, metadata operations and storage layout, then tune with the storage vendor's supported method.
The option would apply to “A storage optimization changes striping width for a parallel filesystem without measuring actual training and checkpoint traffic patterns”. The appropriate intervention is Benchmark representative reader and writer concurrency before choosing supported filesystem striping parameters. This question instead requires “Profile checkpoint file counts, metadata operations and storage layout, then tune with the storage vendor's supported method”.
The related but distinct situation is “A shared storage server reaches a high benchmark throughput but training hosts still see long I/O stalls because their mount and client NIC settings differ”. Therefore Measure throughput end-to-end from compute nodes, including mounts, NICs, permissions and congestion. For the stated problem, “Profile checkpoint file counts, metadata operations and storage layout, then tune with the storage vendor's supported method” addresses the central condition.