NVIDIA NCP-AII Questions & Answers

This option serves the scenario “An AI inference server performs correctly at one request per second but misses latency targets when all accelerators share the CPU and network paths”. Without correction, Single-request smoke tests can conceal contention and poor placement under production demand. The present scenario is distinguished by “An undetected inter-GPU fault can make a nominally healthy compute node unusable for training”.

This alternative addresses the separate case “A node passes short GPU tests but its application intermittently throttles after twenty minutes of sustained work”. Without correction, Short acceptance tests can conceal thermally constrained GPUs that disrupt production jobs. The deciding requirement here is “An undetected inter-GPU fault can make a nominally healthy compute node unusable for training”.

The option would apply to “A GPU server passes nvidia-smi query but CUDA sample binaries fail to allocate device memory under an ordinary user account”. Without correction, A scheduler may dispatch jobs to a node that cannot run real tenant workloads. This question instead requires “An undetected inter-GPU fault can make a nominally healthy compute node unusable for training”.

The option would apply to “A single-node burn-in completes briefly but temperature rises steadily and GPU clocks drop after thirty minutes”. A completed remediation must demonstrate that The long-running stress maintains expected clock behavior without rising error rates or out-of-policy thermals. The present scenario is distinguished by “A combined stress run completes without PSU overload, throttling or NIC link error growth”.

The related but distinct situation is “A stress script records exit code zero even when a GPU has reported uncorrectable memory errors during the test”. The decisive evidence is that Acceptance fails for any nonzero critical hardware error even if the compute test process exits normally. The deciding requirement here is “A combined stress run completes without PSU overload, throttling or NIC link error growth”.

The proposed validation does not isolate the reported fault. An effective closure test requires evidence that A combined stress run completes without PSU overload, throttling or NIC link error growth.

That principle would support Compare BMC inventory, lspci, physical seating and power connections to identify an actually missing device under different conditions, whereas this scenario is primarily explained by Shared bus faults and individual GPU defects require different repair scope and hardware evidence.

That principle would support Check PCI enumeration, slot power, cable seating and platform event logs for the affected adapter under different conditions, whereas this scenario is primarily explained by Shared bus faults and individual GPU defects require different repair scope and hardware evidence.

This alternative addresses the separate case “A rack reports repeated fan alerts from one slot, but operators replace another fan with the same visible color label”. This case illustrates that Correct component identification requires matching reported fault locations to actual serviceable hardware. The deciding requirement here is “Shared bus faults and individual GPU defects require different repair scope and hardware evidence”.

This option serves the scenario “A DGX-class server enumerates eight GPUs but one NVLink peer path is missing from the reported topology”. The controlling explanation is that PCIe device discovery does not establish all intended GPU-to-GPU high-bandwidth connections. This question instead requires “Subset success does not validate the complete intra-node accelerator communication graph”.

This alternative addresses the separate case “An operator measures slow NCCL all-reduce and immediately replaces a GPU even though the device has no errors and topology shows a congested shared fabric”. The key distinction is that Single-node and cross-node collectives exercise different network paths and failure domains. For the stated problem, “Subset success does not validate the complete intra-node accelerator communication graph” addresses the central condition.

The option would apply to “A server with MIG partitioning passes independent tenant GPU kernels but the team assumes every MIG instance supports arbitrary cross-instance NVLink collectives”. The controlling explanation is that Logical GPU partitions have specific communication constraints and should not be treated as full physical GPUs by default. The present scenario is distinguished by “Subset success does not validate the complete intra-node accelerator communication graph”.

Although the alternative can be true, the current evidence is more directly governed by A running chassis can continue on one PSU while having lost the fault tolerance promised by redundant supplies.

The option would apply to “A dual-port NIC exposes only one working port after an optic swap and the operator treats the whole adapter as failed”. The controlling explanation is that A one-port symptom may be caused by replaceable media rather than the network card's common hardware. This question instead requires “A running chassis can continue on one PSU while having lost the fault tolerance promised by redundant supplies”.

This is an incorrect prioritization of the mechanism. The reported fault points to A running chassis can continue on one PSU while having lost the fault tolerance promised by redundant supplies, not the separate condition involving A GPU is absent from the expected device inventory after maintenance, but the technician proposes reinstalling CUDA before checking hardware state.

The option would apply to “Slurm reports GPU resources on a partition, but scheduled jobs fail because the node GRES inventory does not match actual accelerator devices”. Without correction, Incorrect GPU declarations can oversubscribe nodes or leave physical accelerators idle. The present scenario is distinguished by “An open scheduler can expose sensitive datasets or monopolize reserved expensive accelerator capacity”.

The related but distinct situation is “A BCM cluster is ready for general users but no tenant can access the training filesystem from compute nodes”. Without correction, Mismatched identity or file permissions can make queued GPU jobs fail on startup. The deciding requirement here is “An open scheduler can expose sensitive datasets or monopolize reserved expensive accelerator capacity”.

This option serves the scenario “A compute node can reach the login network but fails RDMA collectives because the intended data interface is missing from its BCM network configuration”. Without correction, An absent training fabric can impose severe communication bottlenecks on distributed jobs. This question instead requires “An open scheduler can expose sensitive datasets or monopolize reserved expensive accelerator capacity”.

This response is applicable to a different failure: A Docker image includes CUDA libraries, but the operator launches it without any GPU device allocation flag and wonders why inference runs on CPU. Here the central dependency is that Container images do not virtualize away the host driver compatibility requirements of GPU workloads, so the specific response is Select a driver-compatible CUDA container image or perform a controlled supported host upgrade.

This option addresses an alternative scenario (A Docker image starts on a CUDA-capable node but silently loads a CPU-only framework wheel); the reported issue instead calls for Select a driver-compatible CUDA container image or perform a controlled supported host upgrade.

This option serves the scenario “A Docker test enumerates all GPUs with --gpus all but does not verify whether the intended tenant workload can access them under the production identity”. Therefore Run a user- and image-specific GPU smoke test with the actual deployment security context. The present scenario is distinguished by “Select a driver-compatible CUDA container image or perform a controlled supported host upgrade”.

The related but distinct situation is “A Linux image is cloned with duplicate machine identities and static network parameters across otherwise identical GPU servers”. The primary exposure is that Duplicate identities can cause SSH trust failures and scheduler or monitoring collisions. The deciding requirement here is “Incorrect boot selection can cause inconsistent nodes or loss of expected state”.

This option serves the scenario “An OS image for GPU nodes exposes unrestricted password SSH and lacks synchronized system logs after PXE provisioning”. The primary exposure is that Unhardened newly provisioned compute hosts create cluster-wide credential and incident visibility risks. This question instead requires “Incorrect boot selection can cause inconsistent nodes or loss of expected state”.

The option would apply to “A compute fleet boots an OS image with a kernel version unsupported by the chosen GPU driver and fails after installation”. The primary exposure is that An unsupported OS baseline can make every provisioned GPU node unusable despite successful network boot. The present scenario is distinguished by “Incorrect boot selection can cause inconsistent nodes or loss of expected state”.

This option serves the scenario “A new cluster's ClusterKit run omits three unreachable nodes from the result table and the engineer marks the entire fleet as passed”. A completed remediation must demonstrate that The final report lists every expected node with a pass, fail or explicit approved exception. The present scenario is distinguished by “A corrected harness runs consistently under approved identities and yields valid metrics”.

This alternative addresses the separate case “ClusterKit reports a slow InfiniBand fabric but operators run it during a large unrelated backup and treat the result as hardware failure”. A completed remediation must demonstrate that A controlled retest correlates workload occupancy, NIC counters and fabric throughput to explain the variance. The deciding requirement here is “A corrected harness runs consistently under approved identities and yields valid metrics”.

This is weaker evidence because Invalid test setup results are not interpretable as throughput or device faults without successful job execution. The independent test must show that A corrected harness runs consistently under approved identities and yields valid metrics.

This alternative addresses the separate case “A new server passes isolated GPU tests but fails when eight GPUs and all NICs are stressed concurrently”. Without correction, Independent green component tests can hide shared-resource bottlenecks under real training load. For the stated problem, “A CPU-only test can overlook failed GPU data paths and faulty accelerator memory” addresses the central condition.

The option would apply to “A single-node burn-in completes briefly but temperature rises steadily and GPU clocks drop after thirty minutes”. The acceptance board should flag that Thermal throttling can make production benchmarks substantially worse than short acceptance tests. The present scenario is distinguished by “A CPU-only test can overlook failed GPU data paths and faulty accelerator memory”.

This option serves the scenario “An acceptance engineer benchmarks only GPU number zero on an eight-GPU HGX server and records the entire node as healthy”. The direct failure mode is that One untested defective accelerator can cause whole-node distributed training failures. This question instead requires “A CPU-only test can overlook failed GPU data paths and faulty accelerator memory”.

For this finding, Verify NCCL interface selection, HCA visibility and transport settings for comparable runs is less direct than Isolate rack-to-rack switch and transceiver paths using NCCL scaling tests and per-port telemetry, because A cross-rack-only regression points to an external fabric segment or congestion condition.

The suggested action targets another problem where CPU TCP throughput does not establish GPU communication performance or RDMA configuration. It does not directly address this case, which needs Isolate rack-to-rack switch and transceiver paths using NCCL scaling tests and per-port telemetry.

This alternative addresses the separate case “A link layer reports no packet drops but NCCL performance degrades during simultaneous jobs on neighboring GPU partitions”. Therefore Inspect fabric congestion and PFC/ECN-related counters with workload concurrency during collective tests. The deciding requirement here is “Isolate rack-to-rack switch and transceiver paths using NCCL scaling tests and per-port telemetry”.