NVIDIA NCP-AII Questions & Answers

This option serves the scenario “An HPL result is announced as peak GPU FLOPS without recording matrix size, precision, GPU count or the HPL implementation”. The direct fix is Record a reproducible HPL benchmark configuration, software version and numeric precision before comparing results. The present scenario is distinguished by “Treat numerical validation failures as acceptance failures and examine hardware or software errors”.

This option addresses an alternative scenario (An engineer compares HPL throughput on two GPU nodes but one run uses different process mapping and memory pressure); the reported issue instead calls for Treat numerical validation failures as acceptance failures and examine hardware or software errors.

The related but distinct situation is “A new AI rack posts a good HPL score once but fails to reproduce it after system reboot”. Therefore Repeat HPL with fixed settings across controlled restarts and track device and firmware state. For the stated problem, “Treat numerical validation failures as acceptance failures and examine hardware or software errors” addresses the central condition.

This is an incorrect prioritization of the mechanism. The reported fault points to BCM primary and secondary are labels, whereas active service ownership changes when a validated failover occurs, not the separate condition involving Two BCM head nodes both attempt to mount a shared writable filesystem during a simulated split-brain incident.

The related but distinct situation is “A backup head node has no configured remote power-fencing capability, but the team expects automatic recovery from any loss of heartbeat”. The controlling explanation is that Safe automatic HA normally requires a way to ensure the former active node cannot continue writing shared resources. The deciding requirement here is “BCM primary and secondary are labels, whereas active service ownership changes when a validated failover occurs”.

This option serves the scenario “A Base Command Manager cluster relies on one head node and loses both DHCP provisioning and Slurm scheduling whenever that node powers off”. The operational evidence follows from the mechanism that BCM head-node services can be replicated to a passive peer but require compatible management, storage and failover configuration. This question instead requires “BCM primary and secondary are labels, whereas active service ownership changes when a validated failover occurs”.

The option would apply to “A dual-port NIC exposes only one working port after an optic swap and the operator treats the whole adapter as failed”. The strongest acceptance result is that A known-good transceiver and link test restore the port without unnecessary adapter replacement, not merely that a command executed. This question instead requires “After a targeted change, affected slots stop recording errors while unaffected accelerators remain stable”.

The related but distinct situation is “A rack reports repeated fan alerts from one slot, but operators replace another fan with the same visible color label”. The decisive evidence is that After targeted fan replacement, the specific BMC alarm clears and inlet/cooling behavior stabilizes. For the stated problem, “After a targeted change, affected slots stop recording errors while unaffected accelerators remain stable” addresses the central condition.

This is weaker evidence because Shared bus faults and individual GPU defects require different repair scope and hardware evidence. The independent test must show that After a targeted change, affected slots stop recording errors while unaffected accelerators remain stable.

That principle would support Reconcile live BMC and PCI device inventory against the signed hardware bill of materials before rack acceptance under different conditions, whereas this scenario is primarily explained by A present network adapter may operate at degraded PCIe bandwidth due to slot or firmware conditions.

That principle would support Analyze ECC and hardware event trends and isolate suspect memory or accelerator components before production release under different conditions, whereas this scenario is primarily explained by A present network adapter may operate at degraded PCIe bandwidth due to slot or firmware conditions.

This alternative addresses the separate case “A node's BIOS reports a NUMA topology different from the approved design after a firmware reset”. The controlling explanation is that NUMA placement and I/O topology influence data movement and GPU workload performance even if the OS boots. For the stated problem, “A present network adapter may operate at degraded PCIe bandwidth due to slot or firmware conditions” addresses the central condition.

This option serves the scenario “A new AI factory adds two leaf switches but places their uplinks through a single shared spine and claims full fabric availability”. Therefore Model spine and leaf failures and provide redundant forwarding resources matched to the target recovery level. The present scenario is distinguished by “Define traffic classes and capacity headroom for storage checkpoint writes and synchronous GPU collectives”.

This alternative addresses the separate case “A GPU cluster uses the same unsegmented VLAN for BMC administration and high-volume RoCE traffic across the leaf-spine fabric”. The direct fix is Create explicit management and data-plane isolation with suitable routing, QoS and access boundaries. The deciding requirement here is “Define traffic classes and capacity headroom for storage checkpoint writes and synchronous GPU collectives”.

The option would apply to “An AI training design links all compute servers through one oversubscribed access switch but expects full-bandwidth all-reduce across multiple racks”. Therefore Choose a validated nonblocking or acceptably oversubscribed leaf-spine data fabric sized for east-west collective traffic. This question instead requires “Define traffic classes and capacity headroom for storage checkpoint writes and synchronous GPU collectives”.

This alternative addresses the separate case “A privileged Docker container succeeds with GPU access while an ordinary GPU-enabled container fails the same workload”. The operational evidence follows from the mechanism that Privilege escalation can hide missing runtime integration while exposing unnecessary host resources. For the stated problem, “Container images do not virtualize away the host driver compatibility requirements of GPU workloads” addresses the central condition.

The option would apply to “A Docker image starts on a CUDA-capable node but silently loads a CPU-only framework wheel”. This case illustrates that Visible NVIDIA devices do not guarantee that the application framework contains GPU-enabled binaries. The present scenario is distinguished by “Container images do not virtualize away the host driver compatibility requirements of GPU workloads”.

The related but distinct situation is “A Docker image includes CUDA libraries, but the operator launches it without any GPU device allocation flag and wonders why inference runs on CPU”. The operational evidence follows from the mechanism that Container software packages do not by themselves grant access to host GPU device nodes. The deciding requirement here is “Container images do not virtualize away the host driver compatibility requirements of GPU workloads”.

The option would apply to “A network team enables a BlueField security mode that restricts host-side reads, then declares the card defective when a host monitoring tool loses access”. Without correction, Mistaking intentional host restrictions for defects can trigger unnecessary DPU replacement. This question instead requires “Treating a staged image as active can leave the fleet with inconsistent capabilities”.

The alternative may occur in another situation (A rack of BlueField DPUs displays the expected physical links but several cards report repeated hardware health events from their embedded controller), but the principal consequence here is that Treating a staged image as active can leave the fleet with inconsistent capabilities.

This option serves the scenario “BlueField-3 adapters pass host Ethernet traffic but the embedded Arm software revision differs from the approved fabric interoperability matrix”. Without correction, An unnoticed embedded firmware mismatch can trigger offload faults under distributed workloads. The present scenario is distinguished by “Treating a staged image as active can leave the fleet with inconsistent capabilities”.

The related but distinct situation is “An operations team uses a single successful training run as evidence that six racks meet the contractual availability target”. The controlling explanation is that One completed workload may avoid weak links and says little about redundant paths or intermittent component failures. The deciding requirement here is “A successful OS boot proves provisioning reachability but does not prove all AI workload data paths have been commissioned”.

This option serves the scenario “The build team provisions Slurm users before allocating separate out-of-band access and emergency recovery procedures for the head node”. This case illustrates that Loss of the in-band network during provisioning must not also remove the only way to recover failed systems. This question instead requires “A successful OS boot proves provisioning reachability but does not prove all AI workload data paths have been commissioned”.

This alternative addresses the separate case “An AI factory rack has completed electrical installation, but technicians schedule cluster burn-in before BMC addresses and power checks are signed off”. The controlling explanation is that A benchmark cannot establish deployment readiness while foundational power, management and hardware dependencies remain unverified. For the stated problem, “A successful OS boot proves provisioning reachability but does not prove all AI workload data paths have been commissioned” addresses the central condition.

That change may help in a different context, but it would not establish that A link and transfer stress test completes after fixing the identified PCIe problem with no new AER events. The relevant response is Investigate PCIe link health, slot topology and firmware before assigning the problem to NCCL software.

This alternative addresses the separate case “A GPU cluster logs corrected ECC errors increasing steadily on one accelerator although training jobs usually pass”. Therefore Compare error growth against vendor health criteria and isolate the suspect GPU for extended testing. The deciding requirement here is “Investigate PCIe link health, slot topology and firmware before assigning the problem to NCCL software”.

The related but distinct situation is “The operator sees repeated training slowdowns after a rack cooling change but assumes the NCCL library is defective”. Therefore Compare GPU clock-limit reasons, thermal sensors and fabric telemetry against the prior baseline. For the stated problem, “Investigate PCIe link health, slot topology and firmware before assigning the problem to NCCL software” addresses the central condition.

This alternative addresses the separate case “An eight-GPU server is assigned to a PDU feed sized for average power rather than accelerator transient peaks”. The acceptance board should flag that An undersized circuit can trip during synchronized load and interrupt an entire training rack. For the stated problem, “Poor rack airflow can cause clustered thermal alarms and unpredictable GPU clock reductions” addresses the central condition.

The related but distinct situation is “A liquid-cooled AI server is powered before facility coolant flow and leak sensing are commissioned”. The primary exposure is that Missing cooling interlocks can expose accelerators and facilities to overheating or coolant incidents. The deciding requirement here is “Poor rack airflow can cause clustered thermal alarms and unpredictable GPU clock reductions”.

This option serves the scenario “A data-center cooling report gives acceptable room temperature while the rear GPUs in a dense rack throttle under load”. The primary exposure is that Unidentified hot spots can degrade performance and accelerate failure even when room sensors appear normal. This question instead requires “Poor rack airflow can cause clustered thermal alarms and unpredictable GPU clock reductions”.

The option would apply to “A GPU server's DPU has an approved high-speed physical link but no functioning OOB path for emergency firmware recovery”. The direct failure mode is that Loss of independent management can prolong outages when the DPU blocks host data paths. This question instead requires “Partial updates can produce difficult-to-diagnose network and offload behavior under production traffic”.

The related but distinct situation is “A tenant host administrator can still flash the BlueField firmware even though the project mandates zero-trust DPU management separation”. The direct failure mode is that Host privilege reaching DPU firmware can undermine the intended isolation boundary. For the stated problem, “Partial updates can produce difficult-to-diagnose network and offload behavior under production traffic” addresses the central condition.

This option serves the scenario “A BlueField DPU is operating in NIC mode, but the security architect expects the Arm control plane to enforce independent host network policy”. The primary exposure is that Confusing NIC and DPU modes can leave expected independent data-path protection inactive. The present scenario is distinguished by “Partial updates can produce difficult-to-diagnose network and offload behavior under production traffic”.