Question 380
A Fabric notebook refines semi-structured records with PySpark. The current environment exhibits the following problem: a notebook writes a Delta table without validating input record counts. Which action should the team take next?
Explanation
Correct Answer
Assert input/output cardinality and business-key constraints before commit. Data quality tests catch unintended row loss or multiplication.
Incorrect Answers
A nested field name changes capitalization across source files. Schema normalization avoids duplicate or missing fields from drift.
A groupBy result exceeds memory because one key is extremely hot. Skew concentrates work in a small number of tasks.
A DataFrame is accidentally converted to Pandas on the driver. Distributed processing prevents driver overload.