Question 770
A data lake needs secure ingestion and schema discoverability. The team confirms that a CSV field changes type and Glue jobs fail silently. Which AWS architecture decision should be prioritized?
Explanation
Correct Answer
The scenario identifies this requirement: A CSV field changes type and Glue jobs fail silently. Select Add schema validation and error quarantine before publishing transformed data as the specified technical mechanism. Explicit validation protects downstream table expectations. This response addresses the problem described rather than a different design goal.
Incorrect Answers
The option Convert records to compressed Parquet and partition by common filters addresses a separate condition: ingested CSV queries scan enormous bytes for a few required columns. Columnar compression and pruning reduce query I/O. The actual requirement is a CSV field changes type and Glue jobs fail silently. Recommended corrective action: Add schema validation and error quarantine before publishing transformed data. Explicit validation protects downstream table expectations.
Configure Lake Formation and S3/IAM permissions for governed access would help if different analysts must not access each other's datasets in a shared lake. Data lake governance is an additional authorization layer. That is not the symptom described here: a CSV field changes type and Glue jobs fail silently. The matching response is Add schema validation and error quarantine before publishing transformed data, because explicit validation protects downstream table expectations.
For the alternative failure raw records land in S3 but analysts cannot discover table schema, the control Create Glue Data Catalog tables using crawlers or defined schemas is relevant: catalog metadata exposes datasets to query engines. Here the evidence instead is a CSV field changes type and Glue jobs fail silently. The selected technical mechanism is Add schema validation and error quarantine before publishing transformed data: explicit validation protects downstream table expectations.