Anthropic CCA-F Questions & Answers

The verified manifest already records successful work. The task manifest is the evidence that previous specialist results remain completed. After resuming, the coordinator reruns tasks already completed by an earlier session. Instead, reload a verified manifest of completed tasks and pending dependencies.

A restarted coordinator has not made pending work disappear. Resuming the coordinator does not automatically execute dependencies that were left pending. After resuming, the coordinator reruns tasks already completed by an earlier session. Instead, reload a verified manifest of completed tasks and pending dependencies.

Lifecycle state must be read from verified evidence. Marking every task successful ignores failed or absent completion receipts. After resuming, the coordinator reruns tasks already completed by an earlier session. Instead, reload a verified manifest of completed tasks and pending dependencies.

The weak handwritten class needs specific assessment. Handwritten records need evaluation and possibly expert review, not exclusion from the service. Stratified labeled evaluation exposes that subgroup's error rate and supports appropriate handling of difficult layouts. Instead, measure performance separately for each document class.

This inflates metrics while reducing real-world coverage. Deleting hard cases improves the score only by changing what the metric measures. Stratified labeled evaluation exposes that subgroup's error rate and supports appropriate handling of difficult layouts. Overall extraction accuracy is 97% but handwritten invoices are rarely correct. Instead, measure performance separately for each document class.

Rare handwritten errors can be hidden in the average. The aggregate denominator can be dominated by easy typed forms and hide handwritten failures. Overall extraction accuracy is 97% but handwritten invoices are rarely correct. Instead, measure performance separately for each document class.

Relabeling probability does not improve correctness. Changing numerical labels does not alter whether the extracted identifier matches the source. Model confidence says 0.95 for tax IDs but many are wrong in reviewed records. Instead, build a labeled field-level validation set and compare predicted confidence bins to observed correctness. If 0.95 tax-ID predictions are frequently wrong, their numerical confidence is miscalibrated and must not determine automatic acceptance.

Tax IDs have distinct error patterns and consequences. Different field types need thresholds reflecting their actual error distributions and consequences. Confidence calibration requires labeled observed correctness by field and predicted score bucket, not the model's score alone. Model confidence says 0.95 for tax IDs but many are wrong in reviewed records.

Self-reported scores require empirical calibration. A 0.95 self-score can still be poorly calibrated when labeled tax IDs are frequently wrong. If 0.95 tax-ID predictions are frequently wrong, their numerical confidence is miscalibrated and must not determine automatic acceptance. Model confidence says 0.95 for tax IDs but many are wrong in reviewed records. Instead, build a labeled field-level validation set and compare predicted confidence bins to observed correctness.

Complaint-based samples are biased and incomplete. Recent complaints are a biased slice of failures and do not represent unreported cases. A reproducible random sample provides estimates of misses outside the low-confidence queue without reviewing every record. Instead, review a reproducible high-confidence random sample.

Scores are not evidence of accuracy. Without labeled samples the model's probability cannot be trusted as a correctness rate. A reproducible random sample provides estimates of misses outside the low-confidence queue without reviewing every record. A pipeline routes all low-confidence records for review but never checks confident mistakes.

High-confidence systematic errors remain invisible. A systematic error can receive high confidence and bypass a low-confidence-only queue. A pipeline routes all low-confidence records for review but never checks confident mistakes. Instead, review a reproducible high-confidence random sample. A reproducible random sample provides estimates of misses outside the low-confidence queue without reviewing every record.

Both source values are disputed and cannot be reconciled by arithmetic. A midpoint between two conflicting invoice totals is not documented by either page. Two pages show different invoice totals with no obvious correction note. Instead, send the contradiction and evidence to a human reviewer.

Page order does not establish authority. Later page placement alone does not prove an amendment or superseding correction. Conflicting totals within one invoice cannot both be accepted as authoritative without further documentary evidence. Two pages show different invoice totals with no obvious correction note. Instead, send the contradiction and evidence to a human reviewer.

The financial discrepancy still needs adjudication. Choosing the larger charge remains an unverified financial decision, not cautious validation. Conflicting totals within one invoice cannot both be accepted as authoritative without further documentary evidence. Instead, send the contradiction and evidence to a human reviewer.

Error cost varies substantially across required outputs. Field errors have unequal cost; a missing regulatory flag can exceed minor spelling mistakes. Rare compliance flags can be high-consequence even when overall extraction metrics look excellent. Instead, evaluate rare consequential fields independently.

Removing difficult fields hides critical coverage gaps. Removing consequential rare fields from tests makes the dashboard look better but less useful. Rare compliance flags can be high-consequence even when overall extraction metrics look excellent. A data pipeline is 98% accurate overall but misses important compliance flags. Instead, evaluate rare consequential fields independently.

A rare high-impact field can fail despite high aggregate scores. A rare compliance flag can be systematically missed while easy fields dominate aggregate accuracy. A data pipeline is 98% accurate overall but misses important compliance flags. Instead, evaluate rare consequential fields independently.

A number alone cannot correct or calibrate false amounts. Relabeling confidence numbers cannot correct a wrong source amount or calibrate its risk. Risk-weighted routing uses stricter confidence and validation thresholds for monetary fields and reserves expert attention for exposure. A minor address-format issue and a wrong payment amount are routed identically.

Some high-consequence errors deserve priority handling. Some uncertain monetary values deserve escalation rather than automatic rejection or silent acceptance. A minor address-format issue and a wrong payment amount are routed identically. Instead, use calibrated risk thresholds with stricter handling for monetary fields.

Equal treatment wastes expert attention and fails to protect monetary values. Equal queues consume human review capacity on trivia while wrong amounts may cause payment loss. A minor address-format issue and a wrong payment amount are routed identically. Instead, use calibrated risk thresholds with stricter handling for monetary fields.

A distribution shift may invalidate past score meaning. A new model's score of 0.9 may correspond to different accuracy than the old model. A new model version changes performance on scanned contracts. Instead, rerun stratified labeled evaluation before trusting previous confidence calibration.

Operational decisions require measured postchange behavior. Premature threshold relaxation risks accepting new model errors before measurement. A model upgrade may change the error distribution on scanned contracts even if aggregate benchmark scores appear similar. A new model version changes performance on scanned contracts.

A single case is not representative across scans. One clean contract does not exercise scanner noise, varied layouts and handwritten fields. A new model version changes performance on scanned contracts. Instead, rerun stratified labeled evaluation before trusting previous confidence calibration.

Other reviewers cannot locate the source evidence. Free-form reviewer comments cannot reliably identify the exact line in the original invoice. A human reviewer flags an invoice line but cannot identify its origin page. Instead, store source span and document id for adjudication.

Many invoices can share the same page index. Page 2 is ambiguous without the relevant document identifier among many uploaded files. A human reviewer flags an invoice line but cannot identify its origin page. Instead, store source span and document id for adjudication.

A high-impact disputed line needs traceable origin. A disputed extracted value needs a source span to support correction or adjudication. Human adjudication requires the original document, exact field span and source page tied to the flagged value. A human reviewer flags an invoice line but cannot identify its origin page.

A model can be certain about an unsupported value. The model's certainty can reflect a confident guess even when no date appears in the document. A model asserts high certainty for a date absent from the record. Instead, require evidence-backed field checks and flag missing source values.

The absence remains a factual limitation. Skipping source checks allows unsupported fields to masquerade as verified extraction. High confidence is not evidence that a missing date was actually printed in the source document. A model asserts high certainty for a date absent from the record.

Generated dates cannot stand in for source evidence. Today's date is system context rather than a contract-effective date from the record. A model asserts high certainty for a date absent from the record. Instead, require evidence-backed field checks and flag missing source values.

Critical disagreements still need adjudication. Identity ambiguity and regulated decisions still require accountable expert judgment. Automating trivial exact-match validation can release reviewers to resolve disputed identities and high-impact exceptions. Instead, automate low-risk verified checks and prioritize sampled and high-risk human review.

Reviewer capacity is consumed before ambiguous identities are addressed. Human attention is limited; low-risk exact matches can often be checked deterministically. Automating trivial exact-match validation can release reviewers to resolve disputed identities and high-impact exceptions. A team manually checks every trivial exact-match field while disputed identities remain queued. Instead, automate low-risk verified checks and prioritize sampled and high-risk human review.

High-risk and sampled confident errors also matter. High-confidence errors and high-impact categories may be missed by a queue based only on lowest score. Automating trivial exact-match validation can release reviewers to resolve disputed identities and high-impact exceptions. Instead, automate low-risk verified checks and prioritize sampled and high-risk human review.