Anthropic CCA-F Questions & Answers

Label changes do not improve evidence quality. A new category name leaves the same faulty source-to-sink classifier unchanged. Model review precision must be measured separately for each finding class and tested against labeled examples. Aggregate precision appears high while SQL injection alerts are usually false. Instead, measure per-category precision and tune the failing category.

A global change can suppress legitimate issues elsewhere. Reducing detection everywhere risks hiding true issues in categories that already work. Aggregate precision appears high while SQL injection alerts are usually false. Instead, measure per-category precision and tune the failing category.

SQL-injection false positives can hide behind other accurate categories. Overall precision could remain high because numerous easy categories dominate the benchmark. Aggregate precision appears high while SQL injection alerts are usually false. Instead, measure per-category precision and tune the failing category.

Existing unrelated defects predate the submitted changes. Legacy findings unrelated to changed lines cannot honestly be attributed to the PR author. A CI reviewer flags every known legacy problem as introduced by the current PR. Instead, restrict findings to code behavior changed by the current diff.

A description may omit a real changed behavior. Commit descriptions can omit relevant code changes and should not replace diff inspection. PR review should attribute a finding to behavior introduced or worsened by the proposed diff. A CI reviewer flags every known legacy problem as introduced by the current PR. Instead, restrict findings to code behavior changed by the current diff.

Impact must be judged from the actual changed code. A preexisting problem made worse by the diff may be a valid new regression despite its history. PR review should attribute a finding to behavior introduced or worsened by the proposed diff.

One passing check does not neutralize a validated blocker. Passing one independent check cannot override an unresolved critical failure in another category. A downstream gate should interpret structured severity and explicit thresholds, not treat every message as equally dangerous. Instead, use typed severity with acceptance thresholds for the gate.

An advisory style item should not stop deployment. A typo and a verified security regression should not have identical release-blocking effects. A review downstream job treats any reported item as a release blocker. Instead, use typed severity with acceptance thresholds for the gate.

The gate needs a typed distinction between advisory and critical. Without a severity field the consumer cannot implement evidence-based gate policy. A review downstream job treats any reported item as a release blocker. Instead, use typed severity with acceptance thresholds for the gate.

The team cannot reproduce a threshold-free claim. Without a numeric budget there is no reproducible performance acceptance test. A code assistant describes a slow function as 'inefficient' without a budget or measurement. Instead, define size, latency, or resource thresholds before judging performance.

Optimization risks regressions without a measurable goal. Premature code transformation can introduce defects without measurable performance improvement. A code assistant describes a slow function as 'inefficient' without a budget or measurement. Instead, define size, latency, or resource thresholds before judging performance.

Source size does not prove latency or resource cost. Function length does not determine algorithm complexity or real workload latency. A code assistant describes a slow function as 'inefficient' without a budget or measurement. Instead, define size, latency, or resource thresholds before judging performance.

Security depends on input provenance and downstream effects. Input from a caller may be attacker-controlled even if the local function body appears benign. A reviewer claims a function is safe simply because it lacks known vulnerability keywords. Instead, require concrete validation of relevant trust boundary and data path.

One scanner cannot establish all relevant runtime preconditions. Static tools can miss authorization assumptions and runtime relationships outside their rule set. A reviewer claims a function is safe simply because it lacks known vulnerability keywords. Instead, require concrete validation of relevant trust boundary and data path.

Dangerous data flow can occur without recognizable keywords. The code may expose a sensitive operation through normal application logic with no signature keyword. A reviewer claims a function is safe simply because it lacks known vulnerability keywords. Instead, require concrete validation of relevant trust boundary and data path.

The same word also refers to downloading an invoice. A customer asking to view a bill should not be routed automatically to money movement. A support agent confuses invoice download with payment status when requests use the word bill. Instead, provide examples covering both possible intents and distinguishing conditions.

The available context can often distinguish the desired action. Examples can teach intent recognition without forcing needless clarifying questions in every case. Contrasting labeled requests teach the model to route by user intent and available transaction evidence rather than keyword alone.

The second valid customer intent must remain supported. Removing invoice retrieval would wrongly refuse a common harmless support task. Contrasting labeled requests teach the model to route by user intent and available transaction evidence rather than keyword alone. A support agent confuses invoice download with payment status when requests use the word bill. Instead, provide examples covering both possible intents and distinguishing conditions.

The reviewer may learn to suppress actual null dereferences. Teaching only suppressions may cause the reviewer to overlook an actual null-reference path. A review assistant flags valid error handling as suspicious and misses real null dereferences. Instead, include representative real defects and acceptable patterns.

Repetition omits the contrasting edge cases causing errors. Many normal examples provide no guidance on the error branch that the model misclassifies. Showing both valid guarding and an exploitable dereference reduces false alerts without teaching the reviewer to ignore real faults.

Valid defensive handling should not be penalized. A try-catch can correctly recover from expected failure and is not itself a bug. A review assistant flags valid error handling as suspicious and misses real null dereferences. Instead, include representative real defects and acceptable patterns.

A half-kilogram value would become incorrect. Rounding 0.5 kg to a whole kilogram doubles the actual billed or measured quantity. A small paired example demonstrates that half a kilo and 0.5 kg describe the same amount without dropping source meaning. Invoices express quantities as 'half a kilo' or '0.5 kg' and the extractor varies outputs. Instead, provide examples normalizing both variants to the same unit.

The extractor needs explicit normalization evidence, not invention. An assumed unit can change value when the source lacks an unambiguous measurement basis. A small paired example demonstrates that half a kilo and 0.5 kg describe the same amount without dropping source meaning.

The downstream system requires stable numeric units. Downstream calculation needs a stable unit and numeric representation rather than arbitrary prose. Invoices express quantities as 'half a kilo' or '0.5 kg' and the extractor varies outputs. Instead, provide examples normalizing both variants to the same unit.

A missing date would be fabricated to satisfy the schema. A non-null requirement encourages the model to hallucinate when the source field is absent. A missing renewal date is unknown evidence, not a blank slot that authorizes invented calendar data. Instead, include an example with null for missing fields and demonstrate a case with the field genuinely present.

No source renewal field supports that derived year. The date of issue does not imply a renewal date without a documented contract rule. Contrasting an absent field with a real populated field teaches the extractor when to output null versus a supported date. Instead, include an example with null for missing fields and demonstrate a case with the field genuinely present.

The model must still extract genuinely present values. Examples must include populated dates too, or the model may learn to return null universally. Contrasting an absent field with a real populated field teaches the extractor when to output null versus a supported date. Instead, include an example with null for missing fields and demonstrate a case with the field genuinely present.

Footnote-based citations remain unsupported. A parser trained only on in-text markers misses references attached to footnote entries. Examples for both layouts teach the parser structural alternatives while retaining document/page attribution for each claim. Reports include both footnote citations and inline references and a parser works on only one.

Document layout conventions differ between report formats. Page inference from sentence sequence fails when notes and main text use different numbering. Examples for both layouts teach the parser structural alternatives while retaining document/page attribution for each claim. Instead, supply examples for both source structures.

Evidence provenance would be lost. Stripping footnotes discards the connection between a claim and its evidentiary source. Examples for both layouts teach the parser structural alternatives while retaining document/page attribution for each claim. Instead, supply examples for both source structures.

A narrow sample can miss important edge cases. Taking the first three samples can exclude the most consequential ambiguous document types. A few diverse edge and counterexamples usually define boundaries better than hundreds of almost identical normal cases. Instead, select a small set covering distinct boundary cases.

The target output still needs concrete demonstrations. Purely verbal adjectives do not demonstrate the desired schema or reasoning boundary. A few diverse edge and counterexamples usually define boundaries better than hundreds of almost identical normal cases. Instead, select a small set covering distinct boundary cases.

Volume does not cover distinct failure boundaries. Redundant nearby examples leave uncommon error conditions absent from the training context. A few diverse edge and counterexamples usually define boundaries better than hundreds of almost identical normal cases. Hundreds of near-identical examples crowd the context and reduce responsiveness.