Question 550
A fine-tuned model adds custom special tokens during training, but the release package retains only adapter weights and excludes the expanded tokenizer state. Inference cannot reproduce the trained input representation. What must be packaged?
Explanation
Correct Answer
Added or remapped tokens affect embeddings and parsing; a reproducible release must include the exact compatible tokenizer artifacts.
Incorrect Answers
Adapter weights alone do not define the token IDs used during training.
Equal vocabulary size does not imply identical token-to-ID assignments.
More compute cannot reconstruct missing serialization state.