Upstream contribution · lightseekorg/tokenspeed
tokenspeed · Treating token ID zero as a valid end-of-sequence ID
Problem
TokenSpeed used a truthiness check when collecting model and generation EOS IDs. Zero is a valid token ID, but it could be treated as absent or cause a type error when combined with a set of generation IDs.
Engineering response
The merged config change includes EOS ID zero in the same normalization and union path as other explicit IDs.
Evidence and scope
The PR describes a real ModelConfig loaded from a small temporary Qwen3 configuration. Three of seven fixture cases failed on the baseline, and all seven passed on the candidate. This test covers configuration behavior.