What special: true Actually Changes
It changes nothing when you tokenize. It changes everything when you detokenize, and only if skip_special_tokens is on. Measured against GLM 5.2's tokenizer.
I found the question in my own eval script, which is the embarrassing way to find things.
12text = tokenizer.decode(output[0, input_len:], skip_special_tokens=True).strip()text = re.sub(r"<......