One Byte, One Token: Byteifying a 27B Hybrid-Attention LLM with Self-Generated Data
arXiv:2611.13607v1 Announce Type: new Abstract: Byte-level BPE fragments Chinese text (for Qwen3.8 on Chinese Wikipedia, 3.84 bytes per token on average; 64% of tokens are single characters) and hides spelling from the model. We ask whether an already-trained 27B subword model can be converted directly into a pure byte-level model (one byte, one token), without a patching structure or a local encoder, while keeping most of its ability. Starting from Qwen3.8-27B (48 Gated DeltaNet layers + 16 ful
阅读原始来源来源记录
项目分析
暂无五维分析,可先查看上方来源资料。