WeWorkBuddy

LENS: Replacing LayerNorm and Softmax in Transformers with Learnable Normalization States

arXiv:2611.20331v1 Announce Type: cross Abstract: LayerNorm and attention Softmax are critical operations in Transformers. However, they cannot be efficiently mapped to General Matrix Multiplication (GEMM) computations on hardware. Therefore, modern hardware accelerator designs, such as NPUs on ASIC/FPGA, consume a large amount of hardware resources or chip area to implement their specific arithmetic units. Building on previous work about LayerNorm, Softmax, and their alternatives, we introduce

阅读原始来源
来源参考:LENS: Replacing LayerNorm and Softmax in Transformers with Learnable Normalization States。暂未取得截图。
来源记录 · 预览待采集 · arxiv.org打开来源查看完整网页。

来源记录

项目分析

暂无五维分析,可先查看上方来源资料。