LENS: Replacing LayerNorm and Softmax in Transformers with Learnable Normalization States
arXiv:2611.20331v1 Announce Type: cross Abstract: LayerNorm and attention Softmax are critical operations in Transformers. However, they cannot be efficiently mapped to General Matrix Multiplication (GEMM) computations on hardware. Therefore, modern hardware accelerator designs, such as NPUs on ASIC/FPGA, consume a large amount of hardware resources or chip area to implement their specific arithmetic units. Building on previous work about LayerNorm, Softmax, and their alternatives, we introduce
阅读原始来源来源记录
项目分析
暂无五维分析,可先查看上方来源资料。