WeWorkBuddy

Clustering in pure-attention hardmax transformers and its role in sentiment analysis

arXiv:2407.01602v3 Announce Type: replace Abstract: Transformers are extremely successful machine learning models whose mathematical properties remain poorly understood. Here, we rigorously characterize the behavior of transformers with hardmax self-attention and normalization sublayers as the number of layers tends to infinity. By viewing such transformers as discrete-time dynamical systems describing the evolution of points in a Euclidean space, and thanks to a geometric interpretation of the

阅读原始来源
来源参考:Clustering in pure-attention hardmax transformers and its role in sentiment analysis。暂未取得截图。
来源记录 · 预览待采集 · arxiv.org打开来源查看完整网页。

来源记录

项目分析

暂无五维分析,可先查看上方来源资料。