arXiv / 2026-11-13

Distributionally Robust BEST-RQ For Speech Representation Learning

arXiv:2611.14801v1 Announce Type: new Abstract: BEST-RQ, based on random projection and quantization, has been a well-established self-supervised learning approach to learn speech representations from large-scale unlabeled audio due to its high performance and efficiency. However, in BEST-RQ representations learned by random quantization can be sensitive to the choice of quantization targets. To address this issue, we propose a distributionally robust extension of BEST-RQ that seeks representati