WeWorkBuddy

UltraBERT-JEV: Encoder models for System One Applications

arXiv:2611.17867v1 Announce Type: new Abstract: We study encoder models that pick one of several options in a single forward pass, without generating text. This technical report compares recipes for fine-tuning UltraBERT-380M with one mask per option. The four recipes differ in how an option is scored and how the scores become probabilities; one follows the expert gate of DeepSeek-V3. Trained on the same data as Qwen3-0.6B with the LLM-as-Jev recipe (67.6% test accuracy), the encoder reaches 69.

Read original source
Source reference for UltraBERT-JEV: Encoder models for System One Applications. Screenshot not yet available.
Source record · preview pending · arxiv.orgOpen the source for the full page.

Source record

Project analysis

Project analysis is not yet available. See the source record above.