aiqbench analysis
This research introduces BITEMBED, an extreme low-bit framework that converts LLM backbones into ternary-weight encoders to drastically reduce storage and inference costs for text embeddings. By employing distillation techniques from full-precision teachers, the method achieves performance comparable to high-cost models while offering flexible precision options tailored to specific index requirements. This advancement directly addresses critical deployment bottlenecks in large-scale semantic retrieval systems by optimizing both encoding efficiency and vector storage overhead.
BitNet Text Embeddings
arXiv:2606.25674v2 Announce Type: replace Abstract: LLM-based text embedders have substantially improved retrieval and semantic representation quality, but their deployment remains costly: large backbone models slow down embedding inference, while high-dimensional full-precision embeddings impose substantial storage and bandwidth overhead on large-scale indexes. In this paper, we present BITEMBED, an extreme low-bit framework for LLM-based text embedding that jointly targets encoding efficiency and vector storage. BITEMBED converts pretrained LLM backbones into BitNet-style embedding encoders with ternary weights, quantized activations, and lightweight normalization refinement. The converted model is adapted to representation learning through continual contrastive pre-training, followed by supervised contrastive fine-tuning with both similarity-distribution distillation and attention-relation distillation from a full-precision teacher. Beyond quantizing the backbone, BITEMBED further trains output embeddings to support multiple storage precisions meeting different storage needs in various scenarios. Experiments on MMTEB (eng, v2) with Qwen3-0.6B and Gemma3-270M show that BITEMBED is largely comparable to full precision teacher embedders. Moreover, BITEMBED flexibly obtains text embeddings of various precisions, achieving a trade-off between performance and storage cost.
Abstract:LLM-based text embedders have substantially improved retrieval and semantic representation quality, but their deployment remains costly: large backbone models slow down embedding inference, while high-dimensional full-precision embeddings impose substantial storage and bandwidth overhead on large-scale indexes. In this paper, we present BITEMBED, an extreme low-bit framework for LLM-based text embedding that jointly targets encoding efficiency and vector storage. BITEMBED converts pretrained LLM backbones into BitNet-style embedding encoders with ternary weights, quantized activations, and lightweight normalization refinement. The converted model is adapted to representation learning through continual contrastive pre-training, followed by supervised contrastive fine-tuning with both similarity-distribution distillation and attention-relation distillation from a full-precision teacher. Beyond quantizing the backbone, BITEMBED further trains output embeddings to support multiple storage precisions meeting different storage needs in various scenarios. Experiments on MMTEB (eng, v2) with Qwen3-0.6B and Gemma3-270M show that BITEMBED is largely comparable to full precision teacher embedders. Moreover, BITEMBED flexibly obtains text embeddings of various precisions, achieving a trade-off between performance and storage cost.
Computation and Language (cs.CL); Information Retrieval (cs.IR)
arXiv:2606.25674 [cs.CL]
(or arXiv:2606.25674v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.25674 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 258 words
Related coverage
Best Embedding Models in 2026
OpenRouter · Sep 23, 2026
Introducing Embed 5—A new family of frontier embedding models
Cohere · Sep 30, 2026
NeoMME: an efficient Multimodal-native and Multilingual Encoder
Hugging Face · Sep 3, 2026
Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers
Hugging Face · Aug 18, 2026