Embedding models control which passages an AI agent can see, making the choice of encoder critical for retrieval‑augmented generation, code search, and agent memory. NVIDIA’s Nemotron 3 Embed family gives three ready‑to‑use checkpoints that fit different production needs. The 8 billion‑parameter BF16 version leads the Retrieval Embedding Benchmark with an average NDCG@10 of 78.46, delivering the highest accuracy for large‑scale RAG. A 1.1 billion‑parameter BF16 model keeps most of that performance while cutting compute and memory requirements. The same 1B model is also available in a 4‑bit NVFP4 format that runs up to twice as fast on Blackwell hardware and retains over 99 % of the BF16 retrieval score.
All checkpoints share a transformer encoder trained with bidirectional attention masking and average pooling, supporting sequences up to 32 768 tokens. The 1B variants were created by pruning a 3B parent with NVIDIA ModelOpt NAS, then distilling knowledge from the 8B teacher using cosine distance and mean‑squared‑error losses. The NVFP4 version adds quantization‑aware distillation to recover accuracy on long inputs.
To use the models, prefix queries with “query:” and documents with “passage:” before encoding; the resulting vectors are L2‑normalized so dot product equals cosine similarity. In code, load the checkpoint with Sentence Transformers, set the dtype to bfloat16 for the 8B model, and call encode_query and encode_document. For serving, deploy the vLLM endpoint with the appropriate max‑model‑len and input_type flag to let the server apply the prefixes automatically.
Typical applications include multilingual enterprise search where a German query can retrieve a Japanese document, natural‑language‑to‑code lookup that benefits from the model’s code‑focused training, and agent memory that stores long conversation summaries without aggressive chunking. A cost‑tiered strategy can serve the fast 1B‑NVFP4 index for high‑volume recall and route difficult cases to the 8B‑BF16 index, using separate vector stores for the different embedding dimensions.
#AI #Product #RAG #Embedding #NVIDIA #ML