Enhancing AI Search Precision: NVIDIA Boosts RAG Pipelines with Re-Ranking

In the rapidly evolving landscape of AI-driven applications, re-ranking has emerged as a pivotal technique to enhance the precision and relevance of enterprise search results, according to the NVIDIA Technical Blog. By leveraging advanced machine learning algorithms, re-ranking refines initial search outputs to better align with user intent and context, significantly improving the effectiveness of semantic search.

Role of Re-Ranking in AI

Re-ranking plays a crucial role in optimizing retrieval-augmented generation (RAG) pipelines, ensuring that large language models (LLMs) operate with the most pertinent and high-quality information. This dual benefit of re-ranking—enhancing both semantic search and RAG pipelines—makes it an indispensable tool for enterprises aiming to deliver superior search experiences and maintain a competitive edge in the digital marketplace.

What is Re-Ranking?

Re-ranking is a sophisticated technique used to enhance the relevance of search results by utilizing the advanced language understanding capabilities of LLMs. Initially, a set of candidate documents or passages is retrieved using traditional information retrieval methods like BM25 or vector similarity search. These candidates are then fed into an LLM that analyzes the semantic relevance between the query and each document. The LLM assigns relevance scores, enabling the re-ordering of documents to prioritize the most pertinent ones.

This process significantly improves the quality of search results by going beyond mere keyword matching to understand the context and meaning of the query and documents. Re-ranking is typically used as a second stage after an initial fast retrieval step, ensuring that only the most relevant documents are presented to the user. It can also combine results from multiple data sources and integrate into a RAG pipeline to further ensure that context is ideally tuned for the specific query.

NVIDIA's Implementation of Re-Ranking

In this post, the NVIDIA Technical Blog illustrates the use of the NVIDIA NeMo Retriever reranking NIM. This transformer encoder, a LoRA fine-tuned version of Mistral-7B, uses only the first 16 layers for higher throughput. The last embedding output by the decoder model is used as a pooling strategy, and a binary classification head is fine-tuned for the ranking task.

To access the NVIDIA NeMo Retriever collection of world-class information retrieval microservices, see the NVIDIA API Catalog.

Combining Results from Multiple Data Sources

In addition to enhancing accuracy for a single data source, re-ranking can be used to combine multiple data sources in a RAG pipeline. Consider a pipeline with data from a semantic store and a BM25 store. Each store is queried independently and returns results that the individual store considers to be highly relevant. Figuring out the overall relevance of the results is where re-ranking comes into play.

The following code example combines the previous semantic search results with BM25 results. The results in combined_docs are ordered by their relevance to the query by the reranking NIM.

all_docs = docs + bm25_docs

reranker.top_n = 5

combined_docs = reranker.compress_documents(query=query, documents=all_docs)

Connecting to a RAG Pipeline

In addition to using re-ranking independently, it can be added to a RAG pipeline to further enhance responses by ensuring that they use the most relevant chunks for augmenting the original query.

In this case, connect the compression_retriever object from the previous step to the RAG pipeline.

from langchain.chains import RetrievalQA
from langchain_nvidia_ai_endpoints import ChatNVIDIA

chain = RetrievalQA.from_chain_type(
    llm=ChatNVIDIA(temperature=0), retriever=compression_retriever
)
result = chain({"query": query})
print(result.get("result"))

The RAG pipeline now uses the correct top-ranked chunk and summarizes the main insights:

The A100 GPU is used for training the 7B model in the supervised 
fine-tuning/instruction tuning ablation study. The training is 
performed on 16 A100 GPU nodes, with each node having 8 GPUs. The 
training hours for each stage of the 7B model are: projector 
initialization: 4 hours; visual language pre-training: 30 hours; 
and visual instruction-tuning: 6 hours. The total training time 
corresponds to 5.1k GPU hours, with most of the computation being 
spent on the pre-training stage. The training time could potentially 
be reduced by at least 30% with proper optimization. The high image 
resolution of 336 ×336 used in the training corresponds to 576 
tokens/image.

Conclusion

RAG has emerged as a powerful approach, combining the strengths of LLMs and dense vector representations. By using dense vector representations, RAG models can scale efficiently, making them well-suited for large-scale enterprise applications, such as multilingual customer service chatbots and code generation agents.

As LLMs continue to evolve, RAG will play an increasingly important role in driving innovation and delivering high-quality, intelligent systems that can understand and generate human-like language.

When building a RAG pipeline, it’s crucial to correctly split the vector store documents into chunks by optimizing the chunk size for the specific content and selecting an LLM with a suitable context length. In some cases, complex chains of multiple LLMs may be required. To optimize RAG performance and measure success, use a collection of robust evaluators and metrics.

For more information about additional models and chains, see NVIDIA AI LangChain endpoints.