The RPRS model addresses the limitations of current search methods for long documents. It computes relevance between a query document and candidate documents based on proportional matches across their sentences. The approach consists of three stages: sentence encoding, finding the most relevant sentence sets, and proportion-based relevance scoring. The RPRS method significantly outperforms previous techniques on legal, patent, and Wikipedia datasets, demonstrating its effectiveness in capturing document semantics.
Dominant Search Methods and Their Limitations
Current search methods rely on keywords or vector space similarity to determine relevance between a query and documents. However, these techniques struggle when it comes to searching through entire files, papers, or books as search queries.
Keywords searches are great for short lookups, but they fail to capture the semantics required for long-form content. Documents that discuss “cloud platforms” may be missed if the query is looking for expertise in “AWS.” Exact term matches also face issues with vocabulary mismatch in lengthy texts.
Vector embedding models like BERT can accurately estimate semantic similarity, but they have limitations in terms of the number of tokens they can handle. This limits their ability to fully analyze long documents, and the resulting partial embeddings lose the nuances of meaning across different sections.
The compute complexity of these models also restricts their accuracy when fine-tuning on real-world corpora. Unsupervised learning is an alternative, but solid techniques are lacking.
Introducing the RPRS Model
A recent paper addresses these limitations by introducing the RPRS (Proportional Relevance) model for document search. This model aims to compute relevance between a long query document and candidate documents by analyzing the proportional matches across their sentences.
The RPRS model consists of three key stages:
1. Sentence Encoding
Sentences from queries and candidate documents are encoded into vectors using SBERT, an efficient transformer architecture for sentence embeddings. SBERT allows for the incorporation of full document lengths, avoiding the quadratic complexity faced by other models.
2. Most Relevant Sentence Sets
For each query sentence, the model identifies the most similar candidate document sentences based on vector embeddings. Sets of the most relevant document sentences for every query sentence are determined.
3. Proportion-based Relevance Scoring
The model defines the Query Proportion (QP) and Document Proportion (DP) to compute a final relevance score. QP represents the relative proportion of query sentences that have similarity to document sentences, while DP represents the relative proportion of document sentences that are similar to query sentences. The final relevance score estimates the inter-relatedness of the texts.
The RPRS model accounts for document structure within long-form text and can handle repetition and length bias through an extension called RPRS w/freq.
Results and Implications
The RPRS model has been evaluated on various long-document datasets, including legal case retrieval, patent search, and Wikipedia document similarity tasks. It significantly outperformed previous state-of-the-art techniques and demonstrated its effectiveness with just three tuned parameters.
This model combines semantic matching capability with an intuitive notion of topical relevance, providing interpretable high accuracy retrieval. It addresses the limitations faced by current search methods and expands the scope of neural search paradigms to ultra-long text.
Using AI to Evolve Your Company
If you want to evolve your company with AI and stay competitive, consider incorporating the principles of the RPRS model. AI can redefine your way of work by automating customer interactions, identifying automation opportunities, and selecting AI solutions that align with your needs.
At itinai.com, we offer an AI Sales Bot that automates customer engagement and manages interactions across all customer journey stages. Explore our solutions to redefine your sales processes and customer engagement.
For AI KPI management advice and continuous insights into leveraging AI, connect with us at hello@itinai.com or follow us on Telegram t.me/itinainews and Twitter @itinaicom.
