Fixing Generative AI with Your Own Data

Integrating your own data into Generative AI can feel like a breakthrough, especially when employing Retrieval-Augmented Generation (RAG). Generally, this involves embedding data into a vector store, allowing for the retrieval of relevant context through similarity searches when answering queries.

But here's the catch: this entire process hinges on the assumption that a user's query will closely match documents with the answers they're seeking. Unfortunately, this isn't always the case.

Overcoming Limitations: Rewrite-Retrieve-Read

One strategy to tackle this issue is through a technique called Rewrite-Retrieve-Read. Here, a Large Language Model (LLM) reformulates the user's question to align better with the retrieval process. The outcomes of this method can vary, as we've observed in practice.

Exploring RAG-Fusion

A more promising alternative is known as Multi-Query Retrieval or RAG-Fusion. This method involves generating and combining multiple variations of the initial query. In our experience, this consistently enhances the quality of results. However, it's important to note that this enhancement often comes at the cost of increased latency.

Hypothetical Document Embeddings (HyDE)

Then there's the rather intriguing approach of Hypothetical Document Embeddings (HyDE). With HyDE, a smaller LLM crafts a hypothetical answer which isn't necessarily correct but serves as a better anchor for similarity searches than the original query. We've found that while this approach boosts overall quality in customer projects, it can sometimes detract from the quality of certain reference questions.

Merging Approaches

There's also potential in combining Multi-Query Retrieval with HyDE. If you're willing to set aside concerns about latency, this hybrid approach holds significant promise for improving results.

Final Thoughts

For anyone ready to dive into the depths of optimizing or even creating a RAG application with tailored enhancements, these insights might offer the leap forward you've been looking for.