The magic of RAG is in the retrieval

Consider a system with embedded Tesla data spanning the company’s history. Without efficient chunking and retrieval mechanisms, a financial analyst inquiring about earnings or a risk analyst searching for lawsuit information would receive a response generated from an overwhelming mix of irrelevant data. This data might include unrelated CEO news and celebrity purchases. The system would produce vague, incomplete, or even hallucinated responses, forcing users to waste valuable time manually sorting through the results to find the information they actually need and then validating its accuracy.

RAG agent-based systems typically serve multiple workflows, and retrieval models and LLMs need to be tailored to their unique requirements. For instance, financial analysts need earnings-focused output, while risk analysts require information on lawsuits and regulatory actions. Each workflow demands fine-tuned output adhering to specific lexicons and formats. While some LLM fine-tuning is necessary, success here primarily depends on data quality and the effectiveness of the retrieval model to filter workflow-specific data points from the source data and feed it to the LLM.

Finally, a well-designed AI agents approach to the automation of complex knowledge workflows can help mitigate risks with RAG deployments by breaking down large use cases into discrete “jobs to be done,” making it easier to ensure relevance, context, and effective fine-tuning at each stage of the system.