Retrieval-Augmented Generation (RAG) is the process of retrieving information that the AI does not have in its original training data to answer a question. To set up RAG, the first step is to embed the data you would like to retrieve. Basically, convert the text, image, or other data to a number the AI knows how to find and retrieve when asked a question which relates to the text, image, etc.
If youâre a student and want to reference a textbook for any questions you have to your AI, you would use RAG to ensure it answers based on the textbook. If a business wants to know âWhat was the total amount of the invoice?â the LLM would access the invoice, comb through it to get the total amount, and answer with the correct number.Choosing the right embedding model is a vital part of building a robust pipeline. Check out the Models Page to evaluate different models and continuously add new, clean data to your dataset to improve the quality of your results.
train_data_subset
branch.