A Retrieval Pipeline is the sequence of steps an AI application uses to find and prepare relevant information for a language model. It is the retrieval half of Retrieval-Augmented Generation (RAG): given a user query, the pipeline searches an external knowledge source, ranks the results, and assembles the most relevant passages into the context an LLM uses to generate its answer.
This is distinct from general-purpose data pipelines or ETL tools (such as Apache Beam, Kafka, or Airflow), which move and transform data between systems but are not concerned with ranking content by relevance to a query.
Stages of a Retrieval Pipeline
-
Query processing — The incoming user query is cleaned up and sometimes rewritten or expanded (for example, breaking a multi-part question into sub-queries) to improve retrieval quality.
-
Embedding — The query is converted into a vector representation using the same embedding model that indexed the knowledge source, so it can be compared against stored document vectors. See vectorization.
-
Retrieval / vector search — The query vector is compared against a vector database (or a hybrid of vector and keyword search) to find the most similar or relevant document chunks.
-
Reranking — A secondary, often more precise, model reorders the initial candidate set to surface the passages most likely to answer the query, trading extra compute for better precision.
-
Context assembly — The top-ranked passages are formatted and inserted into the prompt, within the model's context window, along with the original query and any system instructions.
Why Retrieval Pipelines Matter
The quality of a retrieval pipeline directly determines the quality of a RAG system's output. If the pipeline retrieves irrelevant or incomplete passages, the language model has no way to compensate, since it can only reason over what it is given. Common failure points include poor chunking of source documents, embedding models mismatched to the domain, and skipping reranking on noisy retrieval results.
Retrieval pipelines are typically evaluated with metrics like context precision, context recall, and answer relevance, as discussed in RAGAS and covered in the "How do you evaluate Retrieval systems and RAG pipelines?" section of the RAG glossary page.
Related Terms