RAG pipelines look simple in tutorials. In production, they're anything but.
In practice, five layers decide whether a RAG system holds up:
Chunking strategy
Semantic chunking with some overlap usually retrieves better than fixed-size splitting. Use sentence-transformers to find natural boundaries.
Embedding selection
Don't default to one embedding model. For domain-specific content, fine-tuned embeddings can noticeably improve recall. Test with your actual data.
Hybrid search
Combine dense vector search with BM25 keyword search. Pure vector search misses exact matches for product codes, legal citations, and technical terms.
Reranking
A cross-encoder reranker on top-20 results improves answer quality more than any prompt engineering trick. Cohere Rerank and BGE-reranker are our go-tos.
Evaluation framework
Build a golden dataset of 100+ question-answer pairs from your domain. Measure faithfulness, relevance, and context precision on every deployment.