Production RAG Pipeline: What Demos Don't Teach You
Retrieval-Augmented Generation works in demos. Production RAG requires decisions about chunking, embedding models, vector databases, retrieval quality, hallucination mitigation, and cost control.
Architecture
- Ingestion — document parsing, chunking, embedding
- Storage — vector database (Pinecone, Qdrant, pgvector)
- Retrieval — similarity search + reranking
- Generation — LLM with context window management
- Guardrails — hallucination detection, citation grounding
Key Decisions
- Chunking: 512 tokens with 50-token overlap for most documents
- Embedding: OpenAI text-embedding-3-small for cost, BGE-large for self-hosted
- Vector DB: pgvector if you already use PostgreSQL, Qdrant for scale
- Reranker: Cohere Rerank or cross-encoder for precision
FAQ
Cost at scale?
~$0.10-$0.50 per query at 1M docs. Embedding is the biggest cost.
Free AI engineering courses. Code CLOUDPATH for 20% off RAG tools.