RAG Development with Milvus Vector Database
We use Milvus as the vector store for production-scale RAG and semantic search systems when throughput and data volume requirements exceed what hosted alternatives can handle. Milvus is purpose-built for vector similarity search at scale: billions of vectors, horizontal node scaling, and multiple index types including HNSW, IVF_FLAT, IVF_SQ8, and DISKANN. Its hybrid search feature combines sparse BM25 and dense embedding retrieval in a single query. For enterprise knowledge bases, internal document search, and multi-tenant SaaS products, our team deploys Milvus clusters as part of full-cycle RAG implementations.
Our standard setup starts with schema design tailored to your data shape. We define fields for text content, source metadata, document type, and page number alongside the vector fields. For most knowledge-base applications we configure a dense FLOAT_VECTOR field at 1536 dimensions for text-embedding-3-small and a SPARSE_FLOAT_VECTOR field for BM25. This gives two retrieval paths that are fused with Reciprocal Rank Fusion for better coverage on varied query types. We will assess your requirements and recommend index configuration that balances latency, memory footprint, and accuracy.
The ingestion pipeline we build loads documents, splits them into chunks, generates embeddings, and upserts in batches. Milvus partitions let us isolate tenants within a single collection, which simplifies operations without the cost of per-tenant clusters. Our implementation includes connection pooling, retry logic, and health checks for reliable production operation. We deliver full projects within 3–5 weeks from kickoff, included in the scope from infrastructure setup to monitoring.
Installation and Connection
from pymilvus import connections, Collection, FieldSchema, CollectionSchema, DataType, utility
# Connect to Milvus
connections.connect(
alias="default",
host="localhost",
port="19530"
)
# Or via URI (Milvus Lite for local development)
from pymilvus import MilvusClient
client = MilvusClient("./milvus_local.db") # SQLite-like file
Creating Collection with Schema
fields = [
FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, auto_id=True),
FieldSchema(name="text", dtype=DataType.VARCHAR, max_length=4096),
FieldSchema(name="source", dtype=DataType.VARCHAR, max_length=512),
FieldSchema(name="doc_type", dtype=DataType.VARCHAR, max_length=64),
FieldSchema(name="page", dtype=DataType.INT32),
FieldSchema(
name="dense_vector",
dtype=DataType.FLOAT_VECTOR,
dim=1536 # text-embedding-3-small
),
FieldSchema(
name="sparse_vector",
dtype=DataType.SPARSE_FLOAT_VECTOR # BM25
),
]
schema = CollectionSchema(fields=fields, description="Corporate Knowledge Base")
collection = Collection(name="knowledge_base", schema=schema)
# Indexes for vector fields
collection.create_index(
field_name="dense_vector",
index_params={"metric_type": "COSINE", "index_type": "HNSW", "params": {"M": 16, "efConstruction": 200}}
)
collection.create_index(
field_name="sparse_vector",
index_params={"metric_type": "IP", "index_type": "SPARSE_INVERTED_INDEX"}
)
collection.load()
Hybrid Search with RRF
from pymilvus import AnnSearchRequest, RRFRanker
def milvus_hybrid_search(query: str, top_k: int = 5) -> list:
# Dense vector
dense_vec = dense_embedder.embed_query(query)
# Sparse vector (via built-in BM25Encoder)
sparse_vec = sparse_encoder.encode_queries([query])
# Two queries for RRF
dense_req = AnnSearchRequest(
data=[dense_vec],
anns_field="dense_vector",
param={"metric_type": "COSINE", "params": {"ef": 100}},
limit=30,
)
sparse_req = AnnSearchRequest(
data=sparse_vec,
anns_field="sparse_vector",
param={"metric_type": "IP"},
limit=30,
)
# RRF fusion
results = collection.hybrid_search(
reqs=[dense_req, sparse_req],
rerank=RRFRanker(k=60),
limit=top_k,
output_fields=["text", "source", "doc_type"],
)
return results
Case from Our Practice: Search in Product Documentation
Our client needed semantic search across 2.5 million chunks of technical documentation in six languages, including Russian. We deployed a three-node Milvus cluster with HNSW index (M=32, efConstruction=400) for dense vectors and SPARSE_INVERTED_INDEX for sparse BM25. The resulting system handles 850 QPS at P99 latency under 400 ms per node (8 vCPU, 32 GB RAM each). Compared to Pinecone Serverless at the same volume, our Milvus deployment costs roughly eight times less at high QPS. The trade-off is operational overhead, which we covered with automated health monitoring and rolling restarts.
Partitioning for Multi-Tenancy
# Create partitions for client isolation
collection.create_partition(partition_name="client_001")
collection.create_partition(partition_name="client_002")
# Search only in client partition
results = collection.search(
data=[query_vector],
anns_field="dense_vector",
partition_names=["client_001"],
limit=5,
)
When to Choose Milvus
Milvus makes sense when you need more than 10 million vectors, require sub-400 ms P99 latency at high QPS, need multi-tenancy without per-tenant billing, or want to avoid vendor lock-in on your embedding infrastructure. For smaller projects under one million vectors and low QPS, a hosted service or lightweight local index may be a better fit. We help clients evaluate these trade-offs before committing to infrastructure.
Self-hosting Milvus requires DevOps work: cluster provisioning, backup scheduling, index rebuild automation, and monitoring. We set up Prometheus and Grafana dashboards covering query latency, memory usage, and segment merge status as part of every deployment. These dashboards catch degradation early — for example, when growing data volume starts to push HNSW ef-search times above your latency budget. Our Milvus deployments also include documented runbooks for common operational tasks such as collection compaction, snapshot management, and index rebuild after configuration changes.
Timeline
- Milvus cluster setup and schema design: 3–5 days
- Ingestion pipeline with hybrid indexing: 5–10 days
- RAG pipeline integration and evaluation: 1–2 weeks
- Total: 3–5 weeks
Write to us with your data volume estimate and query throughput requirements. We will recommend whether Milvus is the right fit and scope the project accordingly.