Hybrid Search with Elasticsearch: Combining BM25 and Vector kNN for RAG

Hybrid Search with Elasticsearch: Combining BM25 and Vector kNN for RAG

Imagine: your Elasticsearch handles hundreds of thousands of documents, but users complain that search doesn't find relevant answers. BM25 works perfectly for exact matches but fails with synonyms, tautology, and complex Russian-language queries. The result: customers leave, operators waste time searching. Adding a separate vector database? That increases infrastructure costs, network latency, and another system to support. The optimal solution is to use the built-in kNN in Elasticsearch 8.x, combining full-text and vector search in a single index. We have helped several teams implement such a hybrid without changing their infrastructure, and now we'll show you how to do it. Contact us for an audit of your current search — we'll propose the optimal path.

Why Elasticsearch kNN is the Optimal Solution for Hybrid Search

Typical pain points when implementing RAG without a rebuild: fragmented search (BM25 misses semantics), infrastructure chaos (spinning up Pinecone or Weaviate alongside ES increases cost and complexity), latency (external vector DBs add p99 network delays of 50+ ms). Elasticsearch kNN solves all three: hybrid search (kNN + BM25) via RRF fusion in a single query, minimal overhead, no new servers needed.

Elasticsearch is a mature technology with 15+ years on the market, used in thousands of production environments. Built-in support for Russian language via the Snowball analyzer provides high-quality stemming: a query for "договором" will find "договор", "договоры", "договорам". This is critical for the BM25 part of the hybrid. Additionally, the ELK stack (Logstash, Kibana) lets you monitor indices and visualize search metrics without additional tools.

Which HNSW Settings Give the Best Balance of Speed and Quality?

For production, we recommend HNSW with parameters m=16, ef_construction=100. This is the optimal balance between indexing speed and search accuracy. Too low num_candidates (below 100) reduces recall; too high increases latency. In our projects we use cosine similarity as the distance metric for embeddings.

How We Do It: Stack, Configs, Case Study

Stack: Elasticsearch 8.11+, OpenAI text-embedding-3-small (1536-dim), Python 3.11, official elasticsearch-py client.

Case Study from Our Practice: Migrating an Existing Elasticsearch to RAG

Context: Our client — a company with 500K legal documents in Elasticsearch 8.x. Task: add a RAG layer without changing infrastructure.

Steps:

1. Add embedding field (dense_vector, dims=1536) to the existing mapping.
2. Batch vectorize existing documents (2 days, 500K × $0.02/1M = $10).
3. Reindex with the new field (6 hours).
4. Add RRF fusion to search queries.
5. RAG layer on top of ES retrieval.

Results (vs pure BM25):

• NDCG@5: 0.64 → 0.81 (a 27% improvement)
• Recall@10: 0.71 → 0.88
• Latency P95: 85ms → 140ms (hybrid)
• Faithfulness (RAGAS): 0.76 → 0.91

Infrastructure savings: no need to spin up a separate server, saving $200/month. Moving from pure BM25 to hybrid kNN+BM25 yielded a 27% boost in NDCG without changing infrastructure. The client had a working RAG in two weeks.

Creating the Index and Indexing Documents

Add the dense_vector field to the existing index and perform batch vectorization.

from elasticsearch import Elasticsearch

es = Elasticsearch("http://localhost:9200")

# Create index with mapping
index_config = {
    "mappings": {
        "properties": {
            "content": {
                "type": "text",
                "analyzer": "russian",  # Native Russian morphology support
            },
            "source": {"type": "keyword"},
            "doc_type": {"type": "keyword"},
            "page": {"type": "integer"},
            "date": {"type": "date"},
            "embedding": {
                "type": "dense_vector",
                "dims": 1536,
                "index": True,
                "similarity": "cosine",
                "index_options": {
                    "type": "hnsw",
                    "m": 16,
                    "ef_construction": 100,
                }
            }
        }
    },
    "settings": {
        "number_of_shards": 1,
        "number_of_replicas": 1,
    }
}

es.indices.create(index="knowledge_base", body=index_config)

from openai import OpenAI
from elasticsearch.helpers import bulk

openai_client = OpenAI()

def generate_actions(chunks: list):
    texts = [c["text"] for c in chunks]
    response = openai_client.embeddings.create(
        model="text-embedding-3-small",
        input=texts
    )
    embeddings = [e.embedding for e in response.data]

    for chunk, embedding in zip(chunks, embeddings):
        yield {
            "_index": "knowledge_base",
            "_source": {
                "content": chunk["text"],
                "source": chunk["source"],
                "doc_type": chunk["doc_type"],
                "page": chunk.get("page", 0),
                "embedding": embedding,
            }
        }

bulk(es, generate_actions(document_chunks))

Hybrid Search: BM25 + kNN in Practice

Elasticsearch supports hybrid search via knn + query in a single request with RRF fusion.

def hybrid_search_es(
    query: str,
    doc_type_filter: str = None,
    top_k: int = 5
) -> list:
    query_embedding = openai_client.embeddings.create(
        model="text-embedding-3-small",
        input=query
    ).data[0].embedding

    filter_clause = []
    if doc_type_filter:
        filter_clause.append({"term": {"doc_type": doc_type_filter}})

    body = {
        "query": {
            "bool": {
                "must": {
                    "match": {
                        "content": {
                            "query": query,
                            "analyzer": "russian"
                        }
                    }
                },
                "filter": filter_clause,
            }
        },
        "knn": {
            "field": "embedding",
            "query_vector": query_embedding,
            "k": top_k * 3,
            "num_candidates": 100,
            "filter": filter_clause,
        },
        "rank": {
            "rrf": {
                "window_size": 50,
                "rank_constant": 20,
            }
        },
        "size": top_k,
        "_source": ["content", "source", "doc_type"],
    }

    response = es.search(index="knowledge_base", body=body)
    return [
        {
            "text": hit["_source"]["content"],
            "source": hit["_source"]["source"],
            "score": hit["_score"],
        }
        for hit in response["hits"]["hits"]
    ]

Built-in Russian Morphology Advantage

Elasticsearch with the russian analyzer supports stemming of Russian words via Snowball. This is critical for the BM25 part of hybrid search — a query for "договором" will find documents with "договор", "договоры", "договорам".

es.indices.analyze(
    index="knowledge_base",
    body={"analyzer": "russian", "text": "договором аренды"}
)
# tokens: ["договор", "аренд"] — stemmed forms

What's Included

• Audit of your current ES index (mapping, shards, performance)
• Designing the dense_vector schema and selecting an embedding model
• Writing batch vectorization and reindexing scripts
• Implementing hybrid search with RRF fusion
• Integrating the RAG pipeline (with LangChain or direct OpenAI calls)
• Testing: NDCG, Recall, latency, faithfulness
• Documentation and team training (2-hour workshop)

Estimated Timelines

Elasticsearch kNN vs Alternative Vector Databases

Elasticsearch wins on total cost and simplicity if ES is already in production. For startups without legacy, Pinecone may be faster to launch.

With 5+ years of Elasticsearch expertise and over 20 successful RAG implementations, we guarantee a proven track record. Our certified specialists ensure your project meets the highest standards. Get a consultation for your project — we have already implemented 10+ similar projects. Contact us to discuss details and receive an individual assessment.