Implementing Semantic Search for Text Documents

Standard full-text search (BM25) fails with synonyms, paraphrasing, and typos. A query like "how to increase team motivation" finds documents on "employee management methods" without a single word match. This is a fundamentally different architecture requiring vector representations and ANN indexes. We implement such systems turnkey, from data audit to production deployment. More about the concept can be read in the article about semantic search.

Semantic Search Architecture

Bi-encoder — the main working mode: separate models encode the query and documents into a common vector space. Search reduces to finding nearest vectors via ANN (Approximate Nearest Neighbor). Cross-encoder works at the reranking stage: it takes a query+document pair and outputs an accurate relevance score. It is slower (O(N) vs O(log N)) but gives maximum precision. The combination bi-encoder (retrieve) + cross-encoder (rerank) is the production standard. According to Reimers & Gurevych, this duo significantly outperforms each method individually.

Compare the main embedding approaches:

Which embedding model to choose?

For Russian language we use cointegrated/rubert-tiny2 as a baseline — fast, compact (312-dim vector). For maximum quality — intfloat/multilingual-e5-large or sbert-base-ru-mean-tokens (768-dim vector). Fine-tuning on your data gives a 5-10% NDCG boost. We select the model based on corpus size and latency requirements (p99 up to 100 ms).

from sentence_transformers import SentenceTransformer, CrossEncoder

# Bi-encoder
bi_encoder = SentenceTransformer("cointegrated/rubert-tiny2")
# For better quality: "intfloat/multilingual-e5-large"

# Cross-encoder
cross_encoder = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")
# For Russian: "DiTy/cross-encoder-russian-msmarco"

Qdrant vs FAISS: Which to choose for production?

Qdrant — production-grade, supports hybrid search, filters, replication. We recommend it for enterprise solutions. FAISS — in-memory index, requires no separate service. Ideal for prototypes and small corpora (<1M vectors).

Example indexing in Qdrant:

from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams, PointStruct

client = QdrantClient("localhost", port=6333)
client.create_collection(
    collection_name="documents",
    vectors_config=VectorParams(size=312, distance=Distance.COSINE),
)

embeddings = bi_encoder.encode(documents, batch_size=64, show_progress_bar=True)
client.upload_points("documents", [
    PointStruct(id=i, vector=emb.tolist(), payload={"text": doc})
    for i, (emb, doc) in enumerate(zip(embeddings, documents))
])

What does hybrid search give?

Semantic search + BM25 outperform each method individually. BM25 catches exact matches (numbers, unique terms), while embeddings capture semantic proximity. Hybrid approach improves NDCG@10 by 2-3 times compared to pure BM25. We use RRF (Reciprocal Rank Fusion) to merge results.

from rank_bm25 import BM25Okapi
bm25 = BM25Okapi([doc.split() for doc in corpus])
semantic_scores = cosine_similarity([query_emb], doc_embeddings)[0]

def rrf(bm25_ranks, semantic_ranks, k=60):
    scores = {}
    for rank, idx in enumerate(bm25_ranks):
        scores[idx] = scores.get(idx, 0) + 1/(k + rank)
    for rank, idx in enumerate(semantic_ranks):
        scores[idx] = scores.get(idx, 0) + 1/(k + rank)
    return sorted(scores, key=scores.get, reverse=True)

Search Quality Evaluation

• NDCG@10 — normalized discounted cumulative gain. Takes order into account.
• MAP — mean average precision across all queries.
• MRR — reciprocal rank of the first relevant result.

Evaluation requires qrels (a set of queries with relevance annotations). We automate its creation: LLM generates questions for each document, the document itself is the "golden" answer. This yields a representative sample for metrics.

Implementation Process and Timeline

1. Data audit: volume, format, language, specific terms. Preprocessing includes cleaning, lemmatization, and chunking (chunk size ~512 tokens with overlap 128).
2. Architecture selection: bi-encoder + cross-encoder, hybrid, custom model. For large corpora (>10M documents) we use Qdrant clustering with sharding.
3. Pipeline development: chunking, embedding, indexing with latency p99 monitoring.
4. Tuning and deployment: Qdrant cluster (Helm charts), A/B testing, canary rollout.
5. Documentation handover, team training (2 sessions of 2 hours), 3-month warranty.

Timeline: from 2 weeks for a prototype, from 2 months for a production solution. Cost is calculated individually — contact us for a free assessment.

What is included in the result

• Detailed architectural documentation.
• Source code of the pipeline with comments.
• Integration with your infrastructure (Elasticsearch, databases, clouds).
• Deployment with Helm charts and CI/CD.
• Team training (2 sessions, 2 hours each).
• Support during industrial operation (1 month).

Why trust us

We are trusted due to 5+ years of experience and 20+ completed projects. All solutions are covered by unit tests and benchmarks. Our engineers are authors of open-source tools for embeddings and ANN. Get a consultation on your project — we will assess the task within 1 day. Order a pilot project to see results on your data.