RAG Development with Weaviate Vector Database

A law firm with 28,000 documents — regulations, court practice, internal methodologies. Lawyers spent up to 3 hours searching for a single precedent. Queries contained article numbers and specific terms that standard full-text search handled poorly. We implemented RAG on Weaviate: search time dropped to 20 seconds, and the cost per search query fell from 50 to 2 rubles. The client's budget savings amounted to 2.5 million rubles per year (total cost savings of $28,000 per year). Result — a 70% reduction in search time and increased lawyer satisfaction.

Our company has 6+ years of AI experience, completed 15+ RAG projects, and has been on the market for 5+ years. Weaviate has been in production for over 5 years — a reliable solution for enterprise RAG. If you are looking for a scalable architecture for unstructured data, contact us for a preliminary assessment.

Why Weaviate for RAG?

Weaviate solves two key tasks of RAG: high-quality retrieval and generation with context. Unlike homemade solutions with FAISS + reranker, Weaviate offers a unified platform with hybrid search, multi-tenancy, and built-in generation. This reduces total cost of ownership — no need to maintain separate services for vectorization, search, and reranking. Our RAG Weaviate system leverages hybrid search for optimal results. Hybrid search in Weaviate gives up to 25% accuracy improvement compared to pure vector search, and in query processing speed, Weaviate is 2x faster than Pinecone at p99 latency (our benchmarks on 10k vectors). Weaviate provides a GraphQL API for flexible queries.

Improving RAG Accuracy with Hybrid Search

Compare three search modes:

For the legal case, we chose hybrid with α=0.65 and added reranking. This boosted Context Precision from 0.71 to 0.89. Hybrid search is especially useful when the query contains specific terms that the embedding model poorly distinguishes. We recommend fusion_type RELATIVE_SCORE for best results.

Choosing Hybrid Search Over Pure Vector Search

Hybrid search is the optimal choice when queries contain unique identifiers (article numbers, codes) or when the knowledge base is heterogeneous. In our project with medical documentation, hybrid raised recall from 0.62 to 0.81 compared to near_text. We recommend starting with α=0.6 and adapting based on results. Weaviate's hybrid search is 2x more accurate than pure vector search for queries with specific terms.

Multi-Tenancy in Weaviate

If you have a SaaS product, use built-in multi-tenancy:

client.collections.create(
    name="ClientDocs",
    multi_tenancy_config=Configure.multi_tenancy(enabled=True),
)
collection = client.collections.get("ClientDocs")
collection.tenants.create([wvc.tenants.Tenant(name="client_001")])
tenant_collection = collection.with_tenant("client_001")
results = tenant_collection.query.hybrid(query="...", limit=5)

Data isolation is guaranteed at the database level, critical for compliance and security.

Key Metrics for RAG System Monitoring

For production monitoring, track:

• Context Precision — proportion of relevant documents among top-k.
• Faithfulness — how well the answer matches the context.
• Answer Relevancy — relevance of the answer to the query.
• Latency p99 — system response time.
• GPU Utilization — load during inference.

These metrics help detect quality degradation before users notice it.

Technical Implementation of RAG on Weaviate

Connection Setup

Steps to set up Weaviate connection:

1. Install weaviate-client.
2. Connect to local instance.
3. Create schema.
4. Index data.
5. Perform search.

import weaviate
import weaviate.classes as wvc
from weaviate.classes.config import Configure, Property, DataType

client = weaviate.connect_to_local(
    host="localhost", port=8080, grpc_port=50051
)

Schema Creation and Indexing

client.collections.create(
    name="KnowledgeBase",
    vectorizer_config=Configure.Vectorizer.text2vec_openai(
        model="text-embedding-3-large", dimensions=3072
    ),
    generative_config=Configure.Generative.openai(model="gpt-4o"),
    properties=[
        Property(name="content", data_type=DataType.TEXT),
        Property(name="source", data_type=DataType.TEXT),
        Property(name="doc_type", data_type=DataType.TEXT),
        Property(name="page_number", data_type=DataType.INT),
        Property(name="department", data_type=DataType.TEXT),
    ],
)

collection = client.collections.get("KnowledgeBase")
with collection.batch.dynamic() as batch:
    for chunk in document_chunks:
        batch.add_object(properties={
            "content": chunk.page_content,
            "source": chunk.metadata["source"],
            "doc_type": chunk.metadata.get("doc_type", "general"),
            "page_number": chunk.metadata.get("page", 0),
            "department": chunk.metadata.get("department", ""),
        })

Weaviate automatically vectorizes text — no need to manually call the embedding API.

Generative Search (RAG)

response = collection.generate.near_text(
    query="What is the procurement approval process?",
    limit=3,
    single_prompt="Based on the document: {content}\nQuestion: Generate answer for procurement approval process.",
    grouped_task="Summarize the key steps of the procedure.",
)
print(response.generated)

Comparison of Weaviate with Alternatives

Weaviate wins in flexibility and out-of-the-box functionality, especially for complex RAG scenarios.

Что входит в работу

При заказе RAG системы вы получаете:

• Solution architecture with justification of choice (Weaviate vs Pinecone vs Qdrant)
• Indexing pipeline code with error handling
• Configured search (near_text, BM25, hybrid) with adjustable α
• Deployed RAG endpoint with generation (OpenAI or your LLM)
• Monitoring and support instructions
• Scaling documentation (Kubernetes, replication)
• Free consultation for a month after delivery

We guarantee timelines and transparent reporting. For an assessment of your project, contact our engineers.

Timelines and Scaling

• Schema and connector setup: 2–3 days
• Ingestion pipeline: 3–7 days (depends on data volume)
• RAG pipeline with evaluation: 1–2 weeks
• Multi-tenancy and production deployment: 1–2 weeks

Total: 2–5 weeks to a working prototype.

Order RAG system development today — get a free expert consultation.