Enhancing RAG Accuracy with Automated Metadata Filtering

Imagine you're searching for "last year's security policies" in a corporate database of 15,000 documents. A regular RAG returns all documents semantically related to "security" — including archive regulations from five years ago. The user drowns in irrelevant results. Self-Query RAG solves this: the LLM analyzes the query and automatically builds a filter: doc_type=policy AND year>=current_year-1 AND status=active, applying it together with vector search. Precision@5 increases from 0.68 to 0.89, the share of archived documents drops from 42% to 3%. This approach is known as metadata search using LLMs, a key aspect of AI document search.

We implement Self-Query RAG turnkey — from metadata labeling to deploying the assistant. Our engineers adapt the solution to any stack: LangChain, Qdrant, Pinecone, Weaviate. Get a consultation — we'll discuss details for your scenario. Total project cost: $10,000–$25,000; estimated annual savings: $50,000–$100,000.

How Self-Query Solves the Filtering Problem

Without Self-Query, a query like "HR department regulations" searches all documents for the words "regulation" and "HR", without filtering by department. You get regulations from IT, Legal, and even marketing instructions. Self-Query forces the LLM to extract filter department=hr AND doc_type=regulation and discard everything unnecessary at the storage level. This saves search time and reduces query processing costs due to accuracy. Companies save up to 40% time on document search and reduce knowledge base maintenance costs by 25%. RAG with filtering is essential for corporate databases; LLM filter extraction is the key innovation.

Comparison with Regular RAG

Self-Query RAG is 1.3x better than regular RAG in Precision@5 and reduces irrelevant results by 92% compared to standard RAG.

Why Self-Query is a Must-Have for Databases with Metadata

Corporate knowledge bases contain documents of different types, departments, and statuses. Without filtering, users get a jumble. Self-Query automatically classifies the query and applies relevant metadata. This is especially important for legal, HR, and financial documents where accuracy is critical.

Metadata Examples for Self-Query

Implementation via LangChain SelfQueryRetriever

from langchain.retrievers.self_query.base import SelfQueryRetriever
from langchain.chains.query_constructor.base import AttributeInfo
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import Qdrant

# Metadata description for LLM
metadata_field_info = [
    AttributeInfo(
        name="doc_type",
        description="Document type: contract, regulation, policy, faq, procedure",
        type="string",
    ),
    AttributeInfo(
        name="department",
        description="Department or division: hr, legal, finance, it, security",
        type="string",
    ),
    AttributeInfo(
        name="year",
        description="Year of publication",
        type="integer",
    ),
    AttributeInfo(
        name="status",
        description="Document status: active, archived, draft",
        type="string",
    ),
    AttributeInfo(
        name="author",
        description="Author or responsible person",
        type="string",
    ),
]

document_content_description = "Corporate documentation: regulations, policies, contracts, procedures"

llm = ChatOpenAI(model="gpt-4o", temperature=0)
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

retriever = SelfQueryRetriever.from_llm(
    llm=llm,
    vectorstore=vectorstore,
    document_contents=document_content_description,
    metadata_field_info=metadata_field_info,
    enable_limit=True,
    verbose=True,
)

LangChain's SelfQueryRetriever simplifies the implementation of self-query rag.

Self-Query in Action

# Example 1: Filter by year and type
result = retriever.invoke(
    "What security policies were active last year?"
)
# LLM generates filter: {"doc_type": "policy", "department": "security", "year": last_year, "status": "active"}

# Example 2: Filter by department
result = retriever.invoke(
    "Show HR department regulations"
)
# Filter: {"doc_type": "regulation", "department": "hr"}

# Example 3: No filter (pure vector search)
result = retriever.invoke(
    "How to prepare for an audit?"
)
# LLM doesn't extract structured filters — pure semantic search

Custom Self-Query Implementation Without LangChain

from pydantic import BaseModel, Field
from typing import Optional
from openai import OpenAI
import json

class SearchFilter(BaseModel):
    semantic_query: str = Field(description="Pure semantic part of the query for vector search")
    doc_type: Optional[str] = Field(default=None, description="Document type")
    department: Optional[str] = Field(default=None, description="Department")
    year_from: Optional[int] = Field(default=None, description="Year from (inclusive)")
    year_to: Optional[int] = Field(default=None, description="Year to (inclusive)")
    status: Optional[str] = Field(default=None, description="Status: active/archived")

def parse_query_to_filter(user_query: str, client: OpenAI) -> SearchFilter:
    response = client.beta.chat.completions.parse(
        model="gpt-4o-mini",
        messages=[{
            "role": "system",
            "content": "Extract structured filters for document search from the user query."
        }, {
            "role": "user",
            "content": user_query
        }],
        response_format=SearchFilter,
        temperature=0,
    )
    return response.choices[0].message.parsed

def self_query_search(user_query: str, vectorstore, top_k: int = 5) -> list:
    filter_obj = parse_query_to_filter(user_query, openai_client)
    qdrant_filter = build_qdrant_filter(filter_obj)
    return vectorstore.similarity_search(
        filter_obj.semantic_query,
        k=top_k,
        filter=qdrant_filter,
    )

Qdrant filters enable efficient metadata filtering, especially when combined with prompt fine-tuning for LLM filter extraction.

Case Study: Corporate Knowledge Base

Challenge: a search assistant for 15,000 internal documents with metadata (type, department, year, status, author).

Before Self-Query: 42% of queries returned archived documents instead of current ones.

After Self-Query (our client — a company of 500+ employees):

• Archived documents in results for "current" queries: 42% → 3%
• Precision@5: 0.68 → 0.89
• User satisfaction: +31%

Failure cases: LLM may misinterpret filter parameters on ambiguous queries. Solution: add a confidence threshold and fallback to pure semantic search when confidence is low. We fine-tune the prompt if needed to improve filter extraction quality. Prompt fine-tuning is crucial for accurate metadata search.

How to Implement Self-Query: Step by Step

1. Audit documents and metadata — define fields for filtering.
2. Label or automatically extract metadata (NLP classification) — this is a key part of automatic document classification.
3. Choose a vector database and configure indexing.
4. Develop the LLM prompt and integrate the Self-Query Retriever.
5. A/B testing and threshold tuning.

What's Included in the Work

• Documentation: metadata schema, prompt description, extension guide.
• Access control: user permission differentiation via document statuses.
• Training: 2 hours for system administrators.
• Support: 1 month post-launch.

Timeline and Cost

• Metadata labeling: 1–3 weeks (depends on data availability).
• Self-Query Retriever implementation: 3–5 days.
• Testing and prompt tuning: 3–5 days.
• Total: 2–5 weeks. For a typical project, the cost ranges from $10,000 to $25,000, with annual savings of $50,000–$100,000. Implementation costs start at $10,000, with annual savings up to $100,000.

We have been working with RAG for over 5 years and completed 30+ projects. We guarantee transparent architecture and documentation. Get a consultation — we'll discuss the details of your task. Request a demo — we'll show it on your data.

Source: Retrieval-augmented generation (Wikipedia)