RAG with FAISS: Local Vector Search Without Cloud

Vector Search Without the Cloud: FAISS as a Solution for RAG

Cloud vector databases are expensive, slow, and insecure for internal documents. Our team regularly encounters requests for local RAG where every millisecond of latency matters. FAISS by Meta is not a database but a high-performance vector search engine that operates in memory or on disk without network interaction. We use it for embedding into applications, offline scenarios, and situations where an external service is unacceptable. Five years of experience in NLP and 15+ implemented RAG projects allow us to guarantee pipeline stability. According to tests, local FAISS on GPU saves up to 70% compared to cloud services like Pinecone or Weaviate. Switching to local FAISS saves significant amounts monthly for 10 million vectors. Get a consultation on implementing FAISS in your project.

Problems Solved by FAISS

High embedding cost and latency. With batch requests to the OpenAI API, delays grow linearly. FAISS on a local GPU gives p99 latency <5 ms for 100K vectors — 50 times faster than cloud solutions. Data confidentiality. Financial reports, medical records, trade secrets — we do not send embeddings to external services. FAISS stores everything locally. Offline mode. Field devices, closed loops — FAISS works without internet. Order an audit of your current pipeline to identify bottlenecks.

Why Is FAISS Faster Than Traditional Databases?

FAISS vector search does not use SQL, B-trees, or metadata filtering — instead, it applies approximate search algorithms: IVF (Inverted File), HNSW (Hierarchical Navigable Small World), and Product Quantization. Compare:

How We Build RAG with FAISS: Stack and Case Study

Typical stack: Python 3.10+, FAISS 1.7+, OpenAI text-embedding-3-small (1536 dim), GPT-4o-mini for generation. For one client with a corpus of 50,000 technical articles, we chose IndexHNSWFlat (M=16, efConstruction=200). This gave 98% recall and 2 ms latency per query.

import faiss
import numpy as np
import pickle
from openai import OpenAI

openai_client = OpenAI()

def build_faiss_index(texts: list[str], dimension: int = 1536) -> tuple:
    """Creates a FAISS index and corresponding list of texts"""

    # Получаем embeddings батчами
    embeddings = []
    batch_size = 100
    for i in range(0, len(texts), batch_size):
        batch = texts[i:i + batch_size]
        response = openai_client.embeddings.create(
            model="text-embedding-3-small",
            input=batch,
        )
        batch_embeddings = [e.embedding for e in response.data]
        embeddings.extend(batch_embeddings)

    # Конвертируем в numpy float32
    vectors = np.array(embeddings, dtype=np.float32)

    # Нормализуем для cosine similarity (через inner product)
    faiss.normalize_L2(vectors)

    # Создаём HNSW индекс
    index = faiss.IndexHNSWFlat(dimension, 16)  # M=16
    index.hnsw.efConstruction = 200
    index.add(vectors)

    return index, texts

# Сохранение на диск
def save_index(index, texts, path_prefix: str):
    faiss.write_index(index, f"{path_prefix}.index")
    with open(f"{path_prefix}_texts.pkl", "wb") as f:
        pickle.dump(texts, f)

# Загрузка
def load_index(path_prefix: str) -> tuple:
    index = faiss.read_index(f"{path_prefix}.index")
    with open(f"{path_prefix}_texts.pkl", "rb") as f:
        texts = pickle.load(f)
    return index, texts

Search and RAG answer:

def faiss_rag_answer(
    question: str,
    index: faiss.Index,
    texts: list[str],
    top_k: int = 5
) -> str:
    # Embedding вопроса
    query_embedding = openai_client.embeddings.create(
        model="text-embedding-3-small",
        input=question,
    ).data[0].embedding

    query_vector = np.array([query_embedding], dtype=np.float32)
    faiss.normalize_L2(query_vector)

    # Поиск
    distances, indices = index.search(query_vector, top_k)

    # Извлечение текстов
    context_texts = [texts[i] for i in indices[0] if i >= 0]
    context = "\n\n---\n\n".join(context_texts)

    # Генерация ответа
    response = openai_client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Отвечай строго на основе предоставленного контекста."},
            {"role": "user", "content": f"Контекст:\n{context}\n\nВопрос: {question}"}
        ],
        temperature=0,
    )
    return response.choices[0].message.content

How to Accelerate FAISS on GPU?

Transferring the index to GPU gives up to 100× performance boost. We use faiss.StandardGpuResources:

res = faiss.StandardGpuResources()
gpu_index = faiss.index_cpu_to_gpu(res, 0, index)  # GPU 0
distances, indices = gpu_index.search(query_vectors, top_k)

RAG Pipeline Development Process

1. Analysis: Examine the corpus, requirements for latency, accuracy (recall), update volume. Choose the index type.
2. Design: Define the pipeline — embedder (OpenAI / local), index, LLM (GPT-4o-mini / Claude). Work out the context format.
3. Implementation: Deploy indexing and search code, integrate with existing application (API / embedded).
4. Testing: Measure latency, recall, answer quality. Optimize parameters (M, efSearch, batch size).
5. Deployment: Containerization (Docker), CI/CD, monitoring metrics (p99 latency, QPS, GPU utilization).

What's Included in the Work?

• Pipeline architecture (documentation + diagram)
• Indexing and search code (Python, versioned Git)
• Index configuration tailored to your corpus (recommendations for HNSW/IVF)
• Integration with LLM (OpenAI, local models via vLLM)
• Load testing (report with metrics)
• Post-deployment support (2 weeks warranty maintenance)

Typical Mistakes When Implementing FAISS

1. Normalization not performed. Without L2 normalization, inner product is not equivalent to cosine similarity — results degrade by 10–20%.
2. Choosing the wrong index. IndexFlatL2 on 10M vectors consumes >60 GB RAM — use IVFPQ.
3. efSearch not tuned. For HNSW, efSearch < 100 reduces recall to 80% — raise to 200–400.
4. Ignoring GPU memory. For large indexes, the index does not fit on GPU — switch to IVF with quantization.

Comparison of FAISS Index Types

Timeline and How to Order

Estimated timeline: for a corpus up to 500K vectors — 1–2 weeks turnkey. For larger volumes (>10M) — 3–4 weeks with index optimization. Contact us for an assessment of your scenario — we will select a configuration and calculate the cost individually. Get a consultation on implementing FAISS in your project. Order an audit of your current pipeline to identify bottlenecks.