Face Identification System (1:N) Development

Face Identification System (1:N) Development

When searching for a person in a database of 500,000 faces without using ANN, latency exceeds 100 ms — unacceptable for real-time access control systems. We solve this problem using FAISS and optimal index architecture. Our engineers hold PyTorch and FAISS certifications and have experience implementing systems with databases of up to 10 million faces. With over 5 years in the CV market, we develop turnkey systems: from architecture selection to integration into existing infrastructure. We’ll evaluate your project within 2 days and propose the optimal solution.

How to Scale Identification for Millions of Faces?

With a database exceeding 100k faces, brute-force search becomes unacceptably slow. We use a hierarchy of approaches depending on database size. FAISS IVFFlat delivers a 50x speedup over brute-force for a 1 million face database, while IVFPQ compresses vectors by 8x with minimal accuracy loss. The choice of index directly impacts latency and memory consumption.

import faiss
import numpy as np
from dataclasses import dataclass

@dataclass
class IdentificationResult:
    person_id: str | None
    person_name: str | None
    similarity: float
    identified: bool

class FaceIdentificationSystem:
    def __init__(self, embedding_dim: int = 512,
                 n_lists: int = 100,
                 threshold: float = 0.45):
        self.dim = embedding_dim
        self.threshold = threshold

        quantizer = faiss.IndexFlatIP(embedding_dim)
        self.index = faiss.IndexIDMap(
            faiss.IndexIVFFlat(quantizer, embedding_dim, n_lists,
                               faiss.METRIC_INNER_PRODUCT)
        )
        self.index.nprobe = 20

        self.id_map = {}
        self._next_id = 0

    def register(self, person_id: str, name: str,
                 embeddings: np.ndarray) -> int:
        faiss.normalize_L2(embeddings)
        ids = np.arange(self._next_id, self._next_id + len(embeddings))

        if not self.index.is_trained:
            self.index.train(embeddings)

        self.index.add_with_ids(embeddings, ids)

        for fid in ids:
            self.id_map[int(fid)] = {'person_id': person_id, 'name': name}

        self._next_id += len(embeddings)
        return len(embeddings)

    def identify(self, query_embedding: np.ndarray,
                 k: int = 5) -> IdentificationResult:
        query = query_embedding.reshape(1, -1).copy()
        faiss.normalize_L2(query)

        similarities, faiss_ids = self.index.search(query, k)

        best_sim = float(similarities[0][0])
        best_id = int(faiss_ids[0][0])

        if best_id == -1 or best_sim < self.threshold:
            return IdentificationResult(None, None, best_sim, False)

        person_info = self.id_map[best_id]
        return IdentificationResult(
            person_info['person_id'],
            person_info['name'],
            best_sim,
            True
        )

Why Index Choice Is Critical for Latency?

p99 latency is a key metric for real-time systems. With an incorrect index, search time grows linearly with database size. We guarantee that latency will not exceed 15 ms for databases up to 1 million faces with a properly tuned IVFFlat and nprobe=20. For larger databases, we use GPU acceleration via FAISS GPU.

Multiple Images per Person

Registering multiple photos with different angles and lighting conditions improves recall. When identifying with aggregation across all photos of a person:

def identify_with_aggregation(self, query_emb: np.ndarray,
                               k: int = 10) -> IdentificationResult:
    query = query_emb.reshape(1, -1).copy()
    faiss.normalize_L2(query)
    similarities, faiss_ids = self.index.search(query, k)

    votes = {}
    for sim, fid in zip(similarities[0], faiss_ids[0]):
        if fid == -1:
            continue
        pid = self.id_map[int(fid)]['person_id']
        votes[pid] = votes.get(pid, 0) + float(sim)

    if not votes:
        return IdentificationResult(None, None, 0.0, False)

    best_pid = max(votes, key=votes.get)
    best_score = votes[best_pid] / k

    if best_score < self.threshold:
        return IdentificationResult(None, None, best_score, False)

    name = self.id_map[next(
        fid for fid, info in self.id_map.items()
        if info['person_id'] == best_pid
    )]['name']
    return IdentificationResult(best_pid, name, best_score, True)

Closed-set vs Open-set Identification

Closed-set: all queries belong to one of the registered individuals. The task reduces to ranking. Open-set identification is more complex: the system must reject strangers — people not in the database. This requires tuning a rejection threshold. As the database grows, the threshold may need adjustment: from 1,000 to 100k people, the probability of random match increases.

Real-time Database Updates

Adding new users without reindexing: index.add_with_ids() works incrementally. Deletion: IndexIDMap.remove_ids(). Persistence via faiss.write_index().

Production System Metrics

• CMC (Cumulative Match Characteristic): Rank-1, Rank-5, Rank-10 accuracy
• DIR@FAR (Detection and Identification Rate): for open-set
• p95/p99 latency under peak load
• QPS (queries per second) — for capacity planning

What's Included in Our Work?

• Architectural design: selection of embedding model, index, and hardware.
• API implementation for registration and identification.
• Integration with access control systems (ACS).
• Documentation and admin guide.
• Staff training and launch support.
• 12-month code warranty.

Step-by-Step Implementation Process

1. Requirements analysis and load testing.
2. Stack selection and design.
3. Development and testing.
4. Integration and pilot launch.
5. Production load optimization.

Common Mistakes in ID System Design

• Using brute-force with large databases — latency grows linearly.
• Ignoring open-set: threshold not set, leading to false positives.
• Underestimating image quality — model requires at least 100x100 pixel resolution.
• Lack of index replication — single point of failure.

For production systems, we recommend using FAISS GPU for acceleration. The IVFPQ compression ratio of 8x allows storing 10 million faces in 2 GB of memory. According to the FAISS official documentation (GitHub), IVFPQ provides 8x compression with minimal accuracy loss.

We guarantee system quality and accuracy. Our engineers hold PyTorch and FAISS certifications. Experience implementing systems in companies with databases of up to 10 million faces.

Get a consultation for your project — we'll evaluate the architecture and timeline within 2 days. Contact us to discuss the details.