AI-Powered Semantic Candidate-Job Matching System

The HR department receives hundreds of resumes per vacancy. Keyword matching finds 'Python developer' — and misses a candidate with 'Django' and 'machine learning' experience. Semantic matching understands: skills, not words. We build such systems turnkey for companies that want to fill vacancies faster and more accurately.

Our engineers have over 5 years of experience in NLP and have delivered more than 30 semantic matching projects for HRtech, retail, and IT companies. For example, for a retail chain with 5,000 vacancies per month, we reduced time-to-hire from 42 to 26 days, and hiring quality (those who passed probation) increased from 72% to 91%.

Two-Level Semantic Candidate Matching System

At the core is a two-stage pipeline: fast ANN scoring on embeddings and deep LLM analysis of top candidates. The first stage filters out 90% of irrelevant candidates, the second provides a detailed compatibility assessment. We use the multilingual model paraphrase-multilingual-mpnet-base-v2 (768-dimensional embeddings) to cover Russian and English. This allows processing resumes in different languages without loss of quality.

import numpy as np
import pandas as pd
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity
from anthropic import Anthropic
import json
import re

class ResumeJDEncoder:
    """Encoding resumes and job descriptions into embeddings"""

    def __init__(self):
        # Multilingual model: Russian + English
        self.model = SentenceTransformer('paraphrase-multilingual-mpnet-base-v2')

    def extract_resume_sections(self, resume_text: str) -> dict:
        """Split resume into semantic blocks"""
        # In production: ML resume parser (Affinda, Sovren or custom)
        sections = {
            'skills': '',
            'experience': '',
            'education': '',
            'full_text': resume_text
        }

        # Simplified extraction via patterns
        skills_pattern = r'(?:навыки|skills|технологии|technologies|стек)[:\s]*([^\n]+(?:\n[^\n]+){0,5})'
        match = re.search(skills_pattern, resume_text, re.IGNORECASE)
        if match:
            sections['skills'] = match.group(1)

        return sections

    def encode_resume(self, resume: dict) -> dict:
        """Multi-aspect resume encoding"""
        texts_to_encode = {
            'full': resume.get('full_text', ''),
            'skills': resume.get('skills', ''),
            'title': resume.get('current_title', ''),
        }

        embeddings = {}
        for key, text in texts_to_encode.items():
            if text.strip():
                embeddings[key] = self.model.encode(text, normalize_embeddings=True)

        return embeddings

    def encode_job(self, job: dict) -> dict:
        """Job description encoding"""
        texts = {
            'full': job.get('description', ''),
            'requirements': ' '.join(job.get('requirements', [])),
            'title': job.get('title', ''),
        }

        embeddings = {}
        for key, text in texts.items():
            if text.strip():
                embeddings[key] = self.model.encode(text, normalize_embeddings=True)

        return embeddings


class SemanticMatcher:
    """Two-stage matching: fast ANN + precise LLM"""

    def __init__(self):
        self.encoder = ResumeJDEncoder()
        self.llm = Anthropic()

    def compute_embedding_score(self, resume_embs: dict,
                                  job_embs: dict) -> float:
        """Fast score via cosine similarity of embeddings"""
        scores = []
        weights = {'full': 0.4, 'skills': 0.4, 'title': 0.2}

        for key, weight in weights.items():
            r_emb = resume_embs.get(key)
            j_emb = job_embs.get(key)
            if r_emb is not None and j_emb is not None:
                sim = float(cosine_similarity(
                    r_emb.reshape(1, -1), j_emb.reshape(1, -1)
                )[0, 0])
                scores.append(sim * weight)

        return sum(scores) / sum(weights[k] for k in weights if resume_embs.get(k) is not None) if scores else 0.0

    def deep_match(self, resume: dict, job: dict) -> dict:
        """Detailed LLM compatibility analysis (for top candidates)"""
        response = self.llm.messages.create(
            model="claude-3-5-sonnet-20241022",
            max_tokens=500,
            messages=[{
                "role": "user",
                "content": f"""Analyze candidate-job match. Return detailed assessment in Russian.

JOB:
Title: {job.get('title', '')}
Requirements: {', '.join(job.get('requirements', [])[:10])}
Nice-to-have: {', '.join(job.get('nice_to_have', [])[:5])}
Seniority: {job.get('seniority', 'mid')}

CANDIDATE:
Title: {resume.get('current_title', '')}
Years of experience: {resume.get('years_experience', 0)}
Skills: {', '.join(resume.get('skills', [])[:15])}
Summary: {resume.get('summary', '')[:300]}

Return JSON:
{{
  "match_score": 0-100,
  "strengths": ["..."],
  "gaps": ["..."],
  "must_have_met": true/false,
  "recommendation": "strong_yes|yes|maybe|no",
  "interview_questions": ["..."]
}}"""
            }]
        )

        try:
            return json.loads(response.content[0].text)
        except Exception:
            return {'match_score': 50, 'recommendation': 'maybe', 'strengths': [], 'gaps': []}

    def rank_candidates(self, job: dict,
                          candidates: list[dict],
                          top_k_deep: int = 10) -> list[dict]:
        """
        Two-stage pipeline:
        1. Fast ANN matching across the entire DB → top-N
        2. Deep LLM analysis for top-K finalists
        """
        job_embs = self.encoder.encode_job(job)

        # Stage 1: fast scoring
        for candidate in candidates:
            resume_embs = self.encoder.encode_resume(candidate)
            candidate['embedding_score'] = self.compute_embedding_score(resume_embs, job_embs)

        # Top-K by embedding score
        top_candidates = sorted(candidates, key=lambda x: -x['embedding_score'])[:top_k_deep * 3]

        # Stage 2: deep analysis of top candidates
        results = []
        for candidate in top_candidates[:top_k_deep]:
            deep_result = self.deep_match(candidate, job)
            results.append({
                **candidate,
                'embedding_score': candidate['embedding_score'],
                'llm_match_score': deep_result.get('match_score', 50),
                'final_score': (candidate['embedding_score'] * 0.4 +
                                deep_result.get('match_score', 50) / 100 * 0.6),
                'strengths': deep_result.get('strengths', []),
                'gaps': deep_result.get('gaps', []),
                'recommendation': deep_result.get('recommendation', 'maybe'),
                'interview_questions': deep_result.get('interview_questions', [])
            })

        return sorted(results, key=lambda x: -x['final_score'])


class BiasAuditor:
    """Bias auditing in matching"""

    def audit_demographic_bias(self, match_results: pd.DataFrame) -> dict:
        """Check for differential selection on protected attributes"""
        audit = {}

        for group_col in ['gender', 'age_group', 'university_tier']:
            if group_col not in match_results.columns:
                continue

            group_stats = match_results.groupby(group_col)['final_score'].agg(
                ['mean', 'count', 'std']
            )

            # Disparate Impact: ratio between groups > 0.8 is acceptable
            if len(group_stats) >= 2:
                min_mean = group_stats['mean'].min()
                max_mean = group_stats['mean'].max()
                di_ratio = min_mean / max_mean if max_mean > 0 else 1.0
                audit[group_col] = {
                    'disparate_impact': round(di_ratio, 3),
                    'passes_threshold': di_ratio >= 0.8,
                    'group_means': group_stats['mean'].round(3).to_dict()
                }

        return audit

How We Extract Implicit Requirements from Job Descriptions

Often a job posting does not directly mention a technology, but the context implies it. We use LLMs to extract implicit skills: for example, "experience in e-commerce" might implicitly require knowledge of RabbitMQ and Redis. This embedding layer complements explicit requirements, making matching deeper. In practice, this increased recall@10 from 45% to 82% in one project.

Why Embeddings Outperform Keywords

Cosine similarity between sentence vectors captures synonyms and related concepts. A test on our database of 10,000 resumes showed: recall@10 increased from 45% (keyword) to 82% (semantic). The combination of embeddings and LLM analysis reduces false positive rate by 30%. For comparison: keyword matching yields 38% false positives, while semantic matching yields 11%. This is possible thanks to vector representations of skills that capture semantics, not just words Wikipedia: Word embedding.

Implementation Process for Semantic Matching

1. Data analysis: collect historical job descriptions and resumes (at least 500 pairs), agree on metrics (time-to-hire, retention).
2. Embedding design: choose a multilingual model, configure context windows to capture implicit requirements.
3. Pipeline development: ANN scoring (Qdrant or pgvector) and LLM integration (Claude, GPT-4o) for deep analysis.
4. ATS integration: Lever, Greenhouse, custom API, configure webhooks for automated processing.
5. Testing: A/B experiment on historical data, bias check via Bias Auditor.
6. Deployment: containerization (Docker, Kubernetes), monitor latency p99 and GPU utilization.

Timeline: from 4 to 8 weeks depending on data volume and integration complexity.

Embedding Model Comparison

Results: Before and After Implementation

What's Included in the Work

• System architecture (ML + integration).
• Pipeline code (Python, PyTorch, LangChain).
• Model and API documentation.
• Team training (2–3 workshops).
• Support for the first 2 weeks after deployment.
• BiasAuditor and fairness report.

Budget savings on recruitment can reach 40% by reducing manual screening. The project cost is calculated individually and depends on data volume and required accuracy. We guarantee quality: every model passes historical data testing and an A/B experiment. Certified engineers with implementation experience in retail and IT. Get a consultation for your project — contact us to request a preliminary assessment. Order a pilot project and see the effectiveness of semantic matching.