Parsing corporate news often reveals relationships like 'Sberbank appointed German Gref as chairman.' Named Entity Recognition (NER) finds entities, but relation extraction (RE) identifies the 'appointment' connection, converting unstructured text into triples (subject, relation, object). In a project for a legal department, we processed 100,000 court rulings: manual labeling would have taken six months and cost $50,000, but fine-tuning BERT with Distant Supervision achieved F1 78% in 3 weeks for $15,000. Our engineers have delivered over 15 RE projects for financial and legal corpora, including building knowledge graphs. With more than 5 years of NLP experience, we guarantee F1 no lower than 75% on standard benchmarks such as TACRED, DocRED, and SemEval. RE systems already save companies up to 70% of time on contract and judgment analysis. The RE approach is detailed on Wikipedia.
How Relation Extraction Extracts Semantic Links
NER only identifies entities, but does not determine the type of connection between them. Relationship extraction (RE) extends NER: the output is semantic triples like (Sberbank, appoints, German Gref). Such triples are the foundation for building knowledge graphs and automatic text analysis. We use open benchmarks TACRED and DocRED for validation — on these, fine-tuned BERT consistently yields F1 75–80%.
How Fine-Tuning BERT Outperforms Prompt-Based LLMs
Prompt-based approaches with LLMs (e.g., GPT-4) can get RE running in a day, but on large data volumes inference cost is high — latency p99 can reach 1 second per triple. Fine-tuned BERT with entity-marker tokens ([E1]entity[/E1]) achieves F1 10–15% higher with latency 10–50 ms. Comparison of approaches:
| Approach | F1 (TACRED) | Latency p99 | Inference Cost | Schema Flexibility |
|---|---|---|---|---|
| Prompt-based LLM (GPT-4) | 60–70% | 500–1000 ms | High | High |
| Fine-tuned BERT (RoBERTa-large) | 75–80% | 10–50 ms | Low | Low |
| REBEL (T5-based) | 65–72% | 100–200 ms | Medium | Medium |
Fine-tuned BERT is 10 times cheaper at inference and 10–15% more accurate — the optimal choice for high-load systems. Metrics are strict: an answer is considered correct only if entities, direction, and relation type match exactly.
How Distant Supervision Reduces Labeling Costs
Labeling data for RE is an expensive step. Distant Supervision automatically creates a training set by aligning texts with a knowledge base (e.g., Wikidata). This yields 10 times more examples for the same cost as manual labeling, with some loss in accuracy (5–8% F1). To compensate for noise, we use weighted loss and confidence filtering.
Comparison of labeling methods:
| Method | Volume per month | Cost | Accuracy (F1) |
|---|---|---|---|
| Manual labeling | 5–10k examples | High | 100% (gold) |
| Distant Supervision | 50–100k examples | Low | 92–95% (with filtering) |
RE Implementation Process
- Corpus audit and relation schema definition — determine the list of relations and verify NER quality.
- Approach selection — based on data volume and latency requirements, choose prompt-based, fine-tuning, or REBEL.
- Labeling — if labeled data is scarce, apply Distant Supervision.
- Training and validation — tune hyperparameters, monitor F1 on held-out set.
- Deployment — package model into Docker, API on FastAPI with metrics and logging.
- Support — train your team, provide 3 months of maintenance.
What's Included
- Trained RE model with target metrics
- API service on FastAPI in a Docker container
- Documentation: architecture description, fine-tuning guide, API specification
- Source code and configs under version control
- Training for your team (2–3 sessions)
- Technical support for 3 months after deployment
Additional details
- Trained RE model with target metrics
- API service on FastAPI in a Docker container
- Documentation: architecture description, fine-tuning guide, API specification
- Source code and configs under version control
- Training for your team (2–3 sessions)
- Technical support for 3 months after deployment
Common Mistakes and How to Avoid Them
- Missing long-tail relations: rare types introduce noise in distant supervision. Solution — balance the dataset and use weighted loss.
- Entity errors: if NER is inaccurate, RE inherits the errors. We recommend a cascade with intermediate validation.
- Ignoring context: the same word may indicate different relations in different domains. Fine-tuning on the target corpus solves the problem.
To find out which approach fits your corpus, contact us for a free audit. We are ready to discuss details. Get a consultation on choosing the approach for your corpus — it's free.