Automatic Fact Extraction: LLM and NLP Pipelines
Every day, companies process thousands of documents: contracts, invoices, reports, news. Manual entity search — parties, amounts, dates — takes hours and yields 70–80% accuracy. A typical error: missing a date or misattributing a counterparty leads to fines and missed deadlines. Information extraction (IE) solves this: it extracts entities, relations, and events with up to 95% accuracy. Fact extraction from text is no longer a chore.
We automate the full cycle: from parsing to normalization. We use LLMs (GPT-4o, Claude, LLaMA 3) and classic NLP pipelines. We choose the approach based on the task — or combine them. A hybrid architecture achieves F1 up to 95% with latency under 200 ms. Average savings on manual processing reach 60% of costs — or 500 person-hours per month.
What Typical Errors Occur in Data Extraction?
When extracting from contracts, company names are often confused due to typos or abbreviations — "LLC Romashka" and "Romashka LLC" are treated as different. Normalization via edit distance (Levenshtein) and synonym dictionaries solves this. Entity duplicates — the same information appears in different documents with different attributes. We use entity linking through a knowledge graph. Finally, context ambiguity: the word "account" can be a bank account or an invoice. Disambiguation via LLM with few-shot examples.
Why LLM Outperforms Classic Pipelines?
LLMs with structured output (Pydantic, OpenAI function calling) handle extraction without fine-tuning. Example:
from pydantic import BaseModel
from openai import OpenAI
class CompanyInfo(BaseModel):
name: str
revenue: float | None
revenue_year: int | None
ceo: str | None
headquarters: str | None
employees_count: int | None
client = OpenAI()
response = client.beta.chat.completions.parse(
model="gpt-4o-mini",
messages=[{
"role": "user",
"content": f"Extract company information from the text:\n{text}"
}],
response_format=CompanyInfo,
)
result = response.choices[0].message.parsed
This code works for any text — from financial reports to news. No annotation, no pipeline. But for >1000 doc/hour and latency <100 ms, classic pipelines are cheaper and faster. On a marketplace project, we extracted 500k+ product cards from 1 million PDFs. We used a combination of OCR (Tesseract) + LayoutLM for tables + GPT-4o for attribute extraction. F1 on the test set: 93%.
Comparison of Approaches
| Characteristic | LLM-based | Classic pipeline |
|---|---|---|
| Accuracy (F1) | 75–95% | 85–95% |
| Latency | 1–10 s per request | <100 ms |
| Flexibility | High | Low |
| Cost per 10k docs | Medium | Low |
| Data requirement | None | Yes (annotated data) |
| Integration | REST API | Can be embedded |
How Fact Extraction from Text Solves Document Processing
Fact extraction from text is a key step in building knowledge bases. In a typical project, we go through five stages:
| Stage | Duration | Result |
|---|---|---|
| Analysis | 2–5 days | Identification of target entities and relations |
| Design | 2–10 days | Architecture selection (LLM, BERT, spaCy) |
| Implementation | 1–4 weeks | Code writing, pipeline tuning |
| Testing | 1 week | Metric calculation, A/B tests |
| Deployment | 2–5 days | Containerization, REST API, documentation |
Timeline: from 2 weeks to 2 months depending on complexity. Cost is calculated individually.
What's Included in the Work?
- Analytical report with extracted entities and relations
- Trained model or pipeline (LLM or classic)
- REST API with documentation (OpenAPI)
- Integration with CRM (Bitrix24, 1C, etc.)
- Employee training (up to 5 days)
- Technical support for 3 months
Step-by-Step Implementation Process
- Data audit — collect document samples, define target entities and quality metrics
- Architecture selection — test LLM, BERT, spaCy on your data, establish baseline
- Pipeline development — write code, configure inference, optimize latency
- Testing — calculate Precision/Recall/F1, A/B tests, adjustments
- Deployment — containerization, deploy on your infrastructure or cloud
- Support — monitor metrics, retrain when data changes
Extraction Quality Assessment
Precision/Recall/F1 metrics are calculated for each entity type. For relations — relation-level F1, for slots — slot filling accuracy. Typical results: 90–95% F1 on reports, 75–85% on news.
Detailed metrics: F1 is the harmonic mean of Precision and Recall. For each entity type we compute separately. If recall is critical (e.g., finding all mentions), we tune the pipeline for higher recall (up to 98%) at the expense of precision.
We guarantee: fixed quality metrics in the contract, certified engineers with 10+ projects of experience (NLP, Computer Vision, LLM), post-deployment support. If you want to automate data extraction from your documents, contact our engineers for a consultation. Order a pilot project without prepayment — we'll assess your case in 2 days.