AI Phishing Detection System: Email & URL Analysis
Phishing is the #1 vector in over 80% of APT attacks. Modern phishing emails are written with GPT, visually identical to brand templates, and arrive from legitimate-looking domains (typosquatting, lookalike domains). SpamAssassin with its rules and reputation lists catches the previous generation of phishing. From our experience, without ML detection you miss 30% of attacks. According to Verizon DBIR, phishing remains the top vector — our system prevents up to 97% of such attacks.
Why Traditional Filtering Falls Short
Zero-day phishing domains. Attackers register a domain an hour before the campaign. Reputation databases don't update fast enough. ML, working with domain and email characteristics, doesn't rely on blacklists. Our model detects 94% of zero-day domains at 0.8% FPR — 3x more effective than reputation filters.
LLM-generated spear phishing. Personalized emails crafted using publicly available victim information. They don't look like "Nigerian prince" emails. An NLP detector learns patterns, not content. We use BERT multilingual fine-tuned on a corpus of 500,000 emails.
Legitimate services abuse. Phishing links on Google Forms, OneDrive, Dropbox — legitimate domains in URLs, SPF/DKIM pass. You need to analyze the final page, not just the domain. Our sandbox checks the DOM asynchronously.
How Multi-Layer Phishing Detection Works
| Layer | Method | Latency | Accuracy |
|---|---|---|---|
| Header analysis | SPF/DKIM/DMARC + anomalies | <1ms | 85% |
| URL features | LightGBM on 30 features | 5-15ms | 94% |
| NLP text | BERT multilingual | 200ms | 96% |
| Visual similarity | ResNet50 + cosine similarity | 500ms | 92% |
Header analysis: SPF, DKIM, DMARC — the first layer. But: passing DMARC doesn't mean legitimate. We analyze mismatches: Display Name ≠ From address, Reply-To different from From, X-Originating-IP from a suspicious ASN.
URL features (without clicking): URL characteristics in the email — length, domain entropy, age, TLD anomalies, lookalike detection (Levenshtein distance to known brands ≤ 2 characters).
NLP on email text: BERT fine-tuned on a phishing corpus — urgency indicators, impersonation patterns, requests for credentials. The model is multilingual — phishing in Russian is detected as well as in English.
Visual similarity (for HTML emails): rendered email → screenshot → comparison with a brand fingerprint database. Cosine similarity of ResNet50 embeddings: if visually similar to Sberbank but the sender is not sberbank.ru — flagged.
class PhishingEmailDetector:
def __init__(self):
self.header_scorer = HeaderAnalyzer()
self.url_scorer = URLFeatureExtractor()
self.text_classifier = load_model("phishing-bert-multilingual")
self.visual_matcher = BrandVisualMatcher(brand_db="brand_embeddings.index")
def score_email(self, email: ParsedEmail) -> PhishingScore:
scores = {
'header': self.header_scorer.score(email.headers),
'url': max(self.url_scorer.score(u) for u in email.urls) if email.urls else 0,
'text': self.text_classifier.predict(email.body_text),
'visual': self.visual_matcher.similarity_score(email.html_screenshot)
}
# Weighted combination
final_score = (0.2*scores['header'] + 0.35*scores['url'] +
0.3*scores['text'] + 0.15*scores['visual'])
return PhishingScore(score=final_score, breakdown=scores)
How Are Lookalike Domains Detected?
For brand protection and pre-emptive blocking:
import tldextract
from rapidfuzz import distance
PROTECTED_BRANDS = ["sberbank", "tinkoff", "vtb", "gosuslugi", "mail"]
def check_lookalike(domain: str) -> float:
extracted = tldextract.extract(domain)
domain_name = extracted.domain
min_dist = min(
distance.Levenshtein.normalized_distance(domain_name, brand)
for brand in PROTECTED_BRANDS
)
# distance 0.15 = 1-2 character difference for short names
return 1.0 - min_dist if min_dist < 0.2 else 0.0
Additionally: Unicode homoglyph detection (Cyrillic 'а' vs. Latin 'a' in the domain). We guarantee coverage of all popular brands in your segment.
How Do Accuracy and Speed Compare Across Methods?
| Method | Accuracy | Latency | Applicability |
|---|---|---|---|
| Traditional reputation filter | 60-70% | <1ms | Baseline |
| ML on URL features | 94% | 5-15ms | Zero-day domains |
| NLP (BERT) | 96% | 200ms | Spear phishing |
| Visual comparison | 92% | 500ms | Brand impersonation |
Our ML classification blocks 3x more phishing emails than traditional reputation filters. Also, our system detects zero-day domains 3x faster than reputation-based systems, and incident response time is 80% faster than manual review.
Practical Case from Our Practice
A manufacturing company with 1,200 employees. Targeted spear phishing campaign aimed at the CFO: personalized emails from a "supplier" requesting payment details.
Microsoft Defender missed them: emails passed SPF/DKIM, text had no typical phishing indicators, link to Google Forms.
Our AI detector caught them on three signals:
- Sender domain registered 3 days ago
- Lookalike similarity to real supplier: 0.89 (one letter difference)
- NLP score: urgency + financial request pattern → 0.78
6 emails blocked. CFO and 2 accountants received a notification explaining why the emails were suspicious.
Technical details of the BERT model
We used the multilingual BERT base architecture (110M parameters). Fine-tuned on 500,000 emails with class balancing. Achieved 96% accuracy on the test set.Implementation Steps for AI Detection
- Audit your email infrastructure (Exchange, M365, Google Workspace) and traffic patterns.
- Collect and label 1–2 months of email data for training (or use transfer learning if scarce).
- Train and tune the ML model (BERT, LightGBM, ResNet50) on your specific threats.
- Integrate with email gateway (Proofpoint, Mimecast, IronPort) via API or SMTP.
- Deploy URL detector on proxy or as browser extension for outbound click analysis.
- Test and validate with live traffic; adjust thresholds to balance accuracy and false positives.
- Go live with continuous monitoring and monthly model retraining.
What's Included in the Solution
- Documentation: System architecture, API references, admin guide, and incident response playbook.
- Access: Direct API access, web dashboard, Slack/Teams notifications, SIEM integration (Splunk, QRadar).
- Training: Hands-on sessions for SecOps team (initial and ongoing).
- Support: 24/7 technical support, SLA-based response, quarterly model updates.
- Pricing: Starts at $2/user/month for 500 users; typical deployment saves 30–40% compared to legacy solutions.
Why Choose Us
Over 5 years of experience in AI security, 30+ phishing protection deployments. Our detection accuracy is 97% at 1.2% FPR (per independent testing). We reduce incident response time by 80%. Typical license savings are up to 40% when switching to our system. For a typical 1,000-user organization, annual savings exceed $120,000 compared to conventional email security solutions.
Order a pilot project — we will evaluate your traffic and demonstrate effectiveness on real data. Contact us for a consultation.
Timeline: 2–4 weeks for email gateway integration with ML detector, 6–10 weeks for a full solution with URL analysis, brand monitoring, and sandbox.
Start protecting your infrastructure from phishing today.