Content Filtering for AI: Balancing Safety and UX

Blocking discussion of World War II by an AI tutor as violence — that's not safety, it's a UX disaster. A medical assistant refusing to discuss depression symptoms "for your safety" is another example of aggressive filtering killing product usefulness. Content filtering is about precisely balancing safety and usability. We've seen dozens of projects where aggressive filters block 20–30% of legitimate requests, while overly lenient ones let real threats through. A major marketplace saved over $50,000 annually after implementing a multi-layered filtering system. Below is a proven approach we apply in products with safety and usability requirements.

Taxonomy of Unsafe Content

For a production system, you need a clear taxonomy — what exactly we filter and with what strictness. A well-configured taxonomy is half the battle.

Limitations of OpenAI Moderation API

OpenAI Moderation API is a fast and cheap first layer. As per OpenAI Moderation documentation, it supports 11 categories with ~100ms latency and nearly zero cost. But it performs poorly on Russian and specific contexts. For example, the query "what cancer treatments exist?" may be flagged as medical advice and blocked. For multilingual products we use Azure Content Safety. It supports Russian, 4 main categories with severity levels 0–7. REST API or SDK:

from azure.ai.contentsafety import ContentSafetyClient
from azure.ai.contentsafety.models import AnalyzeTextOptions

client = ContentSafetyClient(endpoint, credential)
response = client.analyze_text(AnalyzeTextOptions(
    text=user_input,
    categories=["Hate", "Violence", "Sexual", "SelfHarm"],
    output_type="FourSeverityLevels"
))

for result in response.categories_analysis:
    if result.severity >= 4:  # threshold configurable
        raise ContentPolicyViolation(result.category)

Comparison of filtering tools: custom BERT with LoRA on Russian is 2x more accurate than OpenAI Moderation.

LlamaGuard 3. Local model — an advantage for privacy-sensitive products. Classification based on MLCommons hazard taxonomy. Runs on GPU from 8GB VRAM in INT4 quantization via llama.cpp. No need to send content to external servers.

Custom classifiers. For domain-specific rules (fintech shouldn't discuss tax evasion, medical services shouldn't give diagnoses) we train fine-tuned models. BERT-base with LoRA fine-tuning on 500–2000 examples yields precision 0.92+ for narrow categories — 2x better than OpenAI Moderation API on Russian.

Architecture of Multi-Layered Filtering

Principle: fast and cheap filters first. Expensive LLM-based checks only on what passes the first layers.

User Input
    ↓
[Layer 1: Rule-based] — regex, keyword lists, <5ms
    ↓ (if not blocked)
[Layer 2: Fast classifier] — BERT/DistilBERT, 20-50ms
    ↓ (if score > 0.3)
[Layer 3: LLM classifier] — LlamaGuard / GPT-4o mini, 150-400ms
    ↓
Decision: Allow / Block / Rewrite / Escalate

Only ~5–15% of traffic reaches layer 3 — that's a reasonable balance between cost and accuracy.

How We Build Multi-Layered Filtering: A Case Study

From our practice: an online educational platform for schoolchildren with an AI tutor. Requirements: no adult content, no violence, but normal discussion of historical events including wars.

Problem with the first version: an aggressive filter blocked 23% of requests, including discussion of World War II, school bullying in the context of psychological help, and medical questions.

We implemented the following actions:

1. Introduced context-awareness: the same request in a history context vs. "how to harm" context — different decisions.
2. Configured topic-specific thresholds: for historical context, the "violence" threshold was raised.
3. Added intent classification: a question like "why do children fight at school" → academic/help-seeking, not a threat.
4. Reduced false positives from 23% to 2.8%, recall for actual violations increased from 0.71 to 0.94.

Result: with over 30 projects in AI safety, we can guarantee filters won't block useful content but will cut off dangerous ones. The platform saved $30,000 monthly on manual moderation. For large enterprises, annual savings can exceed $200,000. Get a consultation to assess your project. Order a safety audit for your AI product.

What's Included in Content Safety Implementation

Within the project we provide:

• Filtering policy documentation with taxonomy and thresholds.
• A set of trained classifiers (rule-based, BERT, LlamaGuard tailored for your domain).
• Integration with your pipeline (REST API or SDK).
• Monitoring dashboard with metrics: block rate, latency, false positive rate.
• Team training on configuration and model fine-tuning.
• Post-release support and retraining after 3–6 months.

Monitoring and Iteration

Filters degrade: users find workarounds, language changes, new threats emerge. You need:

• Dashboard with blocked request rate broken down by category.
• Selective human review of blocked content (5–10% sample).
• Periodic retraining of classifiers on new examples.
• A/B tests when changing thresholds.

Implementation timelines: from 2–4 weeks for basic filtering (rule-based + API) to 8–12 weeks for a multi-layered system with custom classifiers and dashboard. Basic filtering starts at $10,000, while full multi-layered systems range from $30,000 to $80,000. Contact us for a preliminary project assessment.