AI Auto-Generation of Test Data
Test data — realistic-looking but synthetic — powers quality testing. Manually crafting test datasets is slow and often misses edge cases. An AI generator creates boundary cases, valid data, invalid data, and special cases based on schema and domain knowledge.
Semantic Data Generation Based on Domain
Key idea: don't just fill random values. Generate data that makes semantic sense for the domain.
from langchain_openai import ChatOpenAI
import json
class TestDataGenerator:
SEMANTIC_PROMPT = """Generate {count} realistic test data records for the schema.
Schema:
{schema}
Domain: {domain}
Requirements:
1. Data must be semantically valid (e.g., order_date < delivery_date)
2. Use realistic values for domain (e.g., valid email formats, phone numbers)
3. Vary the data: different statuses, amounts, dates
4. Include both common and edge cases
5. Return as JSON array with {count} objects"""
def __init__(self):
self.llm = ChatOpenAI(model="gpt-4o", temperature=0.5)
def generate_for_table(self, schema: dict, domain: str, count: int = 100) -> list[dict]:
result = self.llm.invoke(
self.SEMANTIC_PROMPT.format(
count=min(count, 50),
schema=json.dumps(schema, indent=2),
domain=domain
)
)
return json.loads(result.content)
def generate_boundary_cases(self, schema: dict) -> list[dict]:
"""Generates boundary and invalid test data"""
return self.llm.invoke(f"""Create boundary and invalid test data for schema.
Schema: {json.dumps(schema, indent=2)}
Create 2–3 cases of each:
1. Empty values (null, "", [])
2. Maximum values
3. Minimum/negative values
4. Special characters: <, >, ', ", newlines, emoji
5. SQL injection strings (for sanitization testing)
6. Very long strings (>1000 chars)
7. Wrong types (string instead of number, etc.)
Return JSON array with expected_behavior for each case.""").content
Case study: e-commerce platform, 50+ data-driven test scenarios. Manual data creation: 8 hours per scenario. With AI generator: 15 minutes. Coverage improved: boundary cases jumped from 20% to 95%.
Timeframe: basic generator: 1–2 weeks; domain-aware with validation: 3–4 weeks.