AI-Powered Car Damage Assessment from Photos

We developed an AI-powered visual car damage assessment system that solves real-time insurance scoring. Our engineers have over 10 years of experience in computer vision and have delivered more than 50 projects. The company has been in the AI solutions market for over 5 years. Imagine: a customer photographs a car in the parking lot, uploads 6 shots into an app, and within 40 seconds receives a preliminary damage estimate—no expert visit, no inspection queue. This architecture, based on YOLOv8-seg + regression cost model, can be deployed to production in 8–12 weeks. It guarantees a 30–50% reduction in expert costs. For a typical insurance portfolio, the budget savings can reach 2–3 million rubles per year; for a portfolio of 10,000 policies, savings amount to about 15 million rubles annually.

What the Model Actually Detects

The task breaks down into three levels:

• Localization of damage — where and what: dent, scratch, crack, broken glass, body deformation. A segmentation model outputs a polygonal mask for each defect with pixel area.
• Part classification — which car zone the damage belongs to: front bumper, hood, left front wing, door, etc. This is critical for mapping to the hourly rates of specific parts.
• Severity estimation — light / medium / severe damage. A regression head on backbone features provides a score that is multiplied by coefficients from the cost reference table.

Pipeline Architecture

A two‑stage scheme works more reliably than end‑to‑end:

1. Damage detection and segmentation → YOLOv8-seg or SAM 2 with automatic prompts per car zone
2. Part classification → ResNet-50 or EfficientNet-B3 trained on body breakdown (30–50 part classes)
3. Cost estimation → XGBoost or LightGBM on features: mask area, part class, severity score, make/model (from EXIF or a separate classifier)

# Example inference of the segmentation model
from ultralytics import YOLO
import cv2

model = YOLO('damage_seg_v8x.pt')  # fine-tuned on damage dataset
results = model.predict(
    source='car_damage.jpg',
    conf=0.35,
    iou=0.45,
    imgsz=1280,   # high resolution critical for small scratches
    retina_masks=True
)

for r in results:
    masks = r.masks.data       # (N, H, W) tensor
    classes = r.boxes.cls      # damage classes
    areas = masks.sum(dim=[1,2])  # area of each mask in pixels

How We Ensure Detection Quality

Why are small scratches the most common problem?

At standard imgsz=640, a 10 cm scratch at 1.5 m distance occupies 3–4 pixels. YOLOv8 misses them with IoU < 0.3. Raising imgsz to 1280 increases recall for small defects from 0.44 to 0.71, but latency grows from 28 ms to 95 ms on RTX 3090. We balance this by automatically selecting resolution based on camera metadata.

How does the model distinguish new damage from old?

Fresh chips have white edges, old ones have brown rust edges. We add color analysis of mask edges and cross-check with the history of previous inspections. This reduces legal risks for the insurance company.

Dataset and Training

Minimum dataset size for a working model: 5,000 annotated images with masks covering all damage and part classes. In practice we use:

• Open datasets: CarDD (4,000 images), COCO with fine-tuning
• Synthetic from Blender — for rare cases (total front destruction)
• Client data from insurance case archives

Fine‑tuning YOLOv8x-seg on RTX 4090 with 8,000 images takes about 18 hours (100 epochs, batch=16, imgsz=1280). Final metrics: [email protected] = 0.79, [email protected]:0.95 = 0.61.

Performance Comparison on Different Devices

Detection accuracy varies by damage type. For example, recall on dents reaches 0.85, while on small scratches it is 0.71.

What’s Included in the Work

• Data audit — checking dataset representativeness, recommendations on collection and labeling
• Model development — fine-tuning, quantization (INT8/INT4), export to ONNX/TensorRT
• Integration API — FastAPI endpoints for photo upload, mask retrieval, and cost calculation
• Mobile SDK — CoreML (iOS), TFLite (Android) with integration example
• Documentation — OpenAPI spec, model card, labeling instructions for new data
• Support — 3 months warranty after launch

How We Build the Damage Assessment Model: Step‑by‑Step Process

1. Collection and labeling — 5,000+ images with polygonal masks from insurer experts
2. Training — transfer learning from YOLOv8x-seg, 100 epochs, 8–12 hours on RTX 4090
3. Validation — testing on hold‑out set ([email protected] > 0.75) and on real cases from the client
4. Optimization — TensorRT FP16 for server, CoreML/TFLite for mobile
5. Deployment — Docker container on Kubernetes, monitoring p99 latency and drift

Common Model Errors and Their Solutions

• Lighting and glare — simulate RandomSunFlare in augmentations, quality check rejects overexposed photos
• Small scratches — increase imgsz to 1280 and use retina_masks, raising recall from 0.44 to 0.71
• Old vs new damage — color analysis of mask edges + cross-check with inspection history

Project Timelines

8 to 16 weeks depending on availability of labeled data and integration requirements with the insurer’s systems. Cost is calculated individually.

Contact us for a consultation on your project. Order a pilot project to evaluate accuracy on your own data. Get demo access to the system with your photos.