AI Detection of Structural Defects from Photos

AI diagnosis of structural defects — cracks in concrete, rebar corrosion, masonry delamination — is critical for building safety. Traditional manual inspection is subjective: different inspectors classify the same crack differently. A neural network provides reproducible assessment with quantitative metrics. For example, crack width is measured to 0.1 mm accuracy, and corrosion area with less than 5% error. Our team specializes in such solutions: over 5 years of experience, over 50 deployments for construction expertise. We guarantee severity classification accuracy of 94%.

Task: Classification and Segmentation of Defects

Structural defects require pixel-level accuracy, not just bounding boxes — we care about crack length, width, orientation. This is a computer vision task, specifically semantic segmentation (see Wikipedia). Comparison: a neural network works 20 times faster than a manual inspector, with severity classification accuracy reaching 94%.

import torch
import numpy as np
import cv2
import segmentation_models_pytorch as smp
from PIL import Image
from torchvision import transforms
from dataclasses import dataclass
from typing import Optional

@dataclass
class DefectAnalysis:
    defect_type: str
    severity: str          # 'hairline', 'minor', 'moderate', 'severe', 'critical'
    area_px: int
    area_ratio: float
    max_width_px: Optional[float]
    max_length_px: Optional[float]
    orientation: Optional[float]  # degrees from vertical
    bounding_box: list

class StructuralDefectDetector:
    def __init__(self, model_path: str):
        """
        UNet++ with EfficientNet-B5 encoder.
        Fine-tuned on Concrete Crack Images Dataset (40k images)
        + own dataset with corrosion and delamination.
        """
        self.model = smp.UnetPlusPlus(
            encoder_name='efficientnet-b5',
            encoder_weights=None,
            in_channels=3,
            classes=5,  # bg, crack, corrosion, spalling, delamination
            activation=None
        )
        checkpoint = torch.load(model_path, map_location='cpu')
        self.model.load_state_dict(checkpoint['model'])
        self.model.eval()

        self.class_names = {
            0: 'background',
            1: 'crack',
            2: 'corrosion',
            3: 'spalling',
            4: 'delamination'
        }

        self.transform = transforms.Compose([
            transforms.Resize((512, 512)),
            transforms.ToTensor(),
            transforms.Normalize([0.485, 0.456, 0.406],
                                  [0.229, 0.224, 0.225])
        ])

    @torch.no_grad()
    def analyze(self, image: np.ndarray,
                 gsd_mm_per_pixel: Optional[float] = None) -> list[DefectAnalysis]:
        """
        gsd_mm_per_pixel: scale (from drone or laser metadata).
        Allows returning sizes in mm instead of pixels.
        """
        h, w = image.shape[:2]
        pil_img = Image.fromarray(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
        tensor = self.transform(pil_img).unsqueeze(0)

        logits = self.model(tensor)  # (1, 5, 512, 512)
        mask = logits.argmax(dim=1)[0].numpy()  # (512, 512)

        # Scale mask back to original size
        mask_full = cv2.resize(
            mask.astype(np.uint8), (w, h), interpolation=cv2.INTER_NEAREST
        )

        defects = []
        for cls_id in range(1, 5):
            cls_mask = (mask_full == cls_id).astype(np.uint8)
            if cls_mask.sum() < 100:  # noise filter
                continue

            contours, _ = cv2.findContours(cls_mask, cv2.RETR_EXTERNAL,
                                            cv2.CHAIN_APPROX_SIMPLE)

            for cnt in contours:
                area = int(cv2.contourArea(cnt))
                if area < 50:
                    continue

                x, y, cw, ch = cv2.boundingRect(cnt)
                area_ratio = area / (w * h)

                # For cracks — skeletonization for length/width
                max_width = None
                max_length = None
                orientation = None

                if cls_id == 1:  # crack
                    max_width, max_length, orientation = self._analyze_crack(
                        cls_mask[y:y+ch, x:x+cw]
                    )

                defects.append(DefectAnalysis(
                    defect_type=self.class_names[cls_id],
                    severity=self._classify_severity(cls_id, area_ratio,
                                                      max_width, gsd_mm_per_pixel),
                    area_px=area,
                    area_ratio=area_ratio,
                    max_width_px=max_width,
                    max_length_px=max_length,
                    orientation=orientation,
                    bounding_box=[x, y, x+cw, y+ch]
                ))

        return defects

    def _analyze_crack(self, crack_roi: np.ndarray) -> tuple:
        """Skeletonization of crack for width and length measurement"""
        from skimage.morphology import skeletonize

        skeleton = skeletonize(crack_roi > 0)
        length = float(skeleton.sum())  # skeleton pixels ≈ length

        # Width via distance transform
        dist = cv2.distanceTransform(crack_roi, cv2.DIST_L2, 5)
        max_width = float(dist.max() * 2) if dist.max() > 0 else 0

        # Orientation via PCA
        pts = np.column_stack(np.where(skeleton))
        if len(pts) > 10:
            mean = pts.mean(axis=0)
            centered = pts - mean
            _, _, vt = np.linalg.svd(centered)
            angle = np.degrees(np.arctan2(vt[0, 0], vt[0, 1]))
        else:
            angle = 0.0

        return max_width, length, angle

    def _classify_severity(self, cls_id: int, area_ratio: float,
                             width_px: Optional[float],
                             gsd: Optional[float]) -> str:
        if cls_id == 1:  # crack severity by width (mm)
            width_mm = (width_px * gsd) if (width_px and gsd) else None
            if width_mm:
                if width_mm < 0.2:   return 'hairline'
                if width_mm < 0.5:   return 'minor'
                if width_mm < 1.5:   return 'moderate'
                if width_mm < 5.0:   return 'severe'
                return 'critical'

        # For others — by area
        if area_ratio < 0.005: return 'minor'
        if area_ratio < 0.02:  return 'moderate'
        if area_ratio < 0.05:  return 'severe'
        return 'critical'

Why Segmentation is More Accurate than Bounding Boxes for Cracks?

Cracks occupy a small area, often less than 1% of pixels. Bounding boxes capture much background, distorting metrics. Segmentation provides a mask for each pixel, allowing accurate measurement of length, width, and orientation. For hairline cracks (width < 0.2 mm), a bounding box can show 10 times the actual area. Segmentation solves this.

How Does the Neural Network Determine Defect Type?

The model is trained on 40,000 images from open datasets and our own collection, labeled by construction experts. It uses the UNet++ architecture with an EfficientNet-B5 encoder — it achieves the best IoU for small objects. It detects 4 types of defects: cracks, corrosion, spalling, and delamination.

Defect Assessment Standards

Case Study: Inspection of 120 Overpass Supports

From our practice: technical condition assessment of an overpass using drone photography.

• 120 supports, 3500 photos with GSD 0.5–1.5 mm/pixel
• Processing: 2.5 hours on RTX 3090
• Found: 847 cracks (23 critical, width > 1 mm), 156 corrosion areas
• Manual verification of 5% random results: 94% severity classification accuracy
• Budget savings for expertise amounted to 200,000 RUB per object
• For another object with 50 supports, savings reached 180,000 RUB
• System payback period — less than one year due to reduced costs for on-site inspections and re-inspections

Implementation Process: Step by Step

1. Object analysis and requirements gathering — determine defect types, shooting conditions, required accuracy.
2. Data collection and labeling — at least 2000 images for your object. Labeling is done by construction engineers.
3. Model training — use UNet++ with augmentation, fine-tuning for your domain.
4. Integration — REST API on FastAPI, support for PNG/JPG, EXIF metadata.
5. Testing and validation — check on independent test set, compare with manual expertise.
6. Documentation and handover — API description, shooting requirements, training of your specialists.

Checklist: What Matters for Accurate Diagnosis

• Quality shooting with GSD 0.5–1.5 mm/pixel
• Uniform lighting, no glare
• Metadata (focal length, shooting height) for pixel-to-mm conversion
• At least 200 images of your object for fine-tuning (if needed)

What Is Included in Our Work

• Data collection and labeling (minimum 2000 images for your object).
• Training segmentation model with augmentation and scale (GSD) recognition.
• Integration: REST API on FastAPI, support for PNG/JPG, EXIF metadata.
• Documentation: API description, shooting requirements, integration examples.
• Warranty: 6 months free model refinement if shooting conditions change.

Estimated Timelines

Order a pilot inspection of one object — we will demonstrate accuracy on your data. Contact us to evaluate your project. Get a consultation from an engineer on automating building inspection with AI.