AI Safety Monitoring System for Mines and Quarries

Mining is the second deadliest sector after construction. In underground mines, the risk of fatal incidents is 3 times higher than in open pits. A typical challenge: PPE monitoring in poor lighting (down to 10 lux), dust, hazardous zone control, personnel tracking, and emergency detection. Automating PPE and danger zone monitoring can reduce accidents by 30–50%. Classical CV systems fail: helmet detection accuracy in darkness drops to 50%. We use modern neural network detectors like YOLO, fine-tuned on specialized datasets, achieving stable 85–95% accuracy even in harsh conditions.

We develop end-to-end AI solutions: from dataset collection to deployment on video servers. Our experience: over 5 years in industrial CV, 30+ projects for mining companies. We guarantee a compliance rate of at least 85% at the acceptance stage.

Problems We Solve

Underground mines are an extreme environment for computer vision. Low light (down to 10 lux), coal dust suspensions, high humidity. Traditional detectors based on Haar cascades or color thresholding give 50–60% accuracy — unacceptable. Another challenge: false positives on mining equipment. Tracking errors when people cross paths with machinery. Without SCADA integration, it's impossible to automatically block dangerous zones. We implement OPC-UA / Modbus TCP with a latency of no more than 500 ms.

What AI Models Do We Use for Violation Detection?

Our base stack is YOLOv8/9 with ByteTrack tracking. For low-light underground conditions, we apply CLAHE preprocessing (1.5–2x contrast boost) and optionally LWIR thermal cameras. Our proprietary Mine Safety Dataset contains 15,000+ annotated frames with 8 classes: person, mining_helmet, headlamp, reflective_jacket, dust_mask, safety_boots, no_helmet, no_headlamp, mining_equipment.

import numpy as np
import cv2
from ultralytics import YOLO
from dataclasses import dataclass
import time
from typing import Optional

@dataclass
class MineWorkerStatus:
    worker_id: int
    bbox: list
    has_helmet: bool
    has_lamp: bool
    has_reflective_vest: bool
    has_mask: bool
    in_danger_zone: bool
    proximity_to_machinery: bool
    compliance: bool

class MineMonitoringSystem:
    """
    Mining safety monitoring system.
    Accounts for specifics: low light, smoke/dust, IR cameras.
    Mine Safety Dataset (custom):
    - mining_helmet, lamp_headlamp, reflective_jacket
    - mining_equipment, person, danger_zone_marker
    """
    MINE_PPE_CLASSES = {
        0: 'person',
        1: 'mining_helmet',     # helmet with lamp
        2: 'headlamp',          # lamp separately
        3: 'reflective_jacket', # reflective vest
        4: 'dust_mask',
        5: 'safety_boots',
        6: 'no_helmet',         # violation
        7: 'no_headlamp',       # violation
        8: 'mining_equipment',  # combine, loader
    }

    CRITICAL_ZONES = ['roof_instability', 'gas_presence', 'machinery_working']

    def __init__(self, model_path: str,
                  thermal_model_path: Optional[str] = None,
                  danger_zones: Optional[dict] = None,
                  device: str = 'cuda'):
        self.model = YOLO(model_path)
        self.thermal_model = YOLO(thermal_model_path) if thermal_model_path else None
        self.danger_zones = danger_zones or {}
        self.device = device
        self._worker_tracks: dict[int, dict] = {}

    def process_frame(self, frame: np.ndarray,
                       camera_id: str,
                       timestamp: float = None) -> dict:
        if timestamp is None:
            timestamp = time.time()

        # Enhancement for dark/dusty conditions
        enhanced = self._enhance_low_light(frame)

        results = self.model.track(
            enhanced, persist=True, conf=0.40,
            verbose=False, tracker='bytetrack.yaml'
        )

        workers_status = []
        violations = []

        if results[0].boxes is None:
            return self._empty_result(camera_id, timestamp)

        persons = {}
        ppe_items = {}

        for box in results[0].boxes:
            cls_id = int(box.cls.item())
            cls_name = self.MINE_PPE_CLASSES.get(cls_id, 'unknown')
            x1, y1, x2, y2 = map(int, box.xyxy[0])
            cx, cy = (x1+x2)//2, (y1+y2)//2
            tid = int(box.id.item()) if box.id is not None else -1

            if cls_name == 'person':
                persons[tid] = {
                    'bbox': [x1,y1,x2,y2], 'center': (cx,cy),
                    'has_helmet': False, 'has_lamp': False,
                    'has_vest': False, 'has_mask': False,
                    'violations': []
                }
            elif cls_name in ('no_helmet', 'no_headlamp'):
                ppe_items[tid] = {
                    'type': cls_name, 'center': (cx, cy),
                    'violation': True
                }
            elif cls_name in ('mining_helmet', 'headlamp',
                               'reflective_jacket', 'dust_mask'):
                ppe_items[tid] = {
                    'type': cls_name, 'center': (cx, cy),
                    'violation': False
                }

        # Associate PPE with workers
        for item_tid, item_data in ppe_items.items():
            nearest = self._find_nearest(item_data['center'], persons)
            if nearest is None:
                continue
            p = persons[nearest]
            if item_data['violation']:
                p['violations'].append(item_data['type'])
            else:
                t = item_data['type']
                if t == 'mining_helmet':
                    p['has_helmet'] = True
                elif t == 'headlamp':
                    p['has_lamp'] = True
                elif t == 'reflective_jacket':
                    p['has_vest'] = True
                elif t == 'dust_mask':
                    p['has_mask'] = True

        # Evaluate each worker
        for pid, pdata in persons.items():
            in_danger = self._check_danger_zone(pdata['center'])
            compliance = (pdata['has_helmet'] and
                          not pdata['violations'] and
                          not in_danger)

            ws = MineWorkerStatus(
                worker_id=pid, bbox=pdata['bbox'],
                has_helmet=pdata['has_helmet'],
                has_lamp=pdata['has_lamp'],
                has_reflective_vest=pdata['has_vest'],
                has_mask=pdata['has_mask'],
                in_danger_zone=in_danger,
                proximity_to_machinery=False,
                compliance=compliance
            )
            workers_status.append(ws)

            if not compliance or pdata['violations']:
                violations.append({
                    'worker_id': pid,
                    'bbox': pdata['bbox'],
                    'issues': pdata['violations'] + (['danger_zone'] if in_danger else []),
                    'severity': 'critical' if in_danger else 'warning'
                })

        total = len(workers_status)
        compliant = sum(1 for w in workers_status if w.compliance)

        return {
            'camera_id': camera_id,
            'timestamp': timestamp,
            'workers': [w.__dict__ for w in workers_status],
            'violations': violations,
            'compliance_rate_pct': round(compliant/max(total,1)*100, 1),
            'critical_alert': any(v['severity']=='critical' for v in violations)
        }

    def _enhance_low_light(self, frame: np.ndarray) -> np.ndarray:
        """Enhance visibility in dark mine conditions"""
        lab = cv2.cvtColor(frame, cv2.COLOR_BGR2LAB)
        clahe = cv2.createCLAHE(clipLimit=4.0, tileGridSize=(8,8))
        lab[:,:,0] = clahe.apply(lab[:,:,0])
        enhanced = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR)
        # Slight brightness boost
        enhanced = cv2.convertScaleAbs(enhanced, alpha=1.2, beta=20)
        return enhanced

    def _check_danger_zone(self, center: tuple) -> bool:
        for zone_id, polygon in self.danger_zones.items():
            poly = np.array(polygon, dtype=np.int32)
            if cv2.pointPolygonTest(poly,
                                     (float(center[0]), float(center[1])), False) >= 0:
                return True
        return False

    def _find_nearest(self, center: tuple, persons: dict) -> Optional[int]:
        min_dist = 150
        nearest = None
        cx, cy = center
        for pid, p in persons.items():
            px, py = p['center']
            dist = np.sqrt((cx-px)**2 + (cy-py)**2)
            if dist < min_dist:
                min_dist = dist
                nearest = pid
        return nearest

    def _empty_result(self, camera_id: str, timestamp: float) -> dict:
        return {
            'camera_id': camera_id, 'timestamp': timestamp,
            'workers': [], 'violations': [],
            'compliance_rate_pct': 100.0, 'critical_alert': False
        }

How We Collect the Dataset and Train the Model

Data collection is done from cameras on site under various conditions: day, night, dust. Labeling is performed in LabelStudio by safety experts (8 classes). During training, we apply augmentations: brightness variation, adding noise, simulating dust (motion blur + Gaussian noise). We use YOLOv8/9 with pretrained weights on COCO, fine-tuned on our Mine Safety Dataset (15,000+ frames). Training is done on GPU A100 or RTX 4090 for 2–3 days. The final model is validated on a holdout set — target mAP50 0.85+.

How We Test and Guarantee Accuracy

We compare detection with expert video analysis. Typical metrics:

LWIR thermal cameras outperform RGB cameras in dust and smoke by 2–3 times in detection accuracy. We recommend an RGB+LWIR combination for underground sections. Project cost is calculated individually based on the number of cameras and control zones. Savings on fines and downtime amount to up to 40% annually.

What Is Included in the Work?

We handle the full cycle: audit of existing surveillance systems, dataset collection (labeling considering mine specifics), model training, tracker configuration, SCADA and MES integration, server supply or inference on existing equipment.

• Documentation: ETL flow diagrams, API documentation (REST/RTSP), operation manual.
• Access to a web dashboard with real-time compliance_rate_pct, heat map of danger zones.
• Operator training (2 days, online or on-site).
• Technical support for 3 months after launch.

Indicative Deployment Timelines

Exact estimate after audit. We visit the site or analyze camera recordings. We will evaluate your project for free — contact us to get a consultation.

Guarantees and Support

Over 5 years in industrial CV, 30+ implementations, a team of senior AI/ML engineers. Our systems operate in mines in Kazakhstan, Russia, and South Africa. We guarantee a compliance rate of at least 85% at the acceptance stage. According to National Institute for Occupational Safety and Health, automated PPE monitoring reduces injuries by 35–50%. Contact us for a free audit of your facility. Request a demo of the system.