Image Segmentation: Semantic, Instance, and Panoptic Systems

Image segmentation is not just bounding boxes — it's pixel-level object masks. When contours matter: medical scans (tumors), satellite data (wildfires), autonomous driving (pedestrians), quality control (defects with area measurement). We develop end-to-end segmentation systems, from prototype to production. We'll evaluate your project in 2 days, and the savings on manual labeling can exceed the development cost by 10 times.

Choosing the Right Segmentation Model

Semantic segmentation assigns each pixel to a class. All cars become one 'car' class, all pedestrians 'person'. Popular models: SegFormer, DeepLabV3+. Instance segmentation distinguishes individual objects: car #1, car #2. Leaders: Mask R-CNN, YOLOv8-seg, YOLO11-seg. Panoptic segmentation combines both: 'things' via instances, 'stuff' semantically. The gold standard is Mask2Former.

What Does the Segment Anything Model Offer?

Meta's SAM is a game changer. Zero-shot: no training needed for specific classes. Input prompt — point, box, or mask. As noted in Segment Anything paper, the model can segment any object in any image. Read more about SAM on GitHub.

from segment_anything import SamPredictor, sam_model_registry

sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h_4b8939.pth")
predictor = SamPredictor(sam)

predictor.set_image(image)
masks, scores, _ = predictor.predict(
    box=np.array([x1, y1, x2, y2]),
    multimask_output=False
)

The updated SAM2 supports video and tracking segmentation across frames. SAM segments objects 10x faster than manual labeling, but it doesn't classify segments and is too slow for real-time (SAM-ViT-H: ~50ms on A100).

Fine-Tuning for Your Domain

For specialized domains (medicine, industry), SAM can be fine-tuned. Ultra-light models based on EfficientNet achieve mIoU up to 0.92 on internal datasets. To improve accuracy, we use augmentations: random crops, rotations, contrast adjustments.

from ultralytics import SAM

model = SAM('sam2_b.pt')
model.train(
    data='medical_dataset.yaml',
    epochs=50,
    imgsz=1024,
    batch=4,
    lr0=1e-4
)

For semantic segmentation, we use SegFormer (HuggingFace). SegFormer-B5 achieves mIoU 84.0 on Cityscapes. When working with small datasets (<500 images), we employ transfer learning and thorough validation.

U-Net — the Biomedical Standard

U-Net with encoder-decoder and skip connections works well on small datasets (200–500 images). Data augmentation is key: combining shifts, scalings, and Elastic Transform improves mIoU by 5–7%.

import segmentation_models_pytorch as smp

model = smp.Unet(
    encoder_name='efficientnet-b4',
    encoder_weights='imagenet',
    in_channels=1,
    classes=3,
    activation=None
)

Quality Metrics

• mIoU — primary for semantic
• AP — for instance
• Dice coefficient — for medical tasks
• Boundary IoU — contour accuracy

Why Data Preprocessing Matters?

Annotation quality directly affects results. We use polygon annotations in COCO format, check consistency via IoU between annotators. For medical images, normalization and artifact removal are mandatory. A common pitfall is imbalanced classes: solved by weighted loss functions (Focal Loss, Dice Loss).

How Segmentation Reduces Costs

For example, on a metal defect segmentation project, we used YOLOv8-seg fine-tuned on 1500 images. We achieved mIoU 0.91, and automation reduced inspection time from 40 seconds to 2 seconds per part. The customer saved over 2 million rubles per year on quality control. In another case, precise segmentation of fields from satellite data reduced agrochemical costs by 1.2 million rubles per season. Such automation brings significant savings at scale. Contact us to assess potential benefits for your business.

Our Experience in Segmentation

10+ years of experience in computer vision, 50+ projects. We guarantee quality: mIoU not lower than 0.85 on your data. We evaluate your project in 2 days. Request a consultation — we'll help choose the model and set up the pipeline.

How We Work

1. Analysis — study the task, data, annotations. Propose architecture.
2. Prototyping — quickly test 2–3 models on your dataset.
3. Fine-tuning — fine-tune the chosen model, optimize hyperparameters.
4. Production — package into Docker, TensorRT, or Triton Inference Server.
5. Support — documentation, team training, 6-month warranty.

What's Included

• Model trained on your data
• Inference API (REST/gRPC)
• Integration into your existing infrastructure
• Pipeline code and documentation
• Training of your engineers

Timelines

Cost is calculated individually based on scope. Request a consultation — get a project estimate in 2 days.