A surgeon operating through a da Vinci console sees the surgical field via a stereo camera with 10x magnification. His hand movements are scaled and tremor-filtered. But even the best robotic system doesn't answer key questions: is an artery damaged? Is tissue tension sufficient? We develop AI computer vision systems that add this context in real time. Our CV system tracks instrument positions, segments anatomical structures, and builds a 3D map of the surgical field. This is not a research prototype—it's production-ready solutions certified to IEC 62304. Our experience: 10+ years in machine vision and 5 projects in medical robotics. Typical development costs range from $50,000 for a basic tracking module to $500,000+ for a full system with regulatory certification. Investing in our AI surgical robotics can reduce complication rates by up to 30%, saving thousands in potential litigation costs. Get a consultation: we'll evaluate your project in 2 days.
Below we break down three key tasks: instrument tracking, tissue segmentation, and depth estimation for AR overlay. We'll show which models work in the operating room and talk about implementation timelines.
CV Tasks in Surgical Robotics
Surgical Instrument Tracking
The basic task without which most else won't work. The system must know in real time: where each instrument (clamp, scissors, needle) is, its orientation in 3D space, and speed.
Instance segmentation approach: Mask R-CNN or YOLOv8m-seg for pixel-wise segmentation of each instrument. On the Cholec80 dataset (80 videos of laparoscopic cholecystectomies), fine-tuned YOLOv8m-seg yields mAP50 = 0.89 for instrument detection. YOLOv8m-seg is 3x faster than Mask R-CNN with comparable accuracy.
Keypoint detection approach: for precise orientation estimation—detection of keypoints (tip, jaw junction, handle). ViTPose or HRNet adapted for surgical instruments.
How is the problem of instrument occlusions solved?
Partial visibility and occlusions—when instruments overlap each other or go off frame. Temporal prediction (Kalman filter or RNN) restores the track during temporary loss. In our projects we use a combination of YOLOv8-seg and Kalman filter with observability over 5–10 frames back.
Segmentation of Anatomical Structures
Distinguishing critical structures (bile duct, arteries, nerves) from surrounding tissue—a task with the highest accuracy requirements. An error here means patient injury.
Special aspects: intraoperative images are not clean textbook anatomy. Blood, smoke from electrocautery, tissue deformation during manipulations. The model must be robust to these artifacts.
Datasets: CholecSeg8k (8,000 annotated frames of cholecystectomies, 13 tissue classes), Endoscapes (endoscopic scenes), KidneyVSS. Architecture: TransFuse (CNN + Transformer fusion) or HardNet—specialized for medical segmentation.
Why depth estimation is the hardest task?
Technically the most complex part. The surgeon operates in 3D space but sees a 2D image (if no stereo system). Depth estimation allows recovering 3D from monocular endoscope video.
Why it's hard
Endoscopic images violate most assumptions of standard depth estimation models:
- Significant lens distortion (wide-angle endoscope optics)
- Lack of texture on homogeneous tissues (models see no parallax)
- Specular highlights—glare from wet tissues fool the model
- Soft tissue deformation—unlike static scenes, tissues move and deform
Approaches
Self-supervised depth estimation (Monodepth2, DynDepth): training without ground truth depth maps, only from frame sequences. Photometric loss + ego-motion prediction. On laparoscopic data: AbsRel ≈ 0.12–0.18—acceptable for relative estimates.
Stereo + structured light: da Vinci Xi has stereo endoscope. Disparity from stereo matching (RAFT-Stereo, CFNet) gives accurate absolute depth. AbsRel < 0.05 with proper calibration.
Hybrid approach: depth estimation as prior + sparse 3D landmarks from feature matching (SuperGlue + SuperPoint) for refinement. Used for AR overlay of anatomical atlas on video.
| Method | AbsRel | Requires stereo | Latency (ms) |
|---|---|---|---|
| Monodepth2 (self-sup) | 0.16 | No | 12 |
| RAFT-Stereo | 0.04 | Yes | 28 |
| DPT (mono) | 0.14 | No | 18 |
| Hybrid (Stereo + DPT) | 0.03 | Yes | 35 |
Augmented Reality Overlay
Overlaying an anatomical atlas (structures that must not be damaged) on top of the surgical video in real time. Pipeline: Depth estimation → 3D registration with preoperative CT/MRI → tissue deformation model (to account for movements) → AR rendering.
Latency requirement: < 50 ms end-to-end. This dictates architecture choice: only lightweight models or specialized GPU.
End-to-End AI System Development Process
We apply a step-by-step approach to minimize risks and ensure compliance with medical standards.
- Task analysis: requirements gathering, selection of target tasks (tracking, segmentation, depth estimation), assessment of available data.
- Data collection and annotation: frame annotation (instrument masks, keypoints, depth maps). We use internal tools for semantic segmentation. Time savings at the annotation stage—up to 40% via active learning.
- Training and validation: architecture selection, fine-tuning on medical datasets, evaluation by AbsRel, mAP, latency.
- Integration and optimization: ONNX Runtime or TensorRT for acceleration, deployment on NVIDIA IGX Orin.
- Testing and certification: verification on representative data, documentation per IEC 62304.
- Deployment and support: installation in the OR, staff training, warranty service.
What's Included in the Work
- Documentation: model card, architecture description, operation manual.
- Access: source code, trained models, dataset annotations.
- Training: training of the client's team to work with the system.
- Support: technical support during implementation and warranty period.
- Adaptation: to the specific OR and integration with existing equipment.
System Requirements in the OR
Medical requirements are strict: IEC 62304 (medical device software lifecycle), FDA 510(k) or CE MDR for clinical use. This is not just CV development—it's medical development with corresponding documentation, validation, and certification.
Computing platform: NVIDIA IGX Orin (medically-qualified platform) or NVIDIA Clara Holoscan for real-time medical AI systems. Supports 4K 60fps processing with latency < 1 frame.
Comparison of Instrument Segmentation Methods
| Method | mAP50 | Latency (ms) | Occlusion Robustness |
|---|---|---|---|
| Mask R-CNN | 0.91 | 45 | Medium |
| YOLOv8m-seg | 0.89 | 12 | High |
| HRNet + Kalman | 0.87 | 18 | Very High |
Timelines
Research system for a specific task (instrument tracking or segmentation of one structure): 3–5 months. Production-ready system with medical validation: 9–18 months, considering regulatory requirements. Development cost is calculated individually depending on complexity and scope.
Our surgical AI assistant enhances safety by providing real-time alerts. Contact us to discuss your project. We'll help evaluate tasks and timelines. Get a consultation: our engineers will analyze your data and propose the optimal solution. Request a consultation today.
Technical details of the pipeline
For tracking, a combination of YOLOv8-seg + Kalman filter is used. Tissue segmentation is performed with TransFuse. Depth estimation—hybrid of stereo and DPT. The entire pipeline is optimized with TensorRT and runs on IGX Orin with latency < 50 ms.