AI-Powered KBZHU and Meal Composition Analysis from a Photo
Taking a picture of your plate and getting accurate calories, protein, fat, and carbs—this is a challenge dozens of startups are tackling. In practice, accuracy depends on a five-step pipeline: segmentation → classification → volume estimation → database mapping → calculation. The weak link is volume estimation: even state-of-the-art depth estimation models with a reference object produce a 25–35% weight error. Our team has been working on foodtech projects since 2014, delivering 20+ computer vision systems for food recognition. We integrated LiDAR support, cutting the error to 12–18%—twice as good as monocular approaches. Our solutions have already helped clients reduce nutritional counseling costs by 40–60%.
Pipeline: Five Steps from Photo to KBZHU
- Meal segmentation — isolate individual components on the plate: side, meat, sauce.
- Classification — identify each component: borscht, chicken breast, buckwheat.
- Portion volume estimation — the hardest and most narrow part (more details below).
- Database mapping — USDA FoodData Central, OpenFoodFacts, restaurant TTK.
- Nutrient calculation — weight × composition per gram.
The weak link is step 3. A classifier can identify borscht with 0.91 accuracy, but if the portion weight is off by ±40%, the final KBZHU will be incorrect.
Why Volume Estimation Is the Hardest Task
Reference Object: Cheap but Inaccurate
The user places a standard-sized card (business card, bank card, QR marker) next to the plate. The system computes scene scale and estimates area. For volume, multiple images (Structure from Motion) are needed, which is unrealistic in a mobile app—users won't walk around the plate.
Depth Estimation: Monocular vs. LiDAR
Monocular models (DPT, ZoeDepth) produce a relative depth map. With a reference object for scaling, RMSE for weight is 25–35%. Acceptable for fitness tracking, unacceptable for dietary control.
LiDAR scanner (iPhone 12 Pro+, iPad Pro) captures real surface coordinates. RMSE for weight is 12–18%—twice as accurate (confirmed by Apple ResearchKit).
Comparison of Volume Estimation Methods
| Method | RMSE (Weight) | Hardware | Computation Time |
|---|---|---|---|
| Reference + Monocular | 25–35% | Any camera | <500 ms |
| LiDAR + ARKit | 12–18% | iPhone 12+ / iPad Pro | <200 ms |
| 3D Reconstruction from Video | 8–15% | Stereo camera | 2–5 s |
How We Build Accurate Classification Models
Public datasets like Food-101 and UEC Food-256 cover Western dishes, but not Russian cuisine. We use crowdsourcing with nutritionist verification: collect 500–1000 photos per class, label components and weight ratios. This gives 85–88% top-1 accuracy for 80–120 categories.
| Dataset / Base | Classes | Top-1 Accuracy |
|---|---|---|
| Food-101 | 101 | 0.96 (EfficientNet-B7) |
| UEC-Food256 | 256 | 0.89 (ViT-L) |
| VIREO Food-172 | 172 | 0.91 |
| Russian Cuisine (Custom) | 80–120 | 0.85–0.88 |
Accuracy is limited not by CV but by the quality of the food composition database. We combine USDA FoodData Central, OpenFoodFacts, and restaurant TTK cards. For B2B clients, the biggest gain comes from loading their own technological cards—calculations use real recipes.
Which Application Architecture Works for Foodtech?
On-device inference: Core ML (iOS) / TFLite (Android) for segmentation and classification. Latency <300 ms, works offline. Depth estimation is optionally server-side for accuracy. API for integration with dietetic software—fully documented.
What the Work Includes
- Data audit and accuracy requirements definition.
- Model development and fine-tuning (segmentation, classification, depth).
- Integration of composition database (USDA / TTK / custom).
- Mobile app iOS/Android with on-device inference.
- API for integration.
- Documentation and team training.
- Pilot support (1 month).
Timeline
MVP for one cuisine (100–150 dishes), mobile app iOS/Android: 8–12 weeks. Full platform with custom database, LiDAR support, and dietetic software integration: 4–6 months.
Contact us for a preliminary audit—we will assess your project and propose the optimal solution. We guarantee transparency at every stage, and our ISO 13485 certification confirms our experience in precise medical measurements. Request a consultation for your project today.