Plant Recognition from Photos in Mobile Applications
User photographs an unfamiliar flower in a park and instantly sees name, description, and toxicity warning. Technically straightforward—PlantNet API and Google Cloud Vision have ready endpoints. The gap between "works in demo" and "works in production" usually hides in handling poor photos and UX at low model confidence.
Implementation Options
Two paths: cloud API or on-device model.
Cloud APIs (PlantNet, iNaturalist, Plant.id) provide high accuracy and regularly updated database (PlantNet—30,000+ species). Trade-off: internet dependency and 1–3 second latency. For most apps, acceptable.
On-device via CoreML (iOS) or TensorFlow Lite (Android)—works offline, instant response, but model requires OTA updates and accuracy on rare species is noticeably lower. EfficientNet-B4 model fine-tuned on PlantCLEF dataset—around 20 MB in .mlmodel format.
For most projects, optimal: CoreML/TFLite for fast offline results (top-3 candidates) + cloud for refinement with connectivity.
Plant.id API Integration
struct PlantIdentificationService {
func identify(image: UIImage) async throws -> [PlantMatch] {
guard let imageData = image.jpegData(compressionQuality: 0.8) else {
throw PlantError.invalidImage
}
let base64 = imageData.base64EncodedString()
let request = PlantIdentifyRequest(
images: [base64],
modifiers: ["crops_fast", "similar_images"],
plant_language: "en",
plant_details: ["common_names", "url", "description",
"taxonomy", "edible_parts", "toxicity"]
)
let response = try await apiClient.post("/identify", body: request)
return response.suggestions
.filter { $0.probability >= 0.1 } // filter very weak matches
.prefix(5)
.map { PlantMatch(from: $0) }
}
}
The toxicity field is important. For apps with parent audiences or foragers, toxicity warnings should be visually prominent—not item five in details.
Handling Poor Photos
Common problem: blurry shots, bad angles (stem only without leaves/flower), dark background. Detect before API call:
func assessImageQuality(_ image: UIImage) -> ImageQualityResult {
// Blurriness via Laplacian variance
let laplacianVariance = computeLaplacianVariance(image)
if laplacianVariance < 50 {
return .tooBlurry
}
// Check plant present—via CoreML Vision classifier
let plantPresenceScore = runPlantPresenceClassifier(image)
if plantPresenceScore < 0.3 {
return .noPlantDetected
}
return .acceptable
}
On .tooBlurry, ask to retake immediately—don't waste API call.
Timeline Estimates
Integration with one cloud API (Plant.id or PlantNet), processing results, basic plant card UI—1–2 days. Adding on-device model, image quality handling, recognition history, offline mode, and both platform support—1–1.5 weeks.