AI assistant for recipe recommendation in mobile app

AI Recipe Assistant in Mobile Applications

"What can I cook from what's in my fridge?" is a classic bounded inventory problem. The AI assistant isn't merely a recipe database search engine—it generates recipes for specific ingredient sets, dietary restrictions, and cooking time.

Ingredient Input: Text, Photo, Voice

Three input channels—all necessary. Text is obvious. Voice uses SFSpeechRecognizer/Android SpeechRecognizer. Most useful: fridge photo with ingredient recognition.

For ingredient recognition in photos: CoreML with food-dataset fine-tuned models (Food-101 or OpenFoodFacts), or Google Cloud Vision API with label detection.

// iOS - ingredient recognition via Vision + CoreML
func recognizeIngredients(in image: UIImage) async throws -> [String] {
    guard let cgImage = image.cgImage else { return [] }

    let model = try FoodClassifier(configuration: .init())
    let vnModel = try VNCoreMLModel(for: model.model)

    let request = VNCoreMLRequest(model: vnModel)
    request.imageCropAndScaleOption = .centerCrop

    let handler = VNImageRequestHandler(cgImage: cgImage)
    try handler.perform([request])

    guard let results = request.results as? [VNClassificationObservation] else { return [] }

    return results
        .filter { $0.confidence > 0.6 }
        .prefix(10)
        .map { $0.identifier }
}

On Android, use ML Kit ImageLabeler with localModel or remoteModel (downloaded on first use).

Recipe Generation with Constraints

The prompt is key. Constraints can be numerous: allergies, diets, servings, cooking time, difficulty.

struct RecipeRequest: Encodable {
    let ingredients: [String]
    let servings: Int
    let maxCookingMinutes: Int
    let dietaryRestrictions: [String]  // "vegan", "gluten-free", "nut-allergy", ...
    let difficulty: String             // "easy", "medium", "hard"
    let cuisinePreferences: [String]   // optional
}

func buildRecipePrompt(_ req: RecipeRequest) -> String {
    """
    Create a recipe using ONLY these ingredients (you may add basic pantry staples: salt, oil, water, common spices):
    Available: \(req.ingredients.joined(separator: ", "))

    Requirements:
    - Servings: \(req.servings)
    - Max cooking time: \(req.maxCookingMinutes) minutes
    - Dietary: \(req.dietaryRestrictions.isEmpty ? "none" : req.dietaryRestrictions.joined(separator: ", "))
    - Difficulty: \(req.difficulty)

    Return JSON: {name, cookingTime, servings, ingredients: [{name, amount, unit}], steps: [{number, instruction, duration}], nutrition: {calories, protein, carbs, fat}}
    """
}

response_format: json_object is mandatory—parsing markdown-wrapped JSON in production isn't worth the risk.

Recipe Card UI

// Android Compose
@Composable
fun RecipeCard(recipe: Recipe) {
    LazyColumn(
        modifier = Modifier.fillMaxSize(),
        contentPadding = PaddingValues(16.dp),
        verticalArrangement = Arrangement.spacedBy(12.dp)
    ) {
        item {
            Text(recipe.name, style = MaterialTheme.typography.headlineSmall)
            Row(horizontalArrangement = Arrangement.spacedBy(16.dp)) {
                InfoChip(Icons.Default.Timer, "${recipe.cookingTime} min")
                InfoChip(Icons.Default.People, "${recipe.servings} servings")
                InfoChip(Icons.Default.LocalFireDepartment, "${recipe.nutrition.calories} kcal")
            }
        }

        item { Text("Ingredients", style = MaterialTheme.typography.titleMedium) }
        items(recipe.ingredients) { ing ->
            Text("• ${ing.amount} ${ing.unit} ${ing.name}")
        }

        item { Text("Instructions", style = MaterialTheme.typography.titleMedium) }
        itemsIndexed(recipe.steps) { index, step ->
            StepCard(number = index + 1, instruction = step.instruction, duration = step.duration)
        }
    }
}

Timers for Cooking Steps

Each step with specified duration should start a timer directly from the card. Not an AI feature, but what makes the assistant truly useful while cooking.

class StepTimerManager: ObservableObject {
    @Published var activeTimers = [Int: TimeInterval]()
    private var timers = [Int: Timer]()

    func startTimer(for stepIndex: Int, duration: TimeInterval) {
        activeTimers[stepIndex] = duration

        timers[stepIndex] = Timer.scheduledTimer(withTimeInterval: 1.0, repeats: true) { [weak self] _ in
            guard let self else { return }
            if let remaining = self.activeTimers[stepIndex], remaining > 0 {
                self.activeTimers[stepIndex] = remaining - 1
            } else {
                self.timers[stepIndex]?.invalidate()
                self.notifyStepComplete(stepIndex)
            }
        }
    }

    private func notifyStepComplete(_ step: Int) {
        let content = UNMutableNotificationContent()
        content.title = "Step \(step + 1) complete"
        content.sound = .default
        UNUserNotificationCenter.current().add(
            UNNotificationRequest(identifier: "step-\(step)", content: content, trigger: nil)
        )
    }
}

Saving and Personalization

Recipes the user saves and cooks form a preference profile. On next request, add "Previously liked: [list]" to the prompt—this improves relevance without fine-tuning.

Store recipes: SwiftData (iOS 17+) or Core Data—JSON-serialize recipe structure. On Android, use Room with TypeConverter for List<Ingredient> and List<Step>.

Timeline Estimates

Basic assistant (text input + recipe generation)—3–5 days. Full implementation with ingredient recognition from photos, step timers, preference profile, and offline storage—3–4 weeks.