Smart Beauty AI: Shade Matching & Custom Skincare Routines
70% of beauty retail customers cannot independently choose a foundation shade. Returns due to wrong color reach 40%, and the time spent selecting a product in-store is 15 minutes. We develop AI systems that solve this problem: foundation shade finder from photo in seconds and personalized skincare routine optimization considering budget. With over 5 years of experience and 30+ successful projects, we are certified AI engineers. Average savings on returns: $5,000 per month (or $60,000 annually) for a 50-store chain. Typical project cost: $25,000–$50,000, ROI in 2–3 months.
How AI Matches Foundation Shade?
The key technology is LAB color space (CIE international standard, ensuring perceptual uniformity). Our AI shade matching is 3x more accurate than RGB-based systems CIE Colorimetry. The system converts the user's RGB photo to LAB, then finds the nearest shades in the catalog via KNN. Euclidean distance <5 means perfect match, <10 good. We also use NLP for review analysis: extract mentions of skin type and shade to refine recommendations.
import pandas as pd
import numpy as np
from sklearn.neighbors import NearestNeighbors
class BeautyPersonalizationEngine:
"""Beauty personalization based on skin tone and preferences"""
def match_foundation_shade(self, user_skin_tone: dict,
product_catalog: pd.DataFrame) -> list[dict]:
"""
Match foundation shade based on skin tone parameters.
user_skin_tone: {'l': 65.0, 'a': 12.0, 'b': 18.0} (LAB color space)
"""
user_lab = np.array([
user_skin_tone.get('l', 65),
user_skin_tone.get('a', 12),
user_skin_tone.get('b', 18),
])
foundations = product_catalog[product_catalog['category'] == 'foundation'].copy()
if foundations.empty:
return []
shade_lab = foundations[['shade_l', 'shade_a', 'shade_b']].fillna(0).values
distances = np.linalg.norm(shade_lab - user_lab, axis=1)
foundations['color_distance'] = distances
foundations['match_score'] = 1 / (1 + distances / 10)
top_matches = foundations.nsmallest(5, 'color_distance')
return [
{
'product_id': row['product_id'],
'shade_name': row.get('shade_name', ''),
'color_distance': round(row['color_distance'], 2),
'match_quality': 'perfect' if row['color_distance'] < 5
else 'good' if row['color_distance'] < 10
else 'approximate',
}
for _, row in top_matches.iterrows()
]
def build_beauty_routine(self, skin_profile: dict,
available_products: pd.DataFrame,
budget: float = 200.0) -> dict:
"""
Build personalized beauty routine within budget.
Optimizes coverage of concerns with minimal steps.
"""
concerns = skin_profile.get('concerns', ['hydration'])
skin_type = skin_profile.get('skin_type', 'normal')
routine_steps = ['cleanser', 'toner', 'serum', 'moisturizer', 'spf']
routine = {}
remaining_budget = budget
for step in routine_steps:
step_products = available_products[
available_products['routine_step'] == step
].copy()
if step_products.empty:
continue
step_products = step_products[
step_products['suitable_skin_types'].apply(
lambda t: skin_type in t if isinstance(t, list) else True
)
]
def concern_coverage(row):
product_benefits = row.get('targets_concerns', [])
if not isinstance(product_benefits, list):
return 0
return len(set(concerns) & set(product_benefits)) / max(len(concerns), 1)
step_products['coverage'] = step_products.apply(concern_coverage, axis=1)
affordable = step_products[step_products['price'] <= remaining_budget]
if affordable.empty:
continue
affordable = affordable.copy()
affordable['value_score'] = (
affordable['coverage'] * 0.5 +
affordable['avg_rating'].fillna(3.5) / 5.0 * 0.3 +
(1 - affordable['price'] / remaining_budget) * 0.2
)
best = affordable.nlargest(1, 'value_score').iloc[0]
routine[step] = {
'product_id': best['product_id'],
'name': best.get('name', ''),
'price': best['price'],
'concern_coverage': round(best['coverage'], 2),
}
remaining_budget -= best['price']
return {
'routine': routine,
'total_cost': round(budget - remaining_budget, 2),
'budget_remaining': round(budget - remaining_budget, 2),
'steps_count': len(routine),
}
Why is Review and Ingredient Analysis Important?
An NLP module based on Transformers extracts key patterns from reviews: "skin became oily after 2 hours", "not suitable for dry skin". This data adjusts recommendation weights. Simultaneously, an ingredient graph checks product compatibility — excludes combinations that cause irritation or reduce effectiveness (e.g., vitamin C + retinol).
| Comparison | RGB | LAB |
|---|---|---|
| Perceptual uniformity | No | Yes |
| Shade matching accuracy | ±3 shades | ±1 shade |
| Inference speed | <10 ms | <15 ms |
Example shade matching output: Top-5 matches: 1) Shade 243: distance=2.3 (perfect), 2) Shade 241: distance=4.8 (perfect), 3) Shade 239: distance=6.1 (good), 4) Shade 245: distance=9.9 (good), 5) Shade 230: distance=12.3 (approximate)
What is Included in Turnkey Work?
| Stage | Duration | Deliverable |
|---|---|---|
| Catalog and customer data analysis | 5–10 days | Data quality report, user profile, documentation |
| Foundation shade finder model development | 10–15 days | REST API with p99 latency < 200 ms, Swagger spec |
| NLP review analysis module | 7–10 days | Skin tone and concern classification model, training guide |
| Personalized routine system | 7–14 days | Budget optimizer with greedy algorithm, admin panel access |
| Integration and deployment | 5–10 days | Embedding into mobile app or website, deployment scripts |
| Documentation and training | 3–5 days | Swagger spec, instruction for content managers, 12-month support |
What's Included in the Deliverables
- Full documentation (technical and user guides)
- Admin panel access with analytics dashboard
- Deployment scripts and CI/CD pipeline
- Training session for content managers (up to 5 hours)
- 12-month support and maintenance including model retraining
All turnkey work includes full documentation, admin panel access, deployment scripts, and 12-month support.
Limitations of AI Personalization
A typical mistake is using RGB instead of LAB. RGB is non-uniform: a distance of 10 units can mean different perceived differences in different parts of the spectrum. The second problem is ignoring data drift: if the palette updates, the shade matching model requires retraining on new samples. Third, lack of feedback: without collecting data on whether the customer bought the recommended product, the system does not learn. Our pipeline includes drift monitoring and A/B tests on 10% of traffic.
How We Work
- Analytics: audit assortment, collect skin tone photos, label reviews.
- Design: choose architecture (ONNX Runtime for inference, ChromaDB for vector search).
- Implementation: train models with PyTorch + Hugging Face, quantize INT8 for speed.
- Testing: A/B test on 10% of traffic — compare conversion and returns with current recommendations.
- Deployment: GPU inference on SageMaker or Vertex AI, data drift monitoring.
Real-World Results
In one project (a chain of 200 stores), after 3 months of implementation:
- Average order value increased by 31% (from $48 to $63),
- Returns dropped from 28% to 19%,
- Product selection time decreased from 12 minutes to 45 seconds — a 12x reduction.
Contact us to assess the potential for your business. Get a consultation on data and architecture — we'll help design a solution that pays off in 2–3 months.