Building a Production-Ready Image Processing Pipeline

Images are the heaviest part of any website. A single unprocessed camera photo can weigh 10+ MB and take seconds to load. Meanwhile, users abandon a page if LCP exceeds 2.5 seconds. We regularly encounter projects where images are not optimized: no resizing for different screens, no WebP, watermarks applied manually. All of this means lost conversions and wasted traffic costs.

A typical scenario: an e-commerce store with thousands of products. Photos are uploaded through the admin panel, but thumbnails have to be made manually or via workarounds. We offer a ready-made infrastructure: when a file is uploaded, it goes through a pipeline and immediately outputs several versions — thumbnail (150x150), medium (800x600), large (1920x1080) — all in WebP and JPEG. The original is also saved separately.

Watermarks are applied automatically if the image is public. We use a semi-transparent logo in the bottom-right corner — it doesn’t obstruct the view but protects the content. All operations take no more than 200 ms per image.

Our pipeline solves these problems automatically. You upload the original — the system itself generates a set of previews, converts to modern formats, and applies a watermark. And you don’t need to rewrite your existing logic: integration takes between 2 and 3 days. The pipeline can resize images automatically to needed dimensions, crop images exactly, and apply a watermark overlay.

What Problems Does the Pipeline Solve?

Slow loading due to large originals — first problem. One camera photo can weigh 20 MB. On mobile internet this destroys UX. Resizing to the required dimensions and conversion to WebP reduces size by 3–5 times without quality loss.

Lack of adaptive images — second problem. If you show a 1920x1080 image on desktop, on a phone it will be the same size but compressed by the browser, increasing LCP. Our pipeline generates several versions for different resolutions, and we set up srcset — the browser itself chooses the appropriate variant.

Inability to automatically convert formats — third problem. Manual conversion to WebP or AVIF is a slow process that is often forgotten. The pipeline does it on the fly, preserving both the original and the derivatives. Traffic savings reach 40%.

How We Build the Pipeline: Stack and Architecture

For synchronous processing we use Node.js with Sharp. Sharp is 3x faster than Python alternatives, according to Sharp library documentation. For asynchronous processing we use Celery + Pillow. If the load is high, we send tasks to a Redis queue — the server is not blocked. An alternative is imgproxy, which transforms images on-the-fly via URL.

Comparison of approaches:

Which to choose? If you have up to 1000 uploads per day — synchronous is enough. For large projects with millions of images — asynchronous with a queue, enabling batch image processing.

Comparison of formats:

Switching to WebP reduces LCP by 30% and saves up to 40% in traffic. Automatic format conversion (including WebP conversion) is built into the pipeline.

What’s Included in the Work?

The service includes:

• Architecture design of the pipeline (approach selection, stack).
• Development and integration with your project (API, middleware).
• CDN and caching setup (Cloudflare, Vercel) — we integrate a CDN for images to ensure fast delivery worldwide.
• Documentation for usage and further customization.
• Team training (1 hour online).
• 30-day support guarantee for stable operation.
• Access credentials and API keys.

Image optimization is part of the pipeline, reducing file sizes without quality loss.

Our team has 7+ years in web development and over 50 completed image processing projects. Our company has been on the market for over 5 years.

Work Process

1. Analysis — audit of current infrastructure, typical sizes, formats, load.
2. Design — choose approach (synchronous/asynchronous), define set of previews.
3. Implementation — write pipeline code, integrate with storage and CDN.
4. Testing — load testing, cross-browser compatibility checks.
5. Deployment — roll out to production with monitoring.

Timelines and Cost

A basic implementation with Sharp or imgproxy takes 2 to 5 days. Cost is calculated individually after an audit — it depends on complexity, volumes, and whether asynchronous processing is needed. We will assess your project for free within one day. Typical budgets start at $2,000 for a basic pipeline.

Typical Implementation Mistakes

• Ignoring EXIF orientation — photos from phones may be rotated. In Sharp we automatically read metadata and correct it.
• Loss of transparency — when converting PNG to JPEG, the background becomes white. Our pipeline preserves the alpha channel or replaces it with a white background.
• Too many preview sizes — generating 10+ variants slows down processing. Optimal is 3–4 sizes.
• No caching — every request to imgproxy consumes resources. We set Cache-Control to one year.

This image automation pipeline eliminates manual work and ensures consistent results.

# tasks.py (Celery) from celery import Celery from PIL import Image import io, boto3 app = Celery('image_tasks', broker='redis://redis:6379') @app.task(bind=True, max_retries=3) def process_image(self, image_id: int): try: record = db.get_image(image_id) raw = s3.get_object(Bucket='uploads', Key=record.original_key)['Body'].read() img = Image.open(io.BytesIO(raw)) img = ImageOps.exif_transpose(img) if img.mode == 'RGBA': background = Image.new('RGB', img.size, (255, 255, 255)) background.paste(img, mask=img.split()[3]) img = background variants = {} for name, (w, h) in SIZES.items(): resized = img.copy() resized.thumbnail((w, h), Image.LANCZOS) buf = io.BytesIO() resized.save(buf, 'WEBP', quality=85, method=6) buf.seek(0) key = f"processed/{image_id}/{name}.webp" s3.put_object(Bucket='media', Key=key, Body=buf, ContentType='image/webp', CacheControl='public, max-age=31536000') variants[name] = key db.update_image_variants(image_id, variants) except Exception as exc: raise self.retry(exc=exc, countdown=60)