Custom Prompt Registry Development: Versioning, API & SSO Integration

Note: when the number of prompts in LLM systems exceeds a dozen, chaos begins. We once saw a project where versions were stored in Jira, Confluence, Slack, and even in code comments. A typical typo in a prompt — and a $2500 payment request went out with incorrect data. Deploying a change to 50 microservices took 3 days of manual copying. A Prompt Registry is a prompt management system that centralizes storage, versioning, and deployment. We develop custom Prompt Registry turnkey for companies that find ready-made solutions (PromptLayer, Humanloop) insufficient. Our experience — 10+ years in AI/ML, 50+ launched projects, including RegTech and FinTech with strict security requirements. Our certified engineers ensure a seamless deployment and provide a guaranteed uptime SLA.

According to OpenAI developers, prompt versioning is a key element of production-ready LLM systems.

Problems that a centralized prompt hub solves

Without a unified prompt registry, three critical problems eventually arise:

• Lack of versioning. During audits — compliance failure. No one knows which prompt was used for each request. Restoring history is a manual search through logs and chats.
• Manual propagation of changes. Updating a prompt across 50 microservices is a nightmare for DevOps. With a registry — one API request, and all clients pick up the new version in seconds. Rollback — one call.
• No quality monitoring. We implement metric collection: latency p99, tokens, request cost, quality score. This enables A/B testing of prompts and selecting the best one. Typical quality improvement — 15–30%.

Why do companies need a custom prompt registry?

Ready-made services (PromptLayer, Humanloop) are a good entry-level, but they do not meet corporate needs. Here are key differences:

The table below shows real improvements after custom registry implementation:

A custom Prompt Registry pays for itself by reducing deployment time by 10x compared to manual management. Typical MLOps budget savings — 40%, which for a mid-size company translates to $50,000–$100,000 annually. Engineers' time is freed for more valuable tasks. The solution is suitable for LLM prompt management in large companies with high security requirements.

How do we ensure versioning and rollbacks?

The data schema is based on PostgreSQL. Each prompt version is protected by a SHA256 hash — duplicates are eliminated. On deployment, a record is created in prompt_deployments with a reference to the previous version (rollback_of). Rollback — one API call.

-- PostgreSQL schema for our custom prompt registry
CREATE TABLE prompts (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name VARCHAR(255) NOT NULL,
    description TEXT,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    created_by VARCHAR(255) NOT NULL,
    tags TEXT[]
);

CREATE TABLE prompt_versions (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    prompt_id UUID REFERENCES prompts(id),
    version_number INTEGER NOT NULL,
    content TEXT NOT NULL,
    content_hash VARCHAR(64) NOT NULL,  -- SHA256
    model VARCHAR(100) NOT NULL,
    temperature FLOAT DEFAULT 0.0,
    max_tokens INTEGER DEFAULT 1000,
    variables JSONB DEFAULT '[]',
    metadata JSONB DEFAULT '{}',
    created_at TIMESTAMPTZ DEFAULT NOW(),
    created_by VARCHAR(255) NOT NULL,
    UNIQUE(prompt_id, version_number)
);

CREATE TABLE prompt_deployments (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    prompt_version_id UUID REFERENCES prompt_versions(id),
    environment VARCHAR(50) NOT NULL,  -- dev/staging/production
    deployed_at TIMESTAMPTZ DEFAULT NOW(),
    deployed_by VARCHAR(255) NOT NULL,
    is_active BOOLEAN DEFAULT TRUE,
    rollback_of UUID  -- Reference to previous version on rollback
);

CREATE TABLE prompt_executions (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    prompt_version_id UUID REFERENCES prompt_versions(id),
    executed_at TIMESTAMPTZ DEFAULT NOW(),
    input_variables JSONB,
    rendered_prompt TEXT,
    response TEXT,
    input_tokens INTEGER,
    output_tokens INTEGER,
    latency_ms INTEGER,
    cost_usd FLOAT,
    quality_score FLOAT  -- Quality score (if available)
);

Fast API on FastAPI

We use FastAPI to create the prompt registry, ensuring high performance. Example of creating a version and retrieving the latest active one:

from fastapi import FastAPI, HTTPException, Depends
from pydantic import BaseModel
import asyncpg

app = FastAPI(title="Prompt Registry API")

class PromptCreateRequest(BaseModel):
    name: str
    content: str
    model: str = "gpt-4o"
    temperature: float = 0.0
    description: str = None

@app.post("/prompts/{name}/versions")
async def create_version(
    name: str,
    request: PromptCreateRequest,
    db = Depends(get_db)
):
    content_hash = hashlib.sha256(request.content.encode()).hexdigest()
    existing = await db.fetchrow(
        "SELECT id FROM prompt_versions pv JOIN prompts p ON p.id = pv.prompt_id "
        "WHERE p.name = $1 AND pv.content_hash = $2",
        name, content_hash
    )
    if existing:
        raise HTTPException(400, "Identical prompt version already exists")

    version = await db.fetchrow("""
        INSERT INTO prompt_versions (prompt_id, version_number, content,
            content_hash, model, temperature)
        SELECT p.id,
               COALESCE(MAX(pv.version_number), 0) + 1,
               $2, $3, $4, $5
        FROM prompts p
        LEFT JOIN prompt_versions pv ON pv.prompt_id = p.id
        WHERE p.name = $1
        GROUP BY p.id
        RETURNING id, version_number
    """, name, request.content, content_hash, request.model, request.temperature)

    return {"version_id": str(version['id']), "version": version['version_number']}

@app.get("/prompts/{name}/latest")
async def get_latest(name: str, environment: str = "production", db = Depends(get_db)):
    prompt = await db.fetchrow("""
        SELECT pv.content, pv.model, pv.temperature, pv.variables, pv.version_number
        FROM prompt_versions pv
        JOIN prompt_deployments pd ON pd.prompt_version_id = pv.id
        JOIN prompts p ON p.id = pv.prompt_id
        WHERE p.name = $1 AND pd.environment = $2 AND pd.is_active = TRUE
        ORDER BY pd.deployed_at DESC LIMIT 1
    """, name, environment)

    if not prompt:
        raise HTTPException(404, f"No deployed prompt '{name}' in {environment}")
    return dict(prompt)

Python client for integration

For ease of use, we write a client library in Python. It caches the latest active version and substitutes template variables:

class PromptClient:
    def __init__(self, registry_url: str, api_key: str):
        self.url = registry_url
        self.headers = {"X-API-Key": api_key}
        self._cache = {}

    def get_and_render(self, name: str, variables: dict,
                       environment: str = "production") -> str:
        cache_key = f"{name}:{environment}"
        if cache_key not in self._cache:
            resp = requests.get(
                f"{self.url}/prompts/{name}/latest",
                params={"environment": environment},
                headers=self.headers
            )
            self._cache[cache_key] = resp.json()

        template = self._cache[cache_key]['content']
        for var, value in variables.items():
            template = template.replace(f"{{{{{var}}}}}", str(value))
        return template

Prompt versioning for compliance

Full change history with SHA256 hash and metadata allows delivering an audit report in minutes: which prompt, when, and by whom was used. Our RegTech clients reduce audit time from weeks to hours. A custom solution implements changes 5x faster than manual microservice updates. SSO integration for the prompt registry is a standard option that simplifies the audit trail.

Work process

1. Analysis — audit of current practices, authentication requirements, compliance, volumes. Define target metrics (p99 latency < 30 ms, throughput > 1000 rps).
2. Design — database schema, API architecture, deployment model (Kubernetes, bare-metal).
3. Implementation — backend development, client, CI/CD integration (GitLab CI, GitHub Actions).
4. Testing — unit, integration, load tests (p99 latency measured up to 50 ms).
5. Deployment — rollout on your environments, documentation, team training.
6. Support — SLA, monitoring, improvements based on feedback.

Our engineers are proficient in prompt engineering and can tailor the system to any business processes. With over 5 years of MLOps experience and 50+ completed projects, we deliver a proven solution.

Timeline and what's included

Approximately — from 4 to 8 weeks depending on integration complexity (SSO, custom metrics). Includes: working system, documentation (API + administration), training for up to 5 people, source code, 1 month support. Cost is calculated individually — contact us, we will evaluate your project. Typical projects cost $30k–$80k.

Typical result after implementation: deployment speed of changes increases 10x, deployment error rate decreases by 95%, audit time shrinks from weeks to minutes. A unified Prompt Registry becomes a key component of the MLOps infrastructure.

Get a consultation: tell us about your tasks — we will propose a solution considering your data governance and budget. Order a preliminary analysis through the form on the website.