OpenAI API Integration: GPT-4o, o1, o3 — Under the Hood

OpenAI API Integration: GPT-4o, o1, o3 — Under the Hood

Recently, on a project with tens of thousands of requests per day, the team hit a budget wall due to using a single model for all tasks. After migrating to a combination of GPT-4o, o3-mini, and GPT-4o-mini, we managed to reduce costs by 3x without losing quality. Let's dive into how to choose models, configure a client with retries, and implement structured outputs. Our experience — 5 years in AI integrations and over 50 completed projects — allows us to guarantee 99.9% uptime.

Which Models to Choose and Why Beginners Get It Wrong

OpenAI offers a family of models with different architectures. GPT-4o is a universal multimodal soldier: it accepts text and images, outputs structured responses, and works fast. For deep reasoning (mathematical proofs, algorithmic code), use o1 and o3-mini — they spend more time on chain-of-thought. For high-load scenarios with simple tasks (e.g., classification), use GPT-4o-mini: its p99 latency is 2x lower, and cost per token is almost 10x lower.

A typical mistake is using a single model for everything. GPT-4o-mini handles 80% of tasks, but many use GPT-4o everywhere, overpaying. Below is a comparison for three typical scenarios.

How We Configure the Client and Handle Errors

We use the official openai SDK and Pydantic for schemas. We wrap all calls in retry with exponential backoff (tenacity) — on 429 or 5xx we wait with increasing pause. Below is a working example for synchronous and asynchronous modes:

from openai import OpenAI, AsyncOpenAI
from pydantic import BaseModel

client = OpenAI()  # Uses OPENAI_API_KEY from env
async_client = AsyncOpenAI()

# Synchronous call with retry
from tenacity import retry, stop_after_attempt, wait_exponential

@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=4, max=10))
def chat(prompt: str, model: str = "gpt-4o") -> str:
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        temperature=0.1,
    )
    return response.choices[0].message.content

# Structured output
class Extraction(BaseModel):
    name: str
    amount: float
    currency: str

def extract_structured(text: str) -> Extraction:
    response = client.beta.chat.completions.parse(
        model="gpt-4o",
        messages=[{"role": "user", "content": f"Extract data: {text}"}],
        response_format=Extraction,
    )
    return response.choices[0].message.parsed

# Streaming
def stream_response(prompt: str):
    with client.chat.completions.stream(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
    ) as stream:
        for chunk in stream.text_stream:
            yield chunk

# Vision (GPT-4o)
def analyze_image(image_url: str, question: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {"url": image_url}},
                {"type": "text", "text": question}
            ]
        }]
    )
    return response.choices[0].message.content

Why It's Important to Structure Responses?

Without a schema, the API returns free text — hard to parse and validate. We use response_format with Pydantic to guarantee format. This cuts backend processing time and eliminates parsing errors. An example for entity extraction is shown above.

How o1/o3 Work for Reasoning Tasks?

These models do not support system prompt, temperature (fixed), or streaming. However, you can adjust reasoning_effort (low/medium/high). We use them for narrow tasks: code verification, proofs, logical chains. Example:

# o1 does not support system prompt, temperature, streaming
def reason_with_o1(problem: str) -> str:
    response = client.chat.completions.create(
        model="o3-mini",
        messages=[{"role": "user", "content": problem}],
        reasoning_effort="high",
    )
    return response.choices[0].message.content

Embeddings and Semantic Search

For RAG systems, we use text-embedding-3-small (1536 dimensions). It's cheap and effective. We store vectors in Qdrant or pgvector. Example:

def get_embeddings(texts: list[str]) -> list[list[float]]:
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=texts,
    )
    return [item.embedding for item in response.data]

What's Included in a Turnkey Solution

• Client setup with retries, logging, and monitoring (including alerts on p99 latency).
• Selection of the optimal model for each task (cost/quality).
• Implementation of structured outputs with Pydantic.
• Integration of embeddings and vector DB (RAG).
• API documentation and team training.
• Uptime guarantee of 99.9% (our responsibility). Over 50 projects in 5 years — statistics you can trust.

Timeline and How to Get Started

• Basic chat completions integration: 0.5–1 day.
• Structured outputs + tools: 2–3 days.
• Retry logic + cost management: 1–2 days.
• Full RAG pipeline: up to 5 days.

Contact us — we will assess your project in 1 hour. We guarantee transparent code and full documentation. Get a consultation right now.

—

(\text{Learn more about } \text{chain-of-thought} \text{ and } \text{Official OpenAI API docs}.)