Together AI Integration: Turnkey Open LLM Deployment

Deploying your own GPU infrastructure for inference of modern LLMs is an expensive endeavor. A100/H100 clusters require budgets starting from millions of rubles and a team of MLOps engineers. Together AI removes this barrier: we take over the integration of your services with the platform. We connect an OpenAI-compatible API, configure routing between models, optimize p99 latency — and you get a ready-made endpoint for your tasks. With over 5 years of experience, we have completed more than 30 LLM integration projects in production — we will evaluate your project within a day.

Case study: for a fintech startup, we deployed a RAG system on Together AI in 3 days. Previously, they paid for dedicated GPU instances on AWS — Together reduced costs by 5x, p99 latency did not exceed 200 ms, and answer accuracy reached 94%. This is not an isolated example: according to the platform documentation, its use can reduce inference costs by up to 80%. Compared to renting GPU instances on AWS, Together AI provides up to 80% savings with comparable performance.

What problems does Together AI solve?

High GPU cost. Renting dedicated A100/H100 instances in the cloud results in tens of thousands of dollars monthly. Together AI uses a shared GPU pool, reducing inference cost by 5–10x compared to cloud instances. Deployment complexity: selecting CUDA version, optimizing via vLLM or TensorRT-LLM — engineering hours. The platform handles all low-level optimizations. Lack of flexibility: balancing quality and speed — we implement a router that automatically switches the model depending on the context.

Basic integration

from openai import OpenAI, AsyncOpenAI

# Together uses OpenAI SDK
client = OpenAI(
    api_key="TOGETHER_API_KEY",
    base_url="https://api.together.xyz/v1",
)

# Model selection
MODELS = {
    "quality": "meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo",
    "balanced": "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
    "fast": "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    "code": "Qwen/Qwen2.5-Coder-32B-Instruct",
    "reasoning": "deepseek-ai/DeepSeek-R1-Distill-Llama-70B",
}

response = client.chat.completions.create(
    model=MODELS["balanced"],
    messages=[{"role": "user", "content": "Task"}],
    temperature=0.1,
    max_tokens=2048,
)
print(response.choices[0].message.content)

How does model routing work?

The router analyzes the request: if the task requires deep reasoning — it routes to DeepSeek-R1, if speed is needed — to Llama 3.1 8B. This reduces average token cost by 30–40% without quality loss. Under the hood, we use dynamic thresholds based on latency and confidence.

How does fine-tuning work?

Fine-tuning open models on your own data improves accuracy and reduces hallucinations. Together AI provides infrastructure for LoRA and full fine-tuning. We help prepare the dataset, select hyperparameters, and conduct A/B testing. Example launch:

# Together allows fine-tuning open models on your own data
import together

together.api_key = "TOGETHER_API_KEY"

# Upload dataset (JSONL format: {"prompt": "...", "completion": "..."})
file_response = together.Files.upload(file="training_data.jsonl")
file_id = file_response["id"]

# Start fine-tuning
ft_response = together.Finetune.create(
    training_file=file_id,
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Reference",
    n_epochs=3,
    batch_size=16,
    learning_rate=1e-5,
    suffix="my-custom-model",
)
ft_job_id = ft_response["id"]

# Check status
status = together.Finetune.retrieve(ft_job_id)
print(status["status"])  # "running" | "completed" | "failed"

Advantages of fine-tuning on Together AI

The platform manages all compute resources — no need to think about GPUs. Pre-trained LoRA weights are available, which can be customized to your data. We help with hyperparameter selection and metric evaluation on a validation dataset. Fine-tuning pays off by increasing accuracy and reducing the number of expensive tokens.

Embeddings and RAG

For Retrieval-Augmented Generation, we use embedding models such as BAAI/bge-large-en-v1.5. Together AI supports asynchronous embedding generation, which is important under high load:

response = client.embeddings.create(
    model="BAAI/bge-large-en-v1.5",  # One of the best for search
    input=["First text", "Second text"],
)
embeddings = [item.embedding for item in response.data]

Model comparison on Together AI

How to integrate Together AI in 3 steps?

1. Get an API key from the Together AI console.
2. Replace the base URL in your OpenAI client with https://api.together.xyz/v1.
3. Configure routing — we select models for your scenarios. Get your endpoint in half a day.

Contact us for a consultation on Together AI integration.

How long does integration take?

Basic API integration takes half a day: replacing the base URL and key. A full project with fine-tuning and RAG takes from 5 working days. The cost is calculated individually based on traffic volume and model complexity. Request integration — get a ready endpoint in 1 day.

Process overview

What's included

• Connect Together AI to your project (0.5 days)
• Fine-tuning pipeline with A/B testing (3–5 days)
• Embedding integration for RAG (1–2 days)
• Cost optimization (routing, caching, batch processing)
• Documentation and team training
• Post-launch support (monitoring, tuning)

Contact us — we will evaluate your project and propose the optimal solution. We guarantee a 2–5x reduction in inference costs compared to renting GPUs. Additionally, read about LLMs — the foundation for all modern AI solutions.