Fine-Tuning Qwen Models: A Technical Guide

We often encounter situations where a pre-trained LLM doesn't understand corporate terminology or makes calculation errors. Fine-tuning Qwen2.5 from Alibaba solves this — the model adapts to your domain, language, and output format. According to the Qwen2.5 technical report, the MMLU score reaches 85%, outperforming many open-source models. For instance, Qwen2.5-72B beats Llama 3.1 70B in MMLU by 2 percentage points, and on Chinese tasks by 15%. The family ranges from 0.5B to 72B parameters under Apache 2.0 (base versions) and Tongyi Qianwen License (large). Specialized variants: Qwen2.5-Coder for programming, Qwen2.5-Math for mathematics, and Qwen-VL for multimodal tasks. If you need to process long documents (contracts, scientific articles, regulations), Qwen2.5 supports up to 128K tokens context. For most production tasks, 7B or 14B are chosen, but if maximum accuracy is required — 72B. For edge devices, 0.5B and 1.5B fit.

How to Choose the Model Size for Fine-Tuning

For most production tasks, 7B or 14B suffice. 0.5B and 1.5B are for inference on devices, 72B for maximum accuracy on complex scenarios.

QLoRA uses 4-bit weight quantization, allowing fine-tuning a 7B model on a single A100 40GB. Quality drop is no more than 2% compared to full fine-tuning. Cost reduction: QLoRA instead of Full halves GPU hours per experiment on a 7B model.

Why Qwen is Convenient for Multilingual and Long Contexts

Multilingual: Qwen is trained on data with a significant share of Chinese, English, and 27 other languages. Russian is represented much better than in many western models, which is important when working with Russian-language corpora.

Long context: Qwen2.5 supports up to 128K tokens. For fine-tuning tasks with long documents (contracts, scientific articles, regulations), this is a critical advantage.

Qwen2.5-Coder: a specialized version that outperforms most open-source models of the same size on HumanEval. When fine-tuned on a corporate codebase, it gives a better start than fine-tuning a general model.

How to Prepare a Dataset for Qwen Fine-Tuning

1. Data collection: collect 500 to 2000 examples relevant to your task. For financial analysis — reports with calculations.
2. Cleaning: remove duplicates, fix typos, check format compliance.
3. Labeling: each example must contain a user-assistant pair in Qwen chat template format.
4. Validation: create a test set (10% of the dataset) for quality evaluation.

Comparison of Fine-Tuning Methods

QLoRA reduces memory requirements by 7x without critical quality loss — optimal for quick experiments.

Fine-tuning with LLaMA-Factory

LLaMA-Factory is the most convenient tool for Qwen fine-tuning, supporting the full range of methods (Full, LoRA, QLoRA, DoRA) with a unified configuration format:

# config.yaml
model_name_or_path: Qwen/Qwen2.5-7B-Instruct
method: lora
dataset: my_dataset
template: qwen
finetuning_type: lora
lora_rank: 16
lora_alpha: 32
lora_target: q_proj,v_proj
output_dir: ./qwen25-7b-finetuned
num_train_epochs: 3
per_device_train_batch_size: 4
gradient_accumulation_steps: 4
learning_rate: 2.0e-4
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true

llamafactory-cli train config.yaml

Alternatively, using swift from ModelScope (Alibaba):

swift sft \
  --model_type qwen2_5_7b_instruct \
  --dataset my_dataset \
  --train_type lora \
  --output_dir ./output

Data Format: Qwen Chat Template

Qwen2.5 uses a specific chat template with <|im_start|> and <|im_end|> tags:

<|im_start|>system
You are an assistant for financial report analysis.<|im_end|>
<|im_start|>user
Calculate EBITDA from the data: revenue 850M, COGS 420M, OpEx 180M, DA 45M<|im_end|>
<|im_start|>assistant
EBITDA = Revenue - COGS - OpEx + DA = 850 - 420 - 180 + 45 = 295M<|im_end|>

When using transformers directly, apply tokenizer.apply_chat_template() for correct formatting.

Practical Case: Financial Analysis on Qwen2.5-14B

From our practice: a client needed automatic analysis of quarterly IFRS reports with indicator extraction, ratio calculations, and anomaly flags. The dataset — 1800 examples from corporate reporting. We fine-tuned Qwen2.5-14B Instruct via QLoRA (r=32, alpha=64), 4 epochs, on 2×A100 40GB for 6 hours. Results:

• Correctness of ratio calculation: 71% → 94%
• Anomaly flag accuracy (F1): 0.67 → 0.88
• Text summary quality (human eval, 1–5): 3.1 → 4.4

Qwen2.5-14B outperformed Llama 3.1 8B by 12% in indicator extraction accuracy. MMLU and HumanEval confirm the model's competitive position. Inference cost savings: vLLM with INT4 quantization reduces cost by 40% compared to bf16, saving several hundred dollars per month at 100k requests load.

Deploying Fine-Tuned Qwen with vLLM

from vllm import LLM, SamplingParams

llm = LLM(
    model="./qwen25-14b-merged",
    dtype="bfloat16",
    tensor_parallel_size=2,  # 2 GPU
    max_model_len=32768,
    gpu_memory_utilization=0.9
)

sampling_params = SamplingParams(temperature=0.1, max_tokens=2048)
outputs = llm.generate(prompts, sampling_params)

vLLM provides continuous batching and PagedAttention, giving throughput ~240 tok/s on 2×A100 at batch size 16 — 3x higher than vanilla Transformers.

What's Included in Qwen Fine-Tuning Work

• Task analysis and requirements gathering
• Dataset preparation: cleaning, labeling, quality check
• Training configuration setup (LoRA/QLoRA, hyperparameters)
• Training and intermediate metric evaluation
• Baseline vs fine-tuned comparison on test set
• Deployment on chosen infrastructure (vLLM, Triton, SageMaker)
• Documentation and team training

Additionally: we provide access to our test benches and metrics. Assess your project — write to us. Our team has extensive experience in NLP and LLM fine-tuning, having completed over 30 fine-tuning projects.

Timelines

• Dataset preparation: 2–5 weeks
• Training (7B, QLoRA): 3–8 hours
• Training (72B, QLoRA, 4×A100): 24–72 hours
• Iterations and evaluation: 1–2 weeks
• Total: 4–8 weeks

Order end-to-end Qwen fine-tuning — get a model that understands your business. Contact us for a consultation.