Phi (Microsoft) Language Model Fine-Tuning

Fine-Tuning Phi Language Models (Microsoft)

We develop and fine-tune Microsoft Phi language models for enterprise and edge deployments. Our team works with Phi-3 and Phi-4, which match models 3–5× larger on reasoning and programming benchmarks. Microsoft Research trained Phi on high-quality synthetic data and curated academic textbooks — not raw internet text. This explains its exceptional parameter efficiency. Without domain-specific fine-tuning, a base Phi model produces 30–40% irrelevant or hallucinated answers on specialized queries. This makes it unsuitable for production use in regulated industries such as manufacturing, healthcare, or logistics. Our fine-tuning pipeline reduces hallucination rates to under 10%. It raises task accuracy from roughly 55% to over 90%. We achieve this through structured dataset preparation, QLoRA training on high-end GPUs, and rigorous evaluation before deployment. We deliver the complete engagement as a turnkey solution: dataset curation, model training, quantization, device-level testing, and integration support. Contact us to scope your project — we will provide a detailed estimate within one business day.

Phi Model Lineup

Phi-4 with 14B parameters outperforms Llama 3.1 70B on several math and programming benchmarks. This is the result of high-quality training data — synthetic examples and curated academic textbooks — as documented by Microsoft Research.

LoRA Fine-Tuning Phi-4 via Transformers + TRL

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from trl import SFTTrainer, SFTConfig
from peft import LoraConfig
import torch

model = AutoModelForCausalLM.from_pretrained(
    "microsoft/phi-4",
    quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16),
    device_map="auto",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
)

trainer = SFTTrainer(
    model=model,
    args=SFTConfig(
        output_dir="./phi4-finetuned",
        num_train_epochs=4,
        per_device_train_batch_size=4,
        gradient_accumulation_steps=4,
        learning_rate=1e-4,
        bf16=True,
        max_seq_length=8192,
    ),
    peft_config=LoraConfig(
        r=16, lora_alpha=32,
        target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
        task_type="CAUSAL_LM"
    ),
    train_dataset=dataset,
)
trainer.train()

For Phi-4, set trust_remote_code=True and use bfloat16 dtype. This ensures stable training without loss spikes. With 4-bit quantization (QLoRA), 24 GB VRAM is sufficient for Phi-4 fine-tuning.

Why Fine-Tune Phi for Edge and Mobile?

Phi-3/4-mini (3.8B) is our most common choice for mobile apps and browser extensions. After fine-tuning and quantization, the model runs entirely on-device. It requires no cloud connectivity at runtime. This cuts cloud compute costs by up to 60% compared to GPT-4 API calls. Accuracy on domain-specific tasks remains comparable.

Available quantization formats after fine-tuning:

• GGUF Q4_K_M: ~2.2 GB, runs on CPU (Apple M-series: ~12 tokens/s). Supported by llama.cpp and Ollama.
• ONNX INT4: ~2.0 GB, used in ONNX Runtime for Windows and Android.
• ExecuTorch: ~2.5 GB, for iOS and Android deployments via PyTorch Mobile.

Microsoft provides pre-built ONNX versions via microsoft/Phi-3-mini-4k-instruct-onnx. This simplifies integration into .NET and Windows applications.

Dataset Preparation for Phi Fine-Tuning

Dataset quality is the single most important factor in fine-tuning success. Our dataset preparation process includes:

• Collecting real conversations or generating synthetic question-answer pairs using a strong teacher model.
• Filtering duplicates, noisy examples, and statistical outliers.
• Balancing topic distribution — oversampling underrepresented categories to avoid model bias.
• Verifying that all answers are factually correct and consistent with your source documentation.
• Segmenting long documents into chunks of no more than 8,192 tokens to match Phi-4 training context.

Typical dataset sizes range from 500 to 5,000 pairs depending on domain complexity. We have prepared datasets for industrial maintenance, legal document review, and customer support automation.

Common Fine-Tuning Mistakes and How to Avoid Them

Several pitfalls can undermine a Phi fine-tuning project:

• Using torch.float32 causes memory overflow even on 80 GB GPU. Use bfloat16 or fp16 instead.
• Not setting max_seq_length correctly — keep it at 8,192 for datasets with short-to-medium examples to speed up training.
• Applying LoRA to all linear layers increases adapter size without quality gain. Use the target_modules from the example above.
• Forgetting trust_remote_code=True — Phi-4 will not load without this flag.

Case Study: Offline Assistant for Field Engineers

Our client required a mobile app for industrial equipment maintenance engineers. The assistant runs fully offline — there is no internet access at most industrial sites. It answers questions about maintenance procedures and helps diagnose equipment faults.

Base model: Phi-3-mini-128k-instruct (3.8B; the 128K context is necessary for long technical manuals).

Dataset: 1,400 curated pairs — documentation fragment plus engineer question, matched with the correct answer including procedure number and step-by-step instructions.

Results from our practice:

• Answer accuracy (compliance with official procedures): 58% → 86%
• Hallucination rate (invented steps not in documentation): 31% → 8%
• Model size after GGUF Q4_K_M quantization: 2.1 GB, running at 9 tokens/s on a Snapdragon 8 Gen 3 CPU

The client reduced time spent searching documentation by up to 70%. Onboarding time for new engineers also dropped significantly.

What Is Included in Our Fine-Tuning Service

We offer an end-to-end engagement delivered in full:

1. Domain analysis and dataset preparation: 2–4 weeks
2. QLoRA training on A100 GPU: up to 10 hours of compute
3. Quantization and on-device testing: 3–5 days
4. Integration into your application (REST API, SDK, ONNX Runtime)
5. Deployment documentation and one month of post-launch support

Total delivery time: 3–6 weeks, depending on dataset complexity and integration scope. We will assess your project and propose the optimal model and configuration for your use case.