Home › Language Models › Phi-3-mini-4k-instruct

Phi-3-mini-4k-instruct - AI Language Models Tool

Overview

Phi-3-mini-4k-instruct is a lightweight, instruction‑tuned language model from Microsoft in the Phi‑3 family that targets low‑latency and memory‑constrained deployment while preserving strong reasoning and multi‑turn conversation abilities. The model is a 3.8B‑parameter, dense decoder‑only Transformer tuned with supervised fine‑tuning (SFT) and Direct Preference Optimization (DPO) for improved instruction following, structured output (JSON/XML), and safety behavior. It ships in a 4K‑token context variant (with a 128K long‑context variant available in the Phi‑3 family) and is published under an MIT license for broad commercial and research use. ([huggingface.co](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)) Practical deployment options include standard Hugging Face Transformers support (sample code and chat formatting provided), ONNX Runtime and DirectML accelerated builds for Windows/GPU/mobile, and GGUF/llama.cpp friendly builds for local or edge inference. Microsoft reports extensive optimization and quantized/ONNX variants to reduce memory and latency while retaining quality, making the model suitable for on‑device assistants, latency‑sensitive servers, code/math reasoning helpers, and Retrieval‑Augmented Generation (RAG) use cases. The model card and technical report provide training details (trillions of tokens of mixed data, reported training infrastructure) and benchmark improvements versus earlier Phi‑3 mini releases. ([huggingface.co](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct))

Model Statistics

Downloads: 1,407,975
Likes: 1366
Pipeline: text-generation
Parameters: 3.8B

License: mit

Model Details

Architecture and scale: Phi‑3‑mini‑4k‑instruct is a 3.8 billion parameter dense decoder‑only Transformer, optimized for instruction following and chat‑style prompts. The tokenizer supports a vocabulary of 32,064 tokens and the model defaults to a 4,096 token context length; a 128K context variant exists elsewhere in the Phi‑3 family. ([huggingface.co](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)) Training and fine‑tuning: Microsoft reports large‑scale pretraining using a mixture of filtered public text and synthetic “textbook‑like” data, followed by supervised fine‑tuning (SFT) and Direct Preference Optimization (DPO) to align outputs to human preferences and improve structure and reasoning. Published training metadata cites multi‑petabyte token counts (trillions of tokens), training on large H100 clusters, and release timing in mid‑2024. ([huggingface.co](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)) Optimizations and formats: Microsoft provides multiple inference‑ready builds — ONNX Runtime builds (int4, fp16) with DirectML and CUDA support, GGUF quantized files for local use, and transformer checkpoints compatible with Transformers library (tested with specific versions). ONNX/ORT builds are targeted at CPU, desktop GPU, and mobile acceleration to enable low‑latency inference on diverse hardware. ([huggingface.co](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx?utm_source=openai)) Operational notes: The model is best used with chat‑format prompts (system/user/assistant tags), supports flash attention in PyTorch builds, and is released under the MIT license. Users should apply standard RAG, verification, and safety mitigations for high‑risk or high‑accuracy domains. ([huggingface.co](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct))

Key Features

3.8B parameter dense decoder model tuned for instruction following and chat.
Official 4K token context variant; Phi‑3 family includes long‑context 128K variants.
Fine‑tuned with SFT and Direct Preference Optimization (DPO) for better alignment.
Multiple inference formats: Transformers checkpoint, ONNX (int4/fp16), and GGUF quantized builds.
Tokenizer vocab size ~32,064 tokens; chat prompt template and JSON/XML structure guidance.

Example Usage

Example (python):

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

# Example: run the instruction‑tuned Phi‑3 mini 4K model on GPU
model_id = "microsoft/Phi-3-mini-4k-instruct"

torch.random.manual_seed(0)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",            # or "cuda" for single GPU
    torch_dtype="auto",
    trust_remote_code=True,
)

tokenizer = AutoTokenizer.from_pretrained(model_id)

# Prepare chat‑style messages (system/user/assistant format supported by the model)
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain Newton's second law in one paragraph."},
]

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
output = pipe(messages, max_new_tokens=200, temperature=0.0, do_sample=False)
print(output[0]["generated_text"])

Pricing

The model is published under an MIT license and available for free download from Hugging Face and Microsoft resources. There is no model purchase price; deployment on cloud providers (Azure, etc.) may incur standard compute costs.