DeepSeek-V2-Lite - AI Language Models Tool

Overview

DeepSeek-V2-Lite is an open-weight Mixture-of-Experts (MoE) language model optimized for economical training and efficient single-GPU inference. The Lite variant is a 16B-parameter family member (15.7B reported model parameters) that activates about 2.4B parameters per token using sparse MoE routing, enabling stronger capability than many dense models of similar footprint while keeping inference cost low. The model supports both text and chat completion formats and is available on Hugging Face with a dedicated chat SFT release. Technically targeted at research and production teams that need high capability with constrained hardware, DeepSeek-V2-Lite was trained from scratch on a large pretraining corpus (5.7T tokens reported) and uses Multi-head Latent Attention (MLA) plus the DeepSeekMoE FFN design. The authors highlight that the model can run in BF16 on a single 40GB GPU for inference and that a 32k token context length is supported in the published release. For implementation and usage details the Hugging Face model card and the project repository include run examples, evaluation tables and a chat template. ([huggingface.co](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat))

Model Statistics

• Downloads: 120,961
• Likes: 155
• Pipeline: text-generation
• Parameters: 15.7B

License: other

Model Details

Architecture: DeepSeek-V2-Lite is an MoE transformer that combines Multi-head Latent Attention (MLA) to compress KV caches with DeepSeekMoE feed‑forward layers. The Lite configuration uses 27 layers, a hidden dimension of 2048, 16 attention heads (per-head dims described in the model card), and a KV-compression dimension of 512. Most FFN layers are replaced by MoE layers that include a mixture of shared and routed experts. The model card reports 2 shared experts + 64 routed experts per MoE layer, with 6 experts activated per token, yielding ~15.7B total parameters and ~2.4B activated parameters per token. ([huggingface.co](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat)) Training & deployment: DeepSeek-V2-Lite was trained from scratch with AdamW (detailed lr schedule and clipping in the repo) and a maximum training sequence length of 4k during pretraining, followed by long-context extension and SFT to produce a chat variant. The released weights are configured to run with bfloat16 tensors; the model card recommends one 40GB GPU for BF16 inference and suggests vLLM for improved throughput. Support for Hugging Face Transformers, vLLM and LangChain compatibility is described in the model repository. ([huggingface.co](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat))

Key Features

• Mixture-of-Experts: sparse FFNs activate ~2.4B parameters per token for cost-effective compute.
• Multi-head Latent Attention (MLA): compresses KV cache to reduce inference memory and bandwidth.
• Compact runtime: configured to run BF16 inference on a single 40GB GPU.
• Long context: published variants support very long context windows (32k reported for Lite).
• Open-weight release: base and chat weights published on Hugging Face with usage notes and templates.

Example Usage

Example (python):

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig

model_name = "deepseek-ai/DeepSeek-V2-Lite-Chat"
# trust_remote_code is required for this repository's custom model/tokenizer code
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16
).cuda()

# use the model's published generation config
model.generation_config = GenerationConfig.from_pretrained(model_name)
model.generation_config.pad_token_id = model.generation_config.eos_token_id

messages = [{"role": "user", "content": "Write a short Python function that computes Fibonacci numbers."}]
input_tensor = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(input_tensor.to(model.device), max_new_tokens=128)
# decode only the generated portion
result = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True)
print(result)
# Example usage and vLLM guidance are provided on the Hugging Face model card. ([huggingface.co](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat))

Pricing

Model weights: DeepSeek-V2-Lite weights are published on Hugging Face and downloadable under the model's stated license; the weights themselves can be used locally (no model-download charge). Hosted/API pricing: DeepSeek operates a hosted API with separate, evolving token-based pricing. News coverage reported very low introductory prices (for example ~RMB2 per million output tokens reported by Financial Times after launch), and later vendor/API pages documented updated token rates and promotional-period changes; consult DeepSeek's official API docs or provider pages for current rates before production use. ([huggingface.co](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat))

Benchmarks

MMLU (Base, English): 58.3 (DeepSeek-V2-Lite, base reported) (Source: https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat)

MMLU (Chat, English): 55.7 (DeepSeek-V2-Lite Chat, SFT reported) (Source: https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat)

HumanEval (Chat): 57.3 (DeepSeek-V2-Lite Chat, reported) (Source: https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat)

GSM8K (Chat): 72.0 (DeepSeek-V2-Lite Chat, reported) (Source: https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat)

Model size (total parameters): ≈15.7B total parameters; ~2.4B activated per token (Source: https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat)

Key Information

• Category: Language Models
• Type: AI Language Models Tool