Home › Language Models › DeepSeek-V2

DeepSeek-V2 - AI Language Models Tool

Overview

DeepSeek-V2 is an open-source Mixture-of-Experts (MoE) language model designed to deliver near–state-of-the-art capability while minimizing training and inference cost. The model has a total of 236 billion parameters but uses sparse activation so only ~21 billion parameters are active per token, enabling high throughput and much smaller runtime KV-cache footprints. DeepSeek-V2 was pretrained on a large multi-source corpus (reported at ~8.1 trillion tokens) and then improved via supervised fine-tuning (SFT) and reinforcement learning (RL), producing both base and chat variants with long-context support (up to 128k tokens). ([huggingface.co](https://huggingface.co/deepseek-ai/DeepSeek-V2?utm_source=openai)) Practically, DeepSeek-V2 is intended for large-scale generation, long-context reasoning, and code tasks where cost and latency matter. The project provides model checkpoints on Hugging Face (MIT-style model license for the repository; chat and RL-tuned variants available) and recommends specialized runtimes (vLLM/vllm-optimized stacks) for best inference efficiency; local inference typically requires many high-memory GPUs (e.g., multiple 80GB GPUs). The model attracted broad developer interest and press coverage for its cost-efficiency claims and open-source availability. ([huggingface.co](https://huggingface.co/deepseek-ai/DeepSeek-V2?utm_source=openai))

Model Statistics

Downloads: 7,230
Likes: 333
Pipeline: text-generation
Parameters: 235.7B

License: other

Model Details

Architecture and sparsity: DeepSeek-V2 uses a DeepSeekMoE sparse feed-forward design (MoE router + experts) combined with Multi-head Latent Attention (MLA), which compresses key-value state to reduce KV-cache size and speed up long-context inference. The published model card and paper state 236B total parameters with ~21B activated per token and support for 128K token context windows. Training and data: The authors report pretraining on a multi-source dataset of ~8.1 trillion tokens followed by SFT and RL tuning for chat variants. Efficiency claims: DeepSeek-V2 reportedly saves ~42.5% in training costs versus the previous 67B dense model, reduces the KV cache by ~93.3%, and increases max generation throughput by ~5.76× (per the model card and technical note). Deployment notes: Hugging Face-hosted checkpoints are BF16; the project recommends using vLLM or DeepSeek’s optimized runtime for production inference. License and usage: the repository indicates a permissive model license and notes commercial use is supported; hosted API plans are available separately via DeepSeek’s platform. ([huggingface.co](https://huggingface.co/deepseek-ai/DeepSeek-V2?utm_source=openai))

Key Features

Mixture-of-Experts design: 236B total params with ~21B active per token.
Multi-head Latent Attention (MLA) reduces KV cache size for long-context inference.
Supports very long context windows (up to 128k tokens).
Pretrained on large multi-source corpus (~8.1T tokens) with SFT and RL tuning.
Chat and RL-tuned variants optimized for instruction following and code generation.

Example Usage

Example (python):

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# NOTE: local inference for full BF16 DeepSeek-V2 requires large multi-GPU setups
# (model card recommends many 80GB GPUs or vLLM for efficient serving).
model_id = 'deepseek-ai/DeepSeek-V2'

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map='auto')

prompt = 'Write a clear Python function that computes the Levenshtein distance.'
inputs = tokenizer(prompt, return_tensors='pt').to(model.device)
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

# For production/high-throughput inference use vLLM or DeepSeek's recommended runtimes.
# See the model card and DeepSeek docs for runtime recommendations and quantized builds. ([huggingface.co](https://huggingface.co/deepseek-ai/DeepSeek-V2?utm_source=openai))

Pricing

The DeepSeek-V2 model checkpoints are published openly and the model card indicates permissive model licensing for commercial use; the checkpoint itself can be downloaded from Hugging Face for local use (local inference requires large GPU capacity). DeepSeek also offers hosted API access and usage-based pricing for its hosted models (examples from DeepSeek's API docs show per‑1M-token rates for recent hosted models such as DeepSeek‑V3.2: $0.028 per 1M input tokens (cache hit), $0.28 per 1M input tokens (cache miss), and $0.42 per 1M output tokens); confirm current rates on DeepSeek’s official API pricing page before purchasing. ([huggingface.co](https://huggingface.co/deepseek-ai/DeepSeek-V2?utm_source=openai))