Home › Language Models › DeepSeek-MoE

DeepSeek-MoE

DeepSeek-MoE 16B is a Mixture-of-Experts (MoE) language model featuring 16.4B parameters. It employs fine-grained expert segmentation and shared experts isolation to achieve comparable performance to larger models with only around 40% of the typical computations. The repository includes both base and chat variants along with evaluation benchmarks and integration instructions via Hugging Face Transformers.

Key Information

  • Category: Language Models
  • Source: Github
  • Tags: Python
  • Last updated: March 03, 2026

Structured Metrics

No structured metrics captured yet.

Links

Visit official website

Canonical source: https://github.com/deepseek-ai/DeepSeek-MoE

Related Tools

AM-Thinking-v1

AM-Thinking-v1 is a 32B dense language model built on Qwen 2.5-32B-Base, designed to enhance reasoning capabilities. It uses a post-training pipeline that includes supervised fine-tuning and dual-stage reinforcement learning, enabling strong performance in reasoning tasks like code generation, logic, and writing while operating efficiently on a single GPU.
Aria

A multimodal AI model that combines vision, language, and coding tasks, designed to deliver state-of-the-art performance across diverse tasks.
BERT base uncased

A 110M-parameter encoder-only Transformer (BERT) pretrained on BookCorpus and English Wikipedia using masked language modeling and next sentence prediction. Provides bidirectional text representations for downstream tasks like classification, token classification, and question answering. Available on Hugging Face with Apache-2.0 license and ready-to-use across Transformers (PyTorch, TensorFlow, JAX) with export options (ONNX, Core ML).
BGE-M3

BGE-M3 is a versatile embedding model from the Beijing Academy of Artificial Intelligence that supports dense retrieval, multi-vector retrieval, and sparse retrieval for text embeddings. It is designed to work in over 100 languages and can handle inputs ranging from short sentences to long documents of up to 8192 tokens.
Bielik-11B-v2

Bielik-11B-v2 is an 11-billion parameter generative text model trained on Polish text corpora. Initialized from Mistral-7B-v0.2 and fine-tuned using advanced parallelization techniques, it offers robust text generation capabilities in Polish and English, as evidenced by its performance on multiple NLP leaderboards.
BLOOM

BLOOM is a multilingual large language model with 176 billion parameters developed by the BigScience project. It generates text in 46 natural languages and 13 programming languages, and is designed for research and deployment under a Responsible AI License. The release includes access to intermediary checkpoints, optimizer states, and is integrated into the Hugging Face ecosystem.