Home › Language Models › Bielik-11B-v2

Bielik-11B-v2 - AI Language Models Tool

Overview

Bielik-11B-v2 is an 11-billion-parameter causal decoder language model developed by SpeakLeash in collaboration with ACK Cyfronet AGH and released under Apache-2.0. It is purpose-built for Polish-language capabilities: the project reports extensive curation of Polish corpora using an XGBoost-based quality filter and large-scale pretraining on HPC infrastructure (Athena/Helios). Although optimized for Polish, the model retains strong cross-lingual performance on English benchmarks, making it useful for bilingual generation and reasoning tasks. (Sources: Hugging Face model card; SpeakLeash release notes.) The model was initialized from Mistral-7B-v0.2 and trained with Megatron-LM and large-model parallelization. SpeakLeash publishes multiple downstream instruction-tuned checkpoints (v2.0 → v2.2+/Instruct) and several quantized builds for smaller hardware. Benchmarks on the Open PL LLM and Open LLM leaderboards place Bielik-11B-v2 among the top Polish-focused models, and instruction-tuned variants show further gains, making the Bielik family a practical choice for Polish NLP research, prototyping, and fine-tuning workflows. (Sources: Hugging Face model card; Bielik instruct pages.)

Model Statistics

  • Downloads: 682
  • Likes: 46
  • Pipeline: text-generation
  • Parameters: 11.2B

License: apache-2.0

Model Details

Architecture & training: Bielik-11B-v2 is a causal (decoder-only) transformer scaled to ~11 billion parameters and (per the model card) initialized from Mistral-7B-v0.2. Training was performed with Megatron-LM using large-model parallelization on the Helios supercomputer; the authors report use of 256 Nvidia GH200 cards. The model card documents use of curated Polish corpora plus a CommonCrawl subset. Note: the Hugging Face model card contains an inconsistency about token counts — one section states 400 billion tokens while another describes 200 billion tokens (over 700 GB of plain text) used for two epochs; both statements appear on the official model page. (Source: Hugging Face model card.) Precision, files & tooling: distributed weights are provided in safetensors with BF16 tensors; the tokenizer is from the Mistral family (project notes indicate future tokenizer work and a planned v3 that replaces the Mistral tokenizer). The team provides instruct-finetuned variants (v2.0, v2.1, v2.2 etc.) with alignment experiments (DPO-Positive, weighted-token losses, masked prompts) and many quantized builds (GGUF, GPTQ, AWQ, EXL2, MLX, FP8 variants) to support lower-resource deployments. A technical report / paper by the team documents additional training and loss-design choices and is available on arXiv. (Sources: Hugging Face model card; Bielik-11B-v2.2-Instruct page; arXiv technical report.) Recommended use: the base Bielik-11B-v2 is published as a base model intended for further fine-tuning; for immediate instruction-following/chat use, SpeakLeash recommends one of the instruct-tuned checkpoints (for example v2.2-Instruct). Standard Hugging Face transformers (AutoTokenizer, AutoModelForCausalLM) are supported for quick experimentation.

Key Features

  • Polish-optimized: trained on a curated corpus with an XGBoost quality filter for high-quality Polish texts.
  • 11B parameters: depth-scaled causal decoder initialized from Mistral-7B-v0.2.
  • Trained with Megatron-LM on large-model parallelism (reported 256 GH200 GPUs on Helios).
  • Instruction-finetuned variants (v2.0→v2.2+) with alignment experiments (DPO-Positive, weighted losses).
  • Multiple quantized builds (GGUF, GPTQ, AWQ, EXL2, MLX, FP8) for lower-resource inference.

Example Usage

Example (python):

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
import torch

model_name = "speakleash/Bielik-11B-v2"

# load tokenizer and model (use bfloat16 if your runtime supports it)
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16)

gen = pipeline("text-generation", model=model, tokenizer=tokenizer, device=0)

prompt = "Najważniejszym celem człowieka na ziemi jest"
out = gen(prompt, max_new_tokens=100, do_sample=True, top_k=50)
print(out[0]['generated_text'])

Benchmarks

Open PL LLM Leaderboard — average (Bielik-11B-v2): 58.14 (Source: https://huggingface.co/speakleash/Bielik-11B-v2)

Open LLM Leaderboard — average (Bielik-11B-v2): 65.87 (Source: https://huggingface.co/speakleash/Bielik-11B-v2)

Open PL LLM Leaderboard — average (Bielik-11B-v2.2-Instruct): 65.57 (Source: https://huggingface.co/speakleash/Bielik-11B-v2.2-Instruct)

Training hardware reported: 256 × Nvidia GH200 (Helios supercomputer) (Source: https://huggingface.co/speakleash/Bielik-11B-v2)

Model size / tensor type: ≈11B parameters, BF16 safetensors (Source: https://huggingface.co/speakleash/Bielik-11B-v2)

Downloads (recent / Hugging Face listed): Downloads last month: 677 (as listed on model page) (Source: https://huggingface.co/speakleash/Bielik-11B-v2)

Last Refreshed: 2026-03-03

Key Information

  • Category: Language Models
  • Type: AI Language Models Tool
Visit Official Website

Related Tools

Qwen2.5-7B

Qwen2.5-7B is a large language model designed for text generation, featuring improvements in coding, mathematics, instruction following, long text generation, and multilingual support. It supports context lengths up to 128K tokens and is intended for sophisticated NLP tasks.
DeepSeek-V3

Open‑weight flagship LLM (base and chat) with strong reasoning and tool‑use; runs with SGLang/LMDeploy/vLLM and permissive model license.
Llama 3

Llama 3 is an open access large language model (LLM) released by Meta, available in various configurations (8B and 70B parameters) with capabilities for fine-tuning and integrations into platforms like Hugging Face, Google Cloud, and Amazon SageMaker.
UNfilteredAI-1B

A large-scale text generation model designed for creative and unconstrained content production without traditional filtering.
Shuttle-3

Shuttle-3 is a state-of-the-art language model designed for high-quality text generation, particularly suited for complex chat, multilingual communication, and reasoning tasks. It is fine-tuned from the Qwen-2.5-72b-Instruct model and designed to emulate high-quality prose similar to Claude 3 models.
WizardLM

WizardLM is a state-of-the-art large language model designed for complex chat, multilingual tasks, reasoning, and agent functionalities. It features an AI-powered pipeline (Auto Evol-Instruct) that optimizes instruction datasets for improved performance across various domains and leverages Arena Learning for an expanded learning pool of challenging instruction data.