Home › Language Models › OpenLLaMA

OpenLLaMA - AI Language Models Tool

Overview

OpenLLaMA is an open-source reproduction of Meta AI’s LLaMA family that provides pretrained 3B, 7B and 13B parameter models trained on ~1T tokens and released under an Apache‑2.0 license. The project publishes both PyTorch (Hugging Face Transformers) and JAX (EasyLM) weights and evaluation results; v1 models are trained on the RedPajama dataset while v2 models use a refined mixture (Falcon refined‑web, StarCoder, and selected RedPajama subsets) intended to improve code and web-text performance. ([github.com](https://github.com/openlm-research/open_llama)) OpenLLaMA is positioned as a permissively licensed, drop‑in alternative to LLaMA for researchers and engineers who want full-weight access and reproducible training artifacts (JAX training pipeline EasyLM, TPU‑v4 training, and detailed evaluation tables are provided in the repo). The README documents known tokenizer behavior (v1 tokenizer collapses multiple spaces — problematic for code — fixed in v2) and gives example code for loading models via Hugging Face Transformers. ([github.com](https://github.com/openlm-research/open_llama))

GitHub Statistics

  • Stars: 7,533
  • Forks: 407
  • Contributors: 2
  • License: Apache-2.0
  • Last Updated: 2023-07-16T08:35:31Z

Repository shows strong community interest (≈7.5k stars, ≈407 forks) and a permissive Apache‑2.0 license; most public activity (model releases, tokenizer fixes) occurred in mid‑2023. The project provides both checkpoints and evaluation artifacts but has a small core author list (few contributors) and limited ongoing commits since the 2023 release cycle; issues and discussions contain user reports and usage questions (tokenizer, model conversion, inference). ([github.com](https://github.com/openlm-research/open_llama))

Installation

Install via pip:

git clone https://github.com/openlm-research/open_llama.git
pip install -U transformers accelerate safetensors torch
python -c "from transformers import LlamaTokenizer, LlamaForCausalLM; model_path='openlm-research/open_llama_3b_v2'; tokenizer=LlamaTokenizer.from_pretrained(model_path); model=LlamaForCausalLM.from_pretrained(model_path, torch_dtype='torch.float16', device_map='auto'); print('loaded')"
Note: avoid Hugging Face fast tokenizer for some OpenLLaMA checkpoints; use LlamaTokenizer or AutoTokenizer(..., use_fast=False) per project guidance. ([github.com](https://github.com/openlm-research/open_llama))

Key Features

  • Pretrained model sizes: 3B, 7B and 13B parameter checkpoints (v1 and v2 variants). ([github.com](https://github.com/openlm-research/open_llama))
  • Dual weight formats: PyTorch (Transformers) and JAX (EasyLM) for inference and training. ([github.com](https://github.com/openlm-research/open_llama))
  • Permissive Apache‑2.0 license — suitable for research and many commercial use cases. ([github.com](https://github.com/openlm-research/open_llama))
  • v2 data mixture includes Falcon refined‑web, StarCoder, and RedPajama subsets (improved code/web performance). ([huggingface.co](https://huggingface.co/openlm-research/open_llama_3b_v2))
  • Tokenizer notes: v1 merged multiple spaces (bad for code); v2 addresses code tokenization issues. ([github.com](https://github.com/openlm-research/open_llama))

Community

OpenLLaMA attracted rapid attention after release (thousands of GitHub stars and Hugging Face downloads) and active early discussion about tokenizer behavior and model conversions. Community threads and issues document practical guidance (use_fast=False, model loading examples) and experiments (fine‑tuning, quantization, Vicuna-style instruction tuning). Maintainer activity and releases peaked during mid‑2023; the repo remains a popular reference weight source, with much of the real‑world usage and forks happening on Hugging Face and third‑party tooling. ([github.com](https://github.com/openlm-research/open_llama))

Last Refreshed: 2026-03-03

Key Information

  • Category: Language Models
  • Type: AI Language Models Tool
Visit Official Website

Related Tools

Qwen2.5-7B

Qwen2.5-7B is a large language model designed for text generation, featuring improvements in coding, mathematics, instruction following, long text generation, and multilingual support. It supports context lengths up to 128K tokens and is intended for sophisticated NLP tasks.
DeepSeek-V3

Open‑weight flagship LLM (base and chat) with strong reasoning and tool‑use; runs with SGLang/LMDeploy/vLLM and permissive model license.
Llama 3

Llama 3 is an open access large language model (LLM) released by Meta, available in various configurations (8B and 70B parameters) with capabilities for fine-tuning and integrations into platforms like Hugging Face, Google Cloud, and Amazon SageMaker.
UNfilteredAI-1B

A large-scale text generation model designed for creative and unconstrained content production without traditional filtering.
Shuttle-3

Shuttle-3 is a state-of-the-art language model designed for high-quality text generation, particularly suited for complex chat, multilingual communication, and reasoning tasks. It is fine-tuned from the Qwen-2.5-72b-Instruct model and designed to emulate high-quality prose similar to Claude 3 models.
WizardLM

WizardLM is a state-of-the-art large language model designed for complex chat, multilingual tasks, reasoning, and agent functionalities. It features an AI-powered pipeline (Auto Evol-Instruct) that optimizes instruction datasets for improved performance across various domains and leverages Arena Learning for an expanded learning pool of challenging instruction data.