Llama 3 - AI Language Models Tool

Overview

Llama 3 is Meta’s April 18, 2024 release in the Llama family: open-access autoregressive LLMs offered in 8B and 70B parameter sizes, with both pretrained (base) and instruction‑tuned (Instruct) variants tuned for assistant-style dialogue. The release expanded the tokenizer vocabulary to 128,256 tokens, raised the pretraining mixture to over 15 trillion tokens, and standardized an 8,192-token context window to support multi-document and long‑context use cases. ([huggingface.co](https://huggingface.co/blog/llama3?utm_source=openai)) Llama 3 emphasizes deployability and ecosystem integrations: Hugging Face published model cards, Transformers support (including safetensors and quantized 4‑bit/8‑bit loading via bitsandbytes and PEFT), and one‑click inference deployments to Hugging Face Inference Endpoints, Google Cloud Vertex AI, and Amazon SageMaker. Meta also released safety toolkits (Llama Guard / Llama Guard 2 and Code Shield) and a Community License that permits redistribution and fine‑tuning but requires explicit attribution (“Built with Meta Llama 3”) for derivatives. Community testing notes generally praise improved helpfulness and fewer false refusals versus prior Llama versions, while users still report the usual limitations (hallucinations, English‑centric performance, and hardware costs for larger models). ([huggingface.co](https://huggingface.co/blog/llama3?utm_source=openai))

Key Features

• Available as 8B and 70B parameter models with base and instruction‑tuned variants.
• 8,192‑token context window for long‑document summarization and multi‑document QA.
• Tokenizer expanded to 128,256 vocabulary tokens for more efficient multilingual encoding.
• Trained on a reported 15+ trillion tokens (public sources) with extensive instruction fine‑tuning.
• Supports 4‑bit/8‑bit quantized loading (bitsandbytes) and PEFT for low‑cost fine‑tuning.
• Ecosystem integrations: Hugging Face, Google Cloud Vertex AI, Amazon SageMaker, and Inference Endpoints.

Example Usage

Example (python):

# Example: load Llama 3 8B-instruct with transformers and 4-bit quantization (requires transformers>=4.40, bitsandbytes)
# Source/usage pattern based on Hugging Face Llama 3 blog and model cards: https://huggingface.co/blog/llama3

from transformers import pipeline
import torch

model_id = "meta-llama/Meta-Llama-3-8B-Instruct"

pipe = pipeline(
    "text-generation",
    model=model_id,
    device_map="auto",
    torch_dtype=torch.float16,
    model_kwargs={
        "quantization_config": {"load_in_4bit": True},
        "low_cpu_mem_usage": True,
    },
)

prompt = "Summarize the key differences between Llama 3 and Llama 2 in two short paragraphs."
print(pipe(prompt, max_new_tokens=300)[0]["generated_text"])

Benchmarks

MMLU (5-shot, Llama 3 70B): 79.5 (Source: https://huggingface.co/meta-llama/Meta-Llama-3-70B)

MMLU (5-shot, Llama 3 8B): 66.6 (Source: https://huggingface.co/meta-llama/Meta-Llama-3-70B)

Pretraining data volume: 15+ trillion tokens (reported) (Source: https://huggingface.co/blog/llama3)

Context window: 8,192 tokens (Source: https://huggingface.co/blog/llama3)

Tokenizer vocabulary size: 128,256 tokens (Source: https://huggingface.co/blog/llama3)

Key Information

• Category: Language Models
• Type: AI Language Models Tool