Yi - AI Language Models Tool

Overview

Yi is an open-source family of bilingual (English/Chinese) large language models developed by 01.AI. According to the project's GitHub repository, the series was trained from scratch on a multi-trillion-token corpus and includes base and chat variants across multiple sizes (notably 6B, 9B and 34B parameter models), long-context variants (32K and 200K), vision-language members (Yi-VL), and quantized releases for efficient inference. The project team published a technical report on arXiv describing the family, its 3.1 trillion-token pretraining corpus, long-context extension to 200K tokens, and the data/quality engineering that the authors attribute to the models' performance (Yi: Open Foundation Models by 01.AI, arXiv:2403.04652). At release Yi-34B drew attention for strong benchmark results: the instruction-tuned Yi-34B-Chat was reported to place second on AlpacaEval (behind GPT-4 Turbo) and the Yi-34B base model ranked highly on open-model leaderboards. The code and many weights are available under Apache 2.0 from the project's GitHub and Hugging Face model pages, and a lively downstream ecosystem offers quantized variants (GPTQ/AWQ/GGUF) and hosted endpoints on third-party services such as Hugging Face, Replicate and other vendors. Typical uses include bilingual assistants, code/math tasks, long-document retrieval and multimodal (vision+text) workloads; hardware requirements scale from single high-memory GPUs for the 6B models up to multi‑GPU nodes for 34B/200K variants (see official repo and tech report for exact recommendations).

GitHub Statistics

• Stars: 7,842
• Forks: 489
• Contributors: 44
• License: Apache-2.0
• Primary Language: Jupyter Notebook
• Last Updated: 2024-11-27T09:25:55Z

Key Features

• Bilingual modeling with strong English and Chinese performance (trained on large multilingual corpus).
• Multiple sizes: consumer/edge-friendly 6B, high-capability 9B and 34B production variants.
• Long-context variants: standard 4K/32K plus explicit 200K long-context models for large documents.
• Open-source release under Apache 2.0; many weights and code available on GitHub and Hugging Face.
• Chat (instruction-tuned) and base checkpoints, with SFT chat variants optimized for conversational use.
• Quantized distributions (GPTQ / AWQ / GGUF) and community forks for lower‑cost inference.

Example Usage

Example (python):

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Example: run a small Yi model (change to '01-ai/Yi-34B' on multi-GPU / high-memory hosts)
model_id = "01-ai/Yi-6B"

tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=False)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", torch_dtype=torch.float16)

prompt = "Write a short bilingual (English + Chinese) greeting and list three recommended first steps for evaluating a new LLM." 
inputs = tokenizer(prompt, return_tensors="pt")

generated = model.generate(**{k: v.to(model.device) for k, v in inputs.items()}, max_new_tokens=180, do_sample=True, temperature=0.8)
output = tokenizer.decode(generated[0], skip_special_tokens=True)
print(output)

Benchmarks

Parameter count (Yi-34B): ≈34.4B parameters (Source: https://huggingface.co/01-ai/Yi-34B)

Pretraining corpus size: ≈3.1 trillion tokens (English + Chinese) (Source: https://arxiv.org/abs/2403.04652)

Default / extended context length: Default 4K (expandable to 32K); 200K variants available (Source: https://github.com/01-ai/Yi)

AlpacaEval (instruction/chat ranking): Yi-34B-Chat reported 2nd place (behind GPT-4 Turbo) on AlpacaEval (Jan 2024 data) (Source: https://github.com/01-ai/Yi)

Open-model leaderboard placement (at release): Yi-34B reported top-ranked among open-source models on Hugging Face/Open LLM leaderboards (Nov 2023–Jan 2024) (Source: https://www.wired.com/story/chinese-startup-01-ai-is-winning-the-open-source-ai-race)

Key Information

• Category: Language Models
• Type: AI Language Models Tool