Home › Language Models › DeepCoder-14B-Preview

DeepCoder-14B-Preview - AI Language Models Tool

Overview

DeepCoder-14B-Preview is a 14B-parameter code-reasoning large language model fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-14B to improve long-context program synthesis and code reasoning. The preview release was trained with a distributed reinforcement-learning recipe (an enhanced GRPO, called GRPO+) together with iterative context-lengthening so the model can generalize to inference contexts far beyond its training chunk size. According to the model card on Hugging Face, DeepCoder-14B-Preview reaches 60.6% Pass@1 on LiveCodeBench v5 when evaluated at long context lengths and reports competitive Codeforces and HumanEval+ performance, despite having far fewer parameters than some proprietary models. (Source: Hugging Face model card.) The model is oriented toward multi-file and long-function generation, automated problem-solving with many test cases, and long-context code reasoning (evaluated up to 64K tokens). The authors publish usage recommendations (for example: temperature ≈ 0.6, top_p ≈ 0.95 and large max_tokens for long outputs) and provide guidance for serving on vLLM, Hugging Face Text Generation Inference, TensorRT-LLM and other inference engines. DeepCoder-14B-Preview is released under an MIT license and the model card and training/evaluation details are publicly available for reproducibility and inspection. (Source: Hugging Face model card.)

Model Statistics

Downloads: 808
Likes: 681
Pipeline: text-generation
Parameters: 14.8B

License: mit

Model Details

Architecture and base model: DeepCoder-14B-Preview is a 14B-class causal transformer fine-tuned from DeepSeek-R1-Distill-Qwen-14B (the model card reports ~14.8B parameters and a text-generation pipeline). Training recipe: the authors use an enhanced distributed RL approach named GRPO+ (variants described in the model card) and iterative context lengthening (training schedules that progress context windows, e.g., 16K → 32K) so the model learns to reason over longer contexts and generalizes at inference to 64K tokens. Data: the fine-tuning dataset is composed of ~24K unique problem–test pairs drawn from sources such as Taco-Verified, PrimeIntellect SYNTHETIC-1 and LiveCodeBench v5 (as reported on the model card). Evaluation: the model card lists benchmark results including LiveCodeBench v5 Pass@1 (60.6% at 64K context), Codeforces estimated rating (~1936) and HumanEval+ scores (~92.6). Serving & inference: authors recommend serving with high-performance backends (vLLM, TGI, TensorRT-LLM) and using settings that permit long outputs (max_tokens large, temperature 0.6, top_p 0.95). License: MIT (per the Hugging Face model page). (Source: Hugging Face model card.)

Key Features

Fine-tuned for long-context code reasoning and multi-file program generation.
GRPO+ distributed reinforcement-learning recipe for stable policy optimization.
Iterative context lengthening training (16K→32K) with generalization to 64K inference.
60.6% Pass@1 on LiveCodeBench v5 (reported on model card).
Released under an MIT license for unrestricted use and experimentation.

Example Usage

Example (python):

from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM

# Example: simple local generation using transformers (may require large GPU memory).
# For production/long-context serving, use vLLM, TGI, or optimized runtimes.
model_id = "agentica-org/DeepCoder-14B-Preview"

# Load tokenizer and model (device_map='auto' requires accelerate & compatible hardware)
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto",
    trust_remote_code=True
)

gen = pipeline("text-generation", model=model, tokenizer=tokenizer)

prompt = """
# Problem: Given an integer n, return a list of all prime factors of n in ascending order.
# Provide working Python function and a short test.
"""

output = gen(prompt, max_new_tokens=256, temperature=0.6, top_p=0.95)
print(output[0]["generated_text"])