Qwen3-Coder-Next

Summary

• Introduction
• Model Overview
• Parameters
• Usage for Developers
• Reference
• License
• Citation

Introduction

The Qwen team has released Qwen3-Coder-Next, an open-weight language model purpose-built for coding agents and local development. It introduces the following key enhancements:

• High Efficiency with Strong Performance: With only 3B activated parameters out of 80B total, the model delivers performance comparable to systems with 10–20× more active parameters, making it highly cost-effective for agent deployment.
• Advanced Agentic Capabilities: Through a carefully designed training recipe, the model excels at long-horizon reasoning, complex tool usage, and recovery from execution failures, ensuring robust behavior in dynamic coding tasks.
• Seamless Integration with Real-World IDEs: Its 262,144-token native context length and adaptability to a variety of scaffold templates enable smooth integration with popular CLI/IDE platforms (e.g., Claude Code, Qwen Code, Qoder, Kilo, Trae, Cline), supporting a wide range of development environments.

Model Overview

Qwen3-Coder-Next has the following characteristics:

• Type: Causal Language Model
• Training Stage: Pretraining & Post-training
• Number of Parameters: 80B total, 3B activated
• Number of Parameters (Non-Embedding): 79B
• Hidden Dimension: 2,048
• Number of Layers: 48
  • Hybrid Layout: 12 × (3 × (Gated DeltaNet → MoE) → 1 × (Gated Attention → MoE))
• Gated Attention:
  • Number of Attention Heads: 16 for Q and 2 for KV
  • Head Dimension: 256
  • Rotary Position Embedding Dimension: 64
• Gated DeltaNet:
  • Number of Linear Attention Heads: 32 for V and 16 for QK
  • Head Dimension: 128
• Mixture of Experts:
  • Number of Experts: 512
  • Number of Activated Experts: 10
  • Number of Shared Experts: 1
  • Expert Intermediate Dimension: 512
• Context Length: 262,144 tokens natively

Note: This model supports only non-thinking mode and does not generate <think></think> blocks in its output. Specifying enable_thinking=False is no longer required.

Parameters

Input

• prompt (text): A user-provided input, which can be a question, a request, or context to be addressed. The model uses the prompt to analyze, infer, and generate a corresponding response.

Output

• output (text): The generated text returned by the model as its final response to the user.

Usage for Developers

The following details describe how to access and run the model on our platform.

Requirements

pip install -r requirements.txt

Code based on AIOZ structure

from transformers import AutoModelForCausalLM, AutoTokenizer


...

def do_ai_task(
        prompt: Union[str, Path],
        model_storage_directory: Union[str, Path],
        device: Literal["cpu", "cuda", "gpu"] = "cpu",
        *args, **kwargs) -> Any:
    """Define AI task: load model, pre-process, post-process, etc."""
    # Define AI task workflow. Below is an example.
    model_name = os.path.join(model_storage_directory, "Qwen3-Coder-Next")

    # Load the tokenizer and the model
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = AutoModelForCausalLM.from_pretrained(
        model_name,
        torch_dtype="auto",
        device_map="auto"
    )

    ...

    output_text = tokenizer.decode(output_ids, skip_special_tokens=True)
    return output_text

Reference

This repository is based on and inspired by Qwen's work. We sincerely appreciate their generosity in sharing their model and code.

License

We respect and comply with the terms of the original author's license cited in the Reference section.

Citation

@techreport{qwen_qwen3_coder_next_tech_report,
  title        = {Qwen3-Coder-Next Technical Report},
  author       = {{Qwen Team}},
  url          = {https://github.com/QwenLM/Qwen3-Coder/blob/main/qwen3_coder_next_tech_report.pdf},
  note         = {Accessed: 2026-02-03}
}