Qwen3-0.6B

Summary

• Introduction
• Model Overview
• Parameters
• Usage for developers
• Reference
• License
• Citation

Introduction

Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support, with the following key features:

• Uniquely support of seamless switching between thinking mode (for complex logical reasoning, math, and coding) and non-thinking mode (for efficient, general-purpose dialogue) within single model, ensuring optimal performance across various scenarios.
• Significantly enhancement in its reasoning capabilities, surpassing previous QwQ (in thinking mode) and Qwen2.5 instruct models (in non-thinking mode) on mathematics, code generation, and commonsense logical reasoning.
• Superior human preference alignment, excelling in creative writing, role-playing, multi-turn dialogues, and instruction following, to deliver a more natural, engaging, and immersive conversational experience.
• Expertise in agent capabilities, enabling precise integration with external tools in both thinking and unthinking modes and achieving leading performance among open-source models in complex agent-based tasks.
• Support of 100+ languages and dialects with strong capabilities for multilingual instruction following and translation.

Model Overview

Qwen3-0.6B has the following features:

• Type: Causal Language Models
• Training Stage: Pretraining & Post-training
• Number of Parameters: 0.6B
• Number of Paramaters (Non-Embedding): 0.44B
• Number of Layers: 28
• Number of Attention Heads (GQA): 16 for Q and 8 for KV
• Context Length: 32,768

Parameters

Inputs

• prompt - (text): User-provided input can be a question, a request, or the context to be addressed. The system uses the prompt to analyze, infer, and generate a corresponding response.

Output

• thinking_content - (text): The content generated before the special token represents the model's internal reasoning process when enable_thinking=True.
• content - (text): The content generated after the token is the final answer that the model returns to the user.

Usage for developers

Please find below the details to track the information and access the code for processing the model on our platform.

Requirements

pip install -r requirements.txt

Code based on AIOZ structure

import os
from transformers import AutoModelForCausalLM, AutoTokenizer

...

    
def do_ai_task(
        prompt: Union[str, str],
        model_storage_directory: Union[str, Path],
        device: Literal["cpu", "cuda", "gpu"] = "cpu",
        *args, **kwargs) -> Any:
    """Define AI task: load model, pre-process, post-process, etc ..."""
    # Define AI task workflow. Below is an example
    text = "This is the AI task result"

    model_weights_path = os.path.join(model_storage_directory, "Qwen3-0.6B")

    tokenizer = AutoTokenizer.from_pretrained(model_weights_path)
    model = AutoModelForCausalLM.from_pretrained(
        model_weights_path,
        torch_dtype="auto",
        device_map="auto"
    )
    ...

    return thinking_content, content

Reference

This repository is based on and inspired by Qwen's work. We sincerely appreciate their generosity in sharing the code.

License

We respect and comply with the terms of the author's license cited in the Reference section.

Citation

@misc{qwen3technicalreport,
      title={Qwen3 Technical Report}, 
      author={Qwen Team},
      year={2025},
      eprint={2505.09388},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.09388}, 
}