Qwen2 Technical Report

Abstract

We introduce Qwen2, the next generation of the Qwen series of large language models. Qwen2 includes dense language models of 0.5B, 1.5B, 7B, 57B-A14B (MoE), and 72B parameters, trained on data in 29 languages. Qwen2-72B achieves competitive performance with leading proprietary models on a wide range of benchmarks.

Key Findings

• 1Released a family of models from 0.5B to 72B parameters
• 2Trained on data in 29 languages for strong multilingual support
• 3Included a 57B MoE model with only 14B active parameters
• 4Achieved performance competitive with leading proprietary models
• 5Released as open source with commercial-friendly licensing

Impact & Significance