Llama 2: Open Foundation and Fine-Tuned Chat Models

Abstract

We develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases. Our models outperform existing open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for some closed-source models.

Key Findings

• 1Released open-source LLMs from 7B to 70B parameters competitive with closed models
• 2Fine-tuned chat models using RLHF with over 1 million human annotations
• 3Demonstrated that open-source models can approach closed-source quality
• 4Included extensive safety evaluations and red-teaming
• 5Released with a permissive license enabling commercial use

Impact & Significance