Toolformer: Language Models Can Teach Themselves to Use Tools

Abstract

We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction. Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks, often competitive with much larger models, without sacrificing its core language modeling abilities.

Key Findings

• 1Demonstrated that LLMs can learn to use tools through self-supervised training
• 2Showed tool use (calculator, search, calendar, translator) without human annotation
• 3Achieved improved factuality and mathematical reasoning through tool integration
• 4Used a self-supervised approach where the model decides when and how to call APIs
• 5Maintained general language modeling abilities while adding tool competencies

Impact & Significance