Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Introduces the Switch Transformer, a simplified sparse mixture-of-experts model that scales to trillion parameters at constant compute cost.
Unlike standard models that reuse the same parameters for all inputs, Mixture of Experts (MoE) selects different parameters per example, giving huge, sparsely activated models at constant compute. Adoption has been limited by complexity, communication cost, and training instability, which the Switch Transformer addresses by simplifying routing and reducing overheads. New techniques tame instabilities and enable bfloat16 training. Based on T5, it delivers up to 7x faster pre-training, gains across 101 languages, and trillion-parameter models with a 4x speedup over T5-XXL.
Based on: Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity · Journal of machine learning research
Curated by Aramai Editorial
Read summary →