Google Open-Sources AI Language Model Switch Transformer

Researchers at Google Brain have open-sourced the Switch Transformer, a natural-language processing (NLP) AI model. The model scales up to 1.6T parameters (roughly 6x bigger than GPT-3) and improves training time up to 7x compared to the T5 NLP model, with comparable accuracy.