Saijo George

Curated by Saijo George

Read more

thursday9 Jan 2020

Google Open-Sources ALBERT Natural Language Model

https://www.infoq.com

Google AI has open-sourced A Lite Bert (ALBERT), a deep-learning natural language processing (NLP) model, which uses 89% fewer parameters than the state-of-the-art BERT model, with little loss of accuracy. The model can also be scaled-up to achieve new state-of-the-art performance on NLP benchmarks.

The research team described the model in a paper to be presented at the International Conference on Learning Representations. ALBERT uses two optimizations to reduce model size: a factorization of the embedding layer and parameter-sharing across the hidden layers of the network. Combining these two approaches results in a baseline model with only 12M parameters, compared to BERT’s 108M, while achieving an average of 80.1% accuracy on several NLP benchmarks compared with BERT’s 82.3% average. The team also trained aย “double-extra-large” ALBERT model with 235M parameters which performed better on benchmarks than the “large” BERT model with 334M parameters.

Google has released a TensorFlow-based implementation of ALBERT as well as models trained on an English-language corpus and a Chinese-language corpus.ย The ALBERT code and models are available on GitHub.

General
No Media


I love tl;dr Marketing because I can get all the latest SEO news and trends in one spot without having to read lengthy articles. I really look forward to the daily emails to see what's new in our industry!

Bing Is Sending out a Bunch of These Bing Pages Approval Lately 1 - SEO News

Ryan Mews SEO Manager Merkle