Meta on Tuesday announced the release of Llama 3.1, the latest version of its large language model that the company claims now rivals competitors from OpenAI and Anthropic. The new model comes just three months after Meta launched Llama 3 by integrating it into Meta AI, a chatbot that now lives in Facebook, Messenger, Instagram and WhatsApp and also powers the company’s smart glasses. In the interim, OpenAI and Anthropic already released new versions of their own AI models, a sign that Silicon Valley’s AI arms race isn’t slowing down any time soon.
Meta said that the new model, called Llama 3.1 405B, is the first openly available model that can compete against rivals in general knowledge, math skills and translating across multiple languages. The model was trained on more than 16,000 NVIDIA H100 GPUs, currently the fastest available chips that cost roughly $25,000 each, and can beat rivals on over 150 benchmarks, Meta claimed.
The “405B” stands for 405 billion parameters, which are internal variables that an AI model uses to reason and make decisions. The higher the number of parameters an AI model has, the smarter we perceive it to be. OpenAI’s GPT-4 model, by comparison, reportedly has roughly 1.5 trillion parameters, although the company has not disclosed the number so far. In addition, Meta also released upgraded versions of existing Llama models that contain 70 billion and 8 billion parameters each, claiming that the newer versions had stronger reasoning abilities among other things.
Developers can download Llama 3.1 from its official website, while regular users can play with it through Meta AI in WhatsApp or on meta.ai, the company’s website for its chatbot. “Llama 405B’s improved reasoning capabilities make it possible for Meta AI to understand and answer your more complex questions, especially on the topics of math and coding,” Meta’s blog post states. “You can get help on your math homework with step-by-step explanations and feedback, write code faster with debugging support and optimization.” (Editor’s note: Engadget will pit Llama 3.1 against the New York Times Spelling Bee and report back to you). For now, Meta AI on Facebook, Messenger and Instagram are still restricted to the smaller version of Llama 3.1 that uses 70 billion parameters.
Unlike OpenAI, Google, Microsoft and Anthropic that keep their AI models proprietary, Meta’s AI models are open source, which means that anyone can modify and use them for free without sharing personal data with Meta. In a letter published on Tuesday, Meta CEO Mark Zuckerberg argued that an open source approach to AI development will ensure wider access to the technology’s benefits, prevent the concentration of power among a few big companies, and enable safer AI deployment across society. By open sourcing the company’s largest language model to date, Meta aims to make Llama the “industry standard” for anyone to develop AI-powered apps and services with, Zuckerberg wrote.
Open sourcing AI models and adding them to its existing products already used by billions of people could allow Meta to compete more effectively with OpenAI whose ChatGPT and DALL-E chatbots ignited an AI explosion when they launched in 2022. And it could also boost engagement — Meta announced today that users would soon be able to add AI-generated images directly into feeds, stories, comments and messages across Facebook, Messenger, WhatsApp and Instagram.
In his letter, Zuckerberg also criticized Apple and its closed ecosystem, arguing that the iPhone maker’s restrictive and arbitrary policies had constrained what Meta could build on its platforms. “[It’s] clear that Meta and many other companies would be freed up to build much better services for people if we could build the best versions of our producers and competitors were not able to constrain what we could build,” he wrote.
Source: www.engadget.com