Highlights
- Meta’s Llama 3 features 405 billion parameters, surpassing its predecessor.
- Supports eight languages with an expanded context window for larger requests.
- Significant improvements in math and knowledge benchmarks.
- Future multimodal versions to include image, video, and speech capabilities.
Meta Platforms has released its most advanced artificial intelligence model to date, Llama 3, featuring enhanced multilingual capabilities and improved performance in coding and mathematics.
🔵 META UNVEILS BIGGEST LLAMA 3 AI MODEL, TOUTING LANGUAGE AND MATH GAINS
Meta Platforms released the biggest version of its mostly free Llama 3 artificial intelligence models on Tuesday, boasting multilingual skills and general performance metrics that nip at the heels of paid… pic.twitter.com/5vVUBx5d1T
— PiQ (@PiQSuite) July 23, 2024
As reported by Reuters, this latest iteration boasts 405 billion parameters, significantly surpassing its predecessor while still remaining smaller than some competitors’ models.
New Future for Llama 3
According to Reuters, Meta CEO Mark Zuckerberg expressed confidence in the future of Llama models, stating, “I expect future Llama models would overtake proprietary competitors by next year.”
Zuckerberg also predicted that the Meta AI chatbot powered by these models would become “the most popular AI assistant by the end of this year, with hundreds of millions of people using it already.”
The new Llama 3 model offers support for eight languages and features an expanded “context window,” allowing for improved handling of larger user requests.
Ahmad Al-Dahle, Meta’s head of generative AI, told Reuters, “That was the number one feedback we got from the community,” highlighting the benefits for computer code generation in particular.
Updated AI Parameters
Reuters reports that Meta is releasing updated versions of its 8 billion and 70 billion parameter Llama 3 models alongside the flagship 405 billion parameter version.
All three models are multilingual and feature the expanded context window.
| Category | Benchmark | Llama 3.1 8B | Llama 3 8B – April | Llama 3.1 70B | Llama 3 70B – April | Llama 3.1 405B |
| General | MMLU | 73 | 65.3 | 86 | 80.9 | 88.6 |
| MMLU (CoT) | 48.3 | 45.5 | 66.4 | 63.4 | 73.3 | |
| MMLU PRO (5-shot, CoT) | 80.4 | 76.8 | 87.5 | 82.9 | 88.6 | |
| Code | IFEval | 72.6 | 60.4 | 80.5 | 81.7 | 89 |
| HumanEval (0-shot) | 72.8 | 70.6 | 86 | 82.5 | 88.6 | |
| MBPP EvalPlus (base) (0-shot) | 84.5 | 80.6 | 95.1 | 93 | 96.8 | |
| Math | GSM8K (8-shot, CoT) | 51.9 | 29.1 | 68 | 51 | 73.8 |
| MATH (0-shot, CoT) | 83.4 | 82.4 | 94.8 | 94.4 | 96.9 | |
| Reasoning | ARC Challenge (0-shot) | 32.8 | 34.6 | 46.7 | 39.5 | 51.1 |
| GPQA (0-shot, CoT) | 82.6 | 48.3 | 90 | 85.1 | 92.3 | |
| Tool use | API-Bank (0-shot) | 76.1 | 60.3 | 84.8 | 83 | 88.5 |
| BFCL | 8.2 | 1.7 | 29.7 | 14.7 | 35.3 | |
| Gorilla Benchmark API Bench | 38.5 | 18.1 | 56.7 | 47.8 | 58.7 | |
| Nexus (0-shot) | 68.9 | – | 86.9 | – | 91.6 | |
| Multilingual | Multilingual MGSM | – | – | – | – | – |
Meta’s strategy of releasing Llama models largely free of charge aims to hone innovation and reduce dependence on potential competitors.
As per Reuters, Zuckerberg believes this approach will lead to “innovative products, less dependence on would-be competitors and greater engagement on the company’s core social networks.”
The company claims significant improvements in key math and knowledge tests, with results suggesting that the largest Llama 3 model is approaching or surpassing the performance of leading paid models like Anthropic’s Claude 3.5 Sonnet and OpenAI’s GPT-4o in some areas.
On the MATH benchmark of competition-level math word problems, Meta’s model scored 73.8, compared to GPT-4o’s 76.6 and Claude 3.5 Sonnet’s 71.1.
On the MMLU benchmark covering various academic subjects, Llama 3 achieved a score of 88.6, nearly matching GPT-4o’s 88.7 and surpassing Claude 3.5 Sonnet’s 88.3.
Meta researchers also hinted at upcoming “multimodal” versions of the models, which will incorporate image, video, and speech capabilities.
Early experiments suggest these models may compete with other multimodal offerings such as Google’s Gemini 1.5 and Anthropic’s Claude 3.5 Sonnet.
FAQs
What is the Llama 3 AI model by Meta?
The Llama 3 AI model is Meta’s most advanced artificial intelligence model, featuring 405 billion parameters, enhanced multilingual capabilities, and improved performance in coding and mathematics.
What improvements does Llama 3 offer over its predecessor?
Llama 3 supports eight languages, features an expanded context window for handling larger requests, and shows significant improvements in math and knowledge benchmarks.
How does Llama 3 perform in benchmarks compared to competitors?
Llama 3 scores highly on key benchmarks, nearly matching or surpassing leading models like Anthropic’s Claude 3.5 Sonnet and OpenAI’s GPT-4o in some areas.
What is Meta’s strategy for releasing Llama 3?
Meta plans to release Llama 3 models largely free of charge to foster innovation and reduce dependence on potential competitors, aiming to boost engagement on its social networks.
Are there plans for future versions of Llama 3?
Yes, Meta researchers hinted at upcoming multimodal versions of the models that will incorporate image, video, and speech capabilities.
Also Read: Apple Intelligence: AI-Powered Enhancements Boost Company Stock to Record High
Also Read: Apple to Integrate OpenAI Technology in iOS 18: Enhancing AI Features