19-4-2024 (NEW YORK) Meta Platforms has unveiled its latest venture in the generative AI market with the release of Llama 3, a large language model, and an image generator that updates pictures in real time. The move is aimed at challenging the dominance of OpenAI, the current market leader. Meta AI, the virtual assistant, will incorporate these new models, positioning itself as the most advanced among its free counterparts. Meta plans to give the assistant more prominence within its Facebook, Instagram, WhatsApp, and Messenger apps, along with a standalone website, in a direct competition with Microsoft-backed OpenAI’s popular ChatGPT.
Meta’s announcement comes as the company strives to catch up with OpenAI’s leading position in generative AI technology. This endeavor involves significant changes to computing infrastructure and the consolidation of previously separate research and product teams.
According to Chris Cox, Meta’s Chief Product Officer, Llama 3 has been equipped with new coding capabilities and has been trained on both images and text, although it currently generates only text outputs. Cox explained that subsequent versions will include advanced reasoning, enabling the model to craft longer multi-step plans. Meta’s blog posts confirm that upcoming releases will possess “multimodality,” allowing them to generate both text and images.
Cox emphasized the ultimate goal of these advancements: to simplify users’ lives by assisting with tasks such as interacting with businesses, writing, and trip planning. As part of this vision, Llama 3’s training with images will enhance the forthcoming update of Ray-Ban Meta smart glasses, enabling Meta AI to identify objects seen by the wearer and provide relevant information.
In addition to the Llama 3 release, Meta announced a partnership with Google to incorporate real-time search results into the assistant’s responses. This collaboration supplements Meta’s existing arrangement with Microsoft’s Bing. The Meta AI assistant will expand its availability to more than a dozen markets outside the United States, including Australia, Canada, Singapore, Nigeria, and Pakistan. Cox mentioned that the company is still working on implementing this expansion in Europe, where stricter privacy rules and the upcoming AI Act may require disclosure of models’ training data.
Generative AI models heavily rely on data, which has become a significant point of contention in the technology’s development. As part of its effort to catch up with competitors, Meta has been releasing models like Llama 3 for free commercial use by developers. By offering a powerful free option, Meta aims to disrupt competitors’ plans to generate revenue from their proprietary technology. However, this strategy has raised safety concerns among critics who worry about potential misuse of the model by unscrupulous developers.
In a video accompanying the announcement, Meta CEO Mark Zuckerberg acknowledged the competition, describing Meta AI as “the most intelligent AI assistant that you can freely use.” Zuckerberg revealed that the largest version of Llama 3 is currently being trained with 400 billion parameters, achieving an impressive 85 MMLU (Massive Multitask Language Understanding). Smaller versions, with 8 billion and 70 billion parameters, are being rolled out, with the latter scoring around 82 MMLU.
Developers had previously criticized the Llama 2 version of the model for its failure to understand basic context, often confusing computer program-related queries with requests for instructions on committing murder. Google faced similar problems, leading to a pause in the use of its Gemini AI image generation tool due to inaccurate depictions of historical figures.
Meta claims to have addressed these issues in Llama 3 by employing “high-quality data” to enhance the model’s ability to recognize nuance. Details regarding the specific datasets used were not disclosed. However, Meta stated that it fed seven times more data into Llama 3 compared to Llama 2 and utilized “synthetic” or AI-generated data to strengthen coding and reasoning capabilities.
Regarding the sourcing of training data, Cox stated that there has been “not a major change in posture” from the company’s previous approach.
