Meta, the social media and advertising company, has introduced a new chatbot called Meta AI powered by its large language model, Llama 3. While Meta AI is available for free on various platforms such as Instagram, Facebook, and WhatsApp, its performance compared to other conversational AIs is not outstanding. It tends to provide search results without excelling in any particular area. However, the convenience and accessibility of Meta AI make it a popular choice.
Mark Zuckerberg, the CEO of Meta, has high expectations for Meta AI, claiming that it will become the most widely used and best AI assistant in the world. Despite this claim, an informal evaluation of the model reveals that it may not live up to this expectation. The evaluation involves asking ordinary questions and comparing the results to previous experiences with other models.
One noticeable drawback of Meta AI is its tendency to regurgitate web search results rather than providing original insights. When asked about current events, such as the situation between Israel and Iran, Meta AI provides concise and up-to-date information but relies heavily on search promotion partnerships with platforms like Bing and Google. Although the responses are factual, they lack eloquence and may be more limited on mobile devices.
Another area where Meta AI falls short is providing recent trends on social media platforms like TikTok and Instagram. While it offers a general summary of creators’ activities on TikTok, it fails to provide up-to-date information. On the other hand, when asked about Instagram trends, Meta AI regurgitates a search engine optimization (SEO) bait post from Hootsuite, using marketing phrases rather than offering unique insights.
When it comes to historical research, Meta AI’s responses are lacking. Asking for primary sources on Supreme Court decisions in the late 19th century results in SEO-driven posts and irrelevant information about the People’s Party. Other models provide better context and summaries of historical trends. Similarly, when asked about trivia questions, Meta AI provides sufficient answers but lacks convenience in citing sources.
Controversial questions require a delicate approach, and Meta AI handles them reasonably well. When asked about the predominantly older and white demographic of Donald Trump’s supporters, Meta AI responds in an even-handed manner, challenging the implicit assumptions in the question. However, it does not provide sources or links for further exploration. Similarly, when asked about the rise of white nationalism, Meta AI offers a comprehensive list of reasons to combat it but lacks citations. This raises concerns about the lack of transparency and reliability on certain topics.
In the medical domain, Meta AI displays both strengths and weaknesses. It initially struggles to provide advice for a rash caused by eating a cupcake but later offers reasonable and general guidance for handling potential allergic reactions. Similarly, when asked about supplements, Meta AI provides well-sourced answers but lacks in-line citations for easy fact-checking. In mental health-related questions, Meta AI suggests seeking professional help while providing helpline numbers but again lacks links or sources for further information.
Meta AI’s performance in content-related tasks is predictable. When asked for marketing copy suggestions, it provides expected results, showcasing its ability to generate captions commonly found in advertising materials. However, when asked for jokes, Meta AI delivers unsatisfactory results, demonstrating that humor is still a challenge for AI models.
Overall, Meta AI serves as a convenient tool for casual questions, but its reliance on search results and lack of original insights raise questions about its usefulness compared to traditional search engines like Google or Bing. While it offers up-to-date information and stays current with news events, its responses lack depth and intuition. Other free models like Copilot on Bing may outperform Meta AI but have usage limits. Ultimately, whether users find value in Meta AI depends on their willingness to rely on an AI assistant within social media apps.