Meta's Llama 3 vs. OpenAI GPT4?

Along with: Small is beautiful with Microsoft Phi 3

Hey there, 

Llama 3 has arrived, shaking up the LLM scene with its open-source capabilities that measure up to the big-name models. It's not just software; Llama 3 is now embedded in our communication channels like WhatsApp, Instagram, and Facebook.

What's more, it's stepping into the physical world with Ray Ban smart glasses, blending digital intelligence with everyday wearables. The model has multi-modal capabilities - you can navigate environments, make video calls, and more.

Meta is still to release the 400B parameter model, which is still under training. That would be the biggest open-source model available by today’s standards. 

On the other side of the spectrum, Microsoft silently open-sourced Phi 3 - a small, potent model designed to fit in your phone. Early reports claim a GPT 3.5 Turbo performance in under 2GB RAM!

On a side note, if you are in India - don’t miss out on DataHack Summit 2024. It is going to be the place to meet leaders and practitioners and to experience GenerativeAI in person. The super early bird offer will run out soon! Conference dates - 7th to 10th August 2024. Know more here

What would be the format? Every week, we will break the newsletter into the following sections:

  • The Input - All about recent developments in AI

  • The Tools - Interesting finds and launches

  • The Algorithm - Resources for learning

  • The Output - Our reflection 

Please note: This is an abbreviated version of our newsletter due to email length restrictions. For the complete experience, visit our website and enjoy the full, unabridged edition.

Table of Contents

Meta has launched Llama 3, its latest open-source model, offering two versions for developers: an 8-billion and a 70-billion parameter model, both free to use.

Outperforming competitors like Gemini 1.5Pro and Claude 3 Sonnet on standards like MMLU and HumanEval, Llama 3 now features a version with over 400 billion parameters, a significant upgrade from last year's 70 billion.

Enhancements include the ability to create animations and generate high-resolution images in real time.

Additionally, Meta's AI assistant is now integrated into the search functionalities of Instagram, Facebook, WhatsApp, and Messenger. (source)

Microsoft recently unveiled their Phi-3 LLMs shortly after the release of Llama 3, showcasing superior performance despite its smaller size.

The smallest Phi-3 model outperforms the Llama 3 8B model, even though it has been trained on significantly fewer tokens and has less than half its parameters.

Remarkably, the Phi-3-mini, which is compact enough to be used on mobile phones, matches the performance of larger models like Mixtral 8x7B and GPT-3.5, requiring only about 1.8GB of memory when quantized.

This leap in efficiency is attributed to the use of high-quality datasets, consisting of heavily filtered web and synthetic data.

Additionally, there are variants of Phi-3, including one that supports a context window of up to 128k tokens, maintaining the same tokenizer as the earlier Llama 2. (source)

Meta has expanded its Ray-Ban Meta smart glasses collection with new styles and added new features like video calling via WhatsApp and Messenger to share views directly from the glasses.

The new Meta AI with Vision enables hands-free information gathering about the wearer's surroundings. These updates are available in various styles to fit different face shapes and are designed to be prescription lens compatible. (source)

Microsoft's VASA-1 is a new research project designed to create lifelike talking faces for virtual characters, using just a single static image and an audio clip.

This advanced model not only aligns lip movements with spoken audio accurately but also captures emotional expressions to heighten the realism of the characters. However, it is still being trained to better handle aspects like hair and clothing.

This technological development stands out for its ability to generate real-time responses, making it a significant step forward in interactive AI systems.

This innovation could potentially transform how virtual characters are used in various media, offering more engaging and realistic interactions. (source)

Adobe has just released Firefly Image 3, enhancing its AI-powered image generation with photorealistic outputs, better text and prompt interpretation, and upgraded illustration features. 

Additionally, Photoshop has introduced several AI-driven tools like Structure and Style Reference, which give users more control over image generation. 

The updates also improve functions such as Generative Fill and Generate Image, and allow for direct image generation and editing within Photoshop itself. 

These integrations eliminate learning curves and provide instant access to advanced tools, reinforcing Adobe's commitment to enhancing its existing platforms with AI (source)

China's leading tech companies, Ant Group, Baidu, and Tencent, have collaborated with US tech giants like OpenAI and Nvidia to establish the first global standards for GenAI and LLMs.

These standards were introduced at the United Nations Science and Technology Conference in Geneva and are designed to ensure the safety and security of GenAI applications and LLMs.

The standards include comprehensive guidelines for testing and validating the security of these technologies, reflecting a proactive approach as GenAI technologies become increasingly prevalent in both business and personal applications.

This collaboration marks a significant milestone in the global regulation and development of AI technologies, aiming to safeguard the rapid advancement of these powerful tools. (source)

Apple's research introduces a sophisticated architecture for detecting device-directed speech by leveraging a multi-signal large language model (LLM).

This model uses a combination of sound from the speaker's voice and words recognized by speech recognition systems.

By employing a multi-modal fusion approach, the system determines if the command is meant for the device.

This means the device can respond more accurately to voice commands and ignore background noise or conversations not directed at it.

Essentially, it makes voice-controlled devices smarter and more reliable in understanding when to react to commands. (source)

In a recent interview, Anthropic CEO Irina Higgins discusses the potential dangers and ethical implications of AI systems capable of self-replication.

While acknowledging the benefits of such advanced AI, including its potential to accelerate scientific progress and solve complex problems, Higgins emphasizes the need for strict oversight and ethical guidelines to prevent these systems from evolving beyond human control.

She stresses the importance of ensuring that AI aligns with human values and doesn't pose existential risks to humanity.

Overall, the discussion underscores the delicate balance between harnessing AI's capabilities for positive impact while safeguarding against potential risks and unintended consequences. (source)

Hugging Face has unveiled a benchmark specifically designed to evaluate the performance of generative AI models in healthcare-related tasks.

This benchmark aims to assess the ability of AI models to generate accurate and useful outputs in areas such as medical imaging analysis, clinical note generation, and drug discovery.

By providing a standardized evaluation framework, Hugging Face aims to facilitate advancements in AI for healthcare by enabling researchers and developers to compare different models and techniques effectively.

This initiative underscores the growing importance of AI in healthcare and the need for robust evaluation methods to ensure the reliability and effectiveness of AI-driven solutions in medical settings. (source)

Nothing has rolled out an update for its Nothing Phone (2), introducing ChatGPT integration within its operating system, Nothing OS 2.5.5.

This update brings new widgets for voice and text input directly from the home screen and a button for adding screenshots and clipboard content into ChatGPT dialogues, enhancing user interaction.

Additionally, Nothing's latest earbuds, the Ear and Ear (a), now support voice interactions with ChatGPT when connected to a Nothing phone, allowing users to engage with the AI without using their phones directly. (source)

MenteeBot is a versatile robot designed for both home and warehouse tasks.

It can handle domestic chores like table setting, laundry, and more, learning new tasks via voice and visual cues.

In warehouses, it efficiently handles up to 25kg and operates for up to five hours on a single charge.

It uses advanced technologies like Sim2Real for natural movements, NeRF for 3D navigation, and AI for task understanding, ensuring precise and reliable operations with its high-quality integrated electronics and AI capabilities. (source)

Tool: Survicate

Creating surveys can be a time-consuming and meticulous task as it's important to cover all scenarios. A tool that automatically crafts surveys for various purposes would be incredibly useful.

Problem Statement: Develop a survey to gather feedback from subscribers about AI emergence.

Solution:

  1. Sign up or log in.

  2. Create a new survey.

  3. Utilize AI-assisted survey creation.

  4. Set targeting options.

  5. Preview and test the survey.

  6. Launch the survey.

Access the survey here: Take the Survey - We invite you to fill it out and share your feedback. 🙂

For more information about the tool, check out our blog.

  • In the recent episode of Leading with Data, I had a fascinating conversation with Vijay Karunamurthy, CTO at Scale AI. We explored topics ranging from the evolution of search on YouTube to Apple's privacy practices, insights from Vijay's GTC '24 talk, and the future of AI and self-driving cars.

  • In this video, Mark Zuckerberg discusses Llama 3's strides towards Artificial General Intelligence (AGI) through open sourcing, along with innovations in custom silicon, synthetic data, and addressing energy constraints for scaling AI.

  •  In a recent episode of TED Talks, Microsoft AI CEO Mustafa Suleyman delved into the future of artificial intelligence, likening its development to the emergence of a new digital species.

  • Watch this video by Reid Hoffman to find out why he chose to deep-fake himself.

  • Watch this recent interview of Jamie Dimon, CEO of JPMorgan Chase, where he discusses his insights on the banking industry, markets, technology, and geopolitical risks with Bloomberg Originals Host & Executive Producer Emily Chang.

  • TED Talk recently released a video, created in collaboration with artist Paul Trillo and OpenAI. This innovative video, made with OpenAI's yet-to-be-released text-to-video model Sora, envisions how TED Talks will look in 40 years, showcasing a unique and creative application of Generative AI tools.

  • This course by Mistral AI offers an introduction to Mistral's open-source models and aims to enhance participants' prompt engineering skills by covering various examples, enabling them to effectively utilize Mistral's AI models in real-world applications and integrate them seamlessly into software systems.

  • Recently, Drake released a rap track titled “Taylor Made Freestyle,” which directly targets Kendrick Lamar. In an innovative twist, Drake utilized AI technology to emulate the voices of Tupac Shakur and Snoop Dogg—two artists greatly admired by Kendrick, a blend of technology with a deep knowledge of hip-hop culture.

Given the week - I was thinking will large models will contribute more to the development of LLMs or will it be the small models?

Now - it is obvious that you would experience the most advanced models in Large models unless there is a breakthrough. But, large models come with their own set o challenges - high energy needs, high inference memory required, and hence they will take longer to get embedded in Edge devices.

You can see this already. A smaller Llama 3 could sit on RayBan glasses, and WhatsApp and hence might create more interaction and more feedback to these models.

Imagine Airpods and Smartwatches, which can generate music for you on the fly without requiring internet connections. You decide the kind of music you want to train with and it is created on the fly. Just stretch the imagination a bit more and think what if these devices can sense your workout intensity and change the music and beats to amp up / down the intensity?

All with the magic of small models in a few years!

Which models are you more excited about?

Login or Subscribe to participate in polls.

See you again in a week with developments that feel like quarters of development! That’s the world we are in.

How do you rate this issue of AI Emergence?

Would love to hear your thoughts

Login or Subscribe to participate in polls.

Reply

or to participate.