Can Llama 3 bring open source to the lead?

Along with: When is Superhuman Intelligence coming?

Hey there, 

It’s been a big week for AI. 

OpenAI released an improved version of GPT 4 Turbo, Google relaunched Gemini 1.5 Pro, Mixtral launched its frontier model and Llama 3 is set to be launched in the next month. 

What would be the format? Every week, we will break the newsletter into the following sections:

  • The Input - All about recent developments in AI

  • The Tools - Interesting finds and launches

  • The Algorithm - Resources for learning

  • The Output - Our reflection 

  • Question to ponder before we meet next!

Please note: This is an abbreviated version of our newsletter due to email length restrictions. For the complete experience, visit our website and enjoy the full, unabridged edition.

Table of Contents

Meta is set to release Llama 3, the newest version of its large language model (LLM), within the next month. This move aims to advance its generative AI capabilities and keep pace with rivals like OpenAI.

Nick Clegg, Meta's President of Global Affairs, emphasized that Llama 3 will introduce a range of models with unique capabilities, designed to power a variety of Meta products.

Expected to feature around 140 billion parameters, Llama 3 will be made open source. This aligns with Meta's philosophy of promoting widespread technology development and seeks to broaden its question-answering abilities – potentially into more controversial topics.

Additionally, Meta appears to be exploring alternative approaches alongside traditional LLMs.  Their development of the joint embedding predicting architecture (JEPA), which focuses on more precise predictive AI, signals a nuanced approach to the future of generative AI technologies. (source)

This is not the first time Musk has come out with an aggressive timeline. My take is that Musk’s timelines on any predictions have 2 aspects. Firstly, the timelines are aggressive and they usually extend by a few years.

That is still hugely impressive given the kind of problems he is solving. Secondly, all his timelines do happen. Even though his timelines might be delayed - they do happen.

He laid out the vision and timelines for Tesla, SpaceX, and Neuralink, and more often than not, irrespective of the magnitude of the prediction - they do happen. So - I would think we are going to see a SuperHuman AI but likely in the next 5 years, which by itself is still exciting and scary! (source)

What do you think are the timelines for a Superhuman AI?

Login or Subscribe to participate in polls.

OpenAI has launched GPT-4 Turbo with Vision through its API, introducing enhancements like JSON mode and function calling for handling visual data.

It offers improved performance at lower costs, with input tokens now cheaper by threefold and output tokens at half the price.

While unleashing the GPT-4 Turbo, OpenAI also teased the future Voice Engine and the more advanced GPT-5 model, hinting at further innovations. (source)

French AI startup Mistral launched "Mixtral 8x22B," a new large language model (LLM) aiming to outperform its predecessors and competitors like OpenAI's GPT-3.5 and Meta's Llama 2.

This model, notable for its 65,000-token context window and 176 billion parameter size, represents an advancement in AI's ability to process and reference large amounts of text. (source)

Intel unveiled the Gaudi-3 AI accelerator, positioning it as a formidable competitor to Nvidia's H100/200 with claims of 40% better performance and significantly improved power efficiency.

Crafted on the TSMC 5nm manufacturing process and boasting 128GB of HBM3E memory, Gaudi-3 is designed to excel in generative AI applications. 

With the AI industry's demand soaring, Intel's strategy focuses on performance, efficiency, and a more accessible price point.

Despite potential challenges in scalability and future roadmap concerns, Intel's open approach and partnership with major OEMs signal strong market potential for Gaudi-3, particularly among enterprises and emerging sovereign data centers. (source)

Jony Ive, Apple’s former chief design officer, and Sam Altman, CEO of OpenAI, are reportedly collaborating to create the "iPhone of artificial intelligence," with potential funding of over $1 billion from Softbank CEO Masayoshi Son.

They aim to design a consumer device through Ive's firm LoveFrom, aiming for a "more natural and intuitive user experience" with AI, drawing inspiration from the original iPhone's transformative impact. 

The project, currently in the brainstorming phase, aims to create a less screen-reliant device, drawing on Altman’s investment in Humane to develop a wearable AI device envisioned as a smartphone alternative.

The rumors have been around for some months now. Hopefully, we should see something more substantial on this in the coming weeks/months. (source)

Google has enhanced its Gemini 1.5 Pro model with audio processing capabilities, allowing it to analyze and extract information from audio files, such as earnings calls or video soundtracks, without needing written transcripts.

Announced at Google Next, Gemini 1.5 Pro is now accessible through Vertex AI, marking its public debut. This update positions Gemini 1.5 Pro ahead of Gemini Ultra in performance, capable of understanding complex instructions and operating without the need for model fine-tuning. (source)

OpenAI is making it easier for developers to fine-tune and build AI models. Fine-tuning is a technique that allows developers to take a pre-trained model and adjust it to a specific task. This can be a great way to improve the performance of a model on a particular task.

OpenAI has recently added some new features to its fine-tuning platform, including epoch-based checkpoint creation, a comparative UI, and support for third-party integrations. These new features should make it easier for developers to fine-tune and build AI models. (source)

OpenAI faces criticism for potentially violating Google and YouTube's Terms of Service by using YouTube data to train its AI models, including Whisper and Sora.

Google and YouTube executives have expressed concerns about policy violations through data scraping, with Google's spokesperson referring to "unconfirmed reports" of such activities.

This controversy sparks wider ethical debates about data use in AI, leading to increased scrutiny of tech companies like OpenAI, Google, and Adobe. (source)

Researchers at Stanford University have introduced Octopus v2, a new on-device language model that addresses the challenges of latency, privacy, and cost associated with LLMs

Traditional LLMs are often too slow, expensive, and privacy-invasive for use on devices. Octopus v2 overcomes these limitations by using a fine-tuning method with functional tokens, which significantly reduces the context length needed for processing.

In tests, Octopus v2 achieved a 99.524% accuracy rate in function-calling tasks, with a latency of only 0.38 seconds per call. This represents a 35-fold improvement compared to previous models. (source)

Canada, under PM Justin Trudeau, is investing $1.76 billion to enhance its AI sector.

This funding initiative earmarks a substantial $1.47 billion for computing capabilities through the AI Compute Access Fund to support AI startups, research firms, and infrastructure development, along with an additional $220.5 million allocated to enhance AI startups across diverse industries and assist small and medium-sized AI companies in scaling their operations seamlessly. (source)

Huawei Technologies' Pangu-Weather AI model which was hailed as China's top 2023 scientific breakthrough, has transformed weather forecasting by delivering predictions 10,000 times faster with unmatched precision and speed.

Following its success, the team introduced Zhiji, an iteration focused on regional weather prediction, achieving a groundbreaking precision of 3km for five-day forecasts.

Zhiji, trained on high-resolution data from southern China, has markedly enhanced weather forecasting accuracy in Shenzhen and nearby areas, showing potential in typhoon prediction and the integration of AI with traditional methods for superior forecasts during its trial operations. (source)

Archetype, an AI startup, is pioneering a transformative approach that enables users to interact with physical environments—like houses, cars, and factories—through its AI model, Newton.

By analyzing data from various sensors, Newton translates complex sensor inputs into plain language, allowing for real-time, ChatGPT-style communication about the physical state of objects and environments.

Newton can monitor the condition of packages, ensuring their integrity during transport, and offer insights into complex industrial processes without the need for specialized software. (source)

Microsoft has unveiled plans to establish a new AI hub, Microsoft AI London, focusing on advancing state-of-the-art language models and developing tooling for foundation models in collaboration with OpenAI.

Led by AI scientist Jordan Hoffmann, formerly of Inflection and DeepMind, the hub will be situated in London's Paddington office. (source)

Tool - Invideo

Thought of creating a video for marketing purposes without using any additional resources? 

Problem Statement - Create the teaser video for DataHack Summit 2024 happening in Bengaluru from 7-10 August 2024. 

Solution - 

  1. Go to Invideo

  2. Signup/Login

  3. Write your prompt, ideally, add a script to be precise.

  4. Choose a Video Concept and Target Audience.

  5. Generate.

  6. Reviewing and Regenerating the Draft

  7. Making Changes to the Text and Storyboard

  8. Download (without watermark if you buy the subscription)

Here’s the first draft that we got - Here

For more information about the tool, check out this blog

  • In the recent episode of "Leading with Data," I had a chat with Goda Ramkumar about her experience in utilizing machine learning to solve complex business problems at Sabre and Ola, her relentless passion that fuels her journey, and her strategic insights on building effective data science teams.

  • For those interested in enhancing RAG system performance through efficient data processing, this course, "Preprocessing Unstructured Data for LLM Applications" by DeepLearning.AI, teaches techniques for preprocessing various unstructured data types for LLM application development.

  • I went through the recently published book “Co-intelligence: Living and Working with AI” by Prof. Ethan Mollick. Like always, his writing is to the point, fact-based, and conversational, and he leaves you with good guidance. Read my thoughts on the book here.

Ethan Mollick lays out 4 possible scenarios about AI development from where we stand today:

  • Scenario 1 - The current form of AI is the best AI we will see.

  • Scenario 2 - AI continues to evolve slowly / linearly (10-20% improvement) year on year

  • Scenario 3 - AI continues to evolve exponentially

  • Scenario 4 - The Machine God

If you had to take a bet, which scenario would you bet on?

Login or Subscribe to participate in polls.

I will be surprised if a lot of our current subscribers put their money on Scenario 1. We already know that better LLMs are on the verge of coming out.

Earlier, Sam Altman had hinted at something big this summer and now Meta has put a timeline to Llama3. In all likelihood, they are being tested right now.

For the same reasons, the chances of possibility 2 look bleak to me. 

In all likelihood, we are looking at scenario 3 or 4. Musk has announced that we are looking at scenario 4. 

Honestly, I am stuck between scenarios 3 and 4. The human in me is still not able to fathom the possibility that we are on the roadmap to creating SuperHuman intelligence and the math behind is understood by the Research labs out there. 

I think, only time will tell. Would love to hear what you think.

How do you rate this issue of AI Emergence?

Would love to hear your thoughts

Login or Subscribe to participate in polls.

Reply

or to participate.