More Power to Open Source LLMs!

Along with: Who let the (AI)rt out?

Hey there, 

In the true spirit of Emergence, last week saw some awesome open-source models and architectures emerge. Companies from India, and China are announcing their language-specific models and Robotics continues to stay hot in the background.

Let’s Dive In!

What would be the format? Every week, we will break the newsletter into the following sections:

  • The Input - All about recent developments in AI

  • The Tools - Interesting finds and launches

  • The Algorithm - Resources for learning

  • The Output - Our reflection 

  • Question to ponder before we meet next!

Table of Contents

This week saw exciting new Open Source LLMs. With developments ranging from Code Llama to SQLCoder, Open Source models are rapidly advancing. In some scenarios, open-source models are even zooming ahead of their proprietary counterparts.

Meta has just rolled out the latest version of Code Llama -  Code Llama 70B, the open-source language model designed for coding and stands as a competitor to Github CoPilot. It is available in two versions: CodeLlama-70B-Python and CodeLlama-70B-Instruct.

The 70B-instruct variant scored 67.8 on HumanEval, outperforming Gemini Pro in zero-shot prompts.

Touted as the "most comprehensive and high-performing model" so far, Code Llama 70B is capable of handling a larger volume of queries than its previous versions. (source)

A new "multimodal" AI model Yi-VL-34B (Source)was released by China's top AI firm, 01.AI, led by Kai Fu Lee. This model is capable of analyzing images and interpreting their content.

01.AI has chosen to make its AI models open source and is doing interesting work. Their goal is to foster a dedicated community of developers, helping to develop “Killer AI apps”. (source)

You can now phrase queries in everyday language, such as "How many people from India purchased my product?" and SQLCoder will seamlessly translate it into an SQL query.

SQLCoder by defog, belongs to a series of advanced large language models. Impressively, it surpasses both gpt-4 and gpt-4-turbo in converting natural language to SQL queries, as tested on the sql-eval framework.

The models, SQLCoder-70B and SQLCoder-34B have been specifically fine-tuned using the foundational Code Llama model. (source)

The new Eagle 7 Billion model is here to catch your attention. This model marks a shift away from the usual Transformers architecture, leaning towards recurrent neural networks (RNNs).

The foundation of Eagle 7B lies in the RWKV architecture, a combination of RNN and Transformer technologies, harnessing the strengths of both.

Transformers have excelled in handling data and have heavy memory space but are relatively expensive compared to RNNs. RNNs, recognized for their ability to identify sequences and predict data patterns, consume less RAM and offer faster inference. They also can be trained more quickly.

For the first time, an RNN has outperformed all transformers of the same size that were trained with an equal amount of data. (source)

It remains to be seen how these new architectures scale up and whether we get similar scaling benefits as we saw with Transformers.

Generative AI is turning some artists' lives into a nightmare and some artist’s lives into paradise.

Recently, unauthorized sexually explicit deepfakes featuring Taylor Swift became widely circulated on X. The account responsible for posting these images was eventually suspended.

In recent months, this problem has intensified. However, tech platforms such as X, which have created their Gen-AI solutions, have still not implemented or openly addressed the use of tools to identify Gen-AI material that violates their policies. (source)

AI has rapidly become involved in every industry. Recently, A.R. Rahman, the Indian music composer, utilized AI to recreate the voices of the late singers Shahul Hameed and Bamba Bakya in his recent Tamil movie 'Salaam.'

In the song 'Thimiru Yezhuda,' you can hear the voices of these late singers. Additionally, Rahman has obtained permission from the families of the late singers, showcasing the positive use of AI. (source)

  • Google and Hugging Face to collaborate !!! - Google and Hugging Face have partnered to advance open-source AI and machine learning. This collaboration integrates the Hugging Face platform with Google's ecosystem, simplifying GenAI for developers. It enables cost-effective training, fine-tuning, and deployment of open models on Google Cloud, using AI-optimized TPUs and GPUs for better efficiency. (source)

  • Google update comes with a Privacy Nightmare - Google introduced Bard, an AI Assistant, into its messaging platform to enrich the user writing experience. Yet, this update raises privacy concerns, as Bard will learn by analyzing the tone and sentiments of users' conversations to tailor the messaging experience, prompting worries about potential privacy intrusion. (source)

  • Ola - From Cabs to AI Unicorn -  Krutrim, an AI startup founded by Bhavish Agarwal of OLA, has secured $50 million in funding led by Matrix Partners India, becoming India's first major unicorn in pure AI. The company aims to develop India's first comprehensive AI computing stack, marking a significant advancement. (source)

  • Jio Brain- An integration of AI with Telecom - Jio has launched 'Jio Brain,' a cutting-edge AI platform integrating machine learning into telecom and enterprise networks with minimal changes. It features 500 APIs for machine learning and advanced AI capabilities for processing images, videos, text, and speech. (source)

  • Nightshade, an AI tool, was designed to combat AI misuse. Developed by university researchers, it enables artists to make invisible modifications to the pixels in their artwork before uploading it online. This ensures that if the artwork is incorporated into an AI training set, it can disrupt the resulting model, leading to chaos and unpredictable outcomes. The tool has received 250,000 downloads in just five days! (source)

  • Arc Search, is the new iPhone-only app from a browser company. It's all about revolutionizing your browsing experience by creating a customized web page from scratch with tailored search results. The cool part? It has a 'browse for me' feature that does the searching for you. (source)

  • Weights and Biases have recently introduced an engaging course titled "LLM Engineering: Structured Outputs” focusing on efficiently extracting structured data using LLMs and Pedantic.

  • I'm also looking forward to the course “Building Applications with Vector Databases” by deeplearning.ai.  Would be interesting to build a hybrid search app that combines both text and images for a multimodal search experience.

  • Watch this awesome video by roboticist Dennis Hong talking about the potential of the next evolution in robotics engineering. Although Dennis Hong shares how humanoid robots are unstable, this video of Tesla's Optimus Robot taking a walk is fascinating. Yesterday, Elon Musk event went for a walk with Optimus (source)

  • In a recent episode of Leading with Data, I had an enlightening conversation with Julien Simon, Chief Evangelist at Hugging Face, where he shared insights on his tech journey, role at Hugging Face and AWS and the future of open-source LLMs.

  • OpenAI has announced a new feature ‘GPT Mentions’, and it's like tagging someone in a chat, but with custom GPTs or bots. This should increase the usage of custom GPTs in the coming time. Right now, it's in beta and not available to everyone just yet.

Awesome quote from none other than the creator of Linux and git! The beauty is that it applies to almost any creation - not just code. (Tweet)

An awesome week where open-source contributions flexed their muscles! Not just the releases, but this is the current belief when I talk to experts in the industry as well. I think 2024-25 will be bigger years for open source than what we have ever seen!

What are you most excited about in 2024?

How do you rate this issue of AI Emergence?

Would love to hear your thoughts

Login or Subscribe to participate in polls.

Reply

or to participate.