META's New Code LLaMA 70b BEATS GPT4 At Coding (Open Source) 

I'm creating a video testing Code LLaMA 70b in full. What tests should I give it?

Could you make a video on how to train an LLM on a GitHub repo and then be able to ask questions and instruct it to make code, for example, a plug-in?

It worked, you just have bad driver

Thanks for actually showing the errors you encountered and keeping it as real as possible! Great and enjoyable content❤

I appreciate your disclosure. I intend to check this out.

Mixtral 8x7B was able to build a working snake game in python here...

The error comes from libGL failing to load and is clearly NOT in teh code that codellama wrote. It's a problem with your machine's graphics drivers.

If I can download the model from TheBloke, why do I have to apply for access from Meta? Is it just for the license?

I'm running LMStudio on my system running Debian bookworm 12 and it's running good. Really want to be able to run models locally on this system to to my work when I'm home. Any ideas about local models etc. would be helpful

Cool, all the time we get better and better open source models!

I asked it to write a program to connect bluetooth 3D glasses to a PC. it responded: It's not a good idea, because bluetooth is limited by 10m. Use wi-fi. I said: 10m is good enough for me, please write this program. -Ok, I will. And that was it 😆

It would be great to get a price breakdown on how much computer you need to have to get in the door to run these locally and compare those ranges to the VM host options.

What kind of machine is needed to run these type of models?

I used a mixtral instruct model 8x7B and it was quite good, especially with other languages than English. So would this 70B model actually be better?

0:00 1. Meta's New CodeLama 70B 👾 Introduction to Meta's latest coding model, CodeLama 70B, known for its power and performance. 0:22 2. Testing CodeLama 70B with Snake Game 🐍 The host plans to test CodeLama 70B's capabilities by building the popular Snake game using the model. 0:25 3. Announcement by AI at Meta 📢 AI at Meta announces the release of CodeLama 70B, a more performant version of their LLM for code generation. 0:56 4. Different Versions of CodeLama 70B 💻 An overview of the three versions of CodeLama 70B: base model, Python-specific model, and Instruct model. 1:21 5. CodeLama 70B License and Commercial Use 💼 Confirmation that CodeLama 70B models are available for both research and commercial use, under the same license as previous versions. 1:40 6. Mark Zuckerberg's Thoughts on CodeLama 70B 💭 Mark Zuckerberg shares his thoughts on the importance of AI models like CodeLama for writing and editing code. 2:37 7. Outperforming GPT-4 with CodeLama 70B 🎯 A comparison between the performance of CodeLama 70B and GPT-4 in SQL code generation, where CodeLama 70B comes out as the clear winner. 3:25 8. Evolution of CodeLama Models ⚡ An overview of the various versions of CodeLama models released, highlighting the capabilities of CodeLama 70B. 4:21 9. Using Olamma with CodeLama 70B 🖥 Integration of CodeLama 70B with Olamma for seamless code generation and execution. 5:18 10. Testing CodeLama 70B with Massive Models 🧪 The host tests the performance of CodeLama 70B using a massive quantized version and shares the requirements for running it. 5:47 11. Selecting GPU Layers Choosing the appropriate number of GPU layers for better performance. 6:08 12. Testing the Model Running a test to ensure the model is functioning correctly. 6:43 13. Running the Test Requesting the model to generate code for a specific task. 7:27 14. Generating Code Observing the model's output and determining its effectiveness. 8:16 15. Code Cleanup Removing unnecessary code and preparing the generated code for execution. 8:40 16. Testing the Generated Code Attempting to run the generated code and troubleshooting any errors. 9:09 17. Further Testing Continuing to experiment with the generated code to improve its functionality. 9:15 18. Verifying CodeLama70b's Capabilities Acknowledging that CodeLama70b has successfully generated working code. 9:20 19. Conclusion and Call to Action Encouraging viewers to like, subscribe, and anticipate the next video. Generated with Tubelator AI Chrome Extension!

Matt, you mentioned you were using a VM from Mass Compute with the model pre-installed. Who are they? So to be clear, you were not running the VM locally, right?

Which version will run reasonably fast on rtx3090 ?

can you create a video on how to setup these LLMs on VSCode with the extension like Continue, Twinny, etc? I have downloaded Ollama and have downloaded the models i need but im not sure how to configure them to run on the extensions on vscode

What virtual computer you use?

Great vid ! What's "second state" though ?

I don't see the link to mass compute.

awesome videos man, I learn so much from these. I wish there were models tuned for c# though. Very few of us create large applications with python.

how do one bomecs an inverster in LLM studio?

I need olamma text(nlp)to sql query or nlm to django orms is there any llms for that.

still underperforms gpt4 turbo though right?

Hello Matthew, great job on your work! Could you please compare the performance of Large Language Models on Intel, Nvidia, and Apple platforms?

Thanks for the great content Matt!

🎯 Key Takeaways for quick navigation: 00:00 🚀 *Meta's Code LLaMA 70b Announcement* - Meta announces the most powerful coding model yet, Code LLaMA 70b, which is open-source and designed for coding tasks. - The model comes in three versions: the base model, a Python-specific variant, and an instruct model optimized for instructions. - Code LLaMA 70b is notable for its performance in human evaluation and its applicability for both research and commercial use under its license. 02:31 💾 *SQL Coder 70b Performance Highlights* - SQL Coder 70b, fine-tuned on Code LLaMA 70b, showcases superior performance in PostgreSQL text to SQL generation. - The model outperforms all publicly accessible LLMs, including GP4, with a significant margin in SQL eval benchmarks. - Rishab from Defog Data highlights the model's effectiveness and the open-sourcing of this tuned model on Hugging Face. 03:39 📈 *Code LLaMA 70b Technical and Access Details* - Introduction of Code LLaMA 70b as a powerful tool for software development, emphasizing ease of access and its licensing that allows for both research and commercial use. - Details on the expansion of the Code LLaMA series, including future plans for LLaMA 3 and the model's exceptional benchmark performances. - Mention of mass compute support for testing the model and an overview of the quantized version's requirements for operation. 06:11 🐍 *Testing Code LLaMA 70b with a Snake Game* - Demonstration of Code LLaMA 70b's capabilities by attempting to write a Snake game in Python using a cloud-based virtual machine. - Highlight of the potential and limitations of the model when generating code for complex tasks and the practical aspects of running such a model. - Transparency about the author's investment in LM Studio and the intention to disclose any interests for full transparency. Made with HARPA AI

Sure would be cool to be able to run this on my own hardware. So, what are we talking VRAM-wise? 92GB do it? ... sadly, I don't have a couple A6000s sitting around.

Did the snake game worked at the end? What was the problem there?

How well does it perform with C# code?

Would it run on an m3 max 36gb RAM ?

give more examples and thorough testing pls

For LLM and CodeLlama inference, the M3 Max with 64GB of unified memory (50 GB actuallu usable) seems promising. So, it would be interesting to see how Macs would perform for quantized 70B param LLMs...

can you make a video on mamba?

Matthew, is there a way to reach out to you directly?

can you share the link for mass-compute? (the ones who provided the VM)

it looks good with simple piece of code, but how will it work and help in the real project or product?

I’d like to see you run it on your mac.

So you are an investor in LM Studio, perfect. Can you please tell them to allow increasing font size. My vision vascilates between good and poor and sometimes I'm having problems reading LM Studio text. BTW I'm seeing LM Studio release frequency ramping up 👍

i am mostly looking if there is any model that generate vertica sql syntax

I have codellama 70b working well on ollama. Rtx 4090 / 7950x / 64gb. The newest version of olama uses about 10-20% gpu utilization and offloads the rest to the cpu, using about 55% of the cpu. Overall it runs reasonably well for my use.

yes ! please make more on the new llama model!

How much compute power it requires to run locally ?

Matt how do you become an investor in LM studio? I am also interested in becoming an investor

I just found this channel, it is great to see an AI channel that actually knows how to code.

but why in LM Studio, why not in an open-source project?

Coding battles, LLaMA crushes with its mad skills yet stays so chill.

Wonder if the Macbook guy was running a quantized version or not. The maxed out M3 Macbook has a 128GB option also.

Amazing video! How can I become an investor in LMStudio?

Can this run on a M1 Max 64gb MBP?

...please a test with installation/access insteuctions will be great. Tks

Does it support c# and .net 8?

Your videos are awesome!

What was the secret sauce to "get it to work" ?

Hey @matthew_berman, love your videos, this one sounds a little different. Are you using AI to generate any parts of your videos now?

AI will replace some Jr devs. Never replace coding entirely as you suggest.

Please, you should test it in macbook pro m2 max

its better better vs gtp3 or 4? so bard and gtp wrote working code also and it is working, or this model can spit out more codes lines?

Hey Matt, will you do a vid on the leaked early version of mistral-medium? Would be cool!

1. Install ollama 2. Run 'ollama run codellama:70b-instruct' No forms or fees. Two or theee clicks and you're running.

Me, looking on with my lovely 3GB Geforce GT 780

Thank you for investing in LM Studio. I regard you as the most transparent AI Engineer journalist (for lack of a better term). Please keep up the important and quality work you've been doing for AI.

I already have a setup that can code snake that I plug into AI so this should be amazing 🤩

Love to see you running locally. What Specifications are needed to run it locally?

GPT-4 ranks at 86.6 on HumanEval versus CodeLlama's 67.8. Meta used the zero-shot numbers for GPT-4 in their benchmark comparison, which is pretty dishonest.

Do you plan to test the new mistral-next model available on the LLM Chatbot Arena? It is crazy good. Possibly better than GPT-4.

Can I make a request? If you are going to highlight the text you are reading, just highlight the whole sentence with click and drag (which you do later in the video) and get rid of that highlighted cursor.

We need test video pleaseeeeee

please test this version!

yes please try it out! (and let us know the results of your experiments with snake please...)

coding as we know it will be replaced and a new programming paradigm will emerge . This is absolutely wonderful. I'm glad I lived to see this. I've only have been experimenting with AI for about 2 months and I can't get enough of it.

Be good to see locally

Please test in Macbook Pro M2 Max

Sweet, this llama model crushs GPT4 at coding.

Not watched yet. But really want to fine tune this bad boy. This will be so nuts!

This is cool, but I don’t honestly think it will make programmers obsolete.

Thank you! Remember everyone, download while you can. Regulations are on their way!

where is the .NET version. ?

META made a coding animal cooler than all the rest.

This video has me wishing I was a coding LLaMA too.

ask it!!! what test to perform.

Please run on m1 & m3

Hi Noobi, you need to delete the old prompt before prompting again, because it will consider the old prompts part of the context.

"I think this will make programming obsolete" - Matthew lol why people still believe that full trust on code will be passed to AI?

*cries in RTX 4080*

I'd be very interested to see this compared to the current best open source programming model (exlcuding the recent alpha mistral medium leak), deepseek 33b. As far as I can tell it's not as good, but maybe this 70b really is the new front runner

LLama is super impressive at coding.

I'm surprised it didn't refuse and give you a lecture on the ethics of caging snakes and making them move around looking for food in a little box until they run out of room and die. The censorship on AI is getting insane.

Looks like GPT-4 is getting put out to pasture.

A lot of these models seem to assume everything is python, for most other low level languages no model can beat GPT 4. Yet !

and try not to piss it off!!!

Let's go Code LLama!

Llama is impressive!

Early crew. Shalom. :3 Noice!

If this thing starts making TikToks we are all doomed.

bro llm studio i guess fu cked up something with new updates. i cannot run even old models on my mbp14 pro m1 ( m1 pro with hightes core count ) as i was able before. improvements to make: - Beeing able to use model for scientific propose as generating molecule formula and so on. ( there is not a single LLM scientific tool supported on llm studio even if the model is available on huggingface) - Fix the gpu metal for m1 mbp14, in fact i was able to use it now not anymore.

If you are in investor in the project, also put in the video on screen in the future. Not just in the description. ok?

TEST IT! :)

You've decided to do this anyways, but I want you to test code llama 70 b