For about a year now, I came to the realisation that the future of A.I. is open, not because of morals or that closed models are bad, but because of privacy, security and other concerns, especially as closed A.I. models could become very dangerous in the future if 2 or 3 corporations or governments control them, it would give them a massive advantage over everyone else. Because of all that, I feel open source is the only way to go to level the playing field, and even thought that carries its own risk, I feel the risk is much lower than A.I. being controlled by so few people and I like to think that A.I. being open, we can make the right decisions on it together, as the risk to us all is the same so we have a collective impulse on getting it right. I welcome what Meta is doing but we should remember, it's not fully open source in what you think of as open source, but it's close enough to do the job and we need counterweights to what the online closed models are doing, because long term, I think most of us will want to run A.I. at a local level on our own hardware because of privacy and security reasons, this will especially be the case as A.I. becomes better and more useful in more areas and once A.I. starts having long term memory so it can change and adapt to the user, I highly doubt most of us would feel comfortable with data going back and forth to a centralised service, and if that isn't frightening enough, wait till we have robotics around the house, these being run by online A.I. services would be a massive privacy invasion that would make the likes of Google and Facebook look tame in comparison, open source and locally run is the only way forward I think over the long run.
It's a way of working with the hardware companys to help push their upgrades ! . We have still not seen a metric , to prove when it's right to up the parameters past 1b or 3b .. It seems that they are generating large models to displace people from the technology . If you can't train it (unless you have the money to upgrade) ... As well as giving them time to add serial number identification to graphics processors ! Giving them the ability to track GPUs !!! Oh my gosh !!
Stay at 7b Keep training it for the tasks you need , stay with your model and keep training it. You will find your personal model can function better than these commercial models ! .. My mistral out perform the original mistral ? And they did not catch up yet either ? So it's only fine-tuning !! Get too it ! Specialized your own model and adapt your code base for the model , integrating your input output structure .. so you can enter and obtain any media !
Thanks for the walkthrough, quite helpful. It's just hard to think that mr. Zuckerberg has the best interest of people at heart, given the impact of his contribution to the decrease in wellbeing of young people.
He probably doesn't, but this is a win-win for Meta and for the open community. Meta have the resources to create these big models but the open source community have far more resources when it comes to testing and fine-tuning them to get better results over time, Meta benefits from the open source community doing its own thing on these models that Meta can take advantage off in its own products, so both sides benefit from this.
Calm down guys it is just out I am very happy with this I also think in the furuetae all want our own model rather than sending our data into a third party that is too much trust is unsustainable model
It did not cach up. Far behind! GPT4o/Claude 3.5 sonnet: 70.000 tokens in, prompt: Write a summary by chapter, and you get it. Llama 3.1? Just outputs jibberish. (Tested M2 Ultra, 192GB unified RAM, unquantized 8B + Q4, 70b Q4) - in comparison, the 3.0 gradient version was able to, but it didn’t stop, it then hallucinated further chapters. Cannot test 405b, not enough VRAM.
Llama 3 family is extremely sensitive to quantization. Try to run it in Q5 or Q8. Else use some API. This is not limited to LLAMA 3 family, any LLM with lot of tokens went into them are sensitive. Only the smaller LLMs like mistral and older llama are fine with quantization.
@@paul1979uk2000 this was not the point. The point was that llama 3.1 was not able to summarize a long content input. (But it was benchmarked). So how can they benchmark if the model produces jibberish?
Em I don't think you guys understand ! The code base did not change ... Hence it's only a finetuned model ! Or different settings ( they are playing with you and creating brainwashed AI models with bad settings ! And giving them to the public , whilst they enjoy a totally different settings and training set ! ... Unless the code base changes the model is the same !! The most important is the correct settings ! Hence 8b is actually incorrect ! The internal layer count and hidden sizes are incorrect as they are not dividable to binary values , hence training is a bitch and often unstable ! Remebr they used the common crawl first .. now they have used some structured synthetic dataset generated using guardrails and used this dataset to DPO the common crawl , hence after guardrailed the model is highly unturned ..once you get into the Hidden codebase and tokenizer you will find the sub prompts (alao adding abother layer of guaedrailing ) hence making the model tough to get your response no mater the prime pronpt ! . Changing the prompt or query that was given is summon t to intercepting the question and creating a new one ! It's a prompt in the middle attack ! I'm surprised nobody has exposed the hidden prompts in the llama tokenizer and hugging face library ! As well as the unsloth library and better transformer library as well as in the pretrained model !
Which provider are you using for it? Seems like it really dependent on that api you use. If you are running it locally, you need to check your settings. Llamacpp had some issues with this model.
@@engineerprompt API? LLM? No, I'm using it directly from their web site. It needed 3 tries to get a simple Windows batch script working, and when I asked it to tell what it knows about my hometown, it got monuments, location, events, tourist attractions, industry, history, etc wrong. I knew he was talking about the right town by confirming a few things correctly, but it got a lot of things wrong. I asked OpenAI and CoPilot the exact same questions, and they both got the script right in their first attempt, and their description of my town was almost flawless.
Thanks for the delightful review! We now know LLMs weren't OpenAI's moat. I wonder if GPT-3 was the pivotal moment for AI and "us," or if it is now, the release of 405B Meta open weights model?
@@engineerprompt It's so funny, just in a few hours a new model drops and you can ask such questions! What a time to be alive! I checked their blog and was surprised to see the license isn't open. What still makes me lean toward the 405B, besides its open-weight license, is the permission to use it to generate synthetic data. To me, that's the main reason to have such a large model these days.
@engineerprompt It's so funny, just in a few hours a new model drops and you can ask such questions! What a time to be alive! I checked their blog and was surprised to see the license isn't open. What still makes me lean toward the 405B, besides its open-weight license, is the permission to use it to generate synthetic data. To me, that's the main reason to have such a large model these days. Also, I'm not surprised anymore when a smaller model beats a bigger one on a benchmark, because that's exactly the goal of having a teacher model! To generate better data so a 123B parameter model can outperform the teacher. What do you think?
