I thought some viewers might be interested in a more technical video and the rest is hopefully still entertained by Neuro's commentary. For some parts of the video I had to remove the background music for copyright reasons (but the vocals are still audible as the AI tool can't remove those without impacting the dialogue).
@Robotwithtoomuchfreetime It's just short for Neuro-Sama Chronicles, which was my handle before, but YT changed some policies regarding fan channels (they need to be clearly identified as such), so I changed my handle to include fan channel.
It's indeed cool to see this kind of stuff. Neuro feels so much smarter when talking to Vedal while he codes, even when what she says isn't quite right, or is just reiterating.
@@AlexWeinerhe was also formerly of OpenAI and then left Tesla to eventually rejoin OpenAI where he's at right now. ChatGPT was just too interesting to him and he wanted to get back into academic research. The tutorial was made while he was on sabbatical after Tesla before being back at OpenAI.
Not building a model from scratch but training an existing one to act how he would like and respond consistently. However it’s still really interesting to watch!
@@RicardoVermeltfoort i’m going to, unfortunately spoil some of the magic here. Creating a large language model from scratch costs hundreds of thousands to millions of dollars depending on what you’re making. That’s because it either requires extremely expensive hardware or it requires renting out extremely expensive server time. There’s unfortunately no way someone like me you or even vedal can likely create one. we can, however, take a pre-existing model that is given for free that was created with grants and train those models to act differently, and be more like what we are looking for. That’s what vedal does, and what I do on my channel as well. In addition, there’s no way that vedal is running a model that is under 13b parameters. It might even be larger. Meaning, it is definitely an expensive one to make. He’s not taking shortcuts cheating or doing anything less impressive. It’s just how it works.
@@VegaLyrae The twitch chat model he was developing on stream is the one made from scratch. Obviously Neuro is way to complicated for such a thing but we can see his toy model go from outputting random strings of bytes to putting out approx English text as the video progresses.
@@moscacrackreina4457 i dont believe he's ever revealed that. as he says on his how to get started page. he shares some of his stuff but not all of it.
Vedal, don't you think it's a little too early to be having the talk with Neuro? I don't think the world is ready for AI that knows how to reproduce...
This was pretty much the end - I think he let the training run with 10k steps complete and showed the results again, but didn't change much of the code afterwards.
While I was in an AI rabbit hole, I found one in Play store that does what he's programming. Which words follow which and expected responses. I got onto the idea it would be better in a language with masculine and feminine syntax. IDK any languages so brick wall for me.
Is Nuro Samantha Vedal 'just' another'modern' LLM based chatGPT-bot tough, or has 'she' also inherited some other traits from her Great Aunt Eliza? Hmmm...
Internally the neural network does that, so that the output is differentiable. But the model you run has an extra 'layer' which takes the top result (or randomly chooses from the top 10 or something). So it's fine to say it predicts the next token (which requires argmax) or even predicts a sequence of text (which requires repetition and decoding), if that's the external interface you add to it
i think the biggest problem would be that she doesn't seem to have an over arcing goal, but rather produces a result after receiving a prompt. i bet if you very carefully guided her through the process you could get something that worked, but i have my doubts on how far she'd get if someone weren't there holding her hand.
Can anyone prove they're conscious though? You might all just be complex biological machines that claim to be conscious, for all I know I'm the only conscious being in the universe. Only reason I have to believe that the rest of you are conscious is that I trust you when you say you are. Beyond that the only one I can know for sure is conscious is me.
Can you prove you're conscious? Can I prove you're conscious? I'm not just trying to be contrarian, I'm genuinely wondering if that's even a possible task 🤷♂️ I kind of don't think so, because we don't even know what consciousness even _is_ yet
@@vraza14Is a 1 year old conscious? Before they've learned what consciousness even is, nevermind before learning how to even speak! Also it doesn't adrea he fact that an AI has to be created and that in some form as of now includes it learning in some manor and even if it's just learning our language, then that inherently includes some understanding of the definition of words like consciousness. It's a non-starter. Your method doesn't serve to meaningfully answer the question. There's no real answer possible as of yet and there's a reason why it's one of the big questions left in AI and the study of consciousness itself.
@@MrTuneslol A human also has to be created, and trained to understand their language, and everything they know. Humans just involve a more hand off and randomized approach to creation, and a much longer training time.
train her to use the dictionary instead of trying her to learn from google where less information from a dictionary. maybe use some Japanese culture from anime for cuteness i would say convo with brother and sister in amines first
