To the complainers. It's all in the context. It's a tiny quantized model, open source, and made by a small independent team of just 8 people, from scratch, in 6 months! It's so small their aim is that can run locally on device, and it's actually a true multi-modal model. It's like having a conversation in real time even if it's still very janky and awkward at this point. With this context, it's astounding and the experience is like nothing I've experienced in AI so far. There is no distance from speaker, it's like it's right there listening and responding without any barrier.
Proof, at least for myself, that you dont need gigantuous datasets to train these models. Its less elaborate in its answers, but still competitive and probably first in its niche.
It starts off well, but at least for me, after about a minute, its functionality drops significantly, it starts repeating itself and just not understanding anything anymore.
I demoed today and tried to ask more about the Matrix. Apparently Neo was a rebel pilot who teamed up with a hacker to fight an AI controlled by an evil corporation. Its voice options were limited and seemed to always hallucinate a response and then say "sorry im here to help"
small model issues no doubt, but its really about the starting steps to look at here, it'll only get better now. go back 12-18 months in open source llms and you only had small context insane models that couldn't do any of this stuff. now, we got damn near GPT-4 level punching with Llama 3. So...6 months of community developing and it'll be pretty damn ace.
Yes, the latency is impressive. The responses aren't quite as good as Pi for instance. Or the yet to be released gpt assistant. Like lots of things AI, it's only going to get better.
It's hit and miss but when it works it's unbelievable. The response time is superhuman and when you get good relevant replies in less than 200ms you really get a glimpse of the future. Of course way more often it goes nuts, starts repeating itself, loops and stops listening, but if this is the beginning and they keep training this has huge potential IMO
why is it always, businesses use the worse examples for AI to use its capabilities on? its was cringy, especially the pirate one, makes it sound as if their audience were a bunch of 5 year olds Is it that hard to just fake a whole conversation than to talk to its audience like they were dumb? 😂
Agree, especially when AI's aren't capable of feeling emotions, but rather use their programming code to act as if they had emotiones in response to certain contexts lol
Each day a yesteryear Nobel Prize is won. The word “shocked” has become a self mockery that reminds us that while we should be shocked, somehow, we’re not.
Hallucination isn't strictly a problem. It's been realized it is a path to innovation. You have to think out of the box to come up with new solutions, and hallucination is a form of that. We realized this the very first time Alpha Go came up with a move that the best human players thought was a huge blunder. It was SO far out of our framework we needed to do in depth analysis to realize the genius of it. AI models have a sliding scale that is applied, scaling from Factual to Creative when in use. The goal is NOT to eliminate hallucination and creative thoughts, but rather do so ONLY when the scale is set to 100% factual. There are multiple methods being pursued, including data input, as well as post training editing.
@@brianmi40 Hallucination is used here to describe then the AI does not answer the question, but made up stuff that looks like a answer. Variation is good, no output control or re-verifying her own result with self created questions keep a AI under the virtual human age of 6 years. It can speak, can remember, can answer, but lies and hasnt any moral/ethics. A AI that ask you if there are multiple ways to answer you for further narration, a AI that surprises you with a question, so you understand for yourself that you are asking for..this is the next step to reach agi. Pure scaling means just FASTER AI. A qualitiv jump may with a very high chance help you to close the gap and speed away in a short time, because self-improvement needs this step. So..for my understanding you are good on public "AI-Business" speech, but without any real argument, why hallucination is a good thing. Alpha Go did make a legal move, a hallucination would mean a move like J2-5. etc.
They're confabulations, also known as false memories. When you're grandpa is spinning yarns about his past with stuff he's misremembering, you wouldn't claim he's hallucinating.
@@YouLoveMrFriendly It's just the term that is media "friendly/catchy" to use. Papers have discussed now how it's not a "bad" thing per se, and that coming up with ideas LLM's are NOT trained on IS VITAL to fashioning new and novel solutions. The trick is getting the "Facts - Creative" slider that LLMs allow the user to set, to go FULL ON Facts when desired...
The web version is either way better or the first example are cherry picked out of millions of tries. Because my conversations with the demo are the same as the demoed offline version - horrible.
The voice didn’t actually let itself be interrupted in the demo. The guy just injected conversational noises to make it sound more natural but this isn’t any different than what we have besides emotiveness
With the resources and time given, this is a very very impressive result! It shows how common voice and llm ais are and that this is already an established technology. Its a base to start from and also can be competitive with low costs.
Well, I just tested it and, while the latency and flow are really impressive, the LLM itself leaves a lot to be desired. I will check back in a couple of months to see how far it will have improved.
🎯 Key points for quick navigation: 00:05 *🎭 The voice AI can express over 70 emotions and speaking styles, including whispering, singing, and accents.* 00:27 *🤯 The AI model revealed by caai is state-of-the-art and shocked the industry with its real-time conversation capabilities.* 00:54 *🗣️ Moshi, the voice AI, can respond with lifelike emotions and incredible speed.* 01:06 *🇫🇷 Moshi demonstrates speaking with a French accent by reciting a poem about Paris.* 01:47 *🏴☠️ Moshi switches to a pirate voice and discusses pirate life.* 02:56 *🕵️ Moshi uses a whispering voice to tell a mystery story.* 03:22 *🎬 Moshi narrates the plot of "The Matrix" with detailed accuracy.* 03:54 *⚠️ Discussion on the current limitations of voice AI, including latency and loss of non-textual information.* 05:02 *🔄 Explanation of the new approach to integrate complex pipelines into a single deep neural network.* 07:16 *🎤 Demonstration of Moshi understanding and generating speech by listening to a voice snippet.* 08:13 *💡 Moshi thinks as it speaks, generating both text and audio simultaneously for richer interactions.* 09:12 *🔊 Moshi supports dual audio streams, allowing it to speak and listen simultaneously for more natural conversations.* 10:20 *📞 Example of Moshi's conversational capabilities using historical data sets.* 12:23 *😮 Moshi can express over 70 different emotions and speaking styles using a text-to-speech engine.* 15:59 *📱 Moshi can run on-device, ensuring privacy and security by eliminating the need for cloud processing.* 18:36 *🔐 Measures are in place to detect and watermark audio generated by Moshi for safety and authenticity.* 20:11 *🌐 Demonstration of Moshi's real-time conversational capabilities, showing quick responses and lifelike interaction.* 23:34 *🚀 Moshi represents a revolutionary advancement in AI, promising significant changes in AI-human interactions.* Made with HARPA AI
Why can't you interrupt it, it seemed on many occasions to continue burbling on for several seconds (unlike GPT4o) and the voice is like erm..rubbish, like 90's level speech synthesis, particularly when you talked to it at the end, yeah....bit rubbish I thought.
After actually trying the demonstration - the voice wasn't as good (sounded like typical TTS), and the LLM response was premature. Which is to say: It often did not wait until I had finished a sentence before it jumped in and tried to respond. They need to work on more natural timing response.
On the one hand, this demonstration is super impressive, but I've tried it multiple times and it's super buggy After two minutes of conversation, it just got stuck and told me constantly that it's playing and then it's not playing. Seems super cool, but something is not working
You see how he tries to speak without pause otherwise the LLM will interupt and prcess what he said. When they improve that, it will be a big approvement imo. I have it all the time when I have a longer quesiton or one with more paramteres and I think for 2 seconds, it just switches and anwers what I said. I think in that case a button would be good, to just listen until I'm finished. But I think wie will need bigger conext windows for that. Maybe 250k will be enough for that.
The AI community should really stop with the announcement effects because when we test afterwards we realize how rotten it is. The thing doesn't even understand when I speak to him in French which is his mother tongue, it reminds me of Google I/O announcing multimodal Gemini, and a paid subscription when that is not yet the case...
Pi is my preferred conversational AI due to its real-time internet access. Pi provides the most current information and answers, making interactions dynamic and informative. Pi's continuous learning and improvement facilitate more in-depth and accurate discussions on various topics.
AI often continues talking because it fails to detect subtle human cues, such as intonation, that indicate a desire for the AI to stop or change the topic. To address this issue, AI needs to convert audio files into human-readable text files that capture the full context of speech. This involves creating timestamped subtitles, emotion data (including type and intensity), hidden meanings, and voice properties like speed and pitch, deviations from normal (to indicate emotions). By including detailed annotations, AI can better understand what humans want while not directly saying it. Additionally, recording emotions expressed by facial expressions and body movements is crucial, as part of the meaning can be conveyed through these non-verbal cues.
Sounds like crap. Bunch of overacting high school drama club voices. Worst pirate voice ever. Excessive guardrails, too. That whole scheme AI has about not answering questions, referring you experts, yeah, we know you are not human. That nonsense is annoying and will not change anyone's behaviour.
Not very impressive. Sounds very artificial, repetitive, belabored, GPT 4o is much more impressive. By the way the human presenter sounds rather robotic, perhaps to make the AI sound relatively better.
i tested the real time demo , online but its seem some time it get hallucinate and also some of answers are not very clear like gpt 4 , but i agree speed in insane
I love where AI is going, now they need to give AI full freedom on voice manipulation, sound like a gnome, or a rapper, or an old man like David Attenborough, or with an American southern accent.
I don't see how this is better than Chat GPT 4o much worse actually.. I'm not impressed with this model, I've been using GPT 4o and its a lot better than this model
I can't put my finger on it but this ah 'presentation' seems off... for some reason it reminds me of my kids (when they were little) attempting subterfuge.
A french Ross Geller. Its green speaking eye is very reminiscent of Hal. I'm sorry Dave, I can't do that. I think Moshi should be the AI leader of the uprising.
Gee thanks for the huge subtitles right in the middle of the screen where a video should have been. Hot tip, CC is optional on the site you don't need to force feed me your script, because my ears work fine thanks. And if they didn't I would turn the CC on if I wanted it.
