World’s Fastest Talking AI: Deepgram + Groq 

Nice!

yeah they have a ton of good data to work with

Hi is this completely free and opensource?

Cool that sounds good, please share when it's out

Nice! Good luck! What is the project you're building?

@@DataIndependent I didn't started yet. But I will start it soon.

Same, I want to try to do translation (language identification per speaker / switching whisper model could be tricky)

nice to both of you!

@@DataIndependent I ended up pulling down your project and ran into a few issues, fixed them and made a few improvements like response interruption. It's pretty good but groq being limites to llama 2 and Mistral is a shame. It is at least able to fill the time of latency of the more powerful models so that at least it still feels more natural even if the first 10 seconds are filler until openai or Claude 3 ingests input and starts streaming.

@@avi7278 That's awesome! If you were open to doing a PR to add interruptions I would definitely want to share it with the community. Did you add filler too?

@@DataIndependent it's on my list for this weekend, cheers, appreciate the starter.

Nice!

you can only have somewhat close to RT with Google or Deepgram, you should go with on-device

Ha thank you!

Awesome - excited to see what you build

Big time Cam!

This. I need something like this for a project, but I'm not very familiar with Groq or Deepgram yet; just starting to dig in. This thing starts responding with the first little pause, so it constantly cuts me off when I'm just pausing momentarily to think of how I want to phrase the rest of my sentence. If it wants to send data at every minor pause in order to understand context, predict the full query, and begin formulating a response, that's fine-- but it needs to wait until I've finished my entire input before verifying/sending its response. Out of the box, this is like a person who doesn't actually listen to what you're saying and is just waiting for their turn to speak. Is there an easy way to affect the response times and/or understanding of when the user has finished a full thought or do I need to develop logic/rules from scratch?

@@frothyphilosophy7000 not that I’ve seen. And this would be a massive leap in improving conversation. It literally just needs a small model to parse the text at every pause and see if it’s an appropriate time to interject. Just the same as a polite human would do. The groq api should be able to do it. I’m really surprised we haven’t seen this effectively enabled anywhere yet.

@@HideousSlots Gotcha. Yeah, guess I'll need to implement that, as it's unusable otherwise.

I'm not familiar with that - thanks for sharing I'll check it out

I’m in the same situation, got the listening and response working great but no TTS. Spent all day breaking and fixing it 😂 still confused why it won’t talk! Great guide (I needed python 3.11 to get I back working and had some issues with dotenv, ended up hard coding the APIs) - it’s a great piece of work! 👍👍

Try these updates github.com/gkamradt/QuickAgent/commit/21ae2b0e286759e186e12a76addd250a5a491381

Nice - what kind of latency are you getting with those? Along with accuracy

There is also Piper for tts, it's open source and wirks realy well in my non-gpu laptop

there is a caveat with ur solution becuase fatser whisper does not detect silence out of the box and hence, you wouldnt know if the user has finished talking or not, which Deepgram does.

Nice!

Yeah you could for sure. It would take a bit of work back and forth but it’s there

Right? Ya it was super cool

I haven't tried that out yet - sorry!

Same issue here. Hoping for an answer, too.

Try again with these changes github.com/gkamradt/QuickAgent/commit/21ae2b0e286759e186e12a76addd250a5a491381

My guess is no - oss usually isn't as fast as paid

I was able to sneak early access for the video

More coming!

API's being found either!

Thanks for this - I found this same problem. I had it working in beta (before the video game out) so these were the changes needed for their prod version. Updated!

@@DataIndependent Thanks Greg! Awesome content! I appreciate you!

Can you link it?

ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-pRUddK6sxDg.htmlsi=Kg15nRUEEr1AHTGx

Nope not yet - let me know how it goes

Debug the response from TTS without streaming first. I had to change the URL (it was some beta URL), they have changed it now and the one in Github right now returns an internal error (non 200 response), so the audio won't stream.

Both good points, I'll update the URL at least

@@DataIndependent I tried it on Ubuntu Linux and (I think I) also have a problem with ffplay. However, somehow while it does convert my voice to text and LLM responds with text it does not convert text to sound. To test it I did different checks to verify that the connection with Deepgram is working. I used a test script where I used the Deepgram API with a text message and then I streamed the audio directly back to ffplay for playback and it worked. So that is why I am puzzled why the code is not working on the Linux side. Any help is appreciated. I truly enjoy your thoughtful tutorials and videos.

There was an issue with payload. When I commented out the "voice": self.MODEL_NAME it worked. I also changed to pygame from ffplay on Linux. But that may not have been the issue.

Put it in the LLM step, set up a retriever and then call to it

@@DataIndependent Yes, thank you very much. vectordb = FAISS.from_documents(all_splits, embeddings) retriever = vectordb.as_retriever() BUT LLM doesn't accept retriver param, So I used ConversationalRetrievalChain

I’d also love to know this. I’m in the same boat! It works .. just silently for me too

I updated the code, try again or make these changes yourself github.com/gkamradt/QuickAgent/commit/21ae2b0e286759e186e12a76addd250a5a491381

Like what kind of shenanigans?

@@DataIndependent Starter difficulty Steamlit app using Langchain and Deepgramdeepgram that allows you to upload a pdf / embed then when ready we can simply chat with it live , asking a question , retrieving information Advanced difficulty / Aspirational An agent using function calling and a set of tools like web search / calculators etc ... if we can have an asynchronous continues conversation with one supervisor agent who asynchronously can "ask" other agents to do time consuming tasks, like being able to talk to the supervisor agent in Langgraph schema

Haven't tried that yet

Vapi is extremely expensive. I think in a couple of months we should be close to Vapis performance with open sourced models and tooling.

I had this problem a while back too and it was super annoying. The solution was a lot of googling and I think I might have even used it as a push to upgrade my python

@@DataIndependent This doesn't look good for me then, hhhhhh, was the solution obvious or did it require a lot of manipulation?

You trying to use their python-sdk? Maybe try "pip install deepgram-sdk"

Yeah - the suggestion here is the key github.com/deepgram/deepgram-python-sdk?tab=readme-ov-file#installation

Yep it would… But unfortunately lawyers are probably done for… like 99% of them.

It'll help augment roles ha, not replace

Ya…you’ll need an API key which I can call out better in the read me

I have my own api key with 300$ credit. Still it's showing this error Can you please send me a screenshot of your .env(hide your api)file​@@DataIndependent

I have my api key with 300$ credit but still it's showing this error massage. So can you please send me your screenshot of.env( hide your API)​@@DataIndependent

Have you tried this one? www.hume.ai/

Same here..

aww

yeah - it'll be tricky to navigate but there will be use somewhere