How to make an AI VTuber Using GPT 3 and Google Cloud TTS 

can you maker a follow up on how to set up vlc? i am having trouble with this

If you add voice mod to the mix it would be amazing and if you get a anti AI text rescripter then your making free money

@adi-panda Hi, thanks for the awesome tutorial! I've made sure that all of the input/output settings are done correctly but I still can't seem to get the audio to work. I set the Vtube input to VoiceVolumePlusMouthOpen and output to ParamMouthOpenY, just like the tutorial. However, when I type in the chat, no sound comes from OBS studio and the Vtube model mouth does not move. (I also don't get how you would listen to the bot speaking if you set the speaker as the virtual aux) Also, I connected my Twitch account to Vtube Studio and OBS but nothing happened, so I guess I'll disconnect them again? Any help would be greatly appreciated!

How do I feed llama outputs into my output folder..

This tutorial is so much :)

For a chatbot like Neuro though, you'd need a local large language model, as the responses need to be fast. And GPT3 and ChatGPT will go "I'm afraid I can't do that Dave" too often to have the interesting responses Neuro has. ^^

@@2DReanimation to have a local large language model you need a local GPU farm with enough VRAM to fit it all, and stuff on the level of GPT-3 will require you like 300 gigs of it. Which is not feasible. When you use gpt3 or even gpt3.5 via it's api, it won't do "I'm afraid I can't do that Dave" whatsoever, it's your responsibility to filter the responses so they don't break openAi terms and conditions. Edit: not even just 300Gb, 700Gb of vram for gpt3-like nets considering it's 175B parameters...

​@@2DReanimation Nope. for training you would need even more. 700 Gigs of vram is for inference (just running the model with some inputs just to get some outputs) on a GPT-3-like model. LLAMA models are smaller than GPT, yes, but even those are far from tiny. The smallest LLAMA model (7B params, which is considerably worse than gpt on this specific task) needs at least 14 gigs of ram, which is indeed possible for a consumer rig. The proper 65B param LLAMA model needs 130 gigs of ram, which is technically possible if you will run it on the cpu, but for gpu that's already like 10 GPUs worth of vram. Also have you actually tried doing any of that yourself? I'm speaking from experience saying that using the APIs is actually faster than running the net locally, i get 1-3 second responses with the API on average, which is perfectly fine for answering twitch comments. ChatGPT is GPT running with a specific prompt before each of your messages, and openAI's filters before and after. ChatGPT is not a good example of the limitations of GPT.

​@@2DReanimation He did say GPT 3/3.5 API, not chat GPT. I haven't tried that yet so I can't confirm or deny, but I believe that if you skip the chat GPT added functionality and just go with underlying GPT then you have more freedom in those regards

@@alexlarex7773 I've been testing with Vicuna and it's only taken 20ish GB of space. It gets a little confused and starts writing fanfictions halfway through some conversations, but it's still funny.

If you tried it Bro can you telling me what's your Instagram account I want to ask something about this..? Please

thanks man

Fr

How can we do that? total noob here, i'm trying to use chatgpt to figure it out but it's still way over my head. ty

how Do you change it

For an update on this, the git repository now uses gpt 3.5 turbo

Hay mucho contenido en español solo que está todo por partes. No hay un tutorial en especifico w

Knowing your work, I'm curious and excited to see where this goes...

Ya want to share the changes to GPT3.5

yeah you have to change the code though, in the description I put a list of available voices

​​@@adi-panda does the language of the TTS voice use have to be the same as the language that's being used in chat such as if a chinese user is saying Chinese would you then have to use one of the Chinese voices or if English users are using all English would you therefore have to use only English voices or do all the voices basically translate to the same thing meaning it doesn't matter what language you're speaking the language of the voice will still be able to be used.

sorry lol, I tried my best, but this was only really a quick project lol

I am so lost. How can something that seems so simple be so complicated. i just want it to read the most recent chat message instead of reading every chat message top to bottom like a list :(((((((( How is this not something someone has wanted before. Why cant I find anything online

For anyone else wanting this I did finally find how to do it. Just needed to add if not message.content.startswith('-'): return under the async def event_message(self,message):

Though doing this in notepad is very difficult as you have to indent manually

From the current documentation on OpenAI is not possible to fine tune the gpt 3.5 turbo (at the moment of this comment)

same dude, if u have the solution plz tell me ;)

Pretty dang sure it's possible, as you're just streaming from OBS which in turn captures all the input. You'd need to get a youtube API key and modify the code to also connect to youtube using the key. You could even modify the script to take an input parameter so you can default to both but specify if you want to when you run the file. Perhaps youtube has some crucial limitation that Twitch doesn't have that would prevent running this, but I highly doubt it

In main.py at line 74-76

open AI whisper will create a transcript of what youre saying and then you just have to grab that as input somehow, but there will be a lot of latency