Integrate AI Characters with MetaHuman Avatars in Unreal Engine 5 | Convai Unreal Engine Tutorial

Подписаться 4,3 тыс.

Просмотров 6 тыс.

50% 1

In this tutorial, we’ll show you how you can use Convai to add AI functionalities to Metahuman Characters in Unreal Engine 5. Convai enables these Metahuman AI NPCs to perceive their surroundings, perform complex actions and even talk with you or other AI NPCs within your games and virtual worlds. Create amazing virtual experiences in UE5 with Convai today!
Sign up at convai.com to try it out.
Also, learn more about Convai's Safety Guardrails for AI Characters in our latest blog: convai.com/blog/safety-guardr...
Write to us at support@convai.com for any queries, bug reports, collab, or business-related engagement.
Convai plugin for Unreal Engine: www.unrealengine.com/marketpl...
Reallusion Guide: • AI NPCs with Reallusio...
Ready Player Me Guide: • Transform Ready Player...
Custom Character Integration: • AI Characters with Cus...
Time-Stamps:
00:00 Intro
00:45 Setting Up the Convai Plugin
01:25 Adding Metahuman Character to Scene
02:05 Adding Animations
03:18 Applying Lip Sync
05:30 Creating Characters AI Personality
08:13 Enabling Actions
09:16 Scene Perception
10:57 Changing the Character
13:52 Outro
Follow us on your favorite platforms to stay in touch:
Discord: / discord
LinkedIn: / convaitech
Reddit: / convai
Twitter: / convaitech
Instagram: / conv.ai
Facebook: / convaiofficial
TikTok: / conv.ai
ArtStation: www.artstation.com/convai

Опубликовано:

1 авг 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 54

@robot_collective 2 месяца назад

Thank you so much for sharing this tutorial and for your work on Convai! I am so excited to try this out in the next few days :)

@uno27heaven 2 месяца назад

Was legit JUST TRYING TO DO THIS TY

@convai Месяц назад

We are glad you found this helpful! Happy development :)

@robot_collective 2 месяца назад

So awesome, the body and face animations work great on my Metahuman in UE 5.1. and also the conversation works like a charm. You got a new fan here :)

@user-yh3rk6no2s 2 месяца назад

I hope you guys come out with more tutorials on integrating into VR templates!

@convai Месяц назад

Hey! Thanks for your comment. We have a tutorial out for setting up Convai into Unity VR projects! You can check it out here: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-aRHj6RXfErw.html

@RizwanYe Месяц назад

Thank you Convai team and the hard work you do! Please make more of these tutorials! Ill watch all of them!

@convai Месяц назад

Hey! We are glad you like the tutorials. Happy development! :)

@Augmented_AI 2 месяца назад

Please add in the ability for gpt 4o vision. So it can see dynamically what's happening in the scene

@mattbeets Месяц назад

As a total n00b in this, I thought I was going crazy because my ConvaiBaseCharacter's edit blueprint looked nothing like what's shown on the screen @3:04. But somehow I missed that I had to click on "event graph" (Graphs section on (my) left part of the screen) under graphs inside the edit screen. Just if anyone else was confused like I was :)

@mattbeets Месяц назад

Maybe also worth mentioning that using ElevenLabs voices currently requires first a subscription of 6 or 9$/month for Convai, and also a 99$/month subscription to ElevenLabs

@NostromoVA 2 месяца назад

Amazing!

@zimingpeng8418 2 месяца назад

So exiciting! this is a very brilliant tutorial and plugin! I was wandering if this funtion could be packed like a Dektop-program or game version that user can play it with interactive hardware?

@convai 2 месяца назад

Hi @zimingpeng8418 this is definitely possible, simply package the project as normal and you get a playable app.

@zimingpeng8418 2 месяца назад

@@convai cool!!! Thanks! i can´t wait to try it.

@LivioAscione Месяц назад

As soon as I run the convai game mode and try to send a message (vocal or text) ue5 crashes on Mac OS. Maybe a problem with mic permissions?

@xgtwb6473 6 дней назад

Can i run this all on a server using python and render to video?

@NetZeroEarth 2 месяца назад

🔥

@schmutz06 Месяц назад

nice work! i spent a while playing with this earlier with metahuman, in Unreal 5.4.2 (latest stable). I couldn't get the face sync to work unfortunately. I am certain the correct anim class is set to the face, and I have the ConvaiFaceSync component in the stack (tried it in all hierarchies; under ConvaiChatbot, under Root, under face). I was unable to get the navmesh / follow action to work either. The character responded but did nothing. It would turn and face me half the time, and otherwise only look forward. After around 1 hour of starting out, I hit the usage limit. Is it possible to use Ollama / Llama3 local with this to avoid the limit by running all the generative parts locally?

@SethTodd-rl6hr Месяц назад

I am also trying to get a ConvAI metahuman operable, but in 5.3 . Mine also agrees to do some action, like move, but then never does it. Object specified in the details panel so not sure what the issue is.

@SethTodd-rl6hr Месяц назад

I have been trying to get a working MetaHuman with ConvAi in UE 5.3, but with VR. Firstly, I notice that the text box does not appear in the view inside VR (Vive Focus 3). The convai chatbot can respond to input, but the input doesn't show up. Any idea why this might be?

@aw92-v1 9 дней назад

Hello - great video! Previously, I remember there was a built-in mechanism where the actual SB_Main chat appears only as you're nearing or facing the ConvaiBaseCharacter we added in the scene, then it disappears when you face away. How can I get that feature back, please? I do not wish to continue seeing the dialogue box as I move around and interact with the rest of the 3D scene, it clashes with my existing UI widgets. Can you please help me here or guide me to a video or a thread if this has already been addressed in the past? Much appareciated.

@convai 5 дней назад

Hello, We are sorry for the issue you are experiencing. If you wish, please join our Discord server, and our developers will be able to assist you in resolving the problem. discord.gg/Tm9b3FWFJG Thank you!

@Espinosa_render 2 месяца назад

why my metahuman when talks just has an idle pose? he doesn't use the talking animations...why? Thank you for the plugin, is really amazing

@convai Месяц назад

Hi Esinosa_render, thank you for your kind words, the talking animation is a bug that we solved in the latest 3.2.1 version of the plugin, it is not yet out on the marketplace but will be out along with other features and fixes.

@michaelb1099 9 дней назад

is there a way to fix lipsync as it is really off? i am using a reallusion character and not sure how to fix in convai

@convai 5 дней назад

@frunobontes Месяц назад

Are the actions not functional on 5.4? I cannot make it move or follow me.

@convai Месяц назад

Hey! Thanks for your comment. Actions are definitely functional on UE 5.4. You can check out our detailed tutorial on AI NPC actions to learn more: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-phAkEFa6Thc.html

@zhengdonghou178 Месяц назад

In today's test, I found that ConvaiFaceSync in metaHuman doesn't work, and as a beginner I don't know why this is?Maybe someone knows?

@convai Месяц назад

Hi @zhengdonghou178, if the ConvaiFaceSync component is not correctly causing lipsync to work, then double check that you have added the correct face animation blueprint over the face skeleton in the character.

@mikebert 2 месяца назад

is there a way to let the conversation box disappear while talking?

@convai 2 месяца назад

Yes! the conversation UI supports a wide variety of customizable options, and it's even possible to hide it entirely while talking.

@NotFedBait Месяц назад

@@convai i cant find this option, do you have a quick guide? please and thank you.

@twjohnsontsai Месяц назад

Hello, after setting up the Metahuman character, in the game, his eyes always wander and cannot focus on the other person. How can I fix this?

@convai 2 дня назад

Hello! We have a tutorial dropping on customizing metahuman animation blueprint dropping very soon, which dives deeper into the character interaction logic, eg: Character looking at user while talking, character looking away while being in an idle state, etc. Will share it here, and it should be able to help you out! Meanwhile, we recommend you to join our Discord server, and our developers will be able to assist you in resolving any challenges you are facing: discord.gg/Tm9b3FWFJG Happy development! :)

@DIVERGEsound 21 день назад

lol the second comment at the end

@damingy 2 месяца назад

Looks really cool, but I'd imagine just a few players actually utilizing this within a game could easily hit the monthly interaction limit.

@devasCabral 2 месяца назад

When will plugin compatibility for Unreal Engine 5.4 be available?

@convai 2 месяца назад

Hello @devasCabral, Our plugin is compatible with Unreal Engine 5.4. You can download it from our Unreal Engine Marketplace page or access it via this link. www.unrealengine.com/marketplace/en-US/product/convai

@ErikMalkavian 10 дней назад

Is this in Real-Time?

@convai 5 дней назад

Hello @ErikMalkavian, Yes!

@SanjeevKumar-be4dd 2 месяца назад

i'm not receving any response from metahuman after asking questions

@convai 2 месяца назад

Hi @SanjeevKumad-be4dd, is there any errors that pop up in the chat widget? you can share this issue on our Discord channel: discord.gg/uTfQqzUnhf

@SanjeevKumar-be4dd 2 месяца назад

@@convai actually i did receive response from face but hand animation not working tried so many times

@MrBastos2010 Месяц назад

Is it possible to use another language besides English?

@convai Месяц назад

You can select a language for your AI NPC from a wide variety of options that are available on the Convai Playground. Happy development!

@chris09423 2 месяца назад

I wish you dont need a metahuman for this, Is there no way to use your own character?

@convai 2 месяца назад

You can definitely use your custom characters! Check out our tutorial on integrating Convai with Custom Characters: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-69ueqmUShq4.html&t