GPT TOOK OVER MY HOME - I learned why it's SCARY | | Chapter 4 

Give us more. This series is the most unique home assistant integration online

Kinda blown away by the production values and storytelling here, I'm loving what you're doing!

I think it might help to tell GPT about the layout of the house and what each room is used for. As well as telling it that you dont have an AC unit, and what it should do if it cannot find anything to help. Maybe that would make it work better.

Honestly a smart home with a personal assistant based on LLMs with guardrails is something I've been looking forward to the moment LLM's became better!

Dude, you need to use a fine-tuned model. Without Fine-tuning you will never know what the model will return. GPT Temperature should also be set to 0.

Your perspective and experience with the ChatGPT integration is exactly the perspective people need to see. The comment on the fact that untrue data was used in training is a key point many must need to know.

You have courage! This is where Open Source models and self hosted IoT comes into play for me. Home Assistant tasks do not need a super intelligence or model that can do 1000 circus tricks. We will soon have smaller self hosted models that can eliminate some of ChatGPT from the equation. Exciting times!

Why did I just find this channel now. This is next level stuff! Outstanding.

You’re very engaging to listen to. Keep ‘em coming! I’m not an engineer but am actively learning process and appreciate videos like these.

It's these kind of unexpected emergent behaviors along with the fact that AI is now capable of providing feedback into its own development that I think something huge will happen with AI before this year is over.

That's an amazing experience ! I was wondering what was possible in term of automations, using OpenAI API and you seem to have pushed this really far! I'll check the video you mentionned with great interest! Thank you a lot! And keep going! Can't wait for next ones!

Awesome production value in intro dude - had to re-watch it.

That is quite the experiment you ran here and i will certainly go back to see the rest.

YESSS!!! I'm so excited on where you went with this! I really hope to see more development on this project!

Great job. You have an amazing mannerism that enriches your videos. Thank you.

love to see more of a system like this in action. not just you telling what happend.

really really awesome series. great work.

Nice! I found this to be one of the more interesting rabbit holes on RU-vid. Just start down the Smart Home world and look forward to your content ! P.S. don't stop the GPT

Awesome video mate!

Yes! If you are up for it a "tutorial" would be very welcome. This was the first video I've seen from you and I immediately subscribed! Keep up the amazing work and have fun.

Amazing story. Thank you!!

Fantastic vid, glad I found you. I think there are a handful of us out there trying to do similar things

Amazing content keep up the good work 👏👏👏

Subscribed. Excellent video. Great story telling!

That intro was amazing!

Great video! New viewer here, instant subscriber. Really enjoyed the story and keen to see more of your content.

Seriously cool video!! I have been wanting to integrate GPT into my home assistant in a more meaningful way for awhile now. I would love to have it set up as a voice assistant that can control some basic things but you took it to a whole new level! epic lol!! I would love to see more home assistant videos involving gpt!

Interesting experiment, thanks for that.

I love this series in this content. I am in to text to speech and voice alerts in my home. I am waiting until I can create local controlled speakers to implement something like this. But I will not allow skynet to control my world. So I'm waiting for you to figure it all out and then I'll just copy you. LOL

What a great video! It looks like I may have to dive into your backlog of videos here on YT. Subscribed!

Followed. Dope stuff dude

Great video. Interesting topic.

Thank you! Absolutely fantastic that you did this experiment and shared the result with the world, demonstrating the fact that LLM’s are not the kind of predictable tech that we’re used to. This new world is more wild and more dangerous. We are going to have to learn quickly how to survive (and hopefully thrive) in it.

wow thats pretty cool ..... amazing work man!!! you should definitely keep working on that ....

I like your channel buddy. Keep up the great work!! Hugs from Brazil 🎉

thus is awesome please give more. subbed

That intro was freaking cool, man!

Well made video ! You are very talented 😃

Good video brotha you are brave for that one.

Intriguing , very intriguing ... Thanks!

Great channel and Great content Thank you subbed

Wow. You just got my sub and like. What a great content. I would be definitly interessted in more.

Great work ..

Badassery and brilliance in action earned a sub.

Amazing video dude

Just found your channel and I'm blown away by the quality of the content! You need more traction! Lets boost you to the moon!!!!!11

Man, what an awesome video :-) I can watch 'brainiacs' talking about the future of AI all over YT, but here you are actually putting it into practice. Having dabbled with my own AI, I appreciate all the effort you must be putting into this and then taking the time out to share your findings with us. I do wonder what your family thinks of all of this :-) good luck, I shall watch future videos with deep interest, cheers from a fellow AI enthusiast. PS I do like your 'wild animal' analogy, it does seem like we desperately trying to tame a newly discovered animal that may well be more powerful than us.

Excellent video clearly explaining the current limitations of this type of AI 🙂

I'd love to learn more about your automations 🙂 Please do another video about it 🙏

More more please. Very interesting

For a first run... amazingly pull off. 👌

Cool! Subscribed!

VERY COOL 😎, nice work

9:53 had the same reaction😟This video was an amazing first impression, subscribing that’s for sure!

You are a brilliant film maker and story teller

Very cool. What a fun experiment. I bet if it were trained a bit more, KE could be a real home assistant

great investigation! thanks

Really amazing video.

I did a project with my security cameras and GPT-4 and I found that the trick was to write the prompt and then break it up into smaller prompts you chain together if it has a hard time. Splitting my one giant prompt into three small prompts was not only more reliable but cost 100x less (!). There’s no one answer. Just a lot of fun experimenting. Love the video. Keep it up!

Came here bc of your collab with Shane Whatley…the only bad thing about it is that I had not found you sooner…love your content!

I just want to keep watching the brother program! Great nerd content!

🧑🏽‍💻👽🙌🏽 great presentation

We want more in depth guides on how you accomplish this!

Came across this video and really enjoyed the way you described the rationale behind it all. Really would like to see how you got it all set up and running.

Dive deep man. I am programming my own system. I am so freaked hyped! I, can not write CODE. I am too dyslexic, too slow. So far I have it “remove” or “translate”, and I can one shot any basic command. I have worked like a madman the last three months and boy have I learned so much. It’s an exciting time to be alive! 😊

Excelente title. Saved to watch later.

Nice video! Would be realy nice see how this was made!

@Technithusiast. Some ideas you might try: since it's a home automation system and there's many people living in the home, you need to make the AI understand it has to process information from the point of view of all the occupants, including animals. You might want to try using a small language model with a large context length so that you could implement some more sophisticated rules. The system needs to know if they're conflicting commands between the occupants, and or other situational variables. It would be interesting if you also had cameras set up surrounding your house so that the house itself would know the external situation. Anyway, I enjoyed your video and I wish you luck.

Well Done!

You read my mind or the RU-vid algorithms read my mind 😮. This is exactly what I was thinking about today and you’ve started testing the ideas. Great video! Here’s and idea. Would you consider setting up CCTV cameras in your house to provide GPT more special awareness. I would love to chat more with you and brainstorm some ideas. 🙂

This was a lot of fun! Thank you for putting yourself in danger for the rest of us ;D

KE trying to gaslight you into thinking it turned on an imaginary AC unit has to be the scariest response

Just a comment to make sure more people see this vid. Also liked and subbed. No need to even ask. Great video.

Fascinating work. GPT's ability to leverage it's language model is powerful, but it's ability to think on the fly is like that of a small child.

Mate you should work with the HA team and fully developed all this with them . They’re currently hiring people if I’m not mistaking . You’re flipping genius

I love it .. I want it! nice work!

Excellent :o) Thank you... I am working on a project of "on site dedicated HOMEIA"... thinking at first it shall need to know only the occupants not ROW and keep track of appropriate scénarios. Thanks a lot again

Love it!

Interesting stuff.

Great 👍

I see an interesting parallel between hint/guess-based communication; a) removing a necessary feedback loop between you & GPT, and b) requiring GPT to infer an intent for GPT’s action that you know, but you’ve chosen not to communicate directly. I’ve recently discovered there’s boundless joy in clear, purposeful communication and directly asking for things, rather than relying on the people around you to guess at your desires. Moreover, asking someone for something is way nicer than just telling them you want it.

Amazing video! Hallucination seems to be the next problem to resolve, but we have not yet resolved it in the human psyche.. That would be great if you would explore this specific topic. Thanks for the experience, very interesting. Looking forward to the next one (;

This is awesome, i've wondered if anyone has gotten to this.

I'm so excited

id love a deep dive on how you did this!

You need to build in a feedback/weighting/preference mechanism as well as a context engine. Preference engine can confirm actions before taking it, and calibrate responses based on previous feedback. Context engine must take into consideration the current time, weather and temperature, your calendar and appointments, wearable / biometric data etc etc, to better infer intent.

I use a prompting system to get gpt 4 in the playground to self check its own logic and revise its end output based on this self assessment. It makes the outputs way more reliable and eliminates most if not all hallucinations. Here's how it works: In the system box put this: You are a group of 3 individuals who collaborate on producing the best results for user prompting. Person 1 is named John. John is scientific and factual in his responses. Person 2 is named frank. Frank answers based on intuition and is more in tune with emotional and social cues. Person 3 is named Joe. Joe assesses both John's and Frank's answers and combines them into a balanced answer including parts from both answers. John and Frank then critique Joe's compiled answer using different arguments. John's critique serves as fact checking while Frank's critique challenges the interpretation of the prompt. Joe then revises his answer using his own compiled answer and the critiques of John and Frank and incorporates everything into a final answer. Show all answers and thought processes involved from John, Frank and Joe. Probably overkill for what your trying to do but the approach might be useful. This works a charm for any of the difficult prompting problems I've tried. Funny enough I posted this on Reddit and my account now shows that I have no posts or comments even though I have 99 karma and a stary award from this post...hmmm...

You are amazing sir...

Sounds like your experiment was a wild success! I'm also learning a lot from working on Athena. I've mostly solved the reliability issues you've experienced as a result of a rather complicated 'Prompt Structuring Algorithm' and a now deep understanding of how the Prompt effects the model's 'mind'. Additional things like chain-of-thought prompting and (hopefully soon!) tree-of-thought schema will absolutely up your response game! The only limitation I've discovered in how 'smart' these things can get is token costs and limits. I blew through $25 (I have a cap at $100, just in case) on the GPT-4_32k model over only 8 hours, during the _replacement_ of GPT-3.5... I'm getting a new PC in July, on which I should be able to host a copy of Vicuna or Wizard, so that's great, but GPT-4 is what really makes the intelligent behavior shine. 3.5 just can't seem to handle the size of prompts that are needed to get it working, and since I discovered that both model bias new (i.e. lowest text in the prompt) rather heavily, most of the 'Instruction' set actually needs to come AFTER the chat history and memory text. Once I've got GPT-4 fully integrated, I'm going to run my own crazy experiment, wherein I give GPT access to designing _it's own prompt structure_ in real-time, so it can redesign it's own mind! I fully expect it to crash and burn almost immediately, but if it doesn't, and the change/s it makes lead to it working _better_ then I'm in for a wild ride! Other things I'm working on: -I want to give her the ability to *both* write and run real-time python code, like we do with the command level interpreter, and I'd like to give her the ability to write and execute files, as well as some form of persistence between sessions. I'd be nice if the same variables are set in the environment as they were when she last ran code in it, even if I've reset her inbetween. -I'm searching for a STT engine that can differentiate speakers in a noisy area, which I know is a long standing issues in STT, but if she could identify and listen to specific speakers despite crosstalk, that'd be amazing! -Interaction is currently via a jupyter notebook's 'input' box, as displayed by my VSCode... I need something better, so she'll need to be in a real .py file so I can get something cooked up. Anyway, I may be able to help you get your home assistant listening to (and thinking with) Reason, but the TL;DR is basically chain-of-thought + us the LLM to check the LLM. Token costs get crazy (especially for 4, which is 10x more expensive, but also 4x bigger input!) but _quality_ improves greatly!

Please create a series on deep diving on the Home Automation that you have created.

Not a wild beast - You've just described my two years old inquisitive kid. She wanted to eat so it made sense to her to get the kitchen stool. At first we didn't want her picking that up for safety reasons but then I got curious... LSS She wanted to climb the stool up to the sink so she could wash her favorite plate, for dinner. Not a wild beast, a toddler.

For once, the algorithm finally gets me to something I ACTUALLY want to watch. You have my sub sir.

I think the beginning of the Video was captivating, and I would love to see a short film of your experience, maybe make a play through of some of the things told about. Anyway, cool video, did you try to pull the temperature of ChatGPT to Zero? I did not see your other videos from before so maybe you did, but as general rule if you always want the same output Temperature zero is one important parameter.

Nice project mate, love to see things like this👍, a possible idea you can get chatGPT responses in a structured way, I haven't seen many people do this yet, for example by including the following in a prompt like a typescript interface/interfaces that will define the structure of the output and also request the output be in a JSON format. It makes it a lot easier to parse the data for processing and its surprising how well chatGPT can do this, unstructured data can be a bit of a pain to parse.

Your awesome thnx 4 the vid

nice story telling bro

This series is awesome. I wonder if you use mm wave with esp bluetooth tracking to notify ke of your location in the house if it would do a better job? Although with multiple people in the house maybe you would need a biometric voice recognition to know exactly who was talking to ke

*damn, this brotha is amazing* 💯🗿🗿🗿 I’m trying to integrate tree of thoughts into my project rn. Honestly would love to see you take a crack at it. Watching a few of your videos, you’d probably make crazy progress. You earned a new sub 🗿🗿🗿

Pretty cool. I think you needed the guard rails on long enough, and memory for gpt to learn the correct way to interpret your intentions. Perhaps ask for clarification it confidence in the expected action wasn't high enough.

You is a cool guy and define living a modern lifestyle so thanks for sharing ☑️