My GPT-Powered Chatbot Against Home Assistant's Assist 

You are completely right about those extra points! I asked an open-ended request to audience what criteria to compete against and they happened to choose things that Ke excelled at 😛. Stoked you’re enjoying the content! And great work with Assist. It was fun working with it.

Lolol yeah. Trying something new. Over 90% of the video was created by stitching together my YT shorts. The audience who watches the shorts are generally a different audience than the ones who watch the long form so I’m testing to see how people respond.

Thank you 🙏🏾 and class starts here: How To Build A Telegram Chatbot That Controls Your Smart Home | Chapter 1 ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-wl8ZWBjpBoA.html

Glad you enjoyed the video. I was trying something new with this vid. The competition was originally created for YT shorts but my audience primarily watch long form so I knew they probably didn’t see them so I stitched them together into a long form which is why it’s vertical

I’m glad you’re enjoying the videos! I’ve played around with the home assistant pipelines a little but I don’t have a ton of experience with it. Can you elaborate a little further what you mean?

@technithusiast pipelines is the wrong word; flow is more specific and less confusing. On the one hand I'm thinking efficiency and accuracy: preempting a command with the header "household tasks:" would set specific entity filters, like the vacuum or mower and not the holiday light strips or the stereo. Or if certain commands are allowed only to parents (e.g. the alarm system) you could have an "elevated command:" prompt header and have Ke respond with a secure pin prompt in telegram. I know you haven't been doing STT much (or if you have I missed it) but it seems to me like it might be a useful tool for restricting the set of entities to the most relevant for the request. Please forgive my misuse of any terminology - this isn't my professional field

@@sch9986 Lemme see if I understand. From what i gather, you're asking if we can add "prefixes" to commands to allow the automation to narrow down then entities the command applies to or force the automation to request verification, or any other useful technique to increase accuracy of the system? Assuming my understanding of your question is correct, then I think you bring up a point that I feel have some interesting nuance but I'll give the short answer. Yes, you can do something like what you mentioned in your example by combining Home Assistants native Conversation trigger with GPT. However this impacts the user experience and makes the interaction with the smarthome more robotic and would be difficult for people who don't have intimate knowledge of the home and the keywords available to them. But there is technically a way to do what you mentioned and only marginally sacrifice user experience, but that would be a separate convo 😁

Glad you enjoyed! You can check this out: How To Build A Telegram Chatbot That Controls Your Smart Home ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-wl8ZWBjpBoA.html

I’m stuck between a rock and a hard place with this. I would love to keep you all updated but it requires time that my current life doesn’t afford. If this was my day job I could devote a lot of hours in creating something stable for everyone to use with little fuss. I didn’t want the enemy of Good to be Perfect so I opted to release my ideas knowing that they’ll become outdated and hoping people would be able to at the minimum use the resources as a guide or starting point and be able to get there on their own. I am trying to work on a native way to interact with open AI but they change so fast that anything I create would become obsolete and need constant updates and attention. I’ll see if I can put out an update video and code in Chaperone to help everyone jumping in at this point but it may take a bit of time. I have a question for you. Would you find value in being in a mailing list or some type of group where you can get detail updates of the changes I make to my automations?

⁠@@technithusiastDo you see benefit if I help you start a project on GitHub and maintain a community there with regular PRs and contributions from the community. I have started in your footsteps and I have a lot of ideas. Happy to collaborate with you and create a fun space.

@@technithusiast thant would be realy helpful with the mailinglist! I would support this way of bringing out updates!

​@@technithusiast hello sorry for the slow response. i would take anything but if i was in your shoes... i would create a blog.... not a video. a wordpress site for example... it is fast to build and EASY to replace when something changes. so you wouldn't have to create constant updated videos or mailing lists (which are the worst)

@@tedevI think the HASS integration extend_openAI_conversation is probably the best way for most users if he doesn’t have the time. It’s capable of all the same features, call scripts and also allow GPT to create automations by itself. I’m sure he could drop a quick tutorial.

Here is the link to the playlist to build your own version of ke: ru-vid.com/group/PLJFrZG9EZhtoBKZpdc9Ui8aF9X7hneCEp&si=5OfjVeS2MSbQuk8h

I’m trying to get Ke to that level so it becomes a better alternative to the big three. This is highly experimental and I started this just to see what’s possible with LLMs and smart home but now that I got this far I’m trying to see if I could take it all the way. There are 3 big hurdles (and a lot of tiny ones) I need to overcome. 1. I need an easy, highly accessible, mainstream way to talk to Ke the same way we talk to the big 3. I don’t want to build my own speaker or sound system to do it. And if I do have to build it, then it must be so easy my mom can set it up. 2. I need a way for it to be easily installed and utilized. Using GPT is cool but I most people (myself included) would prefer a completely local llm. However, experience taught me that since OpenAI is leading the pack, all the others will simply copy what they are doing and have the highest chances of becoming obsolete so I need LLMs to become more stable 3. Lastly, the access Ke, has is limited and in order to control anything requires some coding knowledge and access to APIs from the system you want to integrate. I need a common protocol (like Thread) that all systems (including Ke) can speak which will make Ke immediately compatible with everything. But one step at a time 😅

Nope. I have Home Assistant OS running in a VM. I’m not using Nabu Casa’s subscription (though I’m pondering it) and I’m not using whisper or piper. Just using the default out of the box features.

@@technithusiast oh right, you're just using text. wyoming pipeline is for voice in/out, duh. its early, im not awake! I run HA-core in a docker, so i dont have addons, i just run separate docker containers. either way you can run faster-whisper, piper and openwakeword for the wyoming protocol pipline. I also sub to nabu, as I have been using HA for 5 years, and they deserve something for all their hard work.

Most of the commands go through home assistants conversation api. But for more complex commands they slightly longer (between 1- 5 seconds)

Another viewer called this out. Try updating to the latest version of gpt (currently gpt-4-1106-preview). Chaperone has the old version since it’s accessible to everyone and the other versions require additional sign up for access

So you are using assist to turn on a light, and OpenAI to turn off multiple lights? How do evaluate which ‘engine’ to use, without asking OpenAI at first?

Great question! I always ask assist first. Recent updates from Home Assistant team now lets the event tell you when the command fails. If i see the failure from assist I call OpenAI.

I have a playlist that goes in depth how to build something like what you saw: ru-vid.com/group/PLJFrZG9EZhtoBKZpdc9Ui8aF9X7hneCEp&si=5OfjVeS2MSbQuk8h

I have seen all your video with great interest, and they really inspired to make something similar. I already made my own, tweaked it a little so it only one call, trying to save waiting time. But would like to see your handling of functions call to get more inspired 🥳😃

Of course! Thanks for the input! I’m curious, would you find value or in being in a mailing list or group where u get more detailed updates about this type of automations?

I would absolutely find value in that! I like taking it beyond the casual smart home, and tweaking things to get the most out it, but in the same time, my time for this varies a lot. That’s why I’m much interested in the more detailed part of Ke 👍 I think your on the right track😃

Lololol. I would for this to be a one-click process too! I’m confident we can get there 😁