The cost to use OpenAI in your Smart Home

Подписаться 14 тыс.

Просмотров 2,3 тыс.

50% 1

Will your cost to GPT increase if you use AI-Intent or Extended OpenAI Conversation? Well lets talk about it.
Enjoying the content? Buy me a ☕️: bmc.link/mmontaque
Or join this channel to get access to perks:
/ @technithusiast
Music
Breakfast For Dinner by Dr. Delight - C1NXIINVBQPCS4DA
Gear I used for this video
- Sony A7S iii: www.amazon.com/Sony-Mirrorles...
- Aputure Light Dome: www.amazon.com/gp/product/B08...
- Aputure Amaran 200X S: www.amazon.com/gp/product/B0B...
- Neewer Heavy Duty Light Stand: www.amazon.com/gp/product/B08...
DISCLAIMER: Links included in this description might be affiliate links. If you purchase a product or service with the links that I provide I may receive a small commission. There is no additional charge to you! Thank you for supporting me so I can continue to provide you with dope free content!

Наука

Опубликовано:

26 янв 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 29

@poshsims4016 5 месяцев назад

Maybe using langchain will help the memory state & controlling it better

@johnlambert7405 5 месяцев назад

Also adds some nice abstractions to use locally hosted LLMs.

@technithusiast 5 месяцев назад

I never worked with langchain but it looks interesting. I’m working on creating a way to use local llm in automations but I’m currently having trouble getting the local llm to work😅

@KeithConley 5 месяцев назад

Thanks! This seems to be a direct answer to my question on your last video.

@technithusiast 5 месяцев назад

Hey, no problem! Glad I could help.

@videoblight 4 месяца назад

It’s a good idea to set up rate limits. I had local voice with extended open-ai on a spare Pi and it went haywire. It sent thousands of requests and chalked up $20 in less than a day.

@technithusiast 4 месяца назад

Yup not surprised. The extended api is really impressive but the trade off is that it’s greedy

@MacDork 5 месяцев назад

ooh ty. been wondering this too

@technithusiast 5 месяцев назад

No problem!

@PontusWelin 6 месяцев назад

This gave me an idea! What about doing two calls to the API? One using register intent/call intent nodes to figure out what it want to do. But this uses as little data as possible. Then when you know the specifics you can make another call where you add the relevant data. You’d get the best of both worlds. Except it will take more time potentially. Though I’m not even sure it would since the big api calls take longer anyway.

@NigelinAustralia 6 месяцев назад

I had the same idea. Get gpt to work out the intent of something. If there was a way gpt could remember context of your house and what devices were there it could work out intent without having to have updated states initially

@PontusWelin 6 месяцев назад

@@NigelinAustralia i feel like that’s what the assistant api is for basically.

@technithusiast 6 месяцев назад

I like your thought process! I liked the way this conversation in this thread was going that i had to create a quick video about my thoughts which you can see here: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-A3uXwZ4_iJ0.html TL;DR - I had a similar idea of separating the calls last fall and rewrote Ke to use this technique and.... it wasn't as good as i thought it would be.

@dragoon347 Месяц назад

Couldnt you just hook this up to an internal house server running a small phi3 model?

@aidanb8719 6 месяцев назад

I have been looking at the litellm and using the proxy you could have it call that and then it might be able to get the response back from a range of LLM some local some in the cloud. Just the local might take some time but if all its doing is returning information you pass in then I can't see it being too slow. Worth a try and if works then it's all free.

@technithusiast 6 месяцев назад

Can you elaborate a little further what you mean? i am not sure i quite follow.

@aidanb8719 6 месяцев назад

@@technithusiast you can host local LLM and there is something called litellm and that will allow you to make api calls to the local LLM or to cloud-stored APIs its useful if you want to use some coding extension such as GPT-Pilot you point the config to call the proxy then it will call where it needs to. A lot of people on here explain it better than me but that's my thinking that you could use that as it would not need the cloud and its all local so safer too.

@PontusWelin 6 месяцев назад

I was also wondering if it is possible to setup an assistant with the assistants API to not have to send all the data all the time? Like you wouldn’t need the system node and the template data could be greatly reduced probably.

@NigelinAustralia 6 месяцев назад

I asked similar. The challenge is the assistant api requires far more conversation management at the moment. Works brilliantly in sandbox but actually calling the api kind of sucks

@PontusWelin 6 месяцев назад

@@NigelinAustralia ok. Conversation management how?

@NigelinAustralia 6 месяцев назад

It's overly complicated at the moment - hoping they simplify

@technithusiast 6 месяцев назад

Correct. I think the Assistant API has promise but as it stands, its complicated and nature of Smart Homes Require that the AI is aware of the state at all times. This mean with each call you have to let the AI know the state of everything. In the end you will always have to send a ton of data, unless you know before hand that what you are asking for does not require a lot of data.

@technithusiast 6 месяцев назад

I just posted a video in Chat that talked about this a little more: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-A3uXwZ4_iJ0.html

@tedev 5 месяцев назад

unless chatGPT is able to store data and status of things, it will be really expensive and time consuming in the long run. i don't want to send an entire list of all my lights everytime i want to say "turn the lights off in the living room". i want to update it once and keep it. maybe a session would work. for example if i start a chat with GPT.... i can list all my lights and then keep talking to that chat later on...

@technithusiast 5 месяцев назад

Technically, you can accomplish something like this but the experience degrades over time. As a simple example, Imagine if you had two lights in you house that were off and you send GPT both lights and their states. Then someone manually turned on a light. How would GPT know with out you sending the state of everything. Now imagine we ask GPT to turn on a light and it sends the command to you but HA failed to turn it on. How would GPT know? I totally agree that it sucks sending all the info every time you call GPT but the way the LLMs work gives us no choice. AI-intent does have a small advantage here where you can send GPT specifically what it needs based on your commands but it requires clever planning.

@OldManShoutsAtClouds Месяц назад

@technithusiast the API call to chat gpt should be pretty much the last in your chain. Prior to that, you should build the context using other more standards API calls. Just ask HA for the status of all devices and include that output as context. The biggest thing you're missing is context. "It's dark".... where? It's dark in the living room... not your daughters room, not the kitchen the freaking living room. Use. More. Context.