I need of tech like that for my desktop virtual 3d assistant. I have a 3d model of a character (AI agent) that has to interact with computer in many interesting ways up to controlling pixels of the screen by itself, for example if it want to impose a an object to interact with virtual space. I hope soon enough we will have enough speed and power for AI agents to be sentient and working seamlessly with any type of information.
Being a member I have been trying to access the github repo, I have sent multiple emails to the provided email address, yet to receive a response it has been 48hrs. Please advise.
Im from Portugal, the portuguese is a mixture of mostly Portuguese from Brasil and a lil bit of Portuguese from Portugal heheh Spanish is not my primary language but it is not that bad also !
🎯 Key points for quick navigation: 00:00 *🖥️ Overview of the project setup* - Setting up for screenshot analysis using GPT-4o - Detailing the low latency approach for image understanding - Collecting documentation and writing the initial iteration of the script 02:18 *🛠️ Implementing functions and configurations* - Fetching documentation from OpenAI for implementing GPT-4o with image inputs - Inclusion of functions from prior projects to streamline the process - Utilizing EnV files to fetch the OpenAI key for configuration 07:21 *🔊 Integrating text-to-speech functionality* - Obtaining OpenAI documentation for speech-to-text-to-speech functionalities - Implementing a feature to read out responses using TTS - Troubleshooting and fixing errors in the TTS APIs and configuration 10:55 *🎛️ Controlling the main function with a trigger key* - Adding a feature to control the main function trigger using a key command - Testing the control setup with screen prompts for AI responses - Demonstrating the capability of the system to respond effectively with controlled triggers Made with HARPA AI
