This is a really full service video. Not only do we get a video of the installation of the MQTT broker and the HA integration, we also get an index, chapters, and a companion blog that includes all the code he uses, so you don't have to scrape the code off the screen. I even appreciate the fact that he doesn't waste my time during his intro and outro. Bravo!
Important to note if you run Docker hass and docker mqtt you neet to enable 'listener 1883' in the mqtt conf for the mqtt container to start and run, otherwise no clients can connect to it as it cant bind ports onto the loopback socket of the docker container. This is a change for mosquitto 2.0
Just wanted to thank you so much for this video! The most informative thing I could find on the topic without having to reference several different articles to get the full guide I needed. Cheers dude!
For real. I really want to go the home assistant route but I wasn't sure if I would be able to do it all. In the last few days I watched 100 of videos about the topic and I already feel confident that I can do it. Big part of it are YOUR videos. I have so much ideas already. I'm testing it in a vm on my gaming rig right now. And when I get a hang of it I'll buy a used intel nuc i5 6200 since the price is roughly the same as a respberry pi (prices of the pis are ridiculous at the moment). Thanks!!!
@@barygol yes.. Got it running and got tons of automations for my house. But it runs on my synology nas in a vm instead of extra hardware. Runs like a dream
@@Kevin-rf9sx cool. I took the route of server with proxmox + VM/docker containers. And doubting whether I'm going with a Synology for storage or another VM with TrueNAS
This didn't fit my exact situation but had enough good details that I was able to figure out what I was doing wrong to get what I needed setup. Thank you!
thanks so much Alan! I remember asking for specifically this a short while back as a comment to one of your docker/Portainer vids, and you replied that you might cover this topic later. I am very thankful, after your next video (Zigbee2MQTT) I can fnally pick up where I got stuck along the way
Thanks again Alan! I'm working though your videos to migrate a Raspberry Pi HA system to an HA Docker system. The knowledge I have picked up on Docker has been invaluable, thanks to your content. I look forward to the next video on Zigbee2MQTT, cause I need that too.
That's a very insightful and useful, clear video. Thanks for taking the time to make it. I look forward to your next video as I want to ditch my Phillips Hue hub and move everything onto my Conbee II.
@@HomeAutomationGuy It's great. Definitely yet another learning curve required, but you can get very far by just basing your configs by community provided templates. What made it worth it for me is that I can place most of the device-specific logic into itself, rather than rely on automations, making it more like a smart device rather than an 'obedient' one.
Nice video and well though out in advance. Been using mqtt for about 4 months now with espresense for room presence detection. I'm running HA and mqtt on a raspberry pi though so easier to setup. But i am running it from and booting from a SSD instead of a micro SD card. Runs much faster that way and don't have to worry about the micro sd card crapping out on me like they do when being written to a lot.
Great video. Very easy to follow with all the Docker instructions. Unfortunately I get to the end, trying to add the MQTT Integration, and I get a "Failed to connect" message, with no context and I can't find any logging anywhere to explain why. Any thoughts on where to look next?
If you are still running into this issue, I figured it out on my end. If you look at the updated blog for this tutorial, it has the line: "listener 1883" that wasn't in the video. I added this into my mosquitto.conf file, restarted the mosquitto container, then tried the MQTT integration again and it connected without an issue. Hope this helps!
@9:50 The reason why it happens is because the user inside the container is "root" and all files created by root are automatically owned by root. Since docker just containerises on top of the kernel of your host system, the access permissions and ownership will also propagate back to your host system. You would need to create a specific user inside the docker container that is allowed to run all necessary commands. Since that is not always straight forward, it is often easier to just run a script once a day to change the ownership of all the data of your /opt/xxx directory (preferably before you start your backup)
Thanks so much for the explanation. Back, early in the initial installation of Docker and Hass, this threw me off so badly. I got through that part ok using the chown workaround. But then when implementing backups (using rsync instead of Duplicati) I started going down the path of permission hell. After countless troubleshooting hours and trying, and failing with Duplicati, I thought maybe RaspberryPi OS was the culprit. When I switched to Ubuntu, I decided to keep permissions cleaner by putting everything into a home directory rather than /opt. When I hit exactly the same problems, I at least had ruled out all the other potential causes. My conclusion was that "ls -l" was showing the correct user name but the culprit is that it was still part of the root group. After I modified chown to be "sudo chown : -R /opt" my rsync worked perfectly. It is still annoying that you need to go back to chown every now and again. I was really wondering if there was something fundamental missing, but this explanation makes it clear that there probably is nothing really cleaner than this workaround.
I tried to create a user named hass inside the container, but the user does not appear to be a root or superuser. Now that I've created this user, how do I give it privileges? Typing things like "usermod" "sudo" and "apt" all return errors.
advice please.... i setup mqtt docker as i followed your explanation and when come to setup authentication all stopped when i clicked on console in mosquitto - error message said "Unable to retrieve image details" where have i missed?
I'd just like to say a brief thank you. I've been building and using PCs for nearly 30 years now but have largely shied away from this particular side of computing. Over the last 3 days I've been trying to get things like docker, portainer, HA, etc all running on a Pi4 (all of which are new to me) and frankly have felt like a complete idiot at every turn. Practically every instruction I've seen and attempted to replicate only to have a problem that somehow didn't exist for the person whos video/blog I was reading. Every. Turn. Yours on the other hand pretty much all clipped along exactly as I had hoped for the most part. The only difference being I had already installed Docker, and so using your docker compose install left me with the 1.2.9 version and for the life of me I couldn't work out how to get a newer build running. Only to find out when attempting to install docker-compose-plugin that version 2.5 was already installed. It does mean console commands are now docker compose, not docker-compose but it seems to work so hopefully world ending outcomes are not on my horizon. But, just thank you. I feel like slightly less of a moron now and hopefully by the end of this weekend I'll get my solar storage inverters talking locally to my new HA via mqtt after this video helped me out so much. Do. Not. Stop.
Assuming you are doing this all on the same server, I would recommend using the "localhost" instead of the IP for setting up the connection to the broker. This way, if the IP changes, it won't break your system.
I have since found that "localhost" does not appear to be supported well, or entirely. Either a limitation of docker or the individual apps. However, the local loopback 'should'. But I have not tried it yet.
I was following the Blog rather than the video and it seems there is a small error on it. In the text that says "If you don't want to set up authentication, you can change the allow_anonymous line in the configuration above to true" it already says TRUE. Then you say "You can now switch back to VSCode and uncomment out the authentication lines." but these lines are not in the existing code (I did however copy them from further down your article). I've managed to set it all up now thanks but thought I would let you know so you can fix these on the blog. I was not in a position to watch the video so dont know how it played out there.
Thank you Alan for this insightful and very useful video! Congrats on your work! I have a question regarding the mosquitto log file. In the video I noticed that whenever you show no errors on the mosquitto log using portainer you are not actually showing anything of what is inside that log file. I believe that is to do with this config parameter "log_dest file /mosquitto/log/mosquitto.log" . I think that if this is enabled the logs are not redirected to Portainer but will be registered real time in this file. If you disable this option and restart the container the logs instantly show up in Portainer container logs. At least I had to comment this config option in order to get them in Portainer, not sure if I am doing something wrong or there is a way to have them both sides, log file and Portainer dashboard. I am really looking forward to your Docker Frigate video. Great work 💪
Thanks for this video. Just wondering why in /opt/mosquitto/data and /opt/mosquitto/log the files that are in the container are not showing. When is console into the mosquito container in the /data and /log there are files that are not there in the /opt/mosquitto folders. I can see the folders were created but they are empty!
I followed the Docker install and ran into problems getting Mosquitto to locate the configuration file, despite it existing with correct permissions. The problem was that despite mapping /opt/mosquitto to /mosquitto in the compose file, for some reason you have to specifically map the config directory as a separate volume. Have a look at issue #2557 on the eclipse/mosquitto github repo for more info.
Love this. I’ve just done this part but this would have saved me time. Are you planning on looking at zigbee2mqtt and node red? That’s what is currently making me pull my hair out.
Zigbee2mqtt isn't to bad.... Mosquitto gave me fits, ended up installing it with portainer which put in in /var/lib/docker... My 'normal docker dir is /opt as per these tutorials... It's cool though cause mqtt keeps running when I fiddle with compose-up in /opt... Zigbee2mqtt, don't change default ports till you have it working, web port is 8080 default. It doesn't start observer until it successfully connects to mqtt. In its yaml I directed it to mqtt:'ipadd:1883'... Good luck, I finally got mine working and set up 4 sengled plugs, Note: You will likely see zha integration try to grab your dongle, TELL IT TO DISMISS... I'm using a Sonoff (flashed) dongle and zha said it couldn't handle it anyway... Logs will show startup, connecting to mosquitto, etc... If you have set allow in yaml you can pair a device even without mqtt working. If you see 'herdsman' messages mqtt didn't connect...
I have a question. How did you create mosquitto.conf 7:12 in VSC? Did you change permition or owner for entire /opt directory. I don't have access from non root user to do that.
@@LukePighetti The devices should automatically get added to Home Assistant via the MQTT HA integration. If they don't, you can go to your Integrations page in Home Assistant, find the MQTT integration and then click the three dots in the bottom right of it, go to System Options and make sure that the Enable Newly Added Entities thing is turned on.
3:55 this is where after 3 hours i give up, dont have the time to deal with the hoops. "Add-ons are only available if you've used the Home Assistant Operating System or Home Assistant Supervised installation method. If you installed Home Assistant using any other method then you cannot use add-ons. Often you can achieve the same manually, refer to the documentation by the vendor of the application you'd like to install."
Mosquitto, Zigbee2mqtt, Frigate, Pi-Hole, mysql, Signal messenger client, Wireguard, Duplicati and Home Assistant. Would you like to know more about any of these in a video? What are you running on yours?
Hi, I need a little help I have a lorawan device publishing mqtt messages can see them on mqtt explorer but nothing is coming up on HA Mqtt broker, listening for the topics and not picking up anything. made everything lowercase that didn't help. does HA have a topic max length maybe?
Running into an issue where I get the error, "Failed to Connect" when trying to set up the MQTT integration. I confirmed the passwords are correct but still can't figure out why it's not connecting.
@@HomeAutomationGuy Hello! Thanks for your reply! It was totally just user error. In the code, I forgot to put the "listener 1883" in the mosquitto.conf file. I corrected this, restarted the container and now it's working like a charm.
Fun fact: If you're running HA in a Docker container, as I was, "Add-Ons" (as well as "Supervisor", "Managed Restore" and "Managed OS") are not supported. I ended up switching to a Linux-based HA-OS VM.
Thank you so much, I am about to buy power multimeter from alibaba, it's an mqtt base, can i integrate it to HA via mqtt broker regardless the device brand ?
Hello, i have a problem when trying to make a new folder like it's shown @6:55 VSCode gives me an "Error: EACCES: permission denied, mkdir '/opt/mosquitto' " when I try to create it. Seems like I have to do some Permission-settings but I don't know where to edit? Maybe anybody can help? And thanks for this awesome instructions, they are great!
Great video, but to be honest.. HA in docker is just extremely complex for ud new ones with a Synology NAS.. just to get a MQTT broker going! i need VScode, 3 extensions, SSH settings enabled i various places, Docker compose stuff installed, Portainer and more docker compose installed, and between all of that is tons of lines of code in different windows and platforms. Its just two complex to get a piece of software going that is 2 clicks away in HA as a OS.. 5+ hours in.. i have to give up. MQTT on Docker HA is just broken IMOO
it would be nice if you actually deploy the mqtt container using command lines instead of relying on vscode, as vscode is also only available on HAOS, so that tutorial section you show there how to manually deploy a container is useless for people who ran HA in a docker container. jeez for all tutorial how to deplot mqtt-broker for HA out there, only one video is actually good enough to follow for most people with that "watches tutorial an mimic it exactly" kind of instruction. And that video is performed by that old guy.
VSCode is free, and so is the extension called "Remote - SSH" he's using that allows you to connect to your raspberry pi/docker containers and edit any files easily within the editor. So it is very applicable!
