New to HA ,and this channel has definitely been our #1 online resource: *** TIP *** not sure if this is known to all, but as a 'newbie' we found out the hard way: if you set up your 'super secure password' utilizing the character '@' in it (for example, to change the letter a,A with @, as in '@ppl3' instead of 'apple'), the database, logs, records, history will NOT START! Only after looking into the logs for an hour, we discovered the @ is used in the [>]recorder code as a designator for db/server location! So the only @ character in the [>]db_url line MUST BE the one between your 'password' and 'core-mariadb' FYI! :D
Just switched to MariaDB using this video after corruption with my SQLlite DB. Really appreciate the clear no-nonsense delivery of information. Surprised how easy it was to do. 5 minutes and all up and running.
I know this video is quite old and a lot has changed with home assistant so Im wondering if things like MariaDB are still the way to go or would it be a good thing to go back to default? as always thanks for the videos looking forward to the new sensor when it comes
@@EverythingSmartHome Hi! I have an update on the Maria DB and a possible issue. I've had to back out to my previous configuration. My hardware is an INTEL NUC Celeron 7CYJH with 8GB Ram and a 128GB SSD and I'm using NABU CASA. Basically Alexa Media TTS fails to work with Maria DB, at least it keeps requesting re-authentication and then the same again a few hours later. I backed out to the previous snapshot and all ok again. Very strange. I'll try installing again to double check, but has anyone else seen this issue?
Hello! That is strange, MariaDB doesn't have anything to do with the workings of HA, it only handles history and log book data, that is all. No integrations or anything is stored in the database. Did you check your logs before reverting? I've seen others complain that different things aren't working but it's because they've made an error in the mariadb setup and HA can't start properly, preventing it from loading components after the recorder. The HA logs should confirm this
@@EverythingSmartHome I will repeat the process to be 100% sure, but it was the only change I made. I have a policy of making one change at a time exactly for this reason, I'll feed back again. No doubt of the performance improvement on the NUC though. I was wondering if I could easily use a remote Maria DB as I run one on my Synology DiskStation for my Amateur Radio logbook.
Finally got round to adding an SSD to my install. These TTT videos are amazing. Every single one of them is so clear and concise. It's epic. Thanks again Lewis
I've only recently installed Home Assistant as a VM on my Desktop running Windows 10 (cause it's always on). I've watched some of your videos and find them very useful and easy to understand. Now I've got all my smart blubs, smart plugs, printer, router info, person tracking working. I'm adding MariaDB and Visual Studio Code after watching this video. Just wanted to say "Thank you". Keep it going :-)
Good info! I had recently set this up but now realized I’ve made some mistakes so will be going back and correcting them like going to an SSD card lol. One thing to note is that the history will have too much data, like all the entities so you can filter what you want in there by typing “include:” or “exclude:”. This is mentioned in the HA page
Very important step missed - around 3:30 Restart HA, and wait and wait, make a cup of tea and wait ** Do not faff about with your config file think you have got your password wrong ....... patience is a virtue I don't have! Great Video again
@@mrphyslaw i dont have nuc, they're somewhat expensive, i already have a rasp pi and a ssd is cheap as. i dont see a need for now with as little devices as i have.
@@EverythingSmartHome It can be done on a Raspberry Pi 3. Would have to be a USB to SATA Adapter with a SSD and would still need a SD Card. Just have to modify the System to use the SSD as the docker container storage. Could also be easily done with a existing system by using a directory link. Another option is a MicroSD to eMMC Adapter.
How about a video of a UPS for a Raspberry Pi? I'm living in an area with a lot of power outages and I am trying to find a solution to keep my Raspberry pi 4 running
@@EverythingSmartHome makes sense, I did the MariaDB on a supervised Deb install and have let it run now a couple of days. I'll have a look into how the disk usage is now. Trying to read up on how I can controll how long data is kept and so on.
Thanks! Tonight, I used this and your related video to setup both influxdb and mariadb with 372 day retention. I guess I'll have to use them to watch my free disk space, now ;) Tonight I also found the Plotly Graph Card (HACS) which I think I'll love. I've been using the apexcharts-card (HACS) and like it a lot.
Another great tutorial.. Thank you.. Like the Cards format you show at the end.. Could you go into more detail for that final monitoring dashboard look?
Thank you! Sure I can look at that, they are made using "Apex" cards from HACS, there are even some example code in their Github. Hope that helps a little!
I'm surprised to hear that SQLite isn't performing well (or reliably) considering Home Assistant only has 10 days of data and this should be way, way below any threshold where performance is suffering. Of course i've never analyzed the queries (the achillieas heel of all SQL database apps) but was wondering if you had any metrics on how MariaDB had improved things.
I thoroughly appreciate your SD card warning. My strong preference is to not run Pi's from SD - I was so glad when the Pi 4's started shipping with built in support for booting from USB. I prefer to run them with m.2 SSDs. All that said, my HA now runs on a VM. I migrated a couple weeks ago and it's a LOT snappier than it was on the Pi4.
Great video! Using Home Assistant over a year now. Just switched to MariaDB. One question,- in the roundup you showed a couple of graphs. Which card did you used for the network network_throughput_in and network_throughput_out. Awesome if you can share this :) Kudos for the video. Keep up the good work.
Love your video's just wondering why you would put you db_url in the configuration.yml instead of only a reference to the secrets.yml seems the better place like that
I actually do this in my configuration. the entire parameter is in the secrets.yaml file. then I reference it in the configuration.yaml like: db_url: !secret mariadb_url .... while secrets.yaml is not any more secure than the configuration.yaml file since they are colocated and not enciphered, it DOES allow me to do things like backup everything to github and not push the secrets file to github. It also allows me to easily share configuration samples with people without having to scrub them.
Waiting to order my blue when they come back in stock (fingers crossed) before i do the mariadb switch over.. my rpi3 is busy enough :-) Another good video.... nice one
Great video, I've set up Maria DB a while back when I saw your video. Do you have any information/video on how to set up reporting or extract data from the events table? Keep up the great work!
Nice video, but this isnt working for me... meybe its because of newer verison of HA ? (2021.12.10 HA OS 7.2) but after MariaDB install whole HA is noticeably slower...
HA newbie here. I'm on the 2024.2.1 version of HA and it looks like things have changed a bit with the UI since this tutorial was created. Would love an update!
Cheers ... sussed out my missing logbook and history. In MariaDB config I had changed the user name but hadn't updated the URL. Thanks for the kick in the arse. Been bugging me for months
If you're seeing issues with recorder, logbook etc after switching over to MariaDB; try a shorter password and remove any symbols, restart after changes.
It worked as you suggested, and I know you mentioned Sensor data will be lost, bu I was hoping I had heard you wrong. Of course, it is lost. Is there any way to get the sensor data back. Losing all the energy stats is a biggy. Is there a solution you are aware of?
Thanks for this. But after following this tutorial and deleted the old database file I see the system monitor-stats didnt get removed. Are those stored somewhere else? Also my energy sensor didnt seem to get resetted!
Lewis thanks again for a great video. I followed along and all went well except I am missing a history tab. Is there something I need to add to get it to appear? I also did not see the old db file to delete?
@EverythingSmartHome if you are running Home Assistant with Proxmox, would you continue to use the Add-On or move the database to a separate container or VM?
Hey, the main devs came out recently and said that most people should not be changing databases. any chances on updating the video so people know not to switch
Any idea how one might swap history recording to a Docker based MariaDB instance? I’m already using MariaDB for a couple of databases and I not sure of the impact of having two MariaDB instances - one in HA and one in my Docker container.
@@EverythingSmartHome no problem, i've tried to watch a number of videos on the subject and they never quite seem to explain it clearly, you have a nice succint way of demonstrating it, definitely can tell you do prep and it shows
adapters that begin with en are cause its in a VM. If you are on bare metal such as a raspberry pi and using the built in ethernet,. the adapter is likely going to be "eth0" or if using built in wifi "wlan0"
Despite the "speed increases" of the past 3 or 4 releases, I'm finding that with 2022.7 it takes much longer to access the HA web UI than it ever has. I'm running a VM with decent and dedicated resource allocation on a fast server and first access to HA feels like I'm on dialup.
Have you tried using MariaDB with a socket connection? This would definitely be faster still. I have a feeling that socket may be obfuscated a bit because I think Maria DB is running inside a docker container and i'm not sure that HA and Maria have a shared volume where this socket can be reached.
Hi, thanks for the video... planning to move to Maria DB... just a quick question. I use RPi 3 and a SSD as external data disk. Home assistant boots from an SD card , but all data is stored on SSD connected via USB port. I heard you mention not to use it with an SD card and to move to SSD. Tried a lot, but can't get to boot from SSD on RPI 3. So only did I start using the data disk feature. So is it ok to use Maria DB on my existing setup? Or will i have to boot from SSD (possible on RPI 4's but hardly available in the market right now)
Thanks for this video. I am also running HA virtualized using qemu/kvm and virtual manager however, I am unable to perform snapshots with the current HA qcow. I was wondering what were you using for snapshots? Thx.
Great video, really easy to follow. Any chance on doing a tutorial on energy cost monitoring. Particularly daily/weekly/monthly running cost using day/night tariffs
Hi Lads, I understand Maria DB is sensitive to backups unless stopped prior to manual backups. Might I decide to use MariaDB in my forthcoming install , would this cause a problem with automatic routine backups with HA OS, generating corrupted files?
Great video. Any suggestions on manually entering some of the historical sensor data ? Example: I have about 12 utility meters that keep count of my solar gen, electric usage, grid usage and battery usage. I have 3 years of history now so it would be a sad day to lost it. I just have 4 sets of sensors for daily, monthly, yearly and all time. :-) any advice on how to add if I convert are welcome
Big problem when the supersecure password has the @ character included. The recorder handshake was not happening, and no data was accumulating. No documentation that I could find. Changed the db name, changed the core-mariadb with the IP address, and eventually I noticed that there was a @#$@ sequence in the yaml file that didn't look good at all. Beginner's mistake, I guess...
I wouldn't say that was a mistake on your part. No reason you wouldn't think @ was OK. It was only chance that I didn't think to include that character in mine. It (the HA interface) should make it clear in the settings if the password is restricted to a set of special characters.
Thanks for another great tutorial. It goes straight to favorites folder. Here's another potential video for us - at 0:48 it shows disk usage as full as 91%. My one is bit lower but I'm sure I'll catch you up soon enough. So what's way to clean the disk and remove old/obsolete files that are not needed in HA any longer. (and how to to delete something essential for sure). Thanks.
Yeah it would be entirely dependent on what you have running etc but are you storing any CCTV images etc? And yes on a VM, I could do with increasing the size at some point 😅
I saw the same thing. Apparently Studio Code Server only shows files it can edit. Do you have Terminal installed? If so. you can do it by opening the terminal and entering the following commands: > cd /config > rm home-assistant_v2.db NOTE: instead of using the rm command (remove), I used the mv command (move - which is also used to rename) to rename it. I did this out of an abundance of caution (paranoia). That command is: > mv home-assistant_v2.db home-assistant_v2.db.bak After a few days I'll delete it with: > rm home-assistant_v2.db.bak
Hello, I'm planning to move my installation to an ODROID with eMMC. Is it ok to move to MariaDB with eMMC? Or is it not recommended like an SD card install?
Great video.... now how about a step further by covering how to keep your database from getting out of hand using recorder to set what to record and also how to cap size of database by setting number of days... is there a simple way to wipe the database or compress it... I don’t know.... I just think there has to be a way to keep the data beast under control and from growing tooooo laaaarrrrrge.
Hey Richard, thanks! I think the default amount of data to retain is 10 days and after that it's purged, but you can lower that value if you want, as Miguel mentioned have a check of the recorder documentation
I tried but never got the feeling it was working as my database seem to constantly grow in size. Is there an way to report that the purge has occurred or... track on a panel the size of the database?
Im running home assistant on a VM in my Synology NAS. I also have a docker container with mariadb running for my kodi database. can i use this for home assistant too?
MariaDb can also get corrupted after power loss and it doesn't recover automatically! the workaround is to uninstall mariadb and install it again! its even worse than to rename the sqlite database. Do you have any advice?
Top work sir, concise and bang on the money, I've been meaning to move off SQL-lite for a while now as I was getting table errors in the logs and the system started to feel a bit sluggish. Personally I use Netdata docker to monitor my systems though, as it's scalable and then centrally managed as I have a few low powered boxes running round the house that I need to keep an eye on. did I hear you say Node red ha ha ha ha
I use raspberry Pi 4 8gb with 240 ssd, I get error message on check config not sure why can you check see what i did wrong on the command sensor: - platform: systemmonitor resources: - type: processor_use - type: processor_temperature - type: memory_free - type: disk_use_percent - arg: / - type: disk_use - type: disk_free my network never show up when i check on terminal
Great video. I initially set up HA on an SD card and RPi 4 4G just to play around and see wat it could do. As I am planning on keeping it, is it possible to transfer HA to an SSD drive withing having to do a complete reinstall?
Having moved over to MariaDb i have run into all sorts of issues with snapshots - specifically the fact that the MariaDb does not appear to function if restored from a snapshot meaning that when i run into issues upon major core releases (which i always do), the restore does not work for history/maria etc and i have to nuke it and rebuild from scratch.. Would you have any recommendations as to how to backup MariaDb? Other users in the forums have recommended stopping the service before taking a snapshot which is fine however it would be great if this could be automated (i am using the GDrive backup solution which is amazing! - apart from with this Db)
Did you ever figure this out? I was looking into moving to MariaDB, but I use the "Home Assistant Google Drive Backup" addon and I'm concerned that my restores won't work, or won't include the MariaDB.
@@davidhill8151 hi David, not as yet no however I would say that HA seems to much better of late in terms of stability that I haven’t needed to restore from backup. Also made some tweaks to my settings so Maria us used got short term only with Influx being used for long term storage, with this setup loosing the Maria would not be too much of a problem.
very useful video. thanks for that. It did however leave me wondering if running the db in mardiab somehow will require me to take some steps to make sure that the database is also included in my auto backup to my nas. if anybody has any thoughts on this I would love to hear them!
Great video. Hoping for some helpfull tips for more stability problems of HA. Almost daily HA is in a hung status. How can I see why this is happening ?
Hey, Lewis! What do you thing about a video thats help us switching our Raspberry Pi SD card to a SSD? Some adaptors and SSDs recommendations might be a good options for the video too!
Hi, again great channel! I have semi-related question, you are suggesting not to base the HA on SD card while using mariaDB and InfluxDB. The new "Home Assistant Blue" is based on eMMC, is it different? Is it reliable enough? Or the SSD suggestion still stands? Thank you in advance!
Excellent...thanks so much. One more question. On my test machine running HA in Docker on Synology, the temperature resource worked fine (which I was a bit surprised) but when I tried the same on my "NUC" (intel laptop running the nuc HA software install....so no VM), it did not. Any idea why?
Question, I use influxDB integration for my historical data and graphs through Grafana. Moving to mariaDB, should the influx data just pick up where it left off?
Great info as always. Quick question, i got mariaDB set up as per your video and all seemed fine, then i noticed a couple of esphome devices had gone offline, any ideas? I went back to a backup i made on proxmox and they came back. Any thoughts would be appreciated, Allan
Thanks Allan, that is strange, the database doesnt contain any information about your sensors and integrations, it only contains sensor history so it's not possible for it to make your device go offline. There was potentially another issue simaltaneously that caused it to go offline? Did you check your HA logs?
@@EverythingSmartHome Hi thanks for your reply, no not yet, I have recovered to a backup just to get things going again. I will try again soon and let you know, busy decorating as well today, thanks, Allan
Hi. Great how to monitor lesson. All worked except the network monitor. Both In and Out return an unknown. I am running HA via a Hyper V VM. Is it possible that this will not work in a VM similar to your description on processor temperature? I have verified my network adapter name. Configuration check is "green" or good. Any thoughts are greatly appreciated.....
Hello and thanks! It should work in Hyper-V no problem, the only thing that won't work as mentioned is temperature. What was the error you got? If you need help feel free to hop into the discord server!
@@EverythingSmartHome I don't get an error. The state just shows "unknown" under developer tools and the history shows "0" for both in and out throughput lines. BTW, I love your channel. You really break things down into simple terms and procedures. I'l be sending you some coffee....
I need to check how my Docker got things set up, I have the "IOTstack" installer. That community is huge about InfluxDB and Grafana and then Mosquitto. Maybe this is overkill, however I wanted to ask if anyone watching this video had any experiences with InfluxDB
Is it easy to migrate homeassistant from an SD card to SSD? And if I run from an SSD shouls I use the 64 bit version? And can you make a guide how to? I'm currently running on an pi4 4GB model. Thanks for the video's!
I don't understand, I did the mariadb update, pretty straight forward, but afterwards I get all kinds of errors. I get an error saying the following integratings and platforms could not start. recorder logbook history energy default config I dont undestand why, I saw a complex pw can cause an issue, mine is pretty straigh forward only 8 characters long. Any thoughts?
The errors are a knock on effect from the recorder not working - make sure add-on is started and check logs for errors. Also no @ symbol in the password
@@EverythingSmartHome did my comment get deleted or something? I left a reply with a link to imjur to see a screenshot to the log I have.... Does RU-vid not like that?
@@EverythingSmartHome I just typed out the errors I was seeing. waiting on integrations to complete setup: spotify (warning) bootstrap.py error setting up entry home for smartthings (error) components/smarrthings/_init_.py setup failed for logbook: (dependencyError(...), 'Could not setup dependencies: recorder') (error) setup.py unable to setup dependencies for logbook. setup faild for dependencies:recorder (error) setup.py setup failed for recorder: Integration failed to initialize. (error) setup.py Thanks for the help!
