Hey, everyone! Thanks for the pointers regarding the servers and a UPS. Aside from simply powering off before a pending storm's arrival, there are other cases where power may cut off intermittently which can absolutely damage servers and corrupt data. The NAS just stores old footage and I've lost data from it before without serious repercussions, but you're absolutely right - it is good practice to keep systems like these attached to a UPS for graceful shutdowns and smooth power delivery. I'll be looking into a second UPS specifically for our servers and may follow up in a subsequent video.
You should host a "nut" server on something low powered like a raspberry pi that will send shutdown commands to all your servers. With it for example all of your high load servers can be shutdown after 60 seconds and then all of your low powered/critical stuff can be shutdown gracefully when the ups only has 240 seconds of power left
I've been using Eaton UPSes lately they have a real nice and easy to use expansion system for extra battery capacity if you have the rack space. Also have Lithum ones (which last a bit longer).
Having your NAS on your UPS is still a proper way of working, especially when you have UPS monitoring in the NAS system, so the NAS can power off when the UPS signals the remaining power goes below a certain threshold. Without powering down gracefully, which you do this way, there is a chance a disk in the NAS gets corrupted, meaning you have to replace and rebuild that raid array again. You can of course power off certain systems manually when a disruptive event starts to take place just to be sure, but what I try to bring across is that an UPS is not just a big battery for systems connected, it also allows the connected systems (when they have the sensing for it) to power down normally before the battery is running empty.
Greg, get a EBM (extended battery module) for the UPS instead of getting a whole second UPS. It plugs into the main UPS and is just an extra battery pack and you get a bit more runtime with an EBM compared to another UPS
Glad you didn't have any major storm damage-though you should definitely keep the NAS plugged into the UPS. Even if you have enterprise storage adapters with built-in supercaps/batteries that will save writes in progress... there are a lot of ways you can end up with corrupt data (even if you're not editing when the power outage happens). The other advantage of plugging servers into UPSes is the much better power filtering/protection a UPS provides (even if you're going through something like a Tesla Powerwall!). My recommendation if you want more runtime for the WiFi especially would be to run two UPSes; one for the NAS and bigger server, maybe figure out how to configure it with 'Network UPS Tools' so it will shut down automatically / gracefully after a couple minutes.
Thanks for the pointers, man! Admittedly, the importance of old footage on my NAS is very low on the totem pole - which is why I was willing to be less careful about power backups for both it and our compute node. That said, I understand your reasoning. I'll see what I can do about running them into a separate UPS with custom shutdown parameters. That part is still very foreign to me.
@@GregSalazar Honestly NUT can be a little confusing even for people experienced with it! But it's well worth getting it working, plus you can do some content on it :) Main thing is to make sure you have backups, of course-for anything important to you!
@@GregSalazar that or just remember if the power does go out to properly shut it down and unplug it or have it on its own power bar you can flip a switch on
They did an excellent job for sure. Stem wall, concrete block exterior, etc. Built to withstand cat 5s. Just hoping I never have to test it through one!
I have something similar to what you are talking about...My tank isn't underground, but I have a 300gal propane tank that I use to heat my house and I have a large generator tied to it that I can use if I need to...I have used it during hurricanes and the occasional ice storms and it works quite well...All my neighbors and family would come to my house to charge there stuff up and take hot showers! lol
@@brucepreston3927 may i ask where you live that gets both hurricanes and ice storms so i know not to move there. cool that you have a set up that keeps the lights on tho.
I have a couple of suggestions for your setup because I live on the Gulf Coast too and have experience many hurricanes. The first thing that I would suggest is to look into a natural gas generator, whether that be a whole home or if you could convert that portable generator into a natural gas generator. Also, if you want to use the portable one, really this applies to both portable or the whole generator, see about have a transfer switch installed in your home. This transfer switch would allow you to directly connect the portable generator into your home power supply without the needs to run extension cables. You will just need that one transfer power cable that will go from the generator to the transfer switch on your home. Downside with the portable generator and the transfer switch is that you will have to pick and choose, which devices you want to run on it because of the limitation of the portable generator.
It's not automatic, but he could just do a (manual) interlock kit and then when he connects the generator to his breaker box he can then pick and choose what circuits he wants active and when. Bit of advise on that, skip the 30amp version, get the 50amp, you can buy adapters to plug in a 30amp plug no-harm-no-fowl, but if he got a bigger generator that could possibly power his a/c and such for creature comforts with the bigger gen set. I'm in pasco, fl just north of landfall, and in my area low places were flooded but I was good, ran my generator from wed night to sat morning powering all but the 220 circuits (generator and 30amp circuit wasn't enough for my 2.5tonn a/c) so jow I'm looking into upping to a 50 amp (maybe 2?) And a bigger generator.
Hey Greg, do yourself a favour and loose the cage nuts & screws by switching to Rackstuds. They make installing or swopping switches so much easier as the stud helps support the equipment while you are tightening the cap nut.
My goals would be to have a power shack with battery backup and a whole house generator like a generac that automatically kicks on after a few mins without power. Our neighbors have one and theirs was running during the storm so they were never without power.
Happy to see that you and your family stayed safe, you were lucky to not get any damage. Watching Hurricane Milton from Europe was scary, can't imagine how was it IRL. Honestly would like to see another video with adding ups, this server stuff is cool!
Glad to hear you Had little to no damage form Milton, I lived in the central FL area while in high school, so i know what you have been through. Love what you have implemented into your home and your future plans.
I relate with only having one ISP. I live in Eastern Tennessee and after watching how easily the storms took out all communications, Starlink is something that is a major contributor to allowing people to continue to contact loved ones in times like these.
Every time I watch one of your Fix or Flop and PCDC videos, I always think to myself I wish I lived in Florida so I could get your help with my rig. But then I see a video like this and I'm lowkey grateful I don't. Glad y'all are alright down there.
Glad to hear you're OK, Greg. Over here in central Seminole County we had 60-80mph winds but we didn't even lose power. I streamed all 8 hours during the worst of it. What a night.
So glad to hear you and your family were safe. I have been through tornadoes, but never a Hurricane. Without power for 3 days is tough with small children. Glad you weren't hammered.
Glad you're safe Greg. I have friends in Florida, you guys are made of different mettle. I live near Toronto Canada, extreme weather isn't really a thing for us. So the fact that there's at least one entire state that accepts hurricanes as a fact of life is just wild to me.
I'll throw a few things in: 1. As others have mentioned, a whole home generator is a good idea. We have one here at home that runs off propane. It kicks in when the power goes down, though there is a very short delay (less than a few seconds), so do have things connected to a UPS to cover the hand-off time. Don't forget that when the power from the grid comes back, that will be another hand-off. 2. Others have mentioned a transfer switch and if you go the whole home generator approach, there will be one to handle the process. Do call in a good electrician for this and check for any building codes and permit requirements as everywhere is different. 3. If your UPS can work with an extra battery pack, that will be cheaper than a new UPS, but having two is a good idea so that one can take over if the other fails. The battery pack will get you more time, but not more power. Two UPS's can also split the load you want to deal with, but regardless, have those on individual electric circuits (If you have any window air conditioners, do that with those too. I have one in my room and it does make the light in the room dim for a moment sometimes). 4. Notes on UPS's (Uniterruptable Power Supply): A. These can get hot and if you are dealing with batteries (Sealed Lead Acid, or Lithium Ion) those can swell up which can make replacement difficult. I should know I had two 1320W UPS's that used Sealed Lead Acid batteries and it took some work to get those out for replacement. B. Prices have gone way up. I paid $300 for each of the Powerconn King Power 2200VA 1320W UPS's I had. Batteries ran me $50 for four (That's the number of batteries in that unit). Now, trying to find that same power level in Watts is very difficult. When I last checked, it was $600 or more for the wattage I'm hunting for. C. Do get a UPS with replaceable batteries. Sure, the new batteries you get may only last a year or two, but that can be far cheaper than replacing the whole UPS. If it's a $60 unit you got at Best Buy or Micro Center, that will be less wattage and probably not have a user replaceable battery. D. About how UPS uptime is measured: Most will list the uptime as a half load measurement (ex. 200W UPS might say 10 minutes of uptime, but that means that if you want all 200W used, then you only have 5 minutes of uptime at that level). Some have started listing full and half load uptimes, but it may not be so with every manufacturer. E. Pro tip: If your UPS can have the batteries swapped, do take photos of how they are set up in the UPS. This will save you headaches and electrocution damage or danger. Do take proper precations, as if things are done incorrectly, you may find things damaged and almost welded together aside from the danger to your well-being both physically and mentally. Hopefully, these few points can help out, if not make for some slight comedy. If you're wondering why the 1320W: 1. 1200W power supply in my main computer. 2. The monitor I'm using consumes 100W when active, 1W when in sleep/standby mode. 3. I have some USB hubs I want to keep powered (a 7 port USB 3.0 hub and a 10 port USB 2.0 hub. Some might say ditch the 2.0 if possible, but I like keeping those apart). For those curious, my home setup includes the following: 1. A 2 port HDMI KVM switch that gets power over USB, and that plug is a USB 2.0 connector. I could have gotten an AC adapter for it, but my powered USB 2.0 hub was nearby, so that got drafted. 2. I do have both a Logitch Gamepad F310 and a Logitech Wingman Extreme 3D Pro flightstick (I use one or the other as unfortunately some of the games I run now have issues when there are two controllers plugged in even if I set one as a default. So, having that 2.0 hub nearby means easy swapping, no need to reach behind the computer or try to hook up the controller in question to a side mounted USB port and risk cables getting broken at the connector (I've had that happen). 3. I have an HDMI to USB 3.0 video capture device (this uses USB for power, but that I have an AC adapter for). 4. I have an HDMI switch that handles 5 inputs, so the old 15 inch ATSC TV (it only has one HDMI port) gets to act as a second monitor with that and as a passthrough for anything I hooked into the capture device (capturing programs have a bit of delay showing recieved content, so having the passthrough means I can see what's coming in and properly deal with that. I'm using it for game console capture, but I can get video from anything that can output an HDMI signal or be adapted to do so). I also have a cable so that I can have that TV used as a third monitor for any laptop I hook up (I have my own and I sometimes do work from home jobs where there is a company supplied computing device, which typically is a laptop). 5. That 2 port KVM switch has four USB 3.0 ports, which get keyboard, mouse, headset, and webcam duty (my setup has me on the very right with a laptop ending up on the far left, so if it has a camera, I'd have to turn the laptop to use that, while the camera I have is already mounted on the monitor I'm facing). That's all for now.
Highly recommend keeping everything on a UPS. If you lose power, whatever is not connected is a risk. I lost 4 servers and a NAS when someone hit a generator down the street and took out the power.
We got hit by Helene, and was without power for 9 days, and internet was gone for almost 2 weeks. We've decided to start working on all the pain points that we experienced with the outage. We bought a charger for phones, and batteries for the many flashlights I have, (which came in super clutch). We got a small fan that has LED lights, and also has a power bank built in. It's funny that you mention the generator buzz, because we used that to put us to sleep at night. You don't realize just how dead silent it gets when everything is shut down, and we are the kind of people who run a stand fan in our room year round just for the white noise. Between the silence, the stagnant air, insane humidity, and how hot it got, getting to sleep was almost a Herculean feat.
Here in England we complain about the weather being dull and overcast BUT we are actually lucky as we don't get the extreme weather that others like yourself do, glad your ok : ) x
Glad you're ok after all the storms. When we moved to NC I was grateful to have a Generac whole home generator already installed to our house. So as long as we're getting natural gas...we're good. I added battery backups in the garage to use up first if we're only down a day or two. Keeps us from having to deal with the generator noise. It was a cheaper fix vs the 80k solution 😂
Watching from Bradenton. We got super lucky during Milton with no damage. We did lose power for about 3 days as well. I looked into an Inverter as opposed to a generator since they are supposed to be quieter. One was even "dual fuel" so it could use gas or a standard propane tank. Something to think about as an alternative until you get the whole home generator.
Greg, look at a propane inverter generator. Propane lasts forever in storage and you can run delicate electronics on a inverted generator. Also you can stock pile proane tanks; you can buy empty tanks at lowes or home depot (or where ever want) get them filled at the propane supplier of your choice.
That dust you pointed out is because the rack's sitting on carpet. Don't disconnect servers from the UPS, losing power will cause data corruption at some point. Use NUT to shut down the servers earlier than the network gear.
We live in Texas and have a whole house generator that runs on our natural gas line. After Hurricane Beryl hit our neighborhood lost power for almost 6 days. With our generator it was like we never lost power it was great to have the A/C running since it was so hot. It is loud but you will easily learn to deal with it if you have power. If your gas and electric are close to each other it is about 12 to 13k, so much cheaper than power walls. If you do get the power walls you still may consider a generator to charge them up in the event of a much longer power outage. Our next house in a few years will be solar with batteries and a standby generator to charge the batteries if the solar generation is too low.
I am glad you and your family are okay Greg! I live in Illinois but my daughter lives in Port Richey, FL. Everyone is okay, but her mom and I were worried sick during the storm.
Greg, what you should do is build a bank of switches to be able to turn on/off certain accessories. kind of like a power srip, except each outlet is switched so you turn off the particular unit. Good to know you survived the weather, stay safe and keep the videos coming!
Glad that you and your family are well. Since there are a lot of good advice about keeping all your servers hooked up to UPS, I will just add something about fuel generators. If the generator is not specifically made to handle computer equipment, the power it generates is usually very low quality, with high spikes and low dips. It's ok for powering the lights, fridges or tools, but not so good for directly powering server equipment. So if you ever will be in the situation where you have to use run-of-the-mill generator for PC/server equipment, do it through UPS, that will filter this power and provide a clean one from batteries if it goes too much out of specs. Also having an "online double conversion" type of UPS for your gear is the best one if you experience often power blinks or have overall dirty power in your location. Those use big capacitors to do a double conversion from AC to DC and then DC to AC again, producing the cleanest sinusoid power. If you add a second UPS to your rack, then for any device with redundant power supply, just connect one PSU to one UPS and second PSU to other UPS. That way you will be able to maximize the time your hardware will be functional in case of the grid failing and you will be able to protect yourself from UPS failing and taking down your whole group of devices offline. Or you can purchase a PDU that will accept two power sources and has a built in switches - this is good if you want to have power redundancy for equipment with only one power brick.
Hey Greg i live in nsb. 4 25 years refreshing to see a young person with the same logic about fla. Yes love the state but the people sometimes r trying..lol..and spectrum dont get me started!!!!. love the content. from a retired pc tech shop owner. keep up the good wok glad you made it ok though the storm.
Keep everything on the UPS. They don't just help with blackouts, but also brownouts. I've had a computer die from brownouts (drive got irreparably corrupted) that would have been preventable had I had a UPS. Any good UPS is also sinusoidal, so they also smooth out "normal" power to your devices.
From the Netherlands, good to hear you and your family didn't had much severe damage from the hurricane and sadly this cannot be said for all the people who lived in the area where the hurricane has moved over. When the hurricane was on the news (yes, in the Netherlands, hurricanes in USA is usually a hot item) I was a bit scared that one of my favorite RU-vidrs (and his family) would be hurt or would take severe damage from the hurricane but I am glad that didn't happen. 😊
TechnoTim has a couple good videos and blog posts on NUT, but I will say it's not for the noob. You can figure it out but I would definitely classify it as more advanced to get set up properly.
You could do something different too. There is a thing called STS which stands for power load balance unit. It balance the power from 2 sources(UPS and Generator). And if you get a beefier UPS something like a 3k or 5k unit you will be good to go with network management and notifications too
You may want to look into a DAC (Direct Attach Copper) Cable when going between your UDM Pro and 48 Port switch. It will increase your bandwith between the two devices in giving you a 10G connection in everything internally. This may not mean a lot, but remember everything goes from the switch to the router and back. having a 10G connection will between the two will not slow down the network in things such as file sharing. you can saturate each connection, with no bottleneck.
Could just do 10gtec sfp+ modules and short fiber jumpers (this is what i have), will probably be close in price and the cables are MUCH thinner and more easily managed.
Hey Greg,here in the Philippines many of us Expats turn to" EcoFlow DELTA 2 Portable Power Station", They sell in the US and are very long lasting. Once properly rated and configured they can run your entire house during outages, best option put it on a transfer switch for non-interruptible service. beats having to gas up the ole genny every couple of hours. Hey they can even come with their own dedicated solar charging system.
For the off the grid side of things youd be well served by looking into something like anker solix. The batteries themselves are portible and could be taken on a trip or moved to a new house if you ever move, with their automatic transfer switch you could have it wired into your house as essentially a whole home ups for battery back up and also have the option to set them up for peak trimming to help reduce the electric bill. If you want to add solar as well its super easy to hook up portobile or permanent panels to them since they have built in solar hookups and the inverter.
There are a lot of whole home battery options that do not require solar panels to be mounted. With the right setup you can keep the generator around to help recharge when they're low. Plug in panels that you can lay out when needed for charging as a backup and some folks even use them as a way to save on their energy bills by running off battery power when rates are high and grid at night to recharge to full. Bluetti, Anker, & Ecoflow come to mind.
For whole-house backup, something like the EcoFlow Delta Pro Ultra could be a good solution. It's modular, so you don't have to buy the whole setup at once. Plus, you would plug it in where you would normally plug in a home generator (e.g. "breakout box"). Once up and running, you can buy battery modules to increase capacity as your finances allow. This keeps the costs more reasonable than an all-at-once built-in battery system.
You don't need to be completely off-grid or self-sufficient for a good amount of solar panels and a battery to cover you! If your battery is hooked into a changeover switch with only essentials on it (fridges, freezers, internet etc.) you can go for quite a long time on a modest 10 kWh battery and 6-10 kW of solar. You'd be surprised how little power your essentials use - most energy use is in heating and cooling.
I'd seriously look at an LP powered generator with a big tank buried in the yard. Not only does LP gas not go bad like gasoline does, but you could also use it to run a gas grill, backup heat, and you could even upgrade your kitchen stove to a gas one. I did that, I love my new stove and during power failures I can still cook.
Rackstuds help when mounting some of these devices as it will more-or-less hold it in place while you tighten everything down. Use them in my entire rack.
I know this is not a prepping channel but a trick I have seen online is to store gasoline for about 6 months,start using it in your car so it doesn't go'stale' and then refill your supply. If you are going to use gas gennys,it is a definite plan.
It's definitely got one of the best climates. But your taxes and home prices are too high for my taste and business. I've tried to justify it in the past but it doesn't work out.
The fires are not our fault. It's mostly on federal land which the federal won't thin or prescribed burn. Calfire does a great job with state owned forest.
I'm just north of Tampa on the coast and lost power for about 24 hours. Spectrum was up before but I had no power to the router/modem at that point. Cell towers were completely useless during this though.
Milton kicked my a$$, Clermont area. Been through all of them in the CFL area since 2000. 10 freaking hours of 40-90 mph winds. Had TWO tall pine trees go down. Thankfully they didnt hit the house. Bought a generac 3 years ago so we would have to go without power. It broke during the hurricane. We only lost power for 20 hours or so. Thankfully the weather was good so the cleanup wasnt the usual hell on earth scenario. Also very thankful that mobile didnt go down in my area. Glad you got through it without anything major. I can recommend a generac. although ours went down during the hurricane, my neighbors had theirs running through the whole thing. You can get a generac for about $6k, get the PAD done for like $1k. If you have natural gas access already hooked up the house you are set. We werent and to get hooked up to natural gas it was around $4k. generac came out and they are taking care of the issue they found. get the extended warranty of you go that way, great value.
I use a generac generator and installed a small propane tank just for it. It sat for a year then when i needed it, it ran for four days in a row no issues could not imagine not having it.
If you are using a generator gas, propane or LP based test it monthly to make sure it works this includes the cutover switches to make sure everything is working.
25 Minutes on the UPS, is about right, it gives you enough time, to get the generator up and running, so that you have generator power before the UPS runs out. Most power failures are either under a minute, or run for hours, The short ones can be more damaging then the long ones. What you CAN do, is have the UPS send signals to the gear, typically over USB these days. You really want the network to shutdown in stages, so everybody gets the power failed notice, and uses a timer to count down, then checks to see if AC is still off, and initiates a shutdown/power off. So for example everything is plugged into the UPS, the servers have a 2 minute timer, then run a shutdown script that powers off. The last thing to power off is the WiFi box.
@GregSalazar IMO, you should keep your NAS connected to the UPS and trigger it to shut down in an orderly fashion after a few minutes of a power outage.
As others have said, I would still have everything on a UPS. I have my networking equipment on it's own UPS because I want it to stay up while the power is down and everything else is on it's own UPS to protect against voltage dips and momentary power outages.
You can always grab those stackable EcoFlow Delta Pro external batteries if you live where power outages are a common thing. I know they're expensive but maybe they can send you a few for "sponsor purposes" 👀👀?
During the storm id considered starlink since my spectrum went out (after the storm had passed, had it the whole time it was on top of us, go figure). Id appreciate a video on how to set it up and a 2nd wan purely for a backup and "tips" on how to get it setup on an as needed not a monthly bassis bill, as id onlybreally need it during such events like these.
For a longer outage a good method is the power walls you talked about and a way to if need be tied the generator in to recharge them in case of a longer term outage that way if you need more power turn gen on excess power charges batteries and once you have enough juice turn it off no more noise also you can get propane generators and get a small propane tank installed same with gas and gas tanks realtively inexpensive
If you have Natural Gas in your home. You should definitely go with a NG power backup generators Pennies on the dollar compared to SOLAR. And it will always work. Will power your entire home. I live in Titusville and before I moved here I owned one up north and it was a blessing. Although I’ve always had Fiber internet everywhere I’ve lived when it became available “ I’m old “, it’s never went out during a storm. Only when they were doing maintenance or service after a storm. But imo a NG backup generator is way better.
Proper setup is everything connected to a proper UPS and software to shutdown nonessential equipment. Putting it off UPS removes a bunch of protection to the equipment. Personally I prefer diesel backup generators (or natural gas).
if you have the room to install it in your yard. you should look into an auto start diesel generator. it sits on a concrete slab and you bolt it down. they are very large and can run your entire house.
Greg, i agree with just the battery bank hooked up. There are many different battery banks that are much more cost efficient than a Tesla wall. As for the backup generator, the big box stores do carry ones that are dual fuel. If you already use propane tanks, it burns cleaner and doesn't need a fuel stabilizer in the off season. At least that is my penny advice. Caffeinator out.
Note on the dual/tri fuel generators, they have different power output ratings for gasoline, propane, and natural gas so make sure you get one bigenough for the fuel you plan to use. Unfortunately I don't have access to NG (least powerful), so my only options are gasoline (most powerful) or propane (second most powerful), I'm not sure how much it effects run duration....
@@AceBoy2099 Thank you for putting that out. I honestly thought I had mentioned that; but, it just was said in my mind. I know that it is a much lower power output, but as long as it can power my refrigerators and refill our UPS's, I can't complain. Luckily I have access to NG, I just want to install a quick connect for easier access.
I also choose to live here in FL mainly for the same business reasons as you said too since I do run a RU-vid channel but not as close in terms of subscribers and not paid by RU-vid yet.. but I do run a side hustle business but one day if it becomes big that I can make a living off of it then yeah that would be make sense here in FL. My family lives here in FL, plus I grew up here in fl ever since I were born. I totally agreed that you said you don’t want to live in the coast, especially for insurance cost and stuff. I’m planning on doing a similar homelab/ server setup as yours as soon as I move out of my parents home.
I just bought the Ubiquiti Enterprise 8 PoE, gives me 8 2.5gb poe ports, then 2 SFP+ ports for 10gb between my desktop pc and my server. Super happy with it.
As much as florida gets hit by hurricane id have a ups and a generator to recharge the ups and power basic stuff. I have one that will run my fridge and deep freeze and my tv and computer. It will run some fans and space heaters in the winter too.
Maybe propane or natural gas generator? I have a propane generator on demand that comes on when power goes out and the propane company monitors the amount I have so if it gets low they automatically come. Not sure what utilities you have down there but just a food for thought. Looks great man. House looks great also!
If you have run out of gas; remember your vehicles still have gas in them. Use a manual transfer pump designed for gas transferring. Grab a gas can and it will get you by a couple more days on your generator.
You will definitely want to keep the UPS for the server even after getting the Powerwall installed. The powerwall cannot switch fast enough to be an UPS from what I understand.
What do you mean you don't have $80k lying around for solar?!? Jokes aside, glad you all are okay. The server rack is coming along nicely. RU-vid just suggested to me your first server build from last year so it's fun to see how far you've come
