Server Room Air Conditioning Failure (Warning) 

Absolutely, I could hear the fans going berserk from down the hall! Not fun at all, we lost a server that day!

Luckily for us we just lost a few drives in our ZFS arrays. Now we have each machine's OS scripted for thermal shutdown limits. You never think you'll need those scripts until something like this happens lol.

SIN3R6Y Scripting my personal home server for thermal shutdown probably saved its life, like, 3 times!

Do server CPUs not turn themselves off when they reach maximum temperature?

Really proves viability of data furnaces to heat buildings

@Android Developer Whats funny is about two years ago we had an issue where the entire school district’s internet went down where someone hacked in and released some sort of a virus. It was some sort of ransomware and I think they shut off the internet connections to prevent any further spread.

@Android Developer agreed

That’s where probably the grading system that teacher entered sits in the school server system.

Jake Billing did she get it all in her house to keep warm? Lol

Are you the IT department or the third party consultant??

We had something like this. our 2 big ACs in the small company datacenter went down, we were wondering why some servers are down. we go to check, "wow the servers are really loud today", went in, wall of 70-80° heat came out, he runs over to the windows, opens all up and checks everything. the servers at the top were off because it was too hot. Then we developed a monitoring sensor system for our monitoring software with a raspi and 10 sensors over 1wire.

I love this comment!

How do you know they didn't? It doesn't seem like he's going to do anything about it at the moment besides record video anyway.

@@NillKitty He appears to have turned the thermostat back on.

@@NillKitty one of the psus died, there’s not much he can do in the moment, especially considering the hardware needs to actually cool off first

@@toranine09​​⁠there is though. Rush over to get the door open to let out the hot air. Bring in exhaust fans immediately instead of making a video nonchalantly. Shut everything down. Find out why the air conditioner was off.

Or you follow industry standards by installing a redundant AC...

Some Guy is that really in “””industry””” standard? so every small company to every large company is expected to run two ac units. And how would that help in a power outage that shuts the ac down for some reason. You could argue that they could use backup power for the ac but that’s an immense amount of power.

@@UlrikBrun-ir7mh uh, that's a giant pile of bull manure

@@UlrikBrun-ir7mh LMFAO

@@UlrikBrun-ir7mh that is not how it works. It's a matter of the compressor compressing refrigerant which increases the refrigerant temperature, which is pumped to the outdoor unit (in this case) to dump the heat. Then the refrigerant pressure is dropped, resulting in cold refrigerant which goes to the indoor unit (in this case) to absorb heat. Repeat the cycle. Basic science principle. If you increase pressure of a gas under constant volume, the temperature increases, or decrease pressure and temperature decreases.

My servers sound like that aswell... But I put them there so i don't need air conditioning and also mine have Gpus in them that need a lot of cooling

I do what my bosses tell me, unless what they tell me is stupid. In which case I tell them and why. If they still don't budge, I go above my boss and sometimes to mutually trusted and respected equity partners in the firm. And if that doesn't work, I find a new job at a workplace which has not become dysfunctional.

Travis Weir Sounds like a U.P.S. was the beeping thing.

Ok, I thought that's what it was but wasn't sure.

@@TravisWeir Yeah UPS, i think the AC tripped a breaker when it blew, probably the normal room breaker while the servers are on their own PDU+UPS, so the watchdog caught em when they overheated, or when they rebooted. Is why my PDU has luascripting in it for overheating scenarios to trigger EPO, or alternate cooling solutions

The only time the fans in an Apple product have been audible haha

it's a G5. Mac Pros have 2 drive slots in the front.

@@jamescollins6085 x serves and g5 powermacs are absolute screamers even at idle at room temp

@@jhonjhon1740 True. I've seen videos on the G5 systems, and they sound like vacuum cleaners.

Holy shit that's hot.

Holy shit, I keep the heated plate on my 3D printer that hot, it's enough to deform plastics

2 3TB HDDs

@@Nevexo287 they lost one server

Thank you so much for doing the conversions for me. :D

no problem, My pleasure :)

Yes and three is highly recommended, because with only two, the instant that one fails... the load on the remaining unit instantly doubles! Dramatically increasing the risk of it suddenly dying too. The number of times I've needed to tell colleagues of this, only to have them claim that the risk of two failures in quick succession is extremely unlikely, has been infuriating for me. More so when I was then proven correct on a number of occasions. One time we had a PSU fail in a SAN disk chassis and as a consequence all the fans in the unit then went to 100% speed (increasing fan failure risk AND increasing remaining PSU power usage) and the load on the remaining PSU thus more than doubled. I warned our storage guy (I was the netsec guy, former server admin) that the faulty PSU needed to be replaced ASAP, because of the dramatic and sudden increased load on the PSU and he insisted that the likelihood of a quick succession failure was low (just based on very rudimentary ideas of probability which completely ignored the age of the equipment and the sudden more than doubling of load for a prolonged period of time). So he told EMC (our SAN provider) that we did not need to use the highest priority callout replacement and that 4 hours response would be fine........ Shortly after, the fan in the remaining PSU died, so that PSU shut down and thus *ALL* SAN LUN's associated with the disks in that chassis went offline and all associated VMs were suspended due to write failures. A major disaster for the firm, due to so many systems going offline and staying offline until they could be brought back up, pending writes written and filesystem and database consistency checked. If they had heeded my warnings and kept cold spares for key items, like I recommended... 🙄

True, however this is a school by the looks and they dont have the budget for that lol. My high school didn't even have A/C, there was an open window and a fan wedged into it and that was it.

When you run EA SPORTS F1 2023 with a Pentium and Windows XP

He purposely turned it off.

+Channel Tech it sounds like the school lost power overnight turning off the AC's, but the servers were still on from and UPS's

+Elliott Gamer Would be my guess aswell.

I think the channel is HVAC, its an AC repair men (or what you call it), and from his storys you see how often they fail if nobody looks after it

Damn Americans with our SI units, it even took me a second 😂

Maybe their servers are good but probably faulty air conditioners? Well it could be.

And if you dont want to spend so much money, a small diy build with an esp microcontroller would cost you less than 15$ in total, and has everything to monitor temps, send it over wifi to your network and if necessary even shut down your servers. And for redundancy you can built 2 of them, link them so if one goes "alarm 1000 degrees in here" it can check the other one to prevent accidents.

Celsius Indeed 👍

No way!! American moment lmao

This place looks like it might be a hospital. Nothing but the best technology is used in healthcare!

@@redsquirrelftw it's a school

@@redsquirrelftw ok american

thats sounds way too overkill for a school XD

@@legominimovieproductions it's more of a engineering college/university

Cryogenic chamber as a server room?

Cigarettes is a danger too.

it's a school

American moment 🐴🐴

Remember to always use C when dealing with electronics temperature, even if you're American.

Recorded: July 2013 jk...

Ok american

More AC units are better, plus it also cools off areas that are otherwise unreachable.

40c is normal temperature

Can't imagine how hot the actual equipments are? When it reaches 70-80 degrees it's high risk of overheating.

Elias Müller it's a KVM

Thx you

Seriously, even with the door opened it was unhealthful for you to be in there alone.