BIG AI Servers NEED This

Подписаться 623 тыс.

Просмотров 135 тыс.

50% 1

We take a look at a state-of-the-art liquid cooled NVIDIA H100 8x GPU server from Supermicro. We show why modern AI servers need liquid cooling. We also show some of the other cool bits from the company's liquid cooling lab including 2U 4-node servers with Intel Xeon and AMD EPYC liquid cooling blocks.
STH Main Site Article: www.servethehome.com/supermic...
STH Top 5 Weekly Newsletter: eepurl.com/dryM09
----------------------------------------------------------------------
Become a STH YT Member and Support Us
----------------------------------------------------------------------
Join STH RU-vid membership to support the channel: / @servethehomevideo
STH Merch on Spring: the-sth-merch-shop.myteesprin...
----------------------------------------------------------------------
Where to Find STH
----------------------------------------------------------------------
STH Forums: forums.servethehome.com
Follow on Twitter: / servethehome
Follow on LinkedIn: / servethehome-com
Follow on Facebook: / servethehome
Follow on Instagram: / servethehome
----------------------------------------------------------------------
Other STH Content Mentioned in this Video
----------------------------------------------------------------------
- Liquid vs. Air Cooling NVIDIA A100 servers: • Liquid Cooling High-En...
- Touring the Liquid Cooling Lab: • Touring the Liquid Coo...
- How Server Liquid Cooling Works: • How Server Liquid Cool...
- Liquid Cooling Takes over SC21: • Liquid Cooling Takes O...
----------------------------------------------------------------------
Timestamps
----------------------------------------------------------------------
00:00 Introduction
01:54 How Liquid Cooled AI Servers Work
05:01 Liquid Cooling an NVIDIA H100 AI server and high-density Intel Xeon and AMD EPYC servers
12:03 Supermicro Liquid Cooling Power Savings
12:31 Why we need liquid cooling

Наука

Опубликовано:

11 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 184

@BiswajitSahu_bisudw 9 месяцев назад

at 5:48 saw a RaspberryPI CM4 is being used as a controller for the liquid cooling.

@logskidder5655 9 месяцев назад

That cooler controller (at 5.50 minutes) was based on a R Pi Compute Module 4.

@ServeTheHomeVideo 9 месяцев назад

Yes, and we have seen several other CDUs use RPi's over the years.

@LeonardTavast 9 месяцев назад

It's fascinating to see a quite ordinary/hobbyist friendly SBC being used as an embedded system like this.

@AriBenDavid 4 месяца назад

I just toured a large data center where experiments in liquid cooling were taking place. They were abandoning the experiment for now as the coolant fluid is considered toxic. Further, conventional cooling including liquid requires huge water usage.

@ServeTheHomeVideo 4 месяца назад

You are right that two phase immersion cooling stopped because the fluid was toxic to manufacture. Direct to chip liquid cooling is usually heavily purified water with anti microbial additives in a loop contained in each rack. Generally D2C can use a lot less fluid and is not an oil base

@AriBenDavid 4 месяца назад

Thx. Interesting that the primary coolant water at Three Mile Island reactor 2 had anaerobic organisms that had to be killed. A lot more heat there at 600 degrees!@@ServeTheHomeVideo

@kellymoses8566 2 месяца назад

Most liquid cooling uses water. Only immersion cooling uses not water.

@AriBenDavid 2 месяца назад

@@kellymoses8566 Somewhere down the line there will be water. The heavy use occurs in the cooling towers. Same for nuclear or conventional power generating plants.

@FireStormOOO_ 9 месяцев назад

Damn that looks slick. Looks like they're doing a lot to reduce the pucker factor of having that much water all around your expensive servers.

@Nobe_Oddy 9 месяцев назад

That was SO AWESOME seeing a CM4 controlling the all the cooling of that entire rack!! HAHA!!!! :D

@JeffGeerling 9 месяцев назад

5:54 I spy the Raspberry Pi!

@C0MPUTERPHILE 9 месяцев назад

And they didn't even liquid cool the CM4

@user-yn2ge6of9q 9 месяцев назад

@@C0MPUTERPHILEر:++=⅜

@eldibs 9 месяцев назад

Just one of those GPU cold plates handles more wattage than my entire gaming PC and my home server put together. Wild to think about.

@rezenclowd3 4 месяца назад

It's not the cold plate, it's the rad capacity at the other end or whatever heat exchanger system they are using which I wish was shown. Either way, still burning the planet to produce...something. I wish it wasn't AI...

@woolfel 9 месяцев назад

that's some sweet modular liquid cooling setup

@MelroyvandenBerg 10 часов назад

This.. THIS.. THISS!.. This is Patrick.

@Veptis 9 месяцев назад

I am close to building a workstation and liquid cooling seems to be the only option to reach the level of density I want. But external liquid cooling is something else.

@ServeTheHomeVideo 9 месяцев назад

Yea I wish we had more external cooling options for workstations as well

@pingtime 9 месяцев назад

5:50 now we know why those CM4 boards going so scarces 😂

@morosis82 9 месяцев назад

Supermicro are such an innovative company and it's awesome to see them out here promoting RU-vid creator content. I'm always amazed at the level of access you guys get to just show off their stuff, from your channel to LTT, etc. Absolutely massive respect for them to put this stuff out there.

@ServeTheHomeVideo 9 месяцев назад

I think we have been reviewing Supermicro products on the STH main site since like 2010 or so. It is fun that they just let us stop by and do this stuff.

@superdave4564 9 месяцев назад

Been talking about this for a week! Thank you for all you do!

@OsX86H3AvY 9 месяцев назад

they're using a raspberry pi in that controller @5:50 - that's fantastic!!!

@ServeTheHomeVideo 9 месяцев назад

They are VERY popular in CDUs

@MrHav1k 9 месяцев назад

Love this kind of content!! Didn't know air cooling was that inefficient.

@mddunlap03 8 месяцев назад

I'm trying to figure out how your using the same amout of power to cool the water and pump it that you get a savings somehow. Also they still use fans to on the end of the rack to cool the DC power supplies so....notice theu use some weird math and don't just say look we will cut Power usage. All the liquid cooles systems I have seen where submerged in a chemical bath that was phase change it's wild but the chemicals are pretty toxic.

@dawnv3436 9 месяцев назад

Hey! At about 6:30 in you mentioned and showed one handed operation of the quick mounting. THANK YOU! I don't need to know that yet but I might eventually and I appreciate the tidbit! One hander here.

@pokeysplace 9 месяцев назад

I'm really hoping this trickles down to the home lab environment as my lab runs at 88F (31c) year round. It would be straightforward to pump a liquid outside to a simple radiator for me. I just need the cold plates and manifolds and couplers that I'd trust.

@ServeTheHomeVideo 9 месяцев назад

We also need this gear that is not crazy expensive for it to be lab-worthy.

@timramich 9 месяцев назад

I did that more than 20 years ago with hardware store parts and a part from an automobile. The water block was Swiftech, back when they were just anodized blue chunks of metal with just their name on it in some cheap font.

@jonathanbuzzard1376 9 месяцев назад

The issue is going to be the cost of the heat exchanger. You don't just pump "facilities water" into the servers on these things. That comes into an in row or in rack heat exchanger and as the water that goes into the server has to be extremely "pure" in that it must be free of particulates which is impossible to achieve pumping it through piping to a radiator outside. The heat exchangers run to thousands of dollars. We seriously looked into it at work with the last system and decided that we would stick with water cooled rear doors on the racks. The only issue is we where persuaded that passive rear doors where better than active ones. They are not and in the next upgrade we will probably need to change them for active doors. Passive doors are good to ~30kW cooling; active doors are optimal up to 65kW and can go to 93kW of cooling.

@timramich 9 месяцев назад

@@jonathanbuzzard1376 A radiator is a heat exchanger. It exchanges the heat in the liquid to the outside air. The water doesn't get exposed to the air, so you don't need a heat exchanger to run a radiator. Regardless, liquid-to-liquid heat exchangers are pretty robust, in that they can transfer a lot of heat for their size and cost.

@jonathanbuzzard1376 9 месяцев назад

@@timramich Look my job is looking after an HPC system. *ALL* of these in server water cooling solutions require vendor-specific water to water heat exchangers that new run to many thousands of dollars each. Exactly how much depends on whether you go in rack or in row. You cannot cycle the water to a radiator on the outside of your house the specification for the purity of the water is too high for that to be viable and the vendors are quite explicit about that. We have had various vendors come in and give us presentations on their solutions. I have the quotes. This technology is never going to filter down to the home lab unless you have a five figure budget for your home lab and have as much again per year to spend on the power. The latest eight way H100 DGX box draws 10kW of power at full tilt. I have a meeting this week with the data centre manager and university electricians to decide how to get more power into the racks to support *exactly* this stuff. The limit is not the cooling the active water cooled rear doors we have on our racks are good for up to 93kW of cooling. Even if I get three 3-phase 32A PDU's into the racks that is only 66kW max of electrical power I can draw so at most six DGX boxes, so cooling is not the problem. Even if I was able to get 80kW of electrical power into the rack to allow eight of these servers to be installed that is 40U and my rack is full, but the door can handle 93kW. TLDR unless you do this for a living you have no idea.

@kungfujesus06 9 месяцев назад

The thought of a data center buying multiple trucks worth of distilled water is kind of funny.

@ServeTheHomeVideo 9 месяцев назад

If you saw our PhoenixNAP tour video, a lot of these facilities purify their own water. Typically they are not running tap water in loops :-)

@kungfujesus06 9 месяцев назад

@@ServeTheHomeVideo that's even funnier to imagine, distilling their own water. I imagine it's rarely flushed and replaced once it's in the loop?

@user-po7se7go6t 9 месяцев назад

very interesting! As always, an exciting video that keeps the viewer watching. The quiet background music was a bit disturbing. they could have safely been left out.

@realandrewhatfield 9 месяцев назад

Even more work for plumbers, remember that the next time you build your home lab! :)

@TAP7a 9 месяцев назад

The suitcase spin at the beginning was buttery smooth

@ServeTheHomeVideo 9 месяцев назад

Over 1.1M miles of travel with that suitcase. Bound to spend some time walking through airports.

@mikegrok 9 месяцев назад

One of the benefits of liquid cooling is the water temperature is warm enough that it can effectively cool the components and still be over 20 degrees warmer than the air temperature in the hottest parts of America. They can run the water loop at 140f to 160f, and cool it off in air heat exchangers down to 120f without the use of compressors.

@tappy8741 9 месяцев назад

Liquid has an incredible energy capacity and liquid heat exchangers are efficient. Win win

@snake_00x 9 месяцев назад

I need one. I don't know what I need it for yet, but I need it ASAP. Great video.

@bits2646 9 месяцев назад

Loved to see rPi cm4 seeing in controller module :)

@OpenAITutor 2 месяца назад

This is fantastic! Thank you for your efforts; they're truly inspiring. I hope everyone recognizes the importance of the work being done at SuperMicro.

@tristanwegner 2 месяца назад

liquid cooling also makes it easier to transfer to district heating as a secondary use of the waste heat

@ServeTheHomeVideo 2 месяца назад

Very true

@g.s.3389 9 месяцев назад

did you notice that they are using a raspberry to control the water distribution unit (5'50'')....

@nicknorthcutt7680 6 месяцев назад

San jose is my hometown, I miss SJ and the Sunnyvale/Mountain View area, so much cool tech stuff in that area.

@ServeTheHomeVideo 6 месяцев назад

I miss it a bit too :-/

@richardgalarza7882 Месяц назад

He does not raise the safety concerns of DLC. Leaks can be catastrophic to a datacenter. Leak detection development is a must!

@ServeTheHomeVideo Месяц назад

All of these have sensors on them. That has been standard for a few years now.

@movax20h 9 месяцев назад

5:50 OOoo. Rasberry Pi compute module in that Supermicro CDU. And USB stick for flash. Nice actualy :) Makes it easier to hack stuff later, or reuse.

@InquiringJoe 9 месяцев назад

One challenge with water cooled systems is resale value. That is, the cost of the initial startup system is so high, and you need everything integrated to avoid failure. So, on the secondary market, when the time comes, they are generally prices lower than air cooled alternatives. There is a similar issue with SXM, where you’ll see the same GPU going for much cheaper prices on the secondary market versus PCIe alternatives. It is curious how SXM together with water cooling will fare in the long run. I’m not sure NVIDIA can get away with pricing their GPUs so high even in the short term, so the total system cost may begin to far outweighing the compute cost may be problematic. Isn’t it obvious that the chip manufacturers should lower their power consumption targets to avoid these issues? Even if we accepted that long term economic risks of water cooling, their complexity would be reduced with more reasonable energy requirements. Why doesn’t NVIDIA refresh the V100 or A100 with more ram on newer nodes?

@ServeTheHomeVideo 9 месяцев назад

Power is still much cheaper than a GPU so it is unlikely power is going down. 1kW accelerators and 500W CPUs are very close to being mainstream. You are right on the liquid cooling. A big win, however, is if you can re-use the liquid cooling for multiple generations instead of replacing the fans/ cooling with each generation. A really interesting part is whether these liquid cooling solutions can be used second-hand not for the resale value, but more for the reuse economy.

@InquiringJoe 9 месяцев назад

Increasing power makes sense when you put it that way, all else equal. Having reusable system components seems ideal, but then the seller’s business model might need to change if they are incentivizing people not to buy new components. However, given how fast the market is growing, that may be good enough to buy time to transition to a supportive structure. Thanks for your great videos!

@Error-403 9 месяцев назад

SuperMicro really need to get that end to end solution to market. My previous employer was way too shaky about having to incorporate the building's chiller system into the server room water cooling that they opted to again go with inefficient air-cooled sytems. Big waste. I love the tech and can't wait to see where it goes.

@ijsklont 9 месяцев назад

At this very moment Supermicro is working on that end to end solutions, however when you start looking at roof mounted chiller systems.... you start looking at govermental building codes etc.... In EU that stuff can be different per country, in the us those rules can be different per state...

@C.J.G. 9 месяцев назад

One thing I am intensely curious about, and I imagine Patrick and others may be thinking about too, is how might this affect the future homelab/deskside server second-hand market. The simple fact is that water cooling is an opportunity to dramatically lower noise levels, and this keeps many away from high end servers. I don't imagine people will be setting up 10 kW DGX systems in their home office, but if systems are quiet, many might choose to set up a rack unit instead of a tower simply because they can. Even right now I would love to use an R930/R940 as a workstation, but the screaming fans make that impossible. Making water cooling the norm might change the game for homelabbers and other deskside compute freaks like myself.

@chublez 9 месяцев назад

Rack stuff really isn't that loud when it's not pinned at max load. On boot at max fan rippems test my stuff screams but once it's up and running it's only about twice as loud as my air cooled gaming rig. That said, it's literally a rack mount server, don't be in the same space, it's kinda the point.

@C.J.G. 9 месяцев назад

@@chublez That's true for some models (like the R730), but even the idle noise level on the R930 is unbearable. It tends to depend on the fans. But also, the purpose of a machine is also to be used, so you will experience high RPM noise intermittently anyway. Even if an R930 is in my garage, I will hear it spin up through two walls and 50 ft. of space.

@coming54 5 месяцев назад

Thanks for the informative video! I noticed that Asetek units, including the CDU, were used in the A100 server. Did Asetek products get replaced with Supermicro's in-house developed products for the H100 server?

@harrytsang1501 9 месяцев назад

Soon you'd need a plumbing license as well as electrician's license to service one of these

@timramich 9 месяцев назад

You don't need a plumbing license to plumb stuff unless you're doing stuff that needs to be checked for building codes (potable water from utility, sanitation to sewer). If you're running plumbing for industrial process cooling, the government couldn't give two poops about a license.

@fuzzyfuzzyfungus 9 месяцев назад

I'm a bit surprised to see the rPi show up. A CM4 certainly does the job and has an attractively low barrier to entry; but I would have assumed that Supermicro would have just slapped an Aspeed onto anything that even resembles a baseboard in need of management out of pure reflex. Is it known whether there's something about an AST2600 that made it unsuitable; or whether the group behind the design of the liquid cooling system is sufficiently distinct from the motherboard side that they aren't in a position to reuse the BMC work already done and so went with the rPi as a better option for working from scratch?

@nexusyang4832 8 месяцев назад

So when could we start seeing geothermal combine with liquid cooling?

@MarekKnapek 9 месяцев назад

There is a data centre with PUE less than 1.0 in Czech Republic. They are drowning the servers in mineral or pharmaceutical oil. The effectiveness of less than 1.0 comes form removing the fans from the servers and replacing them with circuits to pretend to the motherboard the fan is still present. The company is WEDOS and it was all over the news over here few years ago when they announced it.

@ServeTheHomeVideo 9 месяцев назад

Going below 1.0 PUE is very impressive

@m3lbourn3mad3 9 месяцев назад

There are DCs using GPU servers with air cooling achieving 1.1 PUE just by using hot and cold isle containment, and running the cold isle at 35c If you’re building greenfield, you can do enough to achieve a low PUE with air too

@Prophes0r 9 месяцев назад

Fans are above the 1.0 point of PUE. Removal of fans can't get you below 1.0 No amount of anything should get you below 1.0 unless you can somehow harvest electricity from the work being done to power MORE work being done.

@MarekKnapek 9 месяцев назад

@@m3lbourn3mad3 I just watched a presentation by WEDOS from 2022-09-17 (in Czech language only). They said that they have PUE 1.13-1.19 with air. Normal servers, not GPU farm, they are using MoonShot platform. They are using cold/hot aisle + free cooling + A/C. The free cooling is up to 23°C, because any hotter causes lower life of spinning disks. Their oil bath servers use only 70% the energy of air cooled ones (because no fans) plus around 3% for the oil pumps.

@MarekKnapek 9 месяцев назад

@@Prophes0r No, this is where you are wrong. Imagine 100kW worth of servers. You need to supply 100kW of electricity to power them, everything turns into heat. Plus you need some energy to move the heat out, let's say 20kW to do that. That's PUE 1.2. Now rip all fans out (WEDOS say it is 30%, they also say it is 2+6 fans, 28W each, per server). Now you consume only 70kW but have the same amount of compute. Then, add some energy to move hot oil around and you have easily PUE lower than 1.0. They are also heating the offices with this. And are planning to heat up a nearby swimming pool, but I have no details about that, I can imagine the pool would pay them for the heat, that's also better PUE.

@michaelfragrances 8 месяцев назад

Sweden, the heat generated by data centers is used to heat entire cities. The aluminum and copper parts of the radiator look very high quality. Connection from servers looks incredibly cool😎

@glennmcgurrin8397 9 месяцев назад

Any chance these things can run on cooling tower condenser water instead of chiller water? That would really ramp the power savings to the max if you could skip chillers entirely or at least on many days of the year.

@ServeTheHomeVideo 9 месяцев назад

Yes. That is also common. These days it is not like cold/ hot water and often more like warm/ warmer.

@123454142 9 месяцев назад

Hi, what's intrinsic power usage of each CDU for pumps and controls? I looked and couldn't find any data

@MidSeasonGroup 8 месяцев назад

Do you have a homelab AI server recommendation to fine-tune LLMs? Thank you in advance.

@Bob-en1lv 9 месяцев назад

Hi! I deal with a lot of data centre builders through work, and the case for liquid cooling the facilities was hard enough to convince a lot of people because of the extra capital required to offset the opex. If you're saving 10-20% power, how much extra money are they asking you to spend upfront to get that saving?

@ServeTheHomeVideo 9 месяцев назад

Usually the cost of liquid cooling in a rack is much less than a single server. On 8x H100 systems it is a very small cost compared to the hardware

@jonathanbuzzard1376 9 месяцев назад

@@ServeTheHomeVideo You are sort of right, however the real issue is power delivery. Our 11 year old rear door water-cooled 42U racks are good for up to 93kW of cooling, but an eight-way H100 DGX machine consumes ~10kW of power and takes up 5U of space. You can get eight of these in a 42U rack, which means you need ~80kW of power and the existing cooling is enough. Unfortunately, even if I fit three 3-phase 32A PDU's to my rack I can't fricking power them as I am stuck at 66kW of electrical power so only six DGX boxes per rack. I have an on-site meeting with the university electricians and data centre manager to discuss upgrading the power for the existing racks from single to three phase later this week. Just the new PDU's are going be ~70k USD, which admittedly won't buy you a single eight-way H100 DGX machine. If we need a new power feed to the data centre from the basement then we are looking at the price of a DGX machine for the upgrade. Oh and you can't just use the chilled facilities water because it's not pure enough (too much particulates in them), so you are looking at least at in row if not in rack heat exchangers which are not cheap. My take is to just stick with active water-cooled rear doors. On that note passive water-cooled rear doors are not a good idea. Don't give as much cooling and the heat build-up in the rear of the rack is terrible as you are relying on the server fans to push the air out the back over the radiator. We won't be listening to the vendors and getting any more of those.

@CraigLabute 9 месяцев назад

It's great to hear industry mindfulness about how liquid cooling can help reduce the wastefulness of using outdated cooling methods. Better cooling is a win for the planet. There's also great topics to cover there with 208V or 480V vs 120V and DC supply from central rectifiers. Small company DCs can waste an astonishing amount of easily gained efficiency by using 120V power because non-hardware focused engineers think anything that isn't a home wall socket is "scary" or "unnecessary."

@Prophes0r 9 месяцев назад

HAHAHAHAHAHAHAHAHA *gasp* AAAHAHAHAHAHA.... ...it's about saving money. It's ALWAYS about saving money. These companies don't "care" about wastefulness. They CAN'T care about it, because they can't care about anything. They are a chemical reaction that takes in people's work to raise stock prices. Sometimes this reaction results in interesting side-products for us, but those aren't The Point.

@shaun2938 9 месяцев назад

I assume that there would still be system fans for cooling the ram, motherboard etc. Have these changed?

@imeakdo7 9 месяцев назад

@DrivingWithJake 9 месяцев назад

It's quite interesting I feel that the typical data center is going to be a lot different in the next few years. However, I would love to see how much better submersion cooling is vs liquid cooling like this. The problem with submersion is the mess, however I bet the total cooling and cost would be a lot lower. We've done some work on large systems in the past. I think if they could design a rack mounted solution for it that would be more ideal.

@ServeTheHomeVideo 9 месяцев назад

Well the other challenge is that single phase immersion is not that much more efficient than cold plates, but it has operational challenges. Dual phase is better performing, but the hyper-scalers stopped researching them because they use PFAS. If you want to learn more: www.servethehome.com/2-phase-immersion-cooling-halted-over-multi-billion-dollar-health-hazard-lawsuits/

@DrivingWithJake 9 месяцев назад

@@ServeTheHomeVideo Thanks that was a great read. Phase one I believe could still be a great option. It would be quite interesting to see how much of a cost savings this would be even on a smaller data center scale with around 5000-10000 servers. We are noticing the issues with data centers these days being limited on their support. And the amount of power we would save on average just being able to remove fans would save a large amount. :)

@DOM-cc 9 месяцев назад

5:50 im sure this pi cm4 feels right at home as CDU controller xD

@runforestrunfpv4354 9 месяцев назад

When the water loops fails in cooling the colo, then your cold aisle will blow the same temperature as the hottest component on the production colo very quickly.

@chipholland9 9 месяцев назад

How much quieter are these racks? One of the problems with big air-cooled DCs is the noise hazard to employees.

@ServeTheHomeVideo 9 месяцев назад

If we were not in a lab that had air cooled servers next to these, very quiet. Most of the heat goes out through the liquid cooling which makes it eerie to walk through a data center and not hear screaming servers.

@jonathanbuzzard1376 9 месяцев назад

It's not a problem you just wear the appropriate PPE aka ear defenders when you go in. If you are doing any work with a server in the rack you need steel toe-capped shoes anyway

@movax20h 9 месяцев назад

Hi Patrick. Great video. Could you get some hand on some high end but lesser known switch manufacturers. I recently learned of Starry Network, very fordable L3 switches, great for internal networks. They are using switching ASICs from Centec, and from the info and some third hard comments, I found that they perform as advertised, with line rate for all the protocols. It looks like a really good way into 100G networking on a budget. (their highest end switch is like 5-6k$ only, and smaller ones are even cheaper, 2-4k$).

@ServeTheHomeVideo 9 месяцев назад

Where would you buy them? I can have the team look into it.

@movax20h 9 месяцев назад

They can be ordered directly from Starry Networks or from Hohunet reseller. They are very quick responding to emails too.

@zbigniewmalec4816 9 месяцев назад

Hi Patrick. Is there a chance for the article about how to prepare DC space for liquid cooling. Colo space tends to be dry and facility water is not something that exists close by.

@ServeTheHomeVideo 9 месяцев назад

We were thinking of doing something like that early next year.

@zbigniewmalec4816 9 месяцев назад

@@ServeTheHomeVideo thank you very much

@robertballerstedt5688 7 месяцев назад

There are some companies developing closed loop, single server sized cooling solutions. Look at Jetcool.

@magnawavezone 9 месяцев назад

The sad reality of liquid cooling in the datacenter, is that it’s been kicked around for over 20 years, and yet every solution is proprietary and gains about 0 market penetration. Wash rinse repeat with the latest solution. DCs are crazy expensive and infra has to last a long time so customers are rightfully leery. Air cooling is universal and compatible with everything, albeit inefficient. Triply so, hyperscalers are very cost sensitive and this all looks very much the opposite of that. AI may change that but there is some serious uphill battle here.

@bioboi4438 9 месяцев назад

I love this!

@ServeTheHomeVideo 9 месяцев назад

Thanks

@aurimasniekis 9 месяцев назад

I think I have seen these connectors (fittings) but iirc those are like almost 50$/ea plug one side only...

@-MOS- 9 месяцев назад

I cant wait for these to land on ebay for pennies

@kidsythe 9 месяцев назад

looks over at the ole 3960x and 2x 3090s "you look like 100k to me"

@darksam1212 9 месяцев назад

Raspberry Pi spotted 👀🥧

@ServeTheHomeVideo 9 месяцев назад

Got it! Actually, this is not the first RPi CDU we have seen on the STH RU-vid channel

@darksam1212 9 месяцев назад

@@ServeTheHomeVideo have you heard any reasoning why Pis specifically are so popular in infrastructure? There are many industrial SoMs to choose from, and once you are investing in a certain level of software complexity, there isn't a selling point that sticks out to me. EDIT: not just in the data center, open a EV charger and there's a good chance a Pi is hanging out.

@superdave4564 9 месяцев назад

@@darksam1212 my guess is community support. At this point that’s why everyone uses it. Incredible support network

@clausdk6299 9 месяцев назад

3:33 ...thought you said pee wee

@clausdk6299 9 месяцев назад

oh wait you did...

@tmoney1876 9 месяцев назад

If 1u/2u surplus data center servers didn't use 40mm fans, they would be much more acceptable to homelab users... Maybe it'll work its way down to ebay in a few years.

@dota2tournamentss 9 месяцев назад

It's a bit more expensive but you can totally build server within normal tower case with big fans without any screaming 40mm fans

@lvutodeath 9 месяцев назад

Or you could do the complete opposite. Install delta fans everywhere 😁

@tmoney1876 9 месяцев назад

@@dota2tournamentss I'm aware... That's why I avoid 1u and 2u systems... that's the point.

@tmoney1876 9 месяцев назад

@@lvutodeath I live close enough to an airport that I can get that experience whenever I want.

@MatthewSmithx 9 месяцев назад

Supermicro uses 80mm fans on their hgx H100 servers and they are still super loud- that and they use about a month’s worth of household power per hour and cost about half as much as a house (or two pretty nice cars)

@Steve25g 9 месяцев назад

@ServeTheHomeVideo what about immersion ?

@jonlaban4272 9 месяцев назад

Is this proprietary liquid heat transfer hardware system compatible with the open source hardware rack - Open Compute Project Open Rack v3 - which went into mass production in 2023?

@ServeTheHomeVideo 9 месяцев назад

Which ORV3 version? Meta? Google? There are different ORV3 implementations. What we looked at fits in industry standard 19". There is not much stopping a system like this from being used in OCP racks that are larger but that would be the ORV3 version of this, not a 19".

@jonlaban4272 8 месяцев назад

@@ServeTheHomeVideo Steve The OCP ORV3 isn't physically wider or deeper than a standard 19 inch rack. It's just that the old relay rack 19inch pitch and the EIA U height are different inside an OCP rack/box/cabinet.

@DerrangedGadgeteer 9 месяцев назад

And my bootleg AI server has an appropriately bootleg water cooling system... Fun fact: the hardest part is getting the quick disconnects. Almost nobody makes high-flow quick-disconnects that close off on both sides. The ones that do are high-end, and are eyewateringly expensive .

@shephusted2714 9 месяцев назад

lots more points of failure introduced but with power savings you can add another server to every rack - parts is parts - supermicro does not have domain authority here per se but good they are addressing it, not a super advanced technology - it is plumbing

@benjamintrathen6119 9 месяцев назад

Very cool, kid in a candy store vibes, jealous AF.

@ve2jgs 9 месяцев назад

Put the computers in a cold climate and use the resulting heat for living spaces or other productive use..

@Ownermode 9 месяцев назад

Those pump controllers use a Raspberry Pi module

@backacheache 9 месяцев назад

How do they stop condensation?

@accesser 8 месяцев назад

Interesting upload, not something we see everyday

@Yandarval 9 месяцев назад

Of course. If we had much mor efficient gear. We would not need extreme cooling. We are at Netburts levels of power suck atm. We need a rethink and rest across the board.

@concinnus 9 месяцев назад

Dennard scaling broke down ~2006, it's not coming back.

7 месяцев назад

Hey I just tried to read some of your articles over RSS. Is there a chance you could the articles directly into the RSS feed?

@cappie2000 9 месяцев назад

Houses should have hot and cold loops too.

@ahabeger 9 месяцев назад

I'm more likely to use Lenovo Neptune for my cluster.

@ArneiroS Месяц назад

5:50 Is that a CM4?

@alexlandherr 9 месяцев назад

At 5:52, CM4 in the CDU?

@ServeTheHomeVideo 9 месяцев назад

Surprisingly common in CDUs across several vendors. We have shown RPi's (not even CMs) in other CDUs before.

@debugin1227 9 месяцев назад

It’s not cooling. It’s skynet’s circulatory system

@benjamintrathen6119 9 месяцев назад

Imagine the poor old delta fan graveyard on an industrial scale, I can see dump trucks loading them into the hull of a cargo ship headed back to China.

@BredihinKA 9 месяцев назад

5:50 - Raspberry pi cm4. Usb флешки и прочие поделки в серверной стойке. Жуть...

@sativagirl1885 9 месяцев назад

Q: How many Raspberry Pi 5 would it take to replace a Sun Server? ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-cucLQF3TqOg.html

@ServeTheHomeVideo 9 месяцев назад

We replaced a Sun Ultra with an EPYC years ago www.servethehome.com/introducing-the-ultra-epyc-amd-powered-sun-ultra-24-workstation/

@EyesOfByes 9 месяцев назад

But can it cool an overclocked i9-13900KS?

@peppybocan 9 месяцев назад

I kinda disagree with you that liquid cooling needs to happen. I think once the industry switches from silicon based electronics to opto-electronics in 10-15 years, it will be mostly the light that will do the computation and not the electrical current, which in turn, means order of magnitudes of less heat produced, which would drop all the unnecessary cooling systems.

@Prophes0r 9 месяцев назад

Liquid cooling hasn't been State of the Art for over a decade. Direct Phase-Change is the current State of the Art. Instead of using heat pumps to cool water, and then pump that water around, you pump the heat away from the source directly. There is another 25-50% power to be saved by not moving all that water around.

@timramich 9 месяцев назад

Direct phase change cooling is state of the art for overclockers, because it gets down to well below freezing, but it's still a power hog, and not necessary if one doesn't need to cool things down that far. It's also not very stable, whereas a huge mass of water being pumped around is a nice buffer. There's even more power to be saved over direct phase change cooling by using water and evaporative cooling towers. Water pumps really aren't that power-hungry. Geothermal would be the ultimate in efficiency. Refrigeration is just a mess. Every few years the current refrigerant gets banned in favor of a new one that's less efficient.

@Prophes0r 9 месяцев назад

@@timramich Heat is heat. There is no need to go sub-zero to get efficiency from the system. The water chilling systems they use TODAY are cooled by heat pumps. Evaporative (water) cooling systems are an environmental nightmare. We don't need another industry deciding that using water as input is a good/cheap idea. We have enough problems getting clean water to people as it is.

@imeakdo7 9 месяцев назад

Phase change needs pfas forever chemicals as of now. Maybe in the future, water shouldn't be evaporated in cooling towers like it's done now, they could use dry coolers and phase change cooling

@Prophes0r 9 месяцев назад

@@imeakdo7 What? There are PLENTY of refrigerants with no Fluorine in them. R-290 (AKA Propane) is used all over the place. And R-744 seems PARTICULARLY ironic to use in this situation. It's CO2. Using it would be a drop in the ocean of what needs to be contained, but taking it from the air and using it in a closed cycle would still be a good thing.

@imeakdo7 9 месяцев назад

@@Prophes0r phase change cooling in data centers which is what this video is about doesn't use refrigerants and uses pfas containing liquids that can't be used to create temperature differences between areas unlike refrigerants, which need to work at pressures very different from atmospheric and that would make quick connectors bulky, expensive and servers hard to service because of the bulk which is something data centers mind a lot. You need quick connectors to service servers without taking them apart a lot. So unfortunately refrigerants are impractical in datacenters and it would be better to take water from the ocean, cheaper than power for taking water from the air but then you'd have salinity issues or even higher costs because of power for desalination, so you need to choose between power which causes climate change or water, both are bad

@tokkenme 9 месяцев назад

Tiny, mini, mega.

@D8V1Dx 9 месяцев назад

Nah liquid cooling is not it. Might as well go immersion cooling at that point,

@Prophes0r 9 месяцев назад

Immersion cooling is liquid cooling. Might as well go Direct Phase Change cooling and cut out the middle-man. That liquid is already being cooled by heat pumps. Why waste all the energy pumping all the liquid around?

@D8V1Dx 9 месяцев назад

Those phase change liquids are way more expensive. Everything also has to be complete sealed off or the liquid evaporates. @@Prophes0r

@timramich 9 месяцев назад

@@Prophes0rThe liquid doesn't need to be cooled by heat pumps. It can just run through massive radiators, or even cooling towers. I don't get where you get this notion that pumping water is horribly inefficient. Doing so and letting the water evaporate is much more efficient than refrigeration, and uses the same principle. If refrigeration was more efficient, electric generating power plants would be using it. But what do you see at power plants? Cooling towers.

@Prophes0r 9 месяцев назад

@@timramich Pumping water isn't horribly inefficient. My point is that all these systems ALREADY use phase-change heat pumps to chill the water. Cutting out the water step is an efficiency improvement. Water is used because retrofitting facilities with water chilling was the cheap and established technology. Evaporative water cooling is a huge no-no. We do not want yet another industry tapping in to our already overstressed clean water supply. It might be technically more energy efficient, but we NEED that water elsewhere.

@timramich 9 месяцев назад

@@Prophes0r Where's the proof that they're using refrigeration to cool this stuff versus just radiating out the heat? The equipment doesn't need to be that cool. And I don't even know what the hell you're on about with the clean water. Ever notice where power plants reside? Along rivers. They use the river water as the evaporative liquid. It doesn't have to be clean water. You just generally don't know what the hell you're talking about.

@majstealth 9 месяцев назад

sorry, but apple builds nothing, foxconn does. by nnow i even think apple outsources most of the designing of pcb´s...

@hashim5900 9 месяцев назад

This

@msys3367 9 месяцев назад

All that power usage just for lazy people and rendering funny pictures.

@jeffm2787 9 месяцев назад

A million jobs lost to AI, but at least they saved 20% on the electric bill to do so.

@zyeborm 9 месяцев назад

I mean it's happening anyway so you might as well

@offspringfan89 9 месяцев назад

Sure, let's also ban tractors and agricultural machines because they replace farmers, right? Might as well replace automated street lamps with people lighting the lamps with a candle, just like old times, right? We can't hinder technological inovation to save a bunch of jobs. Like Evolution in nature, you either adapt or die.

@tappy8741 9 месяцев назад

A job is a means to an end. It's not the reason to exist. It's not a good part of existence. It's not something to aspire to. Wouldn't it be nice if humans could focus on hobbies and creativity and self-fulfillment as the majority of their activity.

@juanlemod 9 месяцев назад

🫃🏽

@protaginets 9 месяцев назад

What all AI servers need more than liquid cooling is a kill switch and someone with the smarts to know when to use it.

@hiko8822 9 месяцев назад

I know AI won't evolve since they're just glorified selection trees, but I'm just saying It always happens because someone who isn't smart tried messing with it, and no one put in a power cut off.

@ewenchan1239 9 месяцев назад

I was designing my own waterblocks for the Socket940 AMD Opteron back in 2006, when I was in university, doing my bachelor's in mechanical engineering. This isn't new. This story has been, at least, 17 years in the making.