deju vu. this feels like the FUD with crypto mining since 2017-18 or the Intel Pentium 4 FUD with skyrocketing heat from CPUs. I wouldn't be surprised in a few years that writers will be thanking the AI boom for better energy solutions due to steadily growing number of EVs.
That is very true, but it's easy to not recognize how much information ChatGPT knows. For example... my brain is energy efficient, but I just used ChatGPt to create a python script with 100s of lines. I didn't have to learn the syntax (this is my first python script). I didn't have to look up and learn about different modules that it uses. To be honest, it probably saved me 20 to 40 hours and was completed and tested in about 1 hour. I'm not convinced it's less efficient.
@@TwoBitDaVinci So half joking...is the next step in computing an organic computer or some kind of human brain farm (like in the Matrix but for harvesting computational power instead of heat energy)? Lol, it would at least make for an interesting plot for a scifi movie.
It seems like my comment got deleted... I just used a LLM to help me to program a python script. It's up to 3500 lines of code, but more than half was copy and paste from the LLM. Before this, I have never used Python and i didn't have to get a book and learn the syntax, etc. I estimate that it's saved me 80% of the time it would take for me to write it from scratch. Yes it takes power and yes I can write a 10 page essay, but a LLM can write it better and in 1 min. Brains take less power, but they I'm not convinced they are more energy efficient (energy consumed per output unit)
@mccue2439 When talking about energy efficiency, speed is not what we're looking at. What is amazing about the brain is that it could write that script at the energy cost of a cup of coffee.
District heating has higher infrastructure costs up front, but once the infrastructure is built there are lots of inputs. Even waste heat from nuclear plants can go into district heat grids.
@@TwoBitDaVinciplease do and then tell us why this isn’t happening everywhere. Thanks for your awesome videos and for being such a generally cool guy. Not at all surprised why Mrs. Llewellyn liked you so much the instant she saw one of your videos.
Hmmm--I wonder has people thought about putting up server farms in the "safe" areas of Iceland, server farms all powered by plentiful geothermal power.
Happy to hear that analog computation is finding new life. Sixty years ago I had the pleasure of programming the Electronic Associated PACE Model 231R, the last of the large analog computers. It regularly calculated results faster than our department's digital computers. Then we installed a Philco 2000 with 10k core memory! Analog still won the timed races, but lost out on precision with "only" four-digit resolution.
@@TwoBitDaVinci Sure. Residing San Diego now, but relocating to Phoenix in August. BTW, imagine the power requirements if AI only had the vacuum tubes we relied on!
Yes! Make a video on heat redistribution (at least at the data-center level) and how we are not taking full use of this topic. Recycled electricity at the time of use!!!
Well... In Alberta Canada we have flared excess natural gas beyond what the government has set as a limit. For zero usage, not even as heat. It is of course to maximize sales of oil to the USA.
It's just crazy. We've been doing the same here in Venezuela for decades, even when there's no supply of natural gas for households, you would see the flares (called mechurrios here) burning high on the top of tall towers. So sad to see.
I live in the area where the Cedar Rapids data center is to be built. The local nuclear power plant has reached its end of life and is now being replaced by 3,500 acres of solar panels on prime farmland. This video just put the pieces together for me as to why. I can see a lot more expansion of these panels to more prime farmland. Looks like this would have been better served in a location where less productive land is abundant.
40% of the corn grown in the country is for Ethanol. PV solar produces 100 times more energy per acre than corn ethanol. In addition, many are also using the land with PV solar for specialized agriculture, like sheep grazing and part shade crops.
Sorry but commercial nuclear power is not the answer. In part due to its ultra-expensive cost and ultra-long build times, commercial nuclear power will be gone from the grid in about 50 years. Not just because of that but renewables are at historical low cost per MWh generation (and still falling) while nuclear keeps getting more expensive. Furthermore, "firm renewables" at scale are just around the corner with massive government, scientific and industrial interest in grid-scale storage technologies. When firm renewables arrive at scale, all other energy production methods like fossil fuel and nuclear are done on the grid. Nuclear will still be used in small-scale scenarios like military, scientific and deep-space exploration. Commercial nuclear is just the wrong tool for the job on the grid. Commercial nuclear power's own history bears that out with it powering only 10% of the world's grids after a long 70-years of history and its percentage is falling. Nuclear failed us from its promises of abundant energy "too cheap to meter" promised in the 1950s. If nuclear had come close its promises, we wouldn't be so far down the climate crisis hole we're currently in. Nuclear failed us. At 99.9% the mass of the solar system, the sun provides a cosmic 173,000 terawatts to the Earth non-stop. Or about 1000W per square meter peak at the ground and is always peak somewhere on Earth. All the combined fossil and fissile material on Earth would amount to a bucket or two of water in an ocean compared to the daily amounts of energy provided to us from the free clean *fusion* power we get from the sky. Renewables harness this energy directly, and the raw inputs of sunlight and wind cannot be taxed, embargoed, blockaded or sanctioned. With respect to land, much less than 1% of the world's total land surface alone in current generation photovoltaics is enough to power all the world's grids. But why stop there? Imagine a full 1% or 2%? Solar can also produce power 'in situ' on-premises which greatly reduces load on the grid because rooftop solar "unloads" the grid. It helps that datacenters have massive rooftop and parking lot areas. Also, a lot of the biggest datacenters generally have a lot of open land around them as they're built outside of urban areas. A solar farm is actually more environmentally friendly than an actual agricultural farm. Current agricultural methods are ruinous to the environment with animal run-off, pesticide run-off, and soil erosion. So, it's not surprising that with a solar farm on it, the land would live a much more peaceful life than it ever did as an agricultural farm. New agricultural methods are needed. One example is using "agrivoltaics", one can reap two crops off the land: food and energy which humans need a constant supply of.
@@MichaelF350 Soybeans are a thing. Also, I am watching these solar panels go up just 3 miles from my home and they are not planning on raising animals under them-they are too low for that.
I would like to see a deep dive into the techniques for implementing neural networks without matrix multiplication. I did some published research on NNs in the early 1990's but haven't really been involved with them since.
I manage a SCADA system for one of the greenest utilities in the country. Data centers have moved in, and now their baseload is 4x their previous all-time peak. The data centers are eating up all their green energy, and the remaining is coming from firing up old coal plants. AI and Cryptomining are totally negating and alternative energy benefits. Ironic Tesla started all of this. Kind of contrary to their mission statement. Utilities I work with are planning on a 3x increase in electrical demand for upcoming data farms.
While i agree to some point with you, you would want what, that the corporate media feed them their "truth"'? Only way to get the truth right now IS the internet, if you know how to search and have the mental ability and will to analyse and interpret data. AI is no different that any other tool that you need to learn how to use it, and no diferent then any other info source in that you need to analyse and use critical thinking to validate if it is indeed garbage or not. These day you cannot afford to trust anything blindly.
What?? How exactly do GPUs degrade in computation efficiency over time??? They’ll flat out break way before you’re going to measure any type of efficiency decline
@@antonvoltchok7794 Maybe I didn't say that right, but GPUs get more efficient each generation, so they are more efficient power wise, time wise, and usually space wise also for each successive generation. So if don't use them 24/7, you loose that opportunity cost.
you bet, and thanks for being such an active voice in our videos! I see you and appreciate you! don't even know you're real first name, but thank you budgetAudiophile :) great name btw
interesting, so you're saying get the cooling benefits of vaporizing water, then find a way to collect it free of the contaminants? issue is salt would gunk up the meshes... but its an interesting thought!
They should move to Australia. They have some serious issues on how to use their excessive solar energy coming from residential houses. The electric company even stated that the residents have to pay if they send their extra energy back to the grid.
not only are AI data centers chugging energy, but the demand is only increasing. One thing people miss is these companies are getting huge discounts on energy and tax breaks.
simple solution: Make the big data companies pay for the construction of nuclear power plants, to even out the energy it progressively takes out the regular grid from people. Clearly they got the money for it
I've had that thought for a while. Residential power use can be met with grid-tied or off-grid renewables/storage, while industry can run off nuclear/hydro/geothermal if it needs to be so consistently supplied.
Exactly. Tell them to build solar plants and use them. If they don't like it, fine them. We don't even have real AI. It's just google 2.0. it's all marketing and we see through it.
@@stephenbell9324 Hence why industry can use nuclear. Metal refining and AI training are 24/7/365 processes, and any changes to the load would be slow.
nuclear power plants take decades to build, due to high safety and regulation. so there's o practical way we could do this, especially if other countries dont
Yes @TwoBitDavinci this is frigging important. As a comsumer you should be (easily) able to see where your app-energy comes from, and how much it draws. End-to-end... Super complex tho.
Popular models of neural networks that are used in AI models have such problem as catastrophic forgetting. A neural network trained in new tasks dramatically loses the ability to perform previously developed tasks. And what we’re seeing now is NVidia crystallize this drawback. It's a waste of our energy for a while.
DEEP DIVE? HECK YES--on everything you asked if we (i) want to see. Love your explanations and the thoroughness with which you cover each subject you take on. As the Nike ads opine--DO IT!!
A few years ago I though about if we could use the excess heat of these server farms and convert it into motion energy using a sterling motor. Then you could use the motion energy of the motor and power an electric motor, generating electric energy. We could regain some of the energy we used on these server farms and since the heat supply is steady, it would even be base line power with no fluctuations (idk the english term, in Germany it is grundlastfähig)
What people do not understand about AI is that a gaming console becomes an AI when you place a game CD in it. Game programs have become so complex as to, at many levels, outperform most larger server farms in terms of running complex programs. Because of this, no new gaming platforms can be sold outside the U.S. because our government is afraid that terrorists will use the technology within the console to stage attacks. With that in mind, imagine 3 things. First, one building per city block is fitted with enough servers as to be hooked into all other buildings within the same city block as to monitor electrical consumption; building security features; etc.; Second, each apartment, house, business, etc. is also hooked into the system as well; Lastly, because of this, an AI can monitor everything within your residence 24/7 as to reduce the amount of energy consumed during the day when no one is home or otherwise present. Imagine an AI cutting the power to your refrigerator after you left for work because the AI knows that the refrigerator will remain cool enough to keep everything cold because no one will be in your home opening and closing your refrigerator during the day. It can then monitor the temperature to keep it cool, or turn it back on when it knows that you are on your way home. It will know that your on your way home because it has seen you leaving your work from a nearby security camera as well as a traffic camera along the road to your home. Imagine if it can monitor your activities as to make suggestions which will ultimately save you money. There are many cities which uses delivery services for a various number of things. Imagine you coming home to find a box of food just delivered to your front door from Walmart via FedEx. Upon close inspection you ended up saving because the AI used various methods/sales/online coupons/etc. to save you money on the food you eat on a regular basis. Even if you wanted to try new things, it will be able to read various actions you do as to guess what food(s) you are in the mood for eating. Imagine that it figures out that you are in the mood for steak, and reminds you before you leave for work that you have a reservation at a steak house for that evening? Can you afford not to live that way in the current economy?
@@AZOffRoadster He has an unhealthy obsession with Elon Musk, and not in a good way. Apparently Elon is the most evil crooked dumbest snake oil salesman in living history LOL
To me this just indicates a greater concentration in specialization and humans relying upon this specialization for their existence. There is a huge danger for any life form to rely upon more and more specialization as usually it ends up in their extinction.
@@anthonycarbone3826 Well the nice things about humans is we aren't really that specialized like an animal. It isn't like we are born to do a few things, and rely on a single diet. Or a single climate just to breed. If you think about it. Humans are super generalized, and adaptable. General intelligence allows us to change with our conditions or change those conditions entirely. Specialization won't be our downfall. Our downfall will be our greed.
Incredible graphics. Great title, though a lot of competition on that data point. Can you guys a/b test with titles? I noticed this one changed. Did graduate work in 1992 on 'designing' analog neural networks. SWRI did this before me. (In case anyone is looking for IP precedence ;-)
Future video topic suggestion. The discussion can be split between training and inference (and deployment). Training is more batch-able, and can utilize renewable energy, esp. when there's plenty. (There are also cap-ex and capacity factors to consider.) Inference is more real-time, and requires 24/7 power. But it's also a space ripe for innovation, such as quantization, compressed models, new hardware such as analog, or elimination of scheduling and frequent memory transfers in favor of optimized deterministic pipeline via compiler (e.g. Groq). There's also edge devices, consumer AI hardware, such as what Apple is doing, in conjunction with private external servers, and services.
I imagine that if quantum computers truly become a thing, they'll make training even larger models much more accessible, easier, faster, and more energy efficient. This begs the question, how many parameters are enough? Will we simply continue scaling AI models indefinitely? I don't think they will and this is why I think that: When we scale the models and the number of parameters, we need to scale the training data accordingly. With all the issues around AI and copyright, intellectual property, and whatnot, there won't be enough real-world, human-generated data to train the models on. What will happen then is that the models will be trained on synthetic data output by previous models. For me, this points to a future where models will probably start degrading at first and then eventually plateauing, or developers will simply put a cap on how many parameters they add, or the "real" data they scrape from the web will all be AI-generated, in which case it'll be the exact same scenario as using mostly synthetic data. I wonder how AI will evolve after that? The only thing that I can imagine is that they improve the training algorithms to get a better output on the same training data, and one way they'll probably do that is by asking an AI to improve its own training protocol. After that, it's either it's either Skynet, the Architect, or the Oracle. 🤪
Firstly, this is the actual problem with this tech, and a far more relevant topic that what the artists are trying to push. Thanks for making this video. Also, anyone else had the name Ricky ruined by Weird Al? Every time he says his name I think" Hey Ricky..." and have to stop and refocus on the video.
At 1:15 Brad Smith of Microsoft compares the energy consumption problem of AI to a moonshot where the distance from the earth to the moon increasing by a factor of 5 in a very short period of time: “So in many ways the moon is 5 times away than it was in 2020.” I don’t know about you but that analogy suggests that you have not just a very difficult challenge but now a nearly impossible one, certainly in the short term where all of these companies are racing to control AI. For perspective, that meant a moonshot to a moon went from a distance of 238,900 miles to 1,194,500 miles. There isn’t going to be a manned moonshot at that distance, all other aspects of astrophysics and current engineering capabilities held constant. Incidentally, we are not just talking about electric power grids being insufficient; green energy is absolutely unsuitable to scale for realistic energy demand of AI, because of cost, reliability and even material sourcing/supply chains. Perhaps nuclear fission reactors will work, or the adaptation of neuromorphic chip architectures will mitigate the rapacious energy demand of current GPUs. Bottom line, MS, along with everyone else, is way too optimistic about their growth in this area without fully appreciating that physical infrastructure has to be in place for their plans to have any chance to work.
Plot twist: AI already has access to the internet and is using everyone's devices for computing. Granted it still takes a lot of energy, but now it doesn't need to build the infrastructure. ChaosGPT 2024
I am a career data center guy. Every time I see big data centers claim to be ‘green’ by purchasing all the power from a hydro power source I call “BullSH**!” Locking in that power does exactly what you suggest, make utilities find other sources for their other customers!! I also agree that Data Center heat needs to be USED not discarded. But here in the US it’s cheaper to burn new gas than to recycle DC heat, so very few do. It is low grade but it can easily heat space and domestic level hot water. We just need to Do It.
"I am a career data center guy. Every time I see big data centers claim to be ‘green’ by purchasing all the power from a hydro power source I call “BullSH**!” Locking in that power does exactly what you suggest, make utilities find other sources for their other customers!! " If they purchase "green" power they can make that claim. And the more demand for "green" electricity the more it will be built out. This is not a big issue.
What about Jevons principle, in which the more efficient something is the more the demand for it goes up? James Watt’s more efficient steam engine ushered in the industrial revolution.
The comment at 9:15 isn't correct. corporate companies adding their own data to AI's use RAG and not usually fine tuning (eg training the base model) because it's too expensive. I'm not saying that the point you are making isn't correct over all but there are now ways to give AI's access to data you want them to be aware of without actually training them further. MS sells access to openAI APIs on locally run models but you don't usually change the base model. If you read up on RAG, Embeddings and Vector DBs you will see what I mean
Glad you did this video. Its long past time to shine the light on this massive energy consumer, not just on efforts to force those of us that like ice cars to regulatorily mandate evs. Ill make a deal, let me keep my gas car, and i will agree to stay as far away from ai as possible.
Low grade heat from a compute center needs to be concentrated (heat pumps) and sold to offset industrial heat or large scale building heating needs. How about sand batteries storing TWh of energy for winter use.
There is an AI jevon's paradox, where limitations of energy supply will drive efficiency gains out of necessity, which will boost the pace of AI and therefore increase demand. There's many orders of magnitude gains to be had from algrorithimic effciency alone, not considering hardware design, other optimisations. We've already seen AI specific hardware dropping from FP32 FP8 precisions for wicked speed ups. Lots of low hanging fruit in these areas to be picked.
9:22 No, that is LoRA (Low rank adapter), it's training is a tiny fraction of a training a foundational model from scratch. In fact that can be done on the desktop PC.
"There is so much we could do if we got a little more clever." Making electricity from waste heat requires additional processes, equipment, materials, costs, space, maintenance, etc. At some point, chasing incremental efficiency gains isn't worth the additional expenses. Getting clever doesn't do much good when it isn't economically viable or runs into sustainability implications, like Canada coming to the conclusion that most of its hydro potential is tapped out and will need something else, likely new natural gas power plants, to guarantee baseline load through the electrification demand curve. Can't use nuclear for this due to its 15-25 years lead time from proposal to production when you include the endless environmental studies and NIMBY lawsuits.
I run stable diffusion xl purely off my home desktop pc with an rtx 4070 and producing 2000x2000 pixel outputs pretty much uses 100% resources. However 1 inage like that take about 60 seconds. An intereting sidebote i tries soing the same with gpt 8 gig models and bizarrely text models are harder on my system and way more unpredicatable on how much a prompt will stress the system.
Your 100 percent right with AI for searching. I am soooo much more efficient in time just asking ai about coding questions. Also recipes, you just get a tiny write up and thats it no more story and ads their either. I'm not going back to search engines if I can help it
I asked that microshite copilot why we are using all this processing power on AI instead of, say, cracking the nuclear launch codes. It said that was an innapropriate question, I promptly deactivated it off my computer...
As long as the energy is properly priced according to supply and demand, it won't be an issue. Let these AI companies have the energy, but make sure they pay the price for it.
Commercial power rates are typically a fraction of the cost of residential. Same power sold at different rates... surprising? Nah, our system is designed this way. 😅😮.
"As long as the energy is properly priced according to supply and demand, it won't be an issue. Let these AI companies have the energy, but make sure they pay the price for it." Yeah. Market forces will fix everything in the long term.
I always wondered why they don't try to trap all the heat from all those server farms and try to re-use it towards something beneficial to at least in the cities where they are located
Place the servers that generate the models near solar and windfarms and run these servers when there is no demand from the grid. So run on free surplus power.
2 месяца назад
If there is a need there will be a supplier. New tech like first stage Fussion, old tech like solar or wind and water or ancient like nuclar maybe in a new form,or oil and gas.
As many countries no longer have a sustainable replacement birth rate, I wonder what impact a 20-40% reduction in population will have on the overall energy consumption? Korea was at 0.72 birth rate in 2023.
