Please do a vid on the company out of dubai who is creating medical AGI and what kinda of technology they are using and what they plan to do. Medical AGI is very broad. Would love to see what that means.
Linus tech tips is all about it, with excellent up-to-date articles on top of the edge technology, and many other excellent channels, BTW. It's just a tip. Perhaps you are not aware of. Good luck. 🎉❤
Your content is always very special and informative. You tend to choose topics that are not commonly found on other channels. The most important thing is the way you explain complex concepts so easily; that's truly awesome.
The sheer compute power of this chip are promising a new era in AI technology. I’m eager to see how this will be utilized in various applications. Kudos to the team behind this innovation!
yeah dont go counting the benefits just yet.. that is a chip that draws in a 100 fucking megawat hour. that thing cant run for more then moments withouth having parts of it being vaporized into heat. with computing almost ALL of the enrgy goes into heat so that is a bloody 99 megawat heater the size of a chesboard you got there.... you litteraly need a powerplant to run this crazy thing.
It seems like an apples to oranges comparison. Put it against GH200 Superpod with 256 Grace Hopper Superchips. That is Nvidias latest offering. It's not only fast, but energy efficient.
@@635574 VHS vs betamax, Bluray vs HD-DVD... the latter was better in both cases and still lost because they couldn't get adoption. Nvidia gave away a lot of very expensive silicon for nothing in some cases or a small pittance to get CUDA in the hands of research teams at universities, who standardized on it, and eventually started teaching it. I love the idea of a competitor but they won't have an easy road if they're not willing to give away a lot of compute, and unlike Nvidia they don't have the gamers and crypto addicts buying every graphics GPU they could get their hands on for double MSRP to bankroll it.
Nvidia fanboy spotted. Power efficiency is a nonissue when you’re talking about the most powerful compute power cuz most people won’t have access to this power until way later.
You don't get the point: as the complexity of the problems you gave to feed on the AI grows these two systems stop scaling together. 90% of the raw costs is on people working on the project, so having a system that does NOT requires more work at all when your workload increases by orders of magnitudes, is a not brainer. You can start training your AI system months before you will on any Nvidia systems. The only thing Nvidia has on its side right now is the shear mass of chips produced each month, so I gues you can build a GH200 ai much faster then you can on Cerebras: not cheaper, but faster, despite being way behind in practicality and raw results.
Another awesome video Ana! Doing direct interviews is a great addition to your repertoire. I have an interest in AI as a social science person. A lot of videos either go way beyond my ability to comprehend, or are filled with superfluous information just to fill time. You consistently put out interesting and coherent information, that I also trust is valid, because of your background.
wonderful video, did some research about cerebras innovative and found out they really have done different and valuable things. "wafer scale engine" is what cerebras been known for, unlike traditional GPU, it is produced on an entire wafer. Conventionally, multiple cpu or gpu are 'printed' by EV on a single wafer, and later processes will cut them off the wafer. Therefore, one reason cerebras is delivering much better performance is because its 'GPU' is bigger. But this also leads to one problem: its even harder to produce than NVDIA GPUs, wafer often comes with defects, individual defected chips from conventional manufacture technique can be discarded. However, cerabras wafer scale engine needs the whole wafer to have no defects. In addtion, heat dissipation, even powering across whole surface are big challenges. Right now, cerebras is cheaper because it's not yet that popular, once market sees advantages from their super computer, their price can go higher than h100 since they are really difficult to make under current tech level.
I am very interested in Cerebras and Tenstorrent, where they seem to be the most viable alternative to Nvidia, both being companies that makes AI chip that is very scalable. The interesting differentiation between Cerebras and Tenstorrent is that Cerebras started with big chips working their way down (in a sense with enabling PyTorch compatibility) while Tenstorrent works from small chips and evolutionary works their way up. It's interesting to see these different contrasting startup philosophies work in the same industry having basically the same main competitors. Hope to see you cover these two companies in future videos.
Actually the most viable alternative to Nvidia right now is AMD's MI series of processors. The MI300 series is due to be widely available in 2024, and it will probably beat the H100 in terms of performance and flexibility. The research I've done indicates that Cerebras and Tenstorrent are very distant alternatives at this point in time relative to both Nvidia and AMD. There's also Intel with their Gaudi series, where it fits in comparatively is probably along with Cerebras and Tens, the most worrisome aspect being the longevity of the roadmap, Intel has been cutting product lines over the last for years. As we know anything can change quickly since the AI sector is in very early stages, so it's worth looking at all the players including the current batch of underdogs.
@@geekinasuit8333 We agree, if cerebras can lower her prices by 10 can be in the competition if not AMD will be the best alternative. 1 cerebras power computation = 50 nvidia power computation, but for the price of 1 cerebras (2.000.000 $) = you can buy 10 nvidia DGX (200.000 $ , 8 x a100 (10.000 $) ), in price nvidia win. And take in consideration that nvidia is expensive, very expensive. cerebras need to lower her price 4x to be competitive, 10x if want to be competitor.
I remember a time where my uncle as an engineer got a PC with 40MB storage and I wondered how he would ever fill that much space. Today I need that space for one single digital raw photo 😅 It’s amazing how fast and capable hardware and software (some not so much 😂) has become
1998, my first computer (well, the "family computer", because they were very expensive, for what they could do): Pentium II at 300mHz, 32megabytes of RAM and I think something like 500megabytes of hard drive, but I'm not sure about this. Of course with floppy disk and cd drives, in that distinctive "greyed white" of the time. I was 13 back then and it feels like another era in the history of humanity 😅😂
what a time to be alive XD ; luv to see the competition heat up between these top tier tech firms and the smaller startups that are rocking the boat =]
Thank you for doing these videos and helping the rest of us to see what's going on in the world of AI and computing in general. I appreciate your efforts 😊
We need more channels that focus on the compute chips & infrastructure of AI. All the buzz is around the software but its the hardware that makes it work.
Smaller isn't always better. I theorized such a computer with the whole wafer, the whole complier part was out of my skills, parallel data bus would be the only way. They have achieved the removal of the 2 compiler stages to get to machine language, the single stage compiler with whole wafer design has Nvidia beat, for much cheaper for title of most powerful AI. Dude knows what he has, I will seek to buy one of his systems, for a upcoming product. Thank you great video!
Never heard of this channel before, until it just popped up on my homepage. And I'm glad it did: great, clear information, with appropriate graphics (when needed), very in depth, but still understandable. One minor piece of constructive feedback: maybe tweak your audio settings a bit to decrease the harsh 's' sounds. I'm using heaphones, and your 's'-es are a bit uncomfortable. Otherwise: great video!
The bane of wafer scale computing has always been that some percentage of the wafer will have defects and be unusable. Does Cerebras has some way around that problem? There was a famous attempt at this back in the 80's and the company couldn't solve the problem and went bankrupt (Trilogy Systems).
The final wafer processor always is quoted after taking into account the dysfunctional parts of that wafer. Meaning its always assumed to lose some parts to imperfections.
it's great to see so many people and companies working on AI hardware, but without a full software stack, it won't be a credible competitor to NVidia. As ML technology advances, they'll have to make sure their compiler handles the workload scheduling efficiently. That's not an easy task.
What if they made their hardware compatible with the Nvidia software? I think in this video it was mentioned that existing tensorflow code for cuda can also work on their hardware.
Thank you for your deeper introduction on Cereras!!! I won’t know this despite I stayed around Fremont and Santa Clara last month if I didn’t get into it much deeper…😄
very interesting, I wonder how saleable they are for production, honestly seems like companies will be fighting for these limited quantity high speed chips, surprised ive never heard of them! Great vid
Cerebras: AI supercomputer networked across three of the same type Cerberus: Three headed hound that guards the gate to Hades ... just in case you may have been confused. Don't be.
Fantastic video I just subscribed. Mr. Feldman was speaking my mind when addressing the tokenization of the arabic language. I don't speak arabic sadly but have been trying to find good models to handle it and found that only gpt4 and bloom were decent. I think his company is on to something forging connections to the gulf. Great video thank you!
My issue with Cerebas was the moment they said thier wafers had zero defects... this irritated me more than anything since by design, logic and physics that can't be true. Also they have repeatedly refused to release any further information... which seems like a red flag to me personally. This in all gives me memories of the early Rambus.
I'm not sure where you heard that, but that's not correct. By design, their Wafer Scale chips are able to isolate and bypass any defective areas. A defective area in other company's chips invalidates the entire chip.
@anonymousjoe3576 that's still a defect, you've lost some area and the same technique is used everywhere else... its pretty simple to slice off areas of SRAM, interconnect, cores etc... those with many defects simply fall into lower product tiers. They were specific in the interview which could of been by Anandtech... hence why I remembered the quote... "100% yield", I lost interest at that point. Just because you haven't cut up the wafer doesn't mean your yield is 100%, they are bending the metric and not accounting for the dead area... (which they don't disclose as a % obviously).
A scholarly article that I read stated that when the Wafer Scale Engine did have a defective area, it was typically less than 1-2%. Give me a day or two and I will try to find the article.
I think what is meant by that is that every wafer will offer the same compute capabilities / processing power. Internally a wafer may have defects but the user will not suffer from them, meaning that the wafer is still "usable", hence the yield. This is not necessarily doable for smaller chips.
Data quality matters vastly more than parameter count though. Improving LLMs and Stable Diffusion right now is all about figuring out how to get better data.
There's not a lot of information about Cerebras, so thanks for making this video. I'd like to know how flexible a machine like this is with experimenting with different models? Will you be limited to only a few kinds of models, if so then what exactly are those limitations? One known issue that Cerebras acknowledges as an intentional trade off, is that a machine like this is limited with floating point accuracy and will not be suitable for models that require higher 64bit precision. It appears the machine is optimized for 16bit precision only. I expect there will be other limitations besides the FB accuracy and a summary of what those limitations and what the tradeoffs are (pros and cons) will be nice to know about.
Love that everything you post you have a great explanation and always back up your information with real facts as a document video! Thank you 😊 your awesome! Your brilliance in awesome
This is similar to the S1 supercomputer of the 1980s in concept. It was a large wafer but with gates to be connected and could have a new design per week. They used the whole wafer and etched the back side so as to conduct coolan in channels too small to have Bernoulli turbulance.
With a core that size aren't the yields extremely low or is it even possible as there is always an error on the whole wafer? Or do the cores have some sort of fault tolerance built in like deactivating the affected sections?
What an interesting channel and a fabulous, clear presentation on this groundbreaking million dollar AI hardware that will facilitate probably the unimaginable in the near future. Anastasi and her co-host deliver a vivid picture what the company of Nvidia created. It is very exciting ... seeing the future unfold in this enormous leap foward. It tickles me to think that gamers... with their need for the fastest speed ten or fifteen years ago who were willing to pay top dollar to get what they wanted would create a niche market to spawn the likes of this billion plus computer chip made by Ceberas is only the size of an average floor tile, but more powerful than anyting known. This feeling of excitement seems like to me what it must have been to see the Wright brothers flying across the New York City skies for the first time. The significance of this chip is as unknown yet greatly anticipated to become probably the biggest scientific tsunami that will change our civilized world as we know it. Amazing development to learn about and thank you for your excellent presentation.
What is cost per TFLOP? Power per TFLOP? Is it 64 wafers each 50x the power of A100 all taking 1.75 MW? If so, they'll be taking 10 % more power than the 500W NVIDIA A100 (64x50 A100s).
I guessed the company accurately before listening to video. Tech tech potato youtube channel had some good content on this waffersized chip Your video was very well presented
I would imagine that it's more difficult to make it fault tolerant for waferscale than they let on, and it's also harder to program software that computes over such a wide area.
There is a time traveler that said he spoke to a computer that was in charge of the entire Earth. Can’t remember what year he said, but it’s obviously on the horizon.
One thing I've wondered about is why aren't motherboards, or whatever you want to call what the chips are sitting on and the current travels on, printed. Right now, all the circuitry sits on top, but if it was printed, you could print it in 3d so that the circuitry could sit stacked on top of each other. Further more, all the circuitry would be protected from dust, something that slows it down.
Fascinating! I wonder how they bridge all those wafers? Also wonder how they transport heat away from them. There is megawatts of heat produced in a much smaller volume than with GPU cabinets.
the reality is that in anything related to computer performance, going a monolithic approach is always superior, since the beginning we have wanted to get the modular computation or to imitate biological cells ala Alan Kay approach but going monolith reduces the complexity of all the intercommunication problem.
Data parallel training will not allow you to train larger models, just train whatever models that fits faster. If the limiting factor is the model size, the solution has to be to split the model across different compute units.
While this sounds incredible, I was a bit taken back by the unsupported claims that the CEO made. Is their supercomputer x50-x60 or even x200 times faster than a single A100? I want to see real comparable figures detailing the configuration of their solution because only then will we be able to determine the true level of performance that their solution brings. I've seen marketing material from Intel for their AI solutions where they made claims to be faster than NVIDIA, but when looking at their comparisons they were comparing their solutions against 2 generations older NVIDIA hardware. So I don't trust these sort of marketing stories. While building wafer scale processors seems like a great idea, the reality of process yields is that not all of these processors will come out of production functioning, which will drive up costs. The advantages are of course that you can pack more processing capability in a smaller form factor. Each A100 comes with external components that take up space, which also requires a system that is larger to house these multitudes of A100's. The latency issues that you mention that hamper the A100's will also hamper Cerebrus. If data needs to be moved between the wafer scale processors or even the geographically spread out Galaxy supercomputers they will incur latency penalties. There is no way around that. The advantage that NVIDIA has is that their A100 based supercomputers are not limited to AI work, they can also run general High Performance Compute workloads. No word on whether Cerebras's solution can do the same. Don't get me wrong, I see that this is quite an amazing achievement, but there are both technical and economical challenges associated with their solution, and I'm always skeptical when company CEO's start promoting their solutions. I've seen too many presentations where they push the most optimistic view of their product in order to wow the audience.
This is a really interesting video. The part I struggle with is the performance comparison between nvidia and cerebras, seems like comparing apples to oranges. How many nvidia chips are equivalent to 1 cerebras? And then how do you define this equivalence? I suppose those papers you link to will have some details lurking in there somewhere but for now I’ll just rely on what is presented in this video.
I can't get any real use out of the 'waferscale vs A100' comparison... sure, the Cerebras superchip is many times faster, but a server rack full of nvidia GPUs is going to have quite a number of chips inside. As such, a 'single rack' comparison would really interest me in terms of compute power, investment cost and energy consumption.
One thing to keep in mind when assembling a server rack full of GPUs is that you will be far from having perfect strong scaling. In other words, if the Cerebras CS-2 is about 200x faster than a single A100, 200 A100 will still be slower as they will not get perfect strong scalability.
- Always interesting. - Thx. - I especially like the detail about important role of prep time to set up for training. These nuances can be lost in certain presentations of the data. - As a teacher/consultant, I find that the fundamental problem is an incorrect, and/or incomplete understanding of things. One must study wide, and deep, and question understanding along the way. Many people are not willing, or able to do this, or perhaps just don't think it's worth the time - but in some cases, they do so at their detriment; and others will succeed where they fail. But, to each their own, I guess.
I could see Nvidia making a dedicated AI PCIE X16 card. In the future you would be not only upgrading a GPU but an AGI Card too. Thankfully Many boards have multiple PCIE X16 slots.
The correct way of evaluating designs such as this is not to look at raw TFlops number (use your favorite Tflop) but look at TFlop/mm2 (die strate area) Tflop/$ (Product cost -- essentially mm2 die strata x cost to manufacture ) or Tflop/W ( power to get the Tflop). The massive parallelization of AI workloads essentially makes scale-up (large parallel collection of individual components) relatively simple. On this basis the Cerebras chips are not very compelling at all. Today most of the traditional companies are going the route of some kind of 2 1/2 D integration -- essentially putting multi die in a package. Server chips have crossed reticle limits on a package so Cerbras attempting to make the wafer as a package -- rougly 40 550-580mm2 die -- isn't compelling. Single package server chips are 4 die, 6 die configuration ( lets assume 400mm2/die) means that roughly 10 of the packages would hit the wafer scale structure Cerebras shows. A single 1u can easily accomode 4 such pacakges so what Cerebras shows can be accomodate in 2 server 1u racks *easily*. The density of scale-up for the GPU companies is likely even higher. If you now factor in the need to hook large memory (DRAM) to the individual compute, wafer scale really looks unwieldy. Often the external I/O is done at die perimiter with the areal portion chewed up by power and thermals -- here again wafer scale is at a large dis-advantage simply because of perimeter/area being proportional to 1/d. Might be much cheaper to die singulate and put them on freaking large panels used by the large screen industry if all we want is bragging rights on "see I have X TFlops" -- never understand why engineers waste time with such things.
I think we must investigated with AI in vitalizing energies. DNA provides energy to RNA (molecule of live). How did ADN provides energy to RNA? The answer is the best. How would you do it? What do you think would be the best way? That's the best clue to find out.
Do you think this will be successful enough to fill the supply gap in the ai gpu market? It would be amazing because ai startups are starting to buy gaming GPUs which are not desined with enough VRam that it takes to work on ai
How is Cerebras performance compared to Teslas Dojo supercomputer? Cerebras is planning 36 Exafops till second half of 2024 while Tesla is planning for 100 Exaflops till October 2024. But it would be very interesting to compare the systems, what the differences and similarities are.
Really impressive in the evolution of IA and the usage of AI. In the Middle East they will be thousant stepst in front of others. Nice to find out !!!!
