Don't forget that a CPU core also implements the entire x86/x64 instruction set while a shader core is only going to implement a much smaller and simpler instruction set. This is how they fit so many more cores on a GPU die in the first place.
@@freedom_aint_free Basically the more specific an integrated circuit is the more efficient it is at doing that set of specific tasks or singular task. So the most ASIC will be super fast at one thing, but won't be able to anything else. The opposite are FPGA(Field-Programmable Gate Array), which is even more general purpose than the CPUs we have. You can make it do almost everything.
But if a GPU has sufficient instructions it could be Turing complete and accomplish whatever a CPU can GPUs probably are Turing complete its just we would see more complex programming be it either in source or in the compiler
I remember when NVIDIA did this Tegra presentation and I had to cringe when they claimed they had the first 200 core (or something like that) mobile processor. They really just had a generic arm design and a GPU and added those cores up like they were equivalent.
@@CjqNslXUcM and they still do SOC for their own computers like the Nvidia Orin which has 12 Cortex A78E cores (And a Ampere based GPU with 2048 cuda cores and 64 tensor cores)
@@sparkyispog Yes, they're all ARM based CPUs. That's why nVidia wanted to buy ARM. The Nintendo Switch uses an nVidia CPU. GPUs are ussually paired with their CPUs, though. I think the AMD term for that is APU (Accelerated Processing Unit), which means "A chip with a CPU and a GPU in it."
Basically, CPUs are optimized to minimize latency, GPUs are optimized to maximize throughput (bandwidth) While at first glance they seem to imply the same thing, they do not. You could get a result from the CPU in 1ms, but only process 10 items, but a GPU can process 10,000 items in 100ms. You would expect this to mean 10,000/100 = 100 items in 1ms, but yeah that's not how GPUs work. You pay for the the high bandwidth in latency It is nuanced, but once you understand it, the difference is actually night and day. GPUs aren't also flexible. The programs you write, are "inherently" parallel. No std::thread kinda stuff. You write a scalar like program that is "automagically" parallelized, so you have to write thinking about parallel access from the get-go
This is also why GPU's can be saddled with RAM that has massive bandwidth but not so great latency. GPU work load is fairly predictable in that sense and thus the latency can be worked around.
Better analogy - CPUs are surface streets, GPUs are the express lane. You can travel much faster in the express lane but it only goes one direction. You can head anywhere in the city on surface streets or turn around at any point, but you can't go as fast.
TFlops don't mean anything for performance in games, because this is what were talking about here, there's many GPU's like the Titan V which has 110 TFLOPS but is no better than a 2080ti which has 14, also many GPU's that have less than that still outperform lower TFLOP GPU's, don't even get the point of using that as a metric to prove how powerful something is. Especially when you put it in context of Game Performance, sure it might be better for editing or other applications but otherwise, no.
@@marmite2956 doesn’t matter my dude. The info presented in the video was factually incorrect in regards to the specs of the 4090. A factual error was made, corrected in the comments, and acknowledged by the OP. That’s as far as the conversation went and had to go. Functionally, the specs of the gpu don’t make any meaningful impact on the content on video.
A great explanation. Thank you! In a base analogy, it can be taken like thus; CPUs are like 3 architect/builders. They can perform numerous amounts of complex stuff well, but they're limited in number, and therefore efficiency when complexity isn't required. GPUs are like ant colonies; not smart enough to build wonders, but there are many enough to work fast and efficiently on singular tasks.
Good video. I'd like to add that programming GPUs in a way which approaches the advertised performance is rather difficult. You have mentioned the branches, but also, they are lacking features (no call stack, no dynamic memory allocation), ideally need specific memory access patterns (search for memory coalescing, and bank conflicts), have manually managed in-core caches, and important technical info is a trade secret (like their instruction sets).
@@FMHikari VRAM bandwidth is about an order of magnitude higher than system RAM. Due to lower bandwidth, system RAM only consume couple watts, and these modules don’t even really need passive heatsinks. Despite hardware vendors are happy to sell modules with heatsinks, to get a few extra bucks. However, due to the much higher bandwidth, VRAM chips in modern GPUs require not just passive heatsinks, they actually need active cooling. While it’s technically possible to engineer modular VRAM which would support active cooling, I’m afraid the costs of that gonna be prohibitive. CPUs do that, but they cost hundreds of dollars, and there’s normally just a single CPU in the computer.
@@FMHikari It would be good if it was possible though, especially for AI things you tend to need a lot of VRAM but Nvidia would rather sell you an expensive datacenter GPU for that
You should have done the mechanical layout of the a CPU vs GPU core. Its a clear the difference that way. Way more parts for the CPU core as they are very different and not even in the same realm.
You just shouldnt call GPU FP units cores, that's just a marketing term from NVIDIA. Shaders or PF32 units would be better names for what they are. The closest thing in a NVIDIA GPU to a CPU core would be something like a SM. And there are only like 128 SMs even in the highest End GPUs.
@@jennalove6755 First of all, there is no exact definition for what a core is. Typical properties of cores are: 1. Memory is shared across cores 2. Cores can process data and instructions independently (MIMD) 3. (at least for symmetric multi processing) all cores are hierarchically equal (there is no master core that manages all other cores). 4. Each core has its own front end, Execution engine and memory interface I don't know why you're mentioning Bulldozer?
@@jennalove6755 It would be more appropriate to consider the CUDA Cores as FP-ALUs. Making a full comparison is tricky due to FP vs Int and FMA vs FADD. But if we compare FMA only, then one of the dual-core Bulldozer complexes would have "4 Cores", and that goes up to "8 Cores" if you include the FADD units. So really, AMD could have argued that they have at least 2x as many cores as they did (2x FMA per Int core). And yes, I would claim that each complex was 2 cores as a core is typically defined as the number of front-ends (parts that fetch instructions). This is why the appropriate comparison with a GPU is the SM == Core, and the CUDA Cores are SIMD ALUs similar to the SIMD ALU cores from SSE on x86 (or NEON on ARM).
Great presentation! Size / yield / energy : a big clever CPU core is harder to manufacture. Scaling out CPU to the same amount of cores as in a GPU, but keeping the complexity of the CPU… you’ll get insane power draw, low yield and insane prices like super computers. One thing cut for time here: that memory architecture is very different because they are built for so different purposes. GPUs have specialized memory that shuffle a lot of memory at very wide buses; read a lot of closely aligned memory. CPU have very narrow busses (in DDR5, 2x32bit per stick). So a CPU chip can shuffle a lot of different data at the same time while GPUs are good at shuffling a lot more of the same data. So the GPU memory model is bad for running multiple different programs at the same time. So the literal hardware interfaces of the chip are built for extremely different purposes, entire different programming idea :)
@@iamraihan3203 sorry for late reply on something I am not an expert on M2. I imagine that being bandwidth limited is an issue for some applications because when I read about it is based on LPDDR5 DRAM. So maybe extreme gaming at max resolution and max fidelity cannot produce FPS similar to a “real” GPU with monster busses. If there is anything in m2 architecture that makes this less of an issue I don’t know about it. Just being on same chip as CPU shouldn’t make bandwidth larger than the physical RAM bandwidth. CPU having shorter latency to GPU might give some wins in specific workloads where bandwidth isn’t the limiting factor. But max graphics with tons of high resolution textures ought to break it. Similar to consoles I imagine game devs might optimize games for avoiding limits when there are few M chips with known performance. So based on my limited understanding I don’t see how PC games requiring large VRAM + bandwidth requirements could ever be ported to m2 without bottle necking earlier. As long as the graphics problem is bandwidth limited speed between RAM/VRAM to GPU chip is the limiting factor. So with a bunch of caveats; for extreme graphics you want a very fast bus between GPU chip and RAM. Apple is known for good hardware though, such as their video encoder/decoder that was leagues ahead of what NVIDIA shipped. That made a huge difference for Canon R5 footage that that Apple could accelerate but NVIDIA could not. So not all problems are about bandwidth limitations. Caveats caveats caveats :-)
Clocks don't have to do with density directly. It more has to do with how deep the pipeline stages are and transistor yield and quality. Faster clocks demand that the entire pipeline stage, which can be multiple transistor "layers" deep and very wide, be completed by the next clock. You can either shorten or lengthen, narrow or widen these stages for various benefits. GPUs tend to have very wide, very deep pipeline stages which take a long time to process and guarantee data is fully through all the transistors, thus pushing clocks down on GPUs. But it gives them some amazing bandwidth energy efficiency at the expense of a bit of latency. More current gets this happening faster but produces more heat, which is another consideration, as modern computers are far more heat production limited than anything. Heat cannot escape the dies fast enough at the currents necessary for the speeds we demand and they have to be turned off 80-90% of the time. Dark silicon is still a big problem. It's also part of why low power processors aren't not that much weaker yet consume 1/10th the power.
@@badass6300 The clock speeds are typically determined after the design based on the parasitics within the manufactured die. This is also where binning comes into play, chips with higher parasitics are binned to lower clock speeds. The choice of CU/SM size is more of a tradeoff between area (to reduce die size) and pipeline latency (the shorter the pipeline, the more efficient warp switching becomes). Clock speed estimates are taken into consideration at that stage, but in a much more heuristic way that doesn't really consider parasitics (those end up affecting the design as you get closer to tapeout and perform advanced electrical simulations).
Saying a GPU is faster than a CPU is like saying a rock is a much better car than a tennis racket. Unless you have an explicit context and an exact specification compared to _what_ it is supposedly faster there is absolutely no point in even trying to reason what such a statement means.
One thing to note. Starting with Ampere, Nvidias Cuda cores have a FPU, and a combined Int32 and FPU. They aren't completely split anymore. They did this to increase the max theoretical performance, however, there's almost never a case where some Integer calculations are being run. It's really quite interesting actually. AMD has done the same thing with Navi 31, in the Radeon 7000 series. I guess it's a way to squeeze out some extra performance without increasing die size
And also starting from Ada Lovelace (40 series), NVIDIA's SMs support on-the-fly thread reordering which basically lets them reorganise threads across multiple warps such that each thread in a warp is actively executing the same instructions on neighbouring sets of data. Where past architectures may have some threads block execution when the warp enters a branch, Ada Lovelace can detect this and reorganise these threads so that all threads in a warp are taking the same branch, while the other threads that wouldn't have taken that branch are organised into a different warp populated with other threads from other warps that _also_ wouldn't have taken that branch. Caveat with this is it seems like this is only supported with callable shader execution where one shader can invoke another on-the-fly, which I assume basically gives the GPU time to do the reordering. NVIDIA also has another optimisation up their sleeve called subwarp interleaving, which basically lets a warp dynamically switch between two branches of code whenever one of the branches encounters an instruction that would block execution anyways (such as an instruction depending on data being read from VRAM). With current architectures the entire warp would just end up stalling since the first branch is waiting on a blocking instruction to complete and the second branch is waiting on the first branch to complete, but with this optimisation the warp would be able to switch to the second branch from the first branch, execute the second branch until it also encounters a blocking instruction, then switch back to the first branch where the instruction would hopefully have access to the data it was depending on. This isn't currently available in any publicly available GPUs, but NVIDIA was testing this out on a custom Turing (20 series) GPU and saw anywhere from a 6% to a 20% improvement in raytracing workloads, which is one of the most branch-heavy workloads you could give the GPU.
@@jcm2606 some insider information? or is it available to the general public? Take care bro, I hope this doesn't count as casually breaching classified info on a random comment on YT.
No idea how I found this video but I'm glad I watched it! You explained the difference between GPU and CPU cores very well and I could actually follow quite easily. You earned yourself a new sub!
5:38 This specific design of GPU cores is also why modern GPUs tend to have multiple types of cores as well - each core type designed to handle one specific type of task.
Btw you can run any directX graphics on your processor. A 13900K is about equal to a midrange card from 10 years ago, you can play CS Go on your CPU even without APU
Thanks for this correct and honest answer at the very end :) Ultimatively it depends on what should be done and how it's implemented. Certain tasks can't be done efficiently on a GPU. GPUs are great for pipelining. So if you have 10 operations that have to be done in succession, it actually sets up several parallel pipeline you can pump data through them. You essentially get a fix delay which is about equal to the pipeline length but other than that the data can be pumped through at the clock speed. The worst thing that can happen to a GPU is to stall a pipeline. That's why using tons of different shaders is much worse than a few well designed ones. Though with actual CUDA cores they implement more and more concepts of a CPU into the GPU. Though in the end it's still a piece of specialized hardware for certain tasks. Yes, they can do some tasks a CPU can do as well, but are often times slower. Though for pure data crunching they are great, at least when you can adjust the algorithm for the architechture.
It has alaways been slower, the advantage of a gpu is not in speed (as some might think due to how it can render amazing graphics given some pipelines), but in concurrency, the sheer amount of cores with specialized functions does wonders for its specific needs.
Amazing Explanation! :) This explains why Half-Life 1 and Counter Strike 1.5 was the last games of the "Software Renderer" era , and why we can have cool graphics nowaday, even with "low end" gpus
gpu shader cores from my understanding are little more complex than the first pentium cpu cores, but given modern architectural design, emphisis on massively parrallel design, and newer process nodes, you now have a massive number of relatively slow processors but with a huge number of 'threads'
My favorite way to tell people what a cpu is by using a restaurant analogy. The Cores are people, and the customers are incoming data, and each core/person has a task that it has to execute said task. Its the best analogy that i used to understand cpus
Each NVidia warp is basically equivalent to a single CPU thread capable of executing vector instructions of size 32. A simplistic way to the estimate the sequential performance could be to divide the number of NVIDIA cores by 32. That does not work because of other major design differences with CPUs. The GPU cores are themselves grouped into SM ( Streaming Multiprocessors) designed to execute up to 32 warps in a way that is roughly similar to CPU hyperthreading but at a larger scale. Also, the individual warps are not optimized for speed. A CPU can execute simple instructions from a single thread at a rate of 1 per cycle but a GPU SM can only process the instructions of a single warp with a significant latency ; maybe 1 every 10 cycles for the simple ones to a few hundreds cycles for memory accesses. To make things worse, GPU have very little memory caches because they would be inefficient (or very expensive) when running tens of thousands of threads in parallel. Instead, GPUs rely on the principle that the memory latences between instructions are hidden by the fact that each SM is running up to 64 warps in "hyperthreading" mode (and the whole GPU may contain up to 100 SM).
One CPU core can execute several instructions in parallel, this is why SMT/HyperThreading is possible. While it's a bit tricky to define boundaries of cores, one could say that a x86 core with 6 ALUs and 4 FPUs would """equal""" 4 GPU "cores" as far as throughput is considered. Otherwise x86 CPUs would be limited to "number of cores * frequency * 2" FLOPs, but instead the number is much higher (and I think CPU FLOPs are calculated by "number of cores * number of FPUs per core * frequency * 2") And then there is rabbit hole with "double purpose FPU/ALU" + FPU combo in the newer Nvidia GPUs, which essentially means that theoretical performance is anywhere between half of what you'd expect (equal number of INT and FP instructions) and what's advertised (pure FP).
The internet really needs a well-researched video on capability hardware. There's a good book on the subject but it only covers mainframes, and nothing after the 80s. I get that CHERI (including ARM Morello) is the big thing now, but the intel i960 was absolute perfection.
The GPU cores are almost like hardware version of specialised subroutines. In fact, that is what part of the CUDA cores are for, calculation of the triangles for 3D graphics. Meanwhile, CPU are general purposed processor that can execute any of the instructional sets within its CPU family. Think of how Amiga PCs were capable of arcade like graphics and sound because it had specialised sprite and sound processors in addition to the Motorola 68000 CPUs they used. IBM PCs had only Hercules/CGA/EGA graphics card, which were really ram/register buffers with DAC circuits to flush the data from c800 to the VGA analog output. Sound wise, IBM PCs beeper speaker were limited until Yamaha OPL3 FM chip gave it midi and a few voice channels. This off loaded specialised midi, wave, DSP and other related sound processing to sound cards.
I'd say comparing the two is like comparing a big commercial ship to a ship building mashine. Very diffrent use cases but one definetly benefits the othet :)
I think it's also important to highlight the fact that FLOPS only measures the performance with floating point numbers. 80+% of your everyday tasks don't even involve floating point numbers at all
Some older GPU arches actually didn't support dynamic branches at all. And even today result is severe cost in performance: the GPU actually computes both if/else code paths and selects the result via boolean (condition) I.e. the shader if-else is likely compiled into branchless code.
This highly depends on a compiler (both IL and driver ones). If it decides that the difference between both paths and the mere execution of a branch will be negligible, then it most likely will result in a removed branch (so both paths execute). You can directly specify and hint the compiler what you want to generate with a [branch] or [flatten] atrributes on HLSL, don't know about other high level languages (glsl has no support for those iirc).
@1:56 I really thought that'd happen someday. Back in the day I had a socket 939 motherboard with a slot to take an AM2 daughterboard. When GPUs started to do compute workloads and high end SSDs. Came on PCIE cards I thought it wouldn't be long before we get whole systems running on PCIE slots.
It should be noted that there are tasks other than graphics that lend themselves to SIMS processing - more or less anything that involves vector or matrix operations (signal processing, solving massive systems of linear equations...). That's where Nvidia's CUDA API comes in.
from your description, one alder-lake core (with avx512 enabled, so saphire rapids) would be equivalent to 32 cuda cores, since it can initiate 32 floating-point math ops per clock cycle. except that they can be two sets of 16-way simd, and simultaneously can perform another 4 logical ops and more memory ops at the same time, twisting and winding it's way through 2 arbitrary threads at the same time with far greater flexibility, and at a higher clock rate. so top xeon cpu's can theoretically manage >7 terraflops, as long as the memory can keep up. still 7 times slower, but much easier to design diverse code for.
I am thinking you are not understanding cores half as well as you think you do. Compute units in a GPU is equivalent to a CPU "core". Roughly 40 CU's in a gpu on average. You took marketing speak as the tech. Doesn't mean you are wrong in the end. But GPU cores are not what you think they are.
A GPU core is smaller and more stupid, cannot act alone ; everyone in the group does the same instructions (program). A CPU core is much bigger and more independent, and each core run one (or sometimes several) programs at the same time. That was what the SIMD / WARP discussion was about :)
That's a very good takeaway actually. The essence is that a GPU "core" is just a very small math unit, it can do a couple of math operations and that's it, but also it's not an independent unit, it needs to be organised in bigger groups with other "cores" and run the same operations together but just on different data. That's very usefull when for example you want to apply a shader to the entire image, i.e. multiply all the pixels on the screen with a different number to affect their lighting levels. But when e.g. you want to display a webpage that has all kinds of different elements you would have to do that element by element, i.e. the entire GPU thread calculate this button, then another entire GPU thread calculate the position of the other image, etc. it becomes ridiculous. A CPU core on the other hand is essentially like an autonomous computer, you can run hundreds of different instructions, math, logic, weird jumps, memory operations, etc (it's getting really hard to keep this comment high level and not go into details :P).
Those are architecturally very very similar. All of those Out of order superscalar CPU's with wide caches and quite high frequencies. ARM are quite simpler ar decoder stage and beeing ARM64 only (no 32 bit support) helps there a bit. Intel Ring/ AMD IF/ ARM AXI bus - probably the biggest difference, yet the actual biggest differences are between those: Intel 14nm++++ / Intel 7 / TSMC 7nm / Samsung 14nm / Samsung 8mm/ TSMC 5nm
You mean warping material into a form as in related to wrap Old English weorpan "to throw, throw away, hit with a missile," from Proto-Germanic *werpanan "to fling by turning the arm" (source also of Old Saxon werpan, Old Norse verpa "to throw," Swedish värpa "to lay eggs," Old Frisian werpa, Middle Low German and Dutch werpen, German werfen, Gothic wairpan "to throw" Also Old High German warf "warp," Old Norse varp "cast of a net". Most likely where we get the word 'wharf' where ships are moored.
The FPU in a CUDA core are optimized for particular floating point operations. They're really good at addition, subtraction, and multiplication. They suck at division and most any other function like trig and exponents. You're also going to suffer from starting GPU kernels and data transfers between the GPU RAM and main RAM/CPU cache. And the largest float they can handle is double. They can't do long doubles. Doubles are usually enough. But double precision is also slower than single precision, but that may be in common in with CPUs. I haven't checked.
CPUs actually include SIMD execution units and perform tasks in a similar way to GPUs. What sets GPUs apart is their fixed function hardware used for special tasks like performing raster algorithms, sampling textures and in more recent times neural network inference and ray intersection testing. We have also seen that under the right circumstances compute shaders are competitive with fixed function rasterizations and some CPUs including machine learning instructions. This makes me wonder whether someday we might go back to a unified processor, that forgoes that overhead and complicated programming model that comes with using a co-processor.
If you're looking for a layman's terms distinction you could use general purpose for CPUs, as you did, and then specialized on the other end for GPU cores. Cuda cores, and whatever AMDs equivalent is, are specialized for a very specific kind of math in a very specific way and are thus made to be incredibly fast and efficient at that, whereas a CPU has to be able to do everything at roughly the same effectiveness just don't do as well at those tasks specifically.
You could have simplified this video so much and with less name dropping. While supporting it with a tiny spec of math to make it overall more understandable. Even still this was an overall good explenation. If anyone desires a more compact explenation read the following. The CPU has a certain amount of threads (these are comparable to hands, the more hands you have the more you can do at once). Each thread capable of handling a single operation at a time. Usually the CPU has a number of cores and they might or might not be split in two so that there are twice the threads than there are cores (two threads per core), thus doubling the amount of things that can be done at once. A CPU can handle almost any operation but only a certain amount at a time(=amount of threads). A GPU is perfect for Linear Algebra because it has many many tiny "threads*" each solving a small problem of a greater problem toghetter. Which is perfect for Lineair algebra. As Lineair Algebra is an extremely simple and easy form of math. Just takes a really long time for one person or thread to perform or calculate. For simplifications sake Lineair Algebra is just the math revolving around matrices. These are effectively 'packages' containing data. A matrix could for example give you a 3D coordinate on earth with its corresponding temperature and so on instead of having to express each property individually. Summary: CPU is like einstein, you have very few but they can do a wide range of things. GPU is like a peasant, there are many, they can do very few things. But toghetter they topple empires. If my English is broken please reply with where and what so i can correct myself.
This Thumbnail is very misleading since Intel is manufacturing GPUs too, please change it. If you don't read the title the thumbnail is clearly wrong, Nvidia GPUs are clearly faster than Intel's Arc
GPUs may have many ALUs, but the way they're organized matters. I.e. AD102 has 18432 ALUs. These are organized into 144 SMs, with each (simplified) being able to simultanously run 4 different FP32 instructions, each on a group of 32 variables. The SM is what logically and in terms of functionality comes the closest to a CPU core. 144SMs definitely is in the same ballpark as the 128 CPU cores that are now common in servers. On such a GPU however, you need 4 independent threads (with 32 work items ea.) per SM in order to reach full utilization. On a modern CPU that mark is somewhere around 1 up to 1.2 (on average) threads being required to fully utilize a core. When not having enough different and independent work, a GPU may not be able to flex it's muscles. Besides having to execute instructions on groups of 32 (some: 64) items, GPUs usually also suck at control flow (if - else, conditional branching and jumps), aswell as special (transcendental) math operations, such as sin(), cos(), tan(), sqrt(), … This leads to not every workload being able to actually benefit from GPU execution.
And it's not only the core, but the bus that connects the relevant data source and sinks. A GPU would be something like a 64-lane highway full of trucks (VRAM) with only a handful of individual destinations it can reach. That's because a GPU has a reduced instruction set with copy-pasted circuits per lane. A CPU on the other hand would be a large city with maybe a 4 or 8 lane highway (Cache). From any lane you take into the CPU you can reach more or less any destination register in that CPU.
Note that the Nvidia Cuda cores are probably closer to Intel threads. The best Nvidia equivalent for the core is the Streaming Multiprocessor (SM). For the Ada Lovelace Architecture (RTX 40 series), each SM has 128 Cuda cores. 128 * 128 = 16384. CPUs also tend to handle a larger number of data types with additional instructions (and more hardware) needed for both conversions between each, as well as all different operations performed for each different data type. The GPU core, on the other hand, can be a lot simpler and thus smaller.
The funny thing is if I'm not mistaken that the CPU could be the component that feeds instructions to the GPU since it could be independent while the OS is still running. It's also more dynamic in terms of the operations that it can handle so it could be faster or more efficient on a different math operation than the GPU.
Comparing CPU cores to GPU cores is like comparing a FedEx truck to a freight train. The freight train/GPU has a lot of limitations that allow it to be WAY more efficient compared to a FedEx truck/CPU. Sometimes it's worth it, sometimes it isn't.
Versatile abacus vs highly parallelized calculator. BUT don't forget, that GPU design lends itself perfectly to data processing. Best example MCP7A or also known as geforce 9300. A wonderful hybrid chip consisting of an nforce 730i and a geforce 9300 with working drivers. And everything YEARS before cpus with crappy igpus. Out of that DNA, DPUs were born.
We should be doing something "like" using GPUs where we now use CPUs. Not exactly - GPU cores are quite simple in terms of the suite of operations they can perform. But, we screwed the pooch decades ago in CPU design. We decided that we would let legacy C software drive the development of new CPUs - we took as our goal to stay with a single core design but make that core run the old code faster through whatever trickery we could come up with. And while the results have been impressive, they've also turned our CPUs into complex monstrosities that no one can understand completely. And that's exactly how we wound up with things like Spectre / Meltdown and so on. And in the end we had to swallow the multi-core pill ANYWAY. We should have swallowed it decades sooner and moved toward more simple and easy to understand (and easy to secure) cores, and just re-written software as necessary to make use of them. These days there's more logic in a CPU core to do all this trickery than there is actual logic that does the rubber-meets-the-road computation. It was a mistake, and now it's too late to fix it. Just as an example, take instruction re-ordering. That was just a STUPID thing to do in hardware. The programmer or the compiler can handle that part of the problem, but that wasn't good enough because they wanted the chip to run EXISTING CODE BINARIES faster. So we spend logic on that that could have been spent offering us more controllable computing. Come on, it's obvious: just get the instructions in the right order to START WITH, instead of forcing the logic to re-sequence them for you every time. Out of all of the things we "shouldn't have done," this is the one that galls me the most. Speculative execution is another example. Instead of spending logic on that, we could have spent it on "more cores." I'm not even entirely sure about cache memory logic. Having a fast scratchpad on the processor is certainly a good idea, but I suspect that there was a better way to do it that involved the programmer / compiler being conscious of what was going to go into it. Then all that "control logic" could have been used for... you got it - more cores. The general principle here is that we tried to put logic into the chips to make them "think for the programmer." Instead we should have demanded that our programmers LEARN TO THINK. And focused on putting as many simple, easy to understand cores as possible into the programmer's hands. Now, I have no problem with the compiler being smart enough to do some of these things for the programmer - that's still just a one-time cost, and then you reap the benefits forever. But having the hardware do it, at RUNTIME, EVERY TIME? That's just poor resource management.
It's not weird at all. It's also because GPUs are technically in-order processors (the GPUs that are capable of out-of-order execution now also exist - ARM Valhalla and Snapdragon Adreno are examples of out-of-order GPUs now in use), and CPUs are therefore already superscalar out-of-order monsters. However how you program either or both processors matters a whole lot too, because GPUs are meant to chew through the vector math in parallel (ie. you want as many ALUs busy as possible). EDITED: Forgot to add, SIMD vector processors, especially some superscalar varieties (such as AMD GCN shaders), tend to only work on exactly the same item at a time in parallel, unless they explicitly allow dual-issuing of two different SIMD vector math instructions at a time. FPUs are weird in a way.
Cpu can done some complex task with short time, GPU able to perform simple task in big amount in simple example: CPU is like a person who able to complete 10 advanced calculus task, while GPU is like a person who able to do 1000 3rd grader math CPU deliver data at FASTER rate. because it has 2-20 cores but able to perform up to 4-5 ghz and beyond, even the lower end CPU today able to perform above 4 GHz GPU deliver data at BIGGER Rate. because it has thousands of "cores" but the clock somehow are nowhere near 3 ghz, even the most powerful gaming GPU runs at 2,5 GHz normally
Because GPU handles SIMD, it would be more correct to compare it with SIMD unit inside CPUs... My GPU has 14 cores, but each core can handle 1024 threads. Even if I stress my GPU, it is likely to utilize only 10 or 11 cores as it doesn't have enough memory to utilize all cores with own kernels.
In short: Think your GPU is a really fast sport car that reach 250mph which can carry only 100 units inside, and your CPU is a not so fast truck that reachs about 100mph, but can carry 2000 units, and lets use 'sand' as the data we need to process. The sport car is about 2.5x faster then truck, so if your data is 100 units of sand or less, the GPU can just handle it better than CPU. But if you have 2000 units of sand, isn't effective using your GPU to transport that sand, because it will took more time then CPU. So, CPU is for 'heavy' data process, and GPU is for small data that can send faster then CPU. Thats why in games, we use GPU to render textures, effects, shadows, and other small things, and CPU to handle physics, AI, and some other big stuffs.
The difference in critical path length between well designed wallace multipliers is not that different between fp32 and fp64 - it's two transistors in the reduction phase and one in the combine stage. Assuming multiply has the most restrictive timing in the system (unlikely), you'd gain 10% critical clock speed, but you'd pay an extra 25% space cost, including longer wires between cores.
@@capability-snob that's true, but in the end of the day, rtx 4090 makes 1 terraflops in fp64 with large die size and over 400 watt tdp, while CPU Ryzen 9 7950x makes the same nearly 1 tflops of fp64, so I think my comment is kinda relevant though ¯\_(ツ)_/¯
SIMT limitation in GPUs is no longer true in more recent architectures, however, it is true that they still perform best when they are used in a SIMT manner.
Correction* GPUs don't have thousands of cores. For something to be considered a core it needs to be able to fetch, decode, and execute an instruction on it's own. CUDA cores are just a lane of a vector execution unit.
A GPU core is basically a race car. Very good at driving straight forward very fast. A CPU is a pickup. It will get around the track no problem, just really slow. On the other hand, it can most likely drive through 30cm (1ft) water no problem, over uneven terrain, on gravel, etc. Things that the race car most likely can't do, because it's not meant to do that, ever.
My slow laptop became a nuclear reactor, the chipset is working, and the cpu does nothing. The permissions are in the battery’s chip. To program a laptop this way, a simple trick: discharge 10x in Windows, then in bios. It needs more tries, each time some extra will be enabled. For desktops, UPS has the same chip inside.
Well first off, it depends on the workload. Applications that are highly threaded are generally much faster with GPUs. But single threaded stuff or even lightly threaded apps benefit from cpu
GPUs are great at processing consecutive chunks of data. CPUs are great at processing arbitrary bits of data in arbitrary order. Car vs Semi. I would date to do Uber with a semi. I'd hate to move many pallets of product with a car.
are those cores in a GPU just shaders ? As an expierment why couldnt some one emulate a cpu with a gpu , would it be faster if some one forced a gpu to act like a cpu and run windows 10? A cpu is a cisc processor and a gpu is a Risc processor (cisc means complex instruction set chip) (risc means reduced instruction set meaning it can only one run program at a time and cisc can do mulitple programs at once)
I don't think it's sufficient to just say the # of warps is the limiting factor of a GPU's speed because there's still far more warps in a GPU than cores in a CPU. the 4080 has 304 warps, still like 10x more than what a CPU would have. There's 2 big problems really for why we don't just use a GPU for everything. 1. each warp is significantly slower at individual operations than a CPU core. 2. most software isn't going to be able to scale to take advantage of all of the warps on you GPU and will likely just use 1 anyway. Suppose you wrote an OS to utilize a GPU like it's a CPU, you'd need to be running hundreds of applications at once for the scalability of the GPU to overcome each warp being slower. I think the analogy I would use is that trains can move more people per hour than cars, but as an individual it will take you longer to get to your destination via train than driving in most circumstances. It only makes sense to have a train in a situation where you have a lot of people who specifically need to go from one point to another.
I have programmed both 86 and GPU(both AMD and Nvidia) assembler programs It mostly comes down to the warp. The GPU is optimized to run 32 threads doing the same exact thing at once and the CPU is optimized to run 1 thread at a time but MUCH more quickly and with more resources for that single thread.
