Raspberry Pi Supercomputer Cluster 

Does this method your demonstrating using the MPI library allow these RPi's to combine system resources ? I know you said that it sees 4 Pi's = 16 Cores , does it also pool the RAM ? Assuming the answer to both those questions are yes , can you still enable ZRAM within the cluster ? When compiling programs on the Pi it tends to hit swap after a point , ZRAM allows the Pi to swap to the RAM instead of the MicroSD (im sure you familiar). I would want to use a setup like this for compiling , does this cluster configuration help me in that regard ? Thanks Gary .

I suggest another rack with fan kit: www.amazon.com/dp/B07MW24S61 and for the UK: www.amazon.co.uk/dp/B07J9VMNBL

Hello Gary, I would like to thank you for making this video and maintaining such an informative channel. you are very easy to listen to and your explanations are to the point. Keep up the good work. Today was the first day i saw your channel link and decided to give it a go. the raspberry pi is a great pltform. However the raspberry PI Zero is a very compact development board as well. The Pi Zero does not have the compute power of the big pi. however the integration of the wifi on the board makes it one of my favorites. I built a super computer using the PI Zero as the main board. I loaded each PI with aversion of Windows 2008 data Center Server. I used RUFUS to flash 64 GB microSD cards with the OS. Once i had the OS working I cloned it using tis program... clonezilla.org/clonezilla-SE/ I have a multi-slot microSD card reader. It holds 24 microSD cards and connects via usb 3.1. I installed teh Windows 2008 Data Center Server on the microSD cards in about 15 minutes. Windows Server 2008 Data Center can cluster up to 32 machines processors at once. The Windows platform is very stable yet it is a little large. About 3 GB on each microSD. I used the data center server services to aggregate all the Raspberry PI processors and resources. i used the Wifi on each chip to bridge them all together. I only spaced them about 1/2 inch apart. The network and teh data processing is so fast that it returns almost instantly. I mainly use the cluster just to browse the web and play games. It is absolutely over kill on any kind of gaming or graphics program. I set the paging file on all the drives to 512 initial max 4096. I also connected an external 3.0 6 TB hard disk to the USB mini on one of the PI. Number 32 in the cluster. Then all teh data i download goes thru the other 31 and passes down stream directly to the external like a funnel. I can download a 4K movie in about 55 seconds with my AT&T fiber. I thought you might be interested in looking at this flavor of PI super computer. Of course i have Python, PHP 7.2 and AMPPS installed on the cluster. Automatic load balancing and wifi VLAN tagging. The windows 2008 server data center can open up a whole venue of new and powerful applications you may be interested in

@@kevindeng1889 hi

@@eg3730 hi

But if every desktop computer is a supercomputer…

@@SS-ARYAN then we can only dream bigger, my guy 😎

considering the physical limitations of transistors, the only way to turn a single device into a supercomputer is through the cloud.

Guessing you have that 4 year bachelor's degree and you're referring to it aren't you?

No. Just no.

I know you're trying to compliment the video (maybe inflate the audience's ego?) but I think your university ripped you off...

the university teaches you in more detail and less effort. this video is less in detail (in a nutshell)

@@infinity5288 Less in detail with more effort put into explaining the detail. I would say that teaches more, because the less effort put in, the less that you get your point understood, therefore you teach LESS because it is not taught, just stated.

So this video is misleading? 🤔

@@hammercanttouchthis it was a nice simple example.

It depends on the problem you're trying to solve. If the problem you're trying to solve requires minimal network bandwidth(small inputs and outputs) but requires a large amount of CPU processing time... Then the older pi's will work just fine.

@@hammercanttouchthis It was obviously a demo of a theory of clustering put to practise, not a video about optimisating data bandwidth and latency.

WOW!

Yeah, even the chips you find in those musical birthday cards have more computing power than all the Allied Forces put together did in WW2. It's crazy.

@@AbhinavSubramanian but we went to the moon on that power?

@@dashboy007 even your current computer/laptop/phone no matter the model is still many times more powerful than the greatest computers of the ones used for the first few moon landings

@@bnbnism I was trying to be sarcastic. There is no way my phone today could power anything else but itself, let alone a rocket ship.

I agree... He is very good at the explanation and easy to listen to.

Watch tech quickie on yt

@@falcondarkshadow8711 agreed linus and the gang really do good job there

@@philh98 definitely

Wow, thank you!

MARK!!

What version of DOS did it run on? And did you mean it ran on the IBM PC or XT? :)

@@hammercanttouchthis By that time, 1991? I was running it mostly at work on IBM XTs running MSDOS 3.3 or later, as i recall because we had 3.5 inch floppies. The entire programming environment and game fit on a single diskette. It was only 10,000 lines of QuickBASIC code. I never released it to the public, but I had one other friend who was interested in programming who liked writing R-code for the warbots.

Reminds me of Robot Wars. My friend and I had a blast programming our robots to pummel each other in the ring.

Glad you think so!

Glad you liked it!

Yeah, I agree that would be interesting. In fact building different clusters from various bits of historical and new hardware and then benchmarking them would be quite interesting, but alas very time consuming!

@@GaryExplains my master thesis where done in the early 90:es, and it was about creating dynamic computing clusters using heterogeneous computers (various hardware architectures at the time). Given the overall limitations we identified types of problems which could scale using the available technique. Great fun and on the cutting edge for it's time. The main benefit was that the computer clients connected where largely unaware that they committed computing cycles, the jobs was running int the background. The base was done in pvm, in many aspects the successor to mpi.

Closet we can get now a day in my Datacenter that i take cared is FC (Fiber Channel). Every single server in DC and inside cluster are connect together in network via FC for lowest latency and maximum transfer rate.

We use infiniband in an HPC setting. Its connected through the PCIE bus.

That was done with the PS3.

Yup, that was a project that the US Military did because the PS3s had a lot of cores in a relatively compact form. It made being able to source machines from around the world really eaay.

No, unfortunately I am not going to SciPy.

Loren Sims does each node have to match the rest, or can any machine get added to the array?

That would be pointless, because unless your doing instense audio processing, there would be no point in having a dedicated audio pi, and just one pi is not enough to do much video editing on, and at that point what would your third be used for if not everything else?

🖕🏼👎🏼

Yes, there are plenty of different ways to check for primes are likely primes. But that isn't the main point of the video.

You are so kind

PCIe doesn't act super friendly all the time. It's probably better using InfiniBand cards

@@NoorquackerInd Well if you're going to use a comm card, it would be a lot easier to put a ten Gb network card in the pcie slot. I was just thinking to keep costs down, you could try to run the messages over just the pcie lanes.

Which module are you referring to?

I'm also interested

For what I knew it is in arch Linux, maybe