Cloud GPUs Tutorial (comparing & using) 

I'd like to, but there are some hardships there. Any idea on what you'd even like to see trained on one of these machines? I have one idea, but curious to hear.

@@sentdex My idea it's pretty basic, see how fast can a TPU train a huge heavy duty model compared with the best gpu out there and if there is any advantage in price, more specifically at what point the TPU becomes actually more profitable than the GPU or if it is the other way around. It is possible that all this has an easy answer without the need to test anything, but it was the first thing that crossed my mind ¯\_(ツ)_/¯

The TPUs on Google Cloud are almost never available.

I've had some woes with PS, but overall been quite happy. The bigger issues I have with them is the virtualization (which also explains your run-to-run variations). The GPU isn't really attached to the machine, but this is also why you can just power down and not be billed. The virtualization stuff also causes issues when I try to update things like the TF version...etc. We've been able to still do everything, but it can be a pain some times. I do plan to eventually do a cloud vs local video as well. In the end, I do both. Cloud is convenient first, and, can possibly save money. Depends what you're actually doing though. Hope you enjoy your 2x turings :)

For me, as someone who has a casual interest in machine learning, cloud GPU services are just too finicky and take too much effort to set up and get to work well, not even considering that they’re usually quite expensive. I’ve preferred simply having the physical hardware. Just upgraded from a GTX 1070 to an RTX 2060 Super. Easier to work with and cheaper in the long run than renting servers.

@@eyeborg3148 Agreed. That 2060 Super is a great GPU.

@@MachineLearningwithPhil took a quick look at some of your videos, seems interesting. Subbed.

@@sentdex Yeah that's true!

Someone else just pointed out to me that SSH is already avail in cmd.exe. Same with SCP!

@@sentdex To top it off, you can even install openssh server through features in Windows Settings and ssh into your computer, naturally, you can also type bash in the ssh session to manage the Linux subsystem.

Yeah, I got about ~160us/sample on a gtx 1070, 3 seconds per epoch. Did we do something wrong?

Wow no credit card? They're playin fast and loose lol.

Also got a verification requirement screen. Not the best first impression, probably gonna be pain in the ass in the future...

Working now. They require a ticket be open. In the ticket I asked what the problem was and they replied within a few hours that they are having a problem with fraud and immediately enabled me for creating GPU VMs).

@@neilgreisman2687 for me linode support was a pain, and they banned me, although I was filling to provide all required ID docs... Anyway, found better alternative and quite happy now, so it's all good.

@@Noth3ng Good you found an alternative. When I first tried to create a GPU instance, it was refused with a message to open a ticket. I did this and within a few hours, got an email saying this was a check as there is lot of fraud going on but I am authorized. I have created a few GPU instances but they won't boot (but I can see I'm getting charged). Another ticket open and waiting for response. They seem to be having some issues. Mind sharing your alternative??

@@neilgreisman2687 Good to know

Yes I have same problem

I have absolutely no clue. I sadly have not really looked into anything kubernetes. I am aware of its existence and sort of what the point is, but never really dug into it myself. In this case, I am not sure how much that would alleviate things, but however you want to manage the flow the data TO the GPU server is fine. If you know of a faster/better way, then go for it.

Main issue is the cost. I'd like to, but it's fairly prohibitive to test them out.

O.O blew my mind. Thought I had to use powershell. I see scp here too. How wonderful. Thanks for letting me know! Has this been in win10 since the beginning?

@@sentdex It's new as of a recent update in the past few months.

Should be an option to "delete" or "destroy" your linode instance. I forget exact term they use

You could also check out rsync, it's way faster

Doesn't look much like it is. There are some lower end GPUs that you can rent, but you can rent similar ones via AWS...etc. Seems to me like their cheaper only when ... you compare a lesser card on there to one of these high end ones?

@@sentdex depends on workload but the speed will surely be less when compared to hpc gpus. But the prices are low so can be useful for a lot of guys.

Because that was just a sample data set small enough to be viable for a video tutorial. Often your dataset size will exceed storage. Also you want access to things like model files, your entire env...etc. the idea is to be able to quickly spin up and destroy the GPU part.

sentdex nice man, thankss awsome tutorial btw keep rocking

I tried to do it now and the script is not up to date anymore. Additionally, linode doesn't have any GPUs anymore and they told me they try to get more for 2024...

It should work for anything you might use the GPU for. If you need to actually visualize anything on the server itself, you'd need to install some X server, but even that can also be done.

For regular hosting, the same applies. I use linode and digital ocean.

GPU power. If you don't know how or why cloud GPUs could be useful, I suppose stay tuned for a cloud vs local GPU video. Cloud GPUs are more convenient, often cheaper, and more powerful than whatever you're gonna have locally. At some point, it becomes more cost effective to have local power, but this is usually a very large dollar figure that exceeds what most people are after.

@@sentdex ok, thanks for the answare :D Keep up the good work

False. As long as the machine exists and you have a GPU being dedicated to you, youbwill be billed. Thats... kinda the whole reason I made this tutorial, to show how to mitigate this aspect of cloud GPUs

@@sentdex oops, I have edited my question, ok, Thank you very much!!! I get it.

Will the credits work with GPUs? Often "Free credits" don't seem to work on GPU instances (since people would mine crypto with them) ... but I dunno how the credits through that program work.

@@sentdex yes, they do. I have msdn enterprise, which gives around $100 a month of azure usage, which I use for some machine learning experimenting.

Not a theme iirc, just a font and color change, you can change in properties. I forget the font name though.

Aaaaaaaaaaand you can't use it for GPUs. People would just keep making few new accounts and mining crypto :P

I didn't know, lol. I've only used Google Cloud as a VPS. I've never been interested in machine learning up until very recently.

@@sentdex I'm using GCP and I can use GPUs as part of the $300 credit with no issues. But they do prevent it from being used for crypto.

While there is a difference in speed in the OS', I've never seen something this large. There's also a large difference between win10 and 7, and the versions of Ubuntu. Then those differences also break down into the hardware you're running... it goes on and on. To be honest, I have usually found it to be a wash :P

@@sentdex In brief the way the kernel interacts You may have noted amount of VRAM utilized in Windows when not in use its around 1-1.5GB for useless visual quality vs 50-100MB for Ubuntu With GNOME Desktop and eventually almost 0GB for Cli interface/VPS moreover, Nvidia CUDA drivers are specially optimized for ubuntu for GPU accelerated Computing research... Isn't it always good to learn something new that's very minimaly difficult but highly efficient ? Switch to Ubuntu and see its Magic...

It's not like I don't use Ubuntu. I appreciate the info, but I dual boot win/ubuntu, and obviously use Linux servers daily. It's not like I only use Windows. Also have macs. I use all 3 OS, because all three have strengths and weaknesses. Definitely would *not* want to use solely ubuntu. As for the speed delta, something else is going wrong. I've long compared Ubuntu and Windows for TF speeds. It's usually actually a wash, and I've seen Windows be faster :P

@@sentdex In my case RTX TITAN V performs upto 25-30% better at Linux Local Server than on Windows Server, don't know what's on your side ; ). Suggest you to try Meta Machine Learning at Carla for better results ; )

I could

@@sentdex imma try until you do

LMAO right. I tried to edit it out, but damn I hate editing. I did get rid of some of it at least. Still, annoying to say the least. If you watch on 2-3x, it's even more annoying lol.

Dno anything about that

sentdex I found that software very interesting for data mining it has some machine learning things in it ,I just want you to try 🤗 and tell about it.

I don't know for sure. I think you have to delete the instance. With TPUs, stopping them is enough, but I don't know if that's true for the GPUs. I would guess it's not.

No they don't charge stopped VM. They charge the storage though

Careful, I don't think that's right. If you stop the VM, you aren't charged for the VM, but you will still be charged for the things attached to it, including, I believe, the GPU. Rohit, I strongly advise you double check. It'd be nice if Hamza was right, but I don't think so.

Yes they don't charge for CPUs, GPUs, and ssds after stopping instance a/q to this link cloud.google.com/compute/docs/instances/stopping-or-deleting-an-instance

Took a peak. Not sure I'd want to use it over other more ...reliable and dedicated solutions. Yes, some options are cheap there, but you can also renter lower-end cheaper GPUs on AWS, Azure, Google, and Paperspace.

sentdex oh I see

Thanks!