Well you can buy a good cpu laptop with average integrated gpu when you cant afford a laptop with good gpu and later go for this. There are many scenarios like this and you didnt even bother to think of one before writing your comment.@@andrewhill4751
This has not happend, because designing a PCB that can handle 40Gbit/s perfomance is hard and needs extremely expensive test equipment and expensive pcbs to pull it off. Completely different to a USB 3.0 gadget that you can design in half an hour with free software and get it manufactured for a few bucks. And for the MSI/Asus/Asrocks of this world it's a market that's just to niche, they can make way more money on more proven projects.
Not only does it have just 4GB of memory, it also thermal throttles a lot due to it being cooled passively. It's no surprise that it doesn't perform as good as an A500 in a laptop, which would be equivalent to a GTX 1660 or an RX 590
Tbh for the price I would buy a ThinkPad P50 and it would have 4GB of VRAM. Although, scalpers keep selling them for too much smh. Especially, the 4GB M1000M and M2000M ones.
The A500 is a cut-down mobile 3050... I guess it's for tablets or ultralights. Still, I can't fathom why anyone would look at a 3050 and say "too much".
The 4GB of VRAM on this thing are going to be very limiting in terms of the kinds of things you can do with it. For instance, most image generation requires at least 6GB for any reasonable resolution.
I think it's geared towards edge computing (eg. LattePanda) for rapid prototyping, expand existing embedded systems in the field and sell it as a learning tool to schools.
@@Rotwold I can agree with that, but the problem is that this thing costs $500-$520. I'm sure there are narrow use-cases where that still might make sense -- particularly if the absolute smallest form-factor is a must -- but the mass-market/tinkerer appeal just isn't there. In 99% of cases, there are better options.
While I don't think running a language model or image generation model would be feasible on this device, I think some others application (for example image recognition, audio recognition, etc) could benefit a lot from this accelerator.
How much do you think eGPU enclosures are? Often they are in the $300 USD range. And then for $150 probably getting a video card that has similar output to this. But obviously there is a huge difference in portability.
Good start with this type of tech. Excited to see the next few years with more efficient chips with more VRAM etc. This might not be that great especially for the price, but the idea is solid. We just have to wait for advancements. This will be huge.
The portability is the most appealing aspect of this. It's expensive for the raw performance, but the fact you can fit it in your pocket is what makes it so impressive.
Similar GPD G1 or questionable portability but stronger Asus XG mobile for Asus ROG Flow laptops are IMHO much better choices. There is a quite large performance penalty for using laptop own screen so this device will be really bad for FPS/price while the others have stronger GPUs and the option to connect the external display directly to eGPU for best performance if desired.
there was nothing impressive. I don't have so many pockets for a "pocket PC", "pocket keybord", "pocket mouse", "pocket power brick". And on top of that a "pocket GPU"? there are such things as laptops on this planet.
as others said, falls out of scope. But the thing is you don't need those tests since AI/ML and many other types of compute are not limited AT ALL by the 18-22Gbps of Thunderbolt. All you need to check the performance in AI use cases for this is to check any other A500 benchmark.
ETA Prime the gold standard of RU-vid! Tells you specs, Tells you price, Tells you WHERE TO BUY! I hate it when other RU-vidrs DONT EVEN TELL YOU HOW MUCH OR WHERE TO BUY! Sounds simple right? When I make a video I ALWAYS try to remember this!
Unfortunately it is too small for Ai use, I would consider minimum 12gb to be really useful. Shame on AMD and Intel for lagging behind and allowing Nvidia to monopolize Ai market.
for now the AI part of the pocket AI seems like it's nothing more than a buzzword. there is no world where 4gb vram is suitable for anything related to AI and especially training of AI
Very impressive size, tragic about the cost and the significantly lower performance than the 780M. If this was $150 USD it would be much more appealing than $450. I would really really like to see USB C eGPU adapters reduced to this size
The best you could see with this concept would be 100W limited GPU for the PD charging. So the 4080 doesn’t make much sense. I’d like to see a desktop 4060 in a 100x100x50mm form factor with 100W PD power in, oculink AND TB4 input, and a miniDP out. Oculink is good for 60Gbps and has a pretty big performance delta over thunderbolt.
Honestly the best use case for this might be something like someone who does a lot of video editing on location. They can significantly speed up rendering and AI enhancement without needing a 15+ inch laptop. Back in August I was in Japan and needed to upscale some video I had shot, and it took practically forever in Topaz Video AI on the iGPU (I have an eGPU at home, but it’s impractical to take overseas). There’s also music visualization software that supports GPU acceleration, so between that and Ultimate Vocal Remover, DJs might get some use out of this.
It would be awesome if you could drop this in as an accelerator to enable RT/DLSS with any GPU and have the GPU dedicated to rasterization operations while the pocket AI does all of the RT/DLSS calculations.
@pretentious_a_ness which is why I said it WOULD be awesome IF you could...but also just because something is the status quo doesn't mean it can't change. People made using a drop in Nvidia GPU as a physx accelerator a real thing years ago....who's to say someone can't make something that injects into the pipeline, identifies the instructions requiring point-trace math, and "hides" it from your GPU and redirects it to the pocket AI....obviously that's still an oversimplification for a very complex (and maybe impossible) task but everything starts with an idea and a demand.
4GB of vram at GDDR6 is a bit slow for $500 USD. The RTX 3060 12GB vram sells for $250 MSRP right now. Through that into a dedicated eGPU for an extra hundred bucks. Just being honest. You can't do much with 4GB of vram anymore - not in 2024.
This is a great concept, I hope the company further develops it. Biggest issue is the 4GB VRAM, should be 8GB at the least with a 16gb option to get those 13B parameter LLM models working. There should also be a fan that can be switched on by the user whenever noise is not an issue. The company should also look into supporting the hundreds of millions of old Intel Macs out there with Thunderbolt 2/3, this would give them a new lease on life. Glad this product exists!
All things considered, it'd be cheaper to just produce a singular board that has the processor directly wired to the USB4 IC. There's no reason to add modularity to this type of product when there's really no parts on the market other than what they're already producing themselves as an AIB partner. A singular board would also cut down on manufacturing, points of failure, and QC testing, especially when this kind of product is at a much smaller business scale than you've probably considered. Also, MXM has been long dead, the modern alternative Dell introduced is a proprietary pinout, and Framework's re-pinning of these connectors probably won't see widespread adoption considering the history of modular mobile-spec components. It just doesn't make sense for this product to be more than just a singular board.
The jump in performance also come from the fact that the GPU and CPU are sharing the same TDP. But if you use that external GPU, the whole laptop TDP go to the CPU and the GPU use its own TDP.
There is a huge market of people who want these type of portable external GPU. People go on the extreme ends and spend lot of money to somehow make big cards work on their device even of they need to sacrifice a wifi port for that same. So more companies should invest and do RnD in this field it surely is a game changer.
For AI applications 4gb is a bit low, it has one major strength though and that is its size. There is a niche market for mini PCs where size and power restrictions are at a premium. For example i could see this as part of a truck drivers setup or in a tiny home.
not possible, there's no interface other than Thunderbolt which is rare in older machines. And even on the rare ones which do have Thunderbolt, it will be severely limited by an x2 PCIE connection. If you want, you can make your own egpu over m.2 for computers where you can forfeit that, or even go mini-PCIE, but those are extra-exotic, inconvenient setups
Use "integer Scaler" and be happy 😅 Low res gaming on HD display. 720p is minimum to game on a 4k screen. 360p also works but 360p is recommended for 2d games only scale 360p to 4k.
What are they going to accelerate with only 4GB of VRAM? SD considers 8GB low and LlamaGPT requires 6GB - 42GB of VRAM depending on the model you are using.
on that size it's actually great. The closest eGPU to this in size was a much larger Lenovo Dock from like 5y ago, and that sported a 1050 Ti MAX-Q, while the smallest thunderbolt eGPU enclosures you can still get today are as large as a mini ITX case. Yeah, you can do your own m.2-based eGPU, or you can grab an outdated Gigabyte mini box but you will still be severely outdated and increasing the size by about 300%. I am of course excluding some stuff like Dell's or Asus's proprietary GPU docks, because they aren't really thunderbolt, and they only work on an expensive or outdated subset of machines)
I wonder if at some point they will put one of these into a handheld device. Since is so tiny and low power maybe they can make it fit into something like the steam deck, that would be really nice. I hope we are getting closer to dedicated graphics on a handheld.
Yeah . . . You want it integrated with the CPU. Which is why the Switch has an ARM. Nobody is licensing Nvidia to make x86 cores. I'd love to see maybe a software translation layer for x86 code in Proton for example, and maybe some kind of legal hardware acceleration for it in RISC or ARM, but who knows 🤷♂️
As far as I know, that's kinda the entire point of an APU. You're putting the CPU and GPU on the same wafer. The only limitation to how powerful the CPU and GPU combo can be is thermal concerns and production yields and it's typically more space and power efficient than a dedicated GPU.
No. It's a cut down (fewer cores) and power limited (much lower clock speed) 3050 and the 3050 is already slower than the 2060. It's maybe a 1650 Super. Before taking the additional 25% performance hit of looping the video back through TB3/USB4.
@@lewzealand4717 Thanks for the correction! For a graphics card to still have this performance @25 watts, the only other option is a 1030, so look at the relative performance. For that wattage of a GPU, the only thing close is a 750ti.
@@dj4monie because you can play video games in a VM if you have a native platform that doesn't support it? And since it is a workstation card, even shit nvidia will allow it to work with passthrough?
I am just amazed that nowadays integrated graphics even can run some triple A games, I still remember the days when integrated graphics pretty much meant that you can't run anything other than a browser and maybe Minecraft along with some simple 2D games
Pocket AI: $429 MSRP 4GB VRAM 25W TGP 6.54 TFLOPS Thunderbolt 3.0 Connectivity No Expansion Ports Can be Powered by PD3.0+ (40W and up) USB Battery Banks GPD G1: $714 MSRP 8GB VRAM 75-120W TGP 21.54 TFLOPS Three - USB 3.0A, One - SD Card Slot, Two - Displayport, One - HDMI Out, USB4 + Oculink Connectivity, Can Charge Devices with internal power supply. Oculink Support (Faster than Thunderbolt 3 in Gaming performance 15-30% real world) Needs AC Adapter plugged into wall In completely different categories really.
Imagine if the switch had one of these built into the dock. That would be so cool it could still be battery efficient for the lower res handheld display using an integrated gpu and then get actual 60fps 4K likely with the chip built into the dock. They could even sell the two separate or have a dock without gpu leaving an upgrade path. Heck they could probably make such a dock compatible with the current switch too. However that would require admitting that their games don’t run at consistent frame rates at only 1080p lol
The fact that the pocket AI has no video out would make it useless for a scenario like that. If you want to dock the SteamDeck to a powerful GPU and then stream that output to a TV, consider a full fledged GPU with an EGPU enclosure.
Y'all need to stop saying that anything under 60 is not playable. Over 40fps is playable, over 30 can be a bit laggy depending on the game but is still playable.
I dunno bru I just rebought a 360 for the fun of it and 1080. 30 is painful says 60hz so let's even say it's pumping all 60fps is horrid 😅 still the best system ever but it's so so hard after soon many years at 144 then 165 😅 let's face it this just didn't hold true
Between 25 and 30 I would still call playable. Hell, lots of console games capped at 30 because they couldn't hit 60. Above 60 is nice but most people don't have a hugh refresh rate monitor and this won't ever experience the benefits.
I can play 30fps on a switch all day and not notice, on the Steam deck a smooth 40fps doesn't feel any different to me than 60, but on desktop it feels like I always need a minimum of 60. I've had a 144hz monitor before and didn't care about it, I still think 1080p 60fps is high tech. But I grew up gaming on a 2009 iMac with Windows XP installed and I use Chinese mini PC's to this day, I just don't care about the kind of games that need extremely powerful hardware. Some people get motion sickness with the wrong amount of Hz, some people swear 60fps is garbage and 120fps should be minimum, it's really whatever works for you personally but it's not the righteous battle that people make it out to be. Internet folks are a special breed...
As a kid with a very slow machine most games ran around 10 fps (probably less) and that's all I knew until I visited a friend one day and saw the same games at 30-60 fps. It was such a stark difference in experience, as if it were a different game. Then again, I spent hours playing at 10 fps, proving that it was very much playable.
@@KingmanKingman-nb4tn my guy, as he said, he had several unexpected results throughout the thing. it’s worth it just to see anyhow. he has hooked up eGPUs on less.
windows 10 or 11 now has a way to do this integrated on the OS. Before, you could chose it depending on Nvidia/AMD driver support using their buggy user interfaces. Windows seems to do a better, more consistent job. I think even Linux handles this fine these days
With 4GB of VRAM, it is really not going to be much use for AI training, and larger models won't fit into memory for inference. It sounds like a good idea but it will only be useful for people with very specific requirements.
@@ghardware_3034 Most inference LLM's like Llama2 are about 6gb with 7b parameters and 4bit quantization (not very good results) and for 13b parameters (better results) you need at least 12. For Image generation, SD1.5 weighs between 2.5 and 4gb, but even with 2.5gb models, you don't have a lot of space for upscaling or controlNet.
Hope it's not aimed for people who want to do inference, it wouldn't even run LLM, TTS, SD .... this is useless need at the very least 8gb of vram @@lequack6373
It's crazy that I haven't heard about this. Thank you for bringing it to my attention. I've been watching your videos forever but just realized I wasn't subscribed. I fixed that
What this is stunning, I want this thing from future version plus the legion Go, thanks man a lot of time seeing your channel about egpus and video games, great channel
4gb vram is not enough for loading LLM's, and is possible to load SD1.5 models, but they have to be run with some tweaks. They should offer at least 8gb and ideally 12 or 16.
That's a good point, I hadn't thought of the vram limitation. I guess there are ML tasks that aren't so memory intensive but it's a niche within a niche.
this thing could actually be very interesting for a handheld pc with usb4 support or things like that, even the price is not shockingly high, ofc a real egpu with enclosure would give you much higher fps in games, but those cost also much more when you need to buy the enclosure
This first model is likely going to be a test bed for a potential 2nd model. And judging by these comments they might think the demand isn't there when it actually is, they just fell short.
