I supercharged my Nintendo 64 with AI | Nintendrew 

Hey all! 👋 Just wanted to leave a comment to make a couple corrections/clarifications: Firstly, as many have pointed out, I mentioned the N64 outputting over composite with "a maximum resolution of 480p", but that's not quite accurate. While the footage in this video was captured at that resolution, N64 games were more typically presented in 240p or 288p, with the highest fidelity titles running at a max of 480i (interlaced). Secondly, when speculating about the future of video super sampling, I suggested that someone could theoretically use the Nvidia DLSS SDK to upscale video signals in realtime. In reality, DLSS uses additional data provided internally on a game-by-game basis to achieve its end result and could not be used as-is to upscale generic footage. However, Nvidia has just recently announced their RTX Video Super Resolution tech which is designed to do exactly that! Incidentally, 2kliksphillip has a new video on this tech, too, which I would highly recommend watching if you're interested in the topic. As always, thanks for watching! 😄

It's worth noting that the N64 isn't actually 480p. It's usually 240p and has a maximum resolution of 480i.

7:47 To be honest I never grew up with the n64 but when I got one in late 2016, I found something uniquely appealing about the jagged edges on the environments and especially the characters of most games and the low resolution, blurry picture quality. I still can’t explain why I like that

I know people love this but it just looks like melted crayons to me. then again, I need glasses so I could use more pixels in my games.

Pretty neat, but I'd definitely take sharp, crisp pixels, or CRT simulation, from a scaler like the RetroTink over this. In its current state it really looks like a smeary mess, and can be quite unstable at times in motion. It does certainly have its impressive moments, but overall I would not want to play on this. Personally I play on a real CRT whenever possible, and firmly believe this to be the best way of viewing these low-res signals, but I understand that won't always be practical for everyone, especially as CRTs get scarcer over the decades. I'm hoping highly-accurate CRT emulation will be an accessible option down the road (or, better yet, my unrealistic dream of someone manufacturing new CRTs with retro gaming in mind).

I swear, every time you use the Game Boy Camera music it puts me in an amazing mood.

amazing video..however, the footage from the upscaled version looks like someone dumped a bunch of Vaseline and grease on the n64 footage

I would definitely be interested in some kind of real-time upscaling device like that. Being able to do re-stylization would be an amazing option as well; being able to provide an image as a style input which is then automatically applied to the video output. That would be incredible on older and monochromatic consoles like the Gameboy.

In all actuality, the fanmade pc ports of these older titles that have been popping up recently are the best answer. Natively upscales to whatever your pc can handle, and same for the aspect ratio. Sure it's not the most accessible option, but it's simultaneously the most cost efficient and gives you the best results. Not to mention that resolution and aspect ratio aren't the limit when it comes to making the games prettier, plus it allows us to push these versions of the games to other consoles. There's already a proper switch port for the fanmade pc version of OOT for example. Rebuilding the games from the ground up is likely going to be the future for these old titles.

For me personally, I'll just go for the HDMI adapter option (highest quality possible) and not get into supercharging it. That way I can also use the same HDMI adapter for my AV Famicom and Super Famicom systems too (if it works with all AV Nintendo systems anyway). Hopefully I can get one with 4:3 and 16:9 stretch options included, for those 16:9 supported games of course.

There's value in using AI to "upscale" textures. A friend of mine gave me an AI-upscaled version of the PC ports of Dino Crisis and (original) Resident Evil 2, which both looked very crisp. However, I think there's a bigger challenge for the N64, which has always had certain graphic issues when emulating game footage due its source hardware. Either way, it's a cool idea.

i personally use a combination of a rgb modded n64, that passes through a hdmi line doubler (with optional smoother) and it makes the games look quite close to the footage you are showing as well :)

It gets even more impressive when you start using custom-designed models. I’ve been working on a custom model set for the Super Mario Bros Super Show (models available in the Animation Upscale Discord). The results on the animated portion aren’t as good as going back to the source 16mm film would be, but it’s still very impressive. Live action is taking a bit more work since it was shot directly to tape.

1:14 ah yes, my favorite 4 letter word: emulation

Love the video Drew! Great insight into what’s coming in the near future

it's even funnier when you take into account that the emulators themselves already offer the option to increase the internal render resolution

I was theorizing this when I first saw stable diffusion drop a couple years ago. Filter all the games with a prompt to change the art style in real time. Can't wait to be honest

Fascinating stuff! Thank you for painting the future as a bright one for retro gamers!

Wow that looks incredible! Better than any other attempts I've seen over the years to improve the graphics. Would need one powerful af FPGA to pack something like that in a Retrotink or OSSC

What we need is a proper 2.0 HDMI upscaler box with Scart input that can properly handle scanlines. Many of the upscalers we have available introduce some type of issue.

btw the most games on the n64 ran at 256x224 or 320x240 if you downscale your captured footage to the native resolution that game outputs then upscale it will look cleaner and less jaggy.

DLSS uses motion vectors from the game code, unless you also had some sort of high-speed code transport from the N64 to the "DLSS device" it would not be as effective at up-scaling. That said, AMD FSR and CAS algorithms do a pretty good job without motion vectors or AI so who knows what the future might bring. AI will certainly get faster and better at this. You may even get the option to replace all graphics with newly generated realistic ones similar to who AI art tools can take a child's crayon drawing and turn it into a realistic painting.

While it’s one of more harmless uses for AI, it still wouldn’t be that practical. For people that want to use emulation there’s far better texture packs and upscaling options that get the job done, and for people that want to play on the original hardware, I doubt they would want to play through something that is so heavily changing the “original intent” of the graphics. Plus HDMI and upscalers already exist. Interesting I guess, but like a lot of AI things, it feels like they come up with solutions for problems that already have better solutions.

This was very interesting, and the technology could be used to revitalize vintage gameplay! I'd like to experience this myself.

Those games weren't meant to run on those clear image screens. The scan lines and the blurryness makes a huge difference. The more high definition that are implemented on those games, the more ugly they look. Now we can count every pixel and every polygon just by looking at it. In CRT TVs you see less geometric figures in the polygons and less sharpness of the pixels.

the composite video signal and 480p is not why the N64 looks blurry, the N64 uses 3-point bilinear filtering on it's hardware, you'd literally have to solder new hardware for it to be nearest neighbour.

Yes New Video! I hope it's good!

There is a great video called "enhancing photorealistic enhancement" which uses Ai to make GTA footage look like real life. With the progression of AI image models there is the possibility of style transfer. For example you could feed the AI images of the pre-rendered models of the star fox characters and have an AI use them as a base to enhance the N64 footage. Of course, this requires for models with more temporal coherence and we would need much more powerful hardware, but as you said the point of the video was to look at least a decade into the future.

Kliksphillip mentioned. Love to see his very niche style of videos get some spotlight outside those circles. I wonder if it will be possible in the future to hijack the video output directly from the console in this type of manner. Considering there are already console mods for HDMi converters, maybe.

Imagine if emulation could render each frame as a.i upscaling, it would suck but be cool!

Thanks for sharing this!

I actually thought of a idea like this 2 years ago. I said to myself if their was a way for someone to figure out how to get one of these A.I upscalers to work for retro consoles then it would be golden! Imagine being able to have PS2, GAMECUBE, Wii or even PS3 and XBOX360 console games Upscale to a convincing 4K!

I'm personally in the camp that I want to keep the original experience alive. But that's only for video games. For movies I want the highest fidelity so I can understand why people would want games in that way too. I'm sure there are some movie buffs out there that say film projection is the only way to experience a film.

There is literally a $99 device that does this all, and works great with retro consoles. Sold at BestBuy, Amazon, and honestly does what you are trying to do without all this extra work

I'd rather play my N64 with low graphics and not have it look like an ai oil-painting

That's awesome thank you

1:16 It's not 4 letter word, it's a 4 syllable word! :) Great video!

Regardless of OG hardware vs emulation, when AI models fully understand how to remaster a game frame by frame in real time, we could be in store for things like UE5 graphics for 6th, 7th and 8th gen games.

So real time ai upscaling for videos exist on the Nvidia shield and it is pretty good. For retro games, I can't imagine you'll ever get reliably good and stable results without access to a depth buffer, it's a big part of why DLSS works so well and why FSR 1.0 was so bad by comparison

On you go Nintendrew, I have no doubt what you have said is correct for the future - but have to press doubt on you doing it yourself.

A cheap solution for up-scaling video console footage sounds great.

I did buy a plug-in device that upscales my games. These kinds of things aren't new, and advancements are constantly being made to improve them. Just get a RetroTink 5X Pro for more accurate upscaling than the AI did that video

No 480p on the N64, actually. Just 240p or 480i.

DLSS is basically black sorcery, god tech, game changer in every sense of the word. With that said, that's only for modern games designed for higher resolutions in my opinion. For retro games, nothing will beat a good CRT (or complex CRT shader in an OLED panel). That's just me though.

Thing is what you describe as the "original experience" isn't the original experience either. You're playing games specifically made to be displayed on CRT TVs on modern LCD and LED displays.

No! DLSS works completely different! It has a lot more to work with than only an output picture signal. It has 3D information etc.

Clickbait title. I was expecting to see some clever hack or new device. But no, you didn't supercharge your N64. All you did is upscale some footage with topaz ai which has been around for years now.

Easiest nowadays is still use a CRT and if you want to push it get an s-video adapter. Also with a flash cart there are rom hacks that reduce the blurriness.

DLSS is a temporal upscaler. You could not just take it and throw it in an external device or use it at all. You need actual data from the game itself. This is why DLSS is only available in games that implement it. AMD's FSR 1 should work though, but it could be hard to implement in hardware. Emulators already make use of it.

why go through all this trouble, when its cheaper, and easier, to get a CRT, which will always look better...

I like how the N64 need something like this for its low res textures but we see the ps1 being imitated with modern games even today.

I have an M-Classic and I regulary use it with my Switch. It makes a big difference for me considering my TV is 4K and it doesn't upscale a 720p image very well, but a 1080p image looks just fine.

medium textures with clean color is the best for all retrogames!!!

I’d love to see an adapter that decompresses the sound.

To be fair, I don't think it would be possible to use DLSS to do this kind of upscaling, at least not trivially, because DLSS needs more internal data from the game engine, like movement vectors and stuff like that, not just the raw output image. AMD's FSR on the other hand could, I believe. Plus, I think it's open source.

Now combine this with an HD texture pack

Hello Nintendrew! Yeah, I suffer from that blurriness... On a CRT (or those early 4:3 LCDs, like the one I have) with S-Video, the N64 looks good, acceptable. But in modern TVs, I can't stand... What's your actual setup nowadays? Do you plug with those upscalers? Is it HDMI modded?... I think in a near future I may use a modern TV, but I don't want that future right now...

meanwhile me who has played on a pc for years...... looks good bro, totally hd

Nvidia do something similar already with the 2019 shield. It receives a stream from your gaming pc and upscales it to 4k. Im not sure how good it is however.

Take the best CRT from yesterday and take this image (via camera) as comparison and train the ai with this. So de CRT Look how it supposed to look will be burned into the Model and then can be upscaled to 4K

I've played around with the proteus fine tune on Topaz on some of my crappy phone footage. And the artemis

i would just love to use this tech on my DVD collection! 480p to 4k would be incredible.

Man I have feeling that because of those blurry ass pictures and textures gaming through ps1/n64 days from 95-2005 is the reason why I'm wearing glasses 😂 I'd rather use emulator with sharper cleaner images to go easy on my aging eyes

Another killer video bro! This is the future of retro gaming right here. Wow!!

I think this tech would be very good and fun!

The RetroTINK 4K is the best at upgrading graphics to modern TV.

I got 12 CRT's, but when they eventually die will be looking for some kind of solution like this

Yes!

I wish I could do this on my own N64

Real-time isn't likley going to be necessary though. With AI it should be possible to take a rom and remaster it, even with personal request. Maybe a wait for the dozens of hyper focused "minds" to load but...

I love the N64. I run it on a CRT. It bugs out my eyes. I don't remember that happening 25 years ago.

When are you planning to do the Photofast review? You should try it with the EON 64, mClassic and Photofast all connected to see the output.

You didn’t play in 4K so you didn’t supercharge the n64. Also I use topaz and just rendered 12 seconds of footage and that took 35 minutes so best of luck with rendering an hour.

You can get much more consistent and better results RIGHT NOW with original hardware by using high quality video out cables and something like a retrotink 5x or the upcoming 4k edition, and then layering CRT filters over the video. Games from the n64 commonly output 240p and were designed to take advantage of a CRT's display quirks, and while the filters don't capture all of them perfectly, itll look a lot better than the ugly smeared look of consumer AI upscaling tech. In my opinion, AI upscaling is great for use with texture packs (especially when used as a "first pass", and then a human tweaks the final result to remove errors or inconsistencies), but looks absolutely awful for upscaling the raw video output. Selling this as a "supercharged" potential future tech is clickbait when there's better options right now.

You say. Neural networks might play a roll in a few years time. But it definitely will and does already. I have a feeling the next 2 years are gonna change the world as a whole. Leg alone retro gaming. Definitely more excited about the later though lol

Game Boy Camera music

I always use S-video for my N64.

Y'know, we are dangerously close to having AI create entirely new games in whatever style you want. They can already draw art and write music, it's only a matter of time before they can write entire game engines, logic, stories and levels to form new games, even self writing code. At the moment it takes weeks to train an AI just to play a single level of Super Mario, but we're not far off them being able to play games. Once it can do that, it could build new ones. Train it with every good N64 game and have it create its own. Would be interesting to see the results. You might even have AI learn design flaws in creating new tech as cheap as possible, new consoles might be designed by AI, new PC hardware components, everything. AI is going to change the world.

Yeah I think there's something here. With that type of thing.

You should try to turbo charge your Nintendo 64 next time.

What if you connect all these converters and adapters together and see if that improves the quality of the output. Would be interesting to see especially for the wii and switch.

The Eagle filter of the 2020s

Nintendrew It's been a while! 👍🏻

Love the game room and this video rocks! Thanks for sharing!

Retrotink family of products are the answer to your question

I would buy such a device, depending on it’s cost. Have you ever used a Retrotink? I didn’t major in computer science, but would definitely like to get my CompTia A+ certification. As well as build my own gaming PC, which looks really simple to do, based on the RU-vid videos that I’ve watched.

1:14 four-letter word? 😂

Always nice to see another nintendrew video

Nice video!

We're absolutely a long way off from a realtime AI upscaler for retro games. The existence of DLSS doesn't suggest otherwise. A key requirement of DLSS is that the game engine creates motion vectors . In order to create motion vectors accurately, you'd need to have knowledge of the scene's geometry and how it changes between scenes. You can't get that info from the final rendered image. You can try to approximate motion vectors from 2D data but it's imprecise especially at lower frame rates like the 20-30 range that N64 games are rendered in. DLSS also currently requires per-game training based on extremely high resolution renders of the game. All of these reasons contribute to why you can't just enable DLSS for a game in the NVidia control panel. AMD's FSR 1.0 can be enabled as a post-process effect and can be implemented into a scaler today but that's only because it doesn't use temporal information at all. It also wouldn't provide a result anywhere near as good as is shown in this video. The only chance I see for older 3D SD systems to output anything like what was seen in this video is if someone created a replacement GPU for the PS1 with its own VRAM that scales the vertices it gets sent and renders them at a higher resolution. The chances of that are ridiculously slim unless it were FPGA-based.

Honestly looks kinda creepy, reminds me of that AI art which makes sense since its upscaled by AI.

Honestly I'll always prefer using period specific hardware although I do love playing Wii games via an HDMI adapter

Quite a roundabout way just to learn that the N64 is meant to be played on a CRT

This would be ideal for majora and oot.

Now compare it to HD emulation using texture packs

Ah, emulation. My favorite 4 letter word. (1:15)

I wouldn't mind a lower-cost version of the retro tink with an ai or machine-enhanced chip that could make the games less muddy and clearer.

OMFG if there was A Adapter for all My Old School Electronics I would definitely buy this and use it on everything from the Atari 2600 to My Old DVD's from 1997 to 2004 I am using the Retrotink5x Pro for all My Old School Electronics at the moment but if this became A reality this would definitely replace My Retrotink5x Pro.

Amazing discovery! The retro future looks bright 🤩

I hope that nivida makes a adapter for this stuff..