Resizing Images - Computerphile 

"So there's a 3x3 image. It's- y'know, 's- high quality."

Bilinear sampling/scaling is used for almost everything[1] in modern real-time 3D graphics (3D games, etc.) because there are dedicated hardware parts for that operation in every GPU, and as such it's dirt cheap to use. Some games in some situations use nearest neighbour, usually for style, like Minecraft (blocky textures stay blocky however close you are). Back in the early-to-mid '90s it was usually nearest neighbour everywhere, because there was no GPU to help out, and CPU couldn't afford to waste cycles on interpolation math for every single screen pixel (one read, one write VS one read, three pairs of subtraction, addition and multiplication, and then write). Early 3D home consoles, like Playstation 1. did the same. The first Unreal game had an interesting approach (in its software rendering mode) - to use dithering instead of interpolation. In that way, no intermediate (interpolated) colours needed to be calculated, but the exact same source colours (like for nearest neighbour) were "shuffled" around to create an ilusion of gradient. So yeah, those were my 5 cents, maybe someone will find it interesting. (1 - trilinear and anisotropic just build on top of the same basic idea)

"If you're a pixel-artist, doing... uh, you know... pixel-art..."

I just love this guy :). Brilliant at explaining these things. He makes a lot of stuff sound real easy, and in this case it really is, but on myself or by the usual professor it would definitely take longer to understand. Or maybe I just have a different kind of attention in class than while watching RU-vid. And the graphics always are on point as well. Not too much info to distract, but right enough to consume the information easier.

Damn I get so excited when Dr. Mike Pound is in the thumbnail. XD

Explaining bilinear and bicubic with the side view was very illustrative and conceptually helpful, thank you!

A 9x9 parker square

Its 'simple' things like these we take for granted but when looked at closer it makes you realize how absurdly fast an average computer is these days. Being able to do this at a massive scale many times per second.

I think of this kind of stuff every time I see a cop-drama where they have some low resolution image and then magically "blow it up" to get a license plate number or something else that was clearly not visible before. That's not how this works, television!

Why couldn't I have had teachers with such clarity when I was going through school? On some of these videos I gain more perspective in 10 minutes than I did with entire lectures in class. Keep up the great work, this channel is awesome!

"3x3 image. It's you know, high quality" lmfao xD

Mike is my favorite Computerphile member, I just love his videos

Really like this guy, he explains it so well and it's a joy to watch him talk about this stuff!

Mike is just phenomenal at explaining how things work. What a brilliant guy.

more with Mike please! As a student of computer science I can tell that he is way more interesting to listen to than many of my professors!

More Mike videos, please. This guy has so good explanations and the way of talking, just wow. Good.

I love this channel. I know about computers, and it confirms things I thought or gives me the small and fun details that you may not learn in uni.

Dr. Mike Pound. I really enjoy when i listen to this guy. It is like he can explain anything to you. He was born to teach!.

Thank you computerphile. I know this is an old video but this video helped me to understand GIS interpolations.

deep learning ai: "Hold my beer"

Awww. I hope bicubic is coming soon! That looks really interesting! This whole video was really fascinating, to be honest. I had always just assumed it worked in that way Sean Riley described it at the beginning: that way that's similar to but not quite the same as nearest neighbour.

This explanation is sooo much better than the one in the book I studied from! thanks!

The way he explained was outstanding!

Back in the day I did interpolation of wind fields on a map using a sparse set of weather station data. One of my algorithms was inverse-square weighted. I could use a selected set of weather stations to interpolate a particular grid point - even ones far away from the point. The inverse square effect made far ones negligible and heavily favoured near ones.

"But that's for another video" Gah! Again you tease us with a cliffhanger!

This man needs more books on his shelves.

New to computerphile but I must say that this presenter is fantastic!

The animation at 7:49 was a brilliant clarification of what he was trying to get at.

Gee wonder why this has shown up in my feed today 😂

Literally watch the videos for Dr. Mike

It actually helps A LOT to think of pixels not as points in the picture but as areas.

Another great video! It would be helpful to see some animations to show how each sampling type changes the look of a 3x3 array into the output array, just for intuition

Since you mentioned zooming out, can you do a video on anti-aliasing ?

That is awesome, I was googling image resizing algorithms a few hours ago.

Do add this to the Computer Vision series. :) I see a lot more videos on CV but haven't been added to the playlist. Thankyou for making them. ^ ^

Explanation is very concise and simple.

The elements of statistical learning in the background, nice

After seeing this, I feel like I could actually implement it. Great explanation.

He's so good at explaining, thanks!

I understand the "basics" in the videos Dr Mike Pound is starring but the math itself I have no clue whatsoever. I wish my math teacher in highschool was as talented in the art of teaching. I find "math" intriguing but I got lost somewhere along the road some 20 years ago. As usual A+ Dr Mike Pound video :D

You're just awesome explaining image's stuff :) thank you.

A video about perlin noise would make my day!

Love ow you explain IT. Great stuff!

Thanks for the video. Awesome explanation.

Nearest Neighbor is interesting since, in a sense, it allows for lossless scaling. If your scale factor is an integer, if you upscale, you can always downscale later on and return to your original image without any loss. I’ve found this to be a useful method for reducing analog noise in SNES captures since the noise gets averaged out (i.e. capturing at a 2x scale and transcoding losslessly to a 1x scale, allowing for arbitrary Nearest Neighbor scaling later on during post-processing). Nearest Neighbor gets most of its usefulness when paired with an integer scale factor.

Can't wait to see he video on the other interpolation methods

Excellent video, very interesting!

That cliffhanger. Love these videos. Would be cool if you could do a video on neural net upscaling algorithms like NNEDI3 and Waifu2x also.

Great video, as most of them. You know what would be funny to do? The "paper change" moment that happens in some of the Numberphile videos. =D

This helped me understand a Computer vision concept. Thanks!

I'd love to see an explanation of the fractal scaling process that was used in the 90s. Lizard Tech owned the technology at one point. It was used to take early digicam images to post sizes or larger.

this is a bit too simple for me since i already knew this but i am excited for the bicubic interpolation video. i'll definitely watch that one

This is about the only guy I en joy watching in this series.

Thank you!! I have always wondered about this

Of course, there are plenty more methods for resampling an image: "Pixel Mixing" and "Box Filters" are a few of those. My favorite is Pixel Mixing though, but it seems that very few programs use it. Also, keep in mind that when you resize something, the program *should* (but too often will not) have converted the image to a linear RGB colorspace first, to do the actual math of the resizing, and then convert the image back to it's original colorspace. For most images tossed around on the Internet, the standard is to store them in the sRGB colorspace, which is not a linear colorspace.

6:15 It seems that you could simply take a weighted average of all 4 points. You can calculate the distances with the Pythagorean theorem and scale them down proportionally so that they add up to 100%, and use the resulting numbers as weights.

Ha - finally someone could explain that to me so that I understand it. Thank you! Have you done a video on Photoshop bending (screen, multiply, etc) - can you think of doing one? I like the way you explain it.

Very nicely explained

I'm excited to learn how resizing smaller works

Great explaination

Really well explained :D thanks!

I'd love to learn more about image scaling done by tools like Waifu2x that use things like "Deep Convolutional Neural Networks". Really, the images that you can get out of tools like these are AMAZING, especially when it comes to lineart and/or digitally produced art. I believe this particular tool was designed to upscale comic book pages.

Usually when i want to downsize a photograph to release online the way i get the sharpest results with the least amount of artifacts (aliasing and moiré) is to first determine what my final resolution should be, say it's 1600 pixels wide, and the image is currently 5120px wide. What i do is i first upscale it to 6400px wide (which is 1600x4) using bicubic, then i usually blur the image just a tad using gaussian blur (with a strength of maybe 2px) then i usually sharpen the image a slight bit using unsharp mask and a factor of maybe 1.4. What this does is it reduces the amount of moiré in the final image, the blur and sharpening needs to be adjusted for each image depending on the content. Then finally i do a bilinear resize from 6400px to 1600px and it is the sharpest results i've ever gotten with the least amount of artifacts. I'm not really sure exactly why it produces a lot better results than from resizing directly from the original size but it does.

Really nice explanation, but I would love to see the graphical result of these re-sizing methods

Deep learing algorithms like waifu2x have a lot of potential for upscaling images. They are also a way to add real "new" information to images when upscaling. And for video there is Superresolution of course.

Programmers who have implemented Perlin Noise are VERY familiar with pixel interpolation. :D "smoothstep" interpolation is preferred over linear most times for Perlin.

I'd like to see a video on DCCI (Directional Cubic Convolution Interpolation). It's a great but very expensive algorithm, which is "easy in principle but hard to actually build efficiently".

That cliffhanger ...

LOL the thumbnail, "When you start feeling it"

I hope Bicubic also brings up the other methods of which bicubic is just a special case of. And then there's the spline method. And then the lineart methods like hqx and such. They try to only use colors from the original.

what about the upscaling done in emulators, like Super Eagle 2, SAI...

Here are your court summons.

The first book at the back is Elements of Statistical learning. I wish someone makes lectures for that.

thumbnail is just legendary

For linear interpolation between two points, the actual formula is: pixel1 + (pixel2 - pixel1) * weight

This guy is my favourite

In my image editor (RealWorld Paint) I have nearest pixel, linear interpolation and cubic interpolation.

very good explication

The most important part he didn't explain, how to scale down images without creating blockiness or moiré.

This is exciting topic.

Mike is simply the best =)

I really like his watch

I have often rotated digital photo's of LP covers to get nicely aligned pix. Now I think it would be better to leave them sloppily aligned, but more digitally accurate.

What about how image blurring works? Like when one uses Gaussian Blur in something like Photoshop or Paint.NET

Perfect! Thanks!

ahh, so close to covering bilinear vs trilinear filtering that is the choice almost every video game offers yet I never understood why.

awesome !!

Could you leave a link showing what the scaling looks like for the differing techniques?

so good

British Toby McGuire sure knows a lot about computers.

I was kind of disappointed that you didn't show an example of how a scaled image with bilinear (and bicubic) interpolation would look, preferably the same image that was used as an example of nearest-neighbor. Other than that, great video as always!

Could you use deep learning to intelligently upscale and interpolate images, alongside changing exposure and other settings?

This guy is the fastest thinking person I have ever heard.

A short TL;DR: PointResize() -> i.e nearest neighbor, crappy BilinearResize() -> Better, but blurry BicubicResize() -> Sharper, but with ringing LanczosResize() -> Slightly less sharper than bicubic, but with way less ringing BlackmanResize() -> Same as Lanczos but with less ringing Spline64Resize() -> Sharp and with little ringing (slightly prone to aliasing) Nowadays what it's used is something like NEEDI3 (Neural Network Edge Directed Interpolation 3) which uses a Neural Network to upscale images rather than applying normal resizing kernels. Of course there are other resizing kernels like GaussResize(), SincResize() etc but they're for different use.

can you do histogram of oriented gradient next

Actually, if you're a pixel artist its a bad thing to use that form of nearest neighbor because you want to keep pixel aspect ratio. Good pixel artists will use what Sean says, where 1 pixel becomes 4, 9, 16, etc. I think the more practical form of nearest neighbor (and better illustration) is where 1 pixel becomes the center of a larger grid of pixels depending on your scaling ratio.

the way he explained the process, would it matter if i scaled a picture vs turning it by 90°, then scaling then turning it back? would a merging of the two follow a better result? edit: yes paint.net has some difference between the turned bilinear scaled images

How about an algorithm that uses linear interpolation if the 2 original pixels have a similar color, and nearest neighbour if the colors are very different? So gradients would stay nice, while edges would be preserved.

thankyou,..this is the real thing, ..not some shitty tutorial about how to type "import cv2"

I wonder how would it in nearest neighbor if the new pixel is exactly in the middle. Basically happens when scaling ratio is an even number.