Video Game & Complex Bokeh Blurs - Computerphile 

Fun fact: original 1998 Half-Life was going to have an effect where an enemy can hit you so hard your glasses fall off and your vision is blurred, but it was rejected because it was impossible to do at the time.

Outstanding explanation!!! My son was home last weekend trying to explain this to me and I just couldn't picture it... (he works for Ubisoft) Your explanation was superb sir, thank you!!! :-)

0:35 "Gaussian blur, the thing we focused on..." hhehhehheh

I'm from an audio background and the moment he drew the overlaid sine waves it immediately looked to me like sine sound waves of different frequencies overlaid to create square waves. That made me immediately understand what he's talking about.

I love how Mike always is on the defensive with the symmetry of his drawings :D

Doom 2016 does a fast real-time Bokeh where the half-resolution framebuffer is blurred with a circular average (tapped fragments in a circular pattern, no Gaussian used) and then a second blur is performed where the colours are blended with a colour "max" operation (brightest RGB components kept).

Thank you for posting Dr Pound on Valentine's day. In my array of favourites he occupies index 0.

*Game Devs:* **This Video** *Gamers:* **Turns Depth of Field off**

I did this at my first job, after separating it into two cheap draw calls, code sped up 80% lol I was so happy

I see mike. I click like.

stop everything. best host of computerphile is here. i could listen to his explainations the whole day

Real game developers who are supposed to know what they are doing: *furiously taking notes*

Man, it's enjoyable just to hear an academic not be dismissive of game design.

It has surprisingly much resemblance with a drill that can drill square holes. Rotation is an equivalent of a sinus wave - and if you combine right set of waves (in Fourier series), you can get virtually any shape. Consequently if you then translate sinus to rotation, you can drill a hole of any shape - the only limit is a resistance and strength of materials involved.

My brain melted when you showed the bokeh blur of the stars. It went into an infinite loop of trying to focus.

Ah, yes. The Frostbite Engine. Known for its amazing sports games.

A lot of people in the comments objecting to motion blur, which I can understand but it's important to realize that the tech developed for motion blur is also used in per-object motion blur which is typically a lot more tame and doesn't get the motion sickness complaint. DoF similarly can be used really well in cutscenes, but can be distracting during gameplay. On slower paced games, though, it can work with gameplay as well (turn based RPGs especially).

There is also a brilliant separable hexagonal blur to simulate specific lenses that essentially does 3 line passes at 120° to each other. And the technique shown here can be used for much more difficult filters, for my thesis I implemented a realtime fullscreen inverse-square blur to simulate global illumination.

To give an idea of the sort of improvements you can get with separable kernels, I optimized an image processing algorithm with that technique that would run at 4.5FPS at VGA resolution. After I was done, I could run it at 21FPS at 720p resolution. This involves some other tweaks like moving sampling point to reduce and weigh at the same time but a lot of the performance improvement comes from kernel optimization.

That is a fascinating subject, I've always been, fascinated by camera effects in video games. You can tell in an instant when a game is using some other method of blur to simulate depth of speed rather than a proper bokeh approximation.

nice real time bokeh reflecting from the whiteboard in the background!

Would love to see more videos of Mike talking about the implementation of maths in video game development. Would be nice if he/you were able to do a collab with Digital Foundry.

5:14 the accuracy of that hand drawn Gauß curve 😍

I will never be as awesome as this guy

It's always fun to know how much goes into games and people don't even realize it

Digital Foundry collab, when?

Very interesting, I already work on this, to add something on the explaination, what we usually do, is changing the size of the blur size based on the distance to the focal plan by reading the Z-Buffer, what a real lens do. Or the separation between foreground and background is more background, focal region, foreground and blur the front & back separately to not mix color of back & front blury region. And in game we directly work with HDR/Raw buffer where the values are not clamped to 0-1 (aka 0-255) but work with much precise buffer 10 or 16 bits per channel RGBA.

It's just in the game

I think that using the same image to compare both codes is really simple and clever!

I saw a video about the trigonometric sinc function, and it looks very similar to the waveforms that cancel each other to create an approximation of the rectangular function. I guess this is one of the reasons why sin(x) / x is so important in signal processing

Those are the most beautiful freehand Gaussians I've ever seen.

Multi-camera phones must be doing some version of this to let you play with the bokeh.

i wish i were british just so i could "have a go" at things

Mike Pound ❤️ My favourite lecturer

One has to bring e^(-x^2) to the complex plane to reach the Gaussian Integral of sqrt(Pi). The area under the bokeh curve is Pi, from that derivation.

Dr. Mike works with making DSP realtime efficient. I never knew he was truly a man of culture.

Apart from video games, it's also a topic for rendering photorealistic CGI. One of the best plugins to do this in post (on a 2D image) is called Frischluft. If you have a 3D scene and a render engine you can go way crazier. Disney for example scanned real cine lenses to reproduce optical results (including filters and lens effects like adding a split diopter).

Thank you! I hope for much more of this type of video. Image processing is so important

The donut shaped bokeh is what you get when you use a mirror telelens. It uses a combination of concave mirrors to do the magnification instead of lenses.

Love this!!! Please make more about Game Rendering Techniques!!!

Props for the little lego man and his car.

beautiful bespoke bokeh blur depth of field

Using a whiteboard marker on paper? ABSOLUTE MADLAD

MORE MIKE POUND also what model lenovo is that?

Great video! Please do more of these game related videos!

Like adding sine waves together, that converge into a square wave?

EA Tiburon in Maitland, Florida has been producing Madden NFL since 1989. One of Orlando's hidden gems.

I would willingly give this man all of my passwords

I learned imaginary numbers but have completely forgotten what they are and what operations are useful with them. I'd appreciate a refresher video.

Lots of game engines use Bokeh based depth of field. And they will use it more and more. Even Dark Souls had a mod that added this type of blur (you could even choose a Gaussian based one)

I always think that game engine design problems are the best combination of maths and programming that there is and on top of that it has to do in real-time. Even robotics gets shortcuts that games engines don't have the luxury of having.

With talking about imaginary numbers it is important to note: i is not the square root of -1, rather i squared is -1. You would think that's equivalent, but with the square root thing you can proof -1=1 which is not... Ideal

Love this guy

Thanks, Mikey! Was easy to understand, how it works! I see Mike, I like :D

Great explanation, thanks!

Use bessel functions as basis instead of gaussian. In the games i think they would pre render the background if you already have a priori knowledge and not waste the GPU

fantastic. it amazes me how many calculations are happening every second when I play a game on my computer. I play competitively so I generally turn off all the fancy graphics and try to get 200 -300 frames per second.

Mike Pound!!!!! What a guy

Blew my mind... as always!

This is so awesome.

Wow! More please!

Nice explanation.

You made really interesting videos about convolution/blurring. But it would be nice if you could make videos about deconvolution/unblurring as it is sometimes used in astrophotography and microscopy.

What's interesting to me is then in a telescope when you have a star in perfect focus, even then there is that same sort of 2d wave-disk (like your 1 or 2 part approximation) that I'm thinking is connected to that same fourier analysis kind of thing that you're doing here- it's called a diffraction pattern. Interesting. It's similar to how Pi ends up popping out of seemingly nowhere in some math problems...

yes! dr. mike pound!!

I want to see someone try to do an organic bokeh. I had an epiphany once where I realized that the “frozen sparks” shapes surrounding bright point sources at night must be this kind of thing, but with the color part of your eye (your aperture) not perfectly stopping light in a circular shape (and maybe there’s some diffraction going on, as well). Gets harder to calculate with our spherically arranged photoreceptors, and human irises are as unique as fingerprints. I imagine it’s also more pronounced for people with blue eyes.

I love this guy!

I love this prof

As interesting as this is, I can't stand depth of field or motion blur in games and wish more games would make it easier to turn off. (At least outside of cut-scenes.)

That was really interesting!

We need a Pound Grid vs Parker Square showdown

just wait for ray tracing, when we can render our games with actual light, and then do actual lens physics with them.

The only thing missing is a real life comparison of the number of operation / increase in time efficiency between the twin 1D vs the 2D disc convolution methods.

In Computer Vision, you often run a Gaussian blur pass on an image to reduce the amount of data that you might pass into your algorithm (ex: Object Recognition). Seeing this video, I wonder if a Bokeh blur would be better or worse in most CV tasks.

Bokeh is interesting to a Japanese in two ways: why there is h at the end (we don't romanise as such, it's just boke. Maybe to make it look more "oriental"?), and how the word that just means blur or haze in Japanese became the word for specifically this type of lens blur.

Great explanation :)

And then here I am thinking that the only decent implementation of dof/bokeh effect in a game I've encountered is in the witcher 3, and even then I often disable it because it sometimes is too slow to refocus the image when changing where you're looking. Still, it is very interesting and impressive how they achieve the affect in such a short amount of time of 16.67ms (at 60fps) I have to admit that game engine design still blows my mind even though I'm learning programming, and the more I learn about it, the more astonishing it becomes, and the more it starts to feel like wizardry and magic.

This is neat, but I feel like compute shaders have made these sorts of techniques outdated. All of this separable stuff is ultimately to reduce the number of samples you have to take from the source texture for each pixel that you're calculating. But with a compute shader, you can simply have each thread sample the texture, and then share that sample with other threads in the same group using groupshared memory. Other features such as atomics and unordered access view textures open up even more doors.

With stuff like this, if it's done properly, you don't even notice, because it just looks normal. You start noticing when it's not done well.

No Bob and Alice this time... :( Lovely blur filter explanation Dr !

This is a really neat solution. I wonder what it would take to get this to run on a GPU and parallelize it like crazy.

3:47 It's not the best kind of correct!?

Waiting for FFTs!

Very nice. Paint FTW

2:42 Thanks for clarifying it's 60FPS!

Fantastic explanation, but rather heavy on the Frostbyte engine / Madden plugging. Given the uncanny timing of both this and Stephen Colbert's new video having a heavy focus on the game, I'm a little dubious that there aren't some "sponsored content" labels missing in this or that video.

foci blur is the best

as a games dev i never think of adding motion blur as its one of the worse effects and is just there to hide bad performance(while causing some) but nice video though

reshade did this ages ago, also in real-time

Please FourierTransformation ! Cool video by the way 👍🏻

Bokehlicious!

Octopath Traveler has really nice Bokeh, by the way.

Thank you! I’ve always wondered how to they achieve cinematic depth of field (because Gaussian blur always looked awful). Is this the same thechnique used in ray tracing renderers?(blender, cinema 4s) also in portrait mode on smartphones

I don't know/understand the difference between adding two complex filtered image together or adding four real filtered image together. 9:20 is adding real part and imaginary part same as addition done on two real value matrix?

How would a blur using a depth pass work? In the video they showed the depth pass as a mask, but I don't fully get how you would weight the convolution to make it less pronounced in a range of gray values from the depth pass

Actually, plenty of photogs are chasing after lenses with apodisation elements, which produce bokeh that is rather similar to gaussian blur

Depth of field is the first thing I turn off in every game. It seems like waste to render beautiful image just to blur it right before it goes to screen for no reason.

I think Digital Foundry and Computerphile should hook up.

Why would somebody want a blurred image in the first place? I thought background in games is blurred to save gpu (it renders it in some "lazy" way), not that it has to render it correctly and then waste additional resources to blur it...

Dale Cooper , is that you?