Interesting. When I first heard of Lyttro cameras, I suggested this tech be used with eye-tracking so that a video would always focus on wherever the viewer is looking, thereby mimicking real vision and making the scene more immersive. I still think that would be awesome.
+IceMetalPunk Head mounted ones do. But normal viewing LF displays have been made that are around a metre tall using hundreds of projectors. You can even buy small displays like LEIA 3D, though they call it a holographic display.
carrotSnack Leia is not a light-field tech, it's a 3D tech. It doesn't adjust its focus at all, it just sends a left/right image to each eye. Light-field displays need many images from many angles, not just two. As for the meter-tall light field displays, do you have a link to one? I've only been able to find info about head-mounted ones.
Mitchell Gillespie No no, you're misunderstanding. I'm not talking about focusing the *camera* based on where the *shooter* is looking. I'm talking about refocusing the existing video based on where the *viewer* is looking. As in, refocusing after the film has been taken, such as with light-field tech like this video describes, but choose the focal point based on where the viewer is currently looking with eye-tracking software. Both technologies have been around for some years now, but I've never seen them combined, and think it would be amazing if they were.
At about 1:00 he mentions that he'll talk about the more industrial variations of this technology "for a little bit". I'd be interested in seeing that footage if it exists :)
I wasn't ever really very interested in the light field cameras, seeing them as more of a gimmick, but seeing the tech behind them makes them incredibly interesting! Awesome video!
I want to see this technology being used in very strong macro lenses, where you have to deal with excessive DoF by doing focus stacking. This tech could reduce such a photo down to only one image, and would make things a lot easier.
Very interessting - I hadn't heard of light field cameras before, and now I really want one! Also, probably the best (shortest, most concise, most accurate, most memorable) explanation of why some things are in focus and others are not - thanks for that!
@@gfx2006 I imagine this would have to shift hard-edged boundaries, so you'd have to arbitrarily choose which ones to 'prioritize' would it not? (This could be done "automatically arbitrarily" I suppose.)
I remember hearing about this years ago, I think it was at an MIT research lab. They were actually using a perforated piece of paper like a bunch of pinhole cameras instead of a micro lens, but essentially the same thing. I'm kinda surprised it took this long to come out with a commercial product.
Wow! After weeks of watch and reading, I finally found a helpful Lytro video. I have snapped a single picture yet because I wanted to understand the camera first. Thank you. I have both cameras.
Ahh, that kind of reminds me of the grid mask used in front of color CRTs that allow each electron gun to only affect the subpixels for that specific channel! Basicly the individual holes in that mask act like pinhole cameras, so that the way the thing is angled, the electron gun for the green channel can only reach the green phosphors on the screen!
YES you can. In real life you would need a tiny aperture (like a phone camera) to focus nearly everything, but you loose light. In the lytro software you adjust the effective aperture to a tiny value and acheive total focus without losing light. BUT in the norm of photography people think that big aperture looks better (more blurry foreground and backgrounds).
Lytro have something coming out that apparently captures every light ray in a room and using a VR headset you can look around like you were actually there in photographic quality and can move to look around corners
Finally a nice explanation! Not that I was deeply researching the subject but normally I can get my head around of how stuff works, but this wasn't that intuitive...
Then this image can be showed in a ligth field display, and you can focus the background or first plane with your owm eyes , just like a real live scene
Idea: Attach one of these cameras to a drone that takes pictures of 3D objects and then try to reconstruct them in software, maybe even doing machine learning to teach a program to detect a certain object. You no longer have to worry about the machine itself focusing on the objects since the camera does captures all the data itself.
could u use this to make 3d pictures? Generally speaking you need 2 view points to construct the 3D pic, but this techniques seems to show you only need to vary the focus point to get the distance
How about using this tech together with 360 high-res video for VR experience application? You would basically look at different things inside your vr set and they would come into focus, making the whole thing much more believable. You would need to be able to precisely follow the eye movement within the vr set, but still, seems doable to me.
could you link a large amount of these cameras together and create an image with a wider parallax ? kinda like creating another micro lens, or macro lens.
Sounds like they took the idea behind the complex eye and developed this camera. This is some amazing technology. At first I thought it would just be some clever software gimmick that makes the camera take many photos in a short time but I'm delighted to see I'm wrong. If this technology gets integrated into VR applications, combined with eyeball tracking, depth of view would be much more realistic. A simple example would be Google streetview. The future looks bright.
Wait a second, does this mean that the information in such a picture could be used to create a 3D model of the "depth"? I mean, it seems that the image contains depth information, effectively.
Yes that is possible but the effective dept is a proportion of the baseline which is very narrow. For comparison tv gesture recognition use about 15cm to recover dept at 1-5 meters in the living room.
+antiHUMANDesigns As Bogdan said, it is fairly limited in how much depth you can get from the image. The resolution isn't high enough to do something like that on such a small camera, you'd need a much larger setup for that sort of stuff. One thing I would love to see is a setup where it uses 2 light-field cameras to create a scene. It would allow for far better resolution of the full light-field and can be used to create 3D models of a scene (of what can be seen, as usual. damn 3Dness of the universe) Combined with some software, you could use it as a freeform 3D scanner that doesn't lose any focus as you scan the object in question. No more expensive rigs with motors, just hold it in your hand and get spinning.
Bogdan Manciu Yes, the information would be quite... low quality, I suspect. You'd probably get a rather jagged model, but it could probably be smoothed.
I have purchased a LYTRO, but I cannot find the software to view. The link sends me to Raytrix, but they said they do not support it...any suggestions where I can get a download.
+DFX2KX Same for me in a way. I have a carton of 2000 brand new 80 column punch cards in a sealed box, but I have not seen a working (or non-working) card reader since around 1990. I have been using the cards as notepaper for years after our university dumped all computer cards. When I got to the last box, I decided to keep it in case it became interesting for later generations. My kids will probably throw it in the recycle when I die. Oh well.
11:24 explains why in this current form, this is not the future of photography... yet. For now it's completely closed and proprietary. We need an open standard for this and device agnostic tools (ideally open source ones).
+biohazara The problem, besides the patent issues, is the fact that it is still fairly low-resolution. That is more an issue with physics than anything else. A higher-res system will require either considerably more dense CCDs or a larger CCD. (so, in turn, larger camera) Larger camera will be a lot cheaper, which will then turn people off from it. (besides the really hardcore photographers, who would almost certainly welcome it) The CCD density issue will mean a sharp increase in price since it isn't just that which is increasing in density, you are also going to need to increase the density of the microlens grid to improve its accuracy too. Both of them work together to define the actual effective resolution that can be used for images.
+Kris Johnstone +zaco21 I completely agree this will take time to be something usable in the "real world". However, patents and the lack of open standards make progression even slower, much like 3D printing wasn't a thing until some of the patents expired.
+biohazara It will never be the future of photography. No matter what advances are made they are always going to be splitting a normal sensor up into a lot of tiny ones. That is always going to have terrible low light performance and it's always going to be very low resolution in comparison to a normal camera. I would assume it's going to be incapable of achieving the same FPS as a normal camera too considering how much more information it needs to process. It's impossible for it to ever catch up.
+Justin Koenig To make an HDR I would recommend Luminance HDR and then photoshop/gimp/darktable to tweak it (Luminance HDR can take several types of RAW input!)
I'm sorry to hear that, it works pretty well for me, no exposure problem (you can set them manually though if you can find the values in another software). It even has some great batch processing capabilities if you want to create hundreds of HDRs at once. It is very cpu demanding so I do understand that it may not be for everyone. I would advise finding the right profile/settings at a low resolution and only when you're sure let your computer run overnight to produce the high res of the batch of photos. Also make sure to enable multithreading if your cpu has multiple cores.
So it is the effect of lenticular lens hologramms turned backwards. oO I thought it takes multiple pictures in a short time while moving the ccd back and thus having a picture with different focus layers that you later compile to a single picture.
+Morkvonork You could do it that way but I get the sense that it'd be mathematically very difficult to figure out what was going on and reconstruct the rays. It'd also be quite complicated to move the CCD that quickly without changing its orientation at all. You'd be better off using a lens with a variable geometry to bend the light at a slightly different angle.
+DynoosHD Theoretically the same, but in practice the files are larger. I don't know the exact details of their storage format, but while you'd imagine it's just the same size as a normal image (because it's still a regular CCD behind the microlens array) they can't do as much compression because that would risk affecting the image reconstruction. My lytro images were about 10-15 MB, and the newer illium camera is closer to 100 MB per image.
If you ditch the main lens and build a very large sensor panel, with some tweaking it could function as a 3D camera. You'd need a special monitor to display the result, but you wouldn't need 3D glasses.
Please help.. I don't get why you can't just reconstruct the image in the same way without the micro lenses by just keeping in mind that a certain amount of focus on the aperture will have light coming from specific sub-apertures go to specific places... He said something about how since it's not already focused, you have a lot of background noise or something... Let me know if you do.
I know *everything* on the subject in the video but I wanted to give it a go... Then, all of a sudden, 11:15 !! And I was like "You wish!" X) This whole explanation is purposely confusing and makes a burden of simple concepts which could have been described differently. I recognize that this isn't an "in depth" explanation but in +-12 minutes the whole algorithm can be shown easily and it's not "fairly complicated" but relatively simple if you have used it at least once in your life. :P And if you happen to end up in a "black box" it's because you need to search for "integral imaging" to see some relevant results on the matter and you'll discover that "the black box" is actually public knowledge by more than a century. :)
Graham Smith Any picture you take can be paralaxed like seen in the video by tilting your phone (the "3d" option) and any picture can be refocused afterwards pretty much identical to how it looks in this video. You can also upload it to an HTC website page and send that to a friend so they can refocus online. It's pretty amazing but I don't know if it's the same.
+Niels Schellekens I've just had a quick look to see if I could find how the M8 refocusing works, I didn't find a detailed description but I think my first reply was basically right. It seems the two cameras have quite different numbers of pixels. There's a main camera to take the photo and the second camera just captures depth information at a lower resolution. I wonder if they are having the second camera capture multiple shots at different focal points in order to gather the information necessary to correct the focus in the main image.
so why change the focus? Bring the whole picture into focus. Yes, it's a bit unnatural, because an eye could never do that (unless the iris was stopped down to a pinhole), but so what? Everything in view would be clear. Or would that be mathematically impossible?
+rchandraonline It should be possible, but not with the microlens array itself. It would take additional post-processes, along with edge detection to know when a part of the image was at peak sharpness while working through the various focal depths. So basically a program would start at one end of the refocus range, and rank each part of the image as to how sharp it is. Then step the the next depth, and rank it again. Do that for the full range, and then blend the maximally sharp areas together. There will be even more work required to deal with occlusion of a part of the scene by the out-of-focus blur of a nearer object. Though that may fall out automatically from the sharpness evaluation.
+Christopher Meadors I think that the Lytro software can do that easily. You can see at 11:45 on the macro picture of the fly, when he clicks on the fly's rear leg the whole the whole picture is focused for a really brief moment of time (it's better visible if you slow down the video). Also the technique you're talking about is called focus bracketing or focus stacking and can be done even on classic cameras, but to do that you need to take multiple pictures with focus slightly changed on each of them. Then you can use a focus stacking software and process the pictures. You can achieve effects like a macro picture with perfectly sharp background (the effect is stunning, especially if the background is really at some distance from the object being photographed). I've been playing with this technique myself few years ago with my Canon S5IS with CHDK firmware. I've run a Lua script on it that took a pre-set number of photos with focus bracketing :).
+krishnan srinivasan No, consider te Lytro Illum (not the one shown), it has 200,000 micro lenses and a 40 Mpix sensor. The exported image is, howewer, just 4 megapixels. So the very concept of pixel is not that simple in a light field camera.
Could someone use one of these very expensive cameras to take an image of the jastrow duck rabbit "illusion", and see if shifting the point of focus to infront or behind the real image makes the image flip? I suspect such may be the case because if one places a thick glass sheet over a printed copy of the duck rabbit, and then alternates focus point between the top and bottom surfaces of the glass the image can be flipped. there's been a few pyschology phds which used binocular eye trackers, and which say that can't be an explaination, however basic trigonometry shows the eye trackers used just wouldn't have had the angular accuracy to determine depth of focus any closer than +/-5cm over about 30 cm viewing distance.
+steve cummins The retina appears to be non optimal, with light having to pass through nerve fibres etc before hitting the light detectors. Could those those layers be functioning as a micro lens array?
+Imrooot At least for now the image quality is ridiculously bad compared to the traditional imaging. It's going to take a long while before professionals can start switching to these things.
It's not yet comercially viable. This is the future of mobile phone cameras that are 1mm thick and always in focus. Alas the microlenses are hard to mass produce, but give it some time the material engineers will figure it out.
Bogdan Manciu The thing is really that I don't see out-of-focus pictures being such a big problem anyway nowadays. People are fully capable of using cameras with autofocus, so this technology is not really solving any pain points. It looks more like a "nice to have" kinda thing, probably nice if the image quality ever comes on par with the "normal" stuff, but not worth paying significantly more for.
it does not neccesary make sence doing it that way it also would be possible just to increase the sensor size and lens size so they make a bigger focus area the lens in front the 3d lens also would have its limitations it certainaly has a bend point where it cant make a focus depending how big the lens really is and work
+2chws For photographical purposes, astronomical objects are infinitely far away. This means you don't need to refocus and you also don't have any parallax effects, at least not by moving the camera a few centimeters. However, in radio astronomy a similar technique called beamforming is used to get a virtual directional antenna that you can redirect at different objects in post.
***** Yeah, the RU-vid comments have accumulated quite a few bugs in recent times. But this one should be so easy to fix that it's the ultimate prove that no-one in this company has ever come across it themselves, never mind some kind of QA department one could expect. As if a lot of the people here weren't bad enough, now we also have to fight the technology :/
So in other words there are more pixels in this CCD than in a common CCD camera...? It'd be more interesting to use a 50% random mask (instead of a front lens), and do a 'hologram-like' inverse function on the CCD output...
Interesting, but camera can take images very fast nowadays, why wouldn't you just continuously take images at multiple focus levels? And if you wanted the parallax effect then move the lense also?
Because even very fast isn't fast enough when objects or the camera are in motion. You get motion blur. And combining the parallax effect with refocusing would be impossible as well.
