Knowing the tech industry, this is a good challenge for everyone from Apple to Google to Facebook/Meta. It’s definitely going to take a long time, but remember, that VR technology took decades just to get right.
VR took way too long to get off the ground, Carmack was right to say it was "coasting on novelty". These tech monopolies are great at extracting rent from other people's ideas and hard work, not so good at actual innovation amd engineering when it's needed. Facebook hired 10k engineers and it still took a year to rollout 120hz on the Quest 2. Apple has more money than many entire countries and its VR is floundering. Google is in the same position as Apple and has barely started. Microsoft too. All of these companies move too slowly and are too risk averse to introduce these new technologies without the promise of a monopoly over the platform, which itself slows down innovation and engineering.
@@donkey7921 are these huge, or is it that incremental improvements are still required? What drove the issues? My guess is comfort, visual and physical.
@@float32 Id guess the fact that nothing is physically there, and that headsets are inconvenient and uncomfortable, also not everyone has a lot of room in their house.
I love the interviews you are doing with the people who are in the industry for a long time. The have seen hype come and go and can tell a lot of stories about that. Know the history to get an idea about how the future might be.
1:00 I can confirm that's an issue with Hololens specifically, and waveguide AR generally, due to manufacturing limitations. I remember when I tried on a Hololens last week, my initial reaction was "yecch, y'all can tolerate this?" (my Magic Leap doesn't feel like it's on that level of obligatory rainbow, though whites still look like crap). Today, these diffractive waveguides are usually cast (with single-use molds) rather than directly etched like semiconductors, making them rather low quality for their cost. Since every waveguide comes out a little different, it's ridiculously hard to correct for these aberrations, since they'll be different on every device. I suspect Hololenses have especially rough image quality because the waveguides have to magnify the projector output more than in similar devices (that use conventional-ish LCDs instead of beam scanners).
In VR we already have headsets hitting the limits of human vision in at least the vertical FOV. It seems to me, personally, that AR presents a very unique set of requirements.
The current flip down visors for sports/touring motorcycle helmets, with a HUD for gps navigation, speed/rpm/lean angle/G meter/traction control/abs/cruise control/heated grips/trip computer etc, rider entertainment (music) and emergency services, is one of the better use cases for AR.
Let me tell you why I think this guy's opinion is wrong. Not once did this guy ever bring up any kind of UI or ux considerations when talking about the displays. He also seems to have this idea that there's not going to be 6dof tracking because he said when you turn your head the image moves with you. I've been a VR developer for 7 years and I've done an immense amount of UI and ux design. Even if you ignore the background content and just focus on menus and interacting with those things, having it glued to your face is terrible. I do respect the comment about the hard cut off for horizontal field of view, but even with foviated rendering it's still distracting to see the blurred lines. The field of view does matter, but you need a foveiated rendering that has eye tracking. My viewpoint on this is majority of the things we are striving for in the VR market is what we will also be striving for in the AR market. I'd be more willing to wear a visor similar to what you see on Star Trek then a pair of glasses that sits several inches away from your eyes. If we get perfect clarity of your real world surroundings and decent battery life in the headset with 6dof, then we can make it work. I honestly don't think we're going to stumble into AR from a ground up scenario. We're probably going to get into a minimalized VR headset with pass-through and eventually they will boil down the components until it is mostly an AR device. I know there's a lot of intelligent people in the AR field, but every time I look at AR development I just go what the fuck are these guys thinking?
Agreed. He also has Apple-Meta backwards. Apple has had VR working labs since early 90’s when I was contractor and firmly understands VR is neat, but special-case, like cinema or amusement parks experiences. They seem to look at practical applications focused on AR and are actually building a groundswell ecology/economy for devs to supply content to either market. When I took off my first headset I did wonder, for a moment, if my mates had slapped a Kick Me sign on my back and that has remained my fundamental critique of VR in workplaces: it violates our instincts around surprise and safety.
He briefly mentions ui difficulties at 20:35. as for the "when your head turns your image moves with you", I think that was meant in a different context or simply explaining the tech difference (couldn't find the exact moment in the video again so I'm a bit hazy on that). Obviously even laymen know that Hololens had good tracking years back and he even specifically talks about Hololens.
What do you mean by stumble into AR? There are already commercial optical AR glasses and commercial passthrough glasses. The reason the smart glasses form factor is so important over a visor is that it is socially acceptable to wear. Sure maybe you would be willing to wear that in public but not the mass market (which this video focused on)
@@gyroninjamodder they all suck and suck to develop for. Closest thing I see to a good set up is the new Qualcomm reference design. I hate how much it sticks out though
I especially thought 'WTF are you thinking" when the damn Hololens crew pitched it to the Military for...use by grunts. It was at that point I knew the nuts were running the nut house. Not a single soldier on earth wants to pack that wierdness onto his battle rattle. In my time in uniform, as a mid-ranking Joint Fires guy working with JTACs and FOs and FACs, I've strapped on what was probably the earliest known military application of AR, the Optical Relay Tube for the TADS (Apache Helicopter head-tracked gun control system) and wiggled the gun back and forth, and then had a long, enthusiastic discussion with some F-35 pilots at Nellis AFB about how their million dollar whiz-bang helmet (which is probably the most current example of AR in the military) is a game changer in comparison, because of it's incredibly generous FOV and all that it brings. But those are high-value combat vehicles, not meatballs carrying $400 worth of rifle. Even on my Bradley (the IFV, not the RU-vidr), my Thermals, FLIR & Target tracking systems in the IBAS and CIV were about 12 years behind what the civilian market uses, because red tape and beareaucracy constantly keeps us a decade behind on tech. So I have no clue what the hell Microsoft thought they were bringing to the battlefield, other than a 22 billion dollar vaporware contract no one but Microsoft and some old fart of a General wanted, because he read too many Buck Rogers comics as a kid in the 50's and thought it was cool.
Thank you for this, real, accurate, and refreshing. I am amazed at the sci-fi expectations I have seen around Apple's AR/VR. A lot of it comes from what is supposed to be tech-savy sources. Keep up the good work.
I was always about wanting huge super clear FOV, but his comments on how we mostly just care about the image being cut off and not so much about the clarity in the periphery totally makes sense to me. I glance around maybe 30-45 degrees of view but tend to turn my head for anything beyond that.
@@ThepurposeofTime right. Wide FOV is great but super clarity/resolution is not needed for the periphery to feel the immersion effect of the wide FOV .
Breath of fresh air getting these points of views. Very sobering. Absolutely love your work, @sadlyitbradley You're doing god's work out here. Thanks to Karl as well
A low resolution border around the outside of the VR or AR display for the peripheral vision. The resolution could be like 10 pixel per degree. I would prefer that to nothing.
Small FOV is by far the biggest problem here. Everything else isn’t as huge as Guttag makes it out to be, imho. Energy of course always is but that could really be solved with a backmounted battery or at least a cable. Larger FOV on the other hand is absolutely no where in sight!
As an owner of Oculus (DK2), then Quest, then Quest 2, I was pleased last week to finally have an opportunity to try out the hololens 2. Considering the price, I had high expectations. What a disappointment, in almost every respect. Underwhelming display quality, lack of realism, cumbersome to operate. One positive: the headset was comfortable - better than quest or quest2 regarding weight distribution. But other than that I would choose Quest2 even if they were the same price. I know, it’s a different application, but The reason to choose against AR is if hololens is the best that one can do with AR, then Im not interested in AR, until they can do much better. Quest2 is closer to solving VR than Hololens is to solving AR.
A good first step is a Rift CV1 as they are very affordable second-hand now (I paid £166 with controllers and 3 sensors) and are still a great experience with premium sound and comfort.
@@zig131 Thanks for the suggestion! I’m saving up for a good CPU, GPU, and RAM for my PC right now. I don’t trust FB having access to cameras and microphones in my house, so I’m hoping to use something that uses Lighthouse tracking. Sorry for the long reply, have a good night!
I'd be curious what Karl thinks of the near term practicality of very basic "AR" / smart glasses ("basic" at least compared to the hype). E.g. 20 to 30 degree FOV, monochrome, stereoscopic, fixed focus, and a wireless (or even wired) connection to a smartphone that handles the bulk of the processing. Good enough clarity for displaying basic things like text messages, time and calendar events, notifications, diagrams from an textbook you're listening to, timecodes/segments/chapters for a podcast or audiobook, simplistic web pages (at night I use an epaper display to browse the web, so maybe not too far fetched in some cases?), and so on. Almost like a "better smartwatch" but with lower friction, without occupying the hand, and without the focus/display size limitations of this tiny thing on your wrist. Most of it would just be projected at the fixed focus depth rather than even attempting to fuse it with the depth of the surrounding environment and thus triggering VAC, and overlaid content itself would be minimal (perhaps only displayed upon intentional invocation) unless you go out of your way to bring up something more comprehensive. So it's largely just a HUD that's decoupled from the depth of the real world environment (and thus many may not even consider it AR). But would this be practical in relatively compact form factor glasses usable in typical lighting (maybe not direct sunlight on concrete), without significant eyeglow, and without being too heavy? Because I see all of this hype about AR being used for immersive 3d games, watching full color videos, replacing monitors, perfect telepresence video calls, seamless integration with the local environment, dragons and whales flying at you, and so on, and I'm thinking "OK, but what about all of the 'boring' but practical stuff that doesn't need such high fidelity?" Listening to Kai Stroder of tooz on The AR Show, you'd come away thinking this kind of "informational HUD" isn't that far off (though their display is monocular), but obviously I can't rely on their marketing guy for an accurate picture of what's actually practical. However, if I could just avoid having to use a smartphone or smartwatch for this basic stuff (that is, without the friction of taking out and unlocking a smartphone, occupying a hand, focusing on a tiny display in/on my hand, and so on), my sense is that it could still be insanely useful for many people despite not living up to the hype. Of course input and feedback is still a big question ... for the time being you may still need a phone to e.g. input text to respond to a text message (not that touchscreens aren't terrible for inputting text to begin with), but at least your display wouldn't need to be occluded by the on screen keyboard.
Is this why they made apple AirPods max so heavy/expensive. To prepare users for the glasses. Where the glasses will connect to the AirPods max and store the battery and processor.
I very much enjoy the observation at 1:50, about these companies making solutions for someone else. It's fundamental. If you're not building it for yourself - and benefiting from its use - then why are you building it at all? I feel like so many companies are paying lip service. AR/VR/XR are the future, yes, but just like we had Microsoft and Apple dominating desktops at a time when "everyone knew" that computing and the internet was the future (spoiler: everyone didn't know, hindsight 20/20), we will have 1-3 companies that no one's yet heard of dominating this next medium. Metaverse? Idk, weird name. I like to think of it as the "VR-continuum", virtual content via varied visualization technologies.
The talk of bringing your own black made me think of the layered light field display tech, with experiments from e.g. nvidia and MIT. Could we use subtractive colour, drawing in occlusion instead of bright overlays? Could we perhaps use few-colour like Casio's old graphing calculators? There's certainly similar tech being used e.g. for lightfield backlights (Leia) or parallax barriers (Nintendo), but those aren't doing images in the same layer. I know it's just a vague idea, and reality would probably hit us with ridiculously poor transmission, resolution and weight, but it would be nice to know what the options are.
One option you might find interesting is what Magic Leap is doing. We will have an entire video later in this series detailing it. But it uses Liquid Crystal lenses to locally dim the areas that the display is trying to emit images on top of
@@SadlyItsBradley This approach currently has artifacts usually described as a black border. It's another engineering challenge to reduce / get rid of those artifacts.
Why can't we have a half-VR display, like a VR monocle that blocks outside light. Your eyes would naturally overlay whatever is shown there on top of the real world. I know it doesn't really solve a lot of problems but it's an interesting idea.
Great, insightful interview, but that FOV argument makes no sense to me. Comparing HMDs with headtracking to stationary screens designed for framed (!) content like movie theater, TV, laptop or smartphone is just wrong. This isn't just about immersion! The core principle: putting virtual FULL scale objects around you is the same for both VR and AR. With low FOV AR degraded immersion is not the issue, but the objects cut off and you can't use your peripheral vision to constantly update your mental image in your brain of where everything is, but you can do that with everything in real life. Peripheral vision doesn't need a lot of resolution but it has to be there or you have a big problem. A virtual iPhone that can only be seen directly and immediately cuts off and disappears when you tilt your head a little bit would be very irritating to use. That cannot compete with just a normal smartphone or normal laptop. We use saccades to jump to the thing we want to see and keep our eyes rotated usually a small amount, but when you decide where to "jump next" we use wide FOV data from our surrounding to not just move eyes but even rotate head. Even if you are only interested in virtual screens and virtual text when you want to emulate triple monitor or ultrawide setup you will have this problem. AR that can't even be used for that is useless for mainstream.
Respect to homie's resume and all, but I keep hearing the same old refrains. X amount of resolution is _enough_ , Y amount of FOV is _enough_ , and Z amount of refresh rate is _enough_ . Time keeps proving them wrong. How many decades did we hear that the human eye couldn't differentiate above 72hz? Gaming display market has joined the chat.. Same treadworn theories here about moving your head in wider FOV's. I game on a 48 inch ultrawide, and my eyes just flick to things like an FPS radar overlay. I'm not tossing my head to and fro like some meerkat looking for a predator. There's some overly dramatized research being treated as gospel. Maybe too much entrenched dogma in the upper levels of tech experts about what 'people actually do' that makes me think they don't actually know _what we do_ because they themselves are not doing it. Just reading white papers.
He didn't say the things you're saying though. His argument is that wide fov in the AR context is impossible or difficult for practical and physical reasons, not that the imposed limitations are "enough."
I think the misunderstanding is that he's viewing this through the lens(pun intended) of trying to market these things to the average consumer that won't be okay with wearing something too heavy and impractical all day. When even the Ray Ban glasses that don't have a display are bordering on too heavy, how do you expect to fit in all of these "luxuries" into a glasses frame? High resolution, FOV, refresh rate, these things dramatically increase the compute required and the weight would get ridiculous. The technology just isn't there yet and probably won't be for the next 5+ years, we can't make the components compact enough yet, we don't have good enough lightweight batteries, there's a million issues that only get drastically worse if you aren't pragmatic and practical about it.
*Absolutely ANYBODY who's even the slightest bit interested in AR* *should be made to watch this video. All the tech hurdles are laid-out in detail, but explained in a simple,* *super easy-to-understand way so that even dopes like me can come to grips with the FACT* *about just how far away lightweight, self-contained/standalone AR glasses REALLY are.* ⭐️⭐️⭐️⭐️ 🙂👍🏻
Keyword there is self-contained. I personally have zero interest in self-contained AR glasses, because that's clearly too far out at this point to be excited for. I'm not bothered. Glasses with a compute pack that sits in/on your pocket, is our near term future. And these glasses *can* be lightweight and small enough, in the near future. Just look at Lumus Maximus. It needs cameras added still, yes, but that's it. It can still be lightweight and small. With this tech, we *will* potentially have (somewhat thick) Rayban style true AR glasses within a couple of years. Issue is, even then, it may *still* be too heavy. "Lightweight" for AR glasses could be defined as 80g. Best we have is Nreal Air at 77g, which doesn't have any cameras so it's not true AR. Perhaps (with cameras) we need to get this down to 50g. I have no idea on what sort of weight tolerance the general public has for glasses like this. As an enthusiast, I would definitely buy a pair of Rayban style true AR glasses weighing 80g. Meta are probably struggling to reach 80g, let alone 50g. 100g is too heavy (Nreal Light). From impressions, it's seems fine to wear initially at least, but in reality, people aren't going to find that comfortable for any extended amount of time where they can get some good usage out of it.
I think I know why Meta pushed back the consumer release of their AR project... honestly, MR seems overall more useful if usability issues (comfort!) can be solved.
Agreed. However I do expect ""smartglasses" to be a first AR Glasses from these companies in the 2024 time frame. Basic functionality. Mostly focused on overlaying notifications
A smartwatch for the face. Meh. I want my MR superpowers - eyes that can zoom, record, and see in different light spectrums! Cameras already outpace human vision. Why would I choose my weak human eyes? I expect an industry push for AR, but I don't think it's the holy grail of XR technology.
@@SadlyItsBradley Unfortunately that is highly likely. Full AR glasses from Meta and Apple are probably another 5 years out. But I'd still be extremely happy with using basic glasses as a monitor replacement. That alone would be amazing.
Agree! AR is actually out of technological possibility. As always, industry bets on premature tech just to impress consumers; but the latters are not so stupid. Consumer AR is something for the far future, maybe by means of holographic projection and not by glasses, who knows. Now, it's time to focus on consumer VR that comes with most realistic technological foundation today and in the near future.
u can't get to mature tech without betting on the premature.....it's just the days of the fat monochrome monitors now it will take time to get to thin big oled tvs holographic projections will never be better than glasses
@@pumpuppthevolume putting on the market disappointing technologies trying to fool consumers is not the same as betting on new technologies; on the contrary, it means to prematurely kill the new tech! As happened with VR in the '90s.
@@videogamesart8057 I can't say vr actually existed before 2016 besides as a novelty experiment since there was nothing with good enough quality reasonable size and for reasonable price .....as soon as a big company like meta has a 6dof ar headset ar will have its 2016 moment even though nreal light is already good enough 1st gen consumer device
@@pumpuppthevolume VR was put into the market in the '90s with premature technologies just to fool consumers with misleading marketing selling water for wine. It was a complete failure. Today it's the same with AR. Actually we have not adequate technology for AR. Have you listened to Karl?
@@videogamesart8057 Karl likes nreal's image quality .....also he wants perfect ar glasses or nothing ...I'm fine with compromises before 2016 consumer vr practically didn't exist and basically nothing in the 90s could power a good vr experience ....u can't set back a market if there is no way to have a market like that with the tech of the time
I've heard a lot of push for AR from manufacturers, but I haven't heard a lot of demand for AR from consumers. Perhaps I'm not the customer they want. I am very excited about VR. I'm a flight simulator enthusiast, so whatever can put me "even more" in that cockpit, the better. But augmenting my existing reality? Meh.
i've often thought about the human vision being focused downward. none of the VR headsets really seem to use this cuz all the lenses/views are pretty much centered. seems like they waste a lot of pixels on top and lose a lot of potential for immersion. think the floaty feeling a lot of VR has would be solved by being able to see the ground below you better.
That use-case will probably be very possible. And will be the first smartglasses releasing from the big players. But I am not sure if it will be as cheap as an Apple Watch
Regarding the viewing angle comfort curve at 14:00, the wider FOV is good for keeping the data visible as a target. Turning your head in QR could absolutely make sense, to view something a bit off-axis, so you can use your comfortable view for the real world or active overlay, while additional info (for example, a textual manual of the current step in the assembly for AR training) is axessible by looking at it by tilting your head down. Alternatively, combined with eye tracking, you could pin the content with your eyes, at which point turning your head towards it brings it into center field, while otherwise, it would move along with the head. I believe that's a normal locking-on thing we do with our vision anyway, by first fixing with eye movement and then following with the head.
Interesting how AR FoV needs to have different extension following the application of interest. The biggest FoV is not what you always want for whatsoever application, especially in professional applications where you need to focus on details; too big FoV could be distracting and introduce confusion; think at medical applications. On the contrary VR FoV needs to be as big as human eyes FoV.
