First, I want to clarify that I’ve never used any AR, VR, or mixed-reality glasses because I can’t afford them. However, I’ve been fascinated with augmented reality since the first Iron Man movie came out when I was around 6 years old, and I’ve been following AR technology from the sidelines. So, please excuse any ignorance in my question.
One of the most disappointing aspects of the Apple Vision Pro, from the reviews I’ve seen, is the quality of the passthrough. It’s hard to read things up close, there’s pixelation in low light, and so on. My question is, why did Apple choose to display the real world on a screen rather than using transparent glass? Is the technology not yet advanced enough to project onto transparent glass, or did Apple choose the screen route solely to allow users to switch between augmented reality and mixed reality? How close are we to having “Iron Man type” augmented reality with the capabilities of an Apple Vision Pro?
The main issue with using transparent glass is that, in order for the visuals to appear solid, they must be stronger than daylight; this is actually a far stronger projection than what you would find in a virtual reality headset. The headpiece would get extremely hot and the projector alone would require a lot of power. This design has a fundamental physics restriction.
A projector that tracks your eyes and only projects the comparatively small area of the screen that you are looking at is one of the suggested solutions that is currently under development. As you can imagine, this is a pretty complex piece of technology.
Transparent AR doesn’t make the list because projecting images directly onto glass isn’t a valid suggested solution, therefore that isn’t the main issue. would not be conceivable without lenses to adjust for the various focal lengths of the objects we view.
Rather than using glass, the real suggested remedy for this would be to beam an image straight onto our retina.
You describe something that already exists, called foveated rendering, in the second paragraph.
When it comes to augmented reality (AR) and virtual reality (VR) headsets, using a screen or passthrough instead of transparent glass is a deliberate design choice. The main reason is that transparent glass would require a projector to display the virtual information, which would need to be incredibly powerful to overcome the brightness of the real world. This would result in a hot and power-hungry headset. Additionally, transparent glass would also require a complex optic called a “combiner” to make the virtual information appear at the correct distance and allow environmental light through. This technology is still being developed and is not yet mature enough for widespread use. In contrast, using a screen or passthrough allows for better control over the virtual and real environments, enabling features like latency and black levels that are crucial for a seamless AR experience.