Meta unveils its advances in screens for virtual reality headsets this June 20th. The American company, a pioneer in the sector since its acquisition of the start-up Oculus in 2014, has achieved a notable breakthrough on the consumer market with its Quest 2, whose total sales are estimated at 15 million units to date by IDC.
However, the display technologies that it uses (Fresnel lenses, LCD screens) are close to their limits and will not allow to achieve a visual quality and a compactness sufficient to really convince the greatest world. This is where Meta’s research division, headed by Michael Abrash, comes into play. She tackled the problem as early as 2015.
A “visual Turing test” for VR headsets
She has developed what researchers call a “visual Turing test” whose objective is to provide an image so realistic that the user is not able to tell if he sees the real world or if he is using a VR headset.
We are obviously far from it today because of a whole bunch of problems, starting with the resolution (which will have to be higher than 8K per eye) and latency obviously, but also the conflict between vergence and accommodation, ocular parallax, chromatic aberrations and the “pupil swim”, which refers to the distortions of the image that can be seen by looking at different points of the lenses and not just their center.
Meta researchers have created a clinging prototype of this level of quality, named Butterscotch. It provides a resolution of 55 pixels per degree, which is enough to be able to read the last line of a vision test (Snellen chart type) directly in the helmet. To achieve this, the researchers had to reduce the horizontal field of view by half compared to the Quest 2.
For the problems related to the lack of focus, which is typically fixed at 5 meters in current helmets, the solution is the use of varifocal lenses. The latter require having cameras inside the helmet to track the movements of the gaze very precisely in order to dynamically adjust the focus, as the lens naturally does in normal times when looking at an object near or far from us. This is the purpose of the Half Dome prototypes that Meta has already communicated about several times in the past.
“We still have work to do to ensure that their performance is up to date and that eye tracking is reliable enough to be usable by everyone and in all circumstances, but we are optimistic,” Mark Zuckerberg said about this during a press presentation.
Regarding the distortion related to lenses, it is due to the fact that the algorithms that correct their natural distortion are static. Meta seeks to make them dynamic according to what the user is looking at. To speed up research in this area, its researchers have developed a distortion simulator so that they can iterate as quickly as possible instead of waiting weeks to create test lenses.
The last element is the brightness and contrast of the screen. According to Meta, the television industry would have been estimated that the ideal would be to be able to produce peaks of 10,000 nits to provide a perfect image. They are still far from it today, but virtual reality is not even in the mainstream, with a maximum brightness of barely 100 nits for the Quest 2. To test what an “Ultra HDR” image of this type would give, Meta has developed a massive prototype called Starburst (visible above) which places a strong lamp behind the screen.
Holocake 2, an ultra compact helmet prototype
At the moment, the most promising prototype of Meta is called Holocake 2. Its design had leaked a few weeks ago, with some mistakenly speculating that it was the Cambria project, a commercial product that will hit the market by the end of the year.
Holocake 2 combines a holographic display and so-called “pancake” lenses (with a flat design) to which polarized reflectors are added that reduce the distance between the lenses and the screen. The result is an extremely compact headset that also has the same characteristics and performance as the Quest 2 (field of view, resolution, refresh rate, etc.).
The only catch: the light source is a laser, which involves more work upstream to be able to make it a consumer product. In addition, unlike the Quest 2, the prototype works by connecting to a computer, it does not include computing capabilities. That aside, it is 100% functional and can run any virtual reality application on Windows.
Mirror Lake, a very promising concept
But Meta’s vision goes further. Indeed, all the prototypes presented in this article each solve a problem separately, but the real challenge is to be able to solve everything with a single technical solution, in a helmet that is both lightweight and compact.
The last helmet presented is called Mirror Lake and is not even a prototype. Michael Abrash as well as Douglas Lanman (director of research in optical systems) have insisted on this fact. This is a concept of which no functional version exists. But if the research results in something that works, it will be a revolution for virtual reality.
Mirror Lake starts from the base of Holocake 2 and adds everything else to it in order to address all the problems stated earlier. The key to achieving this is the holographic display, because it makes all the other components also flat, supplemented by cameras positioned on the temples. It is even possible to add a correction to it to get rid of prescription glasses.
If the researchers at Meta Reality Labs Research manage to realize their vision, it will be the biggest leap forward that VR has seen so far. But we will not have to wait for it for many more years.