I have reported several times on Finnish XR start-up Varjo, since I first recognised the potential of their concept of ‘foveated resolution’. I reported most recently a few weeks ago from AWE Europe in Munich. (Varjo Pushes The Envelope on VR Performance)
So, I wasn’t sure whether to accept their invitation to a brief meeting in London as they started out on a tour of technology press in Europe and the US. As I was the first meeting on that day, it turns out that I was the first outside the company and its partners to see a really exciting new development using their XR1 headset.
The demo started out by putting me in the headset, which has just been made available as a ‘development product’. It’s a real product, but not yet at an iteration that would go into mass markets. I was strapped into the headset and the first thing was that dual 4K Windows displays were mapped onto the view of the real world coming from the headset’s cameras. The headset is a mixed reality device, with high quality cameras that map the outside world onto the display.
One of the pieces of ‘secret sauce’ that Varjo has, is that it has done a lot of work to create a very fast video mapping pipeline – said to be just 12ms onto the screen. 6ms is taken in exposure time for the camera capture and the other 6ms is used for processing, mapping and synchronising to the display. As a result, the image is so responsive that I was able to handwrite my notes on the experience while wearing the headset – something I could not imagine with any other mixed or VR systems that I have tried.
(I’ve put links to videos at the end of this article and through it. The experience I had was very close to what is shown in the videos)
The company has developed technology to simply allow the mapping of virtual Windows displays into the view of the real world in the headset. That’s great as the technology, combined with the high resolution central area of the display, allows the operation of any 2D app.
In the second demonstration a 3D car was rendered as though it was in a showroom. However, the Unity interface could be used to simply drag and drop items from the Unity menu into the 3D space in a very intuitive way. No need for a special interface – Windows was fine. I quickly found myself dropping items into the scene that accurately scaled to the way they would look in ‘the real world‘.
The next demonstration was of mixed reality. A couple of simple spheres were shown. The Varjo technology allows the lighting from the room to be analysed and exploited so that a virtual reflection of my own hand could be mapped onto one of the spheres in real time – very impressive.
Next up was a demonstration of a planetary simulation. The idea here was to show how virtual objects in the mixed reality world could be used to accurately occlude the background. which was being captured by cameras. It also showed how sophisticated lighting effects – such as a glow around the sun, added to the experience. It seems to me that the possibility of occluding the view of the world behind virtual objects is a key advantage of mixed reality.
The Pièce de Résistance
However the really (and I mean really) impressive demo was the next one. First I was instructed to look to my right where there was a door – like one between an aircraft cabin and the cockpit. I got up out of my chair and moved into the cockpit area of an airliner that was modelled down to the smallest detail. I sat in the Captain’s chair and was able to inspect the different control systems. The headset gave enough resolution that I was able to read the detail on the radar screen. Impressive indeed! All the while, real world items could have been mapped into the simulation (at I/Itsec, the system was being shown with physical controls mapped in).
When I turned around to look back, I could see right through the doorway, of course. It really was very impressive and I don’t remember ever giving a spontaneous round of applause at a demo before!
The mixture of virtual and real world images has been used by Volvo in looking at car interior designs. The models can be shown in the glasses and mapped to the position of the real dashboard, so a current car can be driven around while a designer can sit in the car and see how the interior looks.
The headset I tried is heavy but works well. Image:Meko
Now, the headset is heavy. It’s not cheap and its tethered and I only saw a demonstration, but it really shows how mixed reality could have a dramatic effect on 3D design, simulation and on content creation.
Varjo originally captured my interest because of its innovative display technology, but the company has strengths with its own gaze tracking technology, very good and fast video integration and systems integration with software. Ken has suggested that the display technology might be superseded by displays that are high resolution all over and don’t need the clever foveation technology. Fortunately, the company has a number of other strengths that should allow it to make a big splash, initially in professional applications, but later in a whole raft of areas. (BR)
Analyst Comment
Subsequent to my demo, I heard from someone else that saw it in the US, a few days after. He very much saw it as ‘Meh’, and didn’t plan to write it up (as he has seen Varjo before). Maybe I had drunk too much of Varjo’s Kool Aid! However I was impressed with the tight integration with the real world view and the Varjo optical solution. (BR)
