Canon has developed an augmented reality headset that they refer to as mixed reality. The currently available headset, the HMA-1, uses two OLED panels for a total resolution of 2560×1024 (dual SXGA). There are also two VGA resolution cameras built-in.
There appear to be two main ways to create a mixed reality solution. One is to use a matte or objects printed with bar codes that can be read by the VGA cameras. These trigger the creation of virtual objects that can be inserted into the virtual world or the real world, as tracked by the cameras. This is not an optically transparent headset as it uses OLED panels, but the cameras generate the view from nearly the viewer’s perspective.
The second method attaches a series of tracking balls to the headset and other objects so that the can be accurately measured in a 3D space. For both methods, external cameras need to be in place to accurately track the position and orientation of the headset in the 3D space. Canon offers solutions from several companies with tracking operating at 780 or 800 nm.
At the Canon EXPO, there were three demos. One was an auto repair training demo with bar codes on a matte that inserted views of the engine if you looked up as if in a mechanic’s bay. You are then handed a wrench to try to remove the oil filter.
In the second demo, I was working with another person in a virtual collaborative workspace to discuss repair of a Canon printer. In this case, a headset with tracking balls and 5 cameras was used with two back-to-back sets meant to represent the two collaborators, who can be in different cities.
Placing the headset on, reveals a virtual printer. A tool, also with tracking balls, is used to identify objects in the printer for discussion with the collaborator, who appears as a blue head and tool in the virtual world (see image).
Both of these demos were interesting but did not seem very useful yet. There is significant motion blurring in the images and they are very soft. The headset is tethered with a bunch of heavy cables and the headset is somewhat heavy and not very ergonomically friendly. Plus, there is little sophistication to the app. Perhaps the real life app is better and was just dumbed down for the EXPO demo.
The third demo was more impressive. This uses a next generation headset prototype that appears to have less blurring and other artifacts. Here, the demo focused on virtual car design. Using a headset with track balls and 12 external cameras, you can look at the virtual car from the outside or the inside. I was able to observe the car from the outside, watch as the color was changed, then move around to look at the inside.
In the virtual world, the car doors were open so I walked around to avoid hitting them, but that was unnecessary as they were virtual. The inside shows the dashboard and you can sit in a real driver’s seat with a real steering wheel and transmission shifter. All the instrumentation is then inserted into your view.
What was very clever was the mixing of the demo show floor with the virtual image. For example, sitting in the driver’s seat and looking out the front window revealed the trade show floor as did looking in the rear view mirror and side windows. These were quite accurately tracked so as I moved my head, the position of the windows remained fixed in the 3D space.
The system also inserts an image of your own hands on the steering wheel and allows trying to manipulate virtual buttons. As with most systems to date, the edge detection is poor, with parts of my hands chopped off.
In another part of the EXPO, what looks like a handheld version of the prototype mixed reality headset was on display (see image). Few details were available, but it is believed the device includes twin displays (resolution unclear) with a 120-degree field of view. 360-degree content with a resolution of 5600 x 2800 was streamed to the device, so only a portion is shown at any time to the viewer. The tracking/orientation technology was not described.
