LetinAR Co., Ltd., a startup Augmented Reality company in Seoul, South Korea, has shown its new design for AR glasses at both CES 2018 in Las Vegas in January and at Mobile World Congress in Barcelona in February. Unfortunately, I was not at either event and have not seen these glasses in person but an analysis of the technology shows it is very promising.
Another thing I have not seen is a press report saying exactly how these glasses work. While reports have said there have been problems with the glasses and LetinAR has said that these problems are solvable, without understanding the operation of the glasses, it is impossible to understand either the comments from the reports or evaluate the ability of LetinAR to fix the problems.
An array of 15 pin mirrors embedded in a block of transparent material, as shown at MWC. The black dots are the pin mirrors, the lighter gray dots are the shadows of these mirrors. The company also showed a smaller array said to have a 50° FOV and an element with a single pin mirror and a FOV of 15°. (Credit: Meko)
The LetinAR system is based on pin mirrors embedded in a transparent substrate. These small mirrors are tilted at about 45° to send the image into the viewers eye. Typically they are flat but their diameter is small enough (pin-head size) so they act as a pinhole camera and provide a large depth of field. The optical design of the pin mirrors is application dependent. In addition to a flat mirror design, it can also be a freeform curved mirror which can both fold the light path and focus the light in order to provide the virtual image to the viewer. The advantage of a freeform mirror is that it can have a larger diameter and therefore higher efficiency and still have a very large depth of field. Flat or curved, the mirrors are so small and so close to the eye that the mirrors themselves are not seen – only the image generated by the mirrors is visible. The lenses with the embedded mirrors are essentially see-through to the real world, with the mirrors themselves interfering very little with the outside view.
Each pin mirror both folds and generates a virtual image of the microdisplay by acting as a pinhole lens. (Credit: LetinAR)
Mirrors acting as pinhole lenses or focusing mirrors have a major advantage over the lenses, diffractive optical elements (DOEs) and holographic optical elements (HOEs) used in conventional AR HMD designs. Mirrors and pinhole lenses are achromatic – that is, they have the same focusing properties for all wavelengths of light. Transmissive lenses, especially low-cost plastic lenses used in affordable HMDs, have a significant amount of chromatic aberration. This chromatic aberration is often ignored by AR HMD designers because fixing it adds both cost and weight. DOEs and HOEs are very strongly chromatic, to the point where typically multiple HOEs are needed, one each for red, green and blue at both the input and output. This often also requires the duplication of conventional optical elements such as waveguide plates in the HMD.
Sony see-through AR HMD design with six HOEs and two waveguide plates. (Source: SID 2018 Display Week technical paper 17.3)
Each individual pin mirror has a diagonal field of view of about 15° although the company expects to be able to increase the FOV of an individual pin mirror to 40° in the future. With the array of pin mirrors, as the eye moves, the images from different pin mirrors become visible. This extends the FOV of the current LetinAR system to about 70° horizontally and the company says it can make it 80° or more in the future. This wide FOV image formed by multiple mirrors leads to one of the observations by previous reporters: the image does not transition smoothly from one mirror to the next. The company’s answer to this is more accurate alignment using an aspherical optical design with tight tolerances, such as high-accuracy injection molding of the lenses.
LetinAR system can be built with the imager mounted on either the top or the side of the lens. (Credit LetinAR)
The virtual image to be shown is generated by an OLED microdisplay that can be mounted on either the top or the side of the lens. Normally, the surface of the display would be optically coupled into the lens with no intervening optics. The pin mirrors are arranged in lines that are staggered so no pin mirror shadows another pin mirror from “seeing” the microdisplay. The sample shown at MWC had two rows of pin mirrors, one with 8 elements and the second with 7 elements and a 70° FOV. This wide horizontal array provides a wide horizontal FOV, with a vertical FOV of about 25°. This vertical FOV is provided by two rows of mirrors each with a FOV of 15°. When the image is injected from the side, vertical arrays of pin mirrors would be needed, giving a wide vertical FOV but a more restricted horizontal FOV. For many AR applications, however, a 25° horizontal FOV with the image pushed to the side of the wearer’s FOV would be sufficient.
The depth of field of the virtual images generated by mirrors is said to be from 25 cm to infinity and LetinAR says this is about 10x the depth of focus of an AR head mounted display (HMD) with a conventional imaging system. This bypasses the problem with other AR systems where it is desirable to put the plane of the virtual image in the same plane as the real, see-through image. There were about 16 papers addressing the issue of depth of field using conventional designs of AR HMDs at the recent SID Display Week but I don’t feel any of those 16 papers presented a solution that was both feasible and compact enough for use in a HMD. (See Meko’s Special Report on Display Week – subscription required) The LetinAR pin mirror design would be considered a Maxwellian (i.e. pinhole) design and only one paper on a design of this type were presented at Display Week (Paper 17,2). That design involved a complex approach of steering the pupil to keep it centered on the eye’s pupil as it moved, as opposed to the multiple fixed pupils used by LetinAR. The Maxwellian design type bypasses the problem of adjusting the optics to match the focal plane of the lens of the eye by having all of the AR images in focus, regardless of what the eye is focused on in the real world. This is illustrated below and in a short video simulation from LetinAR.
Three images from LetinAR showing the importance of depth of focus in an AR system. (Credit: LetinAR)
In the top pair of images, the human viewer is looking at the most distant real toy, the blue robot. The eye is focused on this toy and the robot is in sharp focus. In an AR system with a conventional lens, the image plane of the virtual image of the kitten is at approximately the same distance as the robot so the viewer sees both the real robot and the virtual kitten in focus. In the right half of the top image, the situation is similar, with both the real robot and the virtual kitten from LetinAR display in focus.
When the eye shifts to the middle real toy, the brown dog, the conventional AR display displays the kitten partially out of focus. This occurs because the eye is now focused on the brown dog while the kitten remains on the image plane at the same distance as the robot, which is also now partially out of focus.
When the eye shifts to the nearby blue ball of yarn in the bottom image, it refocuses at this shorter distance. The software driving the AR display has moved the kitten and enlarged it to give a sense of near/far perspective. In an AR system with a conventional lens, the kitten is now out of focus because it is still being displayed at the fixed focal plane of the AR imaging system. Like the robot at the same distance, the kitten is now out of focus to the viewer. In the LetinAR system, however, the eye is focused on the real ball of yarn but the virtual kitten is still in focus. This is because of the very large depth of field of the LetinAR system using pin mirrors where the depth of focus ranges from 25cm to infinity.
This works because the diameter of the LetinAR pin lenses embedded in the glasses is very small. While the optical power of the pin mirrors can reportedly range from 0 diopters (flat mirror) up to 5 diopters, because of the small diameter of the pin mirrors, the f/# of the system is very large leading to a very large depth of field.
The small diameter of the LetinAR pin lenses provides a second benefit. When the diameter of the pin lens is smaller than the diameter of the pupil of the human eye, about 3mm, the eye can “see-around” the pin lens and see the real world in the distance. The eye cannot focus on the pin mirror because it is too close to the lens of the eye.
Reports from people who have seen the LetinAR demonstration at CES or MWC are that the display is surprisingly bright. Surprising because pinhole systems are typically very inefficient due to the small diameter of the pinhole. The company says “Our lenses show relatively high optical efficiency since we do not use a polarization, temporal multiplexing, or bandwidth filtering techniques entailing luminance degradation, which are commonly used in other waveguide systems like, DOEs, HOEs, or special optical coatings.”
The specifications for the current pin mirror lens module include:
- General use side-mounted optical module. Infinite DOF and wide FOV.
- Resolution: 1280 x 720 (Lens covers Up to 16K, depends on Display)
- FOV: Diagonal 41° (Up to 70°)
- Lens Thickness: 2.0 – 4.5mm The unit shown at MWC had a thickness of 4.0mm and the latest plastic lens module has a thickness of 3.2mm
- Color: Full Color Expression
- Image Focal Plane: Nearby – Infinity
- Configuration: Side or Top mounted
- Orientation: Vertical or Horizontal
- Transparency: 80-95%
The company adds that the next generation product will have an 80° FOV.
LetinAR was founded in October 2016 by Mr. Jaehyeok Kim and Mr. Jeonghun Ha, now CEO and CTO respectively. The pin mirror technology was developed by LetinAR – the company is not a spin-out of a university or some other lab. Seed money came from Naver Corporation, said to be the biggest Internet corporation in South Korea. Funding for the company is shown in the table. In addition to this funding, the company is also open to global investments, even after the completion of the pre-A funding round.
|
Investor |
Date |
Round |
Amount |
|
Platinum Technology Investment |
2016.12 |
Pre-Seed |
100M KRW (?0.09M USD) |
|
Naver Corporation, Platinum Technology Investment |
2017.06 |
Seed |
700M KRW (?0.65M USD) |
|
Multiple Korean VCs |
To close in next two months |
Pre-A |
TBD |
The company’s business plan is to manufacture and sell the lenses with embedded pin mirrors to customers to be incorporated in their own AR HMDs. They add there are patents pending on the technology globally.
The company concludes: “Our goal is to break down the technical barriers that prevent the commercialization of AR glasses. LetinAR Lens will be provided to selected partners in 2019.” –Matthew Brennesholtz
