The System for Telementoring with Augmented Reality, or the STAR program for short, is a multi-institution project sponsored by the Office of the US Assistant Secretary of Defense for Health Affairs. It includes Purdue University, Industrial Engineering and Computer Science as well as the Indiana University, School of Medicine.
The goal of the program is to use augmented reality for mentoring surgeons without physical presence. One of the goals is to teach and help surgeons on the battlefield in case they have to perform an unfamiliar procedure to save lives. The following image shows how this is supposed to work.
Source: Purdue University STAR Program
The augmented reality part is where the mentor shows the field person on a display device what to do. This overlay of mentoring on the real world image is what makes it possible for the field personnel to perform the various tasks. In a published video you can see how this is supposed to work in the field. You can access the video here.
When it comes to the augmented reality display, the project seems to be using standard tablets by utilizing the rear camera to acquire an image of the real world, or patient in this case. The mentor then interacts with the image that is transmitted back to the tablet and shows the field person what to do. Besides the fact that the system will most likely be used on a battlefield somewhere around the world, and the transmission to the tablet requires a rather good connectivity, the system does not really include a lot of new technology. Today a good tablet or smartphone can provide a decent video connection to anyone around the world, even using cellular technology such as 4G/LTE. Instead of sending a video stream both ways it goes only in one direction and the added content is send back to the field user where it is combined to create AR content.
Overall this concept could easily be extended to other uses in military, law enforcement and first responder situations and makes a lot of sense in using augmented reality in the real world.
Analyst Comment
Interestingly this project focuses on tablets and transparent displays rather than AR head-mounted displays like smartglasses. Maybe the required image quality is beyond the current display technology used in smartglasses. Most tablets and transparent displays will be able to provide resolutions of FHD or higher in a 10” or larger display module. That performance is still outside the current capabilities of any shown head mounted display beside some fighter jet helmets. This is especially true when we consider the actual field of view used by the device. If the tablet is close to the target the size of the tablet display is very close to the actual area on the patient. An augmented reality headset will expand its available pixels over a much larger area, resulting in a rather low resolution representation on a small area. Surely such a system could be developed, but such high quality tablets already exist and can be purchased for $1,000 or less. This could also be the first application where transparent displays could really shine. – NH
