Extended Reality (XR) is earning a lot of ‘street cred’ these days in the medical field. I see creative application and diverse use cases of XR everywhere I tread: in conference presentations, webinars, hospitals (of my grad students), and even the literature that crosses my screen. But in most of the cases I’ve viewed, medical educators and others still seem to struggle to identify meaningful metrics to justify their use and the costs of XR.
Too many practitioners are settling for the pzazz—the ‘cool’ innovation and wow factor that always seems to accompany XR—while less emphasis is placed on meaningful outcomes and precise metrics for measuring the success of this array of technologies. I tackled the ‘metrics’ topic in a past article, How to Score VR for Educational Advantage, and suggested five key metrics for the reader’s consideration, and mention them now:
- Learning Efficiency
- Cost Avoidance
- Accelerating Learning
- Teaching Efficiency
- Increasing Capacity
Alright, so those are some of the most desirable metrics, and they are all defined in the aforementioned article, but what do we do now? Well, what we need now are crystal clear examples of companies that understand and pay attention to these kinds of metrics, and don’t promote XR for less-than-consequential reasons. Enter Precision OS, a Canadian-based XR production facility for medical-grade VR.
With the mission of supporting advanced medical readiness, Precision OS develops highly accurate surgical workflow simulations, and does the best job I’ve seen of boldly addressing the VR performance justification head on. Precision OS takes aim at some lofty goals, mind you, measurable goals like using VR to flatten an increasingly complex learning curve (learning efficiency); helping medical device manufacturers get their devices to market and into use more rapidly with efficient training (increasing capacity); and carving out improved learning opportunities for rural surgeons, thus lessening the need for expensive travel and even more expensive cadavers (cost avoidance).
Additionally, Precision OS does not shy away from evidencing other examples of rich effectiveness metrics. On their web page (education section), they highlight and detail a rich number of studies evidencing such vital metrics as
A “reduction in learning time of 570%”
“Reducing the need for costly operating room time”
“Fewer errors” in the OR leading to “improved outcomes” for patients
“Eight of the 14 residents who participated [in the study] completed their residency training in four years rather than the traditional five. (accelerating learning)
You can see more detail on each of these studies here. This is the kind of measurement thinking that always draws my attention. It would be delightfully worthwhile if the Precision OS team could also elucidate (I haven’t seen one yet) a vibrant example of the teaching efficiency metric (doing more, reaching more in less time, all from the instructor’s perspective) in their collection of studies . One of my grad students, the director of a large OR department in Colorado, recently unpacked for me the familiar teaching mantra of medical educators: “See one, do one, teach one”. So, one wonders whether the use of VR could possibly push that phrase to “See many, do many, teach many”, and if Precision OS or other companies have examples of that. But the Precision OS team gets the big idea. Metrics matter. –Len Scrogan
