by Microsoft:
GE Healthcare manufactures many of its most complex medical equipment, such as CT and MRI scanners, in China. The maintenance of these sophisticated equipment requires that service engineers undergo an intensive training and certification process. Working with Microsoft MRPP partner Adjacentech Corporation, GE Healthcare developed a mixed-reality training system that uses HoloLens 2 to improve training efficiency and expedite knowledge retention.
With Microsoft HoloLens 2 and Azure Mixed Reality, we’ve been able to give students hands-on practice with less risk to the equipment or themselves. Hui Gao: Head of Technical Operation, GE Healthcare
GE Healthcare in China manufactures costly and sophisticated medical equipment, such as CT (Computerized Tomography) and MRI (Magnetic Resonance Imaging) scanners, which make modern medical diagnostics possible. These complex devices, which are used by healthcare facilities around the world, require highly trained engineers for maintenance and repair. GE Healthcare has been exploring new training models and applying new technology to improve training effectiveness and efficiency.
“Engineer technical competency is the key to offering good service to the customer,” says Hui Gao, Head of Technical Operation, GE Healthcare China Service. “Training is one of the most important ways to build competency. The traditional methods of training proved to have some limitations when it comes to helping the engineers master the complex medical equipment, so we were very willing to try any new technology to improve the training.”
Mixed reality offers a training solution
GE Healthcare has more than 1,000 engineers in the field in China. They need to be able to troubleshoot for customers, and if they’re new to the company, they need to learn quickly.” The simulated training modules would give engineers accurate step-by-step guidance, along with holographic 3D visualizations overlaid atop real-world objects, such as medical equipment, to help them understand signal flow or critical maintenance procedures on extremely complex machines, including the Voyager MRI scanner and the Revolution CT scanner.
As an example of a real-world challenge, each Revolution CT scanner has 52 “detector modules” that receive the attenuated X-ray beam. These detector modules are very delicate components. “They’re also expensive, and without good training, it’s very easy to damage a part during service procedures.”
The Voyager MRI also has an intricately wired system cabinet, analogous to a stack of servers in an IT center. Even in a hands-on environment, it’s difficult to understand the connections and how different pieces link together. “Sometimes the theory is not easy for students to understand. And the signal flow—how the signal goes from this port to another port to this circuit board to another board—is difficult to show”.
A complete mixed reality toolkit
To show GE Healthcare a new approach, the develop team demonstrated the HoloLens 2’s capabilities and showed GE Healthcare how Azure’s mixed reality components blend the physical and digital worlds. These components include Azure Spatial Anchors, Azure Remote Rendering, and Azure Object Anchors. With these tools, developers can create interactive 3D visualizations with a high degree of detail.
Two use cases selected by GE Healthcare
After a successful pilot, GE Healthcare proceeded with two use cases.
The first was for the Voyager MRI system cabinet training. Here, the training program overlays holographic schematics and animated connection diagrams onto the real cabinet. HoloLens 2 can identify and locate the components and ports in the real system cabinet when a trainee stands in front of it. The overlay, together with AI-based narration, functions as a virtual 1:1 coach to speed up the learning process.
The second use case was for detector module replacement in the Revolution CT scanner. With a high-fidelity holographic model rendered in HoloLens 2, training is no longer limited to a dedicated classroom. An engineer can observe and learn the process of detector module replacement anywhere, without any risk of damaging sensitive components.
Faster training, increased effectiveness, lower risk
GE Healthcare has seen substantial benefits from deploying these mixed reality solutions and other new technology within the training.
In the two current use cases, an average of four hours of training are saved per trainee.
With reduced training hours, trainees get more hands-on lab time to practice, which can further deepen their understanding and application of key knowledge and service procedures.
GE Healthcare Training Facilities house more than $20 million in medical equipment in China. The risk of incidental equipment damage can be greatly reduced as more mixed reality and HoloLens 2 solutions are used as a method of hands-on lab training.
A mixed reality future
GE Healthcare believes it has only scratched the surface of mixed reality’s potential. “This is the future,” says David Xiao, Instructor Manager for GE Healthcare. “I think this technology will have a very broad use in a wide range of industries.”
Adds Wei Meng, MRI Instructor for GE Healthcare, “We believe mixed reality may enhance traditional training and give students a faster, easier way to learn the technology. Virtual reality is typically video-based, but with mixed reality and HoloLens 2, we can provide a true interactive and immersive experience.”
The Voyager MRI scanner is a very large piece of equipment, filling three rooms. With the HoloLens 2, students can get a view of the entire apparatus without having to move from room to room.