The smartXR Lab equips our research for patient care. We have the following devices:
The lab was presented during the symposium of the Zentrum für virtuelle und erweiterte Realität in der Medizin (ZvRM). Watch the presentation on YouTube.
Our structured-light scanners are the Artec Leo and the AutoScan Inspec. The Artec Leo is a handheld scanner, easy to use and real-time capable, that quickly captures detailed 3D models with high precision. It is well suited for scanning large objects like limbs or even entire rooms.
The AutoScan Inspec, on the other hand, is a tabletop scanner designed for industrial quality control and inspection. Its high-precision scans allow accurate measurement and analysis of complex and small surgical instruments, such as tweezers, scissors, or blades.
Both scanners use structured light technology, making them safe to use on human subjects. Additionally, we provide 3D scanning spray for highly reflective and shiny objects to optimize surface scan quality.
Our resin printer is a stereolithography (SLA) printer from Creality that prints precise, detailed 3D models.
Stereolithography is an additive manufacturing process that uses UV light to selectively harden the resin layer by layer. It can print parts with small features, tight tolerance requirements, and smooth surface finishes. We use the printer to make replicas of our 3D structured-light-scanned objects, CTA/MRI-based surface models, manually created cutting guides or AI-generated patient-specific implants, accurately and within days. For even more rapid prototyping we plan to permanently install an FDM printer as well.
Our smartXR Lab provides a range of mixed reality devices, each with its own capabilities, with a focus on multi-user applications. Mixed reality entails the fusion of the real and virtual worlds, achieved through two approaches: immersive optical-pass-through (mixed reality) and optical-see-through (augmented reality). By combining real and virtual elements, mixed reality goes beyond the capabilities of virtual reality.
Our lab features multiple HTC VIVE Pro 1 and 2, which use base station infrared scanners for user tracking, while the head-mounted display (HMD) captures the surrounding environment. These devices allow for superimposing filmed and registered reality onto the virtual world. The high-end Varjo XR-3 mixed reality device does not require a base station and also incorporates hand tracking.
Besides the HTC VIVE we have acquired 16 Meta Quest 3 mixed reality headsets, which are versatile, wireless, and have a great price/performance ratio, making them ideal for development and research in a clinical setting.
Additionally, we have 10 HoloLens 2 (HL2) devices, which offer optical-see-through functionality. With HL2, users experience minimal discomfort typically associated with mixed reality devices, as the real world is still perceptible through their eyes. These wireless devices include hand tracking, simultaneous localization and mapping technology, and an onboard computer.
Our latest additions are two Apple Vision Pro units. With them, we develop for Android, Microsoft and Apple based devices.
Our smartXR Lab offers the NDI Aurora and 3D Guidance trakSTAR systems, which generate an electromagnetic field and measure changes within it, providing 3D tracking of devices based on these changes. Both track micro sensors embedded in a wide range of medical instruments (ultrasound probes, endoscopes, catheters, guidewires, and even the tip of a needle) in real time and through complex anatomical structures. When integrated into original equipment manufacturer image-guided surgery or interventional systems, the Aurora links patient image sets to the 3D space by localizing the relative positions and orientations of instruments within the operative field. We plan to combine the devices with our mixed and augmented reality headsets, to use them for assessing stand-alone tracking systems, and to keep our research close to current hospital standards.
For motion capture in mixed reality we use the NDI Polaris Vega and an OptiTrack system with the latest PrimeX41 cameras. These systems track spherical infrared-reflective markers by triangulation between at least two cameras. The NDI Polaris Vega is a portable system with high accuracy. The OptiTrack cameras capture live motion within our smartXR Lab at submillimetre accuracy. This brings real-world motion into medical simulations, interactive applications and motion analysis.
The GE Healthcare Ultrasound Scanner Logiq E10 is the lab’s ultrasound system. Its API will allow integration with our Artificial Intelligence and Mixed Reality solutions, as well as our tracking systems, for interactive visualization and real-time guidance. Equipped with an electromagnetic tracking system for the ultrasound probe, the Logiq E10 delivers precise imaging during procedures. Its limited 3D scanning and CT/MRI fusion capabilities are a foundation we aim to expand in the projects to come, along with needle tracking and further features.
Our phantoms are the models on which we develop, train and demonstrate our medical Mixed Reality. With our 3D printers, we can create (miniature) models for our research. More often we use our larger-scale Karstadt models, Ms. and Mr. White, as well as our cranio-maxillofacial surgery phantom from GTSimulators.
Both Mr. White and our surgery phantom have undergone high-dose CT scans, so their imaging data can be used in our Mixed Reality applications. We plan to expand our set of phantoms, particularly for ultrasound applications, and to pair them with our Mixed Reality devices and tracking systems.
