smartXR Lab

Mon, 01 Jan 0001 00:00:00 +0000

Equipped with advanced tools, our smartXR Lab empowers unique and innovative research for patient care. We have the following devices:

Structured Light 3D Scanners (Artec Leo and AutoScan Inspec)
3D Printer (Creality SLA printer)
Mixed/Augmented Reality Glasses (HTC VIVE Pro 1 & 2, Varjo XR-3, Meta Quest 3, Microsoft HoloLens 2, and Apple Vision Pro)
Electromagnetic Tracking Systems (NDI Aurora and NDI 3D Guidance trakSTAR)
Optical Tracking Systems (OptiTrack with PrimeX41 cameras and NDI Polaris Vega)
Ultrasound Scanner (GE Healthcare Logiq E10)
Phantoms (Karstadt models and cranio-maxillofacial surgical phantom)

The lab was presented during the symposium of the Zentrum für virtuelle und erweiterte Realität in der Medizin (ZvRM) — watch the presentation on YouTube.

Structured Light 3D Scanners

Introducing our cutting-edge 3D scanning technology: Artec Leo and AutoScan Inspec. Artec Leo is a handheld scanner known for its ease of use and real-time capabilities, allowing you to quickly capture detailed 3D models with high precision. It is perfect for scanning large objects like limbs or even entire rooms.

AutoScan Inspec, on the other hand, is a tabletop scanner designed for industrial quality control and inspection. With its high-precision scanning capabilities, it ensures accurate measurements and analysis of complex and small surgical instruments, such as tweezers, scissors, or blades.

Both scanners use structured light technology, making them safe to use on human subjects. Additionally, we provide 3D scanning spray for highly reflective and shiny objects to optimize surface scan quality.

From left to right: AutoScan Inspec, Artec Leo and 3D scanning spray from AESUB

3D Printer

Introducing the SLA resin Creality printer: built with advanced stereolithography (SLA) printing technology, this printer offers exceptional precision and detail in creating high-quality 3D models.

Stereolithography is an additive manufacturing process utilizing UV light to selectively harden the resin layer by layer and can print parts with small features, tight tolerance requirements, and smooth surface finishes. This printer allows us to make replicas of our 3D structured-light-scanned objects, CTA/MRI-based surface models, manually created cutting guides or AI-generated patient-specific implants. This allows us to bring our ideas to life accurately and within days. For even more rapid prototyping we plan to permanently install an FDM printer as well.

Creality 3D printer, its basin and print plate

Mixed and Augmented Reality Glasses

Our smartXR Lab provides a wide range of cutting-edge mixed reality devices to enhance multi-user applications. Each device comes with its unique capabilities. Mixed reality entails the fusion of the real and virtual worlds, achieved through two approaches: immersive optical-pass-through (mixed reality) and optical-see-through (augmented reality). By seamlessly combining real and virtual elements, mixed reality takes us beyond the capabilities of virtual reality.

Our lab features multiple HTC VIVE Pro 1 and 2, which utilize base station infrared scanners for user tracking, while the head-mounted display (HMD) captures the surrounding environment. These devices allow for superimposing filmed and registered reality onto the virtual world. For an even more immersive experience, we offer the high-end Varjo XR-3 mixed reality device. Unlike other devices, the XR-3 does not require a base station and also incorporates hand tracking technology, providing an unparalleled blend of both realities.

Besides the HTC VIVE we have acquired 16 Meta Quest 3 mixed reality headsets, which are versatile, wireless, and have a great price/performance ratio, making them ideal for development and research in a clinical setting.

Additionally, we have 10 HoloLens 2 (HL2) devices, which offer optical-see-through functionality. With HL2, users experience minimal discomfort typically associated with mixed reality devices, as the real world is still perceptible through their eyes. These wireless devices include hand tracking, simultaneous localization and mapping technology, and an onboard computer, ensuring a seamless integration of both worlds.

Our latest addition consists of two Apple Vision Pro units, allowing us to stay on top of the latest developments. This allows us to develop for Android, Microsoft and Apple based devices.

From left to right: a HL2 visualizing the insides of the patient, the HL2, HTC VIVE Pro 2 and the Varjo XR-3

Electromagnetic Tracking Systems (EMT)

Our smartXR Lab offers the NDI Aurora and 3D Guidance trakSTAR systems, capable of generating an electromagnetic field and measuring changes within this field, thereby providing 3D tracking of devices based on these changes. These tracking solutions are designed to revolutionize the way medical instruments are tracked and visualized in real time. The Aurora and 3D Guidance trakSTAR electromagnetic tracking solution provides seamless and real-time tracking of micro sensors embedded in a wide range of medical instruments, including ultrasound probes, endoscopes, catheters, guidewires, and even the tip of a needle, enabling precise tracking of instruments through complex anatomical structures. When integrated into original equipment manufacturer image-guided surgery or interventional systems, the Aurora acts as a vital link between patient image sets and the 3D space. It instantly localizes the relative positions and orientations of instruments within the operative field. We plan on incorporating the devices with our mixed and augmented reality devices, frequent use for assessment of stand-alone tracking systems, and to seamlessly intertwine our research with the latest hospital standards.

Both NDI EMT systems and an overview of trackable probes for research

Optical Tracking Systems (OTS)

For advanced motion capture in mixed reality we use the NDI Polaris Vega and an OptiTrack system with the latest PrimeX41 cameras. These systems track spherical infrared-reflective markers by triangulation calculations between at least two cameras. The NDI Polaris Vega, a portable system renowned for its high accuracy, provides seamless and real-time tracking of movement. In addition to the Polaris Vega, the OptiTrack system with PrimeX41 cameras is known for its exceptional performance and reliability for live motion capture. These cameras capture live motion within our smartXR Lab, delivering submillimetre accuracy and precision. With the OptiTrack system, every subtle movement is captured with remarkable fidelity, allowing for seamless integration of real-world motion into mixed reality projects. Whether for medical simulations or interactive experiences, the OTS provides a powerful toolset for motion analysis, enabling us to capture and visualize movements with remarkable realism.

OptiTrack system on the left, on the right a trackable surgical instrument and the NDI Polaris Vega

Ultrasound Scanner

The GE Healthcare Ultrasound Scanner Logiq E10 is a cutting-edge device poised to revolutionize medical imaging. Its API will allow integration with our Artificial Intelligence and Mixed Reality solutions, as well as our advanced tracking systems. This integration will unlock new dimensions of interactive visualization, diagnostic accuracy, and real-time guidance. Equipped with an electromagnetic tracking system for the ultrasound probe, the Logiq E10 ensures precise and accurate imaging during procedures. Its limited 3D scanning and CT/MRI fusion capabilities serve as a solid foundation, which we aim to enhance and expand in the projects to come. Additionally, needle tracking and a multitude of other features await exploration, opening doors to innovative applications and improved patient care. We want to be at the forefront of the future of ultrasound technology with the GE Healthcare Logiq E10.

The Logiq E10 with color doppler to visualize blood flow in the portal vein on the left, on the right the electromagnetic field generator at the top and an overview of ultrasound probes on the bottom

Practice Phantoms

Projects come to life with the help of our phantoms. These models serve as the foundation for developing, training, and showcasing our innovations. With our 3D printers, we can create (miniature) models for our research. However, we frequently use our larger-scale Karstadt models, Ms. and Mr. White, as well as our high-end cranio-maxillofacial surgery phantom from GTSimulators. These phantoms provide an invaluable platform for developing and demonstrating our medical Mixed Reality.

To further elevate our research, both Mr. White and our surgery phantom have undergone high-dose CT scans, enabling us to add a medical component to our Mixed Reality. Looking ahead, we are committed to expanding our repertoire of phantoms, particularly in the realm of ultrasound applications. By integrating these accurate phantoms with our Mixed Reality devices and tracking systems, we can unlock new dimensions of interactive exploration, enhanced visualization, and innovative medical solutions.

Our surgical simulator phantom on the left, in the middle Mr. White with the HL2, on the right an AR-enriched Mr. White

AI-guided Therapies | IKIM MML