Some of our current projects involve markerless patient registration, 3D visualization of CT data and models directly within the patient, multi-user learning experiences, 3D databases, inside-out tracking of patients and surgical instruments, and ongoing work on an augmented-reality-assisted ultrasound scanning approach with artificial intelligence support. Our vision encompasses a promising future where we bring visibility to the previously imperceptible, illuminating the unseen. While we cannot showcase all of our current projects online, we are excited to share a glimpse into our work.
3D Surgical Tool Collection
Over 100 surgical instruments from clinical routine, captured with our structured-light 3D scanners for instrument detection and tracking, medical simulation and training in …
3D Visualization of CT and Segmented Models
An AR technique that visualizes CT data and segmented models directly within the patient, translating 2D CT slices into a 3D representation for diagnosis and treatment planning.
DREAMING Challenge
Diminished Reality for Emerging Applications in Medicine through Inpainting — a challenge we hosted at ISBI 2024 on virtually removing real objects from the surgical field.
HINTS Eye Tracking
Diagnosing the cause of dizziness with the built-in eye tracker of the HoloLens 2, using traditional and machine learning methods to improve — or replace — the current HINTS exam.
Instrument Tracking
Highly accurate inside-out tracking of surgical instruments with the sensors of the HoloLens 2 and infrared reflective markers — surgical navigation without additional expensive …
Markerless Patient Registration
An automated markerless method that registers CT data and segmented models to the patient, giving surgeons a direct 2D-to-3D mapping without dedicated navigation hardware.
MedShapeNet
A collection of medical shapes — surgical instruments, human anatomy and pathology — with over 100,000 shapes across 23 datasets, accessible via web interface and Python API.
Multi-User Learning Experiences
Immersive multi-user experiences for interactive education and training of medical students — and for multidisciplinary teams discussing a treatment plan without sharing a room.
