Augmented reality in rehabilitation: a novel deep learning markerless approach for human pose and gait parameters estimation.
3D Human pose estimation (HPE) is a computer vision task that estimates the configuration of human body parts in the 3D space. HPE is the main technology behind markerless motion capture, which is usually preferred to a marker based one as it is more portable, less expensive, and easier to set up. Nevertheless, its accuracy is not adequate enough to be used in clinical applications such as gait analysis, which therefore often remains restricted to specialized clinics equipped with the gold standard marker-based systems. Thus far, HPE from monocular images and gait parameters extraction are being treated as two separate problems; the former is studied in the computer vision domain using deep learning techniques while the latter in biomechanics. Instead, we define and train both problems in the same deep learning framework to reach higher accuracy in both tasks. We have the ambition to run inference directly and in real time on augmented reality glasses. The developed system will therefore be extremely portable and easy to use and will allow the clinicians to intuitively visualize gait parameters through holograms mixed in the real world. Moreover, additional clinical parameters such as the loads acting inside the knee are, so far, calculated offline through complex modelling simulation. Instead, we propose a completely data driven approach to achieve real time performances.
More information: https://researchportal.vub.be/en/projects/augmented-reality-in-rehabilitation-a-novel-deep-learning-markerl
Funding: FWO PhD fellowship strategic basic research Silvia Zaccardi
RERE PIs: Prof. David Beckwée & Prof. Eva Swinnen