Speech-based Medical Decision Support in VR using a Deep Neural Network (Demonstration)

Speech-based Medical Decision Support in VR using a Deep Neural Network (Demonstration)

Alexander Prange, Michael Barz, Daniel Sonntag

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence

We present a speech dialogue system that facilitates medical decision support for doctors in a virtual reality (VR) application. The therapy prediction is based on a recurrent neural network model that incorporates the examination history of patients. A central supervised patient database provides input to our predictive model and allows us, first, to add new examination reports by a pen-based mobile application on-the-fly, and second, to get therapy prediction results in real-time. This demo includes a visualisation of patient records, radiology image data, and the therapy prediction results in VR.
Keywords:
Natural Language Processing: Speech
Multidisciplinary Topics and Applications: Intelligent User Interfaces
Natural Language Processing: Dialogue
Machine Learning: Deep Learning