Reinforcement Learning for Mechanical Ventilation

Mentored by Jens Lehmann, Jakob Nikolas Kather, Sahar Vahdati, Jakob Wittenstein
at TU Dresden

Invasive mechanical ventilation (MV) is one of the most frequently applied life-saving therapies in the intensive care unit (ICU). The relevance of MV in intubated patients became even more evident within the pandemic caused by the coronavirus infection (COVID-19). However, clinicians provide the setting of MV manually, which is time-consuming for ICU doctors and nurses and error-prone. Furthermore, inappropriate MV settings can further injure the patient’s lungs and increase the risk of death and prolonged ICU stay.

There are few attempts in providing AI-based decision support system where the clinicians can be assisted by recommendations on the setting of MV to reduce potentially competing risks. Such systems can also reduce the error-proneness of human decisions by a bias-aware algorithm. The studies show that Reinforcement Learning (RL) is the core approach for building such a decision support system. However, the exiting RL models lack mathematical foundations to be adjusted for such a complex and vital problem.

Among the major challenges in design and development of such a system are the large action space, scalability and systematic imputation of missing values. Each of these issues require extended research that is the target of this position.

The goal of this project is to extend the theoretical framework of the exiting RL-based approaches with a focus on Dueling Double Deep Q Network (DDDQN) and Discrete Batch-Constrained Deep Q-learning (BCQ) in terms of large action space and scalability by employing classical high dimensionality reduction methods. In addition, the research includes integration of semantic and interconnected complementary hidden knowledge through external knowledge graphs that will replace the normal imputation methods. Within the designed framework, novel RL-based models will be coupled with learning and reasoning to achieve explainable results.

Work Environment

You will be a member of a collaborative project team working at the cutting edge of Computer Science within the SECAI Project. You will be supervised by Jens Lehmann, and Jakob Nikolas Kather and be directly mentored by Dr. Sahar Vahdati, and Dr. Jakob Wittenstein. You will be contributing to the development of theories, models and algorithms for the comparison and approximation of the decision support system in terms of scalability.

Prerequisites

To conduct this research, you should hold a very good university degree (MSc or an equivalent) in computer science or related disciplines (such as mathematics). You should have background in machine learning. Furthermore, the required background knowledge includes:

Technical Requirements

• Reinforcement Learning Experience and Knowledge
• Knowledge about Continues and Discrete Data Analysis
• Experience in general Data Cleaning and pre-processing
• Experience Data Visualization and Data Management
• Interest (and ideally experience) in pre-processing Medical Data
• Fluent English

Programming languages:

• Python -libraries: PyTorch, Tensorflow, Keras 
• Scala, R (desirable)