SAVi: Semantic Analysis of Surgical Videos

Mentored by Stefanie Speidel, Bjoern Andres
at National Center for Tumor Diseases (NCT)

A key challenge in surgical data science (SDS) is accurately understanding surgical workflows and assessing surgical skills. This task is often done via applying machine learning to surgical video analysis tasks. This requires large amounts of annotated data, due to the complexity and variability of surgical procedures. A promising approach to addressing this challenge consists in leveraging machine learning-based methods for full-scene segmentation to enhance the interpretability, translatability, and generalizability of surgical workflow analysis. By integrating a structured understanding of the surgical context, such methods can provide robust insights for downstream tasks such as surgical phase recognition and skill evaluation.

The PhD topic focuses on the development of machine learning models for surgical video analysis, with a strong emphasis on semantic understanding. Responsibilities that are central to this endeavor include:

• Develop and optimize deep learning models for surgical image and video analysis
• Implement semantic segmentation techniques to add meaning to surgical scenes
• Investigate the impact of including semantics on surgical workflow analysis tasks
• Explore methods for surgical skill assessment based on video-derived data
• Collaborate with interdisciplinary teams, including surgeons and computer scientists
• Publish and present findings in international conferences/journals

The candidate will work in a collaborative research environment at the intersection of computer vision, machine learning, and medical informatics, contributing to the development of algorithms that extract and utilize semantic information from surgical videos. The research will explore how these methods impact critical downstream tasks, ensuring that machine learning techniques in surgical data science are both interpretable and clinically relevant.

Work Environment

You will be working at the National Center for Tumor Diseases (NCT) Dresden and the chair of Machine Learning for Computer Vision. The NCT Dresden located at the medical campus combines patient therapy and research under one roof and offers a unique research platform including an experimental operating room and a novel simulation room for robot-assisted surgery. The group researches applied machine learning methods for robot-assisted surgery and surgical data science. The work will be supported by the chair of Machine Learning for Computer Vision at the computer science faculty. The chair is researching computer vision methods for different applications in life sciences and industry.

SECAI offers a first-class environment for advancing your career. You can work with internationally renowned researchers and benefit from the school’s strong networks in industry and research. The graduation of highly qualified researchers is a central project goal in SECAI and doctoral students receive strong support for their professional and personal development.

Prerequisites

• Master’s Degree (or equivalent) in computer science, electrical engineering, applied mathematics or related fields of expertise
• Very good programming skills (e.g. C++, Python)
• Excellent skills and practical experience in one or more of the following research areas is beneficial:
  • Machine Learning
  • Computer vision
• Ability to collaborate well in an interdisciplinary environment