Tact-Morph: Tactile Sensor & Robotics Processing on the SpiNNaker2 Neuromorphic Compute Platform

Mentored by Christian Mayr, Roberto Calandra
at TU Dresden

In tactile sensor processing, the need for data processing in real-time is of utmost importance, as touch is an inherently real-time capability of humans. This is currently limited by the availability of AI hardware that is sufficiently performant at real time, especially as the (quite high) data rate of the tactile sensor should be processed locally, with limited power budget and with low latency.

Neuromorphic computation is considered as one of the most promising approaches to overcome the speed and power limitations of current DNN hardware. The large-scale SpiNNaker2 neuromorphic system employs a massive amount of ARM M4f microcontrollers (153 per chip, 5Mio in overall machine) that are embedded in a slim, low-latency communication fabric inspired from neuronal connections in the brain. It is a generally programmable computing substrate, but greatly differs from classical architectures like CPUs or GPUs, allowing for more distributed and asynchronous processing and simulation approaches. Thus, the SpiNNaker2 architecture with its highly parallel and distributed processing architecture and focus on millisecond latency is a promising candidate for real-time, tactile data processing from edge to cloud.

A tentative workplan could be thus: (1) Analyze tactile sensor data from real-world robotics applications and optimize contemporary machine learning methods to match the spatio-temporal properties of the emerging modality of touch. Refine existing algorithms for efficient preprocessing + features extraction of tactile sensor data. (2) Devise novel machine learning methods such as event-based neural networks for tactile sensing that leverage the dynamic parallelization offered by the SpiNNaker2 Platform. Research and implement optimal parallelization strategies for the developed algorithms on the SpiNNaker2 Platform. (3) Demonstrate the real-time capability of the devised system in robotic systems. (Optional) Develop a system architecture for a dedicated preprocessing chip, meeting the requirements on data processing and limited power consumption in an embedded tactile sensor.

Work Environment

You will be working jointly at the Chair of Highly-Parallel VLSI Systems and Neuromicroelectronics (HPSN) Dresden and at the Learning, Adaptive Systems and Robotics (LASR) Lab, both TU Dresden. At HPSN, you will work with the largest neuromorphic and real-time AI machine currently existing, the 5Mio core, 8 rack SpiNNaker2 machine. Research at HPSN focusses on real-time AI, as well as neuromorphic hardware and algorithms, plus miscellaneous circuit design topics. At the LASR, we focus on research at the conjunction between machine learning and robotics, with the goal of making robots more adaptable and useful in the real world. In the last years, we have developed a family of high-resolution touch sensors for various robotics and sensing applications, including DIGIT – currently most used touch sensor in robotics. These tactile sensors offer a unique research platform to develop the next generation of tactile enhanced robotics and human assistance, while at the same time investigating foundational questions surrounding the representation and processing of touch as a sensing modality. By applying novel machine learning algorithms on the SpiNNaker2 real time AI platform to these groundbreaking tactile sensors, you will unlock a whole new level of touch-based man-machine interaction.

SECAI offers a first-class environment for advancing your career. You can work with internationally renowned researchers and benefit from the school’s strong networks in industry and research. The graduation of highly qualified researchers is a central project goal in SECAI and doctoral students receive strong support for their professional and personal development.

Prerequisites

• Master’s Degree (or equivalent) in computer science, electrical engineering, applied mathematics, physics or related fields of expertise.
• Very good programming skills (e.g. C++, Python, PyTorch)
• Excellent skills and practical experience in one or more of the following research areas is beneficial:
  • Machine Learning, esp Hardware/Embedded Systems
  • Tactile/Sensor processing
  • Neuromorphic Systems
  • Mathematical Modelling
• Ability to collaborate well in an interdisciplinary environment
• Fluency in technical and non-technical English
• A high degree of independence, commitment, team spirit, and good communication skills