Tact-Morph: Tactile Sensor & Robotics Processing on the SpiNNaker2 Neuromorphic Compute Platform

Mentored by Christian Mayr and Roberto Calandra
at TU Dresden, Chair of Highly-Parallel VLSI Systems and Neuromicroelectronics (HPSN) Dresden and Learning, Adaptive Systems and Robotics (LASR) Lab

In tactile sensor processing, the need for data processing in real-time is of utmost importance, as touch is an inherently real-time capability of humans. This is currently limited by the availability of AI hardware that is sufficiently performant at real time, especially as the (quite high) data rate of the tactile sensor should be processed locally, with limited power budget and with low latency.

Neuromorphic computation is considered as one of the most promising approaches to overcome the speed and power limitations of current DNN hardware. The large-scale SpiNNaker2 neuromorphic system employs a massive amount of ARM M4f microcontrollers (153 per chip, 10Mio in overall machine) that are embedded in a slim, low-latency communication fabric inspired from neuronal connections in the brain. It is a generally programmable computing substrate, but greatly differs from classical architectures like CPUs or GPUs, allowing for more distributed and asynchronous processing and simulation approaches. Thus, the SpiNNaker2 architecture with its highly parallel and distributed processing architecture and focus on millisecond latency is a promising candidate for real-time, tactile data processing from edge to cloud.

A tentative workplan could be thus: (1) Develop algorithms for efficient preprocessing + information extraction (candidates: event-based neural networks, sparse CNN) of tactile sensor data. (2) Implement on SpiNNaker2 platform: develop parallelization approaches for handling high input data rate; evaluate energy efficiency and achievable throughput and latency. (3) Work towards integration into robotic environments. (Optional) Develop a system architecture for a dedicated preprocessing chip, meeting the requirements on data processing and limited power consumption in an embedded tactile sensor.

Work Environment

You will be working jointly at the Chair of Highly-Parallel VLSI Systems and Neuromicroelectronics (HPSN) Dresden and at the Learning, Adaptive Systems and Robotics (LASR) Lab, both TU Dresden. At HPSN, you will work with the largest neuromorphic and real-time AI machine currently existing, the 10Mio core, 16 rack SpiNNaker2 machine. Research at HPSN focusses on real-time AI, as well as neuromorphic hardware and algorithms, plus miscellaneous circuit design topics. At the LASR, we focus on research at the conjunction between machine learning and robotics, with the goal of making robots more adaptable and useful in the real world. In the last years, we have developed a family of high-resolution touch sensors for various robotics and sensing applications, including DIGIT – currently most used touch sensor in robotics. These tactile sensors offers a unique research platform to develop the next generation of tactile enhanced robotics and human assistance, while at the same time investigating foundational question surrounding the representation and processing of touch as a sensing modality. By applying novel machine learning algorithms on the SpiNNaker2 real time AI platform to these groundbreaking tactile sensors, you will unlock a whole new level of touch-based man-machine interaction.

Prerequisites

• Master’s Degree (or equivalent) in computer science, electrical engineering, applied mathematics or related fields of expertise
• Very good programming skills (e.g. C++, Python, PyTorch)
• Excellent skills and practical experience in one or more of the following research areas is beneficial:
  • Machine Learning, esp Hardware/Embedded Systems
  • Tactile/Sensor processing
  • Neuromorphic Systems
• Ability to collaborate well in an interdisciplinary environment

Further details on the requirements and application process can be found in SECAI's announcement for open PhD positions in 2024.