Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Objects classification on Tactile Information with Neuromorphic Hardware
This project aims to develop a neuromorphic system for object classification using tactile data, inspired by the human sense of touch. By integrating biomimetic sensors and a neuromorphic chip, the system processes spatiotemporal tactile information with high efficiency and low power consumption. The approach leverages spiking neural networks (SNNs) to encode and shapes in real time. The project focuses on designing algorithms optimized for the unique properties of neuromorphic hardware and evaluating performance in dynamic, real-world scenarios. This work has potential applications in robotics, prosthetics, and intelligent sensing systems, offering an energy-efficient solution for tactile perception tasks.
The project aims to implement an SNN on a Dynapse chip, a neuromorphic platform, for efficient object classification using tactile data. The ETHZ-STAG Dataset, obtained from the Smarthand system, will be used as the benchmark for this study. This dataset includes tactile and IMU data collected at 100 Hz across five sessions, with 16 objects manipulated for 40 seconds each, alongside an additional empty-hand class. In total, the dataset consists of 17 classes and 340,000 frames.
**Data Preprocessing**
The tactile dataset will be analyzed to identify the most critical areas of the hand for tactile sensing, reducing the data to process and optimizing the neuromorphic implementation. Valid frames will be selected by applying a threshold to filter out empty-hand movements, ensuring the dataset captures meaningful object interactions.
**Signal conversion**
To convert the signal into spikes to feed into the chip. A possible conversion algorithm is the asynchronous delta modulator.
**Neuromorphic Implementation**
The SNN will be designed and implemented on the Dynapse chip. This neuromorphic system will process spatiotemporal tactile patterns directly on-chip, leveraging the chip's event-driven architecture to ensure energy-efficient and low-latency computation. The SNN will focus on continuous decoding, enabling object classification and detection of gesture onset and offset.
**Classification Framework**
The project will adopt a hybrid approach:
1. On-Chip Processing: A network will be implemented on the Dynapse chip to process tactile data and extract key features relevant to object classification and gesture dynamics.
2. Offline Readout: A lightweight readout module will be implemented offline to classify all objects in the dataset based on the features generated by the on-chip SNN.
**Available Dataset**
Dataset with references to the paper and to the MIT project: https://iis-people.ee.ethz.ch/~datasets/smarthand/
The project aims to implement an SNN on a Dynapse chip, a neuromorphic platform, for efficient object classification using tactile data. The ETHZ-STAG Dataset, obtained from the Smarthand system, will be used as the benchmark for this study. This dataset includes tactile and IMU data collected at 100 Hz across five sessions, with 16 objects manipulated for 40 seconds each, alongside an additional empty-hand class. In total, the dataset consists of 17 classes and 340,000 frames.
**Data Preprocessing** The tactile dataset will be analyzed to identify the most critical areas of the hand for tactile sensing, reducing the data to process and optimizing the neuromorphic implementation. Valid frames will be selected by applying a threshold to filter out empty-hand movements, ensuring the dataset captures meaningful object interactions.
**Signal conversion** To convert the signal into spikes to feed into the chip. A possible conversion algorithm is the asynchronous delta modulator.
**Neuromorphic Implementation** The SNN will be designed and implemented on the Dynapse chip. This neuromorphic system will process spatiotemporal tactile patterns directly on-chip, leveraging the chip's event-driven architecture to ensure energy-efficient and low-latency computation. The SNN will focus on continuous decoding, enabling object classification and detection of gesture onset and offset.
**Classification Framework** The project will adopt a hybrid approach:
1. On-Chip Processing: A network will be implemented on the Dynapse chip to process tactile data and extract key features relevant to object classification and gesture dynamics. 2. Offline Readout: A lightweight readout module will be implemented offline to classify all objects in the dataset based on the features generated by the on-chip SNN.
**Available Dataset**
Dataset with references to the paper and to the MIT project: https://iis-people.ee.ethz.ch/~datasets/smarthand/
The project aims to achieve accurate classification of the 17 object classes in the ETHZ-STAG Dataset with a target accuracy of at least 85% using the combined on-chip and offline processing approach. The implementation on the Dynapse chip is expected to process tactile data with a latency below 10 ms per frame, demonstrating the efficiency of spiking neural networks for real-time applications.
The project aims to achieve accurate classification of the 17 object classes in the ETHZ-STAG Dataset with a target accuracy of at least 85% using the combined on-chip and offline processing approach. The implementation on the Dynapse chip is expected to process tactile data with a latency below 10 ms per frame, demonstrating the efficiency of spiking neural networks for real-time applications.
