Register now After registration you will be able to apply for this opportunity online.
Computationally Efficient Neural Networks
Computing, time, and energy requirements of recent neural networks have demonstrated dramatic increase over time, impacting on their applicability in real-world contexts. The present thesis explores novel ways of implementing neural network implementations that will substantially reduce their computational complexity and thus energy footprint.
Over the past decade, advances in deep learning and computer vision have led to substantial improvements in robotic perception abilities. It is nowadays possible to use neural networks for reliable object detection, object pose and shape estimation, open-vocabulary semantic interpretation, and the solution of low-level problems such as feature tracking and depth estimation, to name just a few. However, a limiting factor of growing concern is the computational complexity and thus the power consumption/computing hardware vs. latency trade-off of such models. We are therefore also experiencing an increasing demand for cloud-based computation, often with remaining and unpredictable latencies.
As demonstrated through a number of past efforts [2,3,4], the computational complexity of a neural network can be reduced fairly substantially by changes in the selected low-level computing paradigm. Rather than relying on standard matrix-vector multiplications that make use of hardware multipliers, we can choose architectures that will rely more on additive operations [2], thereby reducing computational complexity and thus energy consumption by a substantial amount. The present work aims at the development and testing of novel and efficient network implementations that can be applied to any off-the-shelf network when deployed in custom-programmable hardware.
The proposed thesis will be conducted at the Robotics and AI Institute, a new top-notch partner institute of Boston Dynamics pushing the boundaries of control and perception in robotics. Selection is highly competitive. Potential candidates are invited to submit their CV and grade sheet, after which students will be invited to an on-site interview.
[1] On global electricity usage of communication technology: Trends to 2030, Challenges 6(1), 117-157, 2015
[2] The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits, https://arxiv.org/pdf/2402.17764
[3] XOR-Net: An Efficient Computation Pipeline for Binary Neural Network Inference on Edge Devices,
https://cmu-odml.github.io/papers/XOR-Net_An_Efficient_Computation_Pipeline_for_Binary_Neural_Network_Inference_on_Edge_Devices.pdf
[4] DeepSeek-V3 Technical Report, https://arxiv.org/abs/2412.19437
Over the past decade, advances in deep learning and computer vision have led to substantial improvements in robotic perception abilities. It is nowadays possible to use neural networks for reliable object detection, object pose and shape estimation, open-vocabulary semantic interpretation, and the solution of low-level problems such as feature tracking and depth estimation, to name just a few. However, a limiting factor of growing concern is the computational complexity and thus the power consumption/computing hardware vs. latency trade-off of such models. We are therefore also experiencing an increasing demand for cloud-based computation, often with remaining and unpredictable latencies.
As demonstrated through a number of past efforts [2,3,4], the computational complexity of a neural network can be reduced fairly substantially by changes in the selected low-level computing paradigm. Rather than relying on standard matrix-vector multiplications that make use of hardware multipliers, we can choose architectures that will rely more on additive operations [2], thereby reducing computational complexity and thus energy consumption by a substantial amount. The present work aims at the development and testing of novel and efficient network implementations that can be applied to any off-the-shelf network when deployed in custom-programmable hardware.
The proposed thesis will be conducted at the Robotics and AI Institute, a new top-notch partner institute of Boston Dynamics pushing the boundaries of control and perception in robotics. Selection is highly competitive. Potential candidates are invited to submit their CV and grade sheet, after which students will be invited to an on-site interview.
[1] On global electricity usage of communication technology: Trends to 2030, Challenges 6(1), 117-157, 2015
[2] The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits, https://arxiv.org/pdf/2402.17764
[3] XOR-Net: An Efficient Computation Pipeline for Binary Neural Network Inference on Edge Devices, https://cmu-odml.github.io/papers/XOR-Net_An_Efficient_Computation_Pipeline_for_Binary_Neural_Network_Inference_on_Edge_Devices.pdf
● Literature research
● Development of loss-less post-training adaptations for reducing the computational complexity of neural networks
● Optimization of computational efficiency of standard architectures such as CNNs, MLPs, and Transformers
● Testing in simulation
● Optional: Testing on custom programmable hardware
● Literature research
● Development of loss-less post-training adaptations for reducing the computational complexity of neural networks
● Optimization of computational efficiency of standard architectures such as CNNs, MLPs, and Transformers
● Testing in simulation
● Optional: Testing on custom programmable hardware
Excellent knowledge in either C++ or Python
Knowledge in deep learning
Experience in computer vision
Excellent knowledge in either C++ or Python
Knowledge in deep learning
Experience in computer vision
Laurent Kneip (lkneip@theaiinstitute.com)
Alex Liniger (aliniger@theaiinstitute.com)
Please include your CV and up-to-date transcript when applying
Laurent Kneip (lkneip@theaiinstitute.com)
Alex Liniger (aliniger@theaiinstitute.com)
Please include your CV and up-to-date transcript when applying
202502_Computationally_Efficient_neural_networks.pdf
