Urban areas are expected to grow explosively in the coming years, which requires intelligent decentralized monitoring and control to improve traffic flow and resource allocation. Current cameras and computer-vision algorithms for traffic monitoring exist and perform well, but they require expensive and power-hungry GPU hardware. On a large scale, the power efficiency of neuromorphic systems (brain inspired sensing and computation, ref 1) can save Gigawatts in energy and millions of SEK.
In this project, we will explore the use of event-cameras (ref 2) and train existing spiking neuronal networks (SNN, ref 3) in an existing urban traffic data set recorded in Stockholm. The project will evaluate the trade-off between a networks size (layers, neurons), activity-level, energy consumption, and performance; ideally resulting in recommendations for future real-world applications in traffic monitoring.
In this project, we will explore the use of event-cameras (ref 2) and train existing spiking neuronal networks (SNN, ref 3) in an existing urban traffic data set recorded in Stockholm. The project will evaluate the trade-off between a networks size (layers, neurons), activity-level, energy consumption, and performance; ideally resulting in recommendations for future real-world applications in traffic monitoring.
Tasks
- Implement and understand the trade-off (energy consumption vs. performance) of an SNN to detect traffic participants.
- Evaluate tuning parameters of the SNN and recommend a network for use.
Process
- Read existing literature on SNN and previous (non-spiking) tracking projects
- Familiar yourself with existing libraries to implement SNN (such as SNNTorch or Norse) and the existing data set
- (Re-)implement and train an SNN, study its tracking performance
- Optimize the networks (hyper-)parameter to minimize consumed energy and maximize tracking performance.
- Write report, recommend parameters for existing network implementations
Expected Outcome
A first “proof-of-principle” SNN has been implemented and trained; the data set (spiking input data and labels) exist. The students shall understand and re-consider the current SNN parameters, and re-implement and train multiple networks of different complexity. An assessment of the consumed energy vs. functionality shall be made; ideally minimizing computational resources (neuron count, connectivity, network activity).
Prerequisites
Ideally ML course, programming in Python
Basic understanding of neuronal networks (extension to SNN will be part of this project). Experience in computer vision is a plus, but not required.
KTH Supervisor
Jörg Conradt, conr@kth.se
Relevant reading
A first “proof-of-principle” SNN has been implemented and trained; the data set (spiking input data and labels) exist. The students shall understand and re-consider the current SNN parameters, and re-implement and train multiple networks of different complexity. An assessment of the consumed energy vs. functionality shall be made; ideally minimizing computational resources (neuron count, connectivity, network activity).
Prerequisites
Ideally ML course, programming in Python
Basic understanding of neuronal networks (extension to SNN will be part of this project). Experience in computer vision is a plus, but not required.
KTH Supervisor
Jörg Conradt, conr@kth.se
Relevant reading
- https://www.techtarget.com/searchenterpriseai/definition/neuromorphic-computing
- G. Gallego, et al.,"Event-Based Vision: A Survey" in IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 44, no. 01, pp. 154-180, 2022.
doi: 10.1109/TPAMI.2020.3008413
https://www.computer.org/csdl/journal/tp/2022/01/09138762/1llK3L5znva - Yamazaki K, Vo-Ho VK, Bulsara D, Le N. Spiking Neural Networks and Their Applications: A Review. Brain Sci. 2022 Jun 30;12(7):863. doi: 10.3390/brainsci12070863. PMID: 35884670; PMCID: PMC9313413.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9313413