tinyML Talks: Embedded ML research at TUM: Moving NN Inference to the Extreme Edge


October 7, 2020



Contact us



Timezone: PDT

Embedded ML research at TUM: Moving NN Inference to the Extreme Edge

Rafael STAHL, PhD Candidate

Technical University of Munich

Moving neural network inference near to the data collecting or sensing nodes in IoT networks reduces communication to the cloud, enables always-on devices and improves privacy. The main challenge is the limited amount of computation and memory resources available at these extreme edge IoT devices.
In order to address this challenge we firstly present DeeperThings, a method to enable memory- and compute-constrained devices to run Convolutional Neural Networks (CNNs) in a distributed fashion. The devices join their resources to run CNNs with sizes that otherwise couldn’t fit in each single device’s memory. The inference task is partitioned between all participating devices by utilizing feature partitioning, weight partitioning and communication-aware layer fusion.

Secondly, we present a TinyML code generator tool that transforms a TensorFlow Lite model into static embedded source code. This eliminates the overhead of dynamic interpretation used in the standard TensorFlow Lite for Micro approach and grants increased flexibility for further model optimization such as flexible weight packing support.

Rafael STAHL, PhD Candidate

Technical University of Munich

Rafael Stahl is a doctoral candidate at the Technical University of Munich at the Chair of Electronic Design Automation in his fourth year. He received his Bachelor and Master in “Electrical Engineering and Information Technology” from TU Munich.
He gathered work experience at the Fraunhofer Institute for Embedded Systems and Communication Technology, prototyping demos of connected car systems. Software reverse engineering is a long-time passion of his, that gained him in-depth programming and debugging experience.
Currently he is looking to improve neural network inference through target-aware methods with the wider goal of reducing the memory footprint of embedded software.
He received the Best Paper Award at SiPS 2019.

Schedule subject to change without notice.