tinyML Talks: tinyMLPerf: Deep Learning Benchmarks for Embedded Devices


March 16, 2021



Contact us



Timezone: PDT

tinyMLPerf: Deep Learning Benchmarks for Embedded Devices

Vijay JANAPA REDDI, Associate Professor

Harvard University

Tiny machine learning (ML) is poised to drive enormous growth within the IoT hardware and software industry. Measuring the performance of these rapidly proliferating systems and comparing them in a meaningful way presents a considerable challenge; the complexity and dynamicity of the field obscure the measurement of progress and make embedded ML application and system design and deployment intractable. To enable more systematic development while fostering innovation, we need a fair, replicable, and robust method of evaluating tinyML systems. The talk introduces TinyMLPerf, a machine learning benchmark for ultra-low-power systems that measures both performance and energy. TinyMLPerf extends the existing MLPerf benchmark suite from MLCommons (mlcommons.org) to include tinyML systems. The benchmark suite consists of four ML tasks: small vocabulary keyword spotting, binary image classification, small image classification, and detecting anomalies in machine operating sounds. Thanks to a collaboration with EEMBC, it also features power measurement, which is critical in evaluating tinyML systems. TinyMLPerf will enable device makers and researchers to choose the best hardware for their use cases and provides hardware and software vendors to showcase their offerings. Additional details can be found here: https://github.com/mlcommons/tiny.

Vijay JANAPA REDDI, Associate Professor

Harvard University

Vijay Janapa Reddi is an Associate Professor at Harvard University, Inference Co-chair for MLPerf, and a founding member of MLCommons, a nonprofit ML organization that aims to accelerate ML innovation. He also serves on the MLCommons board of directors.

Schedule subject to change without notice.