Hooray – Exploring the Performance Boundaries of a Head Worn Microphone-Array for Deep Learning based Dynamic Acoustic Scene Analysis

This research project explores the use of head-mounted micro-electro-mechanical systems (MEMS) microphone arrays for direction of arrival (DOA) estimation using deep learning models. Advances in MEMS technology, especially in size and sound quality, allow these microphones to be integrated into devices such as augmented/mixed/virtual reality (XR) glasses, enhancing applications such as hearing aids, hearing protection, and entertainment. A key area of focus is how head motion, in particular the variation of the head-above-torso orientation (HATO) affects the accuracy of sound source localization. To gain insight into the decision making of deep learning models, interpretability techniques, such as layer-wise relevance propagation (LRP), will be applied on the trained models. The primary goals are to evaluate the performance of head-mounted arrays compared to state-of-the-art sound localization methods, e.g. based on spherical microphone arrays or binaural signals from a dummy head, and to determine the microphone quantities required to achieve comparable performance. This analysis involves both generating and evaluating synthetic data sets as well as testing in real-world scenarios.

As part of the Hooray research project, the Low-Cost Open-Source Head Motorization (LoCOMo) Kit for the KEMAR head and torso simulator was developed and published (see go.lu-h.de/locomo). The kit and the evaluation of its acoustic impact on measured HRTFs were presented at the 155th Convention of the Audio Engineering Society:

N. Poschadel, S. Preihs, and J. Peissig, “LoCOMo: A Low-Cost Open-Source Head Motorization Kit,” presented at the 155th Convention of the Audio Engineering Society, http://www.aes.org/e-lib/browse.cfm?elib=22255 (2023 Oct.).