Google AI Presents Health Acoustic Representations (HeAR)
A Bioacoustic Foundation Model Designed to Help Researchers Build Models that Can Listen to Human Sounds and Flag Early Signs of Disease
Health acoustics, such as coughs and breathing, contain valuable health information. Utilizing deep learning models for these acoustics can aid in emotion recognition and detecting diseases like Parkinson’s and Alzheimer’s.
Researchers from Google Research and the Center of Infectious Disease Research in Zambia developed HeAR, a scalable deep-learning system based on SSL. HeAR excels across 33 health acoustic tasks from 6 datasets, showing strong transferability and generalization to out-of-distribution data.
HeAR consists of three main components: data curation, general-purpose training for developing an audio encoder, and task-specific evaluation using the trained embeddings. The system encodes two-second audio clips to generate embeddings for downstream tasks, demonstrating superior performance compared to existing models like TRILL, FRILL, and CLAP.
HeAR outperformed other models across 33 tasks on six datasets, achieving the highest mean reciprocal rank (0.708) and ranking first in 17 tasks. It demonstrated robustness and minimal performance variation across different recording devices, especially in challenging datasets like CIDRZ and SpiroSmart.
The study introduced and assessed the HeAR system, which combines a health acoustic event detector with a generative learning-based audio encoder trained on a large dataset of 313.3 million audio clips using masked autoencoders. Future research should explore model fine-tuning, on-device processing, and bias mitigation.
If you want to evolve your company with AI, stay competitive, and use Google AI Presents Health Acoustic Representations (HeAR) to redefine your way of work. For AI KPI management advice, connect with us at hello@itinai.com. Discover how AI can redefine your sales processes and customer engagement. Explore solutions at itinai.com.
