Contribution
Distilling Efficient Audio Models using Data Pruning with CLAP
* Presenting author
Abstract:
Running state-of-the-art large-scale audio models on edge devices is often infeasible due to their limited storage and computing resources. It is therefore necessary to compress and tune the models for the specific target task and hardware.This is commonly achieved by distilling the audio model, the teacher, to a small target model, the student. However, this approach can be improved by prepending a data set pruning stage and training the teacher on the pruned data set only which contains examples relevant to the target task.Recently, CLAP models have emerged, which embed audio and text examples into a common embedding space. We use the audio embeddings of the CLAP model for the above pruning stage, which is realized using a domain classifier.After knowledge distillation, the student is eventually fine-tuned on some data from the target domain.The CLAP architecture combines text and audio embedding spaces, which allows to search for data given only a textual description, such as a class label. We show how the presented method of compression and domain adaptation can be applied to such a zero-shot setting.