Contribution
Investigating Text-to-Audio Latent Diffusion Model for Industrial Audio Synthesis
* Presenting author
Abstract:
Industrial sound datasets suffer from scarcity and class imbalance, limiting anomaly detection and diagnostic model development. This work adapts the top-performing text-to-audio latent diffusion model from DCASE2023 Task 7 (Foley Sound Synthesis) to industrial machinery sound generation using the DCASE2020 Task 2 development dataset. We implement and train the existing model architecture comprising of a variational autoencoder (VAE), diffusion UNet with conditioning mechanism for iterative denoising. Text prompts describing machine classes (e.g., "sound of toy conveyor") are encoded as 512-dimensional embeddings via pretrained contrastive language-audio pretraining (CLAP) and injected into the denoising process through attention layers, enabling class-specific sound generation. The system performs iterative denoising in the latent space before reconstructing audio using a pretrained HiFi-GAN vocoder. Evaluation employs Fréchet Audio Distance (FAD) alongside mel spectrogram inspection. Classifier-free guidance scaling (ω = 1.0–7.5) controls generation diversity while maintaining class fidelity. Training on industrial samples across six machine types demonstrates that modern text-to audio style architecture, though originally designed for general audio synthesis, can be adapted to specialized industrial domains. This work enables data augmentation for class balanced training and provides a foundation for future extensions to multi-condition modelling for predictive maintenance.