Audio samples from "HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis"

Abstract: Several recent work on speech synthesis have employed generative adversarial networks (GANs) to produce raw waveforms. Although such methods improve the sampling efficiency and memory usage, their sample quality has not yet reached that of autoregressive and flow-based generative models. In this work, we propose HiFi-GAN, which achieves both efficient and high-fidelity speech synthesis. As speech audio consists of sinusoidal signals with various periods, we demonstrate that modeling periodic patterns of an audio is crucial for enhancing sample quality. A subjective human evaluation (mean opinion score, MOS) of a single speaker dataset indicates that our proposed method demonstrates similarity to human quality while generating 22.05 kHz high-fidelity audio 167.9 times faster than real-time on a single V100 GPU. We further show the generality of HiFi-GAN to the mel-spectrogram inversion of unseen speakers and end-to-end speech synthesis. Finally, a small footprint version of HiFi-GAN generates samples 13.4 times faster than real-time on CPU with comparable quality to an autoregressive counterpart.

For more details of our work, please refer to the paper.
Our implementation is available in the github repository.

Contents

 

Single Speaker (LJ Speech Dataset)
Ground Truth
WaveNet (MoL)
WaveGlow
MelGAN
HiFi-GAN V1 (ours)
HiFi-GAN V2 (ours)
HiFi-GAN V3 (ours)


Unseen Speakers (VCTK Dataset)
Ground Truth
WaveNet (MoL)
WaveGlow
MelGAN
HiFi-GAN V1 (ours)
HiFi-GAN V2 (ours)
HiFi-GAN V3 (ours)


End-to-end Speech Synthesis (LJ Speech Dataset)
Ground Truth
WaveGlow
(fine-tuned)
HiFi-GAN V1 (ours)
(fine-tuned)
HiFi-GAN V2 (ours)
(fine-tuned)
HiFi-GAN V3 (ours)
(fine-tuned)
WaveGlow
(w/o fine-tuning)
HiFi-GAN V1 (ours)
(w/o fine-tuning)
HiFi-GAN V2 (ours)
(w/o fine-tuning)
HiFi-GAN V3 (ours)
(w/o fine-tuning)


Ablation Studies (LJ Speech Dataset)
baseline
w/o MPD
w/o MSD
w/o MRF
w/o Mel-Spectrogram Loss
MPD p=[2,4,8,16,32]
MelGAN with MPD
MelGAN
HiFi-GAN V1 (500k step)