论文标题
多阶段的多阶段多编码书VQ-VAE方法用于高性能神经TTS
A Multi-Stage Multi-Codebook VQ-VAE Approach to High-Performance Neural TTS
论文作者
论文摘要
我们提出了一种多阶段的多代码书（MSMC）方法，用于高性能神经TTS合成。基于矢量定量的，变分的自动编码器（VQ-VAE）的特征分析仪用于编码语音训练数据的MEL频谱图，通过在多个阶段中逐渐减小为MSMC表示（MSMCR），并分别使用多个VQ Codebook进行量化它们，并分别对其进行定量。通过最大程度地减少重建均方根误差（MSE）和“ Triplet损耗”的合并损失，对多阶段预测指标进行了训练，以逐步将输入文本序列映射到MSMCR。在合成中，神经声码器将预测的MSMCR转换为最终的语音波形。拟议的方法是由女演讲者使用16小时的英语TTS数据库培训和测试的。拟议的TTS的MOS得分为4.41，其表现以3.62的MOS优于基线。较少参数的拟议TT的紧凑型版本仍然可以保留高MOS得分。消融研究表明，多个阶段和多个代码手册都可以有效地达到高TTS性能。
We propose a Multi-Stage, Multi-Codebook (MSMC) approach to high-performance neural TTS synthesis. A vector-quantized, variational autoencoder (VQ-VAE) based feature analyzer is used to encode Mel spectrograms of speech training data by down-sampling progressively in multiple stages into MSMC Representations (MSMCRs) with different time resolutions, and quantizing them with multiple VQ codebooks, respectively. Multi-stage predictors are trained to map the input text sequence to MSMCRs progressively by minimizing a combined loss of the reconstruction Mean Square Error (MSE) and "triplet loss". In synthesis, the neural vocoder converts the predicted MSMCRs into final speech waveforms. The proposed approach is trained and tested with an English TTS database of 16 hours by a female speaker. The proposed TTS achieves an MOS score of 4.41, which outperforms the baseline with an MOS of 3.62. Compact versions of the proposed TTS with much less parameters can still preserve high MOS scores. Ablation studies show that both multiple stages and multiple codebooks are effective for achieving high TTS performance.
