A Text-to-Speech Pipeline, Evaluation Methodology and Initial Fine-Tuning Results for Child Speech Synthesis

ABSTRACT:

Speech synthesis has come a long way as current text-to-speech (TTS) models can now generate natural human-sounding speech. However, most of the TTS research focuses on using adult speech data and there has been very limited work done on child speech synthesis. In this study, a training pipeline for fine-tuning state-of-the-art (SOTA) neural TTS models using child speech datasets was developed and validated. This approach adopts a multispeaker TTS retuning workflow to provide a transfer-learning pipeline. A publicly available child speech dataset was cleaned to provide a smaller subset of approximately 19 hours, which formed the basis of our fine-tuning experiments. Both subjective and objective evaluations were performed using a pretrained MOSNet for objective evaluation and a novel subjective framework for mean opinion score (MOS) evaluations. Subjective evaluations achieved the MOS of 3.92 for speech intelligibility, 3.85 for voice naturalness, and 3.96 for voice consistency. Objective evaluation using a pretrained MOSNet showed a strong correlation between real and synthetic child voices. The final trained model was able to synthesize child-like speech from reference audio samples as short as 5 seconds.

Objctive Evalaution - Audio samples comparison used in MOSNet(Source vs Synthetic)

Subjective Evaluation - Audio samples used for calculating MOS

Source/Reference audio file for each group

Note: Speaker '002113' represent the child speaker ID. We present results from 4 speaker ID used in our paper.

Speaker '002113' Transcript	Speaker '002113' Audio	Speaker '008045' Transcript	Speaker '008045' Audio	Speaker '013020' Transcript	Speaker '013020' Audio	Speaker '995737' Transcript	Speaker '995737' Audio
i think it's a migration behavior because the black thing is stuck on the human's knee		you can see that the white light makes the candle green because without the light it would not be any color		well the dog is just waging it's tail the bee is just like pollinating and the geese are flying		the energy is flowing into the positive side of the d cell and coming out of the negative side of the d cell

Synthetic audio files for each group

Bonus: Tongue Twisters (Source vs Synthetic)

Transcript: Peter Piper picked a peck of pickled peppers. A peck of pickled peppers Peter Piper picked. If Peter Piper picked a peck of pickled peppers. Where’s the peck of pickled peppers Peter Piper picked?