Style-Adaptive Speech Synthesis Utilizing Supplementary Information

Abstract

Parameter adaptation is necessitated to reduce the mismatch between the training and test conditions caused by speaker variability, channel difference and environmental effects. Particularly, speech data expressing various emotional states and singing voice with different musical notes will show quite different temporal and spectral characteristics compared with those of the reading-style speech. However, it is practically difficult to collect a lot of data from each speaking (or singing) style. For that reason, a small amount of data obtained from the target condition is usually utilized to adapt the parameters of speech recognition or speech synthesis systems. This thesis brings in the performance enhancement of the conventional MLLR adaptation utilizing supplementary information. Since the conventional MLLR could not use the pitch and duration information for the singing voice and the level of emotional intensity for the expressive speech, it degrades the adaptation performance when the conventional MLLR is employed to the adaptation of the singing voice or expressive speech synthesis. Furthermore, the conventional MLLR does not guarantee the performance in case of non-linear mapping because it assumes to a linear mapping between an initial model set and the adaptation data. In the first approach, we extend the conventional MLLR to the factored MLLR (FMLLR) framework where each MLLR parameter is defined as a function of the control parameter vector. And we further improve the performance of FMLLR by extending its structure from the one that is limited by diagonal assumption to generalized form. We also present a training method to estimate the general form of FMLLR parameters based on the expectation-maximization (EM) algorithm and performance evaluation reveals the effectiveness of the proposed technique. To enhance the adaptation performance of the conventional MLLR, we propose a nonlinear generalization of MLLR for HMM-based speech synthesis system. The algorithm performs the nonlinear regression between a mean vector of base model and a corresponding mean vector of adaptation data with the use of kernel methods. Kernels are taken into account both exponential and polynomial forms, and the technique for learning their weights is also presented. One of the advantages of the proposed algorithm is that except for nonlinear mapping of the base model, the rest of operations are linear. Therefore the proposed algorithm can be obtained by solving a system of linear equations in the similar way as the conventional MLLR, which makes an easy implementation, and requires not so much computational loads.