Monitoring Vocal Tract and face movements for 3D modeling and animation for Turkish


Kulak Kayıkcı M. E. (Executive), Gürçay H., Karaosmanoğlu A., Arıtan S.

TUBITAK Project, 2018 - 2021

  • Project Type: TUBITAK Project
  • Begin Date: April 2018
  • End Date: April 2021

Project Abstract

It is very important to obtain accurate data about the movements and shapes of vocal tract and articulator organs during speech in terms of phonetic studies. This data is the basis for the treatment of speech disorders.

In recent years, the use of MRI has become very important in studies about speech production. During speaking, dynamic information about the vocal tract and the movement of the articulator organs can be collected via MRI. The acoustic noise generated during the acquisition of MRI data makes it difficult to record the acoustic signal (speaking voice produced by the person) and to synchronize with the MR image. In addition, the recordings obtained with MRI include vocal tract and articulator organs movement during speech which normally can not be observed externally. With the MR microphone, microphone-camera synchronization set and MR-compatible sound-music system used in this study, it is ensured that acoustic signal is obtained simultaneously with image and acoustic noise suppression. By mounting 2 MR Bore cameras on the MRI device, movement changes of the mouth and around the mouth can also be recorded simultaneously during speaking. Under the Motion Capture (MOCAP) cameras, markers placed on the person's face, the face movements recorded during speaking. While using MRI and MOCAP, target Turkish sounds and words recorded by observing the changes in motion and shape that occured. Data from MRI and MOCAP recordings processed for 3D modeling and animation of vocal tract and articulator organs. With the 3D modeling and animation created, the movement and shape changes of the speech sounds used in Turkish during the speech are visible, eliminating a very important deficiency regarding the visualization of the production of Turkish speech sounds. In addition, by recording the movements and shape changes of the vocal tract and articulatory organs in real time using MRI and MOCAP cameras, it will make significant contributions to the detection of articulation movements that occur in healthy individuals and related speech disorders during speech and their use in speech therapy.