Word Recognition from Continuous Articulatory Movement Time-Series Data using Symbolic Representations

Authors

• Jun Wang, University of Texas at DallasFollow
• Arvind Balasubramanian, University of Texas at DallasFollow
• Luis Mojica de la Vega, University of Texas at DallasFollow
• Jordan R. Green, MGH Institute of Health ProfessionsFollow
• Ashok Samal, University of Nebraska-LincolnFollow
• Balakrishnan Prabhakaran, University of Texas at DallasFollow

Date of this Version

Summer 8-22-2013

Document Type

Article

Citation

Wang, J., Balasubramanian, A., Mojica de La Vega, L., Green, J. R., Samal, A., & Prabhakaran, B. (2013). Word recognition from continuous articulatory movement time-series data using symbolic representations, ACL/ISCA Interspeech Workshop on Speech and Language Processing for Assistive Technologies, Grenoble, France, 119-127.

Comments

Copyright 2013 Association for Computational Linguistics

Abstract

Although still in experimental stage, articulation-based silent speech interfaces may have significant potential for facilitating oral communication in persons with voice and speech problems. An articulation-based silent speech interface converts articulatory movement information to audible words. The complexity of speech production mechanism (e.g., co-articulation) makes the conversion a formidable problem. In this paper, we reported a novel, real-time algorithm for recognizing words from continuous articulatory movements. This approach differed from prior work in that (1) it focused on word-level, rather than phoneme-level; (2) online segmentation and recognition were conducted at the same time; and (3) a symbolic representation (SAX) was used for data reduction in the original articulatory movement time-series. A data set of 5,900 isolated word samples of tongue and lip movements was collected using electromagnetic articulograph from eleven English speakers. The average speaker-dependent recognition accuracy was up to 80.00%, with an average latency of 302 miliseconds for each word prediction. The results demonstrated the effectiveness of our approach and its potential for building a real-time articulation-based silent speech interface for clinical applications. The across-speaker variation of the recognition accuracy was discussed.