A new communication paradigm; Action-to-speech

In this paper, we propose a new technology that enhances the communication ability of human beings. We call it Action-to-Speech (A2S). A2S generates speech from actions or gestures and makes it possible for people who speak different languages or the speaking impaired to communicate. Features of A2S are as follows. (1) Because meanings are defined by mapping functions between action and speech, it is easy for the system to handle new languages. (2) Because prosodic characteristics of speech can be controlled in real time, smooth communication is possible. (3) Because sensors capture actions, reliable responses are guaranteed. A prototype system was developed. Evaluations indicate that A2S makes it possible for persons speaking different languages to easily and effectively communicate.