Autonomous motion recognition by combining reinforcement learning and hidden Markov model

Ken'ichi Morooka, Kazuhisa Hamamoto, Hiroshi Nagahashi

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)


A robot needs the abilities of recognizing motion in the world ("other-motion"), and generating "self-motion" to adaptively behave in a real environment. We have been currently developing a system composed of an "other-motion" recognition module and a "self-motion" generation module. This paper focuses on "other-motion" recognition that is based on "self-motion." The recognition and generation modules are each constructed by reinforcement learning and a Hidden Markov Model (HMM). In this case, the HMM estimation needs many sample data of the motion to be learned. However, there is no guarantee that a sufficient amount of motion data can be acquired in the real world, and the reliability of the HMM may therefore be low. In order to solve this problem, this paper presents a new estimation method of an HMM based on the learning results of reinforcement learning. The state value function of the reinforcement learning is divided into some clusters, and each cluster is made to correspond to a state of the HMM. An output distribution can thereby be estimated on the basis of the value of the value function. Some experimental results show that our method can estimate HMM's model parameters not only from few sample data but also from value functions of the generation module, and that the reliability of the estimated HMM can be improved.

Original languageEnglish
Pages (from-to)34-43
Number of pages10
JournalSystems and Computers in Japan
Issue number14
Publication statusPublished - Dec 2006
Externally publishedYes


  • Hidden Markov model
  • Motion generation
  • Motion recognition
  • Reinforcement learning

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Information Systems
  • Hardware and Architecture
  • Computational Theory and Mathematics


Dive into the research topics of 'Autonomous motion recognition by combining reinforcement learning and hidden Markov model'. Together they form a unique fingerprint.

Cite this