Time filter

Source Type

Kruger V.,University of Aalborg | Herzog D.,CVMI | Ude A.,ATR Computational Neuroscience Laboratories | Ude A.,Jozef Stefan Institute
IEEE Robotics and Automation Magazine | Year: 2010

In the area of imitation learning, one of the important research problems is action representation. There has been a growing interest in expressing actions as a combination of meaningful subparts called action primitives. Action primitives could be thought of as elementary building blocks for action representation. In this article, we present a complete concept of learning action primitives to recognize and synthesize actions. One of the main novelties in this work is the detection of primitives in a unified framework, which takes into account objects and actions being applied to them. As the first major contribution, we propose an unsupervised learning approach for action primitives that make use of the human movements as well as object state changes. As the second major contribution, we propose using parametric hidden Markov models (PHMMs) [1], [2] for representing the discovered action primitives. PHMMs represent movement trajectories as a function of their desired effect on the object, and we will discuss 1) how these PHMMs can be trained in an unsupervised manner, 2) how they can be used for synthesizing movements to achieve a desired effect, and 3) how they can be used to recognize an action primitive and the effect from an observed acting human. © 2006 IEEE.

Forte D.,Jozef Stefan Institute | Gams A.,Jozef Stefan Institute | Morimoto J.,ATR Computational Neuroscience Laboratories | Ude A.,Jozef Stefan Institute | Ude A.,ATR Computational Neuroscience Laboratories
Robotics and Autonomous Systems | Year: 2012

Autonomous robots cannot be programmed in advance for all possible situations. Instead, they should be able to generalize the previously acquired knowledge to operate in new situations as they arise. A possible solution to the problem of generalization is to apply statistical methods that can generate useful robot responses in situations for which the robot has not been specifically instructed how to respond. In this paper we propose a methodology for the statistical generalization of the available sensorimotor knowledge in real-time. Example trajectories are generalized by applying Gaussian process regression, using the parameters describing a task as query points into the trajectory database. We show on real-world tasks that the proposed methodology can be integrated into a sensory feedback loop, where the generalization algorithm is applied in real-time to adapt robot motion to the perceived changes of the external world. © 2012 Elsevier B.V. All rights reserved.

Matsubara T.,Nara Institute of Science and Technology | Matsubara T.,ATR Computational Neuroscience Laboratories | Morimoto J.,ATR Computational Neuroscience Laboratories
IEEE Transactions on Biomedical Engineering | Year: 2013

In this study, we propose a multiuser myoelectric interface that can easily adapt to novel users. When a user performs different motions (e.g., grasping and pinching), different electromyography (EMG) signals are measured. When different users perform the same motion (e.g., grasping), different EMG signals are also measured. Therefore, designing a myoelectric interface that can be used by multiple users to perform multiple motions is difficult. To cope with this problem, we propose for EMG signals a bilinear model that is composed of two linear factors: 1) user dependent and 2) motion dependent. By decomposing the EMG signals into these two factors, the extracted motion-dependent factors can be used as user-independent features. We can construct a motion classifier on the extracted feature space to develop the multiuser interface. For novel users, the proposed adaptation method estimates the user-dependent factor through only a few interactions. The bilinear EMG model with the estimated user-dependent factor can extract the user-independent features from the novel user data. We applied our proposed method to a recognition task of five hand gestures for robotic hand control using four-channel EMG signals measured from subject forearms. Our method resulted in 73% accuracy, which was statistically significantly different from the accuracy of standard nonmultiuser interfaces, as the result of a two-sample t -test at a significance level of 1%. © 1964-2012 IEEE.

Yanagisawa T.,Osaka University | Yanagisawa T.,ATR Computational Neuroscience Laboratories | Hirata M.,Osaka University | Saitoh Y.,Osaka University | And 9 more authors.
Annals of Neurology | Year: 2012

Objective: Paralyzed patients may benefit from restoration of movement afforded by prosthetics controlled by electrocorticography (ECoG). Although ECoG shows promising results in human volunteers, it is unclear whether ECoG signals recorded from chronically paralyzed patients provide sufficient motor information, and if they do, whether they can be applied to control a prosthetic. Methods: We recorded ECoG signals from sensorimotor cortices of 12 patients while they executed or attempted to execute 3 to 5 simple hand and elbow movements. Sensorimotor function was severely impaired in 3 patients due to peripheral nervous system lesion or amputation, moderately impaired due to central nervous system lesions sparing the cortex in 4 patients, and normal in 5 patients. Time frequency and decoding analyses were performed with the patients' ECoG signals. Results: In all patients, the high gamma power (80-150Hz) of the ECoG signals during movements was clearly responsive to movement types and provided the best information for classifying different movement types. The classification performance was significantly better than chance in all patients, although differences between ECoG power modulations during different movement types were significantly less in patients with severely impaired motor function. In the impaired patients, cortical representations tended to overlap each other. Finally, using the classification method in real time, a moderately impaired patient and 3 nonparalyzed patients successfully controlled a prosthetic arm. Interpretation: ECoG signals appear useful for prosthetic arm control and may provide clinically feasible motor restoration for patients with paralysis but no injury of the sensorimotor cortex. Copyright © 2011 American Neurological Association.

Izawa J.,Johns Hopkins University | Izawa J.,ATR Computational Neuroscience Laboratories | Pekny S.E.,Johns Hopkins University | Marko M.K.,Johns Hopkins University | And 5 more authors.
Autism Research | Year: 2012

The brain builds an association between action and sensory feedback to predict the sensory consequence of selfgenerated motor commands. This internal model of action is central to our ability to adapt movements and may also play a role in our ability to learn from observing others. Recently, we reported that the spatial generalization patterns that accompany adaptation of reaching movements were distinct in children with autism spectrum disorder (ASD) as compared with typically developing (TD) children. To test whether the generalization patterns are specific to ASD, here, we compared the patterns of adaptation with those in children with attention deficit hyperactivity disorder (ADHD). Consistent with our previous observations, we found that in ASD, the motor memory showed greater than normal generalization in proprioceptive coordinates compared with both TD children and children with ADHD; children with ASD also showed slower rates of adaptation compared with both control groups. Children with ADHD did not show this excessive generalization to the proprioceptive target, but they did show excessive variability in the speed of movements with an increase in the exponential distribution of responses (τ) as compared with both TD children and children with ASD. The results suggest that slower rate of adaptation and anomalous bias towards proprioceptive feedback during motor learning are characteristics of autism, whereas increased variability in execution is a characteristic of ADHD. © 2012 International Society for Autism Research, Wiley Periodicals, Inc.

People with bipolar disorder, also known as manic-depressive disorder, experience extreme fluctuations in mood and behavior, which may occur in cycles lasting for days, months, or years. Such changes were first described more than half a century ago, but their molecular basis in the brain has remained unclear. Now, researchers at the Institute for Comprehensive Medical Science, Fujita Health University, and ATR Computational Neuroscience Laboratories in Japan have succeeded in predicting states of mood-change-like behavior by studying the gene expression patterns in the brain in a bipolar disorder mouse model. Interestingly, so-called circadian genes, whose expressions increase and decrease over a 24-hour cycle, are overrepresented in the prediction gene sets, demonstrating an intrinsic link between circadian genes in the brain and mood change-like behavior. The research appears in the March 29th the journal Cell Reports. Elucidation of the molecular basis of mood changes occurring with an infradian (longer than a day) rhythm has been hampered by the lack of an animal model that exhibits spontaneous behavioral changes related to the infradian oscillation of mood. In the course of screening over 180 mutant mouse strains with a systematic battery of behavioral tests, Dr. Tsuyoshi Miyakawa and his colleagues found that mice with heterozygous knockout of the alpha-isoform of calcium/calmodulin-dependent protein kinase II (αCaMKII) exhibit behavioral deficits and other brain features consistent with bipolar disorder. Notably, the mutant mice also showed periodic changes in locomotor activity in their home cages with an approximate cycle length of 10-20 days. The changes in locomotor activity are associated with fluctuations of anxiety-like and depression-like behaviors, suggesting that the mutant mice may serve as an animal model showing infradian oscillations of mood substantially similar to those found in patients with bipolar disorder. A recent human study also indicated a genetic association of the αCaMKII gene with bipolar disorder, and decreased expression of αCaMKII has been observed in postmortem brains of patients with bipolar disorder. These findings indicate that αCaMKII mutant mice should serve as a good animal model of bipolar disorder, to elucidate the pathogenesis and pathophysiology of the disorder. In the current study, the researchers used infradian cyclic locomotor activity in the mutant mice as a proxy for mood-associated changes, and examined their molecular basis in the brain by conducting prediction analyses of the gene expression data. At first, researchers longitudinally monitored locomotor activity of 37 αCaMKII mutant mice by calculating the distance traveled in their cage for over 2 months. Subsequently, researchers dissected the hippocampus, a region thought to be involved in the regulation of mood, from the brain. The sampling of the hippocampus was conducted at the same time of the day (between 1 and 2 p.m.). Gene expression patterns in the hippocampus samples were examined using DNA microarrays that measured the expression levels of over 30,000 genes (transcripts) per sample. Based on the gene expression data, they constructed models for retrospectively predicting locomotor activity of individual mice. The researchers found that gene expression patterns in the hippocampus accurately predicted whether the mice were in a state of high or low locomotor activity. "This is the first demonstration, to our knowledge, of successful quantitative predictions of the individual behavioral state from gene expression patterns in the brain of a mammal," says Miyakawa. "Gene expression patterns in the hippocampus may retain information about past locomotor activity." In the current study, prediction analysis of gene expression data was implemented in order to identify the genes that are most useful to determine the state of cyclic changes in locomotor activity. Thus, the researchers examined the list of genes used for the successful prediction of locomotor activity. "To our surprise, the list of 'prediction genes' included significantly higher number of circadian genes, genes that are known to fluctuate according to circadian rhythms. Circadian genes turned out to be also infradian genes, whose expressions go up and down with mood-change-like behaviors in these mice," Miyakawa explains. Researchers also found that levels of cAMP and pCREB, possible upstream regulators of some circadian genes, were correlated with locomotor activity. "The current results provide the evidence for a novel concept that some circadian genes and their regulatory machinery in the brain may be involved in the generation of infradian rhythm behavior," Miyakawa explains. Furthermore, researchers found that drugs that are used to treat bipolar disorder controlled the locomotor activity and changed hippocampal pCREB expression in the mutant mice. These results support the idea that hippocampal pCREB levels may modulate locomotor activity. "While the work so far has been limited to a mouse model of bipolar disorder, regulating effectively such molecular changes might lead to treatment for the disorder," Miyakawa says. "It is also of interest whether certain molecular signatures in the samples, such as blood and cerebrospinal fluid, obtained from living animals can predict past and future locomotor activity. If the successful predictions are confirmed in the mouse model, this strategy may have potential for developing new methods for diagnosis, as well as treatment, of patients with bipolar disorder," Miyakawa says. Explore further: Immaturity of the brain may cause schizophrenia

He Z.,RIKEN | He Z.,South China University of Technology | Cichocki A.,RIKEN | Cichocki A.,Polish Academy of Sciences | And 3 more authors.
IEEE Transactions on Pattern Analysis and Machine Intelligence | Year: 2010

Recently, there has been a growing interest in multiway probabilistic clustering. Some efficient algorithms have been developed for this problem. However, not much attention has been paid on how to detect the number of clusters for the general n-way clustering (n≥ 2). To fill this gap, this problem is investigated based on n-way algebraic theory in this paper. A simple, yet efficient, detection method is proposed by eigenvalue decomposition (EVD), which is easy to implement. We justify this method. In addition, its effectiveness is demonstrated by the experiments on both simulated and real-world data sets. © 2006 IEEE.

Theodorou E.A.,University of Southern California | Buchli J.,University of Southern California | Schaal S.,University of Southern California | Schaal S.,ATR Computational Neuroscience Laboratories
Journal of Machine Learning Research | Year: 2010

With the goal to generate more scalable algorithms with higher efficiency and fewer open parameters, reinforcement learning (RL) has recently moved towards combining classical techniques from optimal control and dynamic programming with modern learning techniques from statistical estimation theory. In this vein, this paper suggests to use the framework of stochastic optimal control with path integrals to derive a novel approach to RL with parameterized policies. While solidly grounded in value function estimation and optimal control based on the stochastic Hamilton-Jacobi- Bellman (HJB) equations, policy improvements can be transformed into an approximation problem of a path integral which has no open algorithmic parameters other than the exploration noise. The resulting algorithm can be conceived of as model-based, semi-model-based, or even model free, depending on how the learning problem is structured. The update equations have no danger of numerical instabilities as neither matrix inversions nor gradient learning rates are required. Our new algorithm demonstrates interesting similarities with previous RL research in the framework of probability matching and provides intuition why the slightly heuristically motivated probability matching approach can actually perform well. Empirical evaluations demonstrate significant performance improvements over gradient-based policy learning and scalability to high-dimensional control problems. Finally, a learning experiment on a simulated 12 degree-of-freedom robot dog illustrates the functionality of our algorithm in a complex robot learning scenario. We believe that Policy Improvement with Path Integrals (PI2) offers currently one of the most efficient, numerically robust, and easy to implement algorithms for RL based on trajectory roll-outs. © 2010 Evangelos Theodorou, Jonas Buchli and Stefan Schaal.

Sugimoto N.,Japan National Institute of Information and Communications Technology | Morimoto J.,ATR Computational Neuroscience Laboratories
IEEE-RAS International Conference on Humanoid Robots | Year: 2011

In this study, we introduce a phase-dependent trajectory optimization method for Central Pattern Generator (CPG)-based biped walking controllers. By exploiting the synchronization property of the CPG controller, many legged locomotion studies have shown that the CPG-based walking controller is robust against external perturbations and works well in real environments. However, due to the nonlinear dynamic property of the coupled oscillator system composed of the CPG controller and the robot, analytically designing the biped trajectory to satisfy the requirements of a target walking pattern is rather difficult. Therefore, using a nonlinear optimization method is reasonable to improve the walking trajectory. To optimize the walking trajectory, a model-free optimal control method is preferable because precise modeling of the ground contact is difficult. On the other hand, model-free trajectory optimization methods have been considered as quite computationally demanding approach. However, because of recent advances in the nonlinear trajectory optimization method, using the model-free optimization method is now a realistic approach fro biped trajectory optimization. We use a path integral reinforcement learning method to improve the biped walking trajectory for CPG-based walking controllers. © 2011 IEEE.

Choi K.,ATR Computational Neuroscience Laboratories
European Journal of Applied Physiology | Year: 2012

To construct and evaluate a novel wheelchair system that can be freely controlled via electroencephalogram signals in order to allow people paralyzed from the neck down to interact with society more freely. A brain-machine interface (BMI) wheelchair control system was constructed by effective signal processing methods, and subjects were trained by a feedback method to decrease the training time and improve accuracy. The implemented system was evaluated through experiments on controlling bars and avoiding obstacles using three subjects. Furthermore, the effectiveness of the feedback training method was evaluated by comparison with an imaginary movement experiment without any visual feedback for two additional subjects. In the bar-controlling experiment, two subjects achieved a 95.00% success rate, and the third had a 91.66% success rate. In the obstacle avoidance experiment, all three achieved success rate over 90% success rate, and required almost the same amount of time to reach as that when driving with a joystick. In the experiment on imaginary movement without visual feedback, the two additional subjects adapted to the experiment far slower than they did with visual feedback. In this study, the feedback training method allowed subjects to easily and rapidly gain accurate control over the implemented wheelchair system. These results show the importance of the feedback training method using neuroplasticity in BMI systems. © 2011 Springer-Verlag.

Loading ATR Computational Neuroscience Laboratories collaborators
Loading ATR Computational Neuroscience Laboratories collaborators