Frankfurt am Main, Germany
Frankfurt am Main, Germany

Time filter

Source Type

Chai T.,Northeastern University China | Jin Y.,University of Surrey | Sendhoff B.,Honda Research Institute Europe
IEEE Computational Intelligence Magazine | Year: 2013

The papers in this special section focus on the challenges and future direction of evolutionary complex engineering. © 2005-2012 IEEE.

Kober J.,Bielefeld University | Kober J.,Honda Research Institute Europe | Bagnell J.A.,Carnegie Mellon University | Peters J.,Empirical | Peters J.,TU Darmstadt
International Journal of Robotics Research | Year: 2013

Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots. We highlight both key challenges in robot reinforcement learning as well as notable successes. We discuss how contributions tamed the complexity of the domain and study the role of algorithms, representations, and prior knowledge in achieving these successes. As a result, a particular focus of our paper lies on the choice between model-based and model-free as well as between value-function-based and policy-search methods. By analyzing a simple problem in some detail we demonstrate how reinforcement learning approaches may be profitably applied, and we note throughout open questions and the tremendous potential for future research. © The Author(s) 2013.

Muhlig M.,Honda Research Institute Europe | Gienger M.,Honda Research Institute Europe | Steil J.J.,Bielefeld University
Autonomous Robots | Year: 2012

In this paper we present a new robot control and learning system that allows a humanoid robot to extend its movement repertoire by learning from a human tutor. The focus is learning and imitating motor skills to move and position objects. We concentrate on two major aspects. First, the presented teaching and imitation scenario is fully interactive. A human tutor can teach the robot which is in turn able to integrate newly learned skills into different movement sequences online. Second, we combine a number of novel concepts to enhance the flexibility and generalization capabilities of the system. Generalization to new tasks is obtained by decoupling the learned movements from the robot's embodiment using a task space representation. It is chosen automatically from a commonly used task space pool. The movement descriptions are further decoupled from specific object instances by formulating them with respect to so-called linked objects. They act as references and can interactively be bound to real objects. When executing a learned task, a flexible kinematic description allows to change the robot's body schema online and thereby apply the learned movement relative to different body parts or new objects. An efficient optimization scheme adapts movements to such situations performing online obstacle and self-collision avoidance. Finally, all described processes are combined within a comprehensive architecture. To demonstrate the generalization capabilities we show experiments where the robot performs a movement bimanually in different environments, although the task was demonstrated by the tutor only one-handed. © 2011 Springer-Verlag.

Dragiev S.,TU Berlin | Toussaint M.,TU Berlin | Gienger M.,Honda Research Institute Europe
Proceedings - IEEE International Conference on Robotics and Automation | Year: 2011

The choice of an adequate object shape representation is critical for efficient grasping and robot manipulation. A good representation has to account for two requirements: it should allow uncertain sensory fusion in a probabilistic way and it should serve as a basis for efficient grasp and motion generation. We consider Gaussian process implicit surface potentials as object shape representations. Sensory observations condition the Gaussian process such that its posterior mean defines an implicit surface which becomes an estimate of the object shape. Uncertain visual, haptic and laser data can equally be fused in the same Gaussian process shape estimate. The resulting implicit surface potential can then be used directly as a basis for a reach and grasp controller, serving as an attractor for the grasp end-effectors and steering the orientation of contact points. Our proposed controller results in a smooth reach and grasp trajectory without strict separation of phases. We validate the shape estimation using Gaussian processes in a simulation on randomly sampled shapes and the grasp controller on a real robot with 7DoF arm and 7DoF hand. © 2011 IEEE.

Knoblauch A.,Honda Research Institute Europe
Proceedings of the International Joint Conference on Neural Networks | Year: 2010

Neural associative networks are a promising computational paradigm, both for modeling neural circuits of the brain and implementing Hebbian cell assemblies in parallel VLSI or nanoscale hardware. Previous works have extensively investigated synaptic learning in linear models of the Hopfield-type and simple non-linear models of the Steinbuch/Willshaw-type. For example, optimized Hopfield networks of n neurons can memorize about n2/k cell assemblies of size k (or associations between them) corresponding to a synaptic capacity of 0.72 bits per real-valued synapse. Although employing much simpler synapses much better suited for efficient hardware implementations, Willshaw networks can still store up to 0.69 bits per binary synapse. However, the number of cell assemblies is limited to about n2/k2 which becomes comparable to the Hopfield nets only for extremely small k. Here I present zip nets being an improved non-linear learning method for binary synapses that combines the advantages of the previous models. Zip nets have, up to factor 2/π ≈ 0.64, the same high storage capacity as Hopfield networks. Moreover, for low-entropy synapses (e.g., if most synapses are silent), zip nets can be compressed storing up to 1 bit per computer bit or, for synaptic pruning, up to log n bits per synapse. Similar is true for a generalized zip net model employing discrete synapses with an arbitrary number of states. © 2010 IEEE.

Einecke N.,Honda Research Institute Europe | Eggert J.,Honda Research Institute Europe
Proceedings - 2010 Digital Image Computing: Techniques and Applications, DICTA 2010 | Year: 2010

The computation of stereoscopic depth is an important field of computer vision. Although a large variety of algorithms has been developed, the traditional correlation-based versions of these algorithms are prevalent. This is mainly due to easy implementation and handling but also to the linear computational complexity, as compared to more elaborated algorithms based on diffusion processes, graph-cut or bilateral filtering. In this paper, we introduce a new two-stage matching cost for the traditional approach: the summed normalized cross-correlation (SNCC). This new cost function performs a normalized cross-correlation in the first stage and aggregates the correlation values in a second stage. We show that this new measure can be implemented efficiently and that it leads to a substantial improvement of the performance of the traditional stereo approach because it is less sensitive to high contrast outliers. © 2010 IEEE.

Lucke J.,Goethe University Frankfurt | Eggert J.,Honda Research Institute Europe
Journal of Machine Learning Research | Year: 2010

We show how a preselection of hidden variables can be used to efficiently train generative models with binary hidden variables. The approach is based on Expectation Maximization (EM) and uses an efficiently computable approximation to the sufficient statistics of a given model. The computational cost to compute the sufficient statistics is strongly reduced by selecting, for each data point, the relevant hidden causes. The approximation is applicable to a wide range of generative models and provides an interpretation of the benefits of preselection in terms of a variational EM approximation. To empirically show that the method maximizes the data likelihood, it is applied to different types of generative models including: a version of non-negative matrix factorization (NMF), a model for non-linear component extraction (MCA), and a linear generative model similar to sparse coding. The derived algorithms are applied to both artificial and realistic data, and are compared to other models in the literature. We find that the training scheme can reduce computational costs by orders of magnitude and allows for a reliable extraction of hidden causes. © 2010 Jorg Liicke and Julian Eggert.

Rodemann T.,Honda Research Institute Europe
IEEE/RSJ 2010 International Conference on Intelligent Robots and Systems, IROS 2010 - Conference Proceedings | Year: 2010

The position of a sound source is an important information for robotic systems to be extracted from a sound. Of the three spherical coordinates (azimuth, elevation, distance) only the azimuth direction is extracted in most robot audition systems. So far rarely investigated is the issue of estimating the distance between robot and sound source. In this article we describe a study on distance estimation using a binaural robot system in an indoor environment for sounds ranging in distance from 0.5 to 6m. We investigated several proposed audio cues like interaural differences (IID and ITD), sound amplitude, and spectral characteristics. All cues are computed within the framework of audio proto objects. In an extensive experimental setup with more than 10000 sounds we found that both mean signal amplitude and binaural cues can, under certain circumstances, provide a very reliable distance estimation. There was no observable effect of frequency dependent attenuation so that the spectral amplitude cue was only slightly above chance level. We also investigated the loss of precision of azimuth estimation with distance. In contrast to what could be expected, the performance does not severely deteriorate when the system is calibrated for different distances. ©2010 IEEE.

Rodemann T.,Honda Research Institute Europe
IEEE 15th International Conference on Advanced Robotics: New Boundaries for Robotics, ICAR 2011 | Year: 2011

The ability to localize a sound source is very important in interaction scenarios where the robot has to face the speaker. It is known that the horizontal position of a sound source can be easily estimated using only two microphones, however, the elevation is more difficult to determine in such a configuration. To deal with these problems the use of special outer ears (so called pinnae) has been proposed in order to allow the use of spectral cues for elevation estimation. Here we compare two algorithms that can extract spectral cues for arbitrary ear shapes and are able to localize a broad class of sounds under challenging real-world conditions. The algorithms run in real-time and are implemented on a real robot-head. © 2011 IEEE.

Knoblauch A.,Honda Research Institute Europe
Proceedings of the International Joint Conference on Neural Networks | Year: 2010

Neural associative memories are single layer perceptrons with fast synaptic learning typically storing discrete associations between pairs of neural activity patterns. For linear learning such as employed in Hopfield-type networks it is well known that the so-called covariance rule is optimal resulting in minimal output noise and maximal storage capacity. On the other hand, numerical simulations suggest that nonlinear rules such as clipped Hebbian learning in Willshaw-type networks perform better, at least for sparse neural activity and finite network size. Here I show that the Willshaw and Hopfield models are only limit cases of a general optimal model where synaptic learning is determined by probabilistic Bayesian considerations. Asymptotically, for large networks and very sparse neuron activity the Bayesian model becomes identical to an inhibitory implementation of the Willshaw model. Similarly, for less sparse patterns, the Bayesian model becomes identical to the Hopfield network employing the covariance rule. For intermediate sparseness or finite networks the optimal Bayesian rule differs from both the Willshaw and Hopfield models and can significantly improve memory performance. © 2010 IEEE.

Loading Honda Research Institute Europe collaborators
Loading Honda Research Institute Europe collaborators