Arleo, A., Smeraldi, F. and Gerstner, W. Cognitive navigation based on non-uniform Gabor space sampling, unsupervised growing networks, and reinforcement learning. IEEE Trans. on Neural Networks, 15(3):639-652, 2004.

Sony CSL authors: Angelo Arleo

Abstract

We study spatial learning and navigation for autonomous agents. A state space representation is constructed by unsupervised Hebbian learning during exploration. As a result of learning, a representation of the continuous two-dimensional (2-D) manifold in the high-dimensional input space is found. The representation consists of a population of localized overlapping place fields covering the 2-D space densely and uniformly. This space coding is comparable to the representation provided by hippocampal place cells in rats. Place fields are learned by extracting spatio-temporal properties of the environment from sensory inputs. The visual scene is modeled using the responses of modified Gabor filters placed at the nodes of a sparse Log-polar graph. Visual sensory aliasing is eliminated by taking into account self-motion signals via path integration. This solves the hidden state problem and provides a suitable representation for applying reinforcement learning in continuous space for action selection. A temporal-difference prediction scheme is used to learn sensori-motor mappings to perform goal-oriented navigation. Population vector coding is employed to interpret ensemble neural activity. The model is validated on a mobile Khepera miniature robot.

Keywords: Gabor decomposition, Hebbian learning, hip-pocampal place cells, log-polar sampling, population vector coding, reinforcement

BibTeX entry

@ARTICLE { arleo:04a, AUTHOR="Arleo, A. and Smeraldi, F. and Gerstner, W.", JOURNAL="IEEE Trans. on Neural Networks", NUMBER="3", PAGES="639--652", TITLE="Cognitive navigation based on non-uniform Gabor space sampling, unsupervised growing networks, and reinforcement learning", VOLUME="15", YEAR="2004", }