This chapter introduces the computational infrastructure that is used to bridge the gap between results from sensorimotor processing and language. It consists of a system called Incremental Recruitment Language (IRL) that is able to configure a network of cognitive operations to achieve a particular communicative goal. IRL contains mechanisms for finding such networks, chunking subnetworks for more efficient later reuse, and completing partial networks (as possibly derived from incomplete or only partially understood sentences).
Grounding language in sensorimotor spaces is an important and difficult task. In order, for robots to be able to interpret and produce utterances about the real world, they have to link symbolic information to continuous perceptual spaces. This requires dealing with inherent vagueness, noise and differences in perspective in the perception of the real world. This paper presents two case studies for spatial language and quantification that show how cognitive operations – the building blocks of grounded procedural semantics – can be efficiently grounded in sensorimotor spaces.
Artificial agents trying to achieve communicative goals in situated interactions in the real-world need powerful computational systems for conceptualizing their environment. In order to provide embodied artificial systems with rich semantics reminiscent of human language complexity, agents need ways of both conceptualizing complex compositional semantic structure and actively reconstructing semantic structure, due to uncertainty and ambiguity in transmission. Furthermore, the systems must be open-ended and adaptive and allow agents to adjust their semantic inventories in order to reach their goals. This paper presents recent progress in modeling open-ended, grounded semantics through a unified software system that addresses these problems.
In this paper we test the dominant paradigm for modeling the semantics of determined noun phrases called Generalized Quantifier Theory in embodied interactions with robots. We contrast the traditional approach with a new approach, called Clustering Determination, which is heavily inspired by research on grounding of sensorimotor categories, and we show that our approach performs better in noisy, real world, referential communication.