Fluid Construction Grammar (FCG) is an open-source computational grammar formalism that is becoming increasingly popular for studying the history and evolution of language. This demonstration shows how FCG can be used to operationalise the cultural processes and cognitive mechanisms that underly language evolution and change.
Almost all languages in the world have a way to formulate commands. Commands specify actions that the body should undertake (such as “stand up”), possibly involving other objects in the scene (such as “pick up the red block”). Action language involves various competences, in particular (i) the ability to perform an action and recognize which action has been performed by others (the so-called mirror problem), and (ii) the ability to identify which objects are to participate in the action (e.g. “the red block” in “pick up the red block”) and understand what role objects play, for example whether it is the agent or undergoer of the action, or the patient or target (as in “put the red block on top of the green one”). This chapter describes evolutionary language game experiments exploring how these competences originate, can be carried out and acquired, by real robots, using evolutionary language games and a whole systems approach.
Cognitive linguistics has reached a stage of maturity where many researchers are looking for an explicit formal grounding of their work. Unfortunately, most current models of deep language processing incorporate assumptions from generative grammar that are at odds with the cognitive movement in linguistics. This demonstration shows how Fluid Construction Grammar (FCG), a fully operational and bidirectional unification-based grammar formalism, caters for this increasing demand. FCG features many of the tools that were pioneered in computational linguistics in the 70s-90s, but combines them in an innovative way. This demonstration highlights the main differences between FCG and related formalisms.
This chapter introduces a new experimental paradigm for studying issues in the grounding of language and robots, and the integration of all aspects of intelligence into a single system. The paradigm is based on designing and implementing artificial agents so that they are able to play language games about situations they perceive and act upon in the real world. The agents are not pre-programmed with an existing language but with the necessary cognitive functions to self-organize communication systems from scratch, to learn them from human language users if there are sufficiently frequent interactions, and to participate in the on-going cultural evolution of language.
This chapter introduces very briefly the framework and tools for lexical and grammatical processing that have been used in the evolutionary language game experiments reported in this book. This framework is called Fluid Construction Grammar (FCG) because it rests on a constructional approach to language and emphasizes flexible grammar application. Construction grammar organizes the knowledge needed for parsing or producing utterances in terms of bi-directional mappings between meaning and form. In line with other contemporary linguistic formalisms, FCG uses feature structures and unification and includes several innovations which make the formalism more adapted to implement flexible and robust language processing systems on real robots. This chapter is an introduction to the formalism and how it is used in processing.
This chapter introduces the computational infrastructure that is used to bridge the gap between results from sensorimotor processing and language. It consists of a system called Incremental Recruitment Language (IRL) that is able to configure a network of cognitive operations to achieve a particular communicative goal. IRL contains mechanisms for finding such networks, chunking subnetworks for more efficient later reuse, and completing partial networks (as possibly derived from incomplete or only partially understood sentences).
This chapter describes key aspects of a visual perception system as a key component for language game experiments on physical robots. The vision system is responsible for segmenting the continuous flow of incoming visual stimuli into segments and computing a variety of features for each segment. This happens by a combination of bottom-up way processing that work on the incoming signal and top-down processing based on expectations about what was seen before or objects stored in memory. This chapter consists of two parts. The first one is concerned with extracting and maintaining world models about spatial scenes, without any prior knowledge of the possible objects involved. The second part deals with the recognition of gestures and actions which establish the joint attention and pragmatic feedback that is an important aspect of language games.
This chapter explores a semantics-oriented approach to the origins of syntactic structure. It reports on preliminary experiments whereby speakers introduce hierarchical constructions and grammatical markers to express which conceptualization strategy hearers are supposed to invoke. This grammatical information helps hearers to avoid semantic ambiguity or errors in interpretation. A simulation study is performed for spatial grammar using robotic agents that play language games about objects in their shared world. The chapter uses a reconstruction of a fragment of German spatial language to identify the niche of spatial grammar, and then reports on acquisition and formation experiments in which agents seeded with a `pidgin German’ without grammar are made to interact until rudiments of hierarchical structure and grammatical marking emerge.
This chapter investigates how a vocabulary for talking about body actions can emerge in a population of grounded autonomous agents instantiated as humanoid robots. The agents play a Posture Game in which the speaker asks the hearer to take on a certain posture. The speaker either signals success if the hearer indeed performs an action to achieve the posture or he shows the posture himself so that the hearer can acquire the name. The challenge of emergent body language raises not only fundamental issues in how a perceptually grounded lexicon can arise in a population of autonomous agents but also more general questions of human cognition, in particular how agents can develop a body model and a mirror system so that they can recognize actions of others as being the same as their own.
The question how a shared vocabulary can arise in a multi-agent population despite the fact that each agent autonomously invents and acquires words has been solved. The solution is based on alignment: Agents score all associations between words and meanings in their lexicons and update these preference scores based on communicative success. A positive feedback loop between success and use thus arises which causes the spontaneous self-organization of a shared lexicon. The same approach has been proposed for explaining how a population can arrive at a shared grammar, in which we get the same problem of variation because each agent invents and acquires their own grammatical constructions. However, a problem arises if constructions reuse parts that can also exist on their own. This happens particularly when frequent usage patterns, which are based on compositional rules, are stored as such. The problem is how to maintain systematicity. This paper identifies this problem and proposes a solution in the form of multilevel alignment. Multilevel alignment means that the updating of preference scores is not restricted to the constructions that were used in the utterance but also downward and upward in the subsumption hierarchy.
The paper surveys recent research on language evolution, focusing in particular on models of cultural evolution and how they are being developed and tested using agent-based computational simulations and robotic experiments. The key challenges for evolutionary theories of language are outlined and some example results are discussed, highlighting models explaining how linguistic conventions get shared, how conceptual frameworks get coordinated through language, and how hierarchical structure could emerge. The main conclusion of the paper is that cultural evolution is a much more powerful process that usually assumed, implying that less innate structures or biases are required and consequently that human language evolution has to rely less on genetic evolution.
Natural languages are fluid. New conventions may arise and there is never absolute consensus in a population. How can human language users nevertheless have such a high rate of communicative success? And how do they deal with the incomplete sentences, false starts, errors and noise that is common in normal discourse? Fluidity, ungrammaticality and error are key problems for formal descriptions of language and for computational implementations of language processing because these seem to be necessarily rigid and mechanical. This chapter discusses how these issues are approached within the framework of Fluid Construction Grammar. Fluidity is not achieved by a single mechanism but through a combination of intelligent grammar design and flexible processing principles.
One of the key components for achieving flexible, robust, adaptive and open-ended language-based communication between humans and robots – or between robots and robots – is rich deep semantics. AI has a long tradition of work in the representation of knowledge, most of it within the logical tradition. This tradition assumes that an autonomous agent is able to derive formal descriptions of the world which can then be the basis of logical inference and natural language understanding or production. This paper outlines some difficulties with this logical stance and reports alternative research on the development of an ?embodied cognitive semantics? that is grounded in the world through a robot?s sensori-motor system and is evolutionary in the sense that the conceptual frameworks underlying language are assumed to be adapted by agents in the course of dialogs and thus undergo constant change.
Aspect is undoubtedly the most capricious grammatical category of the Russian language. It has often been asserted as a mystery accessible only to native speakers, leaving all the others lost in its apparently infinite clutter. Recent work in cognitive linguistics has tried to bring order to the seeming chaos of the Russian aspectual system. But these approaches have not been operationalized so far. This paper demonstrates how the aspectual derivation of Russian verbs can be handled successfully with Fluid Constructional Grammar, a computational formalism recently developed for the representation and processing of constructions.
Open-ended language communication remains an enormous challenge for autonomous robots. This paper argues that the notion of a language strategy is the appropriate vehicle for addressing this challenge. A language strategy packages all the procedures that are necessary for playing a language game. We present a specific example of a language strategy for playing an Action Game in which one robot asks another robot to take on a body posture (such as stand or sit), and show how it effectively allows a population of agents to self-organise a perceptually grounded ontology and a lexicon from scratch, without any human intervention. Next, we show how a new language strategy can arise by exaptation from an existing one, concretely, how the body posture strategy can be exapted to a strategy for playing language games about the spatial position of objects (as in ?the bottle stands on the table?).
Colour naming games are idealised communicative interactions within a population of artificial agents in which a speaker uses a single colour term to draw the attention of a hearer to a particular object in a shared context. Through a series of such games, a colour lexicon can be developed that is sufficiently shared to allow for successful communication, even when the agents start out without any predefined categories. In previous models of colour naming games, the shared context was typically artificially generated from a set of colour stimuli and both agents in the interaction perceive this environment in an identical way. In this paper, we investigate the dynamics of the colour naming game in a robotic setup in which humanoid robots perceive a set of colourful objects from their own perspective. We compare the resulting colour ontologies to those found in human languages and show how these ontologies reflect the environment in which they were developed.
This paper is part of an ongoing research program to understand the cognitive and functional bases for the origins and evolution of spatial language. Following a cognitive-functional approach, we first investigate the cross-linguistic variety in spatial language, with special attention for spatial perspective. Based on this language-typological data, we hypothesize which cognitive mechanisms are needed to explain this variety and argue for an interdisciplinary approach to test these hypotheses. We then explain how experiments in artificial language evolution can contribute to that and give a concrete example.
Learning the meanings of words requires coping with referential uncertainty ? a learner hearing a novel word cannot be sure which aspects or properties of the referred object or event comprise the meaning of the word. Data from developmental psychology suggest that human learners grasp the important aspects of many novel words after just a few exposures, a phenomenon known as fast mapping. Traditionally, word learning is viewed as a mapping task, in which the learner has to map a set of forms onto a set of pre-existing concepts. We criticise this approach and argue instead for a flexible nature of the coupling between form and meanings as a solution to the problem of referential uncertainty. We implemented and tested the model in populations of humanoid robots that play situated language games about objects in their shared environment. Results show that the model can handle an exponential increase in uncertainty and allows scaling towards very large meaning spaces, while retaining the ability to grasp an operational meaning almost instantly for a great number of words. In addition, the model captures some aspects of the flexibility of form-meaning associations found in human languages. Meanings of words can shift between being very specific (names) and general (e.g. ?small?). We show that this specificity is biased not by the model itself but by the distribution of object properties in the world.
Humans maintain a body image of themselves, which plays a central role in controlling bodily movement, planning action, recognising and naming actions performed by others, and requesting or executing commands. This paper explores through experiments with autonomous humanoid robots how such a body image could form. Robots play a situated embodied language game called the Action Game in which they ask each other to perform bodily actions. They start without any prior inventory of names, without categories for visually recognising body movements of others, and without knowing the relation between visual images of motor behaviors carried out by others and their own motor behaviors. Through diagnostic and repair strategies carried out within the context of action games, they progressively self-organise an effective lexicon as well as bi-directional mappings between the visual and the motor domain. The agents thus establish and continuously adapt networks linking perception, body representation, action, and language.
This chapter briefly discusses the issues of symbols, meanings, and embodiment. It explains the solution to the symbol grounding problem. It illustrates the ingredients that are employed in the experiments about language emergence using a specific example of a color guessing game. It argues that these experiments show that there is an effective solution to the symbol grounding problem. The objective test for this claim is in the increased success of agents in the language games.
Language can be viewed as a complex adaptive system which is continuously shaped and reshaped by the actions of its users as they try to solve communicative problems. To maintain coherence in the overall system, different language elements (sounds, words, grammatical constructions) compete with each other for global acceptance. This paper examines what happens when a language system uses systematic structure, in the sense that certain meaning-form conventions are themselves parts of larger units. We argue that in this case multi-level selection occurs: at the level of elements (e.g. tense affixes) and at the level of larger units in which these elements are used (e.g. phrases). Achieving and maintaining linguistic coherence in the population under these conditions is non-trivial. This paper shows that it is nevertheless possible when agents take multiple levels into account both for processing meaning-form associations and for consolidating the language inventory after each interaction.
Our social behaviour has evolved primarily through contact with a limited number of other individuals. Yet as a species we exhibit uniformities on a global scale. This kind of emergent behaviour is familiar territory for statistical physicists.
What processes can explain how very large populations are able to converge on the use of a particular word or grammatical construction without global coordination? Answering this question helps to understand why new language constructs usually propagate along an S-shaped curve with a rather sudden transition towards global agreement. It also helps to analyse and design new technologies that support or orchestrate self-organizing communication systems, such as recent social tagging systems for the web. The article introduces and studies a microscopic model of communicating autonomous agents performing language games without any central control. We show that the system undergoes a disorder/order transition, going through a sharp symmetry breaking process to reach a shared set of conventions. Before the transition, the system builds up non-trivial scale-invariant correlations, for instance in the distribution of competing synonyms, which display a Zipf-like law. These correlations make the system ready for the transition towards shared conventions, which, observed on the timescale of collective behaviours, becomes sharper and sharper with system size. This surprising result not only explains why human language can scale up to very large populations but also suggests ways to optimize artificial semiotic dynamics.
This article proposes a number of models to examine through which mechanisms a population of autonomous agents could arrive at a repertoire of perceptually grounded categories that is sufficiently shared to allow successful communication. the models are inspired by the main approaches to human categorisation being discussed in the literature: nativism, empiricism, and culturalism. colour is taken as a case study. although we take no stance on which position is to be accepted as final truth with respect to human categorisation and naming, we do point to theoretical constraints that make each position more or less likely and we make clear suggestions on what the best engineering solution would be. specifically, we argue that the collective choice of a shared repertoire must integrate multiple constraints, including constraints coming from communication.
The interesting and deep commentaries on our target article reflect the continued high interest in the problem of colour categorisation and naming. clearly, colour remains for many cognitive science related disciplines a fascinating microworld in which some of the most fundamental issues for cognition and culture can be studied. although our target article took the stance of practically oriented engineers who are trying to find the best solution for orchestrating the self-organisation of communication systems in artificial agents, most commentators focus on the implications for cognitive science and we will do the same in our reply.
The computational and robotic modeling of language evolution is emerging as a new exciting subfield in cognitive science. The objective is to come up with precise operational models how communities of agents, equiped with a cognitive apparatus, a sensori-motor system, and a body, can arrive at shared grounded communication systems that have similar characteristics as human languages. Apart from its technological interest for building novel applications in the domain of human-robot or robot-robot interaction, this research is potentially relevant to the many disciplines interested in the origins and evolution of language.
As soon as we stop talking aloud, we seem to experience a kind of `inner voice’, a steady stream of verbal fragments expressing ongoing thoughts. What kind of information processing structures are required to explain such a phenomenon? Why would an inner voice’ be useful? How could it have arisen? This paper explores these questions and reports briefly some computational experiments to help elucidate them.
Behavior-based robotics has always been inspired by earlier cybernetics work such as that of Grey Walter. It emphasises that intelligence can be achieved without the kinds of representations common in symbolic AI systems. The paper argues that such representations might indeed not be needed for many aspects of sensori-motor intelligence but become a crucial issue when bootstrapping to higher levels of cognition. It proposes a scenario in the form of evolutionary language games by which embodied agents develop situated grounded representations adapted to their needs and the conventions emerging in the population.
The paper reports on experiments with a population of visually grounded robotic agents capable of bootstrapping their own ontology and shared lexicon without prior design nor other forms of human intervention. The agents do so while playing a particular language game called the guessing game. We show that synonymy and ambiguity arise as emergent properties in the lexicon, due to the situated grounded character of the agent-environment interaction, but that there are also tendencies to dampen them so as to make the language more coherent and thus more optimal from the viewpoints of communicative success, cognitive complexity, and learnability.
We have been conducting large-scale public experiments with artificial robotic agents to explore what the necessary and sufficient prerequisites are for word-meaning pairs to evolve autonomously in a population of agents through a self-organized process. We focus not so much on the question of why language has evolved but rather on how. Our hypothesis is that when agents engage in particular interactive behaviors which in turn require specific cognitive structures, they automatically arrive at a language system. We study this topic by performing experiments based on artificial systems. One such experiment, known as the Talking Heads Experiment, employs a set of visually grounded autonomous robots into which agents can install themselves to play language games with each other.
Language is a shared set of conventions for mapping meanings to utterances. This paper explores self-organization as the primary mechanism for the formation of a vocabulary. It reports on a computational experiment in which a group of distributed agents develop ways to identify each other using names or spatial descriptions. It is also shown that the proposed mechanism copes with the acquisition of an existing vocabulary by new agents entering the community and with an expansion of the set of meanings.