27 October 2016

La Gaîté Lyrique, 2nd Floor

At a glance

15:00 – 17:00 (Concert Hall) Scientific talks #1 : Language Research: a Tribute to Luc Steels (in English)
17:00 – 18:00 (Concert Hall) Scientific talks #2 : Agroecology & Technology (In French)
17:00 – 17:45 (Historical Foyer) FAUSTO – Fragments of an Opera in progress (in Italian with English subtitles)
15:00 – 20:00 (Modern Foyer) Demos
18:00 – 19:00 (Concert Hall) Scientific talks #3 : The Future of Music Making: Flow Machines (in English)
19:00 – 19:40 (Historical Foyer) Round Table Discussion : The Future of Comics (in French)
20:00 – 20:50 (Concert Hall) Flow Machines Pop Concert
21:00 – 22:00 (Historical Foyer) Cocktail

Concert Hall Talks #1 Language Talks #2
Agroecology & Technology
Talks #3 Music Flow Machines Pop Concert
Modern Foyer Demos
Historical Foyer FAUSTO – Fragments of an Opera in progress Round Table Discussion Comics Cocktail

Scientific talks and musical events

Scientific talks #1 : History and impact of research on the emergence of language: Retrospective of Luc Steels’ Sony CSL (in English)

Time: 15:00 – 17:00

Location: Concert Hall

This series of talks will be introduced and moderated by Remi Van Trijp, researcher at Sony CSL and by Luc Steels, founder and former director of Sony CSL. Invited speakers include:

  • Frederic Kaplan, EPFL (Digital Humanities Laboratory (DHLAB) (Switzerland),
  • Remi van Trijp, Sony CSL (France),
  • Michael Spranger, Sony CSL Inc. (Japan),
  • Eors Szathmary, Parmenides Foundation (Germany),
  • Luc Steels, Fondateur et ancien Directeur de Sony CSL

Scientific talks #2 : Agroecology & Technology (in French)

Time: 17:00 – 18:00

Location: Concert Hall

Agriculture has reached a crossroad. Conventional agriculture is at the eve of a large-scale deployment of automation and robotisation, without having resolved the ecological challenges it faces. In parallel, an alternative agriculture is attentive to the natural processes, to the dynamics of ecosystems, and to the socio-economic impacts, but it is generally sceptical towards new technologies. It is therefore necessary to seek a new middle ground in which agroecological farmers, scientists and engineers can meet.

Invited speakers:

  • Maxime de Rostolan (Fermes d'Avenir)
  • Masatoshi Funabashi (Sony Computer Science Laboratory Inc.)
  • Peter Hanappe (Sony Computer Science Laboratory)

FAUSTO – Fragments of an Opera in progress (in Italian with English subtitles)

Time: 17:00 – 17:45

Location: Historical Foyer

Fausto is the second opera composed by Luc Steels and Oscar Vilarroya, who already successfully presented their first work Casparo in Barcelona, Paris, Brussels and Tokyo.
Taking Intensive Science as an opportunity, we will present some extracts of Fausto, based on the story of Faust, and reinterpreted from a technological, Artificial Intelligence angle.


Scientific talks #3 : The Future of Music Making: Flow Machines (in English)

Time: 18:00 – 19:00

Location: Concert Hall

Introduced and moderated by François Pachet, director Sony CSL.

The talks will present the results of the Flow Machines Project. Flow Machines addresses the issue of enhancing individual creativity by looking at it through the concept of “reflexive interactions”. Reflexive Interactions are human-machine interactions with a system that attempts to imitate the user’s style. The vision behind Flow Machines is that manipulating images of oneself creates novel and very effective ways to boost creativity. The talks will feature this radically new way of looking at content creation tools.

Invited speakers will be announced soon.

Round Table Discussion: ERCcOMICS (in French)

Time: 19:00 – 19:40

Location: Historical Foyer

During this round table, we will discuss in the first place about the future of comics, and also about how comics can help to communicate science. Fiammetta Ghedini will lead the discussion, as she is working as communication officer at Sony CSL and in the ERCcOMICS EU-funded project, that is featuring Flow Machines in Max Order, an AI-based webcomic.

Invited speakers will be announced soon.


Flow Machines Pop concert

Time: 20:00 – 20:50

Location: Concert Hall

At the beginning of the concert, François Pachet will play jazz standards at the guitar with the Reflexive Looper tool. Other artists will join him to play both jazz and pop music.

Going on with the concert, several artists will showcase songs composed with the Flow Machines tools. During the concert, artist-in-residence Benoît Carré and guest musicians will perform some extracts of an album revolving around the tale The Shadow by Andersen, also composed using Flow Machines. The concert will also feature a live, real time song composition with Flow Machines tools and a live performance by Camille Bertault. A video installation created for the event will also be presented.

With François Pachet, Benoit Carré, Camille Bertault, O, Barbara Carlotti, Lescop, ALB, Kumisolo and Housse de Racket.

Demonstrations (continuously throughout the afternoon)

Time: 15:00 – 20:00

Location : Modern Foyer

The Emergence of Communication

How language emerges in a population of robots

This demo will consist of videos about robotic experiments on the development of communication in young children, based on developmental AI. Developmental AI is a new branch of AI that aims to create artificial systems that can develop autonomously, just like a newborn baby. The long-term objective is to create a robot capable of reaching human-level intelligence. Our work, though, focuses on the emergence of communication and language in populations of learner-tutor robots. The demo shows how robots can autonomously acquire aspects of human communication and autonomously go through various stages of language development.


Constructional Language Processing

Web interface that allows participants to interact with Fluid Construction Grammar.

Human language processing is one of the most fascinating mysteries in science. Recent studies strongly suggest that language processing involves much more than simply combining words into syntactic structures: it requires an integrative, multimodal data structure called constructions. This demonstration shows grammatical constructions at work: participants can enter English sentences into a web interface, and inspect how constructions analyze these sentences into meaning. Alternatively, they can specify a particular meaning, and see how the same constructions can verbalize these meanings into sentences. The demonstration is powered by Fluid Construction Grammar, one of the groundbreaking language technologies developed at Sony Computer Science Laboratory, and it offers a “behind-the-scenes” view on what makes the application demonstration possible (see below).

Participative multi-user multi-language chat room

Multi-language chat room with simultaneous automatic translation.

Language technologies have made incredible progress in robust pattern matching. With the advent of big data and faster computers, these technologies have achieved major results in various tasks such as web search, sentiment analysis or word sense disambiguation. However, current technologies are also stuck in an “idiot savant” stage where they can find anything, but comprehend nothing. As a result, language processing remains brittle and limited in semantically challenging domains such as Machine Translation and human-robot interaction. This demonstration will show how the language technologies of Sony Computer Science Laboratory are able to break the glass ceiling through an online translation model for Japanese - English and English - Japanese. Japanese is a notoriously difficult language for translation, because it uses a different word order, grammatical particles, honorifics and optional syntactic elements. By combining state-of-the-art statistical models to constructional language processing, automated translating can however be vastly improved. Visitors will be able to participate in an online chat application through one of the available tablets or by using their own mobile phones. While they leisurely relax in a comfy sofa, their messages and their translations will be displayed on a big screen.


Flow Composer

Interactive web service for the composition of songs, in a lead sheet format

Flow Composer is an interactive web service for the composition of songs in the style of a given composer or corpus. Users can select songs they like from a large database of 12,000 popular music lead sheets, containing jazz standards, Brazilian popular music, and pop songs. The users then specify the structure of a new lead sheet using a graphical editor; then, the system generates a new song in the selected style. The system can also generate new songs from scratch, and the generated songs can be modified by the user. The user can play back the resulting songs using various audio rendering algorithms. This demo will feature videos of actual composers using the system, as well as the possibility to interactively compose a new song.

Bach or not Bach?

Deep Learning on Bach Chorales.

Polyphonic music generation is a difficult task due to the complex interplay between melody and harmony. We developed a model capable of learning to compose chorales in the style of Bach. This model is capable of producing convincing chorales even if it is trained with no other data that the 400 chorale sheets by Johann Sebastian Bach. A key feature is that we are able to constrain the generated chorales in many ways: we can for instance impose the melody, the bass, the rhythm but also the cadences (when the musical phrases end). With this, we can for instance reharmonize in the style of Bach chorales well-known melodies. The demo will consist of a listening test in order to assess if you are able to distinguish real Bach chorales from their generated counterparts.



Generation of realistic accompaniments from song recordings

ReChord uses concatenative synthesis to play an original version of any musical score by reusing chunks of audio extracted from a collection of audio recordings. Through an intuitive web interface, the user can generate an accompaniment of his favorite tune by trying out original combinations of the various recordings provided. The system is full of surprises, because it keeps the groove of the original recording, while actually playing the desired song. Ever wanted to hear Happy in Bossa Nova style ? ReChord will play it for you.

Reflexive Looper

A real-time classifier that identifies the human musician’s playing modes and answers back

The Reflexive Looper is a musical instrument expanding the one-man-band possibilities of the traditional loop pedal. This is mainly because it features an AI live engine that discriminates between bass-line, chords, melody, drums and choirs.

The Reflexive Looper can also intelligently transpose the played chords in order to fill in the missing chords, all in real-time. In addition, when the system finds multiple accompaniment solutions, it chooses the one that best fits the rhythmic pattern played by the musician at any moment. This enables a continuous rhythmic dialogue between the machine and the musician.


Robotics for agroecology

Exploring how technology can support agroecology.

To what extend is it possible, and to what extend do we want to implement automation and robotics in agroecology? In order to answer to this pressing questions, this demo will show a first prototype of an advanced tool for weeding.

Synecoculture - Human Augmentation of Ecosystems

Foundation of Synecoculture: Toward an agriculture of synthetic and profitable ecosystems

Humans have engaged in agriculture for over 10,000 years, but that history has been inevitably based on the trade-off between agricultural production and environmental degradation. Modern agriculture is still situated along the same line, further loading the environment with large-scale monoculture.

Meanwhile, plants have transformed terrestrial environment through surprisingly intelligent adaptation and coevolution throughout the history of Earth's ecosystem. Emergence of complex networks that harbor diverse symbiotic relations formed today's biosphere we live in, as a result of ecological optimum.

Synecoculture aims to yield the potentials of self-organizing nature of ecosystems by augmenting biodiversity beyond natural state, and organizing the ecological interactions toward solving the fundamental conflict between agricultural production and environmental impact. For this purpose, open systems science is applied for the management of crop communities, along with the development of IT support to develop long-tail, distributed frameworks of agriculture transversal to ecosystem processes.


Sony CSL Technology Promotion Office

Videos and posters of projects implemented in real life, as inside businesses or spin-offs

  • Inside business
    • ELFE : machine-learning for image and gesture recognition
    • Stealth project: optimization of production lines with statistical inference
  • Spin-offs
    • Sony Global Education Inc. : Creating a new educational infrastructure for a connected society
    • Koozit Inc. : Sensing and interactive technology solutions using smartphones, digital signage, Internet of Things (IoT) devices, wearable devices, etc., including: PlaceEngine Indoor Positioning Solution, Smile Recognition Solution, Activity Recognition Solution, “Q.board” Sports Sensor Board.

DC-based Open Energy System (DCOES)

Bottom-Up and Scalable Microgrid for Renewable Energy Power Systems

Growing awareness of the scale of problems such as global warming and the rapid depletion of fossil fuels is increasing expectations for sustainable methods of harnessing natural energy. At Sony CSL, as well as maximizing the use of natural sources of energy, our aim is to achieve a stable electricity system based primarily on such sources. For this reason we are conducting research into a wholly new type of super-dispersed, dynamic, reconfigurable network that we call Open Energy Systems (OES). As a practical solution, we have proposed and installed a DC-based, bottom-up, and scalable distributed power system, called DCOES, for a community of 19 inhabited houses at Okinawa Institute of Science and Technology (OIST). Each house is equipped with PV panels, batteries, and the energy exchange subsystem which handles the power transfer between houses via a DC transmission network. It is designed to support various renewable energy sources. The system has been operational since December 2014 and proven in real environment to be efficient in the use of renewable energy, flexible in size, as well as to increase dependability in the case of disaster.

Img Img

JackIn Head

JackIn (3D VR): A human-human telepresence system with omnidirectional wearable camera.

An omnidirectional video from user’s perspective can be captured, stabilized and transmitted to others. Other persons can virtually look around the immersive visual experience with his or her own head motion with HMD. This demo could allow users to remotely experience the extreme sports videos and extraordinary events.

Dissemination : Books & Music by Sony CSL

Showcasing books & music produced by Sony CSL.

  • Language Team:
    • Design Patterns in Fluid Construction Grammar (John Benjamins)
    • Computational Issues in Fluid Construction Grammar (Springer)
    • Language Grounding on Robots (Springer)
    • Experiments in Cultural Language Evolution (John Benjamins)
    • Book series : Computational Models of Language Evolution (Langsci Press)
      • The Talking Heads Experiment (Luc Steels)
      • How Mobile Robots can Self-Organize a Vocabulary (Paul Vogt)
      • Language Strategies for the Domain of Color (Joris Bleys)
      • The Evolution of Case Grammar (Remi van Trijp)
      • The Emergence of Spatial Language (Michael Spranger)
    • Self-Organization in the Evolution of Speech de Pierre-Yves Oudeyer (Oxford University Press)
    • La naissance d’une langue chez les robots de Frédéric Kaplan (Hermes Science)
  • Music Team:
    • Creativity and Universality in Language (Springer, Lecture Notes in Morphogenesis) Editors : Mirko Degli Esposti, Eduardo Altmann, François Pachet
    • Children’s Creative Music-Making with Reflexive Interactive Technology (Routledge, Research in Education) Authors : Victoria Rowe, Angeliki Triantafyllaki, François Pachet, projet MIROR
    • Music Learning with Massive Online Courses (MOOCs) (IOS Press), Editeur: Luc Steels, projet PRAISE
    • The Creative Process: Perspectives from multiple domains (Palgrave Macmillan), Editeur: Todd Lubart
    • Multidisciplinary Contributions to the Science of Creative Thinking (Springer, Creativity in the Twenty First Century), Editeurs : Giovanni Emanuele Corazza, Sergio Agnoli
  • Music albums:
    • “The Shadow” by Benoit Carré and major French pop artists
    • “CountOnIt” (Jazz) by François Pachet and Mark d’Inverno
    • “MarieClaire” (Pop) by François Pachet and Catherine Diran