ABOUT  THE  TRAINING  SCHOOL

The training school on ``Representation Mediated Multimodality'' provides a consolidated perspective on the theoretical, methodological, and applied understanding of representation mediated multimodal sensemaking at the interface of language, knowledge representation and reasoning, and visuo-auditory computing. Intended purposes -addressed in the school- encompass diverse operative needs such as explainable multimodal commonsense understanding, multimodal generation/synthesis for communication, multimodal summarisation, multimodal interpretation guided decision-support, adaptation & autonomy, and analytical visualisation.

The training school hosts an invited faculty delivering lectures, intensive tutorials, and keynotes; opportunities also exist for young researchers to position ongoing / early stage research, discuss, and network with school faculty and participants.

SCHOLARSHIPS    /    Generous scholarships are available to fully support participation. All eligible and accepted participants will be awarded a scholarship.

MULTIMODAL SENSEMAKING

The training school on ``Representation Mediated Multimodality'' provides a consolidated perspective on the theoretical, methodological, and applied understanding of representation mediated multimodal sensemaking at the interface of language, knowledge representation and reasoning, and visuo-auditory computing. Intended purposes -addressed in the school- encompass diverse operative needs such as explainable multimodal commonsense understanding, multimodal generation/synthesis for communication, multimodal summarisation, multimodal interpretation guided decision-support, adaptation & autonomy, and analytical visualisation.

The principal training school programme --consisting of keynotes, invited talks, and tutorials-- will categorically address questions such as:

  • Explainability and transparency in multimodal models
  • Neurosymbolism / Interaction between symbolic & sub-symbolic representations in models
  • Commonsense knowledge representation and reasoning, and its role in multimodal models
  • Complementarity / redundancy among data sources or modalities
  • Situated reasoning, e.g., for language generation, decision-making
Human-Centred Technologies.   Of particular significance to the training school are themes that benefit both public and private human-centred technological services most directly affecting EU citizens in an accessible, multilingual Europe, and have strong economic and societal impacts.
The training school will focus on human-machine interaction considerations in select application areas of emerging societal significance, such as autonomous systems, cognitive robotics, multi-lingual language technologies.

GROUNDED MEANING-MAKING

At the heart of the thematic focus of the school are the themes of grounding and grounded meaning-making:

Linguistic expressions and/or relational categorisations are called grounded when they are linked to non-linguistic, especially quantitative perceptual data, such as information coming from modalities such as vision and audition in space-time. Such perceptual data could pertain to, for instance, dynamic spatio-temporal phenomena both in an embodied as well as disembodied interaction context. Grounding is in essence a key aspect of semiotic construction, e.g., enabling high-level meaning acquisition, analogy, and has been a long-standing challenge in Artificial Intelligence and related disciplines.

The training school particularly consolidates and presents a novel and emerging view of state-of-the-art multimodal, neurosymbolic grounding primarily through a confluence of methods in Artficial Intelligence, Computational Linguistics, Vision, Speech and their impact on interactional and communicative acts/artefacts in everyday life and specialist professional work. Here, multimodal interpretation is to be construed in the context of diverse forms of imagery, e.g., encompassing perceptual and communicative data sources such as image, video, language, audition, text, eye-tracking, neurophysiological markers in behavioural / clinical settings. In this backdrop, the training school particularly focusses on grounded knowledge representation, reasoning, and learning (for linguistic purposes) through the following emerging research-driven themes that are of high interest:

  • Space and Language
  • Computational Linguistics
  • Cognitive Linguistics
  • Cognitive Vision
  • Vision and Language
  • Neurosymbolism
  • Narrative - Discourse
  • Computational Models of Narrative
  • Speech and Language Technologies
  • Human-Centred AI

Lectures cover the full spectrum of research areas within the scope of the school, encompassing the theoretical and methodological aspects of multimodality.

Tutorials cover specific topics in focus, also addressed in the lectures, with significant ``hands-on'' detail and demonstration from the viewpoint of practice and application.

Keynotes present a high-level view and summary of key research and application areas in scope of the school.



The panel discussion will address emerging and outward looking questions relevant to Multimodality Research, its applicatitons, and their significance towards the design and engineering of next-generation digital technologies

The doctoral colloquium is an opportunity primarily (but not exclusively) for early stage doctoral researchers to present ongoing / planned work, and receive feedback from school faculty and participants. On a case-by-case basis, the colloquium is generally open for all early career researchers desirous of discussing a research outlook.

Social events are planned during the training school. Furthermore, the school is hosted with integrated board facilities thereby offering ample opportunity to interact socially and network during the time-frame of the school.



The Training School 2022 is also part of the Multimodality.  Cognition.  Society Intiatiave, closely synergising with the Institute on Multimodality 2022 focussing on Minds. Media. Technologies. Details available via: www


Training School 2022     Training School 2022     Training School 2022