2024   /   CoDesign @ 10

|   January to September 2024   |

The training school on ``Representation Mediated Multimodality'' provides a consolidated perspective on the theoretical, methodological, and applied understanding of representation mediated multimodal sensemaking at the interface of language, knowledge representation and reasoning, and visuo-auditory computing. Intended purposes -addressed in the school- encompass diverse operative needs such as explainable multimodal commonsense understanding, multimodal generation/synthesis for communication, multimodal summarisation, multimodal interpretation guided decision-support, adaptation & autonomy, and analytical visualisation.

At the heart of the thematic focus of the school are the themes of grounding and grounded meaning-making:

linguistic expressions and/or relational categorisations are called grounded when they are linked to non-linguistic, especially quantitative perceptual data, such as information coming from modalities such as vision and audition in space-time. Such perceptual data could pertain to, for instance, dynamic spatio-temporal phenomena both in an embodied as well as disembodied interaction context. Grounding is in essence a key aspect of semiotic construction, e.g., enabling high-level meaning acquisition, analogy, and has been a long-standing challenge in Artificial Intelligence and related disciplines.

The training school particularly consolidates and presents a novel and emerging view of state-of-the-art multimodal, neurosymbolic grounding primarily through a confluence of methods in Artficial Intelligence, Computational Linguistics, Vision, Speech and their impact on interactional and communicative acts/artefacts in everyday life and specialist professional work. Here, multimodal interpretation is to be construed in the context of diverse forms of imagery, e.g., encompassing perceptual and communicative data sources such as image, video, language, audition, text, eye-tracking, neurophysiological markers in behavioural / clinical settings.
In this backdrop, the training school particularly focusses on grounded knowledge representation, reasoning, and learning (for linguistic purposes) through the following emerging research-driven themes that are of high interest:

• Space and Language
• Computational Linguistics
• Cognitive Linguistics
• Cognitive Vision
• Vision and Language
• Neurosymbolism
• Narrative - Discourse
• Computational Models of Narrative
• Speech and Language Technologies
• Human-Centred AI

The principal training school programme --consisting of keynotes, invited talks, and tutorials-- will categorically address questions such as:

• Explainability and transparency in multimodal models
• Neurosymbolism / Interaction between symbolic & sub-symbolic representations in models
• Commonsense knowledge representation and reasoning, and its role in multimodal models
• Complementarity / redundancy among data sources or modalities
• Situated reasoning, e.g., for language generation, decision-making

   TRAINING SCHOOL   >   REPRESENTATION MEDIATED MULTIMODALITY
EU COST ACTION Multi3Generation  /  @University of Malta, Malta - Jan 30 - Feb 2 2024.  /  JOIN US!   

Insitute 2024 on Artificial and Human Intelligence will held during September 22 - 27 2024 at Schloss Dagstuhl, Germany.

/   Artificial Intelligence.  Cognitive Science.  Cognitive Psychology.  Visuospatial Cognition and Computation.   Human-Computer Interaction.  Cognitive Neuroscience.  Design Cognition.  Design Science.   /

Institute 2024 on Artificial and Human Intelligence brings together research methodologies and perspectives from Artificial Intelligence, Cognitive Science, Neuroscience, Psychology & Human Development, Human-Computer Interaction, and Design Science. The institute addresses:

• formal and computational foundations of next-generation AI and cognitive technologies with a principal emphasis on human-centred knowledge representation, semantics, commonsense reasoning, and integration of reasoning & learning. A key focus is reasoning & learning about space, action and motion in human-scale embodied multimodal interactions.
• behavioural research in cognitive science & psychology and neuroscience aimed at investigating human intelligence from the viewpoints of embodiment, multimodal interaction, and visuospatial thinking. Special emphasis is on aspects pertaining to visual perception, high-level event perception, inattention blindness, perception of motion, and narrative-based perceptual sensemaking.