CELEBRATING SEMANTICS

|   August to Septetmber 2021   |

The tutorial on cognitive vision addresses computational vision and perception at the interface of language, logic, cognition, and artificial intelligence. The tutorial focusses on application areas where the processing and explainable semantic interpretation of (potentially large volumes of) dynamic visuospatial imagery is central, e.g., for commonsense scene understanding; visual cognition for cognitive robotics / HRI, autonomous driving; narrative interpretation from the viewpoints of visuoauditory perception & digital media design, semantic interpretation of multimodal human-behavioural data.

The tutorial highlights Deep (Visuospatial) Semantics, denoting the existence of systematically formalised declarative AI methods --e.g., pertaining to reasoning about space and motion-- supporting semantic (visual) question-answering, relational learning, non-monotonic (visuospatial) abduction, and simulation of embodied interaction. The tutorial demonstrates the integration of methods from knowledge representation and computer vision with a focus on (combining) reasoning & learning about space, action, motion, and interaction. This is presented in the backdrop of areas as diverse as autonomous driving, cognitive robotics, eye-tracking driven visual perception research (e.g., for visual art, architecture design, cognitive film studies), and psychology & behavioural research domains where data-centred analytical methods are gaining momentum. The tutorial covers both applications and basic methods concerned with topics such as: explainable visual perception, semantic video understanding, language generation from video, declarative spatial reasoning, and computational models of narrative. The tutorial will position an emerging line of research that brings together a novel \& unique combination of research methodologies, academics, and communities encompassing AI, ML, Vision, Cognitive Linguistics, Psychology, Visual Perception, and Spatial Cognition and Computation.

   International Joint Conference on Artificial Intelligence (IJCAI 2021), Montral, Canada   
   Advanced Course on Artificial Intelligence (ACAI 2021), Berlin, Germany   

The tutorial on “Spatial Cognition and Artificial Intelligence” addresses the confluence of empirically based behavioural research in the cognitive and psychological sciences with computationally driven analytical methods rooted in artificial intelligence and machine learning. This confluence is addressed in the backdrop of human behavioural research concerned with “in-the-wild” naturalistic embodied multimodal interaction. The tutorial presents

• an interdisciplinary perspective on conducting evidence-based (possibly large-scale) human behaviour research from the viewpoints of visual perception, environmental psychology, and spatial cognition.
• artificial intelligence methods for the semantic interpretation of embodied multimodal interaction (e.g., rooted in behavioural data), and the (empirically driven) synthesis of interactive embodied cognitive experiences in real-world settings relevant to both everyday life as well to professional creative-technical spatial thinking.
• the relevance and impact of research in cognitive human-factors (e.g., in spatial cognition) for the design and implementation of next-generation human-centred AI technologies.

   Spatial Cognition Conference 2020/1, Riga, Latvia   

In this full-day workshop, we aim to discussion two main questions:

• How can we learn scalable and general semantic representations? In recent years, there has been a substantial contribution in semantic policy and action representation in the fields of robotics, computer vision, and machine learning. In this respect, we would like to invite experts in academia and motivate them to comment on the recent advances in semantic reasoning by addressing the problem of linking continuous sensory experiences and symbolic constructions to couple perception and execution of actions. In particular, we want to explore how these can make robot learning more scalable and generalizable to new tasks and environments.
• How can semantic information be used to create Explainable AI? We would like to invite researchers from a broad range of areas including task and motion planning, language learning, general-purpose machine learning, and human-robot interaction. Much of action semantics is definitionally tied to how robots and humans communicate, and one fundamental feature of these approaches should be that they allow a broad variety of people to benefit from advances in robotics, and to work alongside robots outside of laboratory environments. Building more understandable action representations is important as a way of building robotic systems that benefit society.

   IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021), Prague, Czech Republic.