Home

Cognitive Vision presents an emerging line of research bringing together a novel and unique combination of methodologies from Artificial Intelligence, Vision and Machine Learning, Cognitive Science and Psychology, Visual Perception, and Spatial Cognition and Computation.

WE DEVELOP

General methods for the processing and semantic interpretation of dynamic visuo-spatial imagery with a particular emphasis on the ability to abstract, learn, and reason with cognitively rooted structured characterisations of commonsense knowledge pertaining to space and motion.

Semantic interpretation of dynamic visual imagery calls for general and systematic methods integrating techniques in knowledge representation and computer vision. Our research emphasises deep semantics, denoting the existence of declarative models -e.g., pertaining space and motion- and corresponding formalisation and methods supporting (domain-independent) reasoning capabilities such as semantic question-answering, relational visuospatial learning, and (non-monotonic) visuospatial abduction.

Presently, we emphasise applications of developed methods and tools in two settings:
(1). explainable visual abduction for active visual sensemaking and control (emphasising human-centred, ethical considerations in autonomous driving); and (2). semantic interpretation of multimodal human behavioural stimuli (emphasising AI foundations for empirically-driven research behavioural research in psychology, social sciences, visual art)

SELECT REFERENCES

Kondyli, V. , Suchan, J., Bhatt, M. (2022). Grounding Embodied Multimodal Interaction: Towards Behaviourally Established Semantic Foundations for Human-Centered AI. In: The 1st International Workshop on Knowledge Representation for Hybrid Intelligence (KR4HI 2022)., part of International Conference on Hybrid Human-Artificial Intelligence (HHAI 2022), Amsterdam, The Netherlands, June 13-17, 2022.

The Economist (2021). Is a self-driving car smarter than a seven-month-old? How to improve the intelligence of self-driving cars. Independent Media Coverage of our Research in Cognitive Vision for Autonomous Driving. (The Economist - Sep 4th 2021 Edition - www ) (Media Coverage)

J. Suchan., M. Bhatt., S. Varadarajan, S. (2021). Commonsense Visual Sensemaking for Autonomous Driving: On Generalised Neurosymbolic Online Abduction Integrating Vision and Semantics, Artificial Intelligence Journal (AIJ), Volume 299, 2021. (Open Access / PDF )

M. Bhatt., J. Suchan. (2020). Cognitive Vision and Perception: Deep Semantics Integrating AI and Vision for (Declarative) Reasoning about Space, Action, and Motion. ECAI 2020: the 24th European Conference on Artificial Intelligence (ECAI)., June 2020, Santiago de Compostela, Spain.

J. Suchan., M. Bhatt. (2020). Driven by Commonsense: On the Role of Human-Centred Visual Explainability for Autonomous Vehicles. ECAI 2020: the 24th European Conference on Artificial Intelligence (ECAI)., June 2020, Santiago de Compostela, Spain.

V. Kondyli., M. Bhatt. (2020). Multimodality on the Road: Towards Evidence-Based Cognitive Modelling of Human Interactions in Everyday Roadside Situations. 6th International Digital Human Modeling Symposium 2020 (DHM2020), Skövde, Sweden

V. Kondyli., M. Bhatt., J. Suchan., (2020). Towards a Human-Centred Cognitive Model of Visuospatial Complexity in Everyday Driving. STAIRS @ ECAI 2020: 9th European Starting AI Researchers Symposium (STAIRS), at ECAI 2020, the 24th European Conference on Artificial Intelligence (ECAI)., Santiago de Compostela, Spain.

Suchan, J., Bhatt, M., Vardarajan, S. (2019). Out of Sight But Not Out of Mind: An Answer Set Programming Based Online Abduction Framework for Visual Sensemaking in Autonomous Driving. IJCAI 2019: the 28th International Joint Conference on Artificial Intelligence (IJCAI) 2019, August 10 - 16, Macao. (Distinguished Paper Nomination, Honourable Mention)

Bhatt, M., Suchan, J., Vardarajan, S. (2019). Deep Semantics for Explainable Visuospatial Intelligence: Perspectives on Integrating Commonsense Spatial Abstractions and Low-Level Neural Features. NeSy 2019: In Proceedings of 14th International Workshop on Neural-Symbolic Learning and Reasoning (NeSy), at IJCAI 2019., August 12, 2019 (accepted for publication - to appear).

J. Suchan., M. Bhatt, Walega, P., Schultz, C. (2018). Visual Explanation by High-Level Abduction: On Answer-Set Programming Driven Reasoning about Moving Objects. In AAAI 2018: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, February 2-7, 2018, New Orleans, USA.

Mehul Bhatt, Kristian Kersting (2018): Semantic Interpretation of Multi-Modal Human-Behaviour Data - Making Sense of Events, Activities, Processes. KI / Artificial Intelligence, 31(4): 317-320 (2017)

Suchan, J., Bhatt, M., Vardarajan, S., Amirshahi, S. A., and Yu, S. (2018). Semantic Analysis of (Reflectional) Visual Symmetry: A Human-Centred Computational Model for Declarative Explainability. Advances in Cognitive Systems, Vol 6: 65:84, 2018.

Jakob Suchan, Mehul Bhatt (2017): Deep Semantic Abstractions of Everyday Human Activities - On Commonsense Representations of Human Interactions. ROBOT (1) 2017: 477-488

J. Suchan., M. Bhatt. (2017). Commonsense Scene Semantics for Cognitive Robotics: Towards Grounding Embodied Visuo-Locomotive Interactions. In ICCV 2017 Workshop: Vision in Practice on Autonomous Robots (ViPAR), International Conference on Computer Vision (ICCV), Venice, Italy.

Jakob Suchan. Declarative Reasoning about Space and Motion with Video. KI, 31(4):321–330, 2017

Suchan, J., Bhatt, M. (2016). Semantic Question-Answering with Video and Eye-Tracking Data: AI Foundations for Human Visual Perception Driven Cognitive Film Studies. IJCAI 2016: 25th International Joint Conference on Artificial Intelligence, New York City, USA.

Michael Spranger, Jakob Suchan, Mehul Bhatt (2016): Robust Natural Language Processing - Combining Reasoning, Cognitive Semantics, and Construction Grammar for Spatial Language. IJCAI 2016: 2908-2914

Suchan, J., Bhatt, M. (2016). The Geometry of a Scene: On Deep Semantics for Visual Perception Driven Cognitive Film Studies., in: WACV 2016: IEEE Winter Conference on Applications of Computer Vision (WACV 2016)., Lake Placid, NY, USA, IEEE.

Suchan, J., Bhatt, M., Santos, P. (2014). Perceptual Narratives of Space and Motion for Semantic Interpretation of Visual Data, in: Proceedings of International Workshop on Computer Vision + Ontology Applied Cross-Disciplinary Technologies (CONTACT). ECCV 2014 -- European Conference on Computer Vision, Zurich, Switzerland.

Suchan, J., Spranger, M., Bhatt, M., Eppe, M. (2014). Grounding Dynamic Spatial Relations for Embodied (Robot) Interaction, in: 13th Pacific Rim International Conference on Artificial Intelligence (PRICAI-2014), Queensland, Australia, 2014.

Bhatt, M., Suchan, J., Schultz, C. (2013). Cognitive Interpretation of Everyday Activities - Toward Perceptual Narrative Based Visuo-Spatial Scene Interpretation. Computational Models of Narrative (CMN) 2013., a satellite workshop of CogSci 2013: The 35th meeting of the Cognitive Science Society., Editors: M. Finlayson., B. Fisseni., Benedikt Löwe., J. C. Meister. OASIcs proceedings volume. OpenAccess Series in Informatics (OASIcs). Dagstuhl, Germany

Bhatt, M., Schultz, C., Freksa, C. (2013). The `Space' in Spatial Assistance Systems: Conception, Formalisation, and Computation. in Thora Tenbrink, Jan Wiener, Christophe Claramunt (editors). Representing space in cognition: Interrelations of behavior, language, and formal models. Series: Explorations in Language and Space. Oxford University Press, 2012. 978-0-19-967991-1.

Bhatt, M. (2012). Reasoning about Space, Actions and Change: A Paradigm for Applications of Spatial Reasoning. in: Hazarika, S. (editor). Qualitative Spatio-Temporal Representation and Reasoning: Trends and Future Directions. IGI Global (PA, USA). DOI: 10.4018/978-1-61692-868-1. ISBN13: 978161692868.

Bhatt, M., Guesgen, H., Woelfl, S., Hazarika, S. (2011). Qualitative Spatial and Temporal Reasoning: Emerging Applications, Trends and Future Directions. Journal of Spatial Cognition and Computation. Issue: Emerging Applications of Spatial and Temporal Reasoning. 11(1). ISSN: 1387-5868 print/1542-7633 online, Taylor & Francis Group 2011.

Bhatt, M., Lee, J. H., Schultz, C. (2009). CLP(QS): A Declarative Spatial Reasoning Framework. Proceedings of the 10th International Conference on Spatial Information Theory (COSIT 11). Belfast, Maine.

Bhatt, M., and Loke, S. (2008). Modelling Dynamic Spatial Systems in the Situation Calculus. Spatial Cognition and Computation, 8(1-2):86–130, 2008.

COGNITIVE VISION AND PERCEPTION

SELECT REFERENCES

COGNITIVE VISION
AND PERCEPTION