Paper accepted in CSSA workshop @ CIKM 2020

Muhammad Jaleed Khan, Edward Curry, “Neuro-symbolic Visual Reasoning for Multimedia Event Processing: Overview, Prospects and Challenges”, Combining Symbolic and Sub-symbolic Methods and their Applications (CSSA), 29th ACM International Conference on Information and Knowledge Management (CIKM), Oct. 2020, Galway, Ireland.

Abstract

Efficient multimedia event processing is a key enabler for real-time and complex decision making in streaming media. The need for expressive queries to detect high-level human-understandable spatial and temporal events in multimedia streams is inevitable due to the explosive growth of multimedia data in smart cities and internet. The recent work in stream reasoning, event processing and visual reasoning inspires the integration of visual and commonsense reasoning in multimedia event processing, which would improve and enhance multimedia event processing in terms of ex-pressivity of event rules and queries. This can be achieved through careful integration of knowledge about entities, relations and rules from rich knowledge bases via reasoning over multimedia streams within an event processing engine. The prospects of neuro-symbolic visual reasoning within multimedia event processing are promising, however, there are several associated challenges that are highlighted in this paper.