Multimodal Summarization : Recent Trends and Applications
Speaker: Sripana Saha – Bihta Patna District, IndiaTopic(s): Artificial Intelligence, Machine Learning, Computer Vision, Natural language processing
Abstract
Large amounts of multi-modal information online make it difficult for users to obtain proper insights. In this talk I will first introduce the concept of multimodal summarization and its various applications in real-life use cases. Thereafter I will introduce and formally define the concepts of supplementary and complementary multi-modal summaries in the context of the overlap of information covered by different modalities in the summary output. In recent years, a new problem statement of combined complementary and supplementary multi-modal summarization (CCS-MMS) is formulated. The problem is then solved in several steps by utilizing the concepts of multi-objective optimization by devising a novel unsupervised framework. An existing multi-modal summarization data set is further extended by adding outputs in different modalities to establish the efficacy of the proposed technique. The talk will also describe some topic aware multimodal summarization techniques. In the modern era, the rapid expansion of social media and the proliferation of the internet community has led to a multi-fold increase in the richness and range of views and outlooks expressed by readers and viewers. To obtain valuable insights from this vast sea of opinions, an inventive and holistic procedure for multi-modal abstractive summarization with comment sensitivity is proposed. This model utilizes both textual and visual modalities and examines the remarks provided by the readers to produce summaries that apprehend the significant points and opinions made by them. My talk will also cover the concept of comment aware multimodal summarization, recent algorithms and evaluation measures. At the end my talk will provide a comparative analysis of LLM produced summaries and deep learning generated summaries.About this Lecture
Number of Slides: 60 - 70Duration: 90 minutes
Languages Available: English
Last Updated:
Request this Lecture
To request this particular lecture, please complete this online form.
Request a Tour
To request a tour with this speaker, please complete this online form.
All requests will be sent to ACM headquarters for review.