中文标题#
透過分層和語義歸一化知識圖譜的視覺敘事魯棒符號推理
英文标题#
Robust Symbolic Reasoning for Visual Narratives via Hierarchical and Semantically Normalized Knowledge Graphs
中文摘要#
理解視覺敘事，如漫畫，需要結構化的表示，這些表示能夠捕捉多個層次的故事組織中的事件、角色及其關係。 然而，符號化敘事圖通常存在不一致和冗餘的問題，其中相似的動作或事件在不同的註釋或上下文中被標記為不同。 這種差異限制了推理和泛化的有效性。 本文介紹了一種用於分層敘事知識圖的語義歸一化框架。 基於認知基礎的敘事理解模型，我們提出了使用詞彙相似性和基於嵌入的聚類來整合語義相關的動作和事件的方法。 歸一化過程減少了註釋噪聲，對齊了敘事層次間的符號類別，並保持了可解釋性。 我們在 Manga109 數據集中的標註漫畫故事上展示了該框架，將歸一化應用於面板級、事件級和故事級圖。 在敘事推理任務（如動作檢索、角色定位和事件總結）上的初步評估表明，語義歸一化提高了連貫性和魯棒性，同時保持了符號透明性。 這些發現表明，歸一化是邁向可擴展的、受認知啟發的多模態敘事理解圖模型的關鍵步驟。
英文摘要#
Understanding visual narratives such as comics requires structured representations that capture events, characters, and their relations across multiple levels of story organization. However, symbolic narrative graphs often suffer from inconsistency and redundancy, where similar actions or events are labeled differently across annotations or contexts. Such variance limits the effectiveness of reasoning and generalization. This paper introduces a semantic normalization framework for hierarchical narrative knowledge graphs. Building on cognitively grounded models of narrative comprehension, we propose methods that consolidate semantically related actions and events using lexical similarity and embedding-based clustering. The normalization process reduces annotation noise, aligns symbolic categories across narrative levels, and preserves interpretability. We demonstrate the framework on annotated manga stories from the Manga109 dataset, applying normalization to panel-, event-, and story-level graphs. Preliminary evaluations across narrative reasoning tasks, such as action retrieval, character grounding, and event summarization, show that semantic normalization improves coherence and robustness, while maintaining symbolic transparency. These findings suggest that normalization is a key step toward scalable, cognitively inspired graph models for multimodal narrative understanding.
文章页面#
PDF 获取#
抖音掃碼查看更多精彩內容