中文标题#
通过语义、谱和结构度量评估知识图谱复杂性以进行链接预测
英文标题#
Evaluating Knowledge Graph Complexity via Semantic, Spectral, and Structural Metrics for Link Prediction
中文摘要#
理解数据集的复杂性对于在知识图谱（KGs）上评估和比较链接预测模型是基础性的。 虽然从谱聚类框架内类之间的概率差异中得出的累积谱梯度（CSG）指标被提出作为一种与分类器无关的复杂性指标，据称其随着类别基数的增长而变化，并与下游性能相关，但迄今为止尚未在 KG 环境中进行评估。 在本工作中，我们批判性地考察了在多关系链接预测背景下的 CSG，并通过 Transformer 派生的嵌入引入语义表示。 与之前的说法相反，我们发现 CSG 对参数化高度敏感，并不稳健地随着类别数量增长。 此外，它与标准性能指标如平均倒数排名（MRR）和 Hit@1 的相关性较弱或不一致。 为了深入分析，我们引入并基准测试了一组结构和语义 KG 复杂性指标。 我们的研究结果表明，通过关系熵、节点级别的最大关系多样性以及关系类型基数捕获的全局和局部关系模糊性与 MRR 和 Hit@1 表现出强烈的反向相关性，表明这些指标作为任务难度的更忠实指示器。 相反，图连通性度量如平均度、度熵、PageRank 和特征向量中心性与 Hit@10 正相关。 我们的结果表明，CSG 所声称的稳定性和泛化预测能力在链接预测设置中未能成立，并强调了在知识驱动学习中需要更稳定、可解释和任务对齐的数据集复杂性度量。
英文摘要#
Understanding dataset complexity is fundamental to evaluating and comparing link prediction models on knowledge graphs (KGs). While the Cumulative Spectral Gradient (CSG) metric, derived from probabilistic divergence between classes within a spectral clustering framework, has been proposed as a classifier agnostic complexity metric purportedly scaling with class cardinality and correlating with downstream performance, it has not been evaluated in KG settings so far. In this work, we critically examine CSG in the context of multi relational link prediction, incorporating semantic representations via transformer derived embeddings. Contrary to prior claims, we find that CSG is highly sensitive to parametrisation and does not robustly scale with the number of classes. Moreover, it exhibits weak or inconsistent correlation with standard performance metrics such as Mean Reciprocal Rank (MRR) and Hit@1. To deepen the analysis, we introduce and benchmark a set of structural and semantic KG complexity metrics. Our findings reveal that global and local relational ambiguity captured via Relation Entropy, node level Maximum Relation Diversity, and Relation Type Cardinality exhibit strong inverse correlations with MRR and Hit@1, suggesting these as more faithful indicators of task difficulty. Conversely, graph connectivity measures such as Average Degree, Degree Entropy, PageRank, and Eigenvector Centrality correlate positively with Hit@10. Our results demonstrate that CSGs purported stability and generalization predictive power fail to hold in link prediction settings and underscore the need for more stable, interpretable, and task-aligned measures of dataset complexity in knowledge driven learning.
文章页面#
PDF 获取#
抖音扫码查看更多精彩内容