We present the first comprehensive study on automatic knowledge base construction for two prevalent commonsense knowledge graphs: ATOMIC (Sap et al., 2019) and ConceptNet (Speer et al., 2017). Contrary to many conventional KBs that store knowledge with canonical templates, commonsense KBs only store loosely structured open-text descriptions of knowledge. We posit that an important step toward automatic commonsense completion is the development of generative models of commonsense knowledge, and propose COMmonsEnse Transformers (COMET) that learn to generate rich and diverse commonsense descriptions in natural language. Despite the challenges of commonsense modeling, our investigation reveals promising results when implicit knowledge from deep pre-trained language models is transferred to generate explicit knowledge in commonsense knowledge graphs. Empirical results demonstrate that COMET is able to generate novel knowledge that humans rate as high quality, with up to 77.5% (ATOMIC) and 91.7% (ConceptNet) precision at top 1, which approaches human performance for these resources. Our findings suggest that using generative commonsense models for automatic commonsense KB completion could soon be a plausible alternative to extractive methods.
核心贡献 · Key contributions
提出 COMET，一种利用预训练 Transformer 生成常识知识库的生成式模型。 Proposes COMET, a generative model for commonsense knowledge base construction using pre-trained transformers.
在 ATOMIC 和 ConceptNet 上分别达到 77.5%和 91.7%的 top-1 准确率，接近人类水平。 Achieves 77.5% and 91.7% top-1 precision on ATOMIC and ConceptNet, approaching human performance.
展示了从预训练语言模型将隐式知识迁移到显式常识生成的能力。 Demonstrates transfer of implicit knowledge from pre-trained language models to explicit commonsense generation.
生成的知识具有高新颖性和多样性，超过 75%的新元组与训练数据编辑距离显著。 Shows high novelty and diversity in generated knowledge, with over 75% novel tuples having significant edit distance from training data.
提供了一个可适应不同常识知识库和关系类型的框架。 Provides a framework adaptable to different commonsense knowledge bases and relation types.
进行了全面的人工评估和消融研究，验证了模型的有效性。 Conducts comprehensive human evaluation and ablation studies validating model effectiveness.
局限 · Limitations
新生成的元组有时会简化训练元组，例如'doctor CapableOf save life' vs 'save person life'。 Novel generations sometimes simplify training tuples, e.g., 'doctor CapableOf save life' vs 'save person life'.
仅使用 1%训练数据时性能显著下降，表明对数据的依赖性。 Performance drops significantly with only 1% training data, indicating data dependency.
模型可能因词义歧义产生错误常识，例如'dove SymbolOf submarine'。 Model may produce incorrect commonsense due to ambiguous word senses, e.g., 'dove SymbolOf submarine'.
评估仅限于英语，未探索其他语言的适用性。 Evaluation limited to English; applicability to other languages is unexplored.
需要种子知识元组，无法在没有现有知识库的情况下从头生成。 Requires seed knowledge tuples; cannot generate from scratch without existing KB.
论文章节 · Sections（共 15）
摘要Abstract
1 引言1 Introduction
2 学习生成常识2 Learning to Generate Commonsense
2.1 任务2.1 Task
2.2 Transformer 语言模型2.2 Transformer Language Model