Fusion of Knowledge: Enhancing AI Reasoning through Language Models and Knowledge Graphs

Abstract

Large Language Models (LLMs) and Knowledge Graphs (KGs) have rapidly emerged as important areas in Artificial Intelligence (AI). LLMs leverage vast amounts of unstructured text to understand and generate natural language. KGs are relational graphs that encode domain expertise and knowledge into explicit semantics. A desideratum of AI is the ability to reason and draw inferences in a rational, sensible way. The present dissertation addresses the following question: How can LLMs and KGs enhance AI reasoning? The core idea of this dissertation is to leverage LLMs as a foundation for understanding and processing natural language, while utilizing KGs to access accurate and domain-specific knowledge. We present our contributions in advancing the capabilities of AI systems with respect to the following dimensions. (1) Faithfulness: We introduce a novel KG retrieval method (GNN-RAG) for grounding the LLM reasoning on multi-hop KG facts, alleviating LLM hallucinations when answering complex questions. (2) Effectiveness: We design a powerful graph model (ReaRev) for improved reasoning over KGs on knowledge-intensive tasks, such as Question Answering. (3) Temporal Reasoning: We propose TempoQR, a method that leverages Temporal KGs and allows LMs to handle questions with temporal constraints. (4) Efficiency: We develop a graph-aware distillation framework (GRAD), in which the LM learns to utilize useful graph information, while being efficient at inference. (5) Robustness: We present SemPool, a simple graph pooling method that offers robustness when critical information is missing from the KG.