Publications - The Year Before Last

This article presents the QUASAR system for question answering over
unstructured text, structured tables, and knowledge graphs, with unified
treatment of all sources. The system adopts a RAG-based architecture, with a
pipeline of evidence retrieval followed by answer generation, with the latter
powered by a moderate-sized language model. Additionally and uniquely, QUASAR
has components for question understanding, to derive crisper input for evidence
retrieval, and for re-ranking and filtering the retrieved evidence before
feeding the most informative pieces into the answer generation. Experiments
with three different benchmarks demonstrate the high answering quality of our
approach, being on par with or better than large GPT models, while keeping the
computational cost and energy consumption orders of magnitude lower.

General-domain knowledge bases (KB), in particular the "big three" --
Wikidata, Yago and DBpedia -- are the backbone of many intelligent
applications. While these three have seen steady development, comprehensive KB
construction at large has seen few fresh attempts. In this work, we propose to
build a large general-domain KB entirely from a large language model (LLM). We
demonstrate the feasibility of large-scale KB construction from LLMs, while
highlighting specific challenges arising around entity recognition, entity and
property canonicalization, and taxonomy construction. As a prototype, we use
GPT-4o-mini to construct GPTKB, which contains 105 million triples for more
than 2.9 million entities, at a cost 100x less than previous KBC projects. Our
work is a landmark for two fields: For NLP, for the first time, it provides
\textit{constructive} insights into the knowledge (or beliefs) of LLMs. For the
Semantic Web, it shows novel ways forward for the long-standing challenge of
general-domain KB construction. GPTKB is accessible at gptkb.org.

Question answering over knowledge graphs and other RDF data has been greatly
advanced, with a number of good systems providing crisp answers for natural
language questions or telegraphic queries. Some of these systems incorporate
textual sources as additional evidence for the answering process, but cannot
compute answers that are present in text alone. Conversely, systems from the IR
and NLP communities have addressed QA over text, but barely utilize semantic
data and knowledge. This paper presents the first QA system that can seamlessly
operate over RDF datasets and text corpora, or both together, in a unified
framework. Our method, called UNIQORN, builds a context graph on the fly, by
retrieving question-relevant triples from the RDF data and/or the text corpus,
where the latter case is handled by automatic information extraction. The
resulting graph is typically rich but highly noisy. UNIQORN copes with this
input by advanced graph algorithms for Group Steiner Trees, that identify the
best answer candidates in the context graph. Experimental results on several
benchmarks of complex questions with multiple entities and relations, show that
UNIQORN, an unsupervised method with only five parameters, produces results
comparable to the state-of-the-art on KGs, text corpora, and heterogeneous
sources. The graph-based methodology provides user-interpretable evidence for
the complete answering process.

Capturing fresh information in near real-time and using it to augment
existing large language models (LLMs) is essential to generate up-to-date,
grounded, and reliable output. This problem becomes particularly challenging
when LLMs are used for informational tasks in rapidly evolving fields, such as
Web search related to recent or unfolding events involving entities, where
generating temporally relevant responses requires access to up-to-the-hour news
sources. However, the information modeled by the parametric memory of LLMs is
often outdated, and Web results from prototypical retrieval systems may fail to
capture the latest relevant information and struggle to handle conflicting
reports in evolving news. To address this challenge, we present the NEON
framework, designed to extract emerging entity interactions -- such as events
or activities -- as described in news articles. NEON constructs an
entity-centric timestamped knowledge graph that captures such interactions,
thereby facilitating enhanced QA capabilities related to news events. Our
framework innovates by integrating open Information Extraction (openIE) style
tuples into LLMs to enable in-context retrieval-augmented generation. This
integration demonstrates substantial improvements in QA performance when
tackling temporal, entity-centric search queries. Through NEON, LLMs can
deliver more accurate, reliable, and up-to-date responses.

Methods for relation extraction from text mostly focus on high precision, at
the cost of limited recall. High recall is crucial, though, to populate long
lists of object entities that stand in a specific relation with a given
subject. Cues for relevant objects can be spread across many passages in long
texts. This poses the challenge of extracting long lists from long texts. We
present the L3X method which tackles the problem in two stages: (1)
recall-oriented generation using a large language model (LLM) with judicious
techniques for retrieval augmentation, and (2) precision-oriented
scrutinization to validate or prune candidates. Our L3X method outperforms
LLM-only generations by a substantial margin.