Natural Language Interfaces to Data
By Abdul Quamar, IBM Research AI, USA, ahquamar@us.ibm.com | Vasilis Efthymiou, FORTH-ICS, Greece, vefthym@ics.forth.gr | Chuan Lei, Instacart, USA, chuan.lei@instacart.com | Fatma Özcan, Systems Research@Google, USA, fozcan@google.com
Abstract
Recent advances in natural language understanding and processing have resulted in renewed interest in natural language interfaces to data, which provide an easy mechanism for non-technical users to access and query the data. While early systems evolved from keyword search and focused on simple factual queries, the complexity of both the input sentences as well as the generated SQL queries has evolved over time. More recently, there has also been a lot of focus on using conversational interfaces for data analytics, empowering a line of business owners and non-technical users with quick insights into the data. There are three main challenges in natural language querying: (1) identifying the entities involved in the user utterance, (2) connecting the different entities in a meaningful way over the underlying data source to interpret user intents, and finally (3) generating a structured query in the form of SQL or SPARQL.
There are two main approaches in the literature for interpreting a user’s natural language query. Rule-based systems make use of semantic indices, ontologies, and knowledge graphs to identify the entities in the query, understand the intended relationships between those entities, and utilize grammars to generate the target queries. With the advances in deep learning-based language models, there have been many text-to-SQL approaches that try to interpret the query holistically using deep learning models. Hybrid approaches that utilize both rule-based techniques as well as deep learning models are also emerging by combining the strengths of both approaches. Conversational interfaces are the next natural step to one-shot natural language querying by exploiting query context between multiple turns of conversation for disambiguation. In this monograph, we review the background technologies that are used in natural language interfaces, and survey the different approaches to natural language querying. We also describe conversational interfaces for data analytics and discuss several benchmarks used for natural language querying research and evaluation.
Natural Language Interfaces to Data
Natural language interfaces provide an easy way to query and interact with data and enable non-technical users to investigate data sets without the need to know a query language. Recent advances in natural language understanding and processing have resulted in a renewed interest in natural language interfaces to data. The main challenges in natural language querying are identifying the entities involved in the user utterance, connecting the different entities in a meaningful way over the underlying data source to interpret user intents, and generating a structured query. There are two main approaches in the literature for interpreting a user’s natural language query. The first are rule-based systems that make use of semantic indices, ontologies, and knowledge graphs to identify the entities in the query, understand the intended relationships between those entities, and utilize grammars to generate the target queries. Second are hybrid approaches that utilize both rule-based techniques as well as deep learning models. Conversational interfaces are the next natural step to one-shot natural language querying by exploiting query context between multiple turns of conversation for disambiguation.
In this monograph, the authors review the rule-based and hybrid technologies that are used in natural language interfaces and survey the different approaches to natural language querying. They also describe conversational interfaces for data analytics and discuss several benchmarks used for natural language querying research and evaluation. The monograph concludes with discussion on challenges that need to be addressed before these systems can be widely adopted.
