Next-Generation Conversational AI for Database Access:
The Technology Behind AutoQL
How innovation in conversational AI is revolutionizing the way today's leading businesses access and leverage their data
Chapter 5
What it Takes to Build Conversational AI for
Database Access
The next step forward is creating a system that can learn to understand meaning, not just intent, in human phrases.
Machine learning models that are trained not only to understand individual words, but that are also capable of reasoning through the relationships between those words and what they mean together, can make using a computer an even more human experience.
This means the computer begins to recognize that “Who owes me?” is synonymous with other questions like “Who’s in debt to me?” and that these mean the querent is asking for information about outstanding invoices (as opposed to who hasn’t e-transferred their share of the dinner bill, which might be an equally relevant question in a different context).
Teaching a machine to fully understand meaning and decipher context in natural language isn’t easy. Computers much prefer structured data, like programming language (code), to natural language, which is completely unstructured data: it doesn’t fit nicely into boxes and its rules are somewhat arbitrary.
Typically, database query language statements are used to search for specific
information in a database when users are looking for answers to their questions
It’s therefore important to build AI that understands meaning in natural language, specifically in the context of database access, in order to bring
the power of a dynamic and intelligent natural language-to-database query language translation system to users.
Translating Natural Language to Database Query Language
To a computer, the meaning behind natural language is largely irrelevant: it understands structured, precise commands and there’s no room for nuance or ambiguity. Computers run on rules, but the rules of human language are hard for even humans to fully understand. What computers do understand is how to execute algorithms and code. They can execute actions based on the instructions given in this particular form. To get a computer system to do what you want it to do, you need to speak its language.
Because computers can’t synthesize language the same way humans can, when it comes to natural language querying, an intermediary step needs to occur between a computer’s understanding of the human phrase and acting on that request or query. Put simply, human language needs to be translated into the computer’s language, so it can know how it should respond.
When it comes to accessing information from databases, specifically relational databases, computer systems already have “languages” for carrying out the process of searching through data and returning what the user wants to know.
Relational databases can be searched and managed using a database query language like Structured Query Language (SQL). SQL can be written and run to identify and retrieve data from tables, columns, and rows in a database so that a user doesn’t have to hypothetically scroll through thousands or even billions of data points contained in a spreadsheet or report to find out, to return to our previous example, who owes them money.
Even though database query languages exist, humans are required to learn how these languages work in order to access data. Writing database query language like SQL is a specialized skill that the average employee doesn’t necessarily possess.
To bridge this skill gap and democratize the information in a database, we can employ AI to translate natural language to database query language. In other words, we can train an AI system to understand what a human is asking in their everyday language and dynamically produce a database query language statement based on that understanding.
Any user can simply enter a natural language query – or ask a question in their own words – to receive information from their database.
This is where the need for improved Natural Language Understanding (NLU) and advanced machine learning come in: once the machine can understand the meaning (not just the intent) of a user’s phrase it should be able to dynamically create a SQL statement that reflects exactly what the user is requesting from the database.
AutoQL dynamically translates natural language questions into database
query language statements in real time.
With this kind of AI in place, any user can simply enter a natural language query – or ask a question in their own words – to receive information from their database.
This makes database exploration intuitive and accessible to everyone, rather than exclusive to developers or other team members who have been trained
in writing database query languages. It’s a major step towards the true democratization of data, making it possible for anyone, regardless of skillset, to search any kind of aggregation of data via natural language queries and commands.
Understanding NL, Understanding SQL
To be effective, a conversational AI system designed for database access needs to deliver the outcomes that the user is expecting, every time.
While intent classification uses keywords in a user’s query to classify specific intents that can then be matched to pre-defined SQL statements, the user would have to use very specific words that the system has been trained to know.
There’s also the potential for the machine to run a SQL statement that doesn’t optimally reflect the user’s needs, since there would only be a single SQL option associated with the user’s intent. This can result in a negative user experience, one that is likely to yield mistrust of the system.
Translating natural language to database query language allows for the flexibility that humans expect from one another when asking for information. There are many ways to ask: “Who owes me?” but in the context the question is being asked, there’s probably only one optimal output.
AutoQL takes into account the context of each query, understanding that the user is looking for information about their AR in this particular example.
To return an accurate and contextually correct answer, the machine needs to understand how to connect all the variables of natural language with the limited, but still extensive, variables of database query language. When a user asks: “Who owes me?” the system understands what the sequence of words mean both in whole and in part, and dynamically generates a corresponding SQL statement based on that understanding. This means multiple unique SQL statements could be generated from natural language queries that vary only slightly in how they are asked by the user.
Much like a Paris city guide fluent in English and French, the machine can interpret natural language and translate that statement to an equivalent statement in database query language to return a meaningful result.
Importantly, the way the user asked for that information might be unique
or be asked in a way that the translator hasn’t experienced before. But this
is irrelevant because the system is designed for this purpose exactly: to decipher meaning by reasoning through the holistic natural language input to dynamically generate an optimal SQL statement that surfaces the desired result.
A system that understands both human language and database query language acts like an interpreter so that no matter how a human chooses to ask for
the information they want to access in their database, the computer can dynamically generate a SQL statement that will retrieve the information that user meant to ask for.
