When artificial intelligence is used to support or make vital decisions in areas such as health care and public administration, it becomes vital to understand how those systems reach at their conclusions. A new doctoral thesis from the University of Gothenburg provides a approach for designing AI systems which can explain the evidence primary their conclusions.
Today’s AI assistants are based on large language models that create responses from statistical patterns in text. Moreover, those systems can also “hallucinate” and generate information that seems credible however is in fact incorrect. One high-profile example happened currently when Sweden’s Deputy Prime Minister Ebba Busch shared a quotation that an AI had falsely claimed to have discovered in two separate sources, when in reality the quotation had been fabricated.
In his doctoral thesis, Alexander Berman shows how AI systems can be formed to explain, in dialogue with the user, the facts, rules or observations that underpin their conclusions—much as people do when they validate their reasoning.
“A main difference as compared with language models is that the explanations authentically show how the system reached at its conclusions, instead of just sounding convincing,” says Berman, a doctoral student in computational linguistics at the University of Gothenburg.
Transparency is vital
Rather than depending on a language model alone to generate both solutions and explanations, the proposed approach lets in the system to access information about the proof underlying its conclusions and use that information when explaining its reasoning to the user.
Language models can still be used as components within the system, for example to interpret the user’s questions or adapt responses to the particular context, without compromising the system’s reliability.
“Reliability and transparency are important when AI systems are used to make vital decisions. People ought to be given the opportunity to assess whether a decision is properly founded,” says Berman.
He hopes that the research will contribute to the development of AI systems that people can authentically trust, and that the thesis will encourage further research in the field.
The thesis, “What do you base that conclusion on? Grounding explainable AI in human dialogue strategies,” can be publicly defended on 4th June at 10:15 a.m. In Jubileumssalen, Humanisten, Renströmsgatan 6, Gothenburg.
