Who do YOU think wins? IBM's 'unsettling' AI supercomputer argues convincingly against humans in landmark debates between man and machine

• e-mail

IBM's new AI supercomputer has been shown to take on humans in a debate and, according to its creators, 'win'.

Named Project Debater, the 'unsettling' AI was pitted against two humans in the first public demonstration of its ability at an event in San Francisco.

It competed in what is known as computational argumentation, where debaters know a subject, present a position and defend it against opposition.

The computer delivered its opening argument by pulling in evidence from its huge database, mainly made up from news articles and journals.

It then listened to a professional human debater’s counter-argument and spent four minutes rebutting it.  

Although the crowd agreed the human beings were unmatched in delivery, it decided the machine made better arguments with more substance.

However many of its responses were plagiarised with full sentences lifted from well-known sources, raising the question of how intelligent the AI really is.

Scroll down for video

IBM Project Debater gathers information from millions of newspaper and magazine articles saved in its database to engage in long-form discussion about a given topic

IBM's AI system is not connected to the internet, but instead uses a stash of hundreds of millions of newspaper and magazine articles on ‘about 100 areas of knowledge’ which the AI is able to draw upon to build its argument. 

During the live on-stage debate, IBM Project Debater also articulated the word 'voiceover' in the middle of a sentence, suggesting that video transcripts are also being categorised and used in its database of information.

According to IBM Principle Investigator for Project Debator Dr Noam Slonim, the computer is not designed to regurgitate its entire argument from a single source.

Speaking on BBC Radio 4’s Today programme, Dr Slonim said: ‘It is not copying a whole article, it is not even copying a whole paragraph either, it is picking single sentences – and in some cases just clauses – taken from many different articles, then it glues that together into a coherent, persuasive narrative.’ 

However, Project Debater manager Ranit Aharanov admitted that if the phrasing of some of the arguments sounds familiar at all – it is because the AI has lifted an argument wholesale from the source material.

‘A lot of content that you see is actually phrases that are taken from the sources, like newspapers,’ Aharanov told VentureBeat.

‘They do undergo rephrasing of various sorts to make them more coherent, to make them align with each other, to sometimes add information about the person mentioned, or so on, so there is phrasing — but a lot of it is taken as-is.’ 

One example of this appeared during the debate when the AI-powered machine argued that ‘having a space exploration program is a critical part of being a great power’.

This exact phrasing has only appeared once before, when it was used in an article by Wall Street Journal contributor Mark Whittington when writing about Japan’s plan to land people on the moon.

To make sure the resulting debate is compelling to watch, IBM also programmed the AI to add jokes into its narrative to keep the crowd entertained.

During its rebuttal, the machine is also designed to suggest its opponent was lying during their argument.

This is a technique used by humans to distract when they do not have a strong argument to rely upon, something the IBM team wanted to replicate with the machine.

The AI-powered machine was required to make a four-minute introductory speech, a four-minute rebuttal to the experienced human debater's arguments, and finish with a two-minute closing statement on the topic 

During the first showcase of the technology, two debates were presented on-stage – whether government subsidise space exploration and if the use of telemedicine should be increased.

IBM Project Debater was pitted against human debaters Noa Ovadia and Dan Zafrir, who each took the reigns for one of the debates.

In both cases, the audience judged that the machine outperformed in its ability to present a wider body of information in its argument.

However, the human beings were universally judged to be better at delivery in their on-stage speeches.

In the end, IBM Debater could not convince the audience to side with its argument in the debate on whether space exploration should be subsidised.

But the machine won the second debate, convincing more audience members with its arguments on telemedicine usage than its human opponent.

IBM has not taught its AI to judge the reliability of the information it draws upon, something that critics believe could lead to the machine using biased sources to successfully argue its point.

Brhmie Balaram, who works as a senior researcher at the Royal Society for the encouragement of Arts, Manufactures and Commerce (RSA) told the Today programme that there were ‘ethical concerns’ with this approach to an AI debate.

‘It’s about considering what sort of information it is analysing.

‘There’s a lot of people who are concerned the data that is going into these machines might be biased, and therefore, there are ethical concerns. But also, where is it getting this data from? If it is about healthcare, it’s raising concerns about privacy.’

However, the team behind IBM Debater believe the ability for the machine to argue both sides of the debate means it can never be seen as biased.

‘The computer is certainly less biased than humans,’ argued Dr Slonim.

‘The computer can argue in favour of both positions, so this is one important point.

‘The second is that, [the computer] can help us to take more informed decisions. Let’s say that we are debating or not to legalise cannabis — a topic that I thought about recently.

‘If we’re debating this topic, obviously we need the technology that will be able to quickly suggest to us: what are the relevant claims that people are making with respect to this topic, what is the relevant evidence that people are suggesting in respect to these claims?’

IBM hopes the project will eventually be used to help present an evidence-based argument on a topic that removes any bias, emotion, or ambiguity.

These arguments can be used to help humans make difficult decisions when there is no black-or-white answer.

Like the human participants, the machine did not have any prior knowledge about what the debate was about. Pictured is Noam Slonim, an IBM researcher

During the live on-stage debate about space, the machine said space exploration was beneficial to the economy. Pictured is Hayah Eichler, a professional debater who has previously debated with IBM Project Debater

Brhmie Balaram of the RSA suggested IBM Project Debater could be deployed to help make critical decisions in sectors including healthcare, criminal justice, and hiring and firing in the workplace.

Should this happen, Balaram believes the public will be frightened by the lack of human input.

‘What humans want is another human in the loop,’ she told BBC Radio 4.

‘They don’t just want the machine to make a decision of a prediction on its own, they want a human to be there to be able to take that into consideration as part of a number of other factors that they use to make a decision.’

Project Debater is still a research project, however, the company claims that some of the technology which underpins the new system has already started to seep into other IBM projects.

IBM is no stranger to grand, public displays of what it claims is superior artificial intelligence.

The Armonk-based company debuted its Watson supercomputer by trouncing human contestants on the US television gameshow Jeopardy, back in 2011. Before that, the company’s Deep Blue system beat world chess champion Garry Kasparov.

HOW ARTIFICIAL INTELLIGENCES LEARN USING NEURAL NETWORKS

AI systems rely on artificial neural networks (ANNs), which try to simulate the way the brain works in order to learn.

ANNs can be trained to recognise patterns in information - including speech, text data, or visual images - and are the basis for a large number of the developments in AI over recent years.

Conventional AI uses input to 'teach' an algorithm about a particular subject by feeding it massive amounts of information.   

AI systems rely on artificial neural networks (ANNs), which try to simulate the way the brain works in order to learn. ANNs can be trained to recognise patterns in information - including speech, text data, or visual images

Practical applications include Google's language translation services, Facebook's facial recognition software and Snapchat's image altering live filters.

The process of inputting this data can be extremely time consuming, and is limited to one type of knowledge. 

A new breed of ANNs called Adversarial Neural Networks pits the wits of two AI bots against each other, which allows them to learn from each other. 

This approach is designed to speed up the process of learning, as well as refining the output created by AI systems.