For first time, AI model beats humans at reading comprehension test

Alibaba’s Institute of Data Science of Technologies (iDST), the fundamental research arm on artificial intelligence under Alibaba Group, developed a deep learning model which outperformed humans on Stanford’s reading comprehension test. 

They said on Monday that its deep neural network model scored 82.44 on the Stanford Question Answering Dataset (SQuAD) on January 11, beating the human score of 82.304. 

This is the first time ever that a machine has surpassed human intelligence in a global reading list. 

The SQuAD is a large-scale reading comprehension dataset comprised of over 100,000 question-answer pairs based on over 500 Wikipedia articles.

The model, with its ability to infer meaning, narrowing down, reads from paragraphs to sentences to words to find the right phrases with the right answers - and is now being seen with lucrative commercial values. 

In a process known as natural language processing (NLP), machines mimic human comprehension of words and sentences. 

The underlying technology has been applied in Alibaba’s Global Shopping Festival over the years, with machines answering huge inbound inquiries during the big sales period.

Luo Si, Chief Scientist of Natural Language Processing (NLP) at Alibaba iDST commented: “It is our great honor to witness the milestone where machines surpass humans in reading comprehension. That means objective questions such as ‘what causes rain’ can now be answered with high accuracy by machines. To our excitement, we believe the technology underneath can be gradually applied to numerous applications such as customer services, museum tutorial, and online response to inquiry from patients, freeing up human efforts in an unpresented way.”

“We are thrilled to see NLP research has achieved significant progress over the year. We look forward to sharing our model building methodology to the wider community and exporting the technology to our clients in the short future.” Si added. 

The iDST is Alibaba’s primary research arm focusing on artificial intelligence. It’s heavily into Natural Language Processing and solving problems that lead to real-world applications.