The digital assistant IQ test from Loup Ventures repeats a similar attempt from April 201
A total of 800 questions were asked for each assistant, with grades given if the question was correctly interpreted and if a correct response was given.
Google Assistant topped the list, giving the right results 85.5 percent of the time, with Siri in second place with 78.5 percent of the questions answered correctly. Alexa scored 61.4 percent right, while Cortana scored 52.4 percent.
There were improvements across the board, with Google Assistant up from the 74.8 percent scored last year, while Siri and Cortana's score from 2017 was 66.1 and 48.8, respectively. The ability of assistants to understand the questions has also improved, with Siri shifting from 95 percent understood to 99 percent, and Google moves from 99 percent to 100 percent.
"Both voice recognition and natural language processing of digital assistants across the board have been improved to the point where they will understand everything you say to them," analyzes Gene Munster and Will Thompson.
As part of the testing, questions were broken into one of five categories, testing capabilities to interpret local knowledge, trade, navigation, information and command searches. It has been noticed that the questions were modified for this year's test to "reflect the ability to change AI assistants", changes that seemed to cause drops in results for navigation issues, but still improvements for the other categories.
Based on a category, Google Assistant handles all others, except for command requests dominated by Siri.
"We found Siri to be a little more useful and versatile (respond to more flexible language) by controlling the phone, smart home, music, etc." Heter report. "Our questionnaire contains some music-related queries (the most common thing for smart speakers.) Apple, true to its roots, has made sure that Siri has the ability to enjoy music on both mobile devices and smart speakers."
The test used iOS apps for Cortana and Alexa, as the analysts claim is not fully reflective of their capabilities when used on other platforms. One example is that the Alexa app's inability to set reminders, alarms, or send emails affected the performance of the command class. As Siri and Google Assistant are baked into iPhone and Android smartphones, it is suggested Cortana and Alexa fight an uphill battle when compared to smartphones.
Analysts hope that voice-based computing will improve over time as it is a function of removing friction from the user's point of view. Along with the ability to understand questionnaires with questionnaires that follow each other, as well as routines and smart home scenes, Siri Shortcuts are also highlighted as another friction reducing feature to create "mini automations" that can be triggered by the command.
Payment and ride hailing are two promising areas for future vocal development, because it requires little or no visual execution to be performed. While Siri and Alexa were able to take a trip, and Siri and Google Assistant could send money, the company expects such features to be adopted by those who do not already offer them.