For the second year running, Google Assistant (Alphabet's Google Assistant) first came in a test intended to measure the effectiveness of smart speaker assistants.
Apple (AAPL) Siri delivered the greatest improvement over the last time it was tested. And all four assistants who were tested on smart speakers made a meaningfully better job compared to their last trip.
Loups Smart Speaker Test
In the 2018 edition of its Smart Speaker IQ Test, the research firm Loup Ventures was asked smart speakers powered by Google Assistant, Siri, Amazon.com (AMZN) Alexa and Microsoft's (MSFT) Cortana a regular set with 800 questions. The questions were divided into five categories: local information, commerce, navigation, information in general and commands (such as alarms or reminders).
Google Assistant, tested on a Google Home speaker, was able to answer 87.9% of the questions correctly, up from 81
Alexa (tested on an Amazon echo) saw its accuracy better at 72.5% from 63.8%, and Cortana (tested on Harman Kardon's Invoke speaker) improved to 63.4% from 56.4% . Google Assistant was the only assistant to understand all 800 questions, but the others could understand 99% or more.
Apple, Alphabet, Amazon.com and Microsoft are holdings in Jim Cramer's Action Alerts PLUS Member Club . Want to be notified before Jim Cramer buys or sells AAPL, GOOGL, AMZN or MSFT? Learn more now.
Of the five categories created by Loup, Google Assistant came first, all saved for commands, where it was completed with others to Siri. Interestingly, Google was good for all rivals (Alexa included) in the trading category, where it had an 86% accuracy rate for Siri's 56%, Alexas 52% and Cortana's 31%.
Meanwhile, both Google Assistant and Siri were well ahead of their peers in the local info and navigation categories, and Google Assistant and Alexa were well advanced in the broader information category.
It is worth noting that Google Assistant and Siri also claimed the top two sites in the Loup July July test of voice assistants running on smartphones. In that test, Google Assistant and Siri were exactly 85.5% and 78.5% of the questions, while Alexa and Cortana made it 61.4% and 52.4%.
Loup's data gives some reason for why Google Assistant has won on Alexa in the smart speaker wars, despite Alexa's start and Amazon's ability to market Echo devices to their giant customer base. Google's undoubtedly unrivaled AI / machine learning abilities have enhanced its assistant, and it also integrates with Google Search and its massive knowledge graph.
One can add that Google's ability to integrate assistant with other Google services is an even greater advantage on devices that have a monitor, either a mobile device or a smart screen such as Google's homepage. On such hardware, Google can respond to a voice command or query by displaying content such as search results, knowledge graph data, YouTube videos, or images from a user's collection in Google Photos.
Another takeaway from Loup's data is that Siri, who has received much criticism over the years, even from magnificent Apple fans, seems to be meaningfully better. Apple, who earlier this week called AI's boss and former Google exec John Giannandrea, has not heard that it considers machine investment investment as an important priority.
And looking largely at Google and Apple's performance in both Loup's smart speaker test and smartphone assistant test in July, the numbers suggest that widespread use of assistant and Siri on smartphones is – by providing both algorithms and human decision makers with more data and experience to work on – improving their competitiveness on mobile devices and smart speakers alike. At this count, it's probably not a coincidence that Google Assistant and Siri easily beat Alexa and Cortana in the two categories of voice assistant (local info and navigation) bound to see a disproportionate number of smartphone requests.
It may not forget that despite growing rapidly over the past two years, the global smart speaker-installed base (thanks to Google and Apple's control of the world's two dominant mobile platforms) is still dwarfed by the installed base of smartphones with Google Assistant or Siri baked in. While Amazon has some strong strengths in the voice assist war and certainly invests aggressively to strengthen Alexa, competitive benefits from Google Assistant and Siri mobile popularity won't go away anytime soon.