The earliest advances in speech recognition focused mainly on the creation of vowel sounds, as the basis of a system that might also learn to interpret phonemes (the building blocks of speech) from nearby interlocutors.

These inventors were hampered by the technological context in which they lived, with only basic means at their disposal to invent a talking machine. Nonetheless, they provide important background to more recent innovations.

Dictation machines, pioneered by Thomas Edison in the late 19th century, were capable of recording speech and grew in popularity among doctors and secretaries with a lot of notes to take on a daily basis.

However, it was not until the 1950s that this line of inquiry would lead to genuine speech recognition. Up to this point, we see attempts at speech creation and recording, but not yet interpretation.

Audrey, a machine created by Bell Labs, could understand the digits 0–9, with a 90% accuracy rate. Interestingly, this accuracy level was only recorded when its inventor spoke; it hovered between 70% and 80% when other people spoke to Audrey.

This hints at some of the persistent challenges of speech recognition; each individual has a different voice and spoken language can be very inconsistent. Unlike text, which has a much greater level of standardization, the spoken word varies greatly based on regional dialects, speed, emphasis, even social class and gender. Therefore, scaling any speech recognition system has always been a significant obstacle.

Alexander Waibel, who worked on Harpy, a machine developed at Carnegie Mellon University that could understand over 1,000 words, built on this point:

“So you have things like ‘euthanasia’, which could be ‘youth in Asia’. Or if you say ‘Give me a new display’ it could be understood as ‘give me a nudist play’.”

Until the 1990s, even the most successful systems were based on template matching, where sound waves would be translated into a set of numbers and stored. These would then be triggered when an identical sound was spoken into the machine. Of course, this meant that one would have to speak very clearly, slowly, and in an environment with no background noise to have a good chance of the sounds being recognized.

IBM Tangora, released in the mid-1980s and named after Albert Tangora, then the world’s fastest typist, could adjust to the speaker’s voice. It still required slow, clear speech and no background noise, but its use of hidden Markov models allowed for increased flexibility through data clustering and the prediction of upcoming phonemes based on recent patterns.

Although it required 20 minutes of training data (in the form of recorded speech) from each user, Tangora could recognize up to 20,000 English words and some full sentences.

The seeds are sown here for voice recognition, one of the most significant and essential developments in this field. It was a long-established truism that speech recognition could only succeed by adapting to each person’s unique way of communicating, but arriving at this breakthrough has been much easier said than done.

It was only in 1997 that the world’s first “continuous speech recognizer” (ie. one no longer had to pause between each word) was released, in the form of Dragon’s NaturallySpeaking software. Capable of understanding 100 words per minute, it is still in use today (albeit in an upgraded form) and is favored by doctors for notation purposes.

Machine learning, as in so many fields of scientific discovery, has provided the majority of speech recognition breakthroughs in this century. Google combined the latest technology with the power of cloud-based computing to share data and improve the accuracy of machine learning algorithms.

This culminated in the launch of the Google Voice Search app for iPhone in 2008.

Driven by huge volumes of training data, the Voice Search app showed remarkable improvements on the accuracy levels of previous speech recognition technologies. Google built on this to introduce elements of personalization into its voice search results, and used this data to develop its Hummingbird algorithm, arriving at a much more nuanced understanding of language in use. These strands have been tied together in the Google Assistant, which is now resident on almost 50% of all smartphones.

It was Siri, Apple’s entry into the voice recognition market, that first captured the public’s imagination, however. As the result of decades of research, this AI-powered digital assistant brought a touch of humanity to the sterile world of speech recognition.

After Siri, Microsoft launched Cortana, Amazon launched Alexa, and the wheels were set in motion for the current battle for supremacy among the tech giants’ respective speech recognition platforms.

In essence, we have spent hundreds of years teaching machines to complete a journey that takes the average person just a few years. Starting with the phoneme and building up to individual words, then to phrases and finally sentences, machines are now able to understand speech with a close to 100% accuracy rate.

The techniques used to make these leaps forward have grown in sophistication, to the extent that they are now loosely based on the workings of the human brain. Cloud-based computers have entered millions of homes and can be controlled by voice, even offering conversational responses to a wide range of queries.

That journey is still incomplete, but we have travelled quite some distance from the room-sized computers of the 1950s.