Why Google’s AI can’t spell the word Google (or anything else)

“Word counting has been a known challenge for LLMs, and we are working to solve this particular problem,” Google told TechCrunch in an emailed statement.

These basic typos may look familiar. LLMs, a type of artificial intelligence that powers chatbots and other text generators, are not designed to understand spelling. It’s been a running joke for years that when a company unveils a new AI model, you have to ask them how many ‘r’s are in the word strawberry. These AI models — which can program an application in seconds, or solve problems that have puzzled mathematicians for decades — are as good as a kindergarten child at spelling.

The problems with Google’s AI Overview go far beyond silly typos, though. Google did correct an issue from last week where a search for the word “ignore” returned what looked like a dictionary definition of the word, only the definition was displayed as “Understood. Let me know when you have a new claim or question!” But these misspellings have remained amusing because they’re so hard to squash.

As the researchers explained previously when we asked about these spelling puzzles, AI does not see sentences as units of language made up of words and letters. Many LLM programs are built on transformer models, which break text into symbols, which can be whole words, phonemes, or letters, depending on the model. Instead of “reading” like a human, the AI ​​converts text into digital representations of itself, which are then contextualized to help the AI ​​come up with a logical response.

“LLM programs rely on this transformer architecture, which doesn’t actually read the text,” Matthew Guzdial, an AI researcher and assistant professor at the University of Alberta, told TechCrunch. “What happens when you enter a prompt is that it gets translated into encoding.” “When he sees the word ‘the’, he has this encoding of what ‘the’ means, but he knows nothing about ‘T’, ‘H’ and ‘E’.

The token-based architecture that supports MBAs like Google’s AI Overview is inherently limiting, and researchers were not optimistic about their ability to solve the orthographic problem.

“It’s kind of hard to get around the question of what exactly a word should be for a language model, and even if we get human experts to agree on an ideal symbolic vocabulary, models will probably find it useful to ‘break down’ things even further,” Sheridan Feucht, a PhD student who studies the interpretability of large language models at Northeastern University, told TechCrunch. “I think there is no such thing as a perfect token because of this kind of ambiguity.”

This is not necessarily a pressing issue in the minds of researchers, as the benefit of an English MA does not come from its ability to spell. But these glaring failures help us remember that AI is not perfect, even if it sometimes seems like an all-knowing force beyond our understanding. We cannot blindly trust AI outputs without verifying their accuracy.