By now, most of us have tried to stump a chatbot — asking if it has feelings, testing it with paradoxical riddles, or tossing it sheer absurdity just to see how it responds.

Figure 1. How Nonsense Tests Expose ChatGPT’s Understanding of Language.

That’s exactly what psycholinguist Michael Vitevitch set out to discover. A professor in the Speech-Language-Hearing Department at the University of Kansas, Vitevitch recently conducted a study where he fed ChatGPT a series of “nonwords” — invented combinations of letters and sounds commonly used in cognitive psychology to probe how people process language. Figure 1 shows How Nonsense Tests Expose ChatGPT’s Understanding of Language.

“As a psycholinguist, I’ve often given people carefully crafted nonsense to better understand what they know,” Vitevitch explains. “Now, I wanted to apply the same methods to AI to see how it processes that same kind of input.”

When speaking gibberish to ChatGPT, Vitevitch observed that while the AI was excellent at detecting patterns, it didn’t always follow the same logic as a human would.

“It identifies patterns — just not always the ones a person would pick up on,” he notes. “Humans and AI process things differently, and that’s okay. The key is knowing where those differences lie so we can build AI systems that complement human thinking — especially in areas where we need extra support.”

Vitevitch also tested ChatGPT on “extinct words” — old English terms that have fallen out of everyday use. Among them was upknocking, a 19th-century job involving tapping on windows to wake people before alarm clocks existed.

Out of 52 archaic terms, ChatGPT accurately defined 36. For 11, it admitted uncertainty. In three cases, it borrowed meanings from other languages. And in two instances, it simply made things up.

“It did hallucinate on a couple of things,” Vitevitch says. “We asked it to define these extinct words. It got a good number of them right. For others, it responded, ‘I don’t know this one — it’s an unusual or very rare word.’ But in a few cases, it invented meanings. I suppose it was just trying to be helpful.”

Next, Vitevitch turned to a phonological test. He presented ChatGPT with a list of Spanish words and asked it to produce similar-sounding English words — a common method used to investigate how humans mentally organize and retrieve speech sounds.

“If I give you a Spanish word and ask for an English word that sounds like it, you — as an English speaker — would likely respond with something from English that phonetically resembles it,” Vitevitch explains. “You wouldn’t suddenly switch languages and give me a word from an entirely different language — but that’s exactly what ChatGPT did.”

“[The AI] used to create ‘sniglets’ — made-up words for things that don’t have names,” says Vitevitch. “Like when you’re vacuuming and there’s a thread on the floor that just won’t get sucked up, no matter how many times you go over it. What do you call that thread? The AI coined the term ‘carperpetuation.’”

Vitevitch found that ChatGPT did “kind of an interesting job” in this area. When asked to invent new words for specific concepts, it often defaulted to a formulaic approach — typically blending two familiar words.

By feeding ChatGPT nonsense prompts, Vitevitch hopes to uncover the unusual, sometimes surprising ways AI processes language. The goal, he emphasizes, isn’t to make AI think like humans — but to explore how its unique strengths can complement our own.

Source: COSMOS

Cite this article:

Priyadharshini S (2025), What Gibberish Reveals About ChatGPT’s Grasp of Language, AnaTechMaz, pp.269