Artificial Intelligence: does state of the art natural language processing really understand language?

As we saw in my last post on GPT-3, state of the art AI can write poetry, computer code, and passages that ruminate about creativity. It can also, with priming from a human being, emulate a particular style:

Before asking GPT-3 to generate new text, you can focus it on particular patterns it may have learned during its training, priming the system for certain tasks… If you prime it with dialogue, for instance, it will start chatting with you.

And if you it to write in the style of, say, Scott Barry Kaufman, it will do that.

But, as the New York Times notes, there are limits:

When Mr. Kaufman actually chatted with his A.I. doppelgänger, he was not nearly as impressed as he was at first:

Scott Barry Kaufman: What is imagination?

“Scott Barry Kaufman”: What is imagination? You should tell me!

Scott Barry Kaufman: You should know!

“Scott Barry Kaufman”: Nope, you should know. You are the one who knows what imagination is. What do you think it is?

Because the only thing that GPT-3 is designed to do is predict the next word in a sequence of words, it doesn’t keep track of what’s already been said, and so its performance in extended dialogs is unconvincing.

In addition, some of its output depends significantly on human input. The computer code it generates requires “fine tuning” (“a line removed here or added there”). Its blog posts, pre-publication, underwent similar editing: a human being providing “the headline and the photo and the first few sentences” and removing some sentences “that were less convincing.”

As the Times notes:

When dogs and other animals exhibit even small amounts of humanlike behavior, we tend to assume they are more like us than they really are. The same goes for machines, said Colin Allen, a professor at the University of Pittsburgh who explores cognitive skills in both animals and machines. “People get sucked in,” he said, “even if they know they are being sucked in.”

More fundamentally, is GPT-3 like us in terms of understanding? Does it understand what it says to us and what we say to it?

Of all its various capabilities, the best candidate for actual understanding is “summarizing email”. Poetry can be generated via patterns learned from the thousands of existing poems; similar pattern learning can generate compelling tweets and short stretches of relatively convincing dialog. Trivia questions, with their straight-forward structures and well-defined responses, can be answered via a combination of key-word extraction and Internet look-up–as Watson did nearly a decade ago on Jeopardy. Large corpora of texts combined with their human authored translations allows machines to learn to do their own translations without knowing the meanings of any of the words in any of the languages.

But producing a decent summary requires actual understanding: no amount of machine learning and pattern recognition suffices for abstracting the gist. Indeed, the typical AI strategy for summarizing can be characterized, not as “gist abstraction,” but as “sentence extraction”.

Sentence extraction means going through each paragraph and identifying the phrases or sentences that, according to some metric, are most “representative” of the entire paragraph. One metric of representativeness: containing the plurality of the paragraph’s “key words”. Key words, in turn, can be identified as words that are relatively “specific”–like “propaganda”–as opposed to “commonplace”–like “statement”– as measured by how frequently they occur across texts in general.

To create a summary, the program takes the extracted sentences (or phrases) and concatenates them together or inserts them into some sort of pre-programmed summary template.

Nowhere in this process is any actual comprehension is involved, and any “gist” that emerges is a function only of how much gist is encapsulated by the human-generated phrases/sentences that the machine’s algorithms selected for extraction.

I’ve taken a look at the tweeted demo that shows GPT-3 summarizing emails and I’m not convinced that GPT-3 does much more than the above.

Real meaning depends on real-world connections. Contrary to claims of some of the most extreme postmodernists, words that are divorced from the real world have little meaning, even when networked together into the kind of semantic Wordnet that some AI systems take advantage of.

In other words, as it were, unless you’ve interacted with dogs, the meaning of “dog” will elude you. Unless you’ve experienced what it is to imagine something, the meaning of “imagination” will, likewise, elude you–as it did “Scott Barry Kaufman.”

And without access to the meanings of individual words, the gist of multiple words interacting in communicative texts will likewise elude you.

As will the meaning of “it.” (Up next).

2 thoughts on “Artificial Intelligence: does state of the art natural language processing really understand language?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s