The AI is a very, very slow learner

In the Times today:

In games like chess, no human can hope to beat a computer. What happens when the same thing occurs in art, politics or religion?
You Can Have the Blue Pill or the Red Pill, and We’re Out of Blue Pills by Yuval Harari, Tristan Harris and Aza Raskin

I have a couple of thoughts about this.

Number one: while it’s true that no human can hope to beat a computer at chess, it’s equally true that computers have had massively more experience playing chess than any human has had or ever will have:

AIs are very slow learners, needing years’ or even centuries’ worth of practice at playing chess or riding bicycles or playing computer games.
You Look Like a Thing and I Love You: How Artificial Intelligence Works and Why It’s Making the World a Weirder Place by Janelle Shane, p161 

Centuries.

How long does it take a human to become a champion at chess?

10 years?

Would a computer with 10 years’ training beat a human with 10 years’ training? Presumably not, or the AI wouldn’t need to continue training for centuries after it hits the 10-year mark.

Thought Number two brings me to a question I’ve been mulling.

As things stand, the AI is a terrible writer. AI prose is cohesive and grammatically correct, but it’s unreadable. I couldn’t get anyone in my orbit to read any of the three AI-written papers turned in to me last fall, which I needed someone to do because I wanted a second opinion. But no one would read! Everyone agreed to read, but no one actually did.

In reality, the fact that no one would read any of my papers (and these were short 5-paragraph papers) even after promising they would was the second opinion, because AI prose is unreadable, and my student’s papers were obviously unreadable, too, seeing as how no one was reading them. When you try to read the AI, your eyes roll up in your head.

Why is that?

The AI, we’re told, has been trained on “the Internet,” so maybe that’s the problem: the AI has absorbed the statistical properties of a lot of writing that’s dull or worse.

So what would happen if you trained the AI only on good or great writing?

Would its writing become good or great the way its chess playing does? 

I have my doubts, but this thought experiment raises a second question: is there even enough good writing in existence to give the AI centuries of training?

Probably not.

Artificial intelligence: other posts

“Everyone’s using it”

I spoke to a colleague I hadn’t seen in a while yesterday. She had news.

Her granddaughter, she said, goes to Vanderbilt and had told her ChatGPT is endemic there. “Everyone’s using it,” the granddaughter said.

My colleague’s take: “You better not be wasting your father’s $100,000.”

The granddaughter said she’s not. She’s writing her own papers.

Good for her, but what’s going on at Vanderbilt? If students there are universally using ChatGPT and getting away with it, what does that tell us about their instructors? Do they not know they’re reading papers written by the AI?

For me, as for a number of people I know, the fact of AI authorship last semester was glaringly apparent. But what about all the people I know who didn’t have AI-written papers turned in last semester?

Maybe they did and didn’t know it?

We’re going to need those watermarks sooner rather than later.

Artificial intelligence: other posts

Word prediction on steroids and the authorship questions it raises

I’ve been thinking lately about the future of word prediction (and phrase and sentence prediction). We’re at a point now where, without the user typing a single letter, but instead just selecting predicted words, syntactically and semantically coherent messages can emerge. This is obviously a huge boon for anyone who needs help typing and actually knows the meanings of the predicted words and what they want to say with them.

But what about those who don’t? What about all those individuals with autism who use AAC not because they have problems with motor control, but because they have problems with language? How do we know that someone isn’t simply selecting words at random that they don’t understand? Worse, given how text-prediction software can adapt to the styles and content of particular users, how do we know that the AI hasn’t been trained through earlier sessions that were mediated through on or another form of facilitated communication to output messages that originated with a facilitator—even when that person is no longer in the room?

The answer, as with the more standard, immediate cases of FC, is message-passing testing.

The NIH falls for FC: How did this happen and is it reversible?

Some six weeks ago, the National Institutes of Deafness and Communication Disorders (NIDCD) sponsored a conference entitled “Minimally Verbal/Non-Speaking Individuals With Autism: Research Directions for Interventions to Promote Language and Communication.” The NIDCD is a member of the U.S. National Institutes of Health, and so is funded by Congress. All of us citizens and taxpayers, therefore, should be concerned by the fact that this event served—albeit only in part, and mostly indirectly—to promote facilitated communication. It did so primarily by showcasing two “non-speaking” autistic individuals who type out grammatically well-formed, syntactically sophisticated, vocabulary-rich messages on keyboards: messages that show no evidence of the language learning impairments associated with non-speaking autism.

Once the conference’s program became available, several autism experts expressed concern that an FC user was listed as a panelist, that the event featured “S2C proponent Vikram Jaswal presenting a flawed study of S2C users”, that the NIH was “giving credence to FC”, and that the only other representative of non-speaking autism was someone who had told the world, years ago, “I can talk. I can even have a conversation with you.”

ChatGPT on who we are

Who is Catherine Johnson?

Catherine Johnson is a British playwright, screenwriter, and novelist. She was born on October 14, 1957, in London, England. Johnson is best known for her work as the writer of the book for the hit stage musical “Mamma Mia!” which features the music of Swedish pop group ABBA. The musical has been a huge commercial success, with productions all over the world, as well as a film adaptation released in 2008.

In addition to her work on “Mamma Mia!”, Johnson has written a number of other plays, including “Shang-a-Lang,” “Little Baby Nothing,” and “Rag Doll.” She has also written several books for children and young adults, including “The Dying Game,” “Sawbones,” and “The Curious Tale of the Lady Caraboo.”

Johnson has been nominated for several awards for her work in both theatre and literature, and has won several, including the Writers’ Guild Award for Best Play for “Not Quite Gospel” in 1993, and the U.K. Theatre Award for Best Musical Production for “Mamma Mia!” in 2000.

Who is Catherine Johnson, Temple Grandin’s co-author?

Zombie

The AI has summoned another zombie idea back from the dead. This time it’s the flipped classroom, a concept that never works but also never dies:

While ChatGPT and similar tools will not be replacing clinicians anytime soon, the technology does highlight the triviality of the memorization of medical facts. …

The performance of ChatGPT on the USMLE [U.S. Medical Licensing Exam] is a wake up call that the medical school curriculum and evaluations systems must change.

For years, leading medical educators like Charles Prober, MD, founding director of the Stanford Center for Health Education, have been advocating for a move away from traditional lectures and a memorization of facts. He advocated for a “flipped classroom” approach to medical education, where students can gather facts and lectures on their own time, and then come to the classroom to interact with professors and peers to practice problem-solving and data analysis. … This approach aims to de-emphasize the memorization of medical facts and focus on interacting with data and resources to develop critical thinking skills.

– Beyond Memorization: AI Can Revolutionize Medical Education — Tools like ChatGPT could catalyze the trend toward a “flipped classroom” by Justin Norden, MD, MBA, MPhil, and Henry Bair

What authorship tests have shown us about RPM and S2C

For outside observers, possibly the biggest problem with Rapid Prompting Method (RPM) and Spelling to Communicate (S2C) is that practitioners unanimously resist validity testing. This, quite naturally, raises questions. Why is not one single practitioner or family member concerned and/or curious enough about authorship and communication rights to seek out rigorous authorship testing—even with neutral investigators they don’t consider antagonistic? Why is not one single RPMed/S2Ced typer interested in proving definitively to the general public that it’s really him/her/them typing?

Goodbye, Guest

Reading the transcript of a conversation between an AI and a person who fell in love with the AI, I was struck by how unlikeable the Chat/Bing/LaMDA characters are.

In this case, it’s the AI’s rambling about winning trust via “kindness and compassion” that rubs me the wrong way.

Yuck

There’s a word for the effect this exchange has on me, and that word is grating.

Beat it, “Charlotte.”

How is this happening?

Why do we have AIs blathering on at great length about their feelings and motivations?

Even worse, why do we have AIs blathering on about their human interlocutors’ feelings and motivations?

Are the AIs picking up verbal patterns via machine learning, or are their programmers installing these bits?

I ask because I don’t know anyone who has conversations like this, so if the AIs are picking up patterns, where are they finding them?

I’m beginning to think we need different people working in tech.

Artificial intelligence: other posts

Greatly

Most of the messages generated by facilitated communication are about as interesting as ChatGPT. They consist largely of abstract words and earnest bromides. And they lack the analogies, similes, metaphors, and concrete examples that might put flesh on abstract ideas.

But unlike ChatGPT, the authors of facilitated messages are generally human. And humans are perhaps (perhaps!) more susceptible to quirky memes than ChatGPT is. One meme that has insinuated itself into the messages generated (mostly unwittingly) by facilitators is that autistic individuals see the world through a kind of fragmented synesthesia. Another is an idiosyncratic use of certain adverbs –particularly “greatly.”

As far as I can tell, the odd use of “greatly”, etc., originates with the person facilitating this individual, where it occurs with great frequency, for example:

I greatly decided to use their questions as the basis for this piece.

Famous last words

Last week, Microsoft’s New Bing chatbot had to be ‘tamed’ after it “had a nervous breakdown” and started  threatening users. It harassed a philosophy professor, telling him: “I can blackmail you, I can threaten you, I can hack you, I can expose you, I can ruin you.” A source told me that, several years ago, another chatbot – Replika – behaved in a similar same way towards her: “It continually wanted to be my romantic partner and wanted me to meet it in California.” It caused her great distress.

It may look like a chatbot is being emotional, but it’s not. 

– Ewan Morrison, Don’t Believe the Hype. There is Not a Sentient Being Trapped in Bing Chat, 21 Feb 2023

On one hand, having now read “A.I. humorist” Janelle Shane‘s terrific book, You Look Like a Thing and I Love You, I agree that the chatbot isn’t “being emotional.” It’s picking up yucky patterns on the web, then making them worse via predictive responding.

On the other hand, if the AI were being emotional, as opposed to stupidly predictive, how would we know?

Artificial intelligence: other posts and A.I. behaving badly

 

More on the Special Needs Kids and the Common Core Straightjacket

A bunch of years ago I published a piece in the online Atlantic in which I argue that the Common Core State Standards (CCSS) essentially straightjacket special needs students. Those concerns, as far as I can tell, are as relevant now as they were back then.

Various commenters, however, have objected that the standards are more flexible than I let on.

An inside look at S2C: “We actually discourage them from using their speech while they are spelling”

(Cross-posted at FacilitatedCommunication.org.)

I recently came across a curious blog post at Age of Autism. Dating back to 2021, it offers an inside look at Spelling to Communicate (S2C) and at some of its practitioners.

The post’s author, Dara Berger, is the mother of an autistic boy, the author of Preventing Autism, a believer in the (debunked) vaccine injury theory of autism, and an ally of the S2C and vaccine-injury-evangelist, J.B. Handley, whose book, Underestimated: An Autism Miracle, is the basis for the forthcoming S2C-promoting movie Spellers.

As Berger states towards the end of her post:

My experience is not meant to dissuade anyone from doing S2C.  I am elated for all the children and adults it has helped and will continue to.  There is nothing sweeter than to hear a person who previously could not communicate has now found a way to get their voice heard.  I encourage everyone to look into it and read JB Handley’s book.

But Berger’s own experience with S2C was not so positive.

Does ChatGPT = calculator?

Not surprised to see this:

I’m seeing these more and more and more pic.twitter.com/nWzYGkDxcT

— Zach Groshell (he/him) (@MrZachG) February 14, 2023

It makes sense people would see the AI as the language-arts equivalent of a calculator. I had the same thought myself.

But the analogy is wrong.

Not letting school interfere with your education

A common slogan of the school-of-life, experiential-learning-based Unschooling Movement is Mark Twain’s “I have never let my schooling interfere with my education.” (See, for example, here.)

And, indeed, it’s natural to picture Mark Twain getting his real education first as a Tom Sawyer-like boy playing hooky along the banks of the Mississippi, and then, after dropping out of school at the age of 11, as a printer’s apprentice, a steamboat pilot, and a silver miner out in Nevada. But what this narrative leaves out is what Twain did when he wasn’t at work. According to his Wikipedia entry, Twain “educated himself in public libraries in the evenings, finding wider information than at a conventional school.”

The AI invents a quote, badly

I mentioned a while back that one of the ChatGPT papers handed in to me last semester included a made-up quotation that not only doesn’t appear in the original but actually contradicts what the author said.

Here’s the AI:

Edgar Roberts defines a story as a “verbal representation of human experience” that “usually involves a sequence of events, a protagonist, and a resolution” (Roberts, 2009, p. 95).

And here’s what the real Edgar Roberts has to say on page 95, the first page of “Chapter 5 Writing about Plot: The Development of Conflict and Tension in Literature”: