Facilitating communication with Dan Willingham

I’ve long respected Dan Willingham’s work, but I’m concerned about a study he tweeted favorably about on twitter this week.


I and others tweeted some criticisms, and Dan posted a rebuttal on his blog. In this post, which I’ll also share on twitter, I’m going to respond, point by point, to that rebuttal.

First, a few notes about my respect for Dan and his work. When his first book came out, I gave it a 5-star review on Amazon; I linked to an article of his on phonics here just 3 months ago; and I’ve had cordial, even friendly, exchanges with him over the years.

However, the study Dan cited has raised red flags for me. Its object of investigation—Rapid Prompting Method—is similar to a discredited method called “Facilitated Communication” (FC). Both have been used with minimally-verbal children with autism, and both are things I’ve blogged about, most recently here.

RPM has never undergone the rigorous testing that FC has, and a number of us, long familiar with the litany of non-rigorous tests, anecdotes, and unsubstantiated claims that purportedly support RPM, took a look at who was involved in this study and immediately had concerns.

One concern: the conflicts of interest of the first author, Vikram Jaswal. First, there’s his long-time collaboration with the person who runs the clinic, whose teaching method (a form of RPM) the study investigated and drew favorable conclusions about. Second, there’s the fact that this method is used with Jaswal’s autistic daughter. From an article in the Washington Post:


Another concern: Jaswal’s collaborator, whom various indicators suggest was also the assistant overseeing the experiment. She–I’ll avoid using her name–has a history of claiming credentials she did not have, including in her dealings with paying clients: dealings relevant to the questions addressed by the experiment.

Unimpressed with our remarks, Dan ultimately tweeted:


Soon a number of threads spun out containing discussions of the study’s methodology and data, including this one from @24shaz. @angelrabanas analyzed one video in detail and posted her analysis on YouTube.

So I pinged Dan, and a day or two later he notified us that that he’d put up a post on his blog in response entitled “Responding to a Study You Just KNOW Is Wrong.”

Dan begins his post with a discussion of the learning styles controversy, and how educators who think they are “in the know” will pounce “with glee” on anyone who mentions learning styles. He goes on to say:


This made me wonder whether any of us had said anything bullying or snobbish about Jaswal et al or their studies. So I went through the various threads, and the only things I found that I thought might possibly be interpreted as snobbish or bullying were these:


Anyway, Dan goes on to explain how if you uncritically dismiss disproven theories you might miss new developments. As an example of a new development that he has not missed, Dan cites new research on learning styles theory and notes that his views have changed as a result.

I’m familiar with Dan’s skepticism about learning styles, and so I was intrigued to hear he’d changed his mind. I followed the link he gave to his new views, read through to the conclusion, and this is what I found:


But anyway, back to Dan’s blog post. Dan next turns to Jaswal et al’s paper:


Missing from this discussion is the most likely way that cuing could occur: not through a distinct cue for each of the 26 letters of the alphabet, but through the movement by the facilitator of the letter board. Such movement is a regular occurrence in videos of RPM, including those that are included in this study.

In particular, the facilitator may (unwittingly) shift the letter board such that a particular letter approaches the subject’s finger and/or enters the path of his/her eye gaze. To pick up on such cues, the subject does not look up at the assistant.

A moving letter board also makes it hard to draw reliable conclusions about intentional eye fixations or about typing rhythm.

Anyway, moving on, Dan turns to some of our concerns. Noting (correctly) that I mischaracterized Jaswal’s relationship to something called “The Tribes” (thus mischaracterizing the precise nature of Jaswal’s collaboration with clinic’s director), he turns to a tweet by Jason Travers, observing that:


Dan leaves it at that. But Jason is making an important point. Yes, he’s asking for a different experiment, but not one that differs in the ultimate questions being explored (i.e., the authorship of the messages). Rather, Jason is asking for an experiment that addresses these same questions using a more rigorous, well-established methodology: one where the possibility of facilitator influence is eliminated, as it is in a message passing test.

Jaswal et al attempt to explain why they think message-passing tests are problematic, making some claims that anyone with any background in language acquisition and linguistic pragmatics would instantly recognize as absurd:


Years of prompting and feedback are needed before children are able to convey new information? Tell that to the parent of a young toddler! Then there’s the fact that all of the participants, although they’re called “nonspeaking,” can talk—just not fluently and interactively.


What would cause these kids to be more fluent and interactive when typing on a letter board with an index finger? That is a question that Jaswal et al leave unexplained.

Even if Jaswal et al had good reasons for rejecting message passing, they’d still need to explain why they didn’t take other influence-reducing steps–like blindfolding the assistant and/or placing the alphabet board on a stationary stand. Such options are never even mentioned.

Dan does not discuss these huge design flaws. Nor does he address our concerns about how a moving board (1) provides cues that don’t involve subjects looking up at the assistant and (2) seriously warps the eye gaze results. Nor does he address various concerns we had about how the data was coded.

Instead, having dispensed with Jason, Dan then reacts to two tweets that, he grants, do address the data and its methods.


I’ve reviewed these and other tweets of @24shaz and @angelrabanas re the videos, and as far as I can tell, Dan’s reaction here is a non sequitur that does not rebut their critiques. Doing that convincingly would require quite a bit more discussion.

Dan then turns to a tweet I posted:


So I went back to the methods section to see if I had missed something:


There’s nothing here, so far as I can tell, that’s relevant to the issue of oral cues being delivered ahead of letter selection. Nor does this issue come up anywhere else in the methods section.

Having dispensed with the above tweets, Dan turns to one regarding the study’s small sample size and explains how these are routine in neuropsychology. He does not address concerns about how the subjects were non-randomly selected–they were hand-picked by the clinic–or about the grounds for eliminating one of the subjects.

Dan then dispenses with the rest of our tweets, among them many more that focused on methodology and data (and a mysteriously missing video) with this:


Alluding to his opening remarks, Dan then contrasts his take on our tweets with his take on his openness to revising his views on learning styles.


He goes on to say that people would be best advised to just ignore studies that they aren’t going to bother to read and that are outside their areas of expertise:


Regarding the second point, on expertise, what’s most relevant here, as far as our critiques go, is the expertise involved in evaluating RPM and FC videos for signs of manipulation. Some of us have spent many hours doing this—perhaps more hours than this paper’s authors, reviewers and advisors have.

Regarding the first point, about bothering to read things, the same applies to Dan himself. If he has the feeling that a conclusion (say one regarding Jaswal’s paper) is probably wrong, but doesn’t want to take the time to properly engage with the arguments, he might be better off ignoring it.

Incidentally, if Dan has a feeling that Jaswal’s critics are probably wrong, he has left out one possible reason why: he and Jaswal are colleagues in the same department at UVA, and Dan is mentioned in the paper’s acknowledgements:


Dan concludes with this:


If Dan were to inform himself about the harsh and abusive practices in both FC and RPM (evident in some of the videos of kids who are subjected to it), and about the linguistic and educational opportunity costs that they potentially impose, and about the evidence of greed and malice within the FC/RPM industrial complex, he might hesitate to interpret our passionate concern as nothing more than righteous indignation.

Facilitating Un-facilitated Communication in Autism

My two hour talk on this charged topic is going live at 5:00 PM EST today, available any time after that:


There’ll be a live q & a on Monday, and I’m hoping someone will bring up a paper, just published in Nature of all places, that appears to provide empirical support for a particular type of facilitated communication:


I’ve had an… interesting exchange with Dan Willingham on twitter about paper:


Monday, Monday

Yesterday was Monday. 

My college had scheduled a budget town hall for Monday, which I wanted to attend (all bad news there), but first, after I opened the email with the Zoom link, I had to be locked out of my account. 

That was actually kind of fun, it was so normal. Locked out of my accountyes! I remember being locked out of my account! I used to get locked out of my account with some regularity back in the day, and it felt good to be doing it again.

This must be what people mean by “stir crazy.”

So I was up for it, but being locked out of my account during quarantine turned out to be different from being locked out of my account in my old life.

Continue reading

“wrong, wrong, and wrong” and back to the future

Great line from Michael Osterholm in a recent CIDRAP podcast (Episode 4: The Reality of Testing):

-24:13: Last week I was at a meeting online with a prominent foundation in which a world-renowned economist, a Nobel Prize Laureate in economy, proposed that we be able to test 30 to 40 million people a week every week starting next week, and when I shared with him that that was not possible, his first reaction was that I’m part of the problem because I’m such a naysayer because I’m in fact you know always beating down these possibilities.

I like to think of myself just as a lighthouse saying, You know you may be a big aircraft carrier, but if you keep coming at me, buddy, I’m not moving, nor is the shore. 

You do like to see Nobel Prize Laureates forcibly informed that supply chains aren’t actual magic every once in a while. At least, I do.

Another great observation:

-26:38: I have raised on multiple occasions … that testing is not going to be widespread available in this country. Just accept that. … It’s what we first put forward more than 7 weeks ago … we are going to have a collision course with destiny called “reagent availability.”

It’s not about money, it’s about physics…. You can’t create the infrastructure overnight.

Wrong, wrong, and wrong:

-17:22: I’m surely not a stranger to or in any way opposed to contact tracing following a valid and comprehensive testing program. I see none of that here. And yet I worry that the whole country opening or reopening or closing or reclosing, or whichever [way] you want to look at [it], are all based on this testing program.

This is wrong, wrong, and wrong.

I had a funny moment yesterday.

As a nonfiction writer, I became pretty good at vetting experts.

With COVID, I chose Osterholm right away, but I was also following Marc Lipsitch, and had just recently discovered John M. Barry. I trust all three. [6/22/2020 UPDATE: I’ve changed my mind on Osterholm, for a couple of reasons. Still find what he had to say early on important–though he may have been wrong about the future availability of reagent, I don’t know. No longer interested in Lipstitch or Barry, either.]

So this week Osterholm’s program, CIDRAP, came out with its first report.

Authors: Osterholm, Lipsitch, Barry.

Plus Kristin Moore, who I hadn’t yet come across.

I have no idea what nonconscious criteria I used to put those three together.


Back to the future

We’re 64 days into lockdown here in New York, and the sun is out. My brain finally feels clear enough to think about education again. Yay!

I’m definitely going back to the classroom in September.  

Assuming classes are held, of course. If they aren’t, Zoom on.