Peer review at Scientific Reports, VII

Reviewer II next turns to the concept of message passing. Here’s what I wrote:

The most direct way to validate authorship is through “message passing” tests involving prompts that the facilitator has not seen ahead of time. For contact-based facilitation, studies dating back to the 1990’s indicate that facilitators do in fact guide typing.^2,3,4 Results show that the overwhelming majority of typed responses—when they occur—are based on prompts that the facilitator witnessed, and not on prompts that only the typist witnessed, strongly suggesting that the facilitator is the actual author. As for assistance via held-up letterboards, practitioners have yet to participate in rigorous, published, message passing experiments.⁵

2. Moore, S., Donovan, B., & Hudson, A. Facilitator-suggested conversational evaluation of facilitated communication. Journal of Autism and Developmental Disorders, 23, 541–552 (1993).
3. Wheeler, D. L., Jacobson, J. W., Paglieri, R. A., & Schwartz, A. A. An experimental assessment of facilitated communication. Mental Retardation, 31, 49–59 (1993).
4. Saloviita, T., Lepannen, M., & Ojalammi, U. Authorship in facilitated communication: An analysis of 11 cases. Augmentive and Alternative Communication, 3, 213-25 (2014).
5. Tostanoski, A., Lang, R., Raulston, T., Carnett, A., & Davis, T. Voices from the past: Comparing the rapid prompting method and facilitated communication. Developmental Rehabilitation, 17, 219–223 (2014).

But for Reviewer II, message passing (MP) means something completely different. Rather than being a means (indeed the best means) to test authorship, it’s a means for interpreting the autistic person’s code.

The Brave New World of Twitter Justice, Act III

As I mentioned earlier, Twitter, having no legal or financial incentive to involve actual humans, appears to leave its decisions about account suspensions entirely up to its AI system. However, as it turns out, there is one way in which Twitter AI does invite human eyes in on the process.

Here’s how it works:

The Philadelphia Inquirer no longer publishes letters on line

…so I’ll have to do that myself:

And, while I’m at it, I might as well re-post it in a searchable format with live links:

To the Editor,
Your recent Op-Ed, Americans with Disabilities Act should cover autism, too, was accompanied by a highly misleading photograph. It shows a boy pointing to letters on a board that is held up by another person. The same photograph has appeared elsewhere as an illustration of Rapid Prompting, also known as Spelling to Communicate. Rapid Prompting is a form of what’s known as “facilitated communication.” Based on a redefinition of autism as a sensory-motor disorder, Rapid Prompting lacks a solid evidence base. Particularly worrying are questions about whether those undergoing Rapid Prompting are actually doing the communicating. A large body of research indicates that the facilitators subconsciously direct the typing—for example, by shifting the board and deciding when a letter has been selected. In using this photograph as your illustration of autism, you unintentionally communicate that the methodology it depicts is generally accepted and supported.

In conversation with Janyce Boynton and Susan Gerbic on Facilitated Communication in Autism

We spoke this past Monday. Janyce is a former facilitator-turned-anti-FC crusader; Susan is the founder and leader of the Guerrilla Skepticism on Wikipedia (GSoW) project.

A footnote about academics who support Facilitated Communication

Way back when, when I was first developing my language training program for people with autism (then called the GrammarTrainer, now called the SentenceWeaver), a linguistic colleague of mine recommended I get in touch with a certain psycholinguist at the University of Wisconsin. Her recommendation stemmed from the GrammarTrainer’s approach to teaching English. While the new SentenceWeaver reads prompts out loud and allows speech as well as text input, the old GrammarTrainer was entirely text and typing-based: written prompts (“Where is the circle?”) soliciting typed responses (“The circle is between the triangle and square.”).

The reason my colleague suggested I contact this psycholinguistic was that she (the psycholinguist) had an autistic son who seemed to embody the potential of autistic individuals to master language entirely through text. The boy, completely non-speaking, had reportedly learned language, at least in part, from TV captions, and was now fully fluent. He could express all sorts of sophisticated thoughts–not by speaking (he remained non-oral), but by typing them out on a keyboard.

I was instantly suspicious, but not for the reasons I would be today. Back then, I had thought that Facilitated Communication was a thing of the past. After all, just a few years back, there’d been a major exposé on Frontline showcasing experiments that completely debunked it.

What made me suspicious, instead, was that it sounded like what this fully fluent, fully conversational boy had wasn’t an autism spectrum disorder, but a movement or motor control disorder–one that so impaired his oral motor functioning that he couldn’t coordinate his lips, tongue, and vocal cavity to make intelligible speech sounds.

Little did I know, back then, that Facilitated Communication not only hadn’t gone away, but that one of the central claims made by its proponents, in fact, is that autism, contrary to everything laid out in the diagnostic criteria (both then and now), is not a social (and social communication) disorder, but a movement disorder.

These days, though, when I hear someone say that autism is a movement disorder, I automatically see a red flag for FC support. And I’ve learned of a few others as well–all of which can, and do, warp the research and/or public statements of even the most accomplished and otherwise reliable academics.

1. Having a non-verbal or semi-verbal autistic child and being unsatisfied with how much that child has been able to communicate via the standard, evidence-based therapies.
2. Offering up lots of warped criticisms of the standard, evidence based therapies.
3. Getting financial support from one or another deep-pocketed, pro-FC charitable foundations (themselves tied by kinship to non-verbal autistic children).
4. Getting support from one or another autism self-advocacy organizations (I explore why autism self-advocates support Facilitated Communication here).
5. Publishing research claiming that autistic people only appear to be unsocial
6. Regularly conflating non-controversial statements like “autistic people do have empathy” and “autistic people do want to connect with others” with unsupported statements like “autistic people are just as good as non-autistic people at perspective taking and Theory of Mind tasks.”

This list has become, for me, a sort of diagnostic checklist–not necessarily for true belief in FC (that may be something that characterizes few people other than those who have chosen it for their children), but for practical entanglement in the FC Industrial Complex.

I made a *HUGE MISTAKE* the other week!

I had gotten the impression that a fatally-flawed Eye Tracking study I blogged about below–the one written by Dan Willingham’s colleague at UVA that supposedly showed support for a form of facilitated communication in autism–was published in the prestigious journal Nature.

It turns out this article was instead published in a completely different publication under the Nature.com umbrella: a publication called “Scientific Reports.”

I found this out when I tried to submit a letter to Nature. Nature wrote back saying that they don’t accept letters about articles in publications other than Nature. It took me a while to figure out that there are three separate entities all involving the name Nature.

There’s Nature.com the journal; Nature.com the publisher of dozens of distinct journals, and Nature.com > Scientific Reports.

And, as it turns out, there are lots of differences between Nature and Scientific Reports. Scientific Reports accepts 56% of submissions; Nature accepts 8%. Scientific Reports charges authors thousands of dollars to publish ($5380 for US authors), and allows them input on who should and shouldn’t review their work.

Scientific Reports has a history of retractions, including, so far in 2020, of a paper claiming the sun causes global warming, one claiming that cell-phone-induced neck-bending causes people to grow horns, and one that was plagiarized from the BA thesis of a Hungarian mathematician.

I’m not sure how much Scientific Reports charges to publish letters to the editor, so as far as the letter I mistakenly wrote to Nature (as in the journal Nature) goes, I’ll do what I did with the one I wrote to the Chicago Tribune on its pro- Rapid Prompting Method piece from early January, which also went unpublished, and post it publicly

To the Editors,

I’m writing with concerns about your article “Eye-tracking reveals agency in assisted autistic communication” (Jaswal, V.K., Wayne, A. & Golino, H. Sci Rep 10, 7882 (2020).

One concern relates to the authors’ justifications for testing the agency of what they call “assisted communication” indirectly, via eye movements, rather than directly, via a message-passing test—the gold standard for establishing authorship.

In a message-passing test, the researcher prompts the subject and/or asks him a question while the assistant, or facilitator, is out of the room. The facilitator then returns and facilitates the subject’s response. If the response is appropriate, message-passing has succeeded.

The authors suggest that, in the case of non-speaking children, message-passing may fail for reasons that have nothing to do with agency. Their claim:

“Children who can talk receive years of prompting and feedback from adults on how to report information their interlocutor does not know, the essence of a message passing test.”

There are several problems with this statement. First, message-passing involves information that is unknown to the facilitator, not to the interlocutor (the researcher). Second, the statement suggests that typical three-year-olds don’t yet know how to talk with people about things that happened while they were out of the room. Third, as the study itself reports, while no participant “was reported to be able to have a ‘to and fro’ spoken conversation involving turn-taking or building on what a conversational partner had said earlier,” all but one “was reported to be able to speak using short phrases or sentences.” (What would cause these individuals to become more conversational when typing on a letter board with an index finger is left unexplored).

My other concern is the letter board, which could have been placed on a stationary stand, but instead is held up by the assistant. As we see in the article’s videos, the board shifts around significantly during typing.

A shifting board makes it hard to draw reliable conclusions about intentional eye fixations or about intentional letter selection—regardless of how high tech the head-mounted eye tracker and video processing software are.

Of course, as far as the authors go, if what you’re after is publicity for, say, an un-disclosed for-profit operation, all that matters is that you get the word out to lots of potential customers before you get scrutinized by actual peer review and ultimately retracted. It could be that spending over $5000 dollars to promote Rapid Prompting Method is a very worthwhile investment.

Especially given how much money desperate parents are willing to pay for anything that appears to boost the communicative potential of their autistic children to the degree promised by RPM.

Facilitating communication with Dan Willingham

I’ve long respected Dan Willingham’s work, but I’m concerned about a study he tweeted favorably about on twitter this week.

I and others tweeted some criticisms, and Dan posted a rebuttal on his blog. In this post, which I’ll also share on twitter, I’m going to respond, point by point, to that rebuttal.

First, a few notes about my respect for Dan and his work. When his first book came out, I gave it a 5-star review on Amazon; I linked to an article of his on phonics here just 3 months ago; and I’ve had cordial, even friendly, exchanges with him over the years.

However, the study Dan cited has raised red flags for me. Its object of investigation—Rapid Prompting Method—is similar to a discredited method called “Facilitated Communication” (FC). Both have been used with minimally-verbal children with autism, and both are things I’ve blogged about, most recently here.

RPM has never undergone the rigorous testing that FC has, and a number of us, long familiar with the litany of non-rigorous tests, anecdotes, and unsubstantiated claims that purportedly support RPM, took a look at who was involved in this study and immediately had concerns.

One concern: the conflicts of interest of the first author, Vikram Jaswal. First, there’s his long-time collaboration with the person who runs the clinic, whose teaching method (a form of RPM) the study investigated and drew favorable conclusions about. Second, there’s the fact that this method is used with Jaswal’s autistic daughter. From an article in the Washington Post:

Another concern: Jaswal’s collaborator, whom various indicators suggest was also the assistant overseeing the experiment. She–I’ll avoid using her name–has a history of claiming credentials she did not have, including in her dealings with paying clients: dealings relevant to the questions addressed by the experiment.

Unimpressed with our remarks, Dan ultimately tweeted:

Soon a number of threads spun out containing discussions of the study’s methodology and data, including this one from @24shaz. @angelrabanas analyzed one video in detail and posted her analysis on YouTube.

So I pinged Dan, and a day or two later he notified us that that he’d put up a post on his blog in response entitled “Responding to a Study You Just KNOW Is Wrong.”

Dan begins his post with a discussion of the learning styles controversy, and how educators who think they are “in the know” will pounce “with glee” on anyone who mentions learning styles. He goes on to say:

This made me wonder whether any of us had said anything bullying or snobbish about Jaswal et al or their studies. So I went through the various threads, and the only things I found that I thought might possibly be interpreted as snobbish or bullying were these:

Anyway, Dan goes on to explain how if you uncritically dismiss disproven theories you might miss new developments. As an example of a new development that he has not missed, Dan cites new research on learning styles theory and notes that his views have changed as a result.

I’m familiar with Dan’s skepticism about learning styles, and so I was intrigued to hear he’d changed his mind. I followed the link he gave to his new views, read through to the conclusion, and this is what I found:

But anyway, back to Dan’s blog post. Dan next turns to Jaswal et al’s paper:

Missing from this discussion is the most likely way that cuing could occur: not through a distinct cue for each of the 26 letters of the alphabet, but through the movement by the facilitator of the letter board. Such movement is a regular occurrence in videos of RPM, including those that are included in this study.

In particular, the facilitator may (unwittingly) shift the letter board such that a particular letter approaches the subject’s finger and/or enters the path of his/her eye gaze. To pick up on such cues, the subject does not look up at the assistant.

A moving letter board also makes it hard to draw reliable conclusions about intentional eye fixations or about typing rhythm.

Anyway, moving on, Dan turns to some of our concerns. Noting (correctly) that I mischaracterized Jaswal’s relationship to something called “The Tribes” (thus mischaracterizing the precise nature of Jaswal’s collaboration with clinic’s director), he turns to a tweet by Jason Travers, observing that:

Dan leaves it at that. But Jason is making an important point. Yes, he’s asking for a different experiment, but not one that differs in the ultimate questions being explored (i.e., the authorship of the messages). Rather, Jason is asking for an experiment that addresses these same questions using a more rigorous, well-established methodology: one where the possibility of facilitator influence is eliminated, as it is in a message passing test.

Jaswal et al attempt to explain why they think message-passing tests are problematic, making some claims that anyone with any background in language acquisition and linguistic pragmatics would instantly recognize as absurd:

Years of prompting and feedback are needed before children are able to convey new information? Tell that to the parent of a young toddler! Then there’s the fact that all of the participants, although they’re called “nonspeaking,” can talk—just not fluently and interactively.

What would cause these kids to be more fluent and interactive when typing on a letter board with an index finger? That is a question that Jaswal et al leave unexplained.

Even if Jaswal et al had good reasons for rejecting message passing, they’d still need to explain why they didn’t take other influence-reducing steps–like blindfolding the assistant and/or placing the alphabet board on a stationary stand. Such options are never even mentioned.

Dan does not discuss these huge design flaws. Nor does he address our concerns about how a moving board (1) provides cues that don’t involve subjects looking up at the assistant and (2) seriously warps the eye gaze results. Nor does he address various concerns we had about how the data was coded.

Instead, having dispensed with Jason, Dan then reacts to two tweets that, he grants, do address the data and its methods.

I’ve reviewed these and other tweets of @24shaz and @angelrabanas re the videos, and as far as I can tell, Dan’s reaction here is a non sequitur that does not rebut their critiques. Doing that convincingly would require quite a bit more discussion.

Dan then turns to a tweet I posted:

So I went back to the methods section to see if I had missed something:

There’s nothing here, so far as I can tell, that’s relevant to the issue of oral cues being delivered ahead of letter selection. Nor does this issue come up anywhere else in the methods section.

Having dispensed with the above tweets, Dan turns to one regarding the study’s small sample size and explains how these are routine in neuropsychology. He does not address concerns about how the subjects were non-randomly selected–they were hand-picked by the clinic–or about the grounds for eliminating one of the subjects.

Dan then dispenses with the rest of our tweets, among them many more that focused on methodology and data (and a mysteriously missing video) with this:

Alluding to his opening remarks, Dan then contrasts his take on our tweets with his take on his openness to revising his views on learning styles.

He goes on to say that people would be best advised to just ignore studies that they aren’t going to bother to read and that are outside their areas of expertise:

Regarding the second point, on expertise, what’s most relevant here, as far as our critiques go, is the expertise involved in evaluating RPM and FC videos for signs of manipulation. Some of us have spent many hours doing this—perhaps more hours than this paper’s authors, reviewers and advisors have.

Regarding the first point, about bothering to read things, the same applies to Dan himself. If he has the feeling that a conclusion (say one regarding Jaswal’s paper) is probably wrong, but doesn’t want to take the time to properly engage with the arguments, he might be better off ignoring it.

Incidentally, if Dan has a feeling that Jaswal’s critics are probably wrong, he has left out one possible reason why: he and Jaswal are colleagues in the same department at UVA, and Dan is mentioned in the paper’s acknowledgements:

Dan concludes with this:

If Dan were to inform himself about the harsh and abusive practices in both FC and RPM (evident in some of the videos of kids who are subjected to it), and about the linguistic and educational opportunity costs that they potentially impose, and about the evidence of greed and malice within the FC/RPM industrial complex, he might hesitate to interpret our passionate concern as nothing more than righteous indignation.

Facilitating Un-facilitated Communication in Autism

My two hour talk on this charged topic is going live at 5:00 PM EST today, available any time after that:

https://thereadingleague.uscreen.io/programs/le-may-beals-cut-1mp4-de3af4?categoryId=24423

There’ll be a live q & a on Monday, and I’m hoping someone will bring up a paper, just published in Nature of all places, that appears to provide empirical support for a particular type of facilitated communication:

https://www.nature.com/articles/s41598-020-64553-9

I’ve had an… interesting exchange with Dan Willingham on twitter about paper:

Very clever, possibly explosively imp #autism study. Some Rs have dismissed non-verbal autistic letterboard use bes the asst holding the board might be cuing. New study: speed, accuracy, timing, of eye movements suggest they are spelling, aren't cued. https://t.co/TZgA0s7W2w

— Daniel Willingham (@DTWillingham) May 12, 2020

 

After 100+ autism interviews, it’s time to debrief

I’m finally coming up for air after an intensive autism project funded by National Science Foundation. We had seven weeks to conduct at least 100 interviews–mostly with parents of autistic kids and with autism-focused teachers and therapists. The unrelenting stress of those seven weeks (which also involved weekly homework, lectures, and presentations, two trips to Boston, and a boot-camp ethos throughout) reminded me of the unrelenting stress I felt during the most difficult eras of J’s childhood.

And the difficulty I found in tracking down autism parents made me wonder whether autism is quite the epidemic people say it is.

My specific target was parents of children somewhere in the middle of the autism spectrum: kids who can recognize and produce at least a few spoken and written words, but who continue to struggle at least to some extent in putting those words together into grammatical phrases and sentences.

In the end, I spoke with about 40 parents, just barely enough to meet our weekly quotas and not get yelled at. Actually, the fear of being yelled at—funny how that doesn’t fade away with age!—was ultimately a good thing, as it resulted in some really interesting interviews.

Here are my main takeaways (some of these will be familiar to anyone familiar with autism):

Final installment on Neurodiversity and Facilitated Communication

So why is it that hard-core Neurodiversity (ND) advocates embrace the thoroughly-debunked and often abusive practice of Facilitated Communication (FC)?

Support by hard-core Neurodiversity advocates for Facilitated Communication

In my previous post I laid out the core tenets of a group of people who identify as autistic and affiliate themselves with a movement called “Neurodiversity.”

…And one more

Here’s one more example of facilitation via held-up alphabet board.

But first, here are a few more!

I’ve been gearing up for a promised post examining the various messages generated by Facilitated Communication, but in the meantime a few other FC examples have crossed my path.

Facilitated Communication: up close and personal

Time for my promised close, critical look at specific instances of facilitated communication—FC for short.

But first, a preliminary note.  In being critical in what is an extremely sensitive area, I don’t want to reveal names of kids and parents. I’ll provide links to material that’s been made publicly available–stuff posted on the Internet, mostly by or in collaboration with family members. But in what I write here, I’ll be avoiding names or abbreviating them.

Why is Facilitated Communication making a comeback? (Part II–a very bleak post)

It’s now time, in this series on autism, neurodiversity, and language learning, to return to two key criteria for autism in the most recent DSM (the DSM V), each of them tapping into deficits in what’s called Joint Attention:

• Deficits in social-emotional reciprocity, ranging, for example, from abnormal social approach and failure of normal back-and-forth conversation; to reduced sharing of interests, emotions, or affect; to failure to initiate or respond to social interactions.
• Deficits in nonverbal communicative behaviors used for social interaction, ranging, for example, from poorly integrated verbal and nonverbal communication; to abnormalities in eye contact and body language or deficits in understanding and use of gestures; to a total lack of facial expressions and nonverbal communication.