Here, as promised is the full text of my critique:
Jaswal et al’s Eye-tracking Reveals Agency in Assisted Autistic Communication (Scientific Reports, May, 2020) reports an experiment that tracks the eye movements of autistic individuals while they point to letters on held-up letterboards in response to questions. The eye-tracking data, the authors argue, indicates that it is the autistic individuals, rather than the person holding up the letterboard, who are the agents/authors of the typed-out messages. This study, however, is based on faulty assumptions that undermine both its rationale and its conclusions: assumptions about test performance; about prerequisites for certain linguistic skills; about a disconnect between speech-based vs. typing-based communication skills; about what it would take for someone to point to letters based on cues from the person holding up the board; and about the study’s central premise: the agency of eye gaze.
The held-up letterboard is one of several related, controversial interventions used with non-speaking individuals. In these interventions, a “facilitator” or “assistant” holds or touches part of the person’s body, or holds up a keyboard or letterboard for them, while the person points to letters and spells out messages. Both contact-based facilitation and held-up-board-based assistance leave wide open the possibility of cuing—e.g., by physical pressure or board movement—and thus raise questions about who is directing the messages. The ideomotor illusion can cause facilitators/assistants to unwittingly project their own movements onto the person they are assisting; Clever Hans-like cuing effects may convey subtle signals that further guide typing. These psychological phenomena raise the possibility that messages are unwittingly directed by the facilitator/assistant.1
The most direct way to validate authorship is through “message passing” tests involving prompts that the facilitator has not seen ahead of time. For contact-based facilitation, studies dating back to the 1990’s indicate that facilitators do in fact guide typing.2,3,4 Results show that the overwhelming majority of typed responses—when they occur—are based on prompts that the facilitator witnessed, and not on prompts that only the typist witnessed, strongly suggesting that the facilitator is the actual author. As for assistance via held-up letterboards, practitioners have yet to participate in rigorous, published, message passing experiments.5
To explain why they have eschewed message passing tests in favor of expensive eye-tracking equipment and painstaking coding of eye-gaze data, Jaswal et al claim that message-passing tests are problematic. They state that (1) “tests that fail to take into account a group’s unique developmental history can underestimate or misrepresent the abilities of members of that group”; (2) non-speaking autistic children miss out on communicative experiences necessary for message passing; and (3) anxiety and lack of familiarity with the test setting may further impede performance.
In support of the first claim, the authors turn to the field of cultural studies, citing a classic paper by Labov and another by Cole and Bruner that date back to the 1970s. These papers discuss how test performance may be affected by cultural context.6,7 The authors also cite Heinrich and Norenzayan, who discuss how empirical psychology has been has been skewed by subjects coming predominantly from industrialized western societies.8 Neither these papers, nor Jaswal et al, connect the dots to message passing tests or to individuals with autism.
Regarding the second claim, the authors state that children who can talk, in contrast to non-speakers, “receive years of prompting and feedback from adults on how to report information their interlocutor does not know.” This, they assert, “is the essence of a message passing test.” (Actually, it is information that is unfamiliar to the facilitator, not to the person asking questions, that is essential to message passing). Their one citation, Nelson and Fivush (2004), specifically addresses autobiographical memory; message passing tests typically involve semantic memory (identifying the names of objects; answering factual questions).9 Furthermore, as Nelson and Fivush note, children report autobiographical memories “at about 18-20 months of age”—well before the hypothetical “years of prompting and feedback” that the authors claim is prerequisite.10 As for reporting information that their interlocutor does not know, as Baker and Greenfield (1988) found, children as young as 17 months, even at the one-word stage, use language to highlight new information.11
The author’s third claim is that the “unfamiliar experimental setting”, combined with “elevated levels of anxiety common in autism”, may help explain difficulties with message passing tests. The anxiety defense is belied by (1) the care taken in many of the tests to make subjects as comfortable as possible12 (in particular, there is no mounting of eye-trackers to subjects’ heads), and (2) the unlikelihood that anxiety would lead—let alone enable—subjects to type out something that the facilitator saw and the typist didn’t. As for “unfamiliar test setting” issues, the authors cite Cardinal et al’s (1996) study in which message passing performance improved over multiple sessions.13 Error rates, however, remained high, and the study has been criticized for several design flaws.14
Jaswal et al, moreover, fail to explain why the subjects were able to deliver the often lengthy responses to the open-ended questions reported in this study when pointing to letters on a letterboard via hunt-and-peck-style typing, but not when articulating words through speech (a much less time-consuming process). The authors state that “all but one participant was reported to be able to speak using short phrases or sentences” but “none could respond verbally to open-ended questions of the type they were asked in this study.” There is, however, no officially recognized language disorder that combines extant oral skills with pragmatic skills that only emerge during hunt and peck typing.
What about the possibility that experimental subjects are pointing to letters based on cues from the assistant? According to the authors:
On a cueing account of a letterboard user’s performance, the assistant would need to deliver a cue that identified which of 26 letters to point to, and the user would need to detect, decode, and act upon that cue. Each of these steps would take time and would be subject to error, especially given the subtlety of the cues the assistant is hypothesized to deliver and the 26 cue-response alternatives.
Completely overlooked here is the most obvious means of cuing: board movements that bring specific letters closer to the person’s extended finger. This involves no decoding whatsoever and speeds up rather than slows down the typing.
Subtle board movements are evident in the study’s supplemental videos, and this highlights a final problem. Even if you accept the authors’ justifications for eschewing message passing tests, and even if you somehow rule out the possibility of board-holding assistants providing cues, a non-stationary letterboard calls into question the study’s central premise: the purported agency of the subjects’ eye gaze. Were subjects intentionally looking at letters, or were letters shifting into their lines of sight? That is something that no eye-tracking equipment, no matter how sophisticated, can answer.
Unless, of course, the board is placed on a stationary surface. But why stop there? If the ultimate goal is to test authorship via eye gaze, why not do so directly, by using the kind of eye-tracking software that lets subjects type with their eyes rather than their fingers—as hundreds of children around the world are successfully doing every day.
1. Wegner, Daniel M., Fuller, Valerie A., & Sparrow, Betsy Clever Hands: Uncontrolled Intelligence in Facilitated Communication. Journal of Personality and Social Psychology. 85, 5-19 (2003).
2. Moore, S., Donovan, B., & Hudson, A. Facilitator-suggested conversational evaluation of facilitated communication. Journal of Autism and Developmental Disorders, 23, 541–552 (1993).
3. Wheeler, D. L., Jacobson, J. W., Paglieri, R. A., & Schwartz, A. A. An experimental assessment of facilitated communication. Mental Retardation, 31, 49–59 (1993).
4. Saloviita, T., Lepannen, M., & Ojalammi, U. Authorship in facilitated communication: An analysis of 11 cases. Augmentive and Alternative Communication, 3, 213-25 (2014).
5. Tostanoski, A., Lang, R., Raulston, T., Carnett, A., & Davis, T. Voices from the past: Comparing the rapid prompting method and facilitated communication. Developmental Rehabilitation, 17, 219–223 (2014).
6. Labov, W. The logic of nonstandard English in Language and Poverty: Perspectives on a Theme (ed. Williams, F.) 153–189 (Academic Press, 1970).
7. Cole, M. & Bruner, J. S. Cultural differences and inferences about psychological processes. Am. Psychol. 26, 867–876 (1971).
8. Henrich, J., Heine, S. J. & Norenzayan, A. The weirdest people in the world? Behav. Brain Sci. 33, 61–83 (2010).
9. Nelson, K. & Fivush, R. The emergence of autobiographical memory: A social cultural developmental theory. Psychol. Rev. 111, 486–511 (2004).
10. Nelson, K. & Fivush, R. The emergence of autobiographical memory: A social cultural developmental theory. Psychol. Rev. 111, 484 (2004).
11. Baker, N. D. & Greenfield, P. M. The Development of New and Old Information in Young Children’s Early Language. Language Sciences, Vol 10, No 1, 3-34, (1988).
12. Saloviita, T., Lepannen, M., & Ojalammi, U. Authorship in facilitated communication: An analysis of 11 cases. Augmentive and Alternative Communication, 3, 213-25 (2014).
13. Cardinal, D. N., Hanson, D. & Wakeham, J. Investigation of authorship in facilitated communication. Ment. Retard. 34, 231–242 (1996).
14. Mostert, Mark P. Facilitated Communication Since 1995: A Review of Published Studies. Journal of Autism and Developmental Disorders, Vol. 31, No. 3, 287-313 (2001).
In the comments below, I will post the reviewer responses I received from Scientific Reports, along with their reasons for not publishing this critique.
Catherine & Katharine 
Reviewer I wrote:
Much of the response appears fixated on message-passing experiments, which is a useful design, but cannot be seen as the only viable means of deriving useful information related to the question of autistic communicative agency. Jawal et al. (2020) point to a variety of reasons the results of prior message passing studies may not fully reflect the agency of all autistic individuals using spelling-based forms of communication. There are certainly points at which I feel Jaswal et al. could have done a better job of connecting the dots, but that alone hardly seems worth pointing out in an alternative publication.
In addition, Beals claim in the 8th paragraph seems to imply that there no officially recognized language disorders in which words could be typed and not spoken, which is simply untrue. There is an entire field of augmentative and alternative communication that has emerged because ideas can be more readily expressed through one modality than the another due to a variety of sensory-motor differences (e.g., ALS, Broca’s aphasia).
I think the most intriguing issue raised by this response is the potential role of letter board movement, although Jawal et al. (2020) addressed that as well (perhaps not definitively but with substance). I hope this helps in your decision-making.
LikeLike
Reviewer II wrote:
This letter responds to the rebuttal letter by Katharine Beals, PhD, who is questioning the results reported in a peer-reviewed study published by Prof. Vikram Jaswal from the U Virginia, Psychology Dept., in the Nature Journal Scientific Reports […] I will start from the end of the letter and will work my way up to the top.
“The eye-tracking software for computer-gaze interface “that lets subjects type with their eyes rather than their fingers—as hundreds of children around the world are successfully doing every day.”
Brain-computer interfaces (BCI) or brain-machine interfaces (BMI) such as those mentioned towards the end of the letter, were my specialty during my postdoctoral years at CALTECH. I developed algorithms and mathematical equations to help accelerate the training of such systems, to help people intentionally direct cursors and moving targets, even when feedback from the peripheral nervous systems signals was corrupted by noise, or absent altogether (due to severed spinal cord, etc.) My algorithms involving the detection of volition have been patented in the US and Europe. As such, I feel confident to speak about this matter. I have also published in peer-reviewed journals on the topic and written books whereby I explain the mathematics behind such seemingly “magic” act.
The process of learning to adapt the signals that one harnesses from the nervous systems of the person, e.g. the EEG brainwaves, the saccadic and pursuit eyes’ signals, the bodily kinematics or EMG biorhythms, the cortical or subcortical spikes, etc. and the external object’s motion (e.g. cursor to jump to a letter, the robotic arm to point, the prosthetic hand to grasp, etc.) happen to follow similar steps as those between a human trying to communicate and another human holding a letter board to receive the action and return a consequential response.
Both endpoints (the eyes traces coordinated with the hand traces of the autistic person) and the handheld letter-board, follow biorhythmic motion trajectories that at a microscopic level inevitably learn to entrain. Simultaneous processes take place that allow that end-product to most of the time be probabilistically correct. These processes are intentional and spontaneous in nature. There is a balance that must develop, such that initially the machine (in this case the facilitator) is in fact doing all of the work; followed by a hybrid phase whereby ~ � machine (facilitator) and � user (eyes-hand traces in this case) take turns in performing the task, and finally 100% the user (the eyes-hand traces in this case) dominate. At that point, the user is performing the task autonomously and independently.
The learning – adaptation process relies of subtle variations of the signals which probabilistically adapt towards the independent performance of the brain signals correctly (most of the time) outputting the desired outcome (the proper letter to form a phrase.)
These BMI/BCI systems are not ubiquitous just yet, but they will be some day owing to the type of research that Prof. Jaswal’s lab is doing. Furthermore, they will open a path to education to the 30% segment of the autism spectrum that has no proper alternative way to the existing methods today.
The nervous systems of these children and adults developed through different paths and rely on different mechanisms that we have yet to fully understand. We have acquired some understanding of these mechanisms in the last decades of intense neuroscience-based research. In the meantime, these minimally verbal folks are being forced to follow the same type of learning / teaching strategies as neurotypical children do, and are being measured with the same ruler to render them with some IQ-level which does not reflect their coping capabilities to function daily. Much is expected from this population without proper understanding of how their systems work, to provide proper support and accommodations and help them flourish […] The BCI/BMIs in use today still work as black boxes, because the map interfacing the nervous systems’ waveforms and those of the motions of the external object forms without explicit mathematical models in closed form. There is no analytic expression
to describe the unfolding learning that takes place throughout. There is trial and error and there are numerical computations using algorithms and local models that guide the process, much as the clinician holding the board does, leading the child to perform volitionally at an independent level, with agency. So, in a way, it is really good that this letter closed with that example, because it is precisely the type of process that goes on while a minimally verbal child learns to communicate with the aid of another person all the way until independence is attained.
“The Static Board”
The movement of the board is what inherently makes this a more natural form of communication, as it provides a subtle feedback to the child to promote conversation, rather than an unnatural one-way type of communication that does not generally exist in the world. Having the board statically placed defeats the purpose of the type of social emotional learning that we are shifting toward in education. Social interactions and the development of emotional support depend fundamentally on the presence of another human -as shown by the problems that social (physical) distancing are creating in the schools across the world. In this particular experiment/board-method the presence of the human is vital to gradually adapt the child until independence is attained. Subtle board movements are part of a dynamically complex interaction where those motions count too. It is part of the learning process that takes place.
“Jaswal et al, moreover, fail to explain why the subjects were able to deliver the often lengthy responses to the open-ended questions reported in this study when pointing to letters on a letterboard via hunt-and-peck-style typing, but not when articulating words through speech (a much less time-consuming process).”
[…] Training eye-hand coordination requires very different techniques than training the vocal apparatus or addressing the swallowing issues or the food-texture issues that many of these folks have. It takes a neuroscientist with proper neuroanatomy and neurophysiology training to know this, and to know it well within the context of autism, where behaviorists force them to do the very things that are asked in this letter. These assumptions that such people without proper training/credentials have made and forced onto these autistic fellows are in fact extremely dangerous.
For example, the anxiety argument that “care is taken to make the subject as comfortable as possible” is one of those which are in the danger zone. The human eye cannot see the distress of the heart signal, the electrodermal activities, the cortisol levels in saliva and the involuntary tremor in fight-flight mode of these folks, particularly when subject to such rigid requests. We have quantified several of these above mentioned levels at baseline in autism, and they are comparable to those of neurotypical folks under pain induced by injury, chronic disorders and sustained pressure/temperature levels. The heart is irregularly beating and the tremor at frequency bands corresponding to stress. Yet, the argument is that these fellows are OK and can be placed under arbitrarily determined situations by people who would not take the time to test their own assumptions / hypotheses, before carrying on their actions on a disabled person at disadvantage. It is something
penalized by the law under other circumstances whereby the person does not have a label of autism, but suffers from other disorders such as Ataxia, ALS, MS, CP, SYNGAP, FX, SHANK3 syndromes, etc. In such cases, support is 100% expected and the types of attitude Dr. Beal expresses here, severely penalized, as they violate the Helsinki Act.
“Message Passing (MP)”
[…] MP can work to a minimally verbal person’s advantage and should be tailored to that person’s non-verbal gestures and cues. In fact, one should develop a system that much like the BMI/BCI systems, learns the person’s code and teaches that to the other person (the facilitator), so they can communicate naturally. MP can be and must be better understood and coded so we can finally help these children communicate using their means, rather than imposing ours, just because it is inconvenient to us to make the effort and take that alternative route of learning who they are and how to best help them […] Human children are certainly capable of acting like Clever Hans when conditioned through Skinnerian animal conditioning techniques that behaviorists have forced on them in the autism world. But when we support them as Prof. Jaswal’s study did, when given the most elemental human right that we all enjoy, they flourish. And yes, there may be a learning
process whereby initially the facilitator may speak on their behalf, but in due time, that help yields its way to allow for an independent child with agency. It is the way we all learned from infancy with our mother and father, until we were able to act independently. Here the path may take a bit longer and be more convoluted than our path, but it is nonetheless destined to succeed when we respect them and nurture them.
“This study, however, is based on faulty assumptions that undermine both its rationale and its conclusions: assumptions about test performance; about prerequisites for certain linguistic skills; about a disconnect between speech-based vs. typing-based communication skills; about what it would take for someone to point to letters based on cues from the person holding up the board; and about the study’s central premise: the agency of eye gaze.”
These are very strong accusatory words. Let’s dissect them here:
“prerequisites for certain linguistic skills” This is not a study of linguistic but rather a study of how utilizing eye signals and pointing behavior combined whenever possible with vocalization of letters, it is possible to form phrases in response to questions. The goals of this study are far more modest than examining language. The level of performance is at the level of quantifiable volitional motor control and dyadic interactions.
“about a disconnect between speech-based vs. typing-based communication skills” The very point of the paper is to show how we have been missing that connection (though certain motor-sensation therapies exist in Argentina, whereby they do start by teaching typing and coordination/self-regulation emerges and leads to spontaneous speech in a significant number of cases.) Information flow exists in both domains but are hidden to the naked eye of the observer. The paper quantifies the type of communication that took place between two people, using several layers of data. This is a basic science study with clear reproducible methods. It is not a study to explain how language is acquired in humans […]
“about what it would take for someone to point to letters based on cues from the person holding up the board” […] To point to letters based on cues from a person holding up a board, it takes in the first place, to process all the sensory information relevant to the task; determine the goal; coordinate the huge number of degrees of freedom of the body in motion, so the brain launches the intended motor command and receives the feedback from the sensory signals that the action generates; consider that signal in relation to the original mental intent and distributes it across the different systems of the eyes in head, head in trunk, arm-hand in trunk standing upright against gravity (operating at different time scales and frequencies) to execute appropriate sensory-motor (coordinates) transformations, so the proper forces are generated to propel the hand in a controlled manner to the letter on the board (without knocking it down or never reaching it) and to do
all of that all over again under different stochastic (subtle) variations, because the person holding the board moved too. I have derived the partial differential equation that models that whole act geometrically, without the forces. It is a feat to do this and no robot can yet do it truly autonomously as these children/adolescents did.
LikeLike
The editor wrote:
Dear Dr Beals,
Thank you for your submission, “Response to Jasal et al’s Eye-tracking Reveals Agency in Assisted Autistic Communication”, to the Matters Arising section. After careful consideration, and consultation with external advisers (comments appended at the foot of this email), I regret to say that we cannot offer to publish it.
From the comments we received, there appear to be a number of limitations in the arguments presented, in the light of which, we have decided that publication of this exchange is not justified, because in our view it does not sufficiently add to our understanding or otherwise clarify the issues for our readership.
I am sorry we cannot be more positive on this occasion, but we appreciate your willingness to bring this matter to our attention.
LikeLike
Reblogged this on Autism Candles Blog.
LikeLike