18 July 2019
Large-Scale Remote Assessment of Verbal Cognitive Function Using Automatic Speech Recognition
At AAIC 2019, we shared how to administer and score verbal neuropsychological assessments on participants own devices in a way that is acceptable, feasible and reliable.
Verbal neuropsychological tests are widely used to identify early signs of neurodegeneration in older adults. To date, employing these tasks in large-scale screening and home-based monitoring programmes has been limited by the dependence on skilled raters.
In efforts to address this need, we report data from a large sample of participants tested in their own homes showing that traditional tasks can be reliably administered and scored via device-agnostic web-based technology (Cambridge Cognition’s Neurovocalix platform).
Participants were tested at home, on their own devices, using the Cambridge Cognition Neurovocalix platform for automated verbal testing. Data was collected on the operating system, web-browser and device used. Following quality review, an analysis set of 2,588 was used.
Results are presented from two verbal tasks:
- Verbal Paired Associates Learning
- Digit Span Forward and Back
For Verbal Paired Associates, in line with other versions of the task, probability of success increased across the three learning trials, particularly for hard trials.
In Verbal Digit Span, accuracy decreased as number of items to be recalled increased.
Relationship with participant demographics
Results of the Ordinary Least Square analysis of the relationship between performance and demographics are presented in Figure 1.
For Verbal Paired Associates, Total Recall (Figure 1A) and ratio of easy to hard pairs recalled (Figure 1B) were both significantly related to Age, consistent with the existing literature. Education and Gender significantly predicted Total Recall, but not easy/hard ratio. Language was not predictive of performance.
In Verbal Digit Span, Education and Gender were significant predictors across both Digits Forwards (Figure 1C) and Backwards (Figure 1D). Age and Language were significantly associated with performance only in the digit forwards condition.
Figure 1. Standardized estimates from OLS models of demographic variables on verbal paired associates total recall (A) and ratio of easy and hard trials (B), and data from digits forward percent recall (C) and backwards percent recall (D).
- These data demonstrate the feasibility of large scale, automated verbal testing.
- We replicate well-established findings from conventionally administered verbal tasks, supporting the validity of remote verbal testing using ASR technology.
- We observed expected associations between verbal episodic memory measures and age, gender and education. Education level and Gender were the strongest predictors of verbal working memory. Language was associated only with worse performance on Forward Digit Span, but with small effect size, supporting the scalability and robustness of the system.
- In future work, we aim to extend these findings in patients with neurodegenerative disease.
Francesca Cormack and Nick Taptiklis