30 July 2021
Feasibility of Repeated Administration of Automated Verbal-Paired-Associate Memory in Older Adults
At AAIC 2021, Alex Kaula - R&D Scientist, hosted an interactive poster session on: Feasibility of Repeated Administration of Automated Verbal-Paired-Associate Memory in Older Adults
Read on for the key findings and full poster.
The Verbal-Paired-Associate (VPA) task has been used as a single-visit assessment of memory in older adults for many years. In its standard form it is a set of eight word-pairings, delivered and tested verbally in clinic by a trained rater.
Cambridge Cognition has developed a system that uses Text-to-Speech (TTS) and Automated Speech Recognition (ASR) to deliver and score a version of VPA that can be used repeatedly, remotely, and at much greater scale than is possible with the human-delivered task.
Our automated VPA assessment is intended to be used for testing the population of older adults who may be suffering from memory problems. Desirable characteristics in our task are that performance is stable across repeated tests, that the user finds it accessible and enjoyable to use, and that the range of performance has useful diagnostic properties.
We therefore present results from two VPA-based studies in older adults where we aimed to address these issues: first, a pilot study that we used for stimulus-selection, and secondly a longitudinal study in which we administered five iterations of the task, one per day, which we followed with a questionnaire
- Participants are instructed to listen carefully and to try to memorise word-pairs
- Text-to-Speech (TTS) engine delivers word-pairs using laptop, tablet, or smartphone audio
- TTS then gives prompts to cue recall (e.g. ‘What word went with River’… ‘…and what word went with Picture’) until all 8 pairs have been tested.
- Task terminates when participants have had three study-test ‘attempts’, or when all pairings have been successfully recalled
To establish a good range of difficulty for task stimuli, we ran an online cross-sectional study on the Prolific platform in 188 adults aged 18-76 (M = 39, SD=17). Participants were paid £3 for ~24 min participation. Participants performed two variations of the NeuroVocalix VPA task, attempting to learn 2 different sets of 8 word-pairs.
Task performance was scored using both ASR and human review processes, and we examined word-pair memorability characteristics. We then selected 5 sets of 8 word-pairs with each set comprising pairs from across a range of difficulty, to be used over the 5-day longitudinal study.
Five-Day Longitudinal Study
Having selected our 5 sets of 8 word-pairs with what we hypothesised to be commensurable difficulty ranges, we used the Prolific online platform to recruit healthy anglophone participants aged 55 and upwards, to take part in a 5-day study. We recruited 42 healthy participants aged 55-85.
Each day, participants logged in and completed up to 3 attempts at learning the new set of word-pairs, and after a short distractor task entered a ‘Delayed Recall’ phase
The main questions we wanted to address in our longitudinal study were:
- First, whether our repeated version of the task would exhibit a good overall range, and if so whether this would be stable over repeated visits.
- Second, it’s important that participant performance is consistent across visits, indicated by test-retest reliability.
- Finally, we are looking for compliance and user satisfaction.
Performance over the task as a whole was relatively stable across visits, and the lower box-plot (Figure 2) shows that some ceiling effects occurred only at the 3 attempt and in the 4th delayed recall phase, where the more able participants did achieve criterion.
We also saw a particular improvement in consistency after the first visit (Figure 3), indicating the importance of a full-length tutorial session during familiarisation.
Figure 4 shows that compliance was at a generally acceptable level for online opportunistic recruitment of volunteers, and questionnaire results indicated participants generally were very happy with the interface and usability of the task.
This first small longitudinal study using the NeuroVocalix platform suggests that the technology and task are well-tolerated by an age-representative sample. The change in test-retest reliability between visits 1&2 and visits 2&3 indicate a full-length tutorial session during screening will be an important feature, and we have now implemented this measure.
The task shows acceptable performance characteristics at the individual test level, where it appears sensitive across a range of ability, and as a repeated test where the range is consistent, as well as showing acceptable visit-to-visit consistency within participants.
R&D Scientist, Cambridge Cognition - Alex Kaula