Geographic Variation and Instrumentation Artifacts: in Search of Confounds in Performance Validity Assessment in Adults with Mild TBI

• Published in: Psychological injury and law Vol. 12; no. 2; pp. 127 - 145
• Main Authors: Lichtenstein, Jonathan D., Greenacre, Matthew K., Cutler, Laura, Abeare, Kaitlyn, Baker, Shannon D., Kent, Katrina J., Ali, Sami, Erdodi, Laszlo A.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01-06-2019; Springer Nature B.V
• Subjects: Achievement tests; African Americans; Base rates; Behavioral Science and Psychology; Clinical assessment; Clinical decision making; Clinical Psychology; Cognitive ability; Confounding factors; Credibility; Demographics; Demography; Estimates; Geography; Intelligence tests; Law and Psychology; Medical diagnosis; Neuropsychology; Psychology; Regional variations; Traumatic brain injury; Validity; United States > US; Geographic variability in cognitive ability; Instrumentation artifacts; Performance validity
• ISSN: 1938-971X; 1938-9728
• DOI: 10.1007/s12207-019-09354-w

Regional fluctuations in cognitive ability have been reported worldwide. Given perennial concerns that the outcome of performance validity tests (PVTs) may be contaminated by genuine neuropsychological deficits, geographic differences may represent a confounding factor in determining the credibility of a given neurocognitive profile. This pilot study was designed to investigate whether geographic location affects base rates of failure (BR
Fail
) on PVTs. BR
Fail
were compared across a number of free-standing and embedded PVTs in patients with mild traumatic brain injury (
m
TBI) from two regions of the US (Midwest and New England). Retrospective archival data were collected from clinically referred patients with
m
TBI at two different academic medical centers (
n
Midwest
 = 76 and
n
New England
 = 84). One free-standing PVT (Word Choice Test) and seven embedded PVTs were administered to both samples. The embedded validity indicators were combined into a single composite score using two different previously established aggregation methods. The New England sample obtained a higher score on the Verbal Comprehension Index of the WAIS-IV (
d
 = .34, small-medium). The difference between the two regions in Full Scale IQ (FSIQ) was small (
d
 = .28). When compared with the omnibus population mean (100), the effect of
m
TBI on FSIQ was small (
d
 = .22) in the New England sample and medium (
d
 = .53) in the Midwestern one. However, contrasts using estimates of regional FSIQ produced equivalent effect sizes (
d
: .47–.53). BR
Fail
was similar on free-standing PVTs, but varied at random for embedded PVTs. Aggregating individual indices into a validity composite effectively neutralized regional variability in BR
Fail
. Classification accuracy varied as a function of both geographic region and instruments. Despite small overall effect sizes, regional differences in cognitive ability may potentially influence clinical decision making, both in terms of diagnosis and performance validity assessment. There was an interaction between geographic region and instruments in terms of the internal consistency of PVT profiles. If replicated, the findings of this preliminary study have potentially important clinical, forensic, methodological, and epidemiological implications.