Quantification of behavioral data with effect sizes and statistical significance tests

This article describes the use of statistical significance tests and distance‐based effect sizes with behavioral data from single case experimental designs (SCEDs). Such data often are interpreted only with visual analysis. However, a growing movement in the field is to quantify results to improve d...

Full description

Saved in:

Bibliographic Details
Published in:	Journal of applied behavior analysis Vol. 55; no. 4; pp. 1068 - 1082
Main Authors:	Costello, Mack S., Bagley, Raymond F., Fernández Bustamante, Laura, Deochand, Neil
Format:	Journal Article
Language:	English
Published:	Hoboken, USA Wiley Subscription Services, Inc 01-10-2022 Blackwell Publishing Ltd
Subjects:	Applied behavior analysis behavior analysis Design of experiments effect size single‐case experimental design single‐case research Statistical significance
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	This article describes the use of statistical significance tests and distance‐based effect sizes with behavioral data from single case experimental designs (SCEDs). Such data often are interpreted only with visual analysis. However, a growing movement in the field is to quantify results to improve decision‐making and communication across studies and sciences. The goal of the present study was to assess the agreement between visual analysis and various statistical tests. We recruited visual analysts to judge 160 pairwise data sets from published articles and compared these analyses to significance tests and effect sizes. One‐tailed significance testing of Tau z and the percentage of pairwise differences in the predicted direction (PWD) generally agreed with each other, and complemented the effect sizes of Ratio of Distances (RD) and g. Visual analysis was somewhat unreliable and should be combined with statistical complements to maximize decision accuracy.
Bibliography:	The authors would like to acknowledge Michael T. Carlin for assistance with the preparation of this manuscript, Michael DeMarco for assistance with a previous version of this manuscript, and Alexus Reyes for assistance with collecting sample data. ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23
ISSN:	0021-8855 1938-3703
DOI:	10.1002/jaba.938