ASSESSING OBSERVER ACCURACY IN CONTINUOUS RECORDING OF RATE AND DURATION: THREE ALGORITHMS COMPARED

The three algorithms most frequently selected by behavior‐analytic researchers to compute interobserver agreement with continuous recording were used to assess the accuracy of data recorded from video samples on handheld computers by 12 observers. Rate and duration of responding were recorded for th...

Full description

Saved in:

Bibliographic Details
Published in:	Journal of applied behavior analysis Vol. 42; no. 3; pp. 527 - 539
Main Authors:	Mudford, Oliver C., Martin, Neil T., Hui, Jasmine K. Y., Taylor, Sarah Ann
Format:	Journal Article
Language:	English
Published:	Oxford, UK Blackwell Publishing Ltd 2009 Society for the Experimental Analysis of Behavior The Society for the Experimental Analysis of Behavior
Subjects:	Accuracy Adult Adults Algorithms Applied Behavior Analysis Behavior Behavior Patterns Biological and medical sciences Computers continuous recording Data Analysis Data collection Female Fundamental and applied biological sciences. Psychology Handheld computers Humans interobserver agreement Interrater Reliability Male Methods Middle Aged New Zealand Numerical Analysis, Computer-Assisted Observation observational data observer accuracy Observer Variation Psychological Patterns Psychology. Psychoanalysis. Psychiatry Psychology. Psychophysiology Psychometrics. Statistics. Methodology recording and measurement Research Assistants Resistance (Psychology) Signal Processing, Computer-Assisted Software Statistics. Mathematics Studies Therapists Time Factors Time Factors (Learning) Undergraduate Students Video Recording - methods Video Technology Young Adult New Zealand Human Evaluation Interrater agreement Observation interobserver agreement continuous recording observer accuracy Test reliability observational data Algorithm recording and measurement
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	The three algorithms most frequently selected by behavior‐analytic researchers to compute interobserver agreement with continuous recording were used to assess the accuracy of data recorded from video samples on handheld computers by 12 observers. Rate and duration of responding were recorded for three samples each. Data files were compared with criterion records to determine observer accuracy. Block‐by‐block and exact agreement algorithms were susceptible to inflated agreement and accuracy estimates at lower rates and durations. The exact agreement method appeared to be overly stringent for recording responding at higher rates (23.5 responses per minute) and for higher relative duration (72% of session). Time‐window analysis appeared to inflate accuracy assessment at relatively high but not at low response rate and duration (4.8 responses per minute and 8% of session, respectively).
Bibliography:	ArticleID:JABA1670 istex:5824935A250D5E5FCFCDEDB454FAFB5C788DEC23 ark:/67375/WNG-8WHTC53Q-R Sarah Taylor is now employed by Odyssey House, Auckland, New Zealand. Jasmine Hui is now employed at Rescare Homes Trust, Manukau City, New Zealand. ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23
ISSN:	0021-8855 1938-3703
DOI:	10.1901/jaba.2009.42-527