Towards greater validity in Schwartz’s portrait values indicator using experimental research

• Published in: Quality & quantity Vol. 50; no. 4; pp. 1567 - 1587
• Main Authors: de Wet, Jacques P., Bacher, Johann, Wetzelhütter, Daniela
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01-07-2016; Springer; Springer Nature B.V
• Subjects: Cognition; Colleges & universities; Conformity; Errors; Experiments; Hypotheses; Measurement; Measurement errors; Methodology of the Social Sciences; Power; Questionnaires; Research design; Social Sciences; Sociology; Statistical analysis; Universalism; Validity; Values; South Africa; Austria; Portrait values questionnaire; Experiment; Measurement error; Rigor
• ISSN: 0033-5177; 1573-7845
• DOI: 10.1007/s11135-015-0221-1

In Schwartz’s 21-item portrait values questionnaire (PVQ) each item consists of two statements which refer to a single underling value. In each of the 21 items the respondents are asked to give a
single
response to the two statements. Anecdotal evidence and cognitive response theory alert us to potential measurement error in this two-statements-single-response approach. Respondents might be influenced by different themes contained in the statements and their answers might not be comparable. This paper addresses the question:
Do the responses to the 21 items in Schwartz’s PVQ
(
where each item contains two statements
)
differ significantly when two separate responses are allowed per item
(
in the split version
)
compared to when a single response is allowed
(
in the combined version
)
?
In order to answer this research question we adopted an experimental design in a two-wave panel study. In the first wave we used Schwartz’s combined version of the PVQ. In the second wave we split the two statements and treated each statement as a separate item, thus requiring responses to each statement. Data was collected from Sociology classes at two universities: one in Austria (n = 52) and the other in South Africa (n = 61). We used statistical and non-statistical methods of analysis. The overall statistical assessment (z test) supports the split version although not all the various z test results unanimously concur. The non-statistical assessment does not support either the split version or the combined version. These mixed results necessitate further interrogation of the continued use of the combined version in the PVQ.