Keywords
assessment, QL/QR
Abstract
This paper introduces a rubric for assessing QR in student papers and analyzes the inter-rater reliability of the instrument based on a reading session involving 11 participants. Despite the disciplinary diversity of the group (which included a faculty member from the arts and literature, two staff members, and representatives from five natural and social science departments), the rubric produced reliable measures of QR use and proficiency in a sample of student papers. Readers agreed on the relevance and extent of QR in 75.0 and 81.9 percent of cases respectively (corresponding to Cohen’s κ= 0.611 and 0.693). A four-category measure of quality produced slightly less agreement (66.7 percent, κ = 0.532). Collapsing the index into a 3-point scale raises the inter-rater agreement to 77.8 percent (κ = 0.653). The substantial agreement attained by this rubric suggests that it is possible to construct a reliable instrument for the assessment of QR in student arguments.
DOI
http://dx.doi.org/10.5038/1936-4660.3.1.3
Recommended Citation
Grawe, Nathan D., Neil S. Lutsky, and Christopher J. Tassava. "A Rubric for Assessing Quantitative Reasoning in Written Arguments." Numeracy 3, Iss. 1 (2010): Article 3. DOI: http://dx.doi.org/10.5038/1936-4660.3.1.3
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License
