PASAA
Publication Date
2017-01-01
Abstract
Using generalizability (G-) theory and rater interviews as both quantitative and qualitative approaches, this study examined the impact of scoring methods (i.e., holistic versus analytic scoring) on the scoring variability and reliability of an EFL institutional writing assessment at a Turkish university. Ten raters were invited to rate 36 undergraduate argumentative essays first holistically and then analytically, with a three-week time interval. The quantitative results indicated that with proper rater training holistic scoring can produce as reliable and dependable assessment outcomes as analytic scoring. Similarly, the qualitative results revealed that all raters prefer using the holistic scoring method because it could help them not only assign fair and objective scores to essays but also facilitate their scoring process. Further, most raters agreed that the content of an essay was the most important factor that most affected their holistic scoring decision making of an essay. In contrast, all aspects of an essay (e.g., grammar, content, or organization) jointly affected their analytic scoring decision making of an essay. Important implications for EFL writing assessment professionals in the institutional assessment context are discussed.
DOI
10.58837/CHULA.PASAA.53.1.5
First Page
112
Last Page
147
Recommended Citation
Han, Turgay and Huang, Jinyan
(2017)
"Examining the Impact of Scoring Methods on the Institutional EFL Writing Assessment: A Turkish Perspective,"
PASAA: Vol. 53:
Iss.
1, Article 5.
DOI: 10.58837/CHULA.PASAA.53.1.5
Available at:
https://digital.car.chula.ac.th/pasaa/vol53/iss1/5