All Journals

Language Teaching Research Quarterly

European Journal of Studies in Management and Business

International Journal of Behavior Studies in Organizations

Studies in Educational Management

New Challenges in Accounting and Finance

Language Testing in Focus: An International Journal

Marketing and Branding Research

Individual Differences in Language Education: An International Journal 

Feedback Research in Second Language 

Management Issues in Healthcare System

Language Teacher Education Research

Artificial Intelligence Knowledge Development

<p style="text-align: justify;">Rater reliability, the consistency of marking across different raters and times, is one important component of reliability regarding the quality of test scores. It is essential regarding performance assessment, such as writing, when the fairness of assessment results can come into question due to the subjectivity when scoring. The present study, part of a larger-scale funded research project, aimed to study this overlooked area in the Omani context, i.e., the reliability in scoring the writing section of the final exams in the University of Technology and Applied Sciences (UTAS). More specifically, the study investigated the estimates of inter-rater and intra-rater reliability among 10 writing markers assessing 286 and 156 students' writing scripts belonging to four levels of proficiency at three different levels of analysis: the whole writing tests, tasks 1 and tasks 2, and the constituent criteria of both tasks. The results indicated a rather high value of inter-rater reliability and a moderate one for intra-rater reliability in general. However, when interpreted regarding raters' personal and background information, some low estimates shed light on the importance of factors influencing scoring consistency across different assessors and times.</p>

ltf.2024.10.03.pdf

Original Research 

ltf.2024.10.03.xml

The inter-rater and intra-rater reliability in scoring writing tests in UTAS