All Journals

Language Teaching Research Quarterly

European Journal of Studies in Management and Business

International Journal of Behavior Studies in Organizations

Studies in Educational Management

New Challenges in Accounting and Finance

Language Testing in Focus: An International Journal

Marketing and Branding Research

Individual Differences in Language Education: An International Journal 

Feedback Research in Second Language 

Management Issues in Healthcare System

Language Teacher Education Research

Artificial Intelligence Knowledge Development

<p style="text-align: justify;">This study conducts a comprehensive quantitative evaluation of OpenAI's language model, ChatGPT 4, for grading Task 2 writing of the IELTS exam. The objective is to assess the alignment between ChatGPT's grading and that of official human raters. The analysis encompassed a multifaceted approach, including a comparison of means and reliability measures such as Cohen's weighted kappa and intraclass correlation. The results revealed a high agreement in means and substantial reliability between the two grading methods on the level of the majority of texts. However, individual discrepancies and outliers were also identified, underscoring the nuanced nature of grading. While ChatGPT demonstrated efficiency and general alignment with human grading, the study concludes that it should not replace human judgment, particularly due to these observed inconsistencies. The findings contribute valuable insights into the potential and limitations of AI in educational grading and emphasize the importance of a comprehensive quantitative evaluation.</p>

ltrq.2024.43.02.pdf

Original Research 

ltrq.2024.43.02 .xml

The Intersection of AI and Language Assessment: A Study on the Reliability of ChatGPT in Grading IELTS Writing Task 2

Artificial Intelligence and ChatGPT for Language Education: A Research Agenda


Thematic Collection