Language Testing in Focus: An International Journal

The role of AI in eco-literacy education is crucial in today's technological era. Students increasingly engage with digital texts, such as fiction, enhancing their understanding of eco-literacy. This study compares the eco-literacy skills of university students and AI to evaluate their effectiveness in environmental education. Participants included 199 students from Makassar State University and Timor University in Indonesia and ten web-based AI chatbots running various large language models. Instruments included a 20-item multiple-choice test for input eco-literacy and a short story synopsis task, "Pohon Larangan di Bengayoan" (Prohibited Tree in Bengayoan) for output eco-literacy, distributed via Google Forms. The study applied independent sample t-test analysis and Yuen's test for robustness using Jamovi 2.4.8.0 software. Results showed no significant gender differences in students' eco-literacy but significant differences between universities, with Makassar State University students performing better. AI outperformed students, highlighting AI's potential in eco-literacy education and the need for curriculum enhancement in universities. Future research should involve broader samples and comprehensive assessment methods.