Yuko Hayashi Associate Professor

Research overview

Development of an Automated Scoring System for Speech and Its Application in a Classroom Environment Towards a system with higher precision through collaborative efforts
Over the past several years, my colleagues and I have been working a project on automated scoring. More specifically we’ve been trying to build and evaluate a new automated scoring system for speech produced in a foreign language by Japanese learners of English, particularly high school and university students. This is a highly interdisciplinary project interacting not just with language teaching and testing but also with natural language processing and computational linguistics.

■ Assessment of speaking ability with tablets and computers
In keeping with the increasingly vital role of English as the lingua franca of various educational and professional settings worldwide, the primary objective of English language teaching lies in preparing learners for communicating effectively in real-world situations. Speaking proficiency constitutes the essential component of such communicative success, and, hence, is often viewed as one of the most important goals of English language learners. However, assessing speaking proficiency in second or foreign language learners comes with a range of challenges including time and financial constraints, shortage of human resources. One solution to alleviating these challenges involving language assessment is the deployment of automated scoring technologies, which have developed prolifically in recent years through increased collaborations across different fields.

Two primary objectives are:
1. To develop an automatic speech evaluation system that will accurately predict the speaking proficiency of Japanese learners of English as a foreign language. The system has been developed using a tablet-adapted oral discourse completion.
2. To evaluate the prediction capability of the system in light of both machine-generated and teacher ratings and feedback with a view to better understanding its efficacy as a pedagogical tool for language teachers to use in a classroom setting.

In measuring the speaking ability of the learners, we utilized an oral discourse completion task (DCT). An oral DCT is a method used to elicit discourse data, whereby learners (test-takers) provide an oral response to pre-recorded situational prompt. It involves comprehending and producing an appropriate and timely response in a multi-turn context and, hence, should help us obtain a more detailed understanding of the learner’s communicative language ability.

■ Project prospects
Our system has been shown to be a reliable tool, in moderate-to-high agreement with human ratings (72–96 %). Such reasonably high human-machine agreement paves the way towards exploring the use of our system in foreign language classroom settings. To that end, we intend to carry out a detailed examination of classroom teachers’ views and attitude towards the use of automated scoring. Building an automated scoring system has become easier and more accessible than before, owing to increasing research collaborations. We hope that our work on automated speech scoring and other ongoing projects in the relevant field will continue to better inform language teachers (as well as researchers) of its pedagogical efficacy.