Xuanye Miao

Vahid Aryadoust

Yanyan Huang

National Institute of Education, Nanyang Technological University, Singapore

Abstract

Over the past few decades, second language (L2) listening assessments have become an integral component of language teaching and learning. However, there have been few systematic reviews examining the validity, reporting practices, and test specifications of existing L2 listening assessments. To gain a comprehensive understanding of L2 listening assessments, we reviewed 126 studies (comprising 160 listening assessments) published between 1997 and 2021. We focused on test specifications, test characteristics, and test validation by adopting established frameworks for test specification alongside the argument-based validation (ABV) framework. The studies were coded using a coding scheme organized into six variable categories, including administrative information, test information, textual features, participant information, validity inferences, and results. The results showed that standardized listening assessments were widely used by L2 listening researchers, and that test specifications across studies were often heterogeneous. In addition, the majority of studies provided limited information on test specifications, thereby limiting the replicability of findings across different contexts. Finally, the distribution of validity evidence across the six ABV inferences was markedly uneven: the generalization inference was the most frequently examined (56.88%), whereas no study provided evidence for the domain description or utilization inferences, and 38.75% of assessments (62 out of 160) presented no validity evidence of any kind. Implications for L2 listening assessment and test development are discussed.

Keywords

Second language listening, systematic review, test specification, validity evidence