Assessing the Quality of a Newly Designed Vocabulary Test for Vietnamese EFL Learners: A Rasch-based Analysis

Authors

  • Thi Kim Phuong Bui Hanoi University of Science and Technology, Vietnam
  • Quy Thanh Nguyen University of Education - Vietnam National University, Vietnam
  • Thai Hung Le University of Education - Vietnam National University, Vietnam

DOI:

https://doi.org/10.52296/vje.2023.270

How to Cite

Bui, T. K. P., Nguyen, Q. T., & Le, T. H. (2023). Assessing the Quality of a Newly Designed Vocabulary Test for Vietnamese EFL Learners: A Rasch-based Analysis. Vietnam Journal of Education, 7(1), 63–73. https://doi.org/10.52296/vje.2023.270

Abstract

Test development involves many required stages, including examining the psychometric characteristics of the test and each test item. This study describes the development of an English vocabulary test designed for EFL learners in Vietnam and provides an analysis using the Rasch model to evaluate the test quality and test items. The data was collected from 202 students from a Vietnamese university and analysed with the use of the Conquest computer program. The results show that the test had high reliability. Four out of 50 items were removed from the test as not fitting the Rasch model. 36 items were ranked very good, and 10 items require some revision efforts for better distractors. The paper is expected to contribute to the wide application of the Rasch model in the practices of test development and suggest the development of more reliable instruments for EFL learners to assess their vocabulary. 

Downloads

Download data is not yet available.

References

Aghekyan, R. (2020). Validation of the SIEVEA instrument using the Rasch analysis. International Journal of Educational Research, 103, 101619. https://doi.org/10.1016/j.ijer.2020.101619

Aviad-Levitzky, T., Laufer, B., & Goldstein, Z. (2019). The new computer adaptive test of size and strength (CATS): Development and validation. Language Assessment Quarterly, 16(3), 345-368.

Bui, T. K. P. (2022). A Review on Vocabulary Tests of High Frequency English Words. VNU Journal of Science: Education Research, 38(3), 51-60.

Burkett, T. (2015). An investigation into the use of frequency vocabulary lists in university intensive English programs. International Journal of Bilingual & Multilingual Teachers of English, 3(2), 71-83. https://doi.org/10.12785/IJBMTE/030202

Dang, T. N. Y. (2020). Vietnamese non-English major EFL university students’ receptive knowledge of the most frequent English words. VNU Journal of Foreign Studies, 36(3).

Duy, V. V., & Nguyen, C. N. (2019). An assessment of vocabulary knowledge of Vietnamese EFL learners. The 20th English in Southeast Asia Conference, Date: 2019/12/06-2019/12/07, Location: National Institute of Education, Nanyang Technological University, Singapore,

Ha, H. T. (2021). A Rasch-based validation of the Vietnamese version of the listening vocabulary levels test. Language Testing in Asia, 11(1), 16. https://doi.org/10.1186/s40468-021-00132-7

Lane, S., Raymond, M. R., Haladyna, T. M., & Downing, S. M. (2015). Test development process. In Handbook of test development (pp. 19-34). Routledge.

Laufer, B. (2013). Lexical thresholds for reading comprehension: What they are and how they can be used for teaching purposes. Tesol Quarterly, 47(4), 867-872. https://doi.org/10.1002/tesq.140

Laufer, B., & Goldstein, Z. (2004). Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning, 54(3), 399-436. https://doi.org/10.1111/j.0023-8333.2004.00260.x

Laufer, B., & Nation, P. (1999). A vocabulary-size test of controlled productive ability. Language Testing, 16(1), 33-51. https://doi.org/10.1177/026553229901600103

Le, T. C. N., & Nation, P. (2011). A bilingual vocabulary size test of English for Vietnamese learners. RELC Journal, 42(1), 86-99. https://doi.org/10.1177/0033688210390264

Le, T. H. & Nguyen T. H. (2021). Experimental Research and Application of Computerized Adaptive Tests to assess Learners’ Competencies. In 2021 3rd International Conference on Computer Science and Technologies in Education (CSTE) (pp. 69-74). IEEE.

Le, T. H., Tang, T. T, Tran L. A., Nguyen T. D., Nguyen P. A & Nguyen T. Q. G. (2019). Developing Computerized Adaptive Testing: An Experimental Research on Assessing the Mathematical Ability of 10th Graders. VNU Journal of Science: Education Research, 35(4).

Lin, L. H., & Morrison, B. (2010). The impact of the medium of instruction in Hong Kong secondary schools on tertiary students’ vocabulary. Journal of English for Academic Purposes, 9(4), 255-266. https://doi.org/10.1016/j.jeap.2010.09.002

McLean, S., & Kramer, B. (2015). The creation of a New Vocabulary Levels Test. Shiken, 19(2), 1-11.

McLean, S., Kramer, B., & Beglar, D. (2015). The creation and validation of a listening vocabulary levels test. Language Teaching Research, 19(6), 741-760. https://doi.org/10.1177/1362168814567889

Nation, I. S. P. (2001). How many high frequency words are there in English. Language, Learning and Literature: Studies Presented to Hakan Ringbom, 4, 167-181.

Nation, I. S. P., & Beglar, D. (2007). A vocabulary size test. The Language Teacher, 31(7), 9-13. http://www.jalt-publications.org/archive/tlt/2007/07_2007TLT.pdf

Nation, P. (1983). Testing and teaching vocabulary. Guidelines, 5, 12-25.

Nguyen, C. D. (2021). Lexical Features of Reading Passages in English-language Textbooks for Vietnamese High-school Students: Do they Foster both Content and Vocabulary Gain?. RELC Journal, 52(3), 509-522. https://doi.org/10.1177/0033688219895045

Nguyen, T. H., Vu , T. L., Le , T. H., & Pham , V. H. (2021). Designing adaptive multiple-choice questions to assess the mathematical ability of 12th grade students. Journal of Education, 508(2), 33-40.

Nguyen, T. M. H., & Webb, S. (2017). Examining second language receptive knowledge of collocation and factors that affect learning. Language Teaching Research, 21(3), 298-320. https://doi.org/10.1177/1362168816639619

Nguyen, V. C., & Nguyen P. H. (2020). Analysing and selecting multiple choice test items based on classical test theory and item response theory. Ho Chi Minh City University of Education Journal of Science, 17(10), 1804-1818.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research.

Şahin, A., & Weiss, D. J. (2015). Effects of calibration sample size and item bank size on ability estimation in computerized adaptive testing. Educational Sciences: Theory & Practice, 15(6), 1585-1595. https://doi.org/10.12738/estp.2015.6.0102

Schmitt, N., Cobb, T., Horst, M., & Schmitt, D. (2017). How much vocabulary is needed to use English? Replication of van Zeeland & Schmitt (2012), Nation (2006) and Cobb (2007). Language Teaching, 50(2), 212-226. https://doi.org/10.1017/S0261444815000075

Schmitt, N., Nation, P., & Kremmel, B. (2020). Moving the field of vocabulary assessment forward: The need for more rigorous test development and validation. Language Teaching, 53(1), 109-120. https://doi.org/10.1017/S0261444819000326

Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language Testing, 18(1), 55-88. https://doi.org/10.1177/026553220101800103

Stoeckel, T., & Bennett, P. (2015). A test of the new General Service List. Vocabulary Learning Instruction, 4(1), 1-8.

Todd, V. (2016). Learn The Oxford 3000™. English Australia Journal, 31(2), 95-97.

Vu, D. V., & Peters, E. (2021). Vocabulary in English language learning, teaching, and testing in Vietnam: A review. Education Sciences, 11(9), 563. https://doi.org/10.3390/educsci11090563

Webb, S. A., & Chang, A. C. S. (2012). Second language vocabulary growth. RELC Journal, 43(1), 113-126. https://doi.org/10.1177/0033688212439367

Webb, S., Sasao, Y., & Ballance, O. (2017). The updated Vocabulary Levels Test: Developing and validating two new forms of the VLT. ITL-International Journal of Applied Linguistics, 168(1), 33-69. https://doi.org/10.1075/itl.168.1.02web

Zareva, A., Schwanenflugel, P., & Nikolova, Y. (2005). Relationship between lexical competence and language proficiency: Variable sensitivity. Studies in Second Language Acquisition, 27(4), 567-595. https://doi.org/10.1017/S0272263105050254

Downloads

Published

2023-03-30

How to Cite

Bui, T. K. P., Nguyen, Q. T., & Le, T. H. (2023). Assessing the Quality of a Newly Designed Vocabulary Test for Vietnamese EFL Learners: A Rasch-based Analysis. Vietnam Journal of Education, 7(1), 63–73. https://doi.org/10.52296/vje.2023.270

Issue

Section

Original Articles