Assessing the Quality of a Newly Designed Vocabulary Test for Vietnamese EFL Learners: A Rasch-based Analysis
DOI:
https://doi.org/10.52296/vje.2023.270-
Downloads
How to Cite
Abstract
Test development involves many required stages, including examining the psychometric characteristics of the test and each test item. This study describes the development of an English vocabulary test designed for EFL learners in Vietnam and provides an analysis using the Rasch model to evaluate the test quality and test items. The data was collected from 202 students from a Vietnamese university and analysed with the use of the Conquest computer program. The results show that the test had high reliability. Four out of 50 items were removed from the test as not fitting the Rasch model. 36 items were ranked very good, and 10 items require some revision efforts for better distractors. The paper is expected to contribute to the wide application of the Rasch model in the practices of test development and suggest the development of more reliable instruments for EFL learners to assess their vocabulary.
Downloads
References
Aghekyan, R. (2020). Validation of the SIEVEA instrument using the Rasch analysis. International Journal of Educational Research, 103, 101619. https://doi.org/10.1016/j.ijer.2020.101619
Aviad-Levitzky, T., Laufer, B., & Goldstein, Z. (2019). The new computer adaptive test of size and strength (CATS): Development and validation. Language Assessment Quarterly, 16(3), 345-368.
Bui, T. K. P. (2022). A Review on Vocabulary Tests of High Frequency English Words. VNU Journal of Science: Education Research, 38(3), 51-60.
Burkett, T. (2015). An investigation into the use of frequency vocabulary lists in university intensive English programs. International Journal of Bilingual & Multilingual Teachers of English, 3(2), 71-83. https://doi.org/10.12785/IJBMTE/030202
Dang, T. N. Y. (2020). Vietnamese non-English major EFL university students’ receptive knowledge of the most frequent English words. VNU Journal of Foreign Studies, 36(3).
Duy, V. V., & Nguyen, C. N. (2019). An assessment of vocabulary knowledge of Vietnamese EFL learners. The 20th English in Southeast Asia Conference, Date: 2019/12/06-2019/12/07, Location: National Institute of Education, Nanyang Technological University, Singapore,
Ha, H. T. (2021). A Rasch-based validation of the Vietnamese version of the listening vocabulary levels test. Language Testing in Asia, 11(1), 16. https://doi.org/10.1186/s40468-021-00132-7
Lane, S., Raymond, M. R., Haladyna, T. M., & Downing, S. M. (2015). Test development process. In Handbook of test development (pp. 19-34). Routledge.
Laufer, B. (2013). Lexical thresholds for reading comprehension: What they are and how they can be used for teaching purposes. Tesol Quarterly, 47(4), 867-872. https://doi.org/10.1002/tesq.140
Laufer, B., & Goldstein, Z. (2004). Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning, 54(3), 399-436. https://doi.org/10.1111/j.0023-8333.2004.00260.x
Laufer, B., & Nation, P. (1999). A vocabulary-size test of controlled productive ability. Language Testing, 16(1), 33-51. https://doi.org/10.1177/026553229901600103
Le, T. C. N., & Nation, P. (2011). A bilingual vocabulary size test of English for Vietnamese learners. RELC Journal, 42(1), 86-99. https://doi.org/10.1177/0033688210390264
Le, T. H. & Nguyen T. H. (2021). Experimental Research and Application of Computerized Adaptive Tests to assess Learners’ Competencies. In 2021 3rd International Conference on Computer Science and Technologies in Education (CSTE) (pp. 69-74). IEEE.
Le, T. H., Tang, T. T, Tran L. A., Nguyen T. D., Nguyen P. A & Nguyen T. Q. G. (2019). Developing Computerized Adaptive Testing: An Experimental Research on Assessing the Mathematical Ability of 10th Graders. VNU Journal of Science: Education Research, 35(4).
Lin, L. H., & Morrison, B. (2010). The impact of the medium of instruction in Hong Kong secondary schools on tertiary students’ vocabulary. Journal of English for Academic Purposes, 9(4), 255-266. https://doi.org/10.1016/j.jeap.2010.09.002
McLean, S., & Kramer, B. (2015). The creation of a New Vocabulary Levels Test. Shiken, 19(2), 1-11.
McLean, S., Kramer, B., & Beglar, D. (2015). The creation and validation of a listening vocabulary levels test. Language Teaching Research, 19(6), 741-760. https://doi.org/10.1177/1362168814567889
Nation, I. S. P. (2001). How many high frequency words are there in English. Language, Learning and Literature: Studies Presented to Hakan Ringbom, 4, 167-181.
Nation, I. S. P., & Beglar, D. (2007). A vocabulary size test. The Language Teacher, 31(7), 9-13. http://www.jalt-publications.org/archive/tlt/2007/07_2007TLT.pdf
Nation, P. (1983). Testing and teaching vocabulary. Guidelines, 5, 12-25.
Nguyen, C. D. (2021). Lexical Features of Reading Passages in English-language Textbooks for Vietnamese High-school Students: Do they Foster both Content and Vocabulary Gain?. RELC Journal, 52(3), 509-522. https://doi.org/10.1177/0033688219895045
Nguyen, T. H., Vu , T. L., Le , T. H., & Pham , V. H. (2021). Designing adaptive multiple-choice questions to assess the mathematical ability of 12th grade students. Journal of Education, 508(2), 33-40.
Nguyen, T. M. H., & Webb, S. (2017). Examining second language receptive knowledge of collocation and factors that affect learning. Language Teaching Research, 21(3), 298-320. https://doi.org/10.1177/1362168816639619
Nguyen, V. C., & Nguyen P. H. (2020). Analysing and selecting multiple choice test items based on classical test theory and item response theory. Ho Chi Minh City University of Education Journal of Science, 17(10), 1804-1818.
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research.
Şahin, A., & Weiss, D. J. (2015). Effects of calibration sample size and item bank size on ability estimation in computerized adaptive testing. Educational Sciences: Theory & Practice, 15(6), 1585-1595. https://doi.org/10.12738/estp.2015.6.0102
Schmitt, N., Cobb, T., Horst, M., & Schmitt, D. (2017). How much vocabulary is needed to use English? Replication of van Zeeland & Schmitt (2012), Nation (2006) and Cobb (2007). Language Teaching, 50(2), 212-226. https://doi.org/10.1017/S0261444815000075
Schmitt, N., Nation, P., & Kremmel, B. (2020). Moving the field of vocabulary assessment forward: The need for more rigorous test development and validation. Language Teaching, 53(1), 109-120. https://doi.org/10.1017/S0261444819000326
Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language Testing, 18(1), 55-88. https://doi.org/10.1177/026553220101800103
Stoeckel, T., & Bennett, P. (2015). A test of the new General Service List. Vocabulary Learning Instruction, 4(1), 1-8.
Todd, V. (2016). Learn The Oxford 3000™. English Australia Journal, 31(2), 95-97.
Vu, D. V., & Peters, E. (2021). Vocabulary in English language learning, teaching, and testing in Vietnam: A review. Education Sciences, 11(9), 563. https://doi.org/10.3390/educsci11090563
Webb, S. A., & Chang, A. C. S. (2012). Second language vocabulary growth. RELC Journal, 43(1), 113-126. https://doi.org/10.1177/0033688212439367
Webb, S., Sasao, Y., & Ballance, O. (2017). The updated Vocabulary Levels Test: Developing and validating two new forms of the VLT. ITL-International Journal of Applied Linguistics, 168(1), 33-69. https://doi.org/10.1075/itl.168.1.02web
Zareva, A., Schwanenflugel, P., & Nikolova, Y. (2005). Relationship between lexical competence and language proficiency: Variable sensitivity. Studies in Second Language Acquisition, 27(4), 567-595. https://doi.org/10.1017/S0272263105050254
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Vietnam Journal of Education
This work is licensed under a Creative Commons Attribution 4.0 International License.
