An Overview of Studies Conducted on Washback, Impact and Validity

Forough Rahimi, Mohammad Reza Esfandiari, Mansour Amini, Mansour Amini

Abstract


This article aimed at presenting a comprehensive overview of three interrelated concepts of washback, impact and validity in language testing and a myriad of studies conducted at different places to investigate the influence of testing on teachers and teaching, textbooks, learners and learning, attitudes toward testing, test preparation behaviors, etc.. Some of these studies present the results of various investigations on the influence of a national English examination on the local English language teaching and learning due to its high-stakes nature in particular countries such as Brazil, China, Hong Kong, Iran, Israel, Japan, Romania, Sri Lanka, and Taiwan. Some others cover a wide range of worldwide investigation on English testing such as the IELTS, TOEFL, and MECC. Moreover, there is a complete report of several important projects appointed by major testing agencies such as Cambridge ESOL and Educational Testing Services (ETS) on washback and impact studies. The article proceeds by reviewing the relevant literature on test validation which is a key concept in language testing domain since it is concerned with test interpretation and use. This domain is characterized and enriched by studies of washback and impact.


Keywords


Washback; Impact; Validity

Full Text:

PDF

References


Alderson, J. C. (1986) Innovations in language testing. In Portal, M. (Eds.), Innovations in language testing. NFER Nelson. 93-105.

Alderson, J.C. & Hamp-Lyons, L. (1996).TOEFL preparation courses: A case study. Language Testing, 13, 280–297.

Alderson, J.C. & Wall, D. (1993). Does washback exist? Applied Linguistics, 14, 115–129.

American Educational Research Association, American Psychological Association &National Council on Measurement in Education (1999).Standards for educational and psychological testing. Washington, DC: Author.

Anastasi, A. (1986). Evolving concepts of test validation. Annual Review of Psychology, 37, 1-15.

Andrews, S. (1995). Washback or washout? The relationship between examination reform and curriculum innovation. In Nunan, D., Berry, V. & Berry, R. (Eds.), Bringing About Change in Language Education, University of Hong Kong, Hong Kong, 67–81.

Andrews, S., Fullilove, J. & Wong, Y. (2002).Targeting washback: A case study.System30, 207–223.

Angoff, W. (1988). Validity: an evolving concept. In H. Wainer & H. I. Braun (Eds.), Test Validity (pp. 19 - 32). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

Bachman, L.F. (1990).Fundamental considerations in language testing. Oxford University Press, Oxford.

Bachman, L.F. (2000). Modern language testing at the turn of the century: Assuring that what we count counts. Language Testing 17(1), 1–42.

Bachman, L.F. (2004).Statistical analyses for language assessment. Cambridge University Press, Cambridge.

Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly,2, 1–34.

Bachman, L.F. & Cohen, A.D. (eds.). (1998). Interfaces between second language acquisition and language testing research. Cambridge University Press, NY.

Bachman, L.F. & Eignor, D.R. (1997).Recent advances in quantitative test analysis. In Clapham, C.& Corson, D. (Eds.),Encyclopedia of Language and Education, Volume 7: Language testing and assessment, Kluwer Academic, Dordrecht, The Netherlands, 227–242.

Bachman, L.F. & Palmer, A. (1996).Language testing in practice: Designing and developing useful language tests, Oxford University Press, Oxford.

Bailey, K.M. (1996). Working for washback: A review of the washback concept in language testing. Language Testing 13, 257–279.

Banerjee, J. &Luoma, S. (1997). Qualitative approaches to test validation, in Clapham, C.& Corson, D. (Eds.),Encyclopedia of Language and Education, Volume 7: Language testing and assessment, Kluwer Academic, Dordrecht, The Netherlands, 275–287.

Borrows, C. (2004).Washback in classroom-based assessment: A study of the washback effect in the Australian adult migrant English program. In Cheng, L., Watanabe, Y. & Curtis, A. (Eds.), Washback in language testing: Research contexts and methods. Lawrence Erlbaum Associates, Mahwah, NJ.113–128.

Chapelle, C. A., Enright, M. K. & Jamieson, J. M. (Eds.).(2008). Building a validity argument for the Test of English as a Foreign Language™. Mahwah, NJ: Lawrence Erlbaum.

Chen, L. (2002). Washback of a Public Exam on English Teaching. Unpublished PhD dissertation, the Ohio State University.

Chen, Z. & He, Y. (2003). Influence of CET-4 on college students and some suggestions. Journal of Technology College Education, 22, 40–41.

Cheng, L. (2005). Changing language teaching through language testing: A Washback study. Studies in Language Testing: Volume 21, Cambridge University Press, Cambridge.

Cheng, L., Watanabe & Curtis, A. (2004).Washback or backwash: A review of the impact of testing on teaching and learning. In Cheng, L., Watanabe, Y. & Curtis, A. (Eds.), Washback in language testing: Research contexts and methods, Lawrence Erlbaum Associates, Mahwah, NJ, 3– 18.

Cheng, L. (2008). Washback, impact and consequences. In Shohamy, E. & Hornberger, N. H.(Eds.),Encyclopedia of language and education. Volume 7: Language testing and assessment,2ndedn. (pp. 349–364). New York: Springer Science and Business Media LLC.

Clark, J. (1975). Theoretical and technical considerations in oral proficiency testing. In Jones, S. & Spolsky, B. (Eds.), Language Testing Proficiency, Center for Applied Linguistics. Arlington, VA, 10–24.

Cronbach, L. J. (1988). Five perspectives on the validity argument. In Wainer, H. & Braun, H. I.(Eds.), Test validity (pp. 3–17). Hillsdale, NJ: Lawrence Erlbaum.

Cronbach, L. J. & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.

Cumming, A. & Berwick, R. (Eds.).(1996).Validation in Language Testing. Multilingual Matters, Clevedon, Avon.

Cureton, E. E. (1951). Validity. In Lindquist, E. F., (Ed.), Educational measurement, 1st edn (pp.621–694). Washington, DC: American Council on Education.

Elder, C. (1997). What does test bias have to do with fairness? Language Testing, 14, 261 –277.

Embretson, S. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93(1), 179-197.

Ferman, I. (2004).The washback of an EFL national oral matriculation test to teaching and learning. In Cheng, L., Watanabe, Y. & Curtis, A. (Eds.),Washback in language testing: Research contexts and methods, Lawrence Erlbaum Associates, Mahwah, NJ, 191–210.

Gosa, G. ( 2004). Investigating washback: A case study using student diaries. Unpublished PhD dissertation, Lancaster University, UK.

Goodwin, L. D., & Leech, N. L. (2003). The meaning of validity in the new standards for educational and psychological testing: implications for measurement courses.(Assessment In Action). Measurement and Evaluation in Counseling and Development, 36(3), 181 - 191.

Green, A. (1997). Verbal protocol analysis in language testing research. Cambridge University Press, Cambridge.

Green, A. (2003). Test impact and English for academic purposes: A comparative Sstudy in BbackwashBbetween IELTS preparation and university pre-sessional courses. Unpublished PhD thesis, Centre for Research in Testing, Evaluation and Curriculum in ELT, University of Surrey, Roehampton.

Guion, R. M. (1977). Content Validity--The Source of My Discontent. Applied Psychological Measurement, 1(1), 1-10.

Hamp-Lyons, L. (1997). Washback, impact and validity: Ethical concerns. Language Testing, 14(3), 295–303.

Hamp-Lyons, L. & Brown, A. (2007). The effect of changes in the new TOEFL format on the teaching and learning of EFL/ESL: Stage 2 (2003–5): Entering Innovation, Submitted to the TOEFL Research Committee, Educational Testing Service.

Hayes, B. & Read, J. (2004).IELTS test preparation in New Zealand: Preparing students for the IELTS academic module. In Cheng, L., Watanabe, Y. &Curtis, A. (Eds.), Washback in language testing: Research contexts and methods, Lawrence Erlbaum Associates, Mahwah, NJ, 97–112.

Irvine-Niakaris, C. (1997). Current proficiency testing: A reflection of teaching. Forum 35, 16–21.

Iwashita, N., McNamara, T., & Elder, C. (2001).Can we predict task difficulty in an oral proficiency test? Exploring the potential of an information processing approach to task design. Language Learning, 51(3), 401–436.

Jin, Y. (2000). Washback of college English test-spoken English test on teaching. Foreign Language World, 80, 56–61.

Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112, 527–535.

Kane, M. T. (2006). Validation.In Brennan, R. L. (Ed.), Educational measurement, 4th edn.

(pp.18–64). Washington, DC: American Council on Education/ Praeger.

Kane, M., Crooks, T. & Cohen, A. (1999).Validating measures of performance. Educational Measurement: Issues and Practice, 18, 5–17. 527–535.

Kunnan, A.J. (1998a).Validation in language assessment. Selected papers from the 17thLanguage Testing Research Colloquium, Long Beach, Lawrence Erlbaum, Mahwah, NJ.

Kunnan, A.J. (Ed.). (1998b). Special Issue: Structural equation modeling. LanguageTesting,15(3).

Kunnan, A.J. (1999). Recent developments in language testing. Annual Review of Applied Linguistics, 19, 235–253.

Kunnan, A. J. (2000). Fairness and justice for all. In A. J. Kunnan (Ed.), Fairness and validation in language assessment (pp. 1–14). Cambridge, UK: Cambridge University Press.

Kunnan, A. J. (2004). Test fairness. In Milanovic, M. & Weir C., (Eds.), European language testingin a global context: Proceedings of the ALTE Barcelona Conference (pp. 27–48). Cambridge, UK: Cambridge University Press.

Lado, R. (1961). Language testing. McGraw-Hill, NY.

Lazaraton, A. (2002). A qualitative approach to the validation of oral tests. Cambridge University Press, Cambridge.

Lederman, L. M., & Burnstein, R. A. (2006). Alternative approaches to high-stakes testing: Mr. Lederman and Mr. Burnstein propose a novel way to increase student engagement and counter the pressures of high-stakes testing. Phi Delta Kappan, 87(6), 429.

McNamara, T.F. (2006). Validity in language testing: The challenge of Sam Messick’s legacy. Language Assessment Quarterly, 3(1), 31–51.

G. J. (1989). Item bias and item response theory. International Journal of Educational Research, 13(2), 127-143.

Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35(11), 1012-1027.

Messick, S. (1988). The once and future issues of validity. Assessing the meaning and consequences of measurement. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 33 - 45). Hillsdale, N.J. :: Lawrence Erlbaum Associates.

Messick, S. (1989). Validity. In Linn, R. L. (Ed.), Educational measurement, 3rd edn. (pp. 13–103). New York: American Council on Education and Macmillan.

Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241–256.

Nazari, A. (2005). Washback effects on TEFL: A case study from Iran, IATEFL Voices 185, 9– 10.

Nemati, M. (2003).The positive washback effect of introducing essay writing tests in EFL environment, Indian Journal of Applied Linguistics, 29, 49–62.

Nguyen, P. (1997). Washback effects of international English language testing system at the Vietnam national university. Unpublished PhD thesis, University of Melbourne.

No Child Left Behind Act (Reauthorization of Elementary and Secondary Education Act) Public Law 107-110 § Section 1202(c)(7)(A)(IV)(2) (2002).

Oller, J.W. (1983).Issues in Language Testing Research. Newbury House, Rowley, Mass.

Pearson, I. (1988) Tests as levers for change. In Chamberlain, D. and Baumgardner, R. (eds.). ESP in the classroom: practice and evaluation. ELT Document 128. Modern English Publications, 98-107.

Pellegrino, J. W. (1988). Mental models and mental tests. In H. Wainer & H. I. Braun (Eds.), Test Validity (pp. 49 - 60). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

Popham, W.J. (1987).The merits of measurement-driven instruction. Phi Delta Kappa, 68, 679– 682.

Qi, L. (2004). Has a high-stakes test produced the intended changes? In Cheng, L., Watanabe, Y. & Curtis, A. (Eds.), Washback in language testing: Research contexts and methods, Lawrence Erlbaum Associates, Mahwah, NJ, 171– 190.

Qi, L. (2005). Stakeholders’ conflicting aims undermine the washback function of a high-stakes Test. Language Testing, 22, 142– 173.

Read, J. & Hayes, B. (2003). The impact of IELTS on preparation for academic study in New Zealand. In Tulloh, R. (Ed.), International English language testing system research reports 2003, Volume 4, IELTS Australia, Canberra.

Robb, T.N. & Ercanbrack, J. (1999).A study of the effect of direct test preparation on the TOEIC scores of Japanese university students. TESL-EJ 3, A2, http://tesl-ej.org/ej12/toc.html.

Saif, S. (2006).Aiming for positive washback: A case study of international teaching assistants. Language Testing, 23, 1–34.

Saville, N. & Hawkey, R. (2004). The IELTS impact study: Investigating washback on teaching materials. In Cheng, L., Watanabe, Y. & Curtis, A. (Eds.),Washback in language testing: Research contexts and methods, Lawrence Erlbaum Associates, Mahwah, NJ, 73 96.

Scaramucci, M.V.R. (2002). Entrance examinations and TEFL in Brazil: A casestudy. Revista Brasileira de Lingüística Aplicada, 2, 61–81, Universidade Federalde Minas Gerais, Belo Horizonte, MG, Brazil.

Shepard, L.A. (1990). Inflated test score gains: Is the problem old norms or teaching the test? Educational Measurement, 9, Issues and Practice, 15–22.

Shih, C. M. (2006).Perceptions of the general English proficiency test and its washback: A case study at the two Taiwan technological institutes. Unpublished PhD dissertation, University of Toronto.

Shohamy, E. (2001). The power of tests: A critical perspective of the uses of language tests. London: Longman.

Shohamy, E., Donitsa-Schmidt, S., & Ferman, I. (1996).Test impact revisited: Washback effect over time. Language Testing, 13, 298–317.

Stobart, G. (2003). The impact of assessment: Intended and unintended consequences. Assessment in Education, 16, 139–140.

Stoneman, B. (2005). An impact study of an exit English test for university graduates in Hong Kong: Investigating whether the status of a test affects students’ test preparation activities. Unpublished PhD thesis, Hong Kong Polytechnic University.

Supon, V. (2008). High-stakes testing: strategies by teachers and principals for student

success. Journal of Instructional Psychology, 35(3), 306-308.

Swain, M. (1985). Communicative competence: Some roles of comprehensive input and comprehensible output in its development. (Eds.). Input in second language acquisition. Cambridge, MA: Newbury House Publishers.

Toulmin, S.E. (2003). The use of argument (updated edition). Cambridge University Press, Cambridge.

Wall, D. (1997).Impact and washback in language testing. In Clapham, C. &Corson, D. (Eds.), Encyclopedia of Language and Education, 291–302.

Wall, D. (2005). The impact of high-stakes examinations on classroom teaching: A case study using insights from testing and innovation theory. Studies in Language Testing, Volume 22, Cambridge University Press, Cambridge.

Wall, D. & Alderson, J.C. (1993,).Examining washback: The Sri Lankan impact study.Language Testing, 10, 41–69.

Wall, D. &Horak, T. (2006). The TOEFL impact study: Phase 1.TOEFL Monograph, 34, Educational Testing Service.

Watanabe, Y. (1996). Does grammar translation come from the entrance examination? Preliminary findings from classroom-based research. Language Testing, 13, 318–333.

Watanabe, Y. (2001). Does the university entrance examination motivate learners? A case study of learner interviews. In Akita Association of English Studies (Ed.), Trans-equator exchanges: A collection of academic papers in honour of professor David Ingram, Author, Adita, Japan, 100–110.

Xi, X. (2005b). Do visual chunks and planning impact performance on the graph description task in the SPEAK Exam? Language Testing, 22(4), 463–508.

Yu, G.K.H. & Tung, R.H.C. (2005). The washback effects of JCEEEs in the past fifty years, Proceedings of 22nd Conference on English Teaching and Learning, 379–403, Normal University, Taipei, Taiwan.

Zhao, L. (2003). College English teaching evaluation system in China: Major problems and corresponding counter measures. Indian Journal of Applied Linguistics, 29, 85–98.




DOI: http://dx.doi.org/10.3968/8877

Refbacks

  • There are currently no refbacks.


Copyright (c) 2016 Mohammad Reza Esfandiari, Forough Rahimi

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Share us to:   

 

Online Submission: http://cscanada.org/index.php/sll/submission/wizard


Reminder

How to do online submission to another Journal?

If you have already registered in Journal A, then how can you submit another article to Journal B? It takes two steps to make it happen:

1. Register yourself in Journal B as an Author

Find the journal you want to submit to in CATEGORIES, click on “VIEW JOURNAL”, “Online Submissions”, “GO TO LOGIN” and “Edit My Profile”. Check “Author” on the “Edit Profile” page, then “Save”.

2. Submission

Go to “User Home”, and click on “Author” under the name of Journal B. You may start a New Submission by clicking on “CLICK HERE”.


We only use three mailboxes as follows to deal with issues about paper acceptance, payment and submission of electronic versions of our journals to databases: caooc@hotmail.com; sll@cscanada.net; sll@cscanada.org

 Articles published in Studies in Literature and Language are licensed under Creative Commons Attribution 4.0 (CC-BY).

 STUDIES IN LITERATURE AND LANGUAGE Editorial Office

Address: 1055 Rue Lucien-L'Allier, Unit #772, Montreal, QC H3G 3C4, Canada.
Telephone: 1-514-558 6138 
Website: Http://www.cscanada.net; Http://www.cscanada.org 
E-mail: office@cscanada.net; office@cscanada.org; caooc@hotmail.com

Copyright © 2010 Canadian Academy of Oriental and Occidental Culture