Personal PD (cont.): Qualitative and Quantitative Methods for CALL Research

Leaky, J. (2011c). Chapter 5: A model for evaluating CALL part 2: Quantitative and qualitative measures. In J. Leaky. Evaluating Computer-assisted Language Learning: An integrated approach to Effectiveness Research in CALL (pp. 115-132). New York: Peter Lang.


In this chapter, Leaky (2011c) continued outlining his evaluation paradigm by examining how qualitative (judgmental) and quantitative (evaluative) approaches can be used to evaluate the efficacy of computer-assisted language learning (CALL) interventions. He provided nine principles to guide CALL research, which included controlling for confounding variables, doing a literature review, using random sampling, and providing a transparent methodology section (among others). He then provided a research design checklist to guide instrument design. This multipoint checklist covers matters of sampling (N size); integration of program, pedagogy, and platform; the conditions of the study (e.g., activity type, variables, etc.); and the quantitative instruments to be used. Qualitative methods and measures are absent from this methodological check list.


In this chapter, Leaky continues his combative approach to building his own framework. This attack on the work that has come before not only sets a disappointing tone, it also introduces weakness into his arguments. He continues to conflate qualitative research with mere subjective storytelling by holding qualitative research to the same validity/reliability measures as quantitative research and presenting a 2D view of qualitative research. His tone throughout the chapter continues to alienate the reader, at least this reader. And, again the most useful elements of the section lie in poorly supported graphical elements. Namely, his checklist for designing CALL efficacy studies. So, if you want to skip the entitled dross, move to pages 125-126 and update the list to be actually mixed-method by considering how robust qualitative research can enrich the quantitative data—something Leaky (2011c) continues to fail to do.

Personalized PD (cont.): CALL Enhancement Criteria

Leaky, J. (2011b). Chapter 4: A model for evaluating CALL part 1: CALL enhancement criteria. In J. Leaky. Evaluating Computer-assisted Language Learning: An integrated approach to Effectiveness Research in CALL (pp. 80-114). New York: Peter Lang.


In this chapter, Leaky switched tone and gears in his exculpation of a reliable model for Computer-assisted Language Learning (CALL) evaluation projects. He began by summarizing Chapelle’s (2001) approach to CALL evaluation, before previewing ways that this model was, while seminal, deficient when compared to the demands of modern CALL efficacy research. He then launched into a 39-page exploration of other models of CALL evaluation to highlight shortcomings in Chapelle and how his own model, which he only introduced in graphic form, fills those gaps. To carry out this work, he presented eight mapping exercises that mapped Chapelle’s, his own, and others’ CALL evaluation criteria to identify areas of overlap, congruity, and incongruity.


This chapter is part one of two in the book dedicated to outlining a framework for CALL evaluation. He begins by saying that he will present his own 12-point framework but only does so in graphic form without much explanation of each point. Instead, he delves into a string of “mapping exercises” that make transparent to the reader why his model is superior to Chapelle’s (2001) model—one that was 11 years old at the time and was developed when CALL interventions weren’t as networked or as interactive as they were in 2011 or as they are today in 2017. To me, his failure to adequately describe his own framework, and the extended attack on another author just to prop up his own work, makes this chapter not only barely readable but utterly useless. The work presented in this section is the work that should have fed his thought processes and his framework development, and then been presented to us in about five pages—not a 41-page journey into his psyche where his linguistic and rhetorical choices take a massive shift from earlier chapters (moving towards the haughty “I know best” end of the spectrum). TL;DR, if you’re reading this book, skip this chapter. Or, find a better book.

A Migrant Academic: Year 3

Well, we’re nearing the end of summer, which I spent in D.C. with my spouse and my dog. And, now it’s time to migrate back to China for another semester at NYU Shanghai—the penultimate semester in my first 3-year contract with the school. I’m not going to go in-depth into the ups and downs of being a migrant academic, as I’ve already done that (see part 1, part 2, and part 3). What I will say is that I will miss my boys (spouse and dog), but I will be happy to get back to the office and back to teaching. Work is very fulfilling to me, even when I complain about the small stuff (like adequate recognition of NTT faculty labor). I love working with students and learning with them. I love contributing to a new campus and hoping to make a difference. Most of all, I appreciate the distraction from the insanity that is consuming the world—not that now is the time to be distracted.

So, what PD goals am I going to set on this, my (potentially) final year at NYU Shanghai? Well, I’m going to compile and submit my reappointment docket. I’m going to complete the TESOL Advanced Practitioner Certificate and the Google Educator Levels 1 & 2 Certificates. I want to get more of my under review submissions accepted, and I want to get more of my nascent ideas off the ground and on the page. I want to finish reading Why Buddhism is True by Robert Wright. And, I want to make sure that I make it back to the U.S. enough to spend quality time with my family. I enjoy what I do. The travel and time away can wear me a little thin at times, but I do enjoy it so very much.

Please note: There will be no updates next week, as I will be traveling back to China and adjusting back to work-life there. I’ll start updates again on 28 August 2017 by diving back into my chronicle of obtaining the TESOL Advanced Practitioner Certificate.


Personal PD (cont.): Does CALL Make a Difference?

Leaky, J. (2011a). Chapter 3: Has CALL made a difference. In J. Leaky. Evaluating Computer-assisted Language Learning: An integrated approach to Effectiveness Research in CALL (pp. 59-72). New York: Peter Lang.


In this chapter, Leaky (2011) examined the various ways that CALL researchers have attempted to explore the effectiveness of CALL interventions. He carried out this work by discussing the four major debates of CALL efficacy research; viz., whether or not CALL leads to learning gains, whether or not CALL is comparable to non-CALL pedagogies, what combination of pedagogical interventions are best for learners, and what measurables should be focused on in efficacy research. His discussion showed the steady evolution of CALL research towards a greater focus on theory-based, data-driven analyses of CALL and its impacts on language teaching and learning. He concluded by offering suggestions for future CALL efficacy research, which were centered on four areas: awareness of resource development and selection criteria, clarity around the teaching context and its influences on language learning, focus on the pedagogical interventions used, and an investigation of how CALL is integrated into the language learning curriculum.


As I learn more about CALL, I’m actively seeking pieces that address one of my greatest concerns. Specifically, most CALL professionals that I have interacted with have been so excited by their work—and, it’s refreshing to see an academic passionate about their field—but, they seem to favor technology over pedagogy. I’ve seen too many presentations on how corpora tool X will revolutionize my students’ ability to acquire academic vocabulary, or how grammar checker Y will lead to professional quality writing by my novice students. In this piece, Leaky states unequivocally that CALL only works when it is a logical extension of pedagogy. That is pedagogy, and the needs of the students must come first. Then CALL should only be implemented as a support for pedagogical choices and to scaffold student learning. This is, to me, critical in thinking about CALL’s place in ELT. Also, Leaky (2011) provides a good tool-for-thought when it comes to evaluating CALL interventions through the paradigm presented in his conclusion (see summary, above)

Personal PD (cont.): The Case for CALL in a Re-mix/Mash-up World

Kessler, G. (2013). Teaching ESL/EFL in a world of social media, mash-ups, and hyper-collaboration. TESOL Journal, 4(4), 615-632.


In this invited article, Kessler provides a sustained argument for teachers to set aside their misgivings about the technological revolution in English language teaching. He adroitly points out that our students live in technological spaces that mediate not only their understanding of their ecosocial spaces but also their views towards learning and their language practices. As this is the case, teachers must come to embrace technology mediated language teaching and seek to integrate it into their practice. To help with this, Kessler provides links to many applications that teachers might find useful. He extends this data dump by providing a rationale for each item he included in his listing, weighing both pros and cons. He also provides a four-point list of actions teachers need to take to better integrate technological interventions with their practices.


For me, this was a very energetic piece by a CALL apologist that I’ve known professionally for some time. That being said, there were many useful aspects of this article. The first is the resources that he directs readers towards. To help the reader jump start their own cognition about CALL’s place in their teaching practice, Kessler provides not only a summary of the resource but suggestions for its use in language teaching. This makes Kessler’s piece infinitely more useful than other pieces that tend more towards being a wall of links. Also useful were his guidelines for approaching technology in the classroom. He encouraged educators to do four things: (1) to focus on pedagogy first, (2) to allow classroom practice to mirror real-world use, (3) to no obsess with becoming an expert of the technology, and (4) to ease into using new tech in teaching.

His second and third points are of particular importance. Allowing classroom practice to mirror authentic use allows students to more immediately see the application, the relevance, of what we’re teaching them. It can also be used to highlight how things like register and medium can influence how our interlocutors parse a message. The third part, not being expert, is instrumental. It’s a well-known maxim that our students will almost always know more about the latest tech than we do. Don’t let that hold you back. Rather, let it push you forward. By allowing students to showcase what they know and can do with new technologies in the classroom, you may be helping to actualize student agency over their own learning, a feat that can lead to increased motivation to carry out educational tasks and to buy into lifelong learning, which is what all effective language learners/users seem to understand—learning a language is a lifelong affair. There is no moment of arrival and completion.