Personalized PD (cont.): CALL Enhancement Criteria

Leaky, J. (2011b). Chapter 4: A model for evaluating CALL part 1: CALL enhancement criteria. In J. Leaky. Evaluating Computer-assisted Language Learning: An integrated approach to Effectiveness Research in CALL (pp. 80-114). New York: Peter Lang.


In this chapter, Leaky switched tone and gears in his exculpation of a reliable model for Computer-assisted Language Learning (CALL) evaluation projects. He began by summarizing Chapelle’s (2001) approach to CALL evaluation, before previewing ways that this model was, while seminal, deficient when compared to the demands of modern CALL efficacy research. He then launched into a 39-page exploration of other models of CALL evaluation to highlight shortcomings in Chapelle and how his own model, which he only introduced in graphic form, fills those gaps. To carry out this work, he presented eight mapping exercises that mapped Chapelle’s, his own, and others’ CALL evaluation criteria to identify areas of overlap, congruity, and incongruity.


This chapter is part one of two in the book dedicated to outlining a framework for CALL evaluation. He begins by saying that he will present his own 12-point framework but only does so in graphic form without much explanation of each point. Instead, he delves into a string of “mapping exercises” that make transparent to the reader why his model is superior to Chapelle’s (2001) model—one that was 11 years old at the time and was developed when CALL interventions weren’t as networked or as interactive as they were in 2011 or as they are today in 2017. To me, his failure to adequately describe his own framework, and the extended attack on another author just to prop up his own work, makes this chapter not only barely readable but utterly useless. The work presented in this section is the work that should have fed his thought processes and his framework development, and then been presented to us in about five pages—not a 41-page journey into his psyche where his linguistic and rhetorical choices take a massive shift from earlier chapters (moving towards the haughty “I know best” end of the spectrum). TL;DR, if you’re reading this book, skip this chapter. Or, find a better book.

Personal PD: TESOL Advanced Practitioner Certificate

So, I’m currently working on obtaining the Advanced ELT Practitioner Certificate from the TESOL International Association. This is a guided certificate program that begins with a PD program and moves through the building of a course to the observation and evaluation of one’s teaching. I’ve decided to focus my PD project for the certificate on CALL environments and blended learning. Something that I’ve always wanted to learn more about, but that my Ph.D. program wasn’t equipped to handle. I’ve decided that over the coming weeks, I’d share the output of my PD project, an annotated bibliography with critical reflection. So, here is the first entry.

Adnan, M. (2017). Perceptions of senior-year ELT students for flipped classrooms: A materials development course. Computer Assisted Language Learning, 30(3-4), 204-222. DOI:10.1080/09588221.2017.1301958


In this article, Adnan examines the use of a “flipped” classroom in an ELT Materials development course at a Turkish university. In their study, they examined how students responded to the flipped classroom, a term they use to mean one where students use web-tool to prepare for the class beforehand (e.g., reading articles or watching a lecture) instead of doing those things in class. They wanted to know how students responded to this organizational pattern for the course and whether it had any influence on academic achievement (grades). Their findings suggest that while it was an empowering approach for students, it did not have any measurable impact on students’ academic performance as represented in test, quiz, essay, and portfolio grades


So, this piece presents an interesting take on flipped learning—I hadn’t seen the term defined quite like this before. To be honest, I find the definition insufficient to the task at hand. They define it merely as using technology to have students prepare ahead of class. To me, it’s more than this. It’s about creating an environment that is student-centered and facilitates students taking increased agency over their own learning. Central to Adnan’s (2017) definition of the flipped classroom is the use of technological interventions to help the students prepare beforehand. Based on what I saw in their report of research, there is nothing here that can’t be done without That is, CALL provides no major advantages here. That being said, they do report that the use of CALL to flip the usual flow of instruction did seem to empower students by creating a sense of control over the speed of the flow of instruction. The flow control mentioned previously, however, is something worth considering. This is where a CALL intervention maybe most helpful. For example, having recorded recall lectures to help students remember the main points of a given class day could be very helpful.

Supporting L2 Writers with Online Writing Labs

In the spring of 2012, I began my first appointment in academic administration. I was appointed to be the content coordinator of the Purdue Online Writing Lab (OWL). It was a bit of a shock, as I didn’t expect to beat out some of my peers from the Rhetoric and Composition program for the position, but the senior leadership at the Purdue Writing Lab were eager to expand some of the L2 writing resources. So, between my background in management outside of the academy and my specialization in L2 writing, I was a good candidate. This was an exciting appointment for me. And, it showed me the potential of online writing resources to support second language writers.

I held that post from 2012 until I finished my Ph.D. in 2015. During my time in this post, I learned a great deal about how OWLs can support L2 writers, and I have become a vocal advocate for OWLs expanding to other national contexts. In today’s blog update, I’m going to share some of my thinking and learning on this topic. I’ll begin by discussing how OWLs can support L2 writers and I’ll wrap up by talking about why I think they should be encouraged “take flight” in other national contexts.

OWLs & Supporting L2 Writers

Since their creation in 1994, OWLs have increasingly included resources specifically targeted at L2 writers to help guide them through the cognitively and linguistically demanding task of composing in the target language. The Purdue OWL’s ESL section includes some resources about composing for U.S. university assignments, business writing in Indian and China Englishes, and a primer on the central concepts of composition in American classrooms. Excelsior College’s OWL ESL section offers a series of workshops on issues that arise throughout the writing process, and they include some animated videos to reinforce the learning.

As wonderful as these resources are, and as many hours as gets poured into their creation, they are always created with a set audience in mind. At the Purdue OWL, the Land Grant University Mission required us to consider how these resources could first be useful to the Purdue University and the Indiana State communities. This means that there will always be the need to change or scaffold these resources before using them in L2 classrooms outside of these contexts. It’s important to be aware of, however, how licensing and copyright may constrain your options in some of these regards.

One of the primary areas that need modification is in regards to Lexile level. A study that I carried out in 2013, and which will appear in Asian EFL Journal next month, shows that many teachers report that their students, especially in the EFL context, have difficulty with some of the language used in online resources—even ones specially designed for L2 writers. This means that classroom teachers will need to dedicate time in class to introducing and defining critical metalanguage about writing, terminology that is typically rather common across these resources. It also means that some of these resources may be more appropriate to helping the teacher prepare for class, and not just giving them to students.

Another way that OWLs can help L2 writers is by helping them to gain agency over their educational experiences. OWLs can be beneficial in this regards by serving as a repository of resources to which students can turn when they are in need. Also, they can allow students to personalize their learning by providing them with a bank of educational resources that cover a range of topics and writing contexts. By helping students to take more agency over their learning OWLs may also contribute to student success and retention at the University level (see Mercer’s “Understanding Learner Agency”).

Why OWLs Should Expand to New Contexts

Since taking over the Purdue OWL, I have been an advocate for OWLs expanding to new contexts. This belief began after a series of conversations with writing lab administrators in Poland and in Japan. During these conversations, I learned about the expansion of writing centers to Asia and Europe. Many of these writing centers followed the U.S. model, making use of peer tutoring and low-stakes conversations about students’ writing to create a safe place for student-writers to work on improving their writing.

OWLs are a logical extension of this growth of in-person centers. And, I would argue, that they help address the affective barriers that the act of writing and revising may raise in ways that physical writing center may not be able to. For example, many of our students have grown accustomed to computer-mediated exchanges. By moving the tutorial and instructional resources to digital spaces, it moves the exchange more closely into the students’ comfort zone. Now, while I firmly believe that we should challenge our students to step outside of their comfort zones, not everyone is ready to do this. For these students, OWL resources can be invaluable. This is one of the reasons that I would love to see more OWLs come online in national contexts across the globe. It makes writing, even in the L1, more affectively accessible. And, it may serve as a vital connection to the writing center that might not occur otherwise—a link which could eventually lead to face-to-face tutorials as the student grows more confident in their writing.

The second reason is that I believe national OWLs can serve a vital function in addressing aspects of writing—especially local L1 writing and multilingual writing—that things like the Purdue OWL and the Excelsior OWL have just been ill-equipped to address strictly from a human resources perspective. A Chinese OWL, for example, could discuss writing strategies and expectations for writing business letters in Chinese, or for using China English to communicate to international and multilingual audiences that the Purdue OWL can’t really hope to replicate. Also, a Chinese OWL could help the millions of L2 Chinese learners across the globe. Heavens knows I’d love a reliable resource to turn to that could guide me through writing a proper business email in Chinese!

Source Articles

Mercer, L. (2011). Understanding learner agency as a complex dynamic system. System, 39(4), 427-436.

Paiz, J.M. (2016). Extending the writing center: Online writing labs as sites of engagement and students learning. [plenary address]. European Writing Centers Association: Łodz, Poland.

Paiz, J.M. (2017). Uses of and attitudes towards OWLs as L2 writing support tools. Asian EFL Journal, 19(1), 56-80.

Helping Students with Word Choice: Recommendations for Online Writing Tutorials

…[F]or many second language writers, the writing process may be less effective, less recursive, and more laborious because their preoccupations with choosing the right words to fit their purpose and audience slow them down (Williams, 2005). –Severino and Prim “Word Choice Errors” p. 116

Words are important. They’re like little Lego bricks of language that we stack together in interesting ways in trying to move some message from our minds to the minds of others. This makes picking the right words for given context rather important. For example, turning to someone you fancy and telling them, “I like you,” will elicit a rather different response than “I love you,” which will get an even more radically different reaction than “I lovey wovey wove you schnookums!” Word choice is so important, and it’s interesting to listen writers talk about their struggles with. One of my Purdue professors once told me that he would often spend weeks reworking a sentence to make sure that it said exactly what he wanted it to say—agonizing over each word, its place in the sentence, and the work that the word was doing. For second language (L2) writers, the challenge of word choice can often seem insurmountable. Year after year, students in my first-year composition and in my graduate professional writing seminar list “increasing my academic/professional vocabulary” among their top concerns, higher even than improving their grammar. So, this means that helping our students develop their word choices is a critical task for L2 writing teachers and for writing center professionals.

Before we can discuss strategies for helping out students develop their vocabularies, it is helpful to understand some of the root causes of the word choice errors. A recent article by Carol Severino and Shih-Ni Prim from the writing center at the University of Iowa provides some key insights in this regard. In their article “Word Choice Errors in Chinese Students’ English Writing and How Online Writing Center Tutors Respond to Them,” Severino and Prim outline six common word choice error sources for L2 writers:

  1. Translation – making errors in moving from the L1 to the L2.
  2. Wrong Context – using a word in a context it’s not typically used in.
  3. Synform – confusing words that look or sound alike.
  4. Idiomaticity – making mistakes in set phrases, expressions, or figures of speech.
  5. Precision – choosing words that over/under-generalize
  6. Register – selecting words more common to spoken vs. written text.

What’s interesting about this list is that only translation errors are reported to be grounded in the L1. All others seem to be rooted in issues in the L2 linguistic system. When it comes to how often these types of errors occur, Severino and Prim’s sample showed that the majority (about 37%) were tied to using words in contextually inappropriate ways. An example that they give is when a student writes something like “I had to accommodate my time (p. 129).” Register errors were relatively rare, accounting for about 6% of the sample, while all others were almost tied with occurrences ranging between 18% (translation) to 12% (idiomaticity).

So, how can we help students with their word choice errors? There are some options, and below, I’ll talk about two. The first is through online writing tutorials from writing center tutors. The other is by scaffolding student agency by teaching them to use online tools target at word choice, like the free-to-use Writefull app. I’ll discuss both a little more detail below.

There are a couple of methods for online tutorials, but they fall into one of two broad categories—synchronous or asynchronous. Synchronous tutorials happen in real-time over platforms like Skype or Google Docs. While asynchronous tutorials usually involve students emailing their papers into the writing center and get feedback a few days later after a tutor has read and commented on the students work. While the synchronous tutorial does a good job of replicating a typical writing center tutorial, asynchronous tutorials have lots of value, especially for L2 writers. Because these tutorials require tutors to write extensive comments on student work, it gives the L2 writers something that they can refer to while they revise. This means that if something doesn’t make sense right away, the student can look back at the comments and discuss those observations with tutors in the lab or with their peers to make better sense of them. However, these comments must be adequately made. Severino and Prim identified five common strategies that tutors had for responding to students’ word choice errors:

  1. Correction – the tutor just corrects the error with a more appropriate choice.
  2. Questions – the tutor will ask a leading question to the writer to get them to think about their word choice.
  3. Explanation – the tutor will explain why the word choice doesn’t work.
  4. Error indication – the tutor will merely mark the error with a highlight or comment bubble.
  5. Options – the tutor will offer alternative word choice options to the student writer.

Now, each of these strategies has (de)merits. For example, just correcting the answer gives the student a more appropriate word choice; but, it also encourages students to just swap out the answers, which means that learning might not happen. Merely marking the error forces the student to think about what might be wrong with the marked word or phrases, but it might be beyond their abilities to correctly identify the kind of error or how to correct it. Asking question is fantastic because it gets the student thinking. However, other research on feedback suggests that L2 students might not understand the point of the question as marking an error. There’s really no silver bullet response to the most efficient way to handle commenting on word choice errors. Rather, a variety of these strategies should be used. Severino and Prim, recommend relying more on strategies like asking questions and offering options or explanation, and less on just making the correct—even if it may seem easier to just edit at the time. Doing the editing for the student actually takes away both a valuable learning opportunity and a student’s agency over their own writing. Severino and Prim also point out that word choice errors make up a notable portion of errors in L2 writing, so it’s important to point students towards resources that they can use to tackle making difficult word choice decisions.

One tool that writers might find useful is the new Writefull App. Writefull works from corpora of texts to allow writers to see how different words are used in context in other texts. Students can even put in a small collocation (e.g., ontogenesis from) to see check if that collocation occurs in the corpora. If not, it may mean that they need to change one or more parts of it. I’ve just started playing around with Writefull, so I can’t say much now—but, I will in a future update—but, I can see some real potential here. One thing that makes it seem so promising is that it provides examples of word choice that come from live texts. This provides crucial contextual information that actually helps with what Severino and Prim have identified as the leading source of word choice errors for Chinese L2 writers.

Source Articles

Severino, C., & Prim, S. (2015). Word choice errors in Chinese students’ English writing and how online writing center tutors respond to them. Writing Center Journal, 34(2), 115-143.

Williams, J. (2005). Teaching foreign and second language writing. Boston, MA: McGraw-Hill.

Writing and Online Feedback

Becoming literate is an important part of achieving educational success. At a surface level, this means learning the ins and outs of reading (parsing information) and writing (transmitting information). However, acquiring academic literacy goes beyond this surface level of recognizing and producing signs. It requires the skillful deployment of different knowledges in understanding others’ messages, planning one’s message, crafting that message, and revising that message. These various acts of literacy require students to deploy critical thinking, rhetorical knowledge, linguistic knowledge, and metacognitive strategies to be successful. It can be a very cognitively demanding task. Doing this in a second language makes it all the more challenging.

Writing plays a crucial role in education, particularly in higher education, where many assessments happen in writing. Science students will often write lab reports; engineering students will write instruction sets; literature students will write critical analyses; and, business students will often write case studies. All of these types of writing will often serve some assessment purposes. Therefore, being proficient in the literacy practices of higher education can mean the difference between success or failure. Anecdotally, there is evidence of students who feel that they have poor language skills picking degrees in the hard sciences thinking they will do less writing. However, the work of Dorothy Winsor shows that these students are often in for a surprise, especially when they hit the job market—they’re going to do a lot of writing.

Now, many of my students seem initially seem to think that writing is something that you sit down, you do, then you’re done. However, this is simply not the case. So, I do my best to get them to buy into the notion of “first ideas are never best ideas.” This saying is especially true when it comes to writing. And, if we accept that the first thing that we spill on to the page at 11:59 pm the night before class isn’t going to be our best work. This means that we must revise. And, in revision, we see how writing isn’t just something you do; it’s something alive and social. In the act of revision, we either evoke the other and how they might read our text; or, we actually give it over to someone else to read and to offer feedback. Thanks to more robust technologic solutions, and the spread of online writing services and pedagogical interventions, students may find that this feedback is coming from online sources.

However, this presents particular challenges. In her article, “Attitudes Towards Online Feedback on Writing” Carola Strobl examined how students interact with online feedback and how, if at all, they make use of that feedback. Strobl outlined some feedback methods, from just providing a model answer to the students submitting their online writing, which has the benefit of providing immediate feedback, to giving students marked up texts outlining their various strengths and weaknesses, which is time displaced but provides the most amount of direct feedback.

What was interesting in Strobl’s study of 38 L2 German students was that these students seemed to have a strong preference for the more immediate feedback of the model. This preference exists because they could see the feedback as soon as they had submitted their answers online. But, the students did not like how hard they had to work to make sense of the comments. That is, they didn’t feel that the model answer met their instructional needs because it was up to them to look at the model answer and then to reflect on their own answer. The reason for this was because they felt that the teacher knew more than they did. Their own linguistic skills were not sufficient enough to take that degree of control, or agency, over their own learning. While the far preferred the direct feedback from the teacher with marked up essays, the lag time often meant that the impact of the feedback was diminished, in part because they were no longer working on that type of writing by the time they go feedback from the teacher.

Most salient were that most students reported just clicking through to the end of the submission, not caring to see the feedback. If we think about how people use websites, this makes sense. When you have a long form to fill out, your mind isn’t on what you can learn, it’s on how quickly you can finish the task and get back to browsing Facebook, looking at pictures of puppies on Imgur, or catching up on your favorite TV shows on 天天美剧.

In these findings, I see much room for improvement in online feedback. But, I also see much promise. I believe that we, as teachers, must work to make ourselves obsolete. That means moving students from needing that specific feedback from us, to being able to better self-regulate. Perhaps a more efficient method here for online feedback of writing would involve training students in what to do with your comments once they have it. How to make sense of it and how to wean themselves off it. This may mean that in work early in the semester, you opt for the time displaced, explicit feedback—marking errors and offering suggestions. Then, at about the mid-point, you can change to marking errors and point students towards online resources that they can read and reflect on to better understand how to improve their own writing. Then, towards the end, you can move towards the non-time displaced method of just providing a model answer to students and having them reflect on that and their own work. What’s important in this approach is creating value around reflection and self-assessment. In my classes, as I work with undergraduate and graduate students, I tend to tie to their future professional lives. I tell them in my time as a manager, all of my employees have had to engage in critical reflection and self-assessment. They’ve also had to take over agency of their advanced training. It was my job as manager to create opportunity. Not to just give them their professional development in a pre-packaged chunk for them to swallow.

Source Articles

Strobl, C. (2015). Attitudes towards online feedback on writing: Why students mistrust the learning potential of models. ReCALL, 27(3), 340-357.

Winsor, D.A. (1996). Writing like an engineer: A rhetorical education. New York: Routledge.