Language Institute 2007-08 Lecture Series: Assessing Language Learning
A Priori Assessment of Complex Language Tasks by Practitioners
Gerard Westhoff
Utrecht University, The Netherlands
3:30 pm, Tuesday, February 12, 2008
254 Van Hise Hall, 1220 Linden Drive
Comments from Thomas A. DuBois, Department of Scandinavian Studies
Download paper (PDF)
Learn more about the WebQuest Model
Abstract:
Several recent developments have stimulated and justified the increased interest in the competence of teachers to estimate the learning impact of instructional materials. Firstly, new technologies such as the Internet encourage teachers to search for a fruitful integration of these new technologies with more traditional printed instructional materials. Secondly, current educational trends encourage more independent and autonomous learning, so teachers need to develop their own (self-access) material for this purpose. Furthermore, there is a growing demand on teachers to differentiate. Published materials are generally less suitable for this purpose. In such circumstances, teachers will often design supplementary course material. Finally, we see a trend towards competence-based learning. In Europe this development has received an impetus by the ‘can-do statements’ of the Common European Framework of Reference (CEFR) (Council of Europe 2001). However, the CEFR is not easily compatible with traditional grammar-oriented pedagogy. Those practitioners who want to shape their work according to principles underlying the CEFR will have to look for alternatives to the usual form- focussed textbook tasks.
These four developments demand from teachers an extended competence in evaluating learning materials. This is especially true of tasks that are lifelike and content-oriented and, as a consequence, not single-issue exercises but sui generis complex and integrating many aspects of language acquisition. However, the critical analysis of such materials has often been overlooked or even neglected in teacher education programmes. In order to fill this gap a scaffolding instrument for a priori assessment of the learning impact of language tasks seems to be useful. In this talk I will describe some characteristics of such an instrument and the construction of an evaluating tool.
The direct, concrete incentive for this project was the fact that FL-teachers looking for learning tasks that were realistic and embedded in a rich, lifelike context in which they were functional, discovered WebQuests. WebQuests, however, are relevant for all subjects in the current school curricula. As a consequence, they mostly lack indications about those characteristics of tasks which specifically enhance language acquisition. Thus, the use of some WebQuests contributed substantially to FL-acquisition, whereas others did not yield much progress. Our aim was to help these teachers to choose and/or adapt or possibly design WebQuests that could be expected to better? foster FL-acquisition. During the development of the instrument we noticed that we actually defined criteria that could be applied to many types of integrated, lifelike and functional tasks.
Theoretical framework
We started from current insights from cognitive psychology about how information is processed in working memory, and stored in long-term memory (LTM) in the form of traces of mental actions in working memory. This means that learning is basically activity-based. Accordingly, tasks can be analyzed with respect to the type, variety and intensiveness of the mental activity they elicit. In reviewing the literature on cognitive psychology we learned that tasks which are lifelike, content-oriented and functional will not only be motivating, but can also be expected to be more effective in terms of learning outcomes.
Secondly, we reviewed current SLA-theory and distinguished five domains in which learning actions could be expected to contribute to SLA. In spite of fierce debates on various issues, a certain consensus seems to exist regarding five basic assumptions on second language:
1. Exposure to input. Without extended exposure to a rich input, there is little SLA
2. Content-oriented processing. Mere exposure is not enough. There seems to be little doubt that being exposed to input is only effective if the input is processed (or in more practical terms, if the learner has tried to understand its meaning).
3. Form-oriented processing. There is far less agreement on the role of grammar, or so-called ‘form-focused instruction’. Yet, support seems to be growing for the weak interface hypothesis (Ellis 1990). The weak interface hypothesis claims that part of the learner output is rule-directed, but that we do not consciously know the rules. Learners form hypotheses about form aspects of the language by processing input. This process of hypothesis formation is supposed to be stimulated by directing the learner’s attention to form aspects of the input.
4. (Pushed) output. There has been support for the facilitating and stimulating role of output production. It is assumed to enhance fluency, and it makes language learners conscious of their deficits, and so increases their motivation to learn. It contributes to form-orientation and gives the teacher or the communication partner the opportunity to provide corrective feedback.
5. Acting strategically. Generally speaking, only limited time is available for foreign-language acquisition. This means that there will always remain gaps in our knowledge. For that reason, it is useful and practical to develop a repertoire of strategies to compensate for these deficiencies.
The construction of a yardstick
Additional to the existing generic criteria for WebQuests as developed by its originator Bernie Dodge, we constructed two sets of criteria. One based on the cognitive psychological principles underlying TBA and one covering the five action domains that were inferred from SLA-theory.
The first version was constructed in the Netherlands in co-operation with a group of practitioners whose main purpose was the construction of new powerful WebQuests for FL-acquisition. The yardstick was a by-product of the construction process. The product was not very user-friendly and its use required more knowledge of SLA theory and terminology than practitioners usually possess.
In a second stage we used a sort of Delphi-like procedure. First we revised the tool quite substantially on the basis of earlier experiences. This new tool was tried out by a group of teachers in a workshop setting. With the help of their (not very favourable) comments and suggestions for improvement, the wording of the criteria was adapted again substantially and the tool was given the form of an elaborated rubric.
This third version was tried out again by a new group of teachers for fine tuning. The final version was translated from Dutch into both English and German. The English version was tried out in a ECML Workshop in Graz (Austria) with satisfactory results. The German version was used in a Seminar for undergraduate students GaFL at the university of Vienna. These students used the yardstick both as an evaluation tool for existing WebQuests and as a structuring tool in the process of constructing new complex tasks. In both functions it was used by the students independently without much help from the teacher. As an evaluation tool the results showed a reasonable correspondence in the judgements. As to the structuring function the students were positive in their evaluation.
The final version was completed with a grading system (one, two or three ‘stars’). Subsequently, the yardstick was put on to the Internet in an inactive format. Once the questions are completed, the end score per category and the total score, including the grading in terms of ‘stars’, are calculated automatically. Comments and suggestions for improvement of the reviewed task can be added by the reviewer
All lectures in the series are free and open to the public. Co-sponsored by the Language Institute and the Center for European Studies.
Language Institute funding is through the College of Letters and Science Anonymous Fund.
|
