Methods of Assessment

• - What are the problems of assessing communication in a L2?
  
  

- 22. What are the problems of assessing communication in a L2?

It is neither possible nor desirable to separate speaking skills from listening skills in many tests of oral production. They are interdependent skills.

The above situation increases the difficulty to analyse precisely what is being tested at any one time. Without tape-recording (to check back) the language is transient. The examiner is making subjective judgements quickly under great pressure.

Limited time. It is often impossible to administer this kind of oral test with large numbers of students.

The nature of language: interaction based - even letter-writing is affected by interaction factors in a real situation.

Unpredictability (Processing of unpredictable data in real time = vital aspect of using language); Context - of situation: physical environment, role/status, formality

Purpose (every utterance is made for a purpose); Performance (successive approximation); Authenticity (Candidate's ability to come to terms with the unknown i.e. the unsimplified authentic text.) Watered down items don't measure communicative ability.

Behaviour-based (behavioural outcomes: primacy of content over form in language. A test of communicative ability will have to be criterion referenced against the operational performance of a set of authentic language tasks.)

Performance Tests: Global Task + Enabling Skills, but what if a candidate can handle all the enabling skills individually but cannot mobilize them in a use situation e.g. driving a car. Need for performance tests.

Summary of problems & book recommendations

1. Which performance operations; level of proficiency anticipated; enabling skills involved in performing tested individually or together; specifying content; format.
2. How do we avoid hopelessly subjective measures of production and how do we set receptive tasks appropriate for different levels of language proficiency?

Note: B.J. Carroll's 1977 specification:

• Size & Complexity(of text)
• Range(of enabling skills, structures, functions handled)
• Speed (at which L can be processed)
• Flexibility (dealing with changes of topic)
• Accuracy(structures)
• Appropriacy(functions)
• Independence (from reference sources)
• Repetition/Hesitation (in processing text)]

• Testing Communicative Performance: An Interim Study (Language teaching methodology series) by B.J. Carroll. This is a theoretical book suitable for applied linguistics study or curriculum planning, but perhaps not so useful directly for lesson planning.
  
  
• Communicative Syllabus Design John Munby (Cambridge 1978) - Taxonomy of Language Skills 1-54 Pages 123-131 Examples: 1. Discriminating/Articulating sounds in isolate word forms/ connected speech 2. Discriminating/Articulating stress patterns within words & connected speech 12. Producing intonation patterns 17. Recognising the script of a language: discriminating the graphemes 19. Deducing the meaning of unfamiliar items, through stems/roots/ 24. Understanding conceptual meaning/ communicative value. Again a theoretical book - commendable in its thoroughness, but which institution or teacher will have the time to implement it? Worth looking at, however!
  
  
• Developing Reading Skills - by Francoise Grellet [ 256 pages: Cambridge 1981 ]. This is a practical book which summarises reading skills succinctly, but also links this theory to practical examples of exercise-types. I would highly recommend it to teachers whose direct interest is lesson planning and materials selection &/or production.
  
  Applied Linguistics Students who wish to review the work on Reading Strategies should also consult:
  Comprehension & Comprehension Tests Lunzer, E. et al (1979)
  Research in comprehension in reading pp499-545 - Davis, F. B. (1968)
  Reading Research Quarterly 3 "Identification of Subskills in Reading Comprehension by Maximum Likelihood Analysis" - Spearritt, D. (1972)
  Reading Research Quarterly 8 "A Model Program for Teaching Advanced Reading to Students of EFL" - Eskey, D. E.,
  Language Learning Vol 23, No 2, pp169-184, and Reading as problem solving: an investigation of strategies - Olshavsky, J.E.
  
  
• Industrial English Jupp,T & S.Hodlin (1975) - Theory & Practice of Functional Language Teaching. This is one of the very earliest attempts at offering a communicative syllabus to people already in an English-speaking workplace who need to learn basic survival English both for integration and to do their jobs more efficiently. I like this work too for its connections between theory and practice.
  
  
• ESU Framework Carroll, B. J. & R. West (Longman 1989) containing operational objectives for language skills and sub-skills and also an attempt to chart the standards set by different British Examination Boards within a single framework. Includes scales: specifications for testing all four skills and more.
  
  
• Communication in the Classroom: Applications and Methods for a Communicative Approach Edited by Keith Johnson and Keith Morrow

PROBLEM AREAS/SOLUTIONS : The test-curriculum relationship (A purposive test framework: Van Ek/Munby); the test content and procedures; levels of performance; methods of data analysis