Methods of assessment
- 17. What are the main requirements of an efficient test?
- 18. The problem of test rubric: How far do format and instructions affect assessment?
- 19. What is test discrimination? When is it not needed?
17. What are the main requirements of an efficient test?
What are the main requirements of an efficient test?
- Reliability and validity (see questions 17–20).
- Comparison and discrimination (see question 24).
- Practicality.
Practicality concerns the practical requirements involved in administering and marking a test. Questions to consider include:
- How long will the test take?
- What will the rest of the class do while the test is being administered?
- What equipment is required?
- How long will it take to mark?
- How many people will be involved in administering and marking the test?
- What will the test cost?
- Can the test materials be reproduced economically in sufficient quantities?
- How will the materials be stored and filed efficiently?
18. The problem of test rubric
The problem of test rubric: How far do format and instructions affect assessment?
The instructions given to students on how to complete a test are an important aspect of validity. The wording of the instructions should be chosen very carefully, particularly at lower levels of language proficiency.
Students' familiarity with the format of the test is also important. If appropriate, the use of the students' first language (L1) in the instructions may improve clarity, particularly where all candidates share the same first language.
In a wider sense, rubric refers to all the arrangements made by the test setter to enable students to give their attention to the content of the test. These may include the layout of the test, the wording and sequencing of instructions, and, in listening comprehension, the length and timing of pauses.
If the rubric is poorly designed, it may be impossible to tell from the results whether students have actually acquired the skill being tested. Their performance may instead reflect their difficulty in understanding the instructions or dealing with the format of the test.
The use of L1 in the rubrics of placement tests is therefore an issue that needs careful consideration.
19. What is test discrimination?
What is test discrimination? When is it not needed?
Discrimination is the capacity of a test to distinguish between candidates and to reflect differences in their performance. A score has little meaning in isolation. For example, a score of 70% tells us very little unless we know how other candidates performed.
A test on which almost all candidates obtain 70% clearly has little power to discriminate between students with different levels of ability or achievement.
Standardised tests are intended for a large population of test-takers and are first tried out on a representative sample. The sample should reflect the characteristics of the much larger group for whom the test is intended.
The results of the trial are then examined to determine the extent to which the test discriminates between individuals who perform differently. The discriminatory power of the test therefore needs to be established.
Discriminatory power may be particularly important in placement tests, where candidates need to be distinguished from one another and placed at appropriate levels.
However, discrimination may be less important in diagnostic tests or in tests designed to determine how much of a particular syllabus students have mastered. In such cases, the main purpose may simply be to establish whether particular learning objectives have been achieved.
The same applies to criterion-referenced tests and some performance tests. The main concern may be whether or not a candidate can perform a specified task or meet a particular criterion, rather than how the candidate's performance compares with that of other candidates.
There may also be a third group of learners who are learning to perform the required task: some elements of their performance may be correct while others are still developing.