Today, I gave a lecture on Test Qualities (Bachman and Palmer, 1996:19-36) in my 7:00 a.m. Test Design class using a power point presentation. Aside from the concepts I presented, I gave examples (from real tests) for each of the test qualities to point out, for example, why a test is not reliable or why a test exercise is not valid, authentic, practical, interactive. Dealing with so many concepts so early in the morning and with the lights dimmed for the ppt presentation, I couldn't help but notice how some of my students were trying hard to follow the lecture and others were trying not to fall asleep! Using all the techniques I knew of to engage students in the lesson, including that where you call on the one whose eyelids were halfway down, I finally succeeded waking them up when I got them to practice detecting the lack of test qualities in more examples I gave them after outlining all of the test qualities. What really got them fully awake was when I gave them the reason for the practice activity, telling them that one other activity they could be involved in after graduation, aside from teaching and designing tests, which they were learning in this course, was to validate tests - to see if the tests are valid measurements of language ability, before they are administered for a final test, for example.
Seeing the relevance of what they were learning in class to their future professional activity as English teachers, as well as having a glimpse of an option where they might be involved in as ELT degree holders, became the factors that awoke their interest in the lesson, regardless of what advanced technological aid I used (well illustrative ppt presentation), and the authentic examples of problem test exercises I had compiled to illustrate test qualities.
I believe this is a one way of understanding why teaching competences could possibly give more results in the field of language teaching because learning can be more effective when students become aware of the real purposes for what we teach them, the relevance of what they learn for their profession or for what they will be involved in away from the classroom, and of learning and developing specific skills, in this case, validating a test, in class. How many students finish their studies, for example in ELT, and are not fully aware of their capabilities in their chosen profession other than just teaching in the classroom, when they can be materials designers, test and curriculum writers, language education administrators, teacher trainers, evaluators and counsellors (even a self-access centre administrator or counsellor).
In language learning, we give them the knowledge of the language, but we need to show them when and where to use the language and for what purposes.
Comments
mceupc
Hello, again, Ellen!
I'm very pleased to have the chance to keep reading on your views related to ELT techniques and beyond it!
Through your words it's pretty clear that on inviting the students/teachers to practise, I mean, getting them really involved in a "do-it-yourself" activity and showing them the relevance of the contents they are learning, they will be more likely to understand the positive impact for their future professional activity.Competence and purpose have come directly related.
It has also been very interesting your reflection and suggestion as for pointing out other"capabilities in their chosen profession other than just teaching in the classroom."
By the way, let me share this with you ... In our case (we must confess) "just teaching in the class" has filled us with much enthusiasm and contributed to develop permanent life learning.Of course, there were strengths and weaknesses. But we always loved the challenging moments, taking the most from them.
Very recently retired, I think I'll always be connected to this inspiring global learning!
Thank you, Ellen.
Warm regards,
Mª do Céu
eleanor_occena
Dear Ma do Céu!
Indeed, there are lots of challenges in the classroom. I forgot to mention that English teachers can do small scale research (action research) in our classrooms and publish and share their experiences to others. I'm sure that many young recent graduates could learn a lot from voices of experience like yours, through teacher forums like this webpage.
Jane Willis, TeachingEnglish guest contributor talks about the book Teachers Exploring Tasks in ELT that she co-edited with Corony Edwards, which is a compilation of teachers' accounts of activities they carried out in class.
I'm happy to hear from you everytime. Thanks again for responding!
Take care,
Ellen
halasalih
eleanor_occena
Hello Halasalih,
Thanks for writing and for your interest on the topic of testing.
Before actually learning how to design tests, students need to look at a lot of concepts realted to testing, one of which are the test qualities - the topic of the lecture I gave.
I chose Bachman and Palmer (1996) as reference for the six test qualities listed below, although I used other references as well:
Reliability - "consistency of measurement...across different characteristics of the testing situation" Bachman and Palmer (1996).
Davies and Pearse (2000) also states that realiability is "a amtter of how far we can believe or trust the results of a test".
Validity - Bachman and Palmer (1996) uses the term construct validity - referring to the extent to which we can interpret a given test score as an indicator of the ability(ies) or construct(s) we want to measure.
Authenticity - the degree of correspondence of the characteristics of a given language test task to the features of the target language use.
Interactiveness - the extent and type of involvement of the test taker's individual characteristics in accomplishing a test task.
Impact - who will be afftected by the results of the test at a micro and macro level.
Practicality- refers to the extent by which the demands of the particular test specifications can be met within the limits of existing resources.
When discussing each test qualitiy, examples of real test exercises I have gathered (from validiting tests in my university) were shown to help them understand what each looked like when lacking of it.
warm regards,
Ellen
englishhh
I really agree with all the different characteristics mentioned before by Eleanor; I think that this are the main aspects at the moment of choosing a test.
Sometimes, I like to measure my English skills answering some English online placement tests, but I think that they are almost full of grammar questions and you are answering like a robot. I think that they are not enough accurate and there is something missing on them. Also, they say that at the moment of arriving to the school you will have an oral test but well I think it measures more the oral part than the online part.
What do you think about time? Usually when somebody is being tested, spends hours answering questions. Do you think you need almost four hours or more to know what is his or her exact English level?Are them practical, have an impact, are authentic and reliable?
Liliana Rodriguez Vega
"I think luck is the sense to recognize an opportunity and the ability to take advantage of it... The man who can smile at his breaks and grab his chances gets on."
S
eleanor_occena
Hi Liliana!
Thanks for writing.
Placement tests aim, as its name suggests, to place you at a certain level of the language. Each item of a placement test corresponds to a certain level of the language. A placement test intends to place you above the level of the items that you successfully answered. But they are indirect tests that use grammar to give us an idea of how much one knows of the language. The problem with indirect tests as Hughes (2003) says, is that "performance of the skills in which we are usually more interested, tends to be rather weak in strength..."
As to the length of tests, Davies and Pearse (2000) are of the opinion that a longer test is more reliable than a shorter one because a longer test gives us more samples of the language to measure the language ability of the test taker. This is problem with placement tests because with a small sample of the language, you are placed at a certain level.
Time should be organised in order to administer tests that take long hours which might not be practical. Tests of long duration is likely to be of high stakes, or having a strong impact on the institution or the test taker, such as when they take proficiency tests to obtain a certification of their level.
Test designers are trained to use authentic language when designing test items. As to reliability, long tests such as proficiency tests are usually validated by a group of people before they are administered internationally or online.
Hope I've answered your questions.
Best regards,
Ellen
englishhh
Hi Eleanor
I really appreciate yout thoughts and knowledge. Right now it has been a complete issue all related with placement test. In my experience sometimes I ask my new students to do the written part and then at the end the spoken one. I have noticed that most of my students they know the grammar pretty well but at the moment of speaking they confuse tenses, lack of vocabulary, mispronounced words etc. In that case based on the results I place them on a lower expected level. At the end I have decided to do only the oral part because it can tell me more what is his or her real level.
Another thing is that students complain a lot about taking tto long to place them and of course they say that at the end they are tired.
What I see I think there must be like another way.. may be I am a little demanding.
Best wishes
Liliana Rodriguez Vega
"I think luck is the sense to recognize an opportunity and the ability to take advantage of it... The man who can smile at his breaks and grab his chances gets on."
S
eleanor_occena
Hi Liliana!
I seem to get the impression that because these are your students, what you are trying to do is looking at their strengths and weaknesses, which means that you are administering diagnostic tests in the different skills. I think what you mean to do is to determine if you have a mixed ability group, not because you want to classify them into different levels as it would be very complicated to teach different levels in every class session, but because you want to plan your classes, based on the results of the diagnostic tests, that you have students of different strengths and weaknesses. I suggest that you administer the tests in each of the skills on different class sessions. Taking the tests when they are already tired or when they are not anymore in the condition to do so, may not produce real results.
warm regards,
Ellen
carladelia
Dear Ellen,
How are you? I've been following your blog post and comments and there's a question I'd like to ask. What are your views concerning grading students? I've been teaching in companies, generally individual classes and each student has a taylor-made course. I use coursebooks and other supplementary materials. I always give my students feedbacks concerning their progress and I've noticed that receiving a grade "means a lot to them". How do you feel about it?
Best Regards,
Carla D'Elia - English Teacher
eleanor_occena