Das Video kommt von YouTube: erst beim Abspielen verbindet sich die Seite mit YouTube (Google).

Understanding speaking assessment: what every teacher should know

Name: Understanding speaking assessment: what every teacher should know
Uploaded: 2016-11-02T16:43:27+00:00
Duration: 54 min 38 s
Channel: English with Cambridge
Description: Für jede Prüfung kläre Zweck, Zielgruppe, Prüfungsgegenstand bzw. Konstrukt, Aufgaben, Bewertung und Wirkung auf das Lernen. Sprechaufgaben können …

English with Cambridge54:38 33.649 Aufrufe veröffentlicht 2. Nov. 2016 Auf YouTube

Das Wichtigste aus dem Video

Tipp auf eine Zeit – das Video springt genau dorthin.

Transkriptautomatisch erstellt · 430 Zeilen

Herunterladen

Thank you very much, people. And hello, everyone. I'm Evelina Galaczi. And I'm Nahal Khabbazbashi.
We work in the Research and Thought Leadership Group of Cambridge English. We'd like to welcome you to the first of a series of six webinars
on understanding assessment. Today, we're going to try to help you understand some key ideas about speaking
assessment and we're going to use simple explanations and examples. Our key aim is to explore key questions about speaking
and to give you some practical tips you can apply in the classroom. There are six key questions which we all need to address when we assess any skill,
whether that's speaking or reading or another skill. The first of these questions is "Why am I testing?"
This relates to the purpose of the test. The second question is "Who am I testing?"
This question relates to the test-takers who will be taking the test. The third question is "What am I testing?"
This refers to the skill that you're testing. The technical term for this is the test construct.
The fourth question is "How am I testing?" And this refers to the tasks in the test.
The fifth question is "How am I scoring?" which refers to how we go about assigning marks to the performance of the test-takers.
And finally, the sixth question is "How is my test benefiting learners?" This refers to the impact of the test on learning.
We covered the six questions in more detail in a previous webinar on understanding assessment.
And if you're interested, it's available for you to watch on the Cambridge English TV channel.
In the webinar today, as we focus on speaking assessment, we're going to look at three of these questions.
And they are "What am I testing?" In other words, what is the speaking test construct?
"How am I testing?" Here, we're going to look at speaking test tasks.
And lastly, "How am I scoring?" In here, we're going to focus on criteria for assessing,
speaking, and the different types of scale that we use to make sure that our tests are reliable.
Now, let's start with a question for you, and that's on your screen. What do you think can make speaking more difficult
to assess than other skills? Please share your ideas in the chat box. And here's the question for you again.
What do you think can make speaking more difficult to assess than the other skills? Please type some of your ideas in the chat box.
We're just waiting for some of the ideas to start coming through. Okay. So, we have someone is saying that it can be quite subjective.
And then, other ideas coming through are about student confidence. So, the confidence of the students is perhaps not
as strong when it comes to speaking assessment. Less objectivity is something from another person.
The word nerves came up in the chat box from one of you, meaning, again, that idea of that sense of anxiety when we have to take
a speaking test. Pronunciation, another person is saying that could be difficult to assess. Confidence, using it in
large classes. So, a lot of really fantastic ideas that are coming through and a whole range of ideas. We're going to just to summarise for you.
The main ideas that came through or the first one I would say is that it was subjective to mark.
A lot of you are mentioning that. And that means that it's difficult to make the test fair. Another set of
ideas that are coming through is that it is very difficult to both give the exam and to score it for the teacher
or the examiner. So, that's difficult to do. You were also mentioning the ideas of the criteria to use to assess
the test that came through. And also, assessing students at different levels of ability could be difficult. How do you decide what tasks are suitable?
And again, and finally, the idea that came through was the idea of anxious, of students being anxious when they take a speaking test.
Really great ideas, and we're going to touch upon all of these questions in our webinar today. Now, let's start with looking at "What am I testing?"
In other words, which features of speaking am I aiming to test? Or what speaking skills am I trying to elicit?
Let's take a look at this task. You will find it in your handouts on page 2. In this task, the test-takers have to read aloud sentences that they see on
the screen. Now, let's think about what this task is actually testing. When we think of speaking, a useful classification for speaking is,
is it a simple reproduction of language is it monologue speech,
or is it interaction? In a reading aloud task like the one you see on the screen,
test-takers just have to read out the text. In other words, they're just reproducing language.
Now, let's take a look at a different task. This one is on page 3 of your handout. In this task, a test-taker has to find out and describe
the differences between two pictures on their own. The speech generated in such a task
is monologue. Unlike the previous task, this is not a simple reproduction of language,
because test-takers have to rely on their own vocabulary and on their own grammar to actually complete the task.
Now, let's take a look at a third task. This one is on page 4 of your handouts. This task involves discussing a topic in a pair or a group.
The topic in this case is going on school trips. This task requires test-takers to go beyond just reproducing language
and beyond just talking on their own to actually interacting with other test-takers, which, as you might agree, might be more challenging for test-takers to do.
Now, what we'd like you to do is look at this task, which is on page 5 of your handout. In this task,
the test-taker is asked to describe a person that greatly influenced them. What do you think this task is testing? Is it, A, a simple reproduction of language?
B, a monologue? Or, C, interaction? Please vote now. Now, here's the question for you again.
What do you think the task which you can see on the screen is testing? Is it, A, simple reproduction of language? B, monologue? Or, C, interaction?
Okay, I can see your answers coming through. Majority of people are choosing option B.
Some people are saying A. There are a few people saying C.
I'm just going to wait a few more minutes for all of the answers to come through. Very much still, the majority believes it's B.
Yes, and we would agree with you that task B is mostly eliciting monologue. I can see why some of you have chosen task A as well
because you might be picking ideas that you see on the task and that can be a reproduction of language.
But I think task B because the learners are not engaging or not interacting with other learners, it can be a monologue.
Now, what we saw in the previous slides is that different tasks elicit different kinds of speaking.
Some elicit simple reproduction of language, other tasks elicit monologue speech, and some elicit interaction.
So, to get back to our question of what am I testing, this question refers to these different aspects of speaking
that can be assessed in a test. So, the next time you have to choose a task to assess speaking,
you should ask yourself which aspects of speaking the task is actually eliciting. Now, it's really important to remember that
none of these aspects of speaking are better or worse. It depends on the purpose of your test.
For example, a test for university entry should probably have tasks which elicit not just monologues but also interaction,
because interaction skills are really important for the purpose of that test. In another context, though,
the purpose of your test might be to quickly and efficiently measure the pronunciation of people applying to work in call centres
In that case, having different tasks such as reading aloud or repeating utterances may be justified.
Okay. Now, why is it important to think of the constructs or the subskills which the tasks assess?
Well, it's actually important because different speaking tasks activate different speaking processes in our brains
and we want to be sure that we assess a range of these processes. So, let's think about what speaking actually is,
what knowledge and what cognitive processes do we need to be able to speak.
Speaking, I think you would agree, is seemingly simple and natural, but it's actually a very, very complex activity.
Now, let's imagine that you're telling a friend about an interesting story you've heard on the news.
First of all, you need to come up with an idea with something you want to communicate.
At this stage, the ideas are not expressed in sentences. They just exist as non-verbal ideas.
In technical terms, we call this the conceptualisation stage.
Then, those ideas have to be converted into language. And to do that, you need to know some words.
In other words, you need to have lexical knowledge. And you also need to know some rules about putting those words together.
In other words, you need to have grammar and syntactic knowledge. This knowledge allows you to put words into phrases and sentences.
Then, these sentences and phrases have to be converted into sounds, which we articulate, which we say.
And that's the path of a non-verbal idea in our brain becoming
an actual idea which we express verbally. But while we're speaking, we're also constantly monitoring the ideas and
the vocabulary and grammar we're producing to make sure that our message is. communicated in the way we want it to be.
So, when we speak a language well, we do all of these activities automatically. But we can see how speaking can be very challenging for learners.
For example, they might not have access to all the necessary vocabulary and grammar and so, the retrieval of that vocabulary grammar might take a lot longer.
And this results in them speaking more slowly and with more pauses and hesitations. In other words, the language knowledge
and cognitive processes are not fully developed, they're not automatic. A useful way to think about
the different aspects of speaking is to compare speaking to driving. When we first learned to drive,
we need to cover the basics, the gears, the brakes, how to turn the steering wheel, and so on.
We typically start driving slowly and all of our attention is focused on getting the basics of driving right.
This is similar to a learner developing core linguistic knowledge and skills such as pronunciation, vocabulary, grammar, and basic functions, such as, for example,
requesting information or giving simple descriptions. Going back to driving, once we've learned the basics of driving,
then we're put in different driving situations. And this is similar to the range of communicative situations that
a learner will be exposed to. In driving, drivers need to be able to drive not just on quiet streets,
but also on busy motorways. Similarly, learners need to be exposed to
a range of communicative situations, which involve monologues and also interactions with others.
This is important because these different situations put different cognitive demands on the learner.
Experienced drivers can deal with different driving situations flexibly. And similarly, the more often learners have to do different kinds of tasks
which represent different communicative situations, the more automatic the processes will
become and the more flexible they'll become in a wide range of situations.
Just a quick mention of the CEFR here as well, because if you look at the level descriptors in the CEFR,
you'll see that as you go up the CEFR levels, they increasingly refer to the ability to be flexible and spontaneous.
And that means the processes have become more automatic as you develop. So far, we've talked about what speaking is.
Let's now think about how we go about eliciting the different subskills of speaking in a test.
The question here is "How am I testing?" And the focus is on the types of tasks that we can use.
Now, here's a list of some tasks that can be used to elicit speaking. For example, we could use a task which asks the learners to talk about a topic,
like talk about yourself or your hobbies. Other tasks might ask the learners to describe a photograph, or to tell a story.
These tasks all involve monologue speech. Other types of tasks tap into interaction.
They could be tasks such as question and answer tasks, discussion tasks, information gap activities, and role plays.
An important point to keep in mind is that all of these tasks have advantages and limitations.
And when using tasks in the classroom or in tests, you should aim to include a range of different tasks.
This will minimise the limitations of the tasks. Let's now look at one specific task in a bit more detail.
So, let's look at a task here together, which involves a discussion between two test-takers.
They are asked to talk together about a question and they're given some ideas to help them.
We'll watch an example of such a task. What we'd like you to do is to think about some advantages
and disadvantages of this kind of task. Please share your ideas in the chat box.
Here's a picture with some ideas to help you. I'll say that again. A girl is going to live in another city.
The people in her tennis club wants to give her a goodbye present. Talk together about the different things they could give her
and then say which would be best. All right. Talk together. Maybe the photo is a good idea because like this, she will have memories
of her friends and —. Yeah, I think it will be really nice. Or maybe a bag so she can put like all the things she needs for tennis.
Like a really big one so she can put everything. Yeah. And when she uses it she can remember also.
Or maybe the T-shirt like this. Yeah, we can make a T-shirt with her name behind. Yeah.
I don't think the ball are a good present. No. She will have a lot of ball. Yeah.
What do you think about the DVD of tennis? I don't know if it's a good gift because she know how to play tennis. Yeah, exactly.
It won't be like really useful. So, here's the question for you again.
What are some of the advantages and disadvantages of this kind of task? I can already see your ideas coming through.
Some people mentioned that one candidate might be monopolising the conversation or they might dominate the conversation.
Yana mentioned one student might feel too shy to talk to another student. I think Lydia mentioned partners may depend on each other as well.
In terms of advantages, I think someone mentioned that it can offer more interaction.
That it allows for more brainstorming. And that it's real task, it's communicative.
So, you're all mentioning a lot of really, really good ideas. So, this task is perhaps more symmetrical,
meaning that it's a task between equals as the candidates, they have less power difference between them.
It's also a common task in a communicative classroom and so it's good to also have it in a communicative speaking test.
It's an idea that you mentioned. And you said the task is quite similar to real life speaking.
However, some of you also mentioned some of the possible disadvantages. For example, it's a problem if one of the partners in the task is stronger in English,
if one of the partners is particularly shy, or the other one is more dominating, and that it's difficult perhaps to give individual scores
when the conversation really depends on both people. But also, that this is perhaps a bit too hard to practice in a big classroom.
So, here's a summary of the advantages and limitations of paired tasks. They have quite a bit of authenticity.
In other words, they reflect tasks that are part of the classroom and in real communicative situations. They also assess interactional skills.
One limitation of this type of task is the fact that one of the test-takers could affect the performance of the pair or the group.
For example, the personality of the test-takers, as you mentioned, they might be shy or they might be extroverted,
that can affect the interaction. Also, how well they know each other.
For example, are they friends or are they strangers? That can also play a role. And finally, the language ability
of the two test-takers could affect the interaction. Now, research has shown that the most important of these factors
is language ability level. This is something you all mentioned as well.
It is really important, therefore, when you're using paired tasks in a speaking test to make sure that the learners
in the pairs are approximately at the same ability level. It's okay to pair up a strong intermediate and a weak intermediate learner,
but it's not okay to pair up a beginner and an intermediate learner. So, all tasks have their advantages and their limitations.
An important question is what we should do to address the possible limitations of a task.
As we mentioned earlier, it is important to include a range of different task types.
For example, a test could consist of a question-and-answer task, a monologue task, and a paired interaction task.
In this way, each task is balancing out the limitation of the other tasks, but also assessing the different aspects of speaking.
Another way is by using different kinds of assessment scales and assessment criteria, which focus on different elements of speech, whether it's grammar,
vocabulary, or pronunciation, or if it's interaction skills. Now, we'll talk about this in a bit more detail later in the webinar.
A final point about task types is that there isn't just one way of testing speaking or there's no one best way to do it.
Different types of tasks have unique advantages and disadvantages. And each type of task should be evaluated against the purpose
for which it is to be used. In other words, the test has to be fit for purpose.
Now, let's look at task difficulty. And this is an issue that some of you have mentioned already in the chat box.
We're going to start with an activity for you. We're going to show you two different tasks
and then we'll ask you to decide which is more difficult and why. You can also find these two tasks on your handout on pages 6 and 7.
Now, here is Task A on your screen. This is a paired interaction task, which asks the test-takers to imagine that
a young man on holiday wants to buy a present to take home to his parents. And the test-takers have to talk together about the different presents he could buy
and then they have to decide which present would be best. They're given some pictures with ideas to help them.
Here is Task B. Again, this is the same kind of task. It's a paired interaction task.
In this case, it asks the test-takers to talk to each other about the advantages and disadvantages of communicating.
They're given some ideas about different ways of communicating to help them. They then have to decide which two ways of communicating are the most effective.
So, we've shown you the two tasks. What we'd like you to do is to tell us which task you think is more difficult,
is it Task A or is it Task B? And also, tell us why you think it's more difficult.
Please type your ideas in the chat box now. So, once again, tell us which task you think is more difficult, Task A or Task B,
and tell us why you think it's more difficult. I see lots of B's coming down.
So, a lot of you are choosing Task B as the more difficult one. Tell us a little bit more about the reason why you chose Task B.
So, what makes Task B more difficult? We're just looking at the answers.
Because there's so many of you, they're going down on my screen very, very quickly. So, some of you are saying that it needs a more advanced level of language.
In other words, you need to have more complex grammar and more developed vocabulary to be able to respond to Task B,
which is absolutely true. I would agree with that. Other people are saying that, again, coming back to the vocabulary,
you need more advanced vocabulary to respond to Task B. Someone else is saying that you need to be able to present an argument.
In other words, you need to be able to persuade your partner to agree with you or not. So, there is more of a discussion element in Task B.
Other people are saying that it's more complicated because of the topic, for example. So, I would agree with that.
There's some really, really good ideas here that I would agree with you. Just to summarise it for you, Nahal and I would say that Task B is more abstract,
which makes it more difficult. So, the ideas and the topic is more abstract.
You need more difficult or it requires harder vocabulary, more difficult vocabulary to complete.
And also, there's more of a discussion there in deciding the outcome of the task. Now, this activity shows you how important the role of topic is in task difficulty.
Task A, which is on your screen now, involves a concrete topic, choose a gift, for example, and a familiar context.
This makes it suitable for a B1 level task. So, concrete topic, familiar context.
In contrast, Task B is a C1 level task because it's more open-ended. It deals with a more abstract topic and a less familiar situation
on which you have to give your opinion. So, to summarise the topic and type of information the task requires can affect
the difficulty of the task. If you refer to the CEFR, the Common European Framework of Reference descriptors,
you'll see topics and/or information that are personal and familiar and concrete are easier than those which are unfamiliar and abstract
and which require more difficult vocabulary and more complex grammar. Another useful way of evaluating tasks is to look at
the functional language that they aim to elicit. These are all in the CEFR,
so you can look at those descriptors to help you decide the level. For example, informational functions, such as expressing opinions,
comparing and contrasting, are generally easier than more demanding intellectual functions, such as persuading or taking the floor.
In designing your own tests, you might want to think about the range of functions that can be elicited
in the tasks and their level of difficulty for the test-takers. For example, here's some questions which you might include in a test.
The general topic is school and studying. And the questions are, what is your favourite subject and why?
And also, tell us about your English teacher. Compare this to this other set of questions also on the topic of studying.
The questions here are, is it a good idea to continue studying throughout your life?
Why or why not? And how important is it to have a routine when working or studying?
The questions are worded so that they elicit a wider range of speech functions such as expressing or justifying opinions, or making comparisons.
They also ask about a more abstract topic, and this makes them more difficult. Here's some other points that you need to consider
when developing or choosing your tasks. First, are the instructions to the task clear?
If the instructions are not clear, the task will not generate the kind of language it is actually designed to generate.
For example, "talk about" is less clear than "talk together" or "talk on your own" or "ask an answer."
Also, is the purpose of the task clear to the test-takers? This is important because the test-takers need to know whether they
need to describe something, whether they have to compare something, whether they need to tell a story, or do they have to answer questions,
or do they have to ask their partners questions and so on and so forth. Another consideration is, is there a range of tasks in the test?
As we've seen, different task types have different strengths and limitations. So, including a range of task types
would minimise limitations and optimise their strengths. What is the order of the tasks?
It is important to start with easier tasks and then move on to more difficult ones. For example, a question-and-answer task may be easier than a paired interaction task.
So, it's better to start the test with a question-and-answer task that warms up the test-taker.
What is the timing for each part of the test? You need to consider how long each task is
and whether you are going to include any planning time before the task. And finally, are the marking criteria clear to the test-takers?
It is important to tell your students how you're going to score the test. Giving them this information will make it a fairer test.
We are going to be looking at marking criteria shortly in this webinar. So far, we've talked about what speaking is and what tasks we could use to elicit
the different speaking subskills. Now, let's think about how we go about scoring the speaking test,
which the task is — sorry, scoring the speaking which the task has generated and what assessment criteria we could use.
Here's a simple way of showing the importance of the assessment criteria and the assessment scales.
Now, assessment scales are the link between the speech which tasks have generated and the score.
The assessment criteria must explicitly state what the test aims to measure. For example, the criteria could be fluency, pronunciation, grammar, and so on.
A key question when assessing speaking is what criteria to use. The criteria have to reflect what the task is aiming to be testing.
Before we answer this question, we're going to watch a short video clip together of a student describing a photo.
As you watch the video, think about what criteria you would use to evaluate that student's performance.
Now, I'd like each of you to talk on your own about something. I'm going to give each of you a photograph of people enjoying music.
Victoria, here is your photograph. Please share it to Chiara.
But I'd like you to talk about it. And Chiara, you just listen.
I'll give you your photograph in a moment. So, Victoria, please tell us what you can see in the photograph.
In the photograph, we've got two girls. One is singing in the microphone.
Another is playing guitar in a bed. So, they are having fun. They're smiling, they're singing.
They are in a bedroom of one of the girls. We can see another guitar. And there's many corners in the bedroom.
And yeah, they're having fun. Yeah. Okay. We're just looking down at the chat box. You're giving us some fantastic ideas.
And you're saying that some of the criteria that you would use to evaluate this task would be grammar.
That's coming up quite often. Vocabulary or lexis, which is just two different ways of referring to the same thing.
Some of you are saying pronunciation as well. Others are saying fluency. So, we agree with all of these. It's just there was a question about fluency.
Fluency means how automated your speech is. So, you might use that criterion to evaluate this performance.
We also have to decide whether to score the quality of the ideas in the answer and some of you are saying you would also count
for how many ideas are mentioned. In this case, we would actually not score for ideas
because the focus of this task is language and it's not the quality of the ideas in the task.
Let's now watch another video clip and this is actually a video clip you have seen before.
This has two test-takers discussing a goodbye present. Again, as you watch the video,
think about what criteria you would use to evaluate the performance of these two test-takers.
Here's a picture with some ideas to help you. I'll say that again. A girl is going to live in another city.
The people in her tennis club wants to give her a goodbye present. Talk together about the different things they could give her
and then say which would be best. All right. Talk together. Maybe the photo is a good idea because like this,
she will have memories of her friends and —. Yeah, I think it will be really nice. Or maybe a bag so she can put like all the things she needs for tennis.
Like a really big one so she can put everything. Yeah. And when she uses it she can remember also.
Or maybe the t-shirt like this. Yeah, we can make a t-shirt with her name behind. Yeah.
I don't think the ball are a good present. No. She will have a lot of ball. Yeah. What do you think about the DVD of tennis?
I don't know if it's a good gift because she knows how to play tennis. Yeah, exactly. It won't be like really useful.
So, a lot of you are saying that you would use the same criteria we used for the previous task.
Plus, you would also use a criteria to include interaction skills of the test-takers. And some of you call it communication skills
or interaction, you're referring to the same things. So, in a task which includes interaction,
it is essential to measure those interactive skills since this is part of the test construct.
Interaction might be, for example, how well test-takers introduce a topic, or how well they develop a topic
which has been introduced by someone else, or how will they relate what they're saying to what has been said previously.
When deciding on assessment criteria, you also need to decide if they're all going to be equally important.
For example, you may decide that pronunciation is more important for the purpose of a test, for example, in a call centre.
In this case, you may decide to give double weighting to pronunciation because it's an important criterion for you.
But you have to remember that what you decide will be based on what aspects of speaking you're aiming to test.
So far, we've talked about the assessment criteria you might use in evaluating speaking tasks.
Let's now see how you can put those criteria into scales. The two most common types of assessment scales are holistic scales,
in which the rater awards an overall mark to the performance based on a general impression of the test-taker's performance.
In analytic scales, a skill such as speaking is broken down into separate features, such as fluency,
vocabulary, pronunciation, etc. And raters give independent marks for each of the assessment criteria.
Here's an example of a holistic scale from Cambridge English. You can find this scale on page 8 of your handout.
When an examiner uses this scale, he or she gives one overall score only for global achievement.
And here's the set of analytic scales. This is also on the handout. This one has four categories.
These are grammar and vocabulary, discourse management, pronunciation, and interactive communication.
When an examiner uses these scales, he or she assigns a score for each category. In this case, he or she would assign four separate scores.
Let's now look at some advantages and disadvantages of these different types of scales. Analytic assessment scales can provide
diagnostic information about candidate's strengths and weaknesses. That is, if the scores are actually reported separately.
They're also useful in rater training since the skill, which has to be measured, is broken down into separate features.
They're also really useful for assessing second language learners who might have uneven profiles.
For example, if they have a wide range of vocabulary, but they're not necessarily very fluent.
This can be captured with your analytic assessment scales. The downside, of course, is that using analytic scales is more time-consuming
and it's also more cognitively demanding for the raters who are also often the interlocutors.
The main strength of the holistic scales is that they're practical and they are much faster to apply.
However, they're also limited as they do not offer useful diagnostic information beyond a single score.
It is also not possible to know if the candidate scored better on features such as grammar or vocabulary, or did not score as high on the fluency or pronunciation.
There's also an underlying assumption that all the relevant aspects of speaking ability develop at the same rate.
But I'm sure you all know learners who might be very fluent, but who may not use grammar or vocabulary very accurately in speech.
Okay. So, looking back now, to summarise for you, we said at the beginning that speaking skills could be more difficult to assess than
other skills and most of you agreed that one of the main reasons for this was because speaking could be quite subjective to assess.
But we know that being fair and equal to all candidates is very important in testing. So, what we've done today is gone through what we have to do,
we have to consider in order to make a test fair. This means that it would be the same for all the candidates at a certain level.
So, to sum up, here are the points that we made today. To make a test fair, you need to pay attention to the test purpose
and have a clear purpose. You have to think about the test construct as well.
In other words, what you're trying to test. You have to make sure that your task types are matched to what you want to assess.
You have to use appropriate assessment scales and criteria. And of course, you also need to provide training to make sure that
your examiners deliver and score test in a consistent and reliable manner. All of these factors will make the test more objective
and so they will make it a fairer test for your students. We've now come to the end of the main webinar and in a moment,
Nahal and I will answer your questions that you've sent about the session. Please type your questions into the chat box.
But before that, Pippa is going to show you how to find some additional resources to support you and your learners.
Thank you so much, Pippa, we're going through the questions. They're fantastic questions.
We won't be able to answer all of them, unfortunately. Let's start with a question from Dave who says,
"What is fluency and do raters agree with it?" This is actually a really good question
because there are different perceptions of fluency. There's the broad perception of fluency, which means language competence.
So, if you're fluent, it means you're fluent in a language. But there is a narrow meaning of fluency
which is how automated your language competence is. So, in the scales that we use at Cambridge,
we use this more narrow definition of fluency. In other words, how automated is your language ability,
and that's measured through how many pauses you make, how many filled and empty pauses you make, how fast or slow you speak.
So, those are some of the measures of fluency in that narrow sense. And do raters agree with it? When you train your raters,
you train them to understand what the meaning of that criterion is. Okay. I'm going to take a question by Heloisa.
I hope I'm pronouncing your name correctly. And I think she is asking, "How can I assess speaking performance
being the interlocutor and the evaluator at the same time?" First of all, it is really cognitive on a person if they're both
the interlocutor and the rater. At Cambridge, the examiners are given lots of training,
but also you have a script to follow. So, your exam is structured and you have — so you know what to follow,
which really gives you a bit of more information, a bit more structure. That is not to say that you have to go exactly by the script.
You can have some flexibility, but just having a bit of structure really helps. The other thing as well is that a lot of times, as we talked about,
you have different task types. And particularly, when you have a monologue,
it gives you a little bit of time when the person is speaking for about a minute or two for you to think about their previous performance and what's coming up next.
So, you actually have to use that time really well to do the work. If you agree?
Yeah, I do. Okay, moving on. A question from Antonella, who says, how can you — I'm sorry,
I just lost the question. It went down a little bit. Is it, is it? But yes, do you have any words — I'm sorry, I'm just going to look at Nahal's iPad.
"Do you have any words of advice about pairing students of different ability in a paired task, because in some context, it's difficult to do," she says.
I would say that it is crucially important to have more or less the same language ability in order to perform a successful paired task.
If you don't have that, if you have students who are, let's say, beginners, intermediates, and advanced in the same context,
then I would advise not to use a paired task, because this is the number one important feature for paired tasks to be successful and
that is, for the language ability of the two test-takers to be more or less the same. Okay. So, I'm taking a question by Patricia.
"What can be done to help students with conceptualisation The generation of ideas in adolescence is quite difficult."
I mean, one idea is to, for example, have a few problems or a few ideas, using pictures where you do provide a few ideas for your test-taker to think about
so that they can start talking about it. If you were in a classroom,
if you've read a phrase, if you've read something already, if you've listened to a text, where you have the ideas already,
then you can ask your students to summarise them or talk about or discuss a particular thing.
You can have them debate something which they've already listened to. So, there are ways that you can engage your students in coming up with a few ideas.
Yeah. Okay. I'm taking a question from Vasuki, who says very succinctly, "How to bring out the inhibited?" I think what Vasuki means here is how to support,
how to really help students who are more shy, students who have a difficult time speaking,
which is a problem that a lot of us face, in fact. I would say that it is really important to develop familiarity
with the tasks that you have in the test. So, that means in the classroom, you need to be giving lots of practice time
to learners to practice all kinds of tasks, which includes reading aloud if that satisfies
the purpose of your test, which includes monologue tasks, which also include interaction task.
Also, to support learners who might have more difficulty on speaking, it's important to give them ideas to talk about.
This goes back to the cognitive processes we talked about, so the conceptualisation stage,
which is the highest stage, that's the generation of ideas. Don't expect them to come up with the ideas.
You give them some ideas to support them, which takes the pressure off generating ideas
and just then gives them time to just put their language resources together. So, that's another way of doing it.
But further way is to maybe step out of the classroom and just have them assess one another.
So, they'll be presenting in small groups to one another, and get your students, the audience listening to their classmates to provide
an assessment of how they've done. So, that's a peer assessment or a self-assessment. So, try to involve them as much as you can in the assessment process, support them,
and hopefully, that'll give them the confidence to be to be less inhibited in speaking. Okay. I'm going to take a question by Lucille.
"What's the exact way to grammar and evaluating spoken interaction? What about successful communication?"
I think one thing that we really try to emphasise is that whatever assessment criteria you focus on depends on the purpose of your test,
but that all of these different features, whether it's grammar, whether it's vocabulary, whether it's fluency, or communication,
they're aspects of speaking. So, in a way, they're all important
for you to be able to communicate coherently. So, I think for raters, a lot of raters pay a lot of attention to grammar
because grammatical error just stands out. However, if we are thinking about communication, that's really the key thing.
Grammar should not be the only aspect you pay attention to. So, I think you need to give a weight to all the assessment criteria equally.
And I don't know. Evelina, what do you think? Absolutely. It depends on your purpose, of course, of the test.
But the Cambridge approach, the Cambridge philosophy is that all four aspects
or all aspects of speaking are equally important. So, you need to have good grammar and good vocabulary because those are
the building blocks of the language. You cannot communicate without that. But that in itself is not enough.
You need to be able to put those into communicative situations. And so, that's why we give marks for other aspects of speaking as well.
Okay. A quick question from — I've just lost it a little bit, yes, Athanasios, I hope I'm pronouncing the name correctly,
who says, "Why isn't the speaking test in Cambridge exams rated as much as the rest of the papers?"
Actually, it is rated as much. In most of the Cambridge exams, we have a reading paper, a listening paper,
writing paper, grammar, vocabulary or use of English paper and the speaking paper, all components have equal weighting.
And that, again, is part of the Cambridge philosophy. We believe speaking to be just as important as the other skills,
so speaking gets equal weighting in the final mark. Equal.
It contributes to the final mark as much as the other papers. Okay, I'm going to take one by Rhea who was asking, "Is it highly recommended
to let the test-taker know how she's being scored in speaking tasks?" I would say absolutely yes.
It's for the sake of transparency. Because in any exam that you go into, you have to know what is expected from you.
So, I think making the descriptors, making the criteria very clear to the test-takers is really important. So, that's a quick one for you.
Absolutely. And another quick one, I think, I lost who asked this. But the question was, should we obsess about pronunciation?
So, in other words, what should be the role of pronunciation? The Cambridge approach is that
you don't need to speak like a native speaker to get top marks. In other words, you don't need to have a native speaker accent. You need to be intelligible.
So, the focus shifts from having an accent or not having an accent to being intelligible.
So, if you look at me, for example, I'm a non-native speaker, I have my accent, but I'm fully, I hope, I'm fully intelligible.
So, in other words, don't obsess with accent, but obsess with intelligibility, how intelligible are your learners.
That is the main focus of pronunciation, we believe. There's a question by Rita. How long should we ask our pupils to speak?
I think there's no really right or wrong answer about how long you're asking people to speak.
I think you can't just expect them to go and talk for, let's say, one or two minutes or like 15 or 20 minutes without giving them structure,
without giving them very clear instructions of what you expect them. Having said that, if you're thinking of designing a test,
you should probably trial the test to see how, generally if you have 10 or 15 students to trial the test with, how long do they, on average, take?
And then, that will give you a good idea of how to design your test and what is a good time.
And also, to have a little bit of leeway, whether it's like don't give exact — it has to be exactly 45 seconds,
because some people take a bit longer, some people take a bit less or more. So, I don't think you should be very precise on exactly how long.
Yeah, I agree with that. Okay, a question from Omida, who says, "How often should we do speaking assessment?"
I would say, and again, this is the Cambridge philosophy, it's not just me, that you shouldn't teach, stop, and assess.
We don't believe that's a very productive way of going about assessment, but you should integrate assessment as part of the learning, as part of the teaching.
What that means is that you should have lots of learning checks of speaking throughout the way.
You can have a bigger speaking test at the end of the term or at the end of the course, whatever works for you.
But don't think of it as now I'm teaching. And now, I'm going to do assessment. Think of integrating the two and you can do that through shorter, but more frequent
and more regular checks along the way. And I would add a really important aspect of that would be to provide feedback
to your learners every time you do a learning check on their speaking, because learning, the number one or one of
the top factors that helps learning is getting feedback. So, take every opportunity during those learning checks or checks on their learning
or checks on their speaking to give feedback to your learners as well. Okay. This is actually not just one person asking.
A lot of people are asking how to deal with larger classes. And this is normally a question we leave for last because we know it's
a difficult one to answer. So, one idea, Evelina's idea last time was that it's good to pair up with another teacher with another class.
So, when you're assessing, let's say, groups of two or three people, you have somebody else is taking over the teaching,
so you're making sure that your students are not just waiting around. Another thing as well is I think it's the use of technology.
These days, you have access to recordings where you can give groups of students a recorder and then you give them a task and say, "Okay,
discuss this topic for the next four or five minutes." And you can record this, they can bring it back to the class.
You can listen to them and give them feedback. But also, they can listen to the task and they can give each other feedback.
So, in a way, you're giving more agency to your students, this is more learner autonomy, and they can self-evaluate a lot better.
So, I think making use of technology, which is out there, might be a good way to go about assessing.
Absolutely. Yes, definitely. We have time for a few more questions. Here's a question from Freddie, who says,
"Is prompting valid during speaking assessment?" In other words, when a student gets stuck, when
they don't know what to say, when they haven't understood the question, is it okay to prompt them? I would say definitely yes,
because what makes face-to-face speaking assessment different from a computer-delivered test is that you have the opportunity for this adaptability.
In other words, when you're going through the test and you realise that a student hasn't been able to understand the question, they need prompting,
step in and give them the prompting with some backup questions, some easier questions, or maybe just the repetition of the question.
Obviously, take that into account when you give them the mark at the end. The fact that they needed prompting should be reflected in their mark.
But definitely, don't stand back and behave like a machine. Behave like a human who is interacting and responding to what they're saying.
This is different from a computer-delivered test where there is no prompting. You get the question, everyone gets the same question,
if you don't understand, tough, the computer moves on. So, yes, be human in a human test, in a face-to-face test,
which includes prompting if necessary. Do we have time for? We do.
I think we have maybe one more question. One more question. Okay, I'm going to answer a question by —
I don't know how to pronounce your name, I'm sorry. I think it's Giuliana. I'm really sorry
if I'm not pronouncing it correctly. So, the question is, what if a student is faced with a topic she is —. Giuliana. Giuliana? Okay. I think Giuliana, maybe.
Okay. Thank you. Is not familiar with enough to be able to provide a solid answer. For example, in IELTS, a lot of questions deal with certain topics,
but they're not familiar to the students, yet student's English is very advanced. So, I think, as we talked about, topic is very important.
What we try in a lot of Cambridge English exams, including IELTS,
is not to only have the one topic. In fact, in IELTS, in the beginning, you have two or three different topics,
then you go to a monologue, which is one topic, and then you go further when you're asked a series of questions
on the general theme of the topic, but the questions are quite different. So, I think even if you don't have the answer to one of the —
you don't have enough ideas about one of the topics, you will still be able to answer others.
But also, you're given, especially like part two of IELTS, you are given some prompts, so you are given something for you to be —
they give you a little bit of structure for you to be able to come up with something to say.
And overall, throughout the whole 15 minutes of the test, you should be able to produce language,
especially if you're saying your students are of higher level of ability. Exactly. So, the key is, I agree with emphasise just to emphasise,
providing some ideas for your learners. So, you have unpacked the topic a little bit for them.
So, the focus is, again, not on generating ideas, but the focus is on generating language.
So, giving them visuals, as we showed you, giving them a few bullet points that unpack the topic
a little bit more would give them that support to generate the ideas. Okay. Unfortunately, this is all we have time for today.
Thank you so much for attending. It's, as always, a huge pleasure for Nahal and myself to be able to share our ideas
with such an international audience. It's a privilege and a pleasure. So, thank you for joining us. We hope you found our ideas useful.
And we look forward to seeing you at our future webinars. The next webinar, which is coming up, is on understanding test scores.
And that webinar is going to come out later this month on the 14th and 16th of November.
So, we hope that you can join us, or join another colleague, but join Cambridge for that webinar later this month. Thank you very much.
Thank you very much.

Zum Nachlesen

ECL-PrüfungssystemDas ECL-Prüfungssystem bietet einsprachige Prüfungen an. · Im Rahmen der ECL-Prüfung werden vier Fertigkeiten getestet: mündliche Kommunikation, Hörverstehen, …