Article 1: Assessment in Education, vol.11 no.1 March 2004 [Original Title: Teachers developing assessment for learning: impact on student achievement]
Article 2: London: King’s College, Department of Education and Professional Studies [Original Title: Working inside the black box]
How can we improve formative assessment - what works?
Formative assessment involves a combination of activities aimed at targeting teacher and student actions increasingly precisely upon the next steps in learning for individuals. It involves dialogue that provides information about students' understandings that helps teachers and students interpret the effectiveness of the activity and tailor subsequent activities using that knowledge. Activities that contribute to formative assessment include different approaches to detailed feedback, teacher questioning and peer and self assessment in the light of learning goals and explicit criteria.
More details about formative assessment are provided in the GTC's Research of the Month summary of 'Inside the Black Box' by Paul Black and Dylan Wiliam. See 'Raising standards through classroom assessment'
Recent research studies have highlighted the value of formative assessment in helping raise pupils' attainment (e.g. Black and Wiliam, 1998a). Although the original work revealed clear messages about what was needed there was little or no exemplification of the strategies. The two studies which are the subject of this digest set out to evaluate and illustrate formative assessment methods in secondary schools in England. The digest is based on the findings and processes described in the first article which focuses on the quantitative evidence of learning gains among the students who experienced the intervention. This information is supplemented in places by extra detail about the process from the second article which is a synthesis of research and development studies which are published elsewhere (see Where can I find out more?)The digest also draws on some of the material in these other articles when appropriate.
Keywords
UK; Key Stage 3; Key Stage 4; Formative assessment; Learning strategies; Mathematics; Science; Teaching styles; Secondary schools; Assessment
Twelve science and twelve mathematics teachers, and, at a later stage, twelve teachers of English, from six comprehensive schools in two local authorities worked with a team from King's College, London. The project became known as the KMOFAP - King's Medway Oxford Formative Assessment Project. At the start of the project, in January 1999, researchers from King's College introduced the teachers to findings and suggestions highlighted by their earlier studies into formative assessment. The teachers used this evidence to help them to plan intervention strategies for development in their own classrooms. They selected their own target classes. Overall the research involved classes from years 7 to 11, including a complete range of ability sets.
Specific topics for classroom research included:
In order to develop their practice in these areas teachers undertook collaborative activities with researchers and others which included:
The reports summarised here are based on the classroom observations, the record of the one-day meetings and written reports made by nineteen of the teachers after the project had been running for eighteen months.
The researchers analysed test data from 25 classes containing pupils who, with their teachers, were participating in formative assessment activities. By comparing the performance of each class with that of a similar class which had not undergone the intervention they sought to measure the effects of the intervention on students' academic achievement. Rather than introducing a new testing regime, the researchers used results from national tests at key stage 3, GCSE examinations and internal school examinations for their analysis. They found that:
The researchers were very cautious in their claims about the effect of the intervention on attainment, not least because of the difficulty of ensuring robust comparisons. The most reliable form of comparison was between the attainment of the intervention class and that of a parallel class taught by the same teacher the previous year, but it was not always possible to achieve this. In a number of cases the researchers made comparisons with parallel classes taught by the same teacher in that year with the inherent risk that the teacher's involvement in the project spread to influence the comparison class as well. Overall the researchers predicted that improvements of the order of half a GCSE grade per subject are achievable.
Teachers involved in this research and development project reported that when compared with the responses teachers usually expected:
Teachers found that by increasing the time allowed for students to answer - the 'wait' time - they could be encouraged to make longer and better responses.
They also found that a 'no hands up' approach made it easier for them to involve a greater number of students and that all students learned to formulate a response.
The first question asked by the teacher in a lesson was important in setting the aims of the lesson and encouraging students to reflect on what they already knew. Open questions were more effective than closed questions which tended to call for memorised facts rather than encouraging students to think.
Students paid more attention to and were more likely to act on teachers' written feedback on their homework if grades were omitted. Students were involved in their own tracking of their responses to teachers' comments and this proved to be effective.
Students conducted peer assessment in groups which encouraged them to reflect on their own learning. They also found a particular and developed use of 'traffic lights' to be helpful in identifying areas for improvement.
All the teachers in the study recognised the importance of questioning as a way to engage students more effectively with the subject matter. Some of the teachers found it helpful to increase the 'wait time' between the teacher asking a question and the students responding. Earlier research (Rowe, 1974) (click to Where can I find out more?) had shown that the short wait time many teachers allowed tended to demand memorised facts as answers rather than reasoned responses. Such short wait times also engaged only a handful of students. By presenting students with much longer wait times, they became used to the idea that they had to think about the problem before answering. One teacher commented on the difficulties and success of this approach:
'Increasing waiting time after asking questions proved difficult to start...The pause after asking the question was sometimes 'painful'...Given more thinking time students seemed to realise that a more thoughtful answer was required...rather than adding a prompting question (as I would have done in the past), a pause is an effective way of indicating that more information is required...'
Teachers also developed a 'no hands up' approach as part of a policy of encouraging all students to prepare verbal answers to questions. As one teacher said:
'...I have also started to move away from hands up. The class know that they will all contribute at some point, and most responses are requested by me rather than volunteered.'
While encouraging all students to be prepared to provide answers, the teachers appreciated that it was necessary to create a non-judgemental environment in the classroom so that weaker students wouldn't feel intimidated:
'...Wrong answers slowly stopped being a problem as the students got used to the idea that I was very unlikely to express an opinion during these sessions.'
Teachers also understood the importance of open-ended questions, particularly when seeking to build on what students know already, as this comment illustrates:
'...when we start a new unit, we start by looking at what we already know. This might be...with open-ended, challenging questions - such as, "If plants need sunlight to make food how come the biggest plants don't grow in the desert where it's sunny all the time?" A far better lead-in...than "What is the equation for photosynthesis?"'
Teachers' concerns about the quality of marking of written work echoed the comment by OFSTED (1998)(click to Where can I find out more?): 'Marking is usually conscientious but often fails to offer guidance on how work can be improved.' Other research illustrated the complexity of this issue by reporting that students tended to ignore teachers' written comments if they were accompanied by marks or grades. One of the project teachers had noticed the same effect in her class:
'Previously I would have marked the work and graded it and made a comment. The pupils only saw the mark and/or credit. After a credit they lost any motivation to improve. Now they get credit only after we have gone over the work so they have an incentive to understand the work.'
Another teacher also moved away from giving grades in favour of written comments only:
'Comment only marking has made a difference in that students do now read what I write....I am not aware of a single parent expressing concern over a lack of grades/levels/marks.'
With the new focus on comment only marking and on students' responding to the comments, teachers were led to consider both the quality of their comments and the monitoring of the students' actions. The following comment illustrates the way the first problem impacted on teachers and how one teacher dealt with it:
'The extra thought needed when making these more relevant comments meant that marking took longer than it had previously. However, after several weeks of this, the time taken decreased. The reason for this...is that I have built up a series of 'good' phrases to use and, more importantly, I realise what makes an effective comment...Knowing the students very well now has also meant that my comments are much more personal.'
Checking to see if students did indeed act on teachers' comments called for new approaches. One teacher adopted an approach which involved students' tracking their own responses:
'I implemented a comment sheet (in) my Yr 8 classes books, the left-hand side is for my comments and the right-side is for the pupil to demonstrate by a reference to the page in their books where I can find the evidence to say whether they have done the work. The comments have been useful for the LSA (Learning Support Assistant) that works with the group.'
The research literature showed that peer- and self-assessment help to raise students test performance, and that they are in fact essential if pupils are to be able to manage their own learning effectively. In response to this evidence, several teachers explored ways of involving pupils in self- and peer assessment processes, including:
One activity, which involved both self- and peer-assessment, required students to explain part of the work to a small group of their colleagues. The other students had to judge whether the explanation was better than, equal to or not as good as the explanation they could have themselves presented. Feedback from students indicated the value of this activity in encouraging them to analyse their own reasoning: 'Explaining to other people and listening to their explanations really helps me to understand it much faster than just reading it at home.'
Teachers reported favourably on the involvement of pupils in marking their own or others' work, 'We regularly do peer-marking - I find this very helpful indeed. A lot of misconceptions come to the fore and we discuss these...'
Teachers developed three main activities to make their summative tests useful in giving feedback for learning:
A 'traffic light' system was used in some classes during revision activities. Pupils coded revision keywords red, amber and green according to their levels of understanding of the subject matter. Red indicated little or no understanding, green represented a high understanding. The pupils' homework tasks were then to try to turn red areas amber or green through further study. In this active approach to revision students also changed some green areas red or amber as they realised that they were not clear about them.
Teachers also explored the idea of engaging pupils in setting limited sets of test questions in order to extend their knowledge and understanding. As one teacher explained:
'...pupils were encouraged to write their own questions based on the work covered...We discussed the difference between learning strings of facts and trying to learn answers to possible questions....I believe those that grasped this apparently simple idea made a real sea-change in the way they reviewed their work prior to a test.'
After a test, pupils were asked to mark one another's test paper in peer-groups. Some teachers provided a mark scheme, but others asked the pupils to compose a mark scheme - an activity which made them think about the purpose of the question and about what might count as a good answer. The approach and the advantages here were very similar to those seen in the peer-marking of homework.
Some teachers felt quite comfortable with the transfer of responsibility to students. Others did not find it easy and felt they had lost some control over classroom activities. In some cases teachers expressed concern about the extra time pressure they anticipated. Evidence, however, showed that time spent on formative assessment was time well spent:
'...I am convinced that it is worth sacrificing the odd 'extension' piece of work, or even the odd small section of the syllabus, in order to ensure that what is covered is well understood.'
The researchers observed that the development of formative assessment practices in classrooms had effects at a number of levels. The first benefit was the improvement in revision and learning techniques and consequent gains in confidence of pupils. Secondly, teachers were enthusiastic about selecting and using assessment methods to enhance their teaching At a third and deeper level some teachers were changing the way they taught from 'delivery of learning' to 'guiding and coaching students' own learning activity':
'The exciting thing for me is that strategies like independent learning, group work and peer assessment are now more than just phrases. They have a legitimate place in a formative learning environment.'
The study also highlighted the continuing challenge of increasing the quality of teachers' responses to the evidence which comes from these formative assessment practices. In particular they drew attention to the need for teachers to include explicitly stated goals for improvement in their feedback and to improve the quality of their questions in the light of their students' responses.
In completing this digest its authors began to ask the following questions about implications for practitioners:
Black, P. and Harrison, C. (2001) Feedback in questioning and marking: the science teacher's role in formative assessment. School Science Review, June, 82(301), pp. 55-61.
Black, P. and Harrison, C. (2001) Self- and peer-assessment and taking responsibility: the science student's role in formative assessment. School Science Review, September, 83(302), pp. 43-49.
Black, P. et al. (2003) Assessment for Learning - putting it into practice. In press, Buckingham: Open University, for publication Sept./Oct. 2003.
Black, P. and Wiliam, D., (1998a) Assessment and classroom learning. Assessment in Education, 5(1), pp.7-71.
Black, P. and Wiliam, D, (1998b) Inside the Black Box: raising standards through classroom assessment. London: School of Education, King's College.
Harrison, C., Drozdowskij, J. and Westhead, K., (2001) Formative assessment in primary classrooms, Primary Science Review, May/June, 68, pp.19-22.
OFSTED (1998) Secondary education - a review of secondary schools in England 1993-1997. London: HMSO.
Rowe, M. B., (1974) Wait time and rewards as instructional variables, their influence on language, logic and fate control, Journal of Research in Science Teaching, 11, pp. 81-94.
King's College website:
http://www.kcl.ac.uk/depsta/education
from which there is a link to the King's Assessment for Learning Group