Certificate free inside

In their widely read article "Inside the Black Box," Mr. Black and Mr. Wiliam demonstrated that improving formative assessment raises student achievement. Now they and their colleagues report on a follow-up project that has helped teachers change their practice and students change their behavior so that everyone shares responsibility for the students' learning.

(ProQuest Information and Learning: ... denotes obscured text omitted.)

IN 1998 "Inside the Black Box," the predecessor of this article, appeared in this journal.1 Since then we have learned a great deal about the practical steps needed to meet the purpose expressed in the article's subtitle: "raising standards through classroom assessment."

In the first part of "Inside the Black Box," we set out to answer three questions. The first was, Is there evidence that improving formative assessment raises standards? The answer was an unequivocal yes, a conclusion based on a review of evidence published in over 250 articles by researchers from several countries.2 Few initiatives in education have had such a strong body of evidence to support a claim to raise standards.

This positive answer led naturally to the second question: Is there evidence that there is room for improvement? Here again, the available evidence gave a clear and positive answer, presenting a detailed picture that identified three main problems: 1) the assessment methods that teachers use are not effective in promoting good learning, 2) grading practices tend to emphasize competition rather than personal improvement, and 3) assessment feedback often has a negative impact, particularly on low-achieving students, who are led to believe that they lack "ability" and so are not able to learn.

However, for the third question - Is there evidence about how to improve formative assessment? - the answer was less clear. While the evidence provided many ideas for improvement, it lacked the detail that would enable teachers to implement those ideas in their classrooms. We argued that teachers needed "a variety of living examples of implementation."

THE JOURNEY: LEARNING WITH TEACHERS

Since 1998, we have planned and implemented several programs in which groups of teachers in England have been supported in developing innovative practices in their classrooms, drawing on the ideas in the original article. While this effort has amply confirmed the original proposals, it has also added a wealth of new findings that are both practical and authentic. Thus we are now confident that we can set out sound advice for the improvement of classroom assessment.

THE KMOFAP PROJECT

To carry out the exploratory work that was called for, we needed to collaborate with a group of teachers willing to take on the risks and extra work involved, and we needed to secure support from their schools and districts. Funding for the project was provided through the generosity of the Nuffield Foundation, and we were fortunate to find two school districts - Oxfordshire and Medway, both in southern England - whose supervisory staff members understood the issues and were willing to work with us. Each district selected three secondary schools: Oxfordshire chose three coeducational schools, and Medway chose one coeducational school, one boys' school, and one girls' school. Each school selected two science teachers and two mathematics teachers. We discussed the plans with the principal of each school, and then we called the first meeting of the 24 teachers. So in January 1999, the King's-Medway-Oxfordshire Formative Assessment Project (KMOFAP) was born.

Full details of the project can be found in our book, Assessment for Learning: Putting It into Practice.3 For the present purpose, it is the outcomes that are important. The findings presented here are based on the observations and records of visits to classrooms by the King's College team, records of meetings of the whole group of teachers, interviews with and writing by the teachers themselves, and a few discussions with student groups. Initially, we worked with science and mathematics teachers, but the work has been extended more recently to involve teachers of English in the same schools and teachers of other subjects in other schools.

SPREADING THE WORD

Throughout the development of the project, we have responded to numerous invitations to talk to other groups of teachers and advisers. Indeed, over five years we have made more than 400 such contributions. These have ranged across all subjects and across both primary and secondary phases. In addition, there has been sustained work with some primary schools. All of this gives us confidence that our general findings will be of value to all, although some important details may differ for different age groups and subjects. Furthermore, a group at Stanford University obtained funding from the National Science Foundation to set up a similar development project, in collaboration with King's, in schools in California. Extension of our own work has been made possible by this funding. And we also acknowledge support from individuals in several government agencies who sat on the project's steering group, offered advice and guidance, and helped ensure that assessment for learning (see "Assessment for Learning," below) is a central theme in education policy in England and Scotland.

THE LEARNING GAINS

From our review of the international research literature, we were convinced that enhanced formative assessment would produce gains in student achievement, even when measured in such narrow terms as scores on state-mandated tests. At the outset we were clear that it was important to have some indication of the kinds of gains that could be achieved in real classrooms and over an extended period of time. Since each teacher in the project was free to choose the class that would work on these ideas, we discussed with each teacher what data were available within the school, and we set up a "mini-experiment" for each teacher.

Each teacher decided what was to be the "output" measure for his or her class. For grade-10 classes, this was generally the grade achieved on the national school-leaving examination taken when students are 16 (the General Certificate of Secondary Education or GCSE). For grade-8 classes, it was generally the score or level achieved on the national tests administered to all 14-year-olds. For other classes, a variety of measures were used, including end-of-module-test scores and marks on the school's end-of-year examinations.

For each project class, the teacher identified a comparison class. In some cases this was a parallel class taught by the same teacher in previous years (and in one case in the same year). In other cases, we used a parallel class taught by a different teacher or, failing that, a nonparallel class taught by the same or a different teacher. When the project and the control classes were not strictly parallel, we controlled for possible differences in prior achievement by the use of "input" measures, such as school test scores from the previous year or other measures of aptitude.

This approach meant that the size of the improvement was measured differently for each teacher. For example, a grade-10 project class might outperform the comparison class by half a GCSE grade, but another teacher's grade-8 project class might outscore its control class by 7% on an end-of-year exam. To enable us to aggregate the results, we adopted the common measuring stick of the "standardized effect size," calculated by taking the difference between the scores of the experimental and control groups and then dividing this number by the standard deviation (a measure of the spread in the scores of the groups).

For the 19 teachers on whom we had complete data, the average effect size was around 0.3 standard deviations. Such improvements, produced across a school, would raise a school in the lower quartile of the national performance tables to well above average. Thus it is clear that, far from having to choose between teaching well and getting good test scores, teachers can actually improve their students' results by working with the ideas we present here.

HOW CHANGE CAN HAPPEN

We set out our main findings about classroom work under four headings: questioning, feedback through grading, peer- and self-assessment, and the formative use of summative tests. Most of the quotations in the following pages are taken directly from pieces written by the teachers. The names of the teachers and of the schools are pseudonyms, in keeping with our policy of guaranteeing anonymity.

QUESTIONING