Transformative Assessment by W. James Popham

I love the first two sentences of this book:

This is a book about classroom assessment, but it's not about giving tests.  Really, it's a book about instruction, because classroom assessment can fundamentally transform the way a teacher teaches.

Acknowledging that such claims about fundamental transformation usually means that you're "dealing with an author in need of reality therapy," Popham supports it later with data from a Black and Wiliam meta-analysis (1998) which reports an effect-size between 0.4 and 0.7.  (and explains that an effect-size "in a recent...study of mathematics would have raised the rank of a middle-of-the-pack country to one of the top 5 nations.)

The Case for Formative Assessment

Popham provides his definition of formative assessment:

Formative assessment is a planned process in which assessment-elicited evidence of students' status is used by teachers to adjust their ongoing instructional procedures or by students to adjust their current learning tactics.

What's so special about formative tests?   There's actually no such thing as "a formative test."  "It's not in the nature of the test that earns the label formative or summative but the use to which that test's results will be put.  If the purpose of Test X is to provide teachers and students with the evidence they need to make any warranted adjustments" then it's part of the formative assessment process."

This definition is particularly important, according to Popham, because companies are trying to sell "benchmark tests" as "formative assessment."  Too, Popham warns that there is currently no-evidence that district-developed or state-developed assessments boost student achievement.  The effect size noted above is reserved for "classroom formative assessment" -- teachers figuring out what kind of (perhaps very quick and dirty, perhaps elaborate) assessment will produce evidence of learning.

Learning Progressions

Popham claims:

Formative assessment is all about decision making.  Those decisions, made by both teachers and students, invariably revolve around the following two-part question: "Is an adjustment needed, and if so, what should that adjustment be?"

But what the context for making adjustments?  Theoretically, you could be making adjustments after every piece of incoming data.  How do you decided where the "decision-making assessments" belong?  Popham says it's "learning progressions" which are composed of the step-by-step building blocks students are presumed to need in order to successfully attain a more distant,  designated instructional outcome."  

A couple caveats...
A learning progression isn't unerringly accurate.  It's teachers' best thinking, hypothesizing.  It's "instructionally defensible."
A particular learning progression isn't suitable for all students.  It's a hypothesis of how the greatest number of students will learn.
A learning progression isn't necessarily better because it's more complex.

And a few ideas about how to build learning progressions

1. When should the teachers assess the sub-skills found at one of these building blocks?  Before proceeding to the next building block.
2. How many building blocks should you build?  only those for which you plan to collect assessment data. (and CAN collect measurable data!)
3. Which building blocks are truly requisite?  those that, "if unmastered, you'd really need to reteach it."
4. When developing assessments, think about creating "behavior-difference situations,"  "settings in which individuals would behave differently depending on whether they have or haven't already mastered the target curricular aim under consideration."

Levels of Implementation
Popham talks about four levels of formative assessment.  Level 2 is when students' are self-assessing and changing the procedures they're using.  Level 4 is schoolwide implementation.  Neither of these were as helpful as Level 1 and Level 3.

Level 1: Teachers' Instructional Adjustments
Teachers collect evidence by which they decide whether to adjust their current or immediately-upcoming instruction in order to improve the effectiveness of that instruction.

Is a teacher good at immediately diagnosing, by sight, the needs of classroom a good formative assessor?  No.  check this out:

Teachers often make on-the-spot instructional adjustments.  If, in the middle of Mr. Howell's explanation to his class of 4th graders, he takes students' glazed-over expressions as a hint that this particular segment of his explanation is confusing, he is likely to make an instant decision to return to that apparently murky explanation in order to do a better  explanatory job.  Such on-the-spot adjustments are usually defensible because teachers have a way of inferring that such instant changes will be beneficial.  These sorts of adjustments, though commendable, are not what the first step of Level 1 formative assessment is all about.

Instead, teachers should look to the pre-decided learning progressions and pre-set "triggers" for what is acceptable evidence of mastery on specific assessments.  The assessments do not have to be pencil-and-paper affairs, either.  Popham talks about letter-card responses, key questioning during discussion (with random responses), whiteboard responses, traffic-signal techniques, etc.

All of this already paints a picture of a highly skillful practitioner.  Popham suggests that the most skill, though, is in HOW to adjust instruction.

Whether  major or minor, adjustments in a teacher's instructional design typically require the teacher to draw on personal pedagogical expertise.  Here's where teachers need to show just how instructionally astute they really are.  This is the moment when top teachers shine.

Level 3: Classroom Climate Shift
Teachers consistently apply formative assessment to the degree that its use transforms a traditional, comparison-dominated classroom, where the main purpose of assessment is to assignment grades, into an a typical, learning-dominated classroom, where the main purpose of assessment is to improve the quality of teaching and learning.

Popham says that you'll notice a teacher who has undergone the shift from Level I to Level III by observing the classroom:

1. Learning Expectations.  Do all assignments, activities, and assessments reflect the teacher's belief that all students will master all curricular aims?  Do students' behavior and actions suggest that they believe success is within their grasp?
2. Responsibility for learning.  Do assignments, activities, and assessments place significant responsibility for learning on students, as individuals and as members of a community of learners?  Do the students' actions and behavior show that they're assuming meaningful responsibility for their own success and the success of their classmates?
3. The role of classroom assessment.  Is classroom assessment consistently employed to generate evidence of learning with the goal of informing teacher and student adjustments?  do both students and the teacher see it as the means to improve learning rather than the means for ranking and comparing students?

These changes can also be seen in students:

• Students need to believe the teacher is invested in having all students succeed -- not just those who are the "best" students.
• Students need to believe the teacher is using formative assessment's test results exclusively for improved student learning -- not for allocating grades and not for determining who is a "smart" or "good" student and who is not.
• Students need to believe the teacher is genuinely seeking their collaboration in assuming responsibility for their own learning.

That's about it.  There's more about how to implement this school-wide.  But that's not my focus now.  There's a chapter on what formative assessment can't do (raise current standardized test scores) and why.  This is a polemic about the current focus on high stakes accountability testing.

One last great Popham quote (from the chapter on Level 2 Formative Assessment)
"None of the assessments functioning as part of the formative assessment process ought to be graded."