West Virginia’s Pilot Evaluation Process for Teachers: a Review

West Virginia has been piloting a new teacher evaluation process over the 2017-18 year. While it has some strong points, it also has problems.

West Virginia’s pilot program for their new teacher evaluation process is now mostly over. Even before the year ended, the state legislature decided to expand the new evaluation process; so while 25 schools in 12 counties participated in this pilot year for the new evaluation, about 700 schools in all 55 counties will be required to use the new process by 2017.

Strong Points of the New Evaluation Process

The new evaluation process for West Virginia teachers has a number of positive aspects. The teachers are now responsible for the outcome of their evaluation to a greater extent than ever before because the new evaluation is basically an online portfolio. To a great extent, teachers themselves decide what gets discussed in their evaluation conference based on what sorts of evidence they decide to submit. And the process strikes a good balance on the question of how much student data to make part of a teacher’s evaluation.

Pilots

Problem One: Quantifying the Evaluation’s Rubric

Probably the biggest single problem with the new process is the issue of quantification. Teachers are rated by an evaluator (usually their principal) on 14 elements divided under 5 professional standards. The process is rubric based (see the Pilot Evaluation Guide). An evaluator assigns a teacher a rating of unsatisfactory, emerging, accomplished, or distinguished on each of the 14 elements. But the rubric is not always completely clear.

The difficulty involved in quantifying the concepts in the new process becomes obvious when you look at the number of times collaboration is referred to in the evaluation. In order to be distinguished, five of the 14 elements require collaboration with other school staff and six of the 14 elements require collaboration with students. But the rubric lacks descriptors that explain exactly what constitutes collaboration. That means it’s possible for an evaluator at one school to view collaboration between teachers as being largely a process of consultation, where they discuss instructional issues together, while a different principal in the same school district may view collaboration as requiring teachers to plan together, synchronize instruction, share resources, and perhaps use common assessments. A distinguished teacher at the first school then becomes just an accomplished teacher at the second school.

Quantifying collaboration with students provides more profound challenges. When teachers collaborate on an issue, collaboration is a conscious process. They collaborate for professional purposes, motivated by the requirements of their job. Students probably are not going to be fully aware of the idea that they are “collaborating” with their teacher. At the very least, the process of a teacher collaborating with students is a less formal, grayer concept. Collaboration with students is probably going to look much different in kindergarten than in seventh grade. But without descriptors for the rubric to define collaboration with students, about a third of the new evaluation process becomes a warm and fuzzy exercise in administrator discretion.

Perhaps the most obvious example of the difficulty that exists in quantifying some of the demands of the professional standard’s rubric is in professional standard number two. The first element calls for a teacher to demonstrate “adequate knowledge of students’ social, emotional and academic needs, interests, learning styles, cultural heritage, and gender” in order to be an emerging teacher. If that knowledge is “thorough” instead of adequate, the teacher is accomplished. And in order to be distinguished their knowledge on the issue must be “extensive.” The difference between adequate and thorough knowledge, or between thorough and extensive knowledge is in the eye of the beholder – or in this case, the evaluator.

As the new evaluation system is implemented with more and more teachers, its success will depend on maintaining a level of consistency that can probably only be reached if descriptors are added to the rubric.

Problem Two: Access to the Online Tool

Another issue faced by teachers who used the new evaluation system this year has to do with access to the online tools. The online evaluation tool allows teachers to perform self-assessments, upload descriptions of the performance evidence they have under each of the standards, and track the progress of their student learning goals. But that tool was only available inside a school building this year. Teachers couldn’t access the tool at home if they chose to reflect on their own self-assessment in the privacy of their living room. They couldn’t access their online student learning goals from home when they graded student work in their own kitchen. That led to a high level of frustration with the online tool. Unless the online tool becomes more accessible, the new evaluation process seems likely to eventually fail.

Problem Three: Developing Learning Goals

There will need to be much more support in the area of establishing acceptable student learning goals as the new evaluation process expands to cover more teachers. The evaluation’s two student learning goals account for 15% of a teacher’s evaluation. The process of writing goals that were quantifiable and realistic (yet ambitious) was one of the most challenging aspects of the pilot evaluation.

Problem Four: Transparency in Scoring

Finally, at the moment the new evaluation process has a transparency issue that will hopefully be solved with time. The Pilot Evaluation Guide shows how individual portions of the evaluation process are weighted. So, for example, standards one, two, three, and five (which all have three elements) are worth 17.14 out of a possible 100 points in the evaluation process. Standard four (which has two elements) is worth 11.44 points.

What’s not clear is whether points are awarded on an element by element basis or on a standard by standard basis. While one can presume that a teacher gets the full 17.14 points if they rate as distinguished on all three elements of standard one, no one seems to know how many points they get if they are rated accomplished or emerging on all three elements. And no one has said how many points (out of 100) a teacher needs in order to be distinguished or accomplished at the end of the overall process.

Five percent of a teacher’s evaluation score won’t be ready until state test scores come out in August, but every teacher in the state should be able to put a numerical score to their evaluation now and know that they got a certain number of points. They should also know already how far above or below a particular overall rating’s cutoff score those points put them. But no one so far can say, “I have 65 points after my evaluation conference in May, and that will make me an Accomplished Teacher. If my school’s test scores are good I’ll have an overall score of 70 points, and that will make me distinguished.” They should be able to say that already. And if that sort of information doesn’t become available soon, teachers will develop trust issues with the new evaluation.