• Quick note - the problem with Youtube videos not embedding on the forum appears to have been fixed, thanks to ZiprHead. If you do still see problems let me know.

What is the Best Way to Evaluate Teachers?

Fnord

Metasyntactic Variable
F
Joined
Oct 31, 2006
Messages
6,623
I used to think that if the majority of students in a classroom failed some standardized examination, that the teacher should somehow be held accountable. Now, after a series of well-publicized strikes (as reported in the CNN article, "Why America's Teachers are Enraged"), I am no longer certain that teachers are the sole cause of student failure.

Yes, it really does take some of us a little longer to see the obvious, but that is not the point I am trying to make, so please don't bag on me for not seeing it sooner. This quote from the article is what gave me that oh-so necessary "DOING!" moment:

"There are many reasons why students do well or poorly on tests, and teachers felt they were being unfairly blamed when students got low scores, while the crucial role of families and the students themselves was overlooked."
Click to expand...
So let's say that more than half of a fifth-grade class has failed some standardized exam, and their teacher, Mrs. Strawman, is facing removal from her position, in spite of numerous reports of how compassionate, dedicated, motivated, and knowledgeable she is.

What factors, aside from Mrs. Strawman's ability and willingness to teach, could be measured, how can those measurements be made, how should the measurements be reported, and what should be done about the findings?

I would really hate to see yet another another generation of teachers get the shaft while the next go-around of the latest educational theories and their applications fail, especially when the future lives of the students (and their teachers) are at stake.
 
Last edited:
There is no single best way to evaluate teachers; multiple measurements should be used. However, simply looking at pass rates on standardized tests is almost certainly a very bad way to do the evaluation. At a very minimum, one wants to look at what are called "value-added measures." These measure student growth, which gives an adjustment for where the students started off the year.
 
IIRC, home emphasis on education and homework outweighs other factors by orders of magnitude. Once I learned that (and that Asian countries with class size of 40 or more lay waste to the US in quality of student education) I stopped being so concerned about

Oh jeeze, it's the part where Ani is riding the giant tick and Padme is laughing. Oh noe, he fell off and is hurt!

Oh, whew! He's just faking to make her laugh. This relieves me.


Anyway, I stopped caring so much about classroom issues per se. But that works in both directions -- the importance of small classes also evaporates. It's not just about teacher evaluations, or books, money, or even public vs. private, compared to home emphasis.
 
Last edited:
Beerina said:
IIRC, home emphasis on education and homework outweighs other factors by orders of magnitude. Once I learned that (and that Asian countries with class size of 40 or more lay waste to the US in quality of student education) I stopped being so concerned about

Oh jeeze, it's the part where Ani is riding the giant tick and Padme is laughing. Oh noe, he fell off and is hurt!

Oh, whew! He's just faking to make her laugh. This relieves me.


Anyway, I stopped caring so much about class size. But that works in both directions -- the importance of small classes also evaporates. It's not just about teacher evaluations, or books, or much of anything compared to home emphasis.
Click to expand...

it's not often that we agree on issues, but this is one.
i was a teacher for over 25 years.

the value that is placed on education in the home is a extremely important factor in a student's success.
i live in northern alberta, where the oil patch pays big wages. a 17 or 18 year old kid can make 2-5 thousand a month in the patch. in families that are all patch workers, fewer kids graduate. 4-5 thousand a month working on the rigs trumps high school.

my students tended to work at provincial averages. i had fewer failures than some teachers, but that came at the cost of fewer at excellence, since i tended to work more with the kids that had the greatest struggle.

there is no doubt that the home environment is the strongest factor affecting student achievement.
 
Lemme re-ask the question, as stated:

Fnord said:
... What factors, aside from Mrs. Strawman's ability and willingness to teach, could be measured, how can those measurements be made, how should the measurements be reported, and what should be done about the findings?
Click to expand...

How can parent participation be measured?

Can (should) parents be held legally accountable for the failure of their children?

Could the same standards be applied to communities and local governments?

I met with some teachers this morning (six, at church), and they seemed to agree that one of their worst problems occur when parents abdicate all responsibility for their children's education.

So how can the parents be evaluated? Should they be evaluated?
 
Statisticaly the general approach is so called value added.

We can tell how a pupil from a certian background and testing at a certian level will do on average.

So suppose at the start of the year you have a bunch of students from sink estate that have previously been failing badly. Statisticaly we would expect a group of 30 to end the year with 2 students getting a C 5 at D and the rest failing spectacularly.

If a teacher gets them up to 4 Cs and 6 Ds they are sucessful. If they only get 3 Ds they are not.

Equaly if you have a bunch of students from the leafy suburbs out of 30 we might expect 25 As and 5 Bs and we can apply the same kind of test.


Run this over a number of years and you can spot the weaker teachers.

This can admitely have some issues in extreme cases. If you are expected to get 30 straight As it's a bit hard to add value. Equaly it's possible to have classes where added value would probably be "didn't add to their criminal record".
 
geni said:
Statisticaly the general approach is so called value added.

We can tell how a pupil from a certian background and testing at a certian level will do on average.

So suppose at the start of the year you have a bunch of students from sink estate that have previously been failing badly. Statisticaly we would expect a group of 30 to end the year with 2 students getting a C 5 at D and the rest failing spectacularly.
<snip>
Click to expand...

One issue with value added. Suppose in the previous year the teacher was first class. Everyone got one grade better than what was expected. For those students this year, what would be expected with a standard teacher?
a. Carry on with the good grade.
b. Go back to standard. Then what was the point of the previous good year.
c. Somewhere between a and b.
 
You can get surprisingly far by just looking at the difference between a September test and a June test, as for the most part background variables don't change. In this way, if a teacher is handed a group of 5th grade students who start at the fourth grade level and the teacher brings them up to half way through grade 5 achievement, it's recognized that the students advanced one-and-a-half years. And that's a good job even though they still haven't entirely caught up with grade level.
 
rjh01 said:
One issue with value added. Suppose in the previous year the teacher was first class. Everyone got one grade better than what was expected. For those students this year, what would be expected with a standard teacher?
a. Carry on with the good grade.
b. Go back to standard. Then what was the point of the previous good year.
c. Somewhere between a and b.
Click to expand...

Probably C but one of the reasons you look at results over several years is to ajust for such things.
 
Fnord said:
I used to think that if the majority of students in a classroom failed some standardized examination, that the teacher should somehow be held accountable. Now, after a series of well-publicized strikes (as reported in the CNN article, "Why America's Teachers are Enraged"), I am no longer certain that teachers are the sole cause of student failure.

Yes, it really does take some of us a little longer to see the obvious, but that is not the point I am trying to make, so please don't bag on me for not seeing it sooner. This quote from the article is what gave me that oh-so necessary "DOING!" moment:



So let's say that more than half of a fifth-grade class has failed some standardized exam, and their teacher, Mrs. Strawman, is facing removal from her position, in spite of numerous reports of how compassionate, dedicated, motivated, and knowledgeable she is.

What factors, aside from Mrs. Strawman's ability and willingness to teach, could be measured, how can those measurements be made, how should the measurements be reported, and what should be done about the findings?

I would really hate to see yet another another generation of teachers get the shaft while the next go-around of the latest educational theories and their applications fail, especially when the future lives of the students (and their teachers) are at stake.
Click to expand...

There are is an issue in standardized testing itself:

-aggregate scores, which is what NCLB uses, they do not track student progress individually, they lump them together and set an arbitrary standard. So every student could show marked gains in all areas but the aggregate would not reflect that.

Off the top of my head, here are the issues that impact students:

-low SES and the stress of that life, especially when your parents works and has no car
-family culture, especially older sibs and the like
-neighborhood culture
-family chaos, domestic violence, substance abuse and unstable housing
-violence in the neighborhood and family
-frequent moves as a result of other factors
-teen parents (twenty when your baby is in kindergarten)
-absent parents

and my favorite

There are no provisions made for the slow learners, these are the students who are one standard deviation below the mean to two standard deviations below the mean.

So they have an IQ of 85-70 and they get no extra services, they are not learning disabled because their test scores (in calls and home work) reflect their IQ, they are not cognitively impaired (ie retarded) because their IQ is above 70.

So five teen percent of the population will start behind and loose grounds as they age, they will not keep up with their peers, no matter what. But no extra services for them,
 
The problem of evaluating teachers is very technically complex. There is a vast technical literature on this. The use of standardized tests is the most common way to do this right now. There are other methods available, but none appear as objective and are as well understood by statisticians as are the methods that use standardized tests. Other methods are under development and will likely emerge as useful alternatives.

Value-Added Methods (VAM) are one of the ways in which standardized tests can be used. This method is widely used in the USA. The idea behind a VAM-based evaluation is that student's past performance can be used to predict future performance and then compared with their actual performance. In actual practice, it is very hard to predict individual trajectory of scores and most experts will tell you that only school performance can be interpreted meaningfully. This has not stopped policy-makers from using individual VAM scores for their own purposes.

There is no real consensus about how to conduct VAM. Different states do it differently. In Tennessee, the argument has been not to include background social variables in the equation used to predict future performance. Since student's current performance has already been caused by these variables, to enter them in the equation would be to count their effect twice. I believe this has been adopted because there is so much missing data from students who transfer in and out of schools, even from different states, that some way of making predictions for these students had to be devised.
 
geni said:
I wasn't aware that postcodes had a race.
Click to expand...
Ahh ... good point.

Let me elaborate.

If the children inside the 90210 zip code do exceptionally well on standardized tests, and the children inside the 39269 zip code do exceptionally bad on those exact same standardized tests, then could a claim of racial bias be made?
 
Many variables are used as indicators of race. Postal code is used. Another widely used variable is proportion of students in a free lunch program. The prediction of VAM scores is not standardized and there are many different formulas used.
 
Scott Sommers said:
Many variables are used as indicators of race. Postal code is used. Another widely used variable is proportion of students in a free lunch program. The prediction of VAM scores is not standardized and there are many different formulas used.
Click to expand...

Free reduced lunch is a measure of low SES, not always race.
 
Best way to evaluate the teachers is to do it within the school taking in to account the local factors, and the best way to encourage this internal evaluation is school choice.
 
It doesn't matter how they are evaluated until the teacher unions are broken so something can actually be done with these evaluations.
 
NewtonTrino said:
It doesn't matter how they are evaluated until the teacher unions are broken so something can actually be done with these evaluations.
Click to expand...

People can be fired under most contracts, what makes you think they can't?

In fact competency is a core of moste valuations, maybe you should actually go to a school and look inside.
 
bikerdruid said:
there is no doubt that the home environment is the strongest factor affecting student achievement.
Click to expand...
On an order of magnitude. That should be a water-is-wet statement. Yet teachers have been taking a pounding and had their authority shredded in oh the last few decades or so. The USA has done an absolutely horrible job of supporting public schools and surprise, they are in big trouble.


If the children inside the 90210 zip code do exceptionally well on standardized tests, and the children inside the 39269 zip code do exceptionally bad on those exact same standardized tests, then could a claim of racial bias be made?
Click to expand...
Only by race card junkies or similarly assorted morons, who I'm sure will do so no matter what anyway. :rolleyes:Alleged "racial bias" in fact was no small part of why we came up with standardized tests in the first place.
 
You must log in or register to reply here.

Back
Top Bottom