Wrong Answer
July 14, 2014 6:56 AM Subscribe

In an era of high-stakes testing, a struggling school made a shocking choice. A look inside the standardized-test cheating scandal in Atlanta.
posted by zeptoweasel (108 comments total) 31 users marked this as a favorite

Unless fifty-eight per cent of students passed the math portion of the test and sixty-seven per cent passed in language arts, the state could shut down the school.

If kids are struggling to learn, the remedy is to close their school. Right. That will solve the problem.

We should not be pretending that this is anything but straight-up class warfare, right here.
posted by mhoye at 7:16 AM on July 14, 2014 [84 favorites]

Missing from this are the dozens of teachers who were harassed or drummed out of teaching in Atlanta for trying to do the right thing.

Pursuing prosecution of Beverly Hall under the RICOH act isn't just talk; her abuse of staff and relentless chasing of bonus checks robbed many young Atlantans of their right to a good education.

Don't blame all the problems on testing. There was real corruption based on cash bonuses in this mess, and children were really hurt as they failed to get the help low test scores may have mandated.
posted by EinAtlanta at 7:25 AM on July 14, 2014 [25 favorites]

This is heartbreaking.

“The people who say poverty is no excuse for low performance are now using teacher accountability as an excuse for doing nothing about poverty.”

This is a devastating article but this is the thing that I think is most important.

To explain the improvement in scores, Hall told the investigators that “an effective teacher three years in a row will completely close the gap between a child born in poverty and a child born to a middle-income family.” This theory, in its earliest form, derives from a study by William L. Sanders, a statistician formerly at the University of Tennessee, but the findings, which have contributed to a nationwide effort to rate teachers rigorously, have been overstated to the point of becoming a myth. According to a recent statement by the American Statistical Association, most studies show that teachers account for between one and fourteen per cent of variability in test scores.

I honestly pity the teachers who felt they had no choice but to cheat. You get what you measure, and what we are measuring is whether or not the teachers figure out how to game the system.
posted by winna at 7:32 AM on July 14, 2014 [21 favorites]

"If they could get a decent education, they wouldn't be poor"

That is so precisely backwards that it hurts.
posted by Dip Flash at 7:33 AM on July 14, 2014 [24 favorites]

Regarding the subhead ("a struggling school made a shocking choice") — (a) it's incorrect: a few teachers, not "the school" made the "shocking choice", and (b) too bad that even the New Yorker is drifting into Upworthy style ("the choice they made will shock you!" would be their version).
posted by beagle at 7:35 AM on July 14, 2014 [8 favorites]

Even if every single special snowflake was a genius and got the best education possible in all worlds someone would still need to deliver your pizza and mow the lawns of the wealthy. And they STILL would be paid for shit.

Pretty soon we're going to have robots and drones for that stuff. And for a lot of other jobs. At some point, we're going to have to agree to let people thrive and survive (and learn) regardless of society's need for them to provide labor.

Or we could just go full-out dystopia.
posted by emjaybee at 7:37 AM on July 14, 2014 [54 favorites]

Tying student funding to the local tax base, then assessing progress against a 'Neutral' standard is basically a Kobayashi Maru scenario for the poor, IMO.
posted by Orb2069 at 7:39 AM on July 14, 2014 [31 favorites]

Also, it's not exactly shocking that given rules where it is impossible to succeed in your environment *without* cheating, that some people will cheat. The shocking bit is that there isn't more.
posted by tavella at 7:40 AM on July 14, 2014 [6 favorites]

Nothing will change with education until you delink the funding from local taxes. Pay for schools at the state level then work on improving it.
posted by blue_beetle at 7:47 AM on July 14, 2014 [26 favorites]

Nothing will change with education until you delink the funding from local taxes. Pay for schools at the state level then work on improving it.

Well, that would work if you really think any of this is about actually funding and improving public schools. The actual agenda at work here is breaking teachers' unions and handing over public education to private corporations.
posted by Thorzdad at 7:52 AM on July 14, 2014 [71 favorites]

emjaybee: Pretty soon we're going to have robots and drones for that stuff. And for a lot of other jobs. At some point, we're going to have to agree to let people thrive and survive (and learn) regardless of society's need for them to provide labor.

People have been doing more with technology since the two industrial revolutions of the 1800s, when people were freed of agricultural jobs and were able to move to the cities to support growing industries that further increased efficiencies. Robots already replaced the auto industry labor. The future arrived a few decades ago, at least, and people are still treated like (and paid) shit.
posted by filthy light thief at 7:55 AM on July 14, 2014 [3 favorites]

> The actual agenda at work here is breaking teachers' unions and handing over public education to private corporations

As we're seeing here in Washington State, where districts "will have to redirect roughly $38 million in federal funding toward private tutoring efforts next year, instead of using the Title I funds to pay for district programs for low-income students."

You get a failing school! And you get a failing school! Failing schools for everyone!
posted by The corpse in the library at 8:01 AM on July 14, 2014 [6 favorites]

Also, it's not exactly shocking that given rules where it is impossible to succeed in your environment *without* cheating, that some people will cheat. The shocking bit is that there isn't more.

This strikes me as being like the way that corporations set quotas and deadlines that cannot be met without ignoring hiring, safety, labor, etc laws/regulations. If no one notices, then profit! If someone notices, then the courts may not care, so profit! If the courts do care, you can always blame it on the plant manager and keep setting impossible quotas.

For schools where the playing field is slanted toward the students, hey, success! Where it isn't, well, get by as best you can, and any missteps will be used as a cudgel on students and teachers alike.
posted by GenjiandProust at 8:05 AM on July 14, 2014 [10 favorites]

At some point, we're going to have to agree to let people thrive and survive (and learn) regardless of society's need for them to provide labor.

Yes, we've reached a point where there are just not enough meaningless jobs to employ everyone. For the next while, we'll see the unemployed be a growing class of scapegoats, until the Establishment surrenders to the notion that people deserve something for ~~nothing~~ existing. If they won't, we're going to have to either scale back automation, enormously increase the numbers of useless overhead trades (such as Branding Consultants), or get ready to deal with food riots and corpses littering the streets.
posted by Kirth Gerson at 8:05 AM on July 14, 2014 [21 favorites]

edgeways: Even if every single special snowflake was a genius and got the best education possible in all worlds someone would still need to deliver your pizza and mow the lawns of the wealthy. And they STILL would be paid for shit.

And people would still get cancer, and cars would still pollute. In this thread, however, we're discussing the standardized-test cheating scandal in Atlanta.
posted by IAmBroom at 8:11 AM on July 14, 2014 [8 favorites]

If the well-oiled machine of teachers who cheated had spent as much thought into teaching as they did into cheating/covering up, their students might have benefited.
posted by Ideefixe at 8:12 AM on July 14, 2014

Ideefixe, that's an easy thing to say, but with this standardized testing nonsense being directly tied to school funding etc, the goal of teachers changes. Instead of actually ensuring their students are learning, they have to prioritize making sure they fill in the right answers on a test. Very different objectives. It's hardly surprising, and damning of the entire system, that cheating seemed like the best solution to an impossible problem.

I'm not absolving them of responsibility for cheating, but when you are functioning in a totally broken and bass-ackwards system, all you ever have is impossible choices.
posted by feckless fecal fear mongering at 8:22 AM on July 14, 2014 [9 favorites]

Ideefixe : If the well-oiled machine of teachers who cheated had spent as much thought into teaching as they did into cheating/covering up, their students might have benefited.

OTOH, if the system didn't encourage a single-minded, easily-manipulated metric above all other reasonable measures success as both a threat and reward for the teachers' entire professional future, maybe we could have had progress. Or at least not gotten worse.
posted by IAmBroom at 8:24 AM on July 14, 2014 [20 favorites]

feckless fecal fear mongering: " Instead of actually ensuring their students are learning, they have to prioritize making sure they fill in the right answers on a test. Very different objectives."

I hear this over and over.

Have we seriously not figured out how to write good tests? Where's the disconnect?
posted by schmod at 8:25 AM on July 14, 2014 [7 favorites]

Where's the disconnect?

Legislatures.
posted by Thorzdad at 8:27 AM on July 14, 2014 [3 favorites]

blue_beetle:
Nothing will change with education until you delink the funding from local taxes. Pay for schools at the state level then work on improving it.

Quoted for motherfucking truth. Perhaps the most insidiously systematic racism imbedded in our government (as well as generally keeping the poor of every race in their place - but nonwhites are of course disproportionately affected.).
posted by IAmBroom at 8:28 AM on July 14, 2014 [8 favorites]

I think we do know how to write good tests that actually demonstrate knowledge and skills learned.

The problem is, passing those kinds of tests means teachers needing smaller classroom sizes, more support staff for special needs students, more support for students at home (incl. nutrition etc).

That means spending money, and we all know that spending public money on education is a really bad way to make sure Republicans get re-elected.
posted by feckless fecal fear mongering at 8:30 AM on July 14, 2014 [7 favorites]

I had a grad school teacher tell me that, here in michigan, they tried that, but the rich towns demanded more funding and were willing to put it forward themselves. And then the state said "Well these schools are supporting themselves" and stopped increasing funding to meet rising costs over the decades.
posted by rebent at 8:30 AM on July 14, 2014 [1 favorite]

Punishing underperforming schools and their students is like taking crutches away from someone who's not hitting their physical therapy goals as fast as you think he should. Cruel and pointless. If a sympathetic PT started faking data so that the patients could keep their crutches..or even get a wheelchair they desperately need...I would feel the same as I do about these teachers.

In the end, the kids lose out (because they need honest evaluations) and the teachers lose out (because they are forced/drafted into a corrupt system). A system in which it's impossible to succeed no matter how hard you try--unless you cheat--is the definition of dysfunctional. The blame for that goes to the people who put that system in place.
posted by emjaybee at 8:34 AM on July 14, 2014 [17 favorites]

If the well-oiled machine of teachers who cheated had spent as much thought into teaching as they did into cheating/covering up, their students might have benefited.

As I quoted above:

According to a recent statement by the American Statistical Association, most studies show that teachers account for between one and fourteen per cent of variability in test scores.

It is poverty that is the problem.
posted by winna at 8:35 AM on July 14, 2014 [5 favorites]

There's something really sad about the fact that people are still pushing "If they could get a decent education, they wouldn't be poor" when there are so many people who went to college, grad school, law school, struggling to find a decent job. Even if all these kids were getting taught by the best teachers in the US -- even if they didn't have the learning issues caused by home life problems, nutrition problems, and so on -- the jobs wouldn't magically appear. And people keep chasing after test scores as if that's going to do anything to fix it.

I know there are tons of annecdotes out there (and in here) and unemployment and underemployment are certainly still not where they should be, but there are still clear correlations between educational level attained and income and education level and unemployment.
posted by The 10th Regiment of Foot at 8:37 AM on July 14, 2014 [2 favorites]

winna: It is poverty that is the problem.

And parenting, because poverty is a multigenerational curse. And societal pressures and expectations. And nutrition - children who don't get breakfast do much poorer.

It's complicated, is the answer. But it starts with behaving as though every child in this country were important to the future of this country.
posted by IAmBroom at 8:38 AM on July 14, 2014 [1 favorite]

The school my daughter goes to was getting extra money since it was underperforming. Now that they are doing better, guess what, they are taking away the very money that helped them perform better. stupidity.
posted by bottlebrushtree at 8:38 AM on July 14, 2014 [7 favorites]

Where's the disconnect?

Legislatures.

Yup.
posted by Kirth Gerson at 8:38 AM on July 14, 2014

I had a grad school teacher tell me that, here in michigan, they tried that, but the rich towns demanded more funding and were willing to put it forward themselves.

The solution to the problem of better funding for richer neighborhoods is harder to solve than merely decoupling school funding from local property taxes (although that's obviously step one). I live in the District of Columbia: it's one jurisdiction, school funding is the same, and theoretically there should be the same funding for every school, but you wouldn't know that if you looked at schools in the richer, whiter parts of Northwest versus poorer minority neighborhoods. The reason? PTAs. Parents of means will donate large sums of money to the PTA, who then spends it to improve the school. It's cheaper than private school, and their kids get the education they would have to pay double or triple their donation for if they left the public schools. Donations pay for new playgrounds, school remodeling, even specialist positions that wouldn't exist otherwise. Rich kids are always going to have nice schools.

The solution is almost certainly to fund poor performing schools at higher levels than other schools, but merely by thinking that sentence I've rendered myself unelectable, so here we are.
posted by Bulgaroktonos at 8:40 AM on July 14, 2014 [29 favorites]

If we decided that students should be provided for based on need, we would find that poorer districts would have a lot more kids who needed tutoring, assistance with behavioral issues, supplemental nutrition available at school and family counseling and support and would permanently need higher levels of funding.

If we really started being accountable to facing the effects poverty has on childrens learning and paying for the support services to meet those extra needs, maybe people would wise up and start seeing addressing poverty itself, directly, with raising minimum wage, paying for public works and services that result in more job creation, and supporting single and disabled parents homes with supplemental income and services to help single parents work part time and do the parenting and cooking/cleaning etc themselves or have assistance from others with some of that labor instead of their children simply being home alone, roaming around unsupervised, eating an easy cheap unhealthy diet etc.
posted by xarnop at 8:44 AM on July 14, 2014 [7 favorites]

According to a recent statement by the American Statistical Association, most studies show that teachers account for between one and fourteen per cent of variability in test scores.

I'm buggered if I know what the best approach to ensuring that kids get a decent education is, but I do find myself troubled by the reflexive "TEACHERS CAN DO NO WRONG!!" attitude that immediately crops up on the left on this issue. Because if teacher quality is unimportant and it is impossible to measure what makes one teacher good and another poor then why should teachers receive any training at all and be paid more than minimum wage? I mean, if it is rigorously impossible to bring any measure into a classroom that demonstrates that teacher A is doing a satisfactory job and teacher B is not, then it's clearly pointless hiring highly trained teacher A at $60,000 p.a. rather than untrained teacher B at $25,000 p.a., right? And, in fact, by the same token, it's clearly pointless hiring two teachers to teach two classes of 30 kids each when it is impossible to demonstrate that that is in any way to the students' benefit: so we should hire one person to teach a class of 60. Or maybe 90. Or maybe 120? I mean, if no measure of student performance or student improvement is credible there can be no basis for the argument that a class of 30 is better than a class of 120, right?
posted by yoink at 8:46 AM on July 14, 2014 [11 favorites]

But it starts with behaving as though every child in this country were important to the future of this country

Yeah but that means you have to give money to the poors who don't deserve it. Sigh. I wish that were a joke, or even ironic, and not stuff I've actually heard and read.

The school my daughter goes to was getting extra money since it was underperforming. Now that they are doing better, guess what, they are taking away the very money that helped them perform better. stupidity.

This is more or less the same reasoning behind anti-vaxxers saying "Well measles isn't a problem anymore!"

The USA has, indeed, been educated stupid. For at least three or four decades now, and on purpose. Partly due to racism, partly because less educated people are less likely to think critically about who they vote for, meaning we end up with all these people voting directly against their interests when they tick off whoever has an R next to their name on a ballot. Which starts the whole thing over again.

Not saying that schools in Canada are perfect, not by a long shot. Yes, we have private schools for the wealthy. Those will always exist everywhere I suspect. But for the most part (except on reservations, but that's slowly changing) there is a robust public school system, that (for the most part) doesn't rely on standardized testing, and (I'm pretty sure I'm correct here) standardized testing isn't linked to school funding.
posted by feckless fecal fear mongering at 8:47 AM on July 14, 2014 [2 favorites]

Someone smarter than me can refute your arguments, yoink. I'd rather just use a match for all the strawpeople you just made.
posted by feckless fecal fear mongering at 8:48 AM on July 14, 2014 [5 favorites]

I'm buggered if I know what the best approach to ensuring that kids get a decent education is, but I do find myself troubled by the reflexive "TEACHERS CAN DO NO WRONG!!" attitude that immediately crops up on the left on this issue.

Is that what I said?

Wow I just thought I was pushing back against the notion that teachers are magic.
posted by winna at 8:51 AM on July 14, 2014 [6 favorites]

> Nothing will change with education until you delink the funding from local taxes. Pay for schools at the state level then work on improving it

They do that in Vermont, with Act 60. I don't live there any more and don't know how well its worked out. Do local PTAs throw everything off?
posted by The corpse in the library at 8:51 AM on July 14, 2014 [2 favorites]

I began teaching in 1990, first in the southeast and into the midwest, then Colorado and finally California. Always in urban, low resource schools. I saw lots of reform and response to immigration. Some of it amazing, like the Algebra Project. Some of it not. This will sound far-fetched, but not until coming across Dr. Suess' posthumous work Hooray for Diffendoofer Day!, 1998, did I have a key to what agenda had drafted Bush's reforms.

Recall that NCLB was accompanied by Charter School reform and Vouchers. The policy was "comprehensive" and its architecture is one of MBAs and neoconservative policy-- a supposedly pragmatic solution to the collapse of Great Society education.
The testing was to shatter concentrations of poverty, Charters were to compromise unions and vouchers to "tinder" a marvelous transformation.
Geisel's last work addressed a dystopian future in education and there is an explanation of how its drafts remained hidden for years in a desk after his death in 1991.

That's my "understanding" and the details are numerous and sickening. Bush's cousins made money from funding cut loose for tutoring and classroom tech. I was yelling about that in 2004. The businesses of textbooks and testing were promulgating the terms "gold standard" and "data driven" with more assertions and lingo than evidence.

We know how classes learn-- it's the ratio. That's what you see in privates-- sufficient attention to an individual. But education is a business to be maximized.

The latest news on Charters was that around 50% are successful. That makes sense for about any enterprise. Vouchers never got off the ground except for a few cities. Likely a good thing. But hospitals have been turned over to business. Schools will be too, I guess.

But the issue of testing is the only "prong" of a comprehensive agenda now contested. Merit pay is a sickening idea. NCLB itself was discredited as utopian in its ends. The numbers are simply unattainable.

I've lived long enough to see a cycle around politics and education. The challenges of education policy come to the fore only between periods of unrest in foreign policy.
posted by lazycomputerkids at 8:54 AM on July 14, 2014 [17 favorites]

Yeah, I think in some areas, PTAs are a big part of the divide between rich and poor schools. I pass a school every day in the wealthy neighborhood where I work that is always advertising events like SoulCycle fundraisers (Donate $90 to ride a bike for an hour!) to benefit the PTA. Meanwhile the schools in the neighborhood where I live are doing, what, bake sales? Even if they were able to host a big, $50+ per donation fundraiser, you'd have to have enough people living in the neighborhood who are willing and able to donate that much money, and that's just not the case. I don't know what the solution is. I do think people should have the right to donate to improve their local schools, but as long as neighborhoods remain so heavily segregated, students living in poverty will continue going to schools that don't have as many resources.
posted by matcha action at 8:56 AM on July 14, 2014

pushing back against the notion that teachers are magic

The issue isn't whether or not they're "magic"--the issue is whether or not we can significantly improve children's education by imposing measures on teacher quality and rewarding teachers of demonstrably higher quality. If the answer to that is "no" then the very clear conclusion is that teachers are wildly overpaid and overtrained. If it is impossible to improve student outcomes by improving teacher quality then there is no reason not to use the lowest quality teachers available. Otherwise you might as well be spending the school budget on solid gold desks for the students on a faith-based belief that the higher the quality of the desk the better the education the student will receive.
posted by yoink at 8:59 AM on July 14, 2014 [1 favorite]

Have we seriously not figured out how to write good tests? Where's the disconnect?

The disconnect is that high-stakes testing is not about student performance. It's about punishing schools and teachers and students (mostly in poorer school districts). The tests are more about shifting blame that encouraging success.

I work in higher education, and the high-stakes testing environment seems pretty well designed to create incoming classes lacking critical thinking, intellectual curiosity, and good study habits while not really giving them a solid foundation in math, writing, etc. Maybe every generation feels this way, but it's disturbing.

I do find myself troubled by the reflexive "TEACHERS CAN DO NO WRONG!!" attitude that immediately crops up on the left on this issue.

Honestly, I think it's more that the left notices that NCLB is closely attached to the privatization of primary education and the transfer of public funds to commercial institutions while simultaneously attacking unions. Sure, there are teachers who "do wrong," but NCLB is not actually designed to identify teachers "doing wrong" or to help students, teachers, and schools do better. It's all about bludgeoning down costs and transferring public funds.
posted by GenjiandProust at 9:00 AM on July 14, 2014 [17 favorites]

I think there's a simple solution. Money donated via PTA to schools gets pooled and redistributed to all schools in the district. But that's socialist or something.

yoink, are you being disingenuous or what? Teacher quality can be measured, and is done so all over the place. Teacher quality matters. What matters also are a) nutrition for children, b) home support for education, c) wealth inequality, d) much lower teacher:student ratios.

Even a mediocre teacher with a class of 15-20 students is going to have better outcomes than a really good teacher with 30-40 students. Focus on each student really, really matters. And it's the great teachers that try to do this no matter what class size they're handed, which often leads to burnout and either leaving education entirely or moving to private schools with lower classroom sizes.
posted by feckless fecal fear mongering at 9:04 AM on July 14, 2014 [9 favorites]

EinAtlanta: Pursuing prosecution of Beverly Hall under the RICOH act isn't just talk

It may seem harsh but it'll deter copier-cat crimes. Let's wait until all of the fax are in.
posted by dr_dank at 9:06 AM on July 14, 2014 [8 favorites]

Or maybe, yoink, the issue is that while there are some gains to be had in improving the quality of teachers, we would see much greater gains by actually focusing on the social issues that have a much greater impact on the education of children. Not to mention that a lot of our issues with teachers are thanks to cultural nonsense that makes teaching very unattractive.
posted by NoxAeternum at 9:06 AM on July 14, 2014 [1 favorite]

(re: burnout.. speaking from direct experience of teachers I've known in a personal sense; my best friend's mother was a teacher who busted her ass to focus on every student. She too burned out eventually and went into consulting and research. Anecdata but it happens all the time.)

Actually, yoink, you can replace everything I've said to you with "You are using the arguments used to break teachers' unions."
posted by feckless fecal fear mongering at 9:07 AM on July 14, 2014 [2 favorites]

yoink, are you being disingenuous or what? Teacher quality can be measured, and is done so all over the place. Teacher quality matters

Then I would argue that the teachers unions and their allies would be on much stronger political grounds if their position was "you are using the wrong measures of teacher quality and here are the alternative measures you should be using and we will support your use of them" than the current position which is, essentially, that teaching is a Great Mystery and that no one can possibly know what works and what does not. Which is why teachers unions insist, all over the country, that pay advances solely with seniority.
posted by yoink at 9:08 AM on July 14, 2014 [6 favorites]

And, to that end, may I ask: what do you think would be reliable measures of teacher quality that could safely be used in hiring, retention and promotion decisions?
posted by yoink at 9:09 AM on July 14, 2014 [1 favorite]

Sometimes while it's more economical to support people at the start of them having needs than once they are in crisis, it's even cheaper to just discard everyone who needs help and just hope you can build a police state strong enough to keep them away from everyone else.

Although I think unless you plan to just destroy all those discarded, the police state you need to manage their insanity, criminality, and illness causing trouble to everyone may not actually be all that much more economical as it first seems.

The thing is illnesses are more likely to thrive in poor conditions, so costs for EVERYONE goes up because that's a great place for diseases to adapt and become resilient against other techniques we are using to beat disease (and hygiene, cleanliness, and healthy physical and emotional living conditions are a big part of those exact advances).

On the one hand, I don't think this is about DIVINE justice, because why would divine justice send disease to the most injured and discarded people? It just so happens though, that we all need each other, and we are all better off when everyone is well provided for, healthy, nurtured, supported, and mentored/trained to develop their skills to share their gifts.

When kids are getting cognitive impairments and brain damage from poverty conditions while they are in the womb and early childhood, the children really need occupational therapists to work one on one with them and help them overcome intellectual, behavioral, and physical developmental difficulties they face.

Certain types of learning environments may be better or worse, and certainly I think teaching matters and low teacher/student ratio is critical for some students, but the level of needs poverty creates in children is well beyond what a teacher with 30 students can meet the needs of in order to help them learn. Not to mention many of these kids lack bonding and emotional attachments with familial adults that can not be duplicated in a professional environment that would be worth assisting their families with accounting for (which includes allowing parents to have time off to heal from being overworked and the trauma of their own experiences and to actually be with their children instead of working two jobs to still not be able to afford a healthy quality of life).
posted by xarnop at 9:09 AM on July 14, 2014 [3 favorites]

yoink, from the review paper cited in the article [pdf] (emphasis mine):

Research on VAMs has been fairly consistent that aspects of educational effectiveness that are measurable and within teacher control represent a small part of the total variation in student test scores or growth; most estimates in the literature attribute between 1% and 14% of the total variability to teachers. This is not saying that teachers have little effect on students, but that variation among teachers accounts for a small part of the variation in scores. The majority of the variation in test scores is attributable to factors outside of the teacher’s control such as student and family background, poverty, curriculum, and unmeasured influences.

In other words, it's not that teachers aren't doing anything useful, but good teachers can't overcome things like high poverty rates and parental neglect, the impossible situation in which many American teachers seem to find themselves.

The issue isn't whether or not they're "magic"--the issue is whether or not we can significantly improve children's education by imposing measures on teacher quality and rewarding teachers of demonstrably higher quality. If the answer to that is "no" then the very clear conclusion is that teachers are wildly overpaid and overtrained.

No. You are committing a statistical error. If teachers right now are held to a certain baseline of competence and training, that will tend to reduce the variance explained by teachers. After removing that baseline, the variance explained could go up significantly. If you take a population of plants where sunlight availability is sufficient and similar, sunlight will seem to contribute very little to the variance in the health of the plants. But if you starve half of them by putting them in the dark, suddenly sunlight will emerge as a highly-significant covariate.

(On top of that, it may be that teacher quality metrics are flawed, or that there's too much variability in the per-teacher estimates to use them to reliably rank teachers, two issues the linked article brings up.)
posted by en forme de poire at 9:11 AM on July 14, 2014 [29 favorites]

Every political issue has an emotional key that can be used to access people's less reasonable, emotional responses to it. In the case of education, that key is the image of the bad teacher because merely through statistical inevitability (assuming some bell curve distribution of quality to poor teachers) we have all had experiences with very bad teachers at some point in our educations. People make their judgments based on personal biases, so if you can put a human face (literally) on a problem, you get more political traction. Everyone remembers that one teacher they hated so much. Everyone blames Bush and Obama personally even for choices thousands of other people encouraged or went along with that led to those choices.

In reality, the only general problem with the population of teachers is that they are human beings and there is always going to be considerable variation among the population of teachers in terms of teaching style and competence.

As much as we'd prefer to live in a fantasy world where we can excise the humanity portion from humanity's problems, we're never going to get that.

That's why the focus on weak teachers as the be-all end-all of the quality of education seems particularly misplaced to some of us "on the left."
posted by saulgoodman at 9:16 AM on July 14, 2014 [5 favorites]

The whole approach of universal testing against the entire curriculum for a grade level is deeply flawed as a means of testing school and teacher performance. It also fails to provide a meaningful benefit to the student or parents at an individual level. A better approach would be to sample student work product at regular intervals and subject it to an independent review.
posted by humanfont at 9:18 AM on July 14, 2014 [2 favorites]

The actual agenda at work here is breaking teachers' unions and handing over public education to private corporations.

Quoted for truth. As a teacher for 19 years, it is perfectly clear to me that public, secular education has been under active, large-scale attack my entire career. It's also now matriculated up to public higher education.

(I don't think disproportionately hurting students from poor homes is actually an explicit goal, just a nice by-product for those driving this, because fuck the poor, they won't be able to pay for anything anyway.)
posted by LooseFilter at 9:19 AM on July 14, 2014 [19 favorites]

What do you think would be reliable measures of teacher quality that could safely be used in hiring, retention and promotion decisions?

the only jobs for which there are reliable measures of quality are ones that are so routine and repetitive that the statistics are the same as the failure rate for a manufacturing line... which, unsurprisingly, is where the whole idea of statistically objective quality control comes from.

so, you can see that, in the end, thanks to objective metrics and a focus on quarterly performance, the Atlanta schools work exactly like American big business.
posted by ennui.bz at 9:21 AM on July 14, 2014 [5 favorites]

If teachers right now are held to a certain baseline of competence and training, that will tend to reduce the variance explained by teachers.

Sure. But I appeal to your own experience as a student in high school, and that of other Mefites. Did every single one of your teachers seem to be held "to a certain baseline of competence"? I didn't go to school in the US, but I went in a Western country with a highly regarded education system. And I certainly encountered some stunningly good teachers in my time. But I also encountered teachers who were, quite literally, wasting everybody's time. I had a teacher who sat knitting at her desk all through class and chatting with a small group of students about things utterly unrelated to the class subject while the rest of us sat around and played cards. I had teachers who had no understanding of their subject at all and others who were so burnt out they couldn't get through an entire period without bursting into tears and fleeing the classroom. I'm sure many of the horror stories one reads about US high school and elementary school teachers are exaggerated, but I'm equally sure that many of them are true.

If teachers are being punished by having inappropriate or unreliable measures of quality imposed upon them, that of course, is profoundly wrong and needs to be resisted. But so far the central thrust of the teachers' unions resistance is not "let's get the right measures in place" it is "let's not have any measures at all other than seniority."
posted by yoink at 9:21 AM on July 14, 2014 [2 favorites]

blue_beetle: "Nothing will change with education until you delink the funding from local taxes. Pay for schools at the state level then work on improving it."

Honestly, I think the status quo is the closest American public schools will get to equal. Quality childhood education is something the rich are prepared to spend a lot on, and if you come up with a policy that replaces the 'good suburb/bad urban core' school model with one that is 'every school is equal' model, you've widened the quality gap between the best public schools and the average private school, and removed any incentive for the wealthy to give two shits about public education.

And local school quality is a factor in home prices and rent. So if you retain the 'must attend the closest school' policy, you've just raised the rents on everyone who accidentally lives in a newly improved school district, whether or not they have children.

yoink: "The issue isn't whether or not they're "magic"--the issue is whether or not we can significantly improve children's education by imposing measures on teacher quality and rewarding teachers of demonstrably higher quality."

The system, as currently designed, cannot deliver this. This has been repeatedly pointed out by technocrats, and broadly ignored. It's possible to believe that quantitative metrics may improve education quality and that the current method fails to deliver them. You might prefer to choose to read the OP's article as an expose of the current failing system, rather than systems in general. Self-administration of tests is a huge problem, which leads to other problems -- if you're being judged on the year over year improvement and last year's school cheated for their students, you're basically on track to be fired for not cheating yourself!
posted by pwnguin at 9:27 AM on July 14, 2014 [1 favorite]

This article misses the forest for the trees, in part by ignoring the reporting of The Atlanta Journal-Consitution, especially the fact that the cheating is nationwide and "Improbable scores were twice as likely to appear in charter schools as regular schools."
posted by ob1quixote at 9:31 AM on July 14, 2014 [8 favorites]

If the schools were closer to equal, surely school quality would be less of a factor in housing prices?
posted by zeptoweasel at 9:31 AM on July 14, 2014 [4 favorites]

The whole approach of universal testing against the entire curriculum for a grade level is deeply flawed as a means of testing school and teacher performance. It also fails to provide a meaningful benefit to the student or parents at an individual level

It's almost like it's intended to do something else....

No Child Left Behind worked exactly as designed, IM(professional)O. If the framing continues to be "well, some form of universal testing has to be found to assess all students identically and to hold teachers accountable," the battle is already lost because the framing is wrong:

--Bad, tenured teachers were never the problem. As thoroughly summarized upthread, we have lots and lots of data telling us that the real problems are poverty, etc., which are problems at home, not school.*
--Universal assessments, especially in an objective format, will always suck. They are far from our best assessment tools, and NO assessment I've ever seen could work universally.

So now we have Common Core which, while better than NCLB, accepts flawed premises as its foundation, and is also quite problematic as a result.

*-this has been a self-fulfilling perspective: as the "epidemic" of bad teachers (WITH TENURE!!) has been addressed over the past 20 years, teaching has become a less and less desirable profession, so more and more bright young adults avoid it as a vocation (and of those who choose it, over 50% leave the profession permanently in their first five years...and it's not the worst half who leave). (I see this part up close, I teach teachers.)
posted by LooseFilter at 9:32 AM on July 14, 2014 [15 favorites]

I honestly pity the teachers who felt they had no choice but to cheat.

Students would say they also felt no choice but to cheat, but cheating is just a way to keep a facade and lie going on for as long as it can.

That said, poverty sucks...the life out of you. The other day I saw two kids no older than seven begging for spare change in front of a grocery store and then went in, bought a box of donuts and pop, and split both as they used the top of a garbage bin as a table. That is how poor children have to survive. Luxuries such as nabbing that A on your history test takes a backseat to warding off the effects of vitamin deficiency and hustling for your next meal or even place where you can sleep for the night. That is stress that leaves scars -- no stability, no peace of mind, and room for concentration.

That is the problem. The best teachers would have to do find these children financial security to raise those test scores because then they are no longer tax with other problems and can focus. These kids are fighting at a such a tender age for mere survival. Deal with that problem first; otherwise, we are just enabling people who want to ignore the real problems by looking at test scores and thinking everything will be all right.
posted by Alexandra Kitty at 9:40 AM on July 14, 2014 [6 favorites]

I haven't read the New Yorker article yet, but the report produced by the special investigators on the Atlanta scandal back in 2011 is a fascinating read, if you're into primary source documents. One of the points some of the teachers made is that they were evaluated on how students improved on previous years' test scores at different schools, and they were convinced that those prior test scores were themselves bogus.

I think there's a really interesting comparison to be made with the recent VA wait list scandal, in terms of the social dynamics of how these patterns of defrauding performance metrics come about.
posted by yarrow at 9:43 AM on July 14, 2014 [4 favorites]

Hall belonged to a movement of reformers who believed that the values of the marketplace could resuscitate public education. She approached the job like a business executive: she courted philanthropists, set accountability measures, and created performance objectives that were more rigorous than those required by No Child Left Behind

Let's not lose sight of the bottom line here: this woman succeeded entirely at what she set out to do. The numbers improved! Her personal star rose to the highest point in the night sky! Atlanta did so well that it started attracting private equity money (well, Gates Foundation money, which is sort of like a public-sector stand-in). The system to which they attributed their success after the fact (because they needed to credit SOMETHING for the meteoric improvement) is now a legitimate, accepted standard. The audit reported that she hadn't ORDERED her subordinates to cheat, she just created a system where it was obvious you'd be fired if you weren't a teamplayer. Everything went great! Only problem is she did it in a place that doesn't guarantee you a golden parachute, so she might have to lay low for a few years until her name is no longer synonymous with scandal. Then she can get right back to working as an educational reform consultant, at a generous hourly rate.

Everyone wins! Except for those poor black kids in Atlanta whose futures are forfeit, and clearly those are just an unfortunate bit of collateral damage in this grand experiment of how the values of the marketplace shape public policy.
posted by Mayor West at 9:47 AM on July 14, 2014 [4 favorites]

Mayor West: “Then she can get right back to working as an educational reform consultant, at a generous hourly rate. ”

Presuming she lives. As it stands it's six to five and pick 'em whether or not she ever actually stands trial.
posted by ob1quixote at 10:05 AM on July 14, 2014 [1 favorite]

Have we seriously not figured out how to write good tests? Where's the disconnect?

Some standardized tests are better than others, but the better tests require a lot of resources just to create and grade (because they evaluate things like creative and critical thinking, and therefore require students to do more than just bubble in answers, and can't be graded by a machine).

But an intrinsic problem with standardized tests is that *kids* aren't standardized. Aside from issues surrounding poverty or even school "quality," a kid who is just learning English, or who is developing at a rate different from his peers, or who has a disability (either diagnosed or not), or who tends to choke under pressure because that's just his personality or *whatever* is going to "test" idiosyncratically. Culture also isn't standardized, and it's immensely difficult to create a "culturally neutral" standardized test.

If a school is involved in high stakes testing and so a lot of its students are feeling pressure and choking/getting psyched out or a lot of its kids have undiagnosed or still-changing learning or physical differences, or a lot of its kids are immigrants or speak a language other than English at home, or a lot of its kids had poor prenatal/natal care or early education or early nutrition or current food insecurity and so are physically playing catch-up, or a lot of its kids have all those issues and/or more (and a lot of those issues go hand-in-hand), that school is going to have funky test scores. If a school has a lot of kids from populations that the test is *not* culturally "neutral" toward (kids from cultures underrepresented in terms of the makers of the test -- meaning, maybe immigrant kids, minority kids, urban/rural kids, etc etc etc), that school is going to have funky test scores. It's not necessarily because *that* standardized test that the school is using is "bad," it could be just that standardized tests aren't that useful in evaluating what's really going on in a classroom or whether kids are being educated well or in evaluating heterogeneous groups of children at all.

As a culture, we've been edging away from relying on traditional IQ tests for a long time for similar reasons, and instead using other ways of evaluating cognition (such as professional observation, scans, etc). In higher education, we also tend to rely heavily on professor and teacher evaluations rather than on standardized tests (including the SAT and ACT). It seems bizarre to me that when it comes to younger children, who we can expect to be *even more* varied than adults in terms of development since they're growing at all kinds of different rates and exposed to a relatively limited number of things because they're so young anyway, we would rely on standardized tests to evaluate them. I don't actually think it's possible to make a standardized test that would be very useful for evaluating thousands/millions of different kids, when it comes to questions as granular as "are the kids learning the right stuff and enough of it?," and I wish we would stop trying and pour the massive amounts of money and effort that we currently pour into standardized testing into programs that would actually improve children's quality of life.
posted by rue72 at 10:06 AM on July 14, 2014 [14 favorites]

Sure. But I appeal to your own experience as a student in high school, and that of other Mefites. Did every single one of your teachers seem to be held "to a certain baseline of competence"?

You are responding to statistical explanations and reasoned arguments regarding the complex, multivariate, and not merely cumulatively linked factors that affect school and student performance by reciting a bunch of personal anecdotes, noting that you expect many similarly unverifiable anecdotes you've heard must also be true, and then claiming that this reflects substantial, meaningful support for a public policy position.
posted by kewb at 10:10 AM on July 14, 2014 [18 favorites]

I was a teacher, now I'm an administrator and I will say only this:

THIS IS NOT A SHOCKING CHOICE.

Erasing answers and bubbling in new ones HAPPENS ALL THE TIME.
posted by kinetic at 10:21 AM on July 14, 2014 [3 favorites]

Forgive me for responding to a whole bunch of comments in this thread at once:

1. Anytime legislators talk about education, I am amazed at the concept that anyone believes there is one single approach to teaching anything that will work for EVERY student. I teach at the college level; I am a popular teacher, with good evaluations. But even I am not vain enough to think that my approach will work for every student. That's why there are several teachers to choose from for every course. The idea that we can find one successful teacher (say, Jamie Escalante) and then xerox him and have that approach work universally is just idiotic. And yet, that seems to be how legislatures (and often, administrators) think.

2. I am about to turn 50, and I have the golden ticket - a tenure track position (or, at least, what passes for tenure in Florida at the state college system - "continuing contract"). I always thought that was the goal, but now I worry that the college I teach at might not even exist in 15 - 20 years when I am ready to retire; I worry that the state will have let the for profits run amok (Florida doesn't seem to regulate them at all) or else sold the right to run my college to some corporation. I know this is a first world problem, but it's what I think about at 2:00 in the morning.

3. Legislatures, even assuming good faith, don't get "regression to the mean." Even if a school is very successful this year, it's unlikely they can continue to perform at that level year after year after year because of regression to the mean. (Textbook example: Tall parents have children who are taller than average, but not quite as tall as their parents.) But that is exactly what schools get dinged for; the inability to improve each year forever.

I was foolish enough to think that if I went into teaching math, I would be immune from politics (as opposed to teaching English, or History.) I never expected to be dealing with the nationwide attack on the entire idea of public education.
posted by wittgenstein at 10:27 AM on July 14, 2014 [18 favorites]

yoink, you mentioned your high school and your experiences with both excellent teachers and awful ones. Now, for the sake of argument, judge the high school overall. Come up with a single metric that completely encompasses the quality of that school.

Let's say, just for the sake of making the point, that the rating of your high school is 50% on whatever scale we devise. But what does 50% mean? There is a world of difference between a school where all teachers are mediocre (all at 50%), and a school where exactly half are perfect and half are potted plants - the 100% and 0% averaging out to 50%.

Now let's assume that we've got two teachers - Mr. Allen and Ms. Brown. Ms. Brown has a slightly higher rating than Mr. Allen. Is it because Ms. Brown is a better teacher? Did Mr. Allen volunteer to get extra training as an inclusion teacher, and thus his class has many more special-needs students than Ms. Brown? Alternately, let's say Ms. Brown is an art teacher and Mr. Allen is a math teacher. Should their ratings count equally in the school's metric? No? But what if the school is a magnet school for the arts?

The point of this intellectual exercise is to point out that schools are insanely complex organizations that do not lend themselves to being summed up by a single number. Same for teachers. Same for students. And when you accept that multiple measures are needed, you have to start prioritizing them in some way - and with that comes the fact that different people will prioritize differently, and that it genuinely makes sense to prioritize differently depending on the circumstances. And that necessity for prioritizing and understanding circumstances means that somewhere along the line there'll have to be judgment. And the determination of what it means to be a good school - or a good teacher, or a good student - becomes not a single metric or even multiple metrics, but a narrative. Or, perhaps more to the point, an analysis of strengths and weaknesses backed up by many things, only some of which are quantitative - and resting on the judgment of who is doing the judging.

During the years that I was a public school teacher, I saw the beginnings of what was described in the OP in my own district (minus the cheating). My school was a good but imperfect school, and I was a good but imperfect teacher. When the numbers came out one year that showed us not making Adequate Yearly Progress in sixth grade math - I was a sixth grade math teacher - I was not surprised. The previous year I'd had a strong class, so the bar (you always have to improve on last year's scores) was set high. And this year I'd had a class that was overall weaker, and because of a high number of special needs students in that grade my math class became the inclusion class. On paper, it looks like I became a terribly worse teacher from one year to the next. I didn't. I actually became much better, because the close work with the special education teachers really helped me to understand better how to reach a wide range of learners.

And keep in mind that if (under the rules at the time) if one grade level failed to reach the bar, the entire school fails. So we were declared a school that was failing math, and in marched in a parade of folks from the central office to help us improve our math instruction. And the powers that be not only had the mandate to make major changes in our curriculum (including things that were working), but also had the mandate of tearing out other parts of our curriculum (the heart of our social studies program), because math was the perceived problem and by God we were going to do more math.

So you ask, reasonably, if the existing measures don't work, what will? The roundabout answer to the question is that you don't need measures, you need measurers. You, for example, are a good judge of the strengths and weaknesses of those teachers you mentioned, both good and bad. You are not all-seeing, however, and so your determinations may miss some things. A set of evaluators, from the principal to the parents to the central office to a committee of peers is what's needed to evaluate teachers - and yes, the cold reality check of standardized test scores are inarguably one part. It ain't cheap to do it that way, but it's the only thing that will actually work.

TL; DR: You can't machine score critical thinking skills, inspirational teaching, or what makes a "good" school.
posted by Chanther at 10:36 AM on July 14, 2014 [27 favorites]

As a middle aged Canadian I am curious how we avoided all this madness.
posted by srboisvert at 10:52 AM on July 14, 2014 [1 favorite]

I have a story I like to trot out any time people talk about education in the US; I think it's a pretty good summation of why the whole system is almost totally fucked:

Some time ago, my youngest brother went to a very good public school. The school was a model for what public education can be: great outcomes (both in terms of test scores and college admittance), a (relatively) diverse student population, engaged, thoughtful teachers. At the time, my brother had a truly fantastic math teacher. This guy had a PhD in math and years of teaching experience, and he was passionate about the material and his students. Students were learning a ton, parents loved him, the administration liked him. Really a model teacher, all in all.

One year, testing rolled around, and his students' pass rate dropped. Not a ton, and largely because scores were super high in general (I think his rate dropped from something like 96% to like 90%), and it was the kind of fluke that just happens some years. But for whatever reason, this drop - be it because of the huge numbers involved, or the general prestige of the school, or what (nobody is really sure why) - was enough to have him formally labelled the 'Worst Teacher In [city]' by the DoE. Of course, the tabloids jumped on this like sharks to blood, and for days he was paraded around the media as being the 'worst teacher in the city', with people who had never so much as laid eyes on him calling for his resignation. To their credit, both his students' parents and the school made it clear that they were on his side 100%, but being painted as a villain on a grand scale pushed him past his tolerance for bullshit. He wound up taking his PhD and becoming a quant at a hedge fund, where he now (by his own admission) 'gets a ton of respect and is paid five times more to work half as hard'.

It was the perfect storm of terrible testing, terrible policy, and a terrible media; and a brilliant example of why it's so damn hard to recruit and retain teaching talent, especially in STEM fields.
posted by Itaxpica at 10:52 AM on July 14, 2014 [32 favorites]

To put this another way, arsenic poisoning would explain very little of the variance in public health in the USA, because arsenic poisoning is rare here. This doesn't mean that arsenic is not dangerous or that we should relax our standards about arsenic contamination: it only means that our existing standards are high enough that it does not represent a public health hazard when compared to, say, the flu or getting hit by cars. Likewise, if most people have only had a few really bad or neglectful teachers in their lives, most of the variance in their test scores will tend to have been driven by other factors. If having really bad and neglectful teachers becomes a more routine experience (and, importantly, uniformly distributed across e.g. income brackets - non-independence between covariates like teacher quality and poverty can also lead to misleading estimates of variance explained), that amount of variance could go up significantly -- until bad teachers become the norm, in which case the amount of variance explained would again go down. (There's also the issue of single vs. repeated exposure to bad teaching: these VAM models are generally focused on estimating teachers' performance in a single given year.)

You therefore simply cannot advocate for a policy of flooding schools with cheap, low-quality teachers based on this single estimate. Those models were not designed to be able to offer evidence about that policy recommendation.
posted by en forme de poire at 10:57 AM on July 14, 2014 [6 favorites]

yoink: Did every single one of your teachers seem to be held "to a certain baseline of competence"? I didn't go to school in the US...

Then why are you even bringing up your experiences with teachers? It's totally irrelevant! We have a totally different certification structure for teachers, administrative system for public schools, etc. You cannot draw conclusions about our teachers from yours.

On a side note, I would suggest that student anecdotes about teacher quality are questionable at best, both from my experiences as a student and my experiences as an instructor. First of all, I used to talk to other students about it a lot, and I know that they often had drastically different opinions about any given instructor than I did. Secondly, I know from my teaching experience that students love easy, fun, or attractive instructors and dislike difficult ones, regardless of how much you actually learn from either.
posted by Mitrovarr at 10:57 AM on July 14, 2014 [7 favorites]

I'm a teacher. This part sticks out for me:

"[Waller] told me that he was offended by the idea that he would cheat in order to get what amounted to five thousand dollars in bonuses. He and other teachers at Parks spent their own money to buy groceries, H.I.V. medications, furniture, and clothes for students and their mothers, and this continued even after he was fired. “It wasn’t because of the money—I can promise you that,” he said."

Doesn't shock me at all, and I believe him. Even with all of the dishonesty and conspiracy and denial here, I totally believe this part. It's the same reason why people still go into teaching despite knowing that the pay will be crap and that the workload will be too much... and yet I'm also not at all surprised to see this conspiracy portrayed as a quest to earn bonuses by people outside the educational system.

I never met a single teacher who ever went into teaching for the money. And I've never met one who would want a raise more than they would want a smaller class size, or better facilities, or better equipment or additional help in the classroom like an instructional aide or an interpreter.

It's never about the fucking money.
posted by scaryblackdeath at 11:06 AM on July 14, 2014 [9 favorites]

On a side note, I would suggest that student anecdotes about teacher quality are questionable at best, both from my experiences as a student and my experiences as an instructor. First of all, I used to talk to other students about it a lot, and I know that they often had drastically different opinions about any given instructor than I did. Secondly, I know from my teaching experience that students love easy, fun, or attractive instructors and dislike difficult ones, regardless of how much you actually learn from either.

Research on reference transactions in libraries shows that people rate wrong information delivered in a polite cheery way much more highly than correct information delivered in a curt or rude way. Social interactions are complex, huh?
posted by GenjiandProust at 11:08 AM on July 14, 2014 [8 favorites]

Just a little note because teachers' unions have come up a couple times here: Georgia is a "right to work" state, and state employees, including teachers, are forbidden from unionizing. The AFT is mentioned in the article because national unions frequently provide advisory and legal support in these situations in these states, but teachers in Georgia have no negotiating power, no collective bargaining, and no strikes. They also (obviously) get paid lower salaries compared with states without these laws and have less protection from firing without cause.

And yet, as scaryblackdeath notes above, they spend their own money helping their students both in the classroom and in their home lives because that's what public school teachers do.
posted by hydropsyche at 11:12 AM on July 14, 2014 [10 favorites]

As a middle aged Canadian I am curious how we avoided all this madness.

Because like most sensible industrialized countries, funding and almost all administration is provided at the provincial level instead of a patchwork of state, local and federal funding sources and administrators.
posted by Talez at 11:19 AM on July 14, 2014 [1 favorite]

Then I would argue that the teachers unions and their allies would be on much stronger political grounds if their position was "you are using the wrong measures of teacher quality and here are the alternative measures you should be using and we will support your use of them" than the current position which is, essentially, that teaching is a Great Mystery and that no one can possibly know what works and what does not.

That's bogus nonsense. There have been very effective tools for evaluating teacher performance for generations--most of them involve occasionally monitoring classroom performance, letting peers and students perform evaluations of teachers, and other less easily standardized and more qualitative metrics. The problem is no one in policy world wants to use or acknowledge the need to use less rigid qualitative metrics because they're using business world thinking and are fixated on using quantitative measures as proxies for quality measures.

Appropriate and meaningful qualitative metrics of teacher performance would really have to consider on a case-by-case, student-by-student basis, the full range of background factors and influences influencing the more easily quantifiable education outcomes (which by the way aren't without their own problems), but that would require way more complex analytic work and administrative overhead than anyone in politics or any where else has the nerve or money for, so we're left with this mangled worst-of-all-worlds situation that misuses half-measures on a high-stakes basis.
posted by saulgoodman at 11:23 AM on July 14, 2014 [12 favorites]

I taught at the smallest public school in Massachusetts. Typical graduating class size was between 17 and 30 kids. The school's size is a blessing and a curse. On the one hand, the teacher:student ratios were terrific, but on the other hand the school was faced with a daunting problem in standardized testing: when 1 kid in a grade scores poorly, that's between a 5.8% to 3.3% increase in the number of kids who failed the MCAS, which was the difference between being a performant school or not.

The principal developed two coping strategies. The first was that he had the most senior math teacher develop a curriculum to teach kids how to game the math portion of the test (which, by the way, is totally doable) and made that class a requirement for all students. The second and far more insidious thing was that after 8th grade MCAS results came back, he contacted all the parents of students who were risky and had a discussion about wouldn't you like your kids to go to the vocational school? Which is pitiable on a number of levels - I saw several kids who in 7th grade I could see the spark of a good engineer but they needed a little more maturity to see the value of doing the up front work in math. Nope, voc school.

Awful.
posted by plinth at 11:26 AM on July 14, 2014 [2 favorites]

So now we have Common Core which, while better than NCLB, accepts flawed premises as its foundation, and is also quite problematic as a result.

And in two years or less, Common Core will be old news and there will be an all new Silver Bullet That Will Save Education. The New Silver Bullet, like Common Core and the other Silver Bullets before it, will require tons of expensive training seminars, and will generate lots of "how to implement" books that teachers and "educators" all over the country will buy, and we'll all jump through the new hoops and tell the emperor how great his clothes look.

Education is no different than the military-industrial complex that Eisenhower warned about. Every two years there's a New Thing, because the New Thing always makes money. And the teachers and the students never know whether they're coming or going, but the students aren't the point of these Silver Bullets. Successful teaching isn't the point. The money is the point.
posted by scaryblackdeath at 11:28 AM on July 14, 2014 [5 favorites]

The point is the money.

Current USA, distilled. It's so very depressing to see the general abandonment of principles--freedom, equality--that has been accelerating in the USA in my lifetime, and has been metastasizing here to Canada (fuck you, Harper).

Education is a basic human right. So is access to food and shelter and a meaningful existence (call it "pursuit of happiness" if you like). But, as with healthcare, the root problems are a) profit motive, and b) poverty.

I can't think of a way to eliminate the first point without some kind of all-out civil war resulting in a totally new country rising phoenix-like from the ashes. And there's no political will to really deal with the second problem.

So you end up with public school teachers living on tuna sandwiches (hyperbolic example) because they're spending so much of their (pitiful) salaries on food and class supplies and HIV medications (!!!!) for students.

Teachers are goddamn saints given the shit they have to put up with and I'm sick and bloody tired of them being scapegoated for everything that legislators do.
posted by feckless fecal fear mongering at 11:33 AM on July 14, 2014 [7 favorites]

Teachers are goddamn saints given the shit they have to put up with and I'm sick and bloody tired of them being scapegoated for everything that legislators do.

Sure. But every voter remembers that one teacher they hated. I remember that algebra teacher who "flunked me" in high school and sucked at his job. And I've resented it forever. So when I hear people complain about teachers, I think about That Guy, and then I figure that most if not all of them are really just like That Guy, right?

And when it's my own kid who is flunking--sorry, who is being flunked by his teachers--and I think about That Guy, then I'm doubly angry. And I'm like, "You know what? I'm a good parent. And my kid is a good kid who doesn't lie to me and who tries hard, because I'm a good parent and I love my kid. So it's obvious that the problem here is the teachers. Fuck those guys."

We cannot legislate good parenting. We cannot legislate our kids into being good kids. But we can sure as hell legislate the teachers and the schools, and since all I have is this hammer and all I see are nails...
posted by scaryblackdeath at 11:39 AM on July 14, 2014 [1 favorite]

Yeah, I know. That's part of why it's all so depressing.
posted by feckless fecal fear mongering at 11:43 AM on July 14, 2014

saulgoodman's comment needs to be required reading before anyone comments on these topics, here or anywhere. Especially this:

The problem is no one in policy world wants to use or acknowledge the need to use less rigid qualitative metrics because they're using business world thinking and are fixated on using quantitative measures as proxies for quality measures.

This mistaken thinking with regard to valuing programs and majors has thoroughly corrupted higher ed, as well. The only reason that the teacher evaluation aspect hasn't also hit professors is because there is a robust and collaborative faculty-admin review process already in place, with lots of eyes on it and many voices in the process.
posted by LooseFilter at 11:50 AM on July 14, 2014 [1 favorite]

The actual agenda at work here is breaking teachers' unions and handing over public education to private corporations.

Honestly, if that's what it takes to get children educated so they can pass fair tests without cheating, that's what it takes.
posted by roomthreeseventeen at 11:59 AM on July 14, 2014

Honestly, if that's what it takes to get children educated so they can pass fair tests without cheating, that's what it takes.

But so that's the point; these aren't fair tests, and given the requirements for continual improvement, they're impossible for schools to pass without cheating. The requirements are reminiscent of nothing so much as the old Soviet Five Year Plans. Saying "well if breaking teachers unions are what's required to pass this test then let's do it" is like saying "well if sending all the wreckers off to Siberia is what's required to meet the production targets, let's do it!" Breaking the unions won't allow schools to meet their targets, because the targets are unmeetable by design.
posted by You Can't Tip a Buick at 12:08 PM on July 14, 2014 [13 favorites]

An ever intensifying urbanization has brought about competing values of social welfare and (relatively) free markets. Or, as another poster wrote-- narratives. (Except when it comes to national, social objectives, so many interests are near monopolies)

What I learned in California was that unions and admin/legislative parties went to court over curriculum (and its metrics) and a thinking to win out (that astonished me) was: Evaluations are compromised by diversity. So the solution was uniformity of curriculum. Teacher evaluations were to include criticism for "divergent" activity. In the case of California, the practical solution was huge contracts with curriculum creators and the elimination of music, art, and physical education.

NCLB is more or less dead now, and thanks to Mr. Gates, Common Core was adopted state by state through two iterations-- really, just set of standards, intended to achieve what California set out to do and make the whole system more uniform. Such reform was attempted in the 1980s as well, but this effort has more supporting resources than the near legalese of the 80s. If you're old enough, you might recall those standards being printed and laminated and put all over the place, throughout the nation.

So a more soft sort of policy assertion is what's going on currently-- after A LOT of sickening waste, of which Atlanta is the tip of an iceberg.

"No excuses" testing (yet another twisted version of 'don't coddle/limit the disadvantaged') was never supported by voucher infused public/charter choice. Now, I don't believe the neoconservative supposition that corporate markets will positively supplement public education and make a better world, but so much money is at stake, this "battle" within the republic will go on for generations.
posted by lazycomputerkids at 12:11 PM on July 14, 2014 [1 favorite]

I know from my teaching experience that students love easy, fun, or attractive instructors and dislike difficult ones, regardless of how much you actually learn from either.

I will freely admit I was in an academic track at an unusually competitive public high school and my experience might be atypical, but - just because students might sometimes seek out easy classes does not mean they would respect those teachers or describe them as "good." My experience was that many students had a pretty solid sense about this, in fact, but actually learning was not what everybody was after.
posted by atoxyl at 12:14 PM on July 14, 2014

The actual agenda at work here is breaking teachers' unions and handing over public education to private corporations.

Honestly, if that's what it takes to get children educated so they can pass fair tests without cheating, that's what it takes.

Once again, the teachers in Atlanta are not unionized. You need to find a different scapegoat for this case.
posted by hydropsyche at 12:18 PM on July 14, 2014 [7 favorites]

My son's school, for example, is one of the oldest and best regarded schools in town, as far as the community's concerned. But they got hit with a C evaluation a few years ago and it was almost catastrophic.

They got the C not because their kids weren't able to pass "a fair test," but because the kids in the current year didn't demonstrate enough improvement as measured against the performance of the kids from the previous year.

The school didn't really have much room for improvement when the school grading measures first started (being an A school with consistently good performance on standardized tests), but in the years after their initial good scores, they started getting graded as a C school for not showing enough improvement over their own previous year's performance, which significantly hurt the school's standing and funding.

This year, with a lot of extra community support, they've managed to get back up to a B, but really, the only reason they had the room to improve in the first place was because they were punished for their success at the beginning. Meanwhile, the economic conditions in the area have dramatically worsened, so you'd expect to see declining performance due to those outside factors even if they schools was still delivering the same quality of service as it always had in the past.

The evaluation system in some states at least almost seems to have been designed to guarantee that eventually even the successful public schools would start to look like failing ones.
posted by saulgoodman at 12:22 PM on July 14, 2014 [29 favorites]

Bing bing bing, saulgoodman. That is exactly the purpose, and the of course the mandated solution is to turn the schools over to corporations.
posted by tavella at 12:28 PM on July 14, 2014 [5 favorites]

I forgot something. Before Gates was funding Common Core initiatives, he liked "for profit" schools, and they were, along with the agenda of closing "under performing schools" through testing, weakening unions with Charters and proposing "cash" vouchers to subsidize corporate enterprises, the comprehensive alternative to Great Society education.

No "for profit" school still operating does so through returns-- they are all still running off their initial grants.
posted by lazycomputerkids at 12:35 PM on July 14, 2014

yoink: But so far the central thrust of the teachers' unions resistance is not "let's get the right measures in place" it is "let's not have any measures at all other than seniority."

Would it be possible for you to produce any support for this statement? It just comes off as the type of assertion that a person who has uncritically absorbed anti-union rhetoric would make. From what I can tell it's been a very long time (if ever) that any teachers unions have dared to make this type of demand. And even from a teacher's point of view, it makes no sense. What teacher wants to inherit a bunch of students who were ruined by a worthless instructor from a prior class? So I think teachers want their fellow teacher to perform up to standards. What they don't want (some of them, anyway) is for their pay to be dependent upon matters that are largely out of their control. Student poverty, parental involvement, school politics, etc.

yoink: Because if teacher quality is unimportant and it is impossible to measure what makes one teacher good and another poor then why should teachers receive any training at all and be paid more than minimum wage?

Others here have already argued that it is a fallacy to conclude that teacher quality is unimportant based on the statistics presented. But I think, further, that few are claiming that it is impossible to measure teacher success. What they are claiming is that it is great deal more complex than measuring student test scores. And that because schools and student bodies vary so widely, finding a method that works across the board is tremendously elusive.

But even if the difference between teachers were irrelevant, that would not justify paying them minimum wage. Think of tractor trailer drivers. The median wage is over $18/hr. because, to begin with, there is a barrier for entry (having a Commercial Driver's License in good standing) and secondly because there are a limited number of people who are willing and able to perform the work. Even if existing drivers were all exactly equally competent, hence interchangeable, you can't replace them with random minimum wage employees because they still have to meet basic standards. It's the same for teachers. Teachers require, as far as I know in most places, require a bachelor's degree and a teaching certificate at a minimum. There are a limited number of people who are able and willing to perform the work even up to minimal standards. (The burnout rate among new teachers is pretty high, almost 50% in five years.) In a world where teachers could be fired at will regardless of tenure, holding on to good performers would be even more of an issue. (Teacher turnover is higher in private schools than in public, unsurprisingly.) Whom are you going to replace them with? Where is this bountiful supply of star teachers just waiting for their chance to break into the unionized public school cabal?
posted by xigxag at 2:09 PM on July 14, 2014 [7 favorites]

schmod: "Have we seriously not figured out how to write good tests? Where's the disconnect?"

Good tests are expensive to grade.

The College Board's AP tests are "good" standardized tests. They are way, way more expensive to grade than fill-in-the-bubble old-style SATs.

matcha action: "Yeah, I think in some areas, PTAs are a big part of the divide between rich and poor schools. ... I do think people should have the right to donate to improve their local schools, but as long as neighborhoods remain so heavily segregated, students living in poverty will continue going to schools that don't have as many resources."

PTAs are literally trivial, budgetarily; the issue is local tax funding based on the local tax base. Wealthy suburban areas have property worth a lot of money, so they have a higher base value to be taxing. I just looked this up and felt literally dizzy when I saw the numbers -- I went to high school in a pretty wealthy public school district, and I'm on the school board in a very poor public district in the same state.

Wealthy District has 4700 students and a yearly operating budget of $100 million. Poor District has 14,000 students and a yearly operating budget of $150 million. Only 1/3 of Poor's budget (or about $50 million) comes from local property taxes; 2/3 comes from state and federal funding (mostly related to our high poverty status). Wealthy District raises NINETY-FIVE PERCENT of its funding locally, with only 1% coming from the state and 2% from the feds. Wealthy District's tax rate is 3.9% of assessed value (which is 1/3 of the actual value of the home, usually). Poor District's tax rate is 4.8% of assessed value.

Unsurprisingly, Wealthy's average home value is $500,000 and rising. Poor's is $87,000 and flat or falling.

Wealthy is spending $21,000 per student enrolled, virtually all of it coming from local sources. Poor is spending $10,500 per student enrolled, only $3500 of it from local sources, and Poor has a substantially higher tax rate to even get that $3500.

Compared to funding inequality like that -- where even with state and federal poverty funding, the poor school has HALF the per-student resources as the rich one -- PTA fundraising is completely trivial.

yoink: "And, to that end, may I ask: what do you think would be reliable measures of teacher quality that could safely be used in hiring, retention and promotion decisions?"

Given that you have teachers working across ages 3 to 21, working as generalists and subject specialists, in special ed and gifted, in math and gym and language arts and foreign languages, before we even introduce the inherent variability of the students they work with, I would not expect that ONE measure would be effective for all of them.

We're actually a very data-focused district and find some standardized testing data helpful to us in assessing how our teachers are doing and where there are problems. But data is the START of the story for teacher effectiveness; it can identify (some) problems but it can't tell you what's causing them or how to fix them. That's what the professional expertise of the teachers and other educational staff is for.
posted by Eyebrows McGee at 3:06 PM on July 14, 2014 [19 favorites]

zeptoweasel: "If the schools were closer to equal, surely school quality would be less of a factor in housing prices?"

This works both ways -- it reduces premiums, and lowers discounts. If you magically fix urban schools, the fact that they're in a bad district no longer depresses prices.
posted by pwnguin at 3:15 PM on July 14, 2014

Even though Atlanta is a single school system with a single tax-base which should be distributed equally throughout the city, the schools in the mostly white and all wealthy Druid Hills and Virginia Highland look really different from the schools in the Southwest neighborhood of Pittsburgh like the one profiled in the article (which has since closed). Whether that is due to PTA funding or simply evil, I do not know.
posted by hydropsyche at 5:50 PM on July 14, 2014 [2 favorites]

Compared to funding inequality like that -- where even with state and federal poverty funding, the poor school has HALF the per-student resources as the rich one -- PTA fundraising is completely trivial.

I am sure you know the numbers better than I do, but that's surprising to me, given how much difference there seems to me to be between schools in the same district in my city. My frame of reference is urban schools, not suburban ones, and even schools a couple of miles apart and even in the same district seem extremely far apart when it comes to resources. I am trying to look up data but all the numbers I can find lump together all the NYC schools.

It just makes me sad every time I pass this wealthy school and they have a sign up that asks parents "WHO DO YOU KNOW?" (who can give us money to help us build a fancy greenhouse) and I know there are so many schools nearby that don't even have the basics.
posted by matcha action at 6:13 PM on July 14, 2014 [1 favorite]

Whether that is due to PTA funding or simply evil, I do not know.

Anecdotal and possibly bullshit: I interviewed at a public middle school in a very affluent neighborhood in the LA area years ago. The topic of neighborhood wealth came up, and the principal told me that there were very strict legal guidelines on what locals could and could not pay for with donations (borne out of equity concerns). He said that yes, they have a nice scoreboard on their playing field and such, but the local parents would gladly pay out of their own pockets to double the teaching staff if they were allowed.

As a note on that: in every teaching job interview, you'll get one or two questions that reveal the core concerns of a given administration. Sometimes they want to see if you're familiar with current eduspeak. Sometimes they ask about your class management plans. Sometimes they ask about testing. In that interview, the principal asked me how I would handle a parent who got upset and started pressuring me because I gave the parent's child a (well-deserved) B or C rather than an A. He said I would likely hear things like, "My son/daughter is going to Harvard. It's already arranged."
posted by scaryblackdeath at 6:34 PM on July 14, 2014

hydropsyche: “Whether that is due to PTA funding or simply evil, I do not know.”

Almost all financial information for APS is a matter of public record. The FY15 budget is available, although that doesn't appear to be what they sent to the printer like the one for FY14. The FY2013 (and earlier) actuals are searchable online as mandated by the Georgia Transparency in Government Act of 2010. There is also the extremely interesting Financial Deconstruction blog, which provides outside analysis of APS' finances.
posted by ob1quixote at 7:25 PM on July 14, 2014 [1 favorite]

He said I would likely hear things like, "My son/daughter is going to Harvard. It's already arranged."

I'd be all like "grease my palm, beyotch!"
posted by telstar at 6:09 AM on July 15, 2014

Any system where primary motivators are continued success, will inevitably result in gaming of said system to bypass non-ideal results. Example: I know my class is unintelligent. I know that this particular class has 10 or so clowns in it that plan to fill out the Scantron in the shape of an image, for kicks. I don't want to be fired. I juke the stats. Anywhere where reportable stats are used to determined quality of work, on a large scale those stats will be juked.
posted by Zenfoldor at 7:16 AM on July 15, 2014

I live in the APS district. PTA budgets make a massive difference. I'm on my phone at a camp ground but have an article at home I can post later. One ele school has a PTA budget of several hundred thousand dollars and several schools had budgets of under 1000. That extra money brings in teachers, enrichment, equipment, supplies, parties, etc. I'm not sure how anyone could maintain that a large budget wouldn't make a big difference to a school.
posted by pearlybob at 8:30 AM on July 15, 2014 [2 favorites]

“ARRA's Actual Per Pupil Expenditure Data Reveals Inequities in School Funding,” Jennifer Cohen Kabaker, New America Foundation Ed Money Watch, 7 December 2011

Comparability of State and Local Expenditures Among Schools Within Districts: A Report From the Study of School-Level Expenditures [PDF], Ruth Heuer & Stephanie Stullich, U.S. Department of Education, 2011
posted by ob1quixote at 8:30 AM on July 15, 2014 [1 favorite]

pearlybob: “ I'm on my phone at a camp ground but have an article at home I can post later.”

Gotcha covered, pearlybob: “Study: Inequity common in Atlanta Public Schools,” Molly Bloom, The Atlanta Journal-Constitution, 23 June 2014

pearlybob: “I'm not sure how anyone could maintain that a large budget wouldn't make a big difference to a school.”

In theory, the PTA can do a lot, but if the money affects comparability, the district will be in trouble. The way it's supposed to work is that the bargain school systems make for Title I money is that they must spend comparable amounts of local and state funding per pupil, and that the Title I money must supplement and not supplant those funds. A fair amount of time and effort is put into proving it to the DoE, although v.s.
posted by ob1quixote at 9:00 AM on July 15, 2014 [3 favorites]

Just caught this: Down with America's Kid Competition Complex -- Come for the article, stay for the comments (no really. it's worth it just for the chicken picture story alone).
posted by Mchelly at 1:32 PM on July 15, 2014

My kids go to one of the better scoring and funded APS schools - Mary Lin Elementary. I promise it's not evil that results in funding disparity - the neighborhoods donate heavily to the high scoring campuses. Because neighborhood associations also know that house value is correlated to the desirability of the schools a neighborhood is zoned for, each neighborhood association also cuts a generous check to their public schools once a year as well.

The public schools in nicer neighborhoods are nicer because there's more money. More wealth in general around a school also means that parents are likely to have more time to be involved as well. With the tax base of Atlanta shrinking as wealthy suburbs pull away to form their own cities, there's a feeling among public school parents that donating a few hundred - if you can - to the school is only fair if the public school is good enough that you don't have to pay for private.

One good result of this scandal is that more attention is being focused on fund raisers for the lower performing schools. Also a good number of corrupt officials were rolled out of their positions - including Hall. She profited from millions in bonuses, and I want our taxpayer dollars back, in full, and invested in those schools that suffered.
posted by EinAtlanta at 7:18 PM on July 15, 2014 [1 favorite]

Also a good number of corrupt officials were rolled out of their positions - including Hall. She profited from millions in bonuses, and I want our taxpayer dollars back, in full, and invested in those schools that suffered.

I mean, on the one hand, on a knee-jerk lizard-brain level I kind of like the idea of Hall's metaphorical head being placed on a metaphorical spike as a warning to other corrupt school administrators. But on the other hand, corrupt school administration isn't the problem here; the problem is a testing regime designed to dismantle public schooling altogether through a process of demanding increasingly impossible productivity standards.

Like, Hall's a creep, but if one thinks that public schooling is a good thing, and one doesn't adhere to the type of deontological ethics that would forbid you from (to take the canonical example) lying to a murderer about which way their victim ran, then juking the stats is absolutely, positively the right thing to do here.
posted by You Can't Tip a Buick at 9:25 AM on July 17, 2014 [2 favorites]

“Atlanta Public Schools Employees To Receive Pay Increase,” Talk Up APS Blog, 31 July 2014

Atlanta Public Schools employees will see their first pay increase in over five years. The school system has fully implemented a district-wide pay increase plan that was announced by the Atlanta Board of Education in late June.

posted by ob1quixote at 4:36 PM on July 31, 2014 [1 favorite]

« Older Stan Lee and Jack Kirby’s Akira | Happy Birthday, MetaFilter Newer »

This thread has been archived and is closed to new comments

MetaFilter

Wrong Answer
July 14, 2014 6:56 AM Subscribe

Tags

Share

Wrong Answer July 14, 2014 6:56 AM Subscribe

Tags

Share

Wrong Answer
July 14, 2014 6:56 AM Subscribe