"We got him."

September 2, 2013 10:44 AM

"Maria Ridulph was 7 when she was kidnapped from a street corner in Sycamore, Illinois, on December 3, 1957. Her kidnapping and murder is the nation's oldest cold case to go to trial. It required family members to turn against one of their own and haunted a small town for 55 years. Even now, the case may not be over." CNN: Taken: The Coldest Case Ever Solved

Chapters (This is a special report divided into six sections)
1) A child vanishes. (Linked above.)
2) A trail of women wronged: "He left Sycamore to become a soldier, then a cop. "My DNA is protector," Jack McCullough says. But some women who crossed his path see him as a sexual predator."
3) Bulldogs on the Case: "It's not every day the cops get a tip about a 1957 murder based on something a dying mother said. Tony Rapacz knew this wasn't a whodunit. It was a canweproveit."
4) "That's Him": "Her mother told her: "Remember his face." Half a century later, Kathy Sigman Chapman nailed the man she believes snatched her best friend."
5) The Whole Truth?: "A 55-year-old murder mystery is solved. But was there justice for all?

Additional Reference Material
* 8 questions from armchair detectives about the oldest cold case ever solved
* Cast of Characters (click on each image for more info)
* Timeline
* Evidence (Includes video, photos and documents)
* Chicago Tribune: 1950's Ridulph Case Photos and Witnesses remember anxious night when Maria Ridulph went missing

Other Coverage
* A blog defending Mr. McCullough
* The New York Times: "It was the coldest of cold cases — a crime so old that when the director of the F.B.I. requested regular updates on the investigation, it was J. Edgar Hoover who did the asking."
* CBS 48 Hours Mystery report on the case.
* Jim Fisher True Crime: The Jack McCullough Case: Reasonable Doubt?
* AP Big Story: "Man convicted in 1957 murder of 7-year-old girl"
* Chicago Sun-Times: Former neighbor found guilty of 1957 murder of 7-year-old girl

posted by zarq (85 comments total)

The blog defending Mr. McCullough is apparently run by his son-in-law Casey. (First listing under 'Other Coverage" above.)
posted by zarq at 11:47 AM on September 2, 2013 [1 favorite]

Great post, zarq. Best reporting that I've read on CNN for a long while. McCullough strikes me as a nasty piece of work, and from what I've read I think it's likely that he killed Maria. I hope that the verdict allows her family closure.
posted by arcticseal at 11:50 AM on September 2, 2013 [3 favorites]

Fascinating -- thanks for posting this, zarq.
posted by scody at 11:53 AM on September 2, 2013 [1 favorite]

Great article, and great post - thanks, zarq.

from CNN article: "The Chicago G-man found it 'most peculiar' that such a rigorous investigation had not turned up a suspect. The locals were passing on tips about 'all of their homosexuals, queers and fairies, etc.' when the FBI was looking for 'sex deviants of a different kind,' the supervisor wrote in the pejorative and politically incorrect language of 1957."

Wow - so he's saying that branding homosexuals as "deviants" and lumping them together with pedophiles makes it more difficult to catch actual criminals. Who would've thought?
posted by koeselitz at 12:03 PM on September 2, 2013 [11 favorites]

The last few chapters of the CNN article are super confusing at first because McCullough's defense attorney's last name is apparently McCulloch.
posted by elizardbits at 12:07 PM on September 2, 2013 [4 favorites]

Jim Fisher (3rd to last link) pretty well sums up how I feel about this:

While it is possible, even probable, that Jack McCullough had committed this 54-year-old crime, the state had not met its burden of proof beyond a reasonable doubt. Had the defendant been tried before a jury, I can't imagine a guilty verdict in this case. We can only hope that this man is in fact guilty. But that is not how our criminal justice system is supposed to work.

posted by localroger at 12:19 PM on September 2, 2013 [6 favorites]

I agree that McCullough is a nasty piece of work, but that does not mean that he committed this crime. Jailhouse snitches are less than useless as sources of information. A 54 year old memory is, well, not terribly reliable. I am 31. My memories of when I was seven are incredibly foggy. Most are memories of memories, reconstructions of stories that I told myself to make sense of what I did remember. I cannot imagine just how faulty they will be when I am twice my age.

On preview, what localroger said.
posted by Hactar at 12:20 PM on September 2, 2013

Thanks for this.

A very interesting long read.
posted by TheLittlePrince at 12:35 PM on September 2, 2013

I cannot imagine just how faulty they will be when I am twice my age.

I'm 49, and the news isn't good.

I am appalled that a US court convicted someone on such flimsy evidence. "Probably" isn't supposed to be the standard for a murder conviction.

I'd be much, much happier if instead of just listing other women who went missing when McCullough lived nearby they had actually gone and got some physical evidence from one of those crimes that actually matched McCullough and put him in jail for that. When we start locking people up because the town and the family need closure and not because we have satisfied the standard of proof we're supposed to be using, it's a dark day for everyone.
posted by localroger at 12:42 PM on September 2, 2013 [6 favorites]

I agree with those who said that the standard of proof wasn't met.

At the same time this John Tessier sounds like a truly nasty piece of work and I can't feel any real regret that he's behind bars, because he deserves, not in the legal sense but in terms of natural justice, to be there for all the other stuff he did. His daughter was murdered and there's a homicide investigation underway, so it'll be interesting to see what happens with that.

It's always so frustrating to read about old murders like this that could have been solved if the police had only had the use of the forensic science we have today.
posted by orange swan at 2:04 PM on September 2, 2013

I'm still a bit confused about why they wouldn't have recognised him. This was discussed in the 'eight questions' link, with the note that seven and eight year olds often don't know the eighteen year olds in their neighbourhood by sight, which is entirely true. But it seems John Tessier was fairly notorious in the town and known to other children. I lost track--were the people who talked about him being known to them as a kids significantly older than Maria and Kathy?
posted by hoyland at 2:36 PM on September 2, 2013

When he moved to the town at a much younger age, he was known to other kids of the same age as an odd kid, aka Commando. There's no indication if he was still considered odd at 18 by neighbour kids.
posted by rainy at 2:47 PM on September 2, 2013 [1 favorite]

How odd that he decided to enclose his unused (and damning) train ticket in the cardboard frame of the photo he gave to his girlfriend, rather than destroying it - like he was making sort of confession to her, or deliberately planting a clue for future investigators.
posted by Flashman at 3:13 PM on September 2, 2013 [3 favorites]

he showed military recruiters his "little black book." It contained the names and addresses of girls in Sycamore, as well as their bust and hip measurements.

...Asked whether he ever had sex with children, Tessier admitted being "involved in some sex play" with a younger girl but said it happened years earlier.

What in the hell was going on at that recruiting office?
posted by hobo gitano de queretaro at 3:35 PM on September 2, 2013 [5 favorites]

Several details of the case were strange and somewhat confusing, including, as Flashman said, the fact that the unused train ticket was hidden behind that picture. How would that have happened?
Also, its made clear that John Tessier had a car that he cherished back in 1957, and a witness saw someone driving the car around Sycamore the day of Marie's disappearance. So why would he be calling collect from Rockford for a ride home? It was never explained why he didn't drive himself to Chicago and Rockford, instead of hitchhiking, as he claims now.

I agree also that even if Tessier didn't murder Marie Ridulph, he very likely did rape at least a couple of young girls and definitely produced a lot of child pornography.
Its difficult to have a great deal of sympathy for the man being incarcerated for something he possibly did, when he definitely did a bunch of other abhorrent things which he never got punished for.

In any case, let's just hope justice was served.
posted by Alonzo T. Calm at 3:36 PM on September 2, 2013 [1 favorite]

There's an incredibly strange detail in this case that isn't elaborated on. In part 3, investigators interviewed the suspect's old girlfriend, who he was supposed to meet later that night (but apparently didn't). The investigators asked her if she had a photograph of the suspect from the time of the crime, and apparently she did, and mailed it to them. Then this:

Another break came when he pulled the photo from its cardboard frame. A small, yellowed square of paper fluttered out. It appeared to be a government-issued train ticket. Tessier was supposed to use it to travel to Chicago for his induction physical on December 2. The train would leave from Rockford; there was no passenger train from Sycamore.

It was a one-way ticket, and it didn't appear to have been punched.

Wait just one minute here! So the suspect's old girlfriend just happened to have, 50 years later, a super-incriminating train ticket from the very night of the crime inside some photo frame, which she just happened to find and send to investigators? How did she even get the ticket, why in the world would she have it? And why would she keep it in a photo frame?

This is either one of the most staggering coincidences in crime solving history, or there's something fishy going on. The girlfriend was unaware of it, remember, it was only the investigators who found it. If that historical society hadn't authenticated the ticket, I'd think the cops planted it.

I still kinda think that, actually.
posted by gkhan at 3:38 PM on September 2, 2013 [3 favorites]

Should have pressed that button to show the recent comments, apparently!
posted by gkhan at 3:39 PM on September 2, 2013

Glad some of you enjoyed the articles. Thanks.

arcticseal: " Best reporting that I've read on CNN for a long while. "

Same. Every once in a while they do some truly excellent reporting.

---------

Aaaand I noticed a typo in the post. There are 5 chapters. Not six. Sorry.

I should have used my fingers to count 'em.
posted by zarq at 3:48 PM on September 2, 2013

Yeah, great post, zarq. Fascinating story.
posted by gkhan at 3:54 PM on September 2, 2013

Its difficult to have a great deal of sympathy for the man being incarcerated for something he possibly did, when he definitely did a bunch of other abhorrent things which he never got punished for.

In any case, let's just hope justice was served.

Justice is never served when we compromise its principles for a result we want for emotional reasons.

Tessier should have probably been in jail a long time ago, most likely. But "most likely" is not the standard we use for these things, and he should not be in jail right now for Marie's murder because, no matter how likely it seems, nobody in their right mind would think that the state proved its case in that matter.
posted by localroger at 4:01 PM on September 2, 2013 [3 favorites]

Yeah, you're right, localroger.
What I meant by hoping that justice was served was that I hope he is indeed guilty of the crime he has been convicted of, instead of being yet another erroneously convicted person languishing away in our barbaric penitentiaries.
posted by Alonzo T. Calm at 4:06 PM on September 2, 2013

Yeah Alonzo I get that, but my point is that while it is disturbing that a person who may have been so violent and dangerous operated with impunity for so many years, I find no comfort in the idea that such a monster was fortunately the victim of our stopped-clock justice system happening to be right this time.

As efforts like the Innocence Project have shown using new forensic capabilities our justice system doesn't work all that well even when it works right. When we start deciding it's OK to lock people up because, well, there's a shitton of circumstantial evidence and they're shifty looking and the town and the family need closure, well which monster has eaten more people, Tessier or the justice system convicting innocent people? Both of these things worry me but one worries me quite a bit more.
posted by localroger at 4:21 PM on September 2, 2013 [1 favorite]

I love this sort of long form reporting and found the story fascinating. Thanks for posting!
posted by SweetTeaAndABiscuit at 5:23 PM on September 2, 2013

I am puzzled by that sentence from mom that started the new investigation:

"Those two little girls, and the one that disappeared, John did it. John did it, and you have to tell someone."

What two little girls? Was there any investigation done about that?
posted by TheLittlePrince at 6:07 PM on September 2, 2013

Yikes. This puts me in mind of the scene from The Dead Zone where Johnny happens to touch the serial killer's mother in the course of brushing by her and relizes that she knew and has been protecting him for years.
posted by localroger at 6:13 PM on September 2, 2013

I am puzzled by that sentence from mom that started the new investigation:

"Those two little girls, and the one that disappeared, John did it. John did it, and you have to tell someone."

What two little girls? Was there any investigation done about that?

She's probably referring to Kathy, the other child who was playing with the victim but left to go get her mittens when she was taken.
posted by gkhan at 6:17 PM on September 2, 2013

Had the defendant been tried before a jury, I can't imagine a guilty verdict in this case.

I dunno about that. People don't like child-molesting murderers very much. Accusing someone of that tends to make the jury think ill of the accused. "After all," the question goes, "If the police didn't think he did it, why would he be in the box? And would the police charge him if they didn't have reason to?"

We can talk about "Innocent until proven guilty" until we're blue in the face, but in the practical world of the criminal trial, it's not one of our more convincing legal fictions.

Here's the thing: Prosecutors tend to win more cases than they lose. A whole hell of a lot more if you count guilty pleas. People know this. Prosecutors also tend to bring criminal charges in a minority of the cases that come across their desks. Far too many potential cases, finite prosecutorial and judicial resources. As a result, they tend to try to start with the cases they're most likely to win. People know this too. And finally, by the time we get to a jury trial, there has almost certainly been a judicial determination that the prosecutor has established--at least by a preponderance of the evidence if not more than that--that the defendant is guilty of the charged offense(s). Otherwise the case gets dismissed, the grand jury refuses to bring an indictment, etc. People also know this.

The result is that by the time a criminal trial actually happens, in many cases it's all over but the shouting. The result is that, for all its logic, this state of affairs drastically prejudices defendants, permitting misconduct--innocent and deliberate--to go unchecked in many instances. Particularly among the majority of defendants that can't afford private criminal attorneys.

Indeed, not to denigrate Mr. McColloch in the slightest (or public defenders in general), public defenders' offices don't really have the time or resources they need to adequately defend this sort of thing. The prosecutors spent three years putting this case together prior to McCollough's arrest. McCollough was convicted about fourteen months later. So the public defender had a total of fourteen months to investigate and defend a murder and a rape charge, while the prosecutor had that plus another three years. A private attorney could probably (1) devoted more time to the case during that period than McColloch was likely to have been able to devote, given that he could easily have had a hundred other clients from July 2011 to September 2012, and (2) successfully moved to delay the case pending further investigation by the defense.
posted by valkyryn at 6:27 PM on September 2, 2013

All of that being said, and given the facts as described in the article, I think an appellate court is likely to find some combination of no error and harmless error. The defense identified three things it thought constitute reversible error: the lineup, admitting the mother's deathbed hearsay, and excluding the FBI reports.

The lineup doesn't necessarily need to be admissible. If that's the only thing tying the defendant to the case, then yes, it's essential. But in this case, it simply pointed the police in the right direction. This isn't a Fourth Amendment issue, where an illegal search can jeopardize all related evidence. Even a bad lineup can be irrelevant if it leads to other evidence, as it seems to have here.

The same is basically true of the confession. Did it spark the investigation? Yes. Did the police discover enough to have brought the charges without the confession? Seems that way to me.

As far as keeping out the FBI reports, yes, we're in an odd area of the law, and that might well have been an error. But the prosecutors could probably show that the only independent evidence of the version of the alibi in the reports was the defendant himself. That would do a huge amount to neutralize their significance. It might actually then enable the prosecutor to challenge that account in a number of other ways which might not be beneficial to the defendant.

Take away those three things and you're left with one claim on appeal: verdict contrary to the evidence. Now we're arguing, not that the judge screwed in in handling the evidence, but that he simply evaluated the import of that evidence incorrectly. That's the last resort of the desperate in terms of viability, and I don't see any likelihood of it working here. The verdict may be incorrect, but it's not a blatantly, obviously false interpretation of the available evidence. It would have to be for such an appeal to succeed.

For good or ill, my money says McCollough dies in prison.
posted by valkyryn at 6:35 PM on September 2, 2013 [1 favorite]

For good or ill, my money says McCollough dies in prison.

I tend to think so too, while I'm not a lawyer I know appeals tend not to like to reconsider "facts" established in the course of the trial. And the broader problem, that "facts" involving fifty year old eyewitness testimony and jailhouse snitches should be treated skeptically, are unlikely to be given up by the justice department any time soon.

And this is bad, because he shouldn't be in prison for this crime on these premises. This is a broken system that can't admit it is broken and has no means to fix itself. And frankly that is just as terrifying as the existence of serial killers. There are a lot more judges out there than there are serial killers, and a lot more people in jail than victims of serial killers.
posted by localroger at 6:43 PM on September 2, 2013 [1 favorite]

I feel I'd have been able to convict him on the strength of the evidence presented: the inconsistent stories about whether he was in Rockford or Sycamore; his alleged history of sexual assaults; his attempt to conceal the fact that he had a car. Against all this is the recorded time of the collect call, which doesn't fit well with the estimated time of the abduction.

The only solidly verifiable facts about Tessier from the morning of December 3rd to the morning of December 4th are that he was in Rockford from around 7-7:30 PM in the evening, and that he was also there on the morning of the 4th. Everything else has been contradicted. So where was he? Did he come back to Sycamore at all that evening? If not, why?

Tessier's car is the thing that destroys his story, which is probably why he lied about it. Assuming a car he probably would have gone directly from Chicago to Sycamore. So why was he in Rockford at 7:00 PM? And if he had stopped in Rockford (assuming a car) then why call his family collect, if he didn't need to be picked up? If he stayed overnight in Rockford, why lie about that? Or if he drove to Sycamore after 7:30 PM, why lie about it? It was clearly after the time of the abduction.

The simplest explanation is that Tessier went from Chicago to Sycamore in the morning (where his car was seen) then he went to Rockford, called his parents, met the recruiting officer, then probably stayed in his car overnight. That's the big secret. He lied about it because he was trying to conceal his presence in Sycamore earlier that day. He was in Sycamore hunting for a victim, and all his subsequent lies were to conceal that fact and to conceal the fact that nobody saw him from 7:30 PM on December 3rd to the morning of December 4th.
posted by Joe in Australia at 7:48 PM on September 2, 2013 [3 favorites]

Isn't it odd that he used his real name when talking to the girls?
posted by rainy at 9:11 PM on September 2, 2013 [1 favorite]

I found Tessier's actions on the day suspicious, but the evidence against him is weak, especially regarding the timeline.

The girls were seen playing alone by two different witnesses at 6:02 and 6:05pm.

If Tessier did it, the timeline is:

• Second witness sees them at 6:05pm.
• Tessier shows up, chats with the girls, gives Maria a piggy back ride, Maria runs home to get her doll, comes back, Kathy asks what time it is, runs home, Tessier grabs Maria, gets her to his car, kills her (presumably putting her body in the trunk of his car), and starts driving toward Rockford all in a maximum of 11 minutes.
• Tessier makes a collect call from Rockford at 6:57pm. According to Google Maps, Archie Place in Sycamore to the center of Rockford is 51 minutes. But if he just drives to the outskirts of Rockford, the prosecution thinks it could be done in 40.
• Tessier drives to the Air Force recruiting office, arriving at 7:15pm.

Also, when Kathy asks "Johnny" what time it is, he says 7:00. Of course he could have lied, but he couldn't have known in advance that she was going to ask what time it was. And if he was going to come up with a fake time to support an alibi in the event he killed one of them, why not a fake name?

Finally, two neighbors claimed to hear a child shriek at 7:00 and looked out their window, but saw no one.

Everything outside that time frame makes Tessier looks guilty, but everything from 6-7:30 makes it very hard for him to be the killer.
posted by justkevin at 9:36 PM on September 2, 2013 [1 favorite]

I think the phone call must have been made from what was probably the main Rockford Post Office at the time, 401 South Main Street. It's a substantial building of the right era and it has a third floor, which matches the description of the recruiting office's location.

I see the timeline as being like this:

Tessier returns from Chicago in a bad mood after having been turned down. He hunts for a victim and after a close call at 4:15 PM he finds the two girls shortly after 6:00 PM. Maria runs off to get her doll and Tessier "asked whether [Kathy] wanted to take a walk around the block or go on a trip in a truck, car or bus." Kathy didn't take him up on the offer, but Maria did, after Kathy ran off.

Tessier drives to Rockford and calls his parents from the Post Office, collect. Maria may have been killed by that time; she may have been locked in the trunk; she may just have been intimidated into keeping quiet. Tessier's parents sound calm and he realises that nobody knew what he had done. He probably tells them that he's in Rockford and will return in the morning. There's potentially fifteen to thirty minutes unaccounted for here, but in any event he sees Colonel Liberwitz near the Post Office and (either as a planned alibi or out of panic) asks him about the Recruiting Office. Liberwitz thinks that Tessier looks like a "narcotic" but takes him to Sgt John Froom upstairs from the Post Office. Froom describes Tessier as looking bewildered and acting like a "lost sheep". Tessier claims that he will call his father to come and get him, and leaves.

Tessier returns to his car and kills Maria, if she isn't already dead, and dumps her body. He returns to the Recruiting Office the next morning and talks to Sgt Oswald, making some incriminating statements. He then leaves and returns home to Sycamore.

Tessier doesn't need superhuman powers to do this. Maria may not have been his first victim (speculation based on his interest in the Grimes girl); we know he was already in the right frame of mind because of his approaches to other girls; we know Maria was comfortable with him and may even have consented to come with him. I don't know what the speed limits were then, or if there were any, but the old route on Google Maps is more direct than the new one and country roads are pretty empty at night.
posted by Joe in Australia at 10:55 PM on September 2, 2013 [3 favorites]

Joe in Australia: "Tessier's parents sound calm and he realises that nobody knew what he had done. He probably tells them that he's in Rockford and will return in the morning."

I feel weird getting into this honestly, but - if he told them he was spending the night, that gives his mother no reason to lie to police telling them he's been there the whole night.
posted by koeselitz at 11:31 PM on September 2, 2013

Tessier's siblings didn't see him, and the girl he allegedly dated denies it, and the boy with whom he allegedly hunted for Maria denies it. Surely this implies that he wasn't actually in Sycamore that evening. So why make up a story about his father bringing him home? Probably to explain the phone call and how he actually did get home from Rockford, if he didn't have a car. In hindsight it would probably have been best if he had just just stayed in Rockford and hitched a lift in the morning, but what could he do with his car - which very likely had a dead body in it at the time.
posted by Joe in Australia at 12:53 AM on September 3, 2013

elizardbits, good thing to point out. FYI, it's easy to miss, but there's a link in the right column to a "cast of characters" gallery.

This case has always interested me, as this is within my own stomping grounds -- we have close family friends who live in Sycamore today (but had no connection to it then), I'm closer geographically to Rockford, and I've often driven on US 20 past Woodbine, where her body was found. I am too pressed for time to review this all at once in time to comment substantially, alas.
posted by dhartung at 3:09 AM on September 3, 2013

What in the hell was going on at that recruiting office?

Well, the second question was from the FBI investigators. It's the story that links the two sets of questions. Nevertheless, one of the primary jobs of the military recruiting staff was to screen out undesirables, including "sexual deviants", such as homosexuals (cf. Alice's Restaurant, Stripes, etc.).

In retrospect, we probably sense the possibility that showing the recruiter his little black book was a way of underlining normal sexual tendencies. But we can never really know for sure; men did brag about their conquests more bluntly then. I mean, it was certainly uncommon enough by my time, but I knew a guy who showed one off to me and asked if I was interested in any of them.

This puts me in mind of the scene from The Dead Zone

Frankly, I'm watching Broadchurch at the moment, and there's a lot of resonance there (murder of a child in a small town, and the series does a good job of withholding information from the viewer such that a reviewer who's seen two more episodes than I have can't figure out who the killer might be).

The verdict may be incorrect, but it's not a blatantly, obviously false interpretation of the available evidence. It would have to be for such an appeal to succeed.

Yeah, I have to say that my own brushes with the criminal justice system tell me that more cases are decided on fuzzy, circumstantial evidence than anyone wants to believe. Something of the CSI effect has taken hold, and we can't imagine a case being proven without incontrovertible results from a mass spectrometer or a DNA analysis.
Disclaimer: Right now I am carefully watching the appeal of a guy who mugged me two summers back, on identification and chain of evidence issues.

Tessier's car is the thing that destroys his story, which is probably why he lied about it.

Pretty much, especially since as noted he had to find a way to dump the body, and the place he did so was 3-4 counties away. And we probably also need to presume that his car was, somehow, already outside the dragnet when it started -- or emptied of incriminating evidence.

It's really sad -- there's clearly a family dynamic at work here, right up to and including probable sexual abuse of his own daughter -- eventually bringing her into sex work and a relationship with an abusive and possibly homicidal boyfriend. And then you have the covering, the secrets, the deathbed confession (although, I have to say, she should have written a statement or a letter -- it really would have helped matters).

I can't explain the train ticket -- what a staggering coincidence indeed. But I'm pretty confident it's beyond the capability of 2000s law enforcement to correctly and convincingly fake a 50-year-old train ticket, which wasn't even used at trial. But put together with the photograph and it looks like it may have been because he considered her part of at least one version of his alibi. I'm not clear on the chain of evidence here, and maybe the investigators aren't either -- I can imagine the photo being taken, and developed and delivered perhaps still at the event (this was barely possible in those days, and you see a similar thing in the first James Bond picture a few years later), and somehow he just tucks it in there because he's been carrying it around in his pocket and is worried about its significance.

Having used paper tickets on the descendant commuter rail Metra, I know that the common practice for all of my experience is just to walk away from your ticket, for which the seats have a clip. So if he'd used it it would have almost certainly been left on the train; it isn't a ticket that they punch and you take back. You're to display it. And even if you have an unused/unpunched/obsolete ticket, the vast majority of those will end up in a trash bin at the end of the platform provided for the purpose (although, I don't know, litter was more common so maybe they just all got strewn on the platform and some hapless custodian was charged with sweeping them up) -- main point being that discarding the ticket would have been the normal course of a train ride no matter what interaction with the conductor he had. So it's burning a hole in his pocket and his mind and for some reason he doesn't want to entirely get rid of it, so into the photo frame it goes. How very, very odd.

I find the family-defenders' blog unconvincing due to its aggressive and smarmy tone.
posted by dhartung at 5:32 AM on September 3, 2013 [2 favorites]

I don't think you can read too much into the ticket based on the current Metra ticketing system, which doesn't sell you a ticket to station X on date Y, but between zones A and B within a year of the date of purchase.
posted by hoyland at 6:20 AM on September 3, 2013

There are a lot more judges out there than there are serial killers

Doubtful. If by "serial killer" you mean "disturbed person who kills people in a ritualistic manner and leads the police on a merry chase while making national headlines," then yeah, probably. But if you mean "person who has killed more than one victim," then almost certainly not. Any given criminal court judge will likely sentence a few of those people almost every year. If the average judge spends only ten years on the bench, at two per year that's twenty. For one judge. I think it's probably safe to say that there are more people guilty of multiple homicide than there are judges.

and a lot more people in jail than victims of serial killers.

Well that's certainly true. But only because most of the people in jail aren't there for homicide offenses. So I'm not really sure what your point is.

the broader problem, that "facts" involving fifty year old eyewitness testimony and jailhouse snitches should be treated skeptically, are unlikely to be given up by the justice department any time soon.

Interpreting "justice department" as "prosecutors offices around the country," you're probably right. And I think that's a problem, I grant you.

But here's the thing: there aren't really many alternatives. "Science" is not the answer here, as it doesn't provide nearly the level of certainty that people seem to think that it does, and it's only ever as good as the lab that produces it. Even the FBI isn't immune, to say nothing of the 40,000 convicts in Massachusetts who were convicted in part due to evidence by a lab tech now accused of faking the results. The surveillance state which might possibly be (though I tend to think not) is a bad idea for different reasons.

And make no mistake: there are crimes out there to prosecute! But you're attacking two of the mainstays of evidence in criminal trials, i.e., eyewitness testimony and defendants turning states' evidence. Both have significant problems. I am a criminal defense attorney, so believe me, I know. But take those away and. . . what are we supposed to do? As a society, I mean. We're not allowed to make defendants testify against themselves. Which is good! But seriously, what then? Even the current state of affairs is a far sight better than medieval or early-modern criminal processes, where (1) most crimes went completely unpunished, but (2) anyone even suspected of a crime got an incredibly raw deal even if they were found innocent. The only difference between the medieval "hue and cry" and a lynch mob is that the former was legal.

So really, I'm asking, what are we supposed to do? Just stop prosecuting crimes? Well we're likely to have to think about that simply because we can't afford to pay for all the inmate-days that the criminal justice system generates. But there really are criminals out there, who really do commit acts which everyone agrees are wrong (e.g., assaults, robberies, thefts, etc.). We cannot function as a society if we do not do something significant about that. Lawlessness does not seem real to us here in the US (or the UK, or Australia, or much of Western/Northern Europe) but it is a very real thing in many parts of the world, and it's only been maybe two hundred years since large swaths of our own countries were significantly lawless. What we've got may be problematic, but is it really worse than anarchy?

I tend to think not. Doesn't mean there shouldn't be reforms, but it does put the need for said reforms in context.
posted by valkyryn at 10:34 AM on September 3, 2013 [5 favorites]

"Science" is not the answer here, as it doesn't provide nearly the level of certainty that people seem to think that it does, and it's only ever as good as the lab that produces it.

The problem is that, unlike the fundamental problem of witness memory and snitch self-promotion, science and lab quality can be improved.

I know something about this too because I happen to work for an industry where that's taken seriously. I have to have a license to open the case of a scale which is used to buy and sell. Such scales have to be inspected regularly and are regularly tested by the state. If your scale fails its state test it is red-tagged and you cannot use it until someone who is certified to do so fixes it. Removing that red tag is a crime.

Lately everything is going ISO. We have to have procedures which document where everything comes from, how it's recorded, how it's tracked, and how it's released. After I certify your scale I can follow a paper trail back to the technician, the test weights he used, the last time they were calibrated, and that trail leads all the way back to the Standard Kilogram. Weights and measures are provided for in the Constitution because even in the 18th century people knew commerce would fall apart if they couldn't be trusted.

So where is the equivalent care for these incompetent crime labs? Who randomly checks their methods? Who audits their procedures? I can tell you it is utterly impossible that your truck scale could break that badly and be used for 40,000 transactions before someone noticed. If it did break there would be a clear recent point at which it could be established to be reliable, limiting the number of suspect results.

And I can answer my own question here, since some of our customers are crime labs whose balances we calibrate: For the most part nobody bothers. Quality assurance costs money and states don't have much. This doesn't mean science is somehow on the same plane with asking a 70 year old woman to remember a man's face she saw when she was 7, or forgetting to mention to the jury that a snitch received positive favors for giving up the dirt on someone. It means that science can be done wrong on the cheap. This is a problem that can and should be fixed.

There is also, as it happens, a right and a wrong way to use witness testimony and snitches, and that's to not depend completely on it, and to carefully weigh the factors that influence its reliability. I feel very strongly that this prosecution should not have been brought forward with such meager and unreliable evidence, and that it sets an extremely dangerous precedent to say it's OK because oh dear it's all we got.

Asking for physical evidence and letting a few people go because none exists isn't going to lead to anarchy. The justice system needs to understand that nearly everything science has discovered about the reliability of memory and eyewitness testimony in the last century is at best terrifying.

My wife was the subject of a demonstration when she was in high school. During a lecture two men rushed into the room, had a rapid altercation, and then ran out. The instructor then asked the class to write down what happened; every single student wrote that a black guy chased a white guy into the room and tried to hold him up with a gun, but then the white guy ran away. Then the actors were brought back and showed what really happened: The white guy chased the black guy into the room and tried to hold him up with a banana.

That's eyewitness testimony. You want me to vote for a conviction, at least prove that the gun isn't a banana.
posted by localroger at 12:24 PM on September 3, 2013 [1 favorite]

The problem is that, unlike the fundamental problem of witness memory and snitch self-promotion, science and lab quality can be improved.

Not in a way that would solve the kinds of problems we're talking about here. Even if we did have the money to run the labs the way you want to run them, you've touched on a basic problem with all testimony: trust. Or "credit" to use the language of philosophy of science. Unless the jury is actually doing the forensics analysis themselves, all they're really getting on the witness stand is somebody testifying about stuff.

"But ah," you say, "That's stuff that can be validated and checked!"

Well, sure, in theory. But when you get that into the courtroom, it's just. . . somebody testifying about the other witness's testimony about stuff. As with eyewitness testimony, jurors have to decide whether to credit the testimony of scientific experts or not, and that decision can't be based on any empirical observation or cross-checking of the testimony.

There's your vaunted "physical evidence": it ultimately boils down to whether the jury, who has not performed the scientific procedures in question nor observed the witness who says they did, believes what the expert tells them. It's not "scientific." Not in the slightest. It's someone telling the jury that they did thus-and-such and that X and Y were the results, and they used method Alpha and Beta, which were validated and audited by agencies ABC and PQR, and none of this has any actual empirical validity. It's all just testimony that the jury has to decide to believe or not, because they haven't personally witnessed any of it. For all the jury knows, the guy on the stand is a hobo off the street that defense counsel paid $50 to take a shower and dress in a suit. There's no "scientific" way for them to know anything about anything.

So no. While increasing budgets at forensics labs is really something that needs to happen, it isn't going to do anything about the problems we're talking about here. "Science" as evidence, when it comes right down to it, is no more-or-less epistemically valuable than eyewitness testimony from the perspective of the jury. Courtrooms are not laboratories.
posted by valkyryn at 4:06 PM on September 3, 2013 [1 favorite]

But when you get that into the courtroom, it's just. . . somebody testifying about the other witness's testimony about stuff. As with eyewitness testimony, jurors have to decide whether to credit the testimony of scientific experts or not, and that decision can't be based on any empirical observation or cross-checking of the testimony.

Look, this is just wrong. I have testified twice in the course of doing my job and both times the testimony started with my bona fides. Now you may think that's "just testimony" but no, it's not; my license from the state of Louisiana and my company's license and documentation trail are things which could be challenged. (Nobody bothered because such a challenge would be trivial to refute. There is a state agency out there with records that could easily be produced.) So at a fundamental level if I am testifying about methods of possibly cheating a scale, my testimony is worth more than yours, full stop. In metrology that trust thing you're talking about has a documentation trail.

Where is the comparable documentation trail in the matter of the 70 year old woman trying to match a face she hasn't seen in over 50 years? The very comparison is absurd.

You are talking like a defense attorney who is trying to weasel around the fact that a test has just said something bad about your client. And the fact that the system makes such weaseling possible is both stupid and unnecessary. Particularly in a narrow field like forensic DNA analysis it should be possible to have a licensing and trust verification regime at least as good as what the scale industry, which is much larger and more spread out and rather poorly funded, manages to implement.

It is not an absurd burden to ask that labs be run in such a way that there are multiple independent ID trails, that ID's are not known to test technicians or the people doing the match detection except as the end result of a double-blind comparison. Mostly it's paperwork, and the reason it isn't done is probably that certain people have a vested interest in maintaining the argument the mushiness of these occasionally so embarrassing results.

What happens in a testing lab isn't really "science." They aren't doing experiments to see if some theory of nature works or not. They are doing a repeatable procedure for which those experiments have long already been done. The jury doesn't need to know how the goddamn test works. They need to know that a reputable authority has verified that the test works and is maintaining standards to ensure that the particular test in question was done according to those established procedures and should be trusted. If you are presenting a DNA result to a jury and the words "polymerase chain reaction" pass your lips, you are doing it wrong.

That testimony should go like this: "I am a licensed DNA testing technician employed by *** which is a licensed DNA testing laboratory. Our facilities are inspected at *** intervals by auditing and random checks are conducted of our methodology, which is established by NIST publication ***. According to those procedures, I established a probability of better than 99.4% that these samples in evidence match, which are identified by our audit trail as belonging to the perpetrator and defendant, respectively."

That's pretty much the way it went the two times I testified, except you can substitute weight accuracy for DNA matching. This is absolutely no different than what humans have been doing with weights and measures for 5,000 years. Measurement is always better than an eyewitness account, and measurements can be certified by a chain of evidence. The ancient Egyptians worshipped scales precisely because they were revealers of hidden truth and, properly used, they do not misremember or lie. To argue otherwise is to fundamentally misunderstand not the courtroom, which is at the end of the day part of the world, but the world itself.
posted by localroger at 4:45 PM on September 3, 2013 [3 favorites]

And it bears repeating, forensic science is, quite problematically, not really science in certain fundamental ways that lead it to the level of trust that some would give it.

We also have tremendous societal bias against circumstantial evidence -- such that the very words are a sneer. But circumstantial evidence -- precisely because of the problems with testimonial evidence that localroger notes -- is among the strongest in most criminal cases.

DNA evidence isn't always available, or conclusive (imagine the situation where someone's DNA is found precisely as expected, e.g. in a home they frequent). Many murders and even other crimes are among intimates. Many, many crimes take place without eyewitnesses at all. I think this notion that we're drowning in a sea of bad convictions based on unreliable testimony a bit hard to swallow. We should invalidate testimony that can be demonstrated to be wrong or otherwise countered. But the idea that we can simply wave away the need for eyewitness testimony because, I don't know, the invention of the scale? Absurd. Devices can only measure certain things. They can't substitute for things they can't measure, observe, record, or otherwise serve.
posted by dhartung at 4:51 PM on September 3, 2013 [1 favorite]

Now that I think about it though this conversation reveals an interesting bit about the first time I testified, when one my customers sued our competitor who sold them a system that did not weigh accurately, costing them $50,000+ in overpayments before the error was discovered, and were subsequently unable to fix it.

What the defense attorny did not understand is that in weights and measures there is a very fixed chain of responsibility reaching from the person who operates the scale back to the maintenance of the Standard Kilogram. Our competitor had bought a scale that worked and an automatic feed system that was workable if installed right and had put them in, but they did not understand a basic requirement of the feed system which the scale did not happen to meet (nor did they know how to troubleshoot the problem, which I fixed in half an hour). The defense lawyer tried to pin it on both of the manufacturers involved but I testified that both machines worked as designed. It would be my responsibility as the person who assembled the system and put it into service to make sure it is working correctly.

In the end the jury made a judgement for my customer that put our competitor out of business.

dhartung: Nowhere have I said anything as stupid as that testimony isn't necessary. I have said that it should be taken with a grain of salt (sometimes, as in the OP, a very large grain) and backed up by physical evidence unless there is very strong believable testimony from multiple credible witnesses. That some bad people might go free because there is no evidence is a better result than locking a similar number of innocent people up. It is not "anarchy" to ask that the prosecution meet its burden of proof with a modern understanding that eyewitness testimony has sharp limits.
posted by localroger at 5:01 PM on September 3, 2013

Localroger, I totally agree with you about the nature and quality of scientific evidence. That only goes so far, though: most other forms of evidence are uncertain or subjective and can be hard to quantify, which is why they are left to human judgment. The defense can remind the "trier of fact" (i.e., the jury, or the judge in a non-jury trial) about the dangers of eyewitness identification, but it is up to the trier of fact to decide how much weight to give it. Kathy's identification is worth something, even fifty years later. If you construct a Bayesian equation you'd have to say that it makes Tessier's guilt somewhat more probable. In this case it was one of a number of factors, many of which we can't assess ourselves, that convinced the judge he was guilty.
posted by Joe in Australia at 5:15 PM on September 3, 2013

If you construct a Bayesian equation you'd have to say that it makes Tessier's guilt somewhat more probable. In this case it was one of a number of factors, many of which we can't assess ourselves, that convinced the judge he was guilty.

Included in that Bayesian equation as applied by the judge, I am certain: the family and the community needed "closure," and hey the guy was most likely guilty of something.

That is not, however, the way it is supposed to work. Frankly I would disagree that Kathy's ID is worth anything at this time and place. I realize that other countries fall differently on this but in the US it is most absolutely not supposed to work that way.
posted by localroger at 5:19 PM on September 3, 2013

Once in a while, CNN hits one out of the ballpark. Thanks for the post, zarq!
posted by deborah at 6:53 PM on September 3, 2013 [1 favorite]

No, Bayesian inference is about degrees of certainty and there is literally no room in those equations for factors that mean "I want to get the right answer". Yes, the judge may have given too much weight to her testimony, but it was absolutely definitely positively worth something, even if it was only a minute amount.

It's quite a serious accusation to say that he convicted Tessier because "the guy was most likely guilty of something" or because the family needed closure. Where things like "closure" are supposed to come in is sentencing: the judge in this case was working initially as a trier of fact, and after he had determined Tessier's guilt he presumably applied some guidelines to determine the sentence. But that's after the conviction and Kathy's identification was no longer relevant by that point.
posted by Joe in Australia at 7:01 PM on September 3, 2013

Joe, I agree it's a serious accusation and I'm totally making it. The judge was derelict in his duty. I am in total agreement, as I wrote above, with Jim Fisher that a jury (particularly from another venue) given typical instructions would have been very unlikely to return a guilty verdict on the evidence in this case. I believe hizzoner walked into that courtroom looking for a justification to "do the right thing," where "the right thing" was to put this bad man who had been free too long behind bars.

Unfortunately, making the case that that happened on appeal is an extremely high bar to clear if it can even be raised at all, and so this travesty is probably going to stand.
posted by localroger at 7:12 PM on September 3, 2013

Oh, and Joe, how would you estimate the Bayesian worth of my wife's class of 17 year olds every single one of whom saw a banana as a gun and the black man wielding it instead of the white guy? Sometimes testimony has negative value. It can be worse than worthless.
posted by localroger at 7:18 PM on September 3, 2013

localroger: I agree, absolutely, that Kathy's id is worth almost nothing.

The lineup is almost comical. Here are the differences: 1 photo has something written under it, has dark background, has some white lines where photo seems to had been bent, is much blurrier than the rest, the young man's eyes are focused on the camera, and he is dressed differently, and it is obviously an amateur photo.

The other 5 are studio photos, all boys are looking to the side, background is light, they are equally sharp.

It's hard to imagine that the detectives were naive enough to think it's a fair lineup. I can see how it was tough for them to get similar photos of the boys from that time and place, but it would be immensely more fair if they were just any old photos shot in a similar way.

When Kathy was 7 or 8, her mother constantly telling her to make sure to remember the abductor's face, because she's the only one who saw him that evening. Was that supposed to help her memory or to help her be mistakenly certain of the recollection?

What they definitely did accomplish was go back and look at his alibi and establish that he could have been, might have been the killer. If all of this was turned up in 1957, they would almost certainly prove or disprove his guilt conclusively.
posted by rainy at 7:22 PM on September 3, 2013

rainy, one of the things that has been learned about memory in the last 20 years or so, especially thanks to the Satanic abuse hysteria in the US, is that that kind of pressure will cause you to think you vividly "remember" astonishingly detailed things that completely never happened. One of the most damning things in the lineup story is that Kathy actually struggled for a couple of minutes over a couple of the pictures. Is it that she couldn't tell which of the two boys was the abductor, or was it that it had to be one of them and neither of them quite matched, and she had to convince herself?

Or, even worse, was she worrying that if she got it wrong, the killer (everyone after all knowing he was the guy) might go free?
posted by localroger at 7:34 PM on September 3, 2013

A jury would likely have failed to convict him because, statistically, it is harder to get agreement among a group of people than it is to convince a single person. I speculate that the defense chose to waive a jury hearing because they were afraid that a jury would be swayed by the unsympathetic defendant and the tragic nature of the case. It was a gamble: do you try for a hung jury in a case where the rational arguments are pretty thin, or do you go for a judge and hope that emotional arguments will carry less weight? It's a hard choice.
posted by Joe in Australia at 8:27 PM on September 3, 2013

What the defense attorny did not understand is that in weights and measures there is a very fixed chain of responsibility reaching from the person who operates the scale back to the maintenance of the Standard Kilogram.

That's as may be, and I believe you.

The problem is that for that "chain of responsibility" to actually mean anything in court, you'd have to get every single person responsible for every single step of that chain into the courtroom to testify that yes, they did what they were supposed to do. And the jury would have to weigh their testimony just like any other testimony.

Why? Because the jury isn't personally overseeing that chain of responsibility. They only know what they're told.

And why is that a problem? This is why:

They need to know that a reputable authority has verified that the test works and is maintaining standards to ensure that the particular test in question was done according to those established procedures and should be trusted.

Your argument boils down to "Because I'm a SCIENCE PERSON!" It has to. The "reputable authority" item in your formulation does an incredible amount of work. In your little statement, here's what the jury has to accept based solely on the witness's say so: "verified," "works," "maintaining," "standards," "done according to those established procedures." The jury has no way of verifying any of what independent of the witness telling them beyond "Take my word for it because I'm a SCIENCE PERSON!" In other words, "I don't make mistakes, I have no bias (which wouldn't matter anyway), and you should take what I say at face value."

And you're telling me that that's less problematic than eyewitness testimony?

What's really interesting to me about this is that this is basically exactly the same argument that the first court ever to admit expert testimony made in admitting said testimony. Only rather than saying "Yeah, this looks like real science that has the hallmarks of general acceptance and/or repeatability and peer review," they said "Oh, hey, this guy's a gentleman. Of course he'll testimony honestly."

Nothing has changed. Your argument for why scientific testimony is better than eyewitness testimony is explicitly "Because science people can be trusted." Well f*ck that.
posted by valkyryn at 4:17 AM on September 4, 2013 [2 favorites]

The problem is that for that "chain of responsibility" to actually mean anything in court, you'd have to get every single person responsible for every single step of that chain into the courtroom to testify that yes, they did what they were supposed to do.

This is so wrong I don't even know where to begin.

My argument has to do with how standards are written and applied. When you present fingerprint evidence you don't have to present testimony from everyone involved in the development of matching techniques back to the 19th century, you have a system of standards certification which establishes that the expert you hired should be taken seriously and the chain of evidence maintained by the cops to make sure what he's matching was collected under the stated circumstances. You know how to do this.

In the case of DNA you also have the lab, equipment, and operators which must be trusted. No, you do not get their testimony, that is the stupidest thing I've ever heard. What you get is a standard written which can be generally accepted by the court system. Then you get paperwork, which is presented to the jury if necessary, establishing that the lab has passed testing and conformed to the standards which are generally agreed to produce the results which will be described. Part of that testing is that chains of evidence are preserved. Part of it is more like ISO than metrology, but the methods are there and known to work.

My argument does not boil down to "Because I'm a SCIENCE PERSON." It boils down to "I have been trained, tested, and operate a facility which adheres to these standards which are periodically verified and which are agreed to work by a body of experts whose authority the court accepts." Note that last part. You do not get to nit pick the meaning of a word like "verified" in that statement because the standards body whose authority is accepted by the court has already established what that means and there is a document which spells it out. NIST handbook 44, which specifies the calibration and testing standards for scales, is 100 pages long mainly because of people like you.

I can't legally take the red tag off your scale because I'm a SCIENCE PERSON, I can do that because I have a license in my pocket which says I've met certain standards which everyone has agreed are adequate. Note that word LEGALLY. The courts accept my license as something more than "I'm a SCIENCE PERSON." They've told two juries that too.

This isn't even SCIENCE. It's quality assurance and industry has been doing it for over a century. The whole point of ISO certification is that you don't have to audit the whole chain of evidence to have confidence that you're getting what you expect. We manage to do this to make sure you get an honest gallon of gas and an honest pound of ground beef and that the plastic in your countertop is to spec and won't fall apart in a year. We do it to make sure your car won't crash and your food won't poison you and the bridge you're driving over won't fall down.

That we can't do it to make sure we are sending the right man to prison is baffling.

I am not saying DNA testing is done right now; it obviously isn't. I am saying that it is possible to do it right though. You do not seem to understand that SCIENCE!!! is not a war of words won by whoever can best persuade a group of earnest but naive citizens. What the scale industry has been doing since the 18th century, what industry has been doing internally since the 19th, what international trade organizations have been mutually enforcing since the mid-20th, could be done by the crime lab if anybody cared, and if that were done then yes it would be infinitely more valuable than eyewitness testimony.

Yes, you would still need testimony for when the evidence is lacking or to place it, but you would not need to rely on it so much and could afford to regard it with the skepticism it has proven to warrant.

P.S. In other news, are you really entirely ignorant of the Innocence Project?
posted by localroger at 5:36 AM on September 4, 2013

My argument has to do with how standards are written and applied.

I know it does. My argument is that doesn't matter a damn. Because "standards certification," from the jury's perspective, is little better than someone saying "This is accurate. You can take my word for it."

My argument does not boil down to "Because I'm a SCIENCE PERSON." It boils down to "I have been trained, tested, and operate a facility which adheres to these standards which are periodically verified and which are agreed to work by a body of experts whose authority the court accepts.

This is a distinction without a difference, and you explain why here:

I can't legally take the red tag off your scale because I'm a SCIENCE PERSON, I can do that because I have a license in my pocket which says I've met certain standards which everyone has agreed are adequate. Note that word LEGALLY.

Exactly. You can do what you do, not because a jury has any empirical reason to believe you, but because someone has said that you're trustworthy. It's not a question of science. It's not a question of certification, or standards, or whatever. It's a question of credit. In the eighteenth century (and before) credit was based on class. Now, apparently, it's based on legal investiture with a license. One may be a little more transparent than the other, but when it comes down to juries evaluating evidence, it's still a question of whether they are going to credit your testimony, not a question of whether you've employed the proper standards, because they'll just have to take your word that you have anyway.

The whole point of ISO certification is that you don't have to audit the whole chain of evidence to have confidence that you're getting what you expect.

That confidence is based on trusting the people doing the certification.

We manage to do this to make sure you get an honest gallon of gas and an honest pound of ground beef and that the plastic in your countertop is to spec and won't fall apart in a year. We do it to make sure your car won't crash and your food won't poison you and the bridge you're driving over won't fall down.

Yes, of course we do. But those are different problems than the question of evidence in a courtroom. Two reasons. First, whether a scale measures accurately or a car crashes or food makes you sick or a bridge falls down are all purely empirical questions. We'll know, pretty much right away, whether the people involved have done their jobs or not because if they haven't, the results will be immediately and empirically obvious. This is manifestly not the case in the courtroom. Whether or not standards have been adhered to is never obvious there, because the jury doesn't have the physical access to observe them. And even if there are observable effects of failure, they frequently don't occur for years.

Second, those problems are all about what will happen. That's almost never the question in a courtroom. It's all about what did happen. That's generally not an empirical question anyway, even when done outside the courtroom.

You do not seem to understand that SCIENCE!!! is not a war of words won by whoever can best persuade a group of earnest but naive citizens.

You do not seem to understand that what is entirely useful and appropriate for practical and industrial applications has no bearing whatsoever on what happens in the courtroom or on the issues that a jury has to resolve in reaching a verdict. It's a different epistemic environment interested in different questions with different methodologies. If you're building a house, you want to make sure that it doesn't fall down. The "knowledge" in question only really needs to be functionally valid, or at least close enough to work. But when you're a jury, you can't actually evaluate the functionality or practicality of testimony you've been given. You need to make sure that it is epistemically valid, i.e., convincing enough to allay reasonable doubt.

Let's run with the house example again. Let's say we've got a house, and we need to tell whether it's been built correctly. If you're the inspector, you can poke around. Check directly to see whether codes have been followed, standards applied, best practices employed, etc. But if you're a jury, you can't do that. You have to listen to the inspectors and evaluate what they have to say. The fact that they've got home inspector licenses doesn't matter at all. If he says that standards were/were not followed, you can choose to believe him or not, but none of the epistemic machinery of empirical observation, the scientific method, or even licencing procedures, is going to help you in the slightest. You can't go see if he's right. All you've got is his word. Which likely conflicts with the other side's inspector anyway, making the standards you're talking about doubly worthless, as they seem to produce two results. Or, at least, you do not have the resources to determine which one is objectively correct.

So questions of credit have an entirely different character in the courtroom, one which adding standards and licenses and certifications does nothing to resolve. Expert testimony is, from the perspective of the jury, no different from eyewitness testimony.

P.S. In other news, are you really entirely ignorant of the Innocence Project?

Oh, not at all. And what they're doing is admirable. But what they're doing is not bringing the pure, holy light of SCIENCE to bear to correct horrible, no-good eyewitness testimony. It's a lot more complicated than that. Quite frequently, DNA evidence in their cases is often just a wedge in the appellate courtroom door they can use to explore issues like misconduct on the part of the prosecutors and/or police and a lot of crappy lawyering by underpaid and over-worked public defenders. We're talking about correcting a whole host of arguably objective errors on the part of trial court judges which somehow fail to rise to the level of reversible error for one reason or another. Trial court judges have a lot of discretion, and reversing them on issues where they have discretion is incredibly difficult. DNA is often just a tool to get a federal judge's attention focused on the real problems at trial. So if you think that the Innocence Project is all about just refuting faulty eyewitness testimony with objectively reliable DNA evidence, you're badly mistaken.
posted by valkyryn at 6:26 AM on September 4, 2013 [1 favorite]

localroger: "especially thanks to the Satanic abuse hysteria in the US, is that that kind of pressure will cause you to think you vividly "remember" astonishingly detailed things that completely never happened. "

Can SOMETIMES cause one to vividly "remember." Not always. Let's please be crystal clear about that.

I've known quite a few child abuse survivors and am one myself. Most of them are quite clear in what they do and do not remember. Please don't assume that all child abuse survivors who are asked to recall their abuse are misremembering what was done to them.
posted by zarq at 7:26 AM on September 4, 2013 [1 favorite]

localroger: “You do not seem to understand that SCIENCE!!! is not a war of words won by whoever can best persuade a group of earnest but naive citizens.”

I agree that science itself is more than that. But science doesn't have a spotless record as it's played out in courtrooms. The most vivid example, to my mind, is the example of Cameron Todd Willingham, an almost certainly innocent man who was executed in Texas because of court-endorsed scientific experts who weren't actually using scientific methods. Courtrooms are always down to a war of words. I hope that we can make that war of words more efficient, more just, and that the scientific truth can be made more evident – I hope that's what Willingham's case has done for arson laws – but inevitably it'll always come down to competing sides. All we can hope is to make the scientific truth more attractive and more obvious.

I know that eyewitness reports should be treated with a huge amount of skepticism, though. The unfortunate thing, however, is that science is often on the same footing as eyewitness reports in the courtroom. The best that scientific people can hope for is to make the science itself clear to juries and judges, so that they can see for themselves. When scientific people are treated merely as experts who give an account, they're really nothing more than (admittedly more-reliable) eyewitnesses who tell what they saw.
posted by koeselitz at 7:46 AM on September 4, 2013

Let's run with the house example again. Let's say we've got a house, and we need to tell whether it's been built correctly. If you're the inspector, you can poke around. Check directly to see whether codes have been followed, standards applied, best practices employed, etc. But if you're a jury, you can't do that.

This is not how it works ANYWHERE.

Let's say you are taking a delivery of plastic pellets from the chemical plant down the road, and you want to tell whether they meet spec. You CANNOT just go down to the chemical plant and poke around. And it used to be a problem if the guy at the chemical plant was too busy looking at the naked lady calendar to load the right amount of reagant into the reactor.

So what you do is you demand ISO paperwork which establishes that the procedures are audited, tested, and followed to standards agreed upon by everyone in the industry. The chemical plant does the same thing with the people who supplied the reagant, so you don't have to. The reagant people do the same thing with the extraction company that provides their raw material, so you don't have to.

NOBODY expects to personally audit the entire chain, which is much deeper and more complicated than anything you will ever see in a courtroom, because it is physically impossible. Standards are NOT the same as personal claims. All it takes is for the state to pass a law, as all 50 of them have done with respect to my own field, stating who the reliable standards body is and what publications need to be followed in what circumstances.

I have personally observed the diligence with which standards are written and updated and how proper quality assurance procedures work. If you want to say that someone's brief recollection of a face they glimpsed in marginal lighting is even remotely comparable to the result of a piece of physical evidence gathered and documented under existing chain of evidence rules and tested by procedures agreed upon and validated by national and international standards bodies and performed by regularly inspected and certified labs, well as someone once said f*ck that.

Industry used to do it your way. They learned the hard way that leaving important procedures up to individual human motivation is unreliable and produces poor results. So while they do need human workers, they use written procedures, training, and testing to make sure the process is as uniform and reliable as possible. Where appropriate they use double-blind techniques to make sure individuals can't game the system either inadvertently or deliberately.

Obviously you do need humans in crime investigation to find and document the evidence and to present it in court. But the claims you are making about how test results have to be presented is just wrong. That may be the way the court treats it now, but there is a better way and I can promise you hardly anything that has been produced by technology since 1950 would be possible if we hadn't figured it out.
posted by localroger at 7:47 AM on September 4, 2013

zarq: I didn't mean at all to imply that abuse doesn't occur. But it is proven that people can be convinced under the right circumstances that they "remember" very elaborate things which never really happened. This is a particular problem, in fact, for the people to whom those things really do happen.

koeselitz: The big problem is that courts and police departments do not seem to have a clue how to do quality assurance. I realize there is a problem now mainly because they aren't doing it right. But to claim that such tests can't ever be more reliable than eyewitness testimony because hey experts are just witnesses is unbelievably ignorant.
posted by localroger at 7:51 AM on September 4, 2013

localroger: "zarq: I didn't mean at all to imply that abuse doesn't occur. But it is proven that people can be convinced under the right circumstances that they "remember" very elaborate things which never really happened. This is a particular problem, in fact, for the people to whom those things really do happen."

Do you have any evidence that it happened in this case? Do you know whether what you are proposing is a likely possibility? You're not just engaging in idle speculation. You're using it as an example to discredit her.
posted by zarq at 7:56 AM on September 4, 2013

I mention this because it doesn't sit well with me. I have no idea if what you are saying happened in this case. But I object to adults who endured abuse as children being tarred as unreliable witnesses on the basis of 'well, it's happened before,' without some concrete supporting evidence.

When I began reading the CNN story, I was convinced of McCullough's guilt. Which also probably speaks to my own personal biases. The story lays out a convincing picture of a creep with a disturbing life history who had opportunity and motive. And then the fifth chapter introduced enough doubt in my mind that I now think he was wrongly convicted. Not because he could not have committed the crime, but rather because I don't think the evidence was entirely convincing. I also think the judge made some mistakes in not allowing evidence to be introduced.

Do I personally think he did it? Yes. Do I also think we should only put people in jail unless we're sure they're guilty? Almost always. Even with a crime this awful and horrifying -- and one that disturbs me viscerally -- I don't think the case was open and shut.

I would like to see justice done for Maria Ridulph. Even 50 years later. I'd like to see her killer locked away. But even if he did it, I'd rather the evidence not be so damned flimsy. For many of the reasons you mention.
posted by zarq at 8:09 AM on September 4, 2013

zarq, I don't know if it happened in this or any other particular case, but it has been very well documented to be possible and thus must be treated seriously.

In this case it's a 50 year old memory and there is pressure to find "closure" for the family and town. She's given five pictures with the strong implication that one of them is the guy. This is not how you do a lineup. She could have just picked a picture at random and had a 20% chance of putting somone away. As it is she spent several whole minutes on the last two of the five, which suggests to me that it was 50% in the end.

In other matters, the particular warning klaxon is "recovered memories," especially memories recovered under therapy. If you have a clear and continuous memory that something happened to you then that's obviously a thing to be taken seriously, but if you only remember it years after the fact and only then after being placed in a suggestible state, then I want to know who the therapist was and who else they've treated.
posted by localroger at 9:32 AM on September 4, 2013

But the claims you are making about how test results have to be presented is just wrong.

As you seem to be okay making arguments from authority, now might be the time to point out that I'm a lawyer and you aren't. If we're going to place bets on the one of us who has a better grasp on the nature of evidence in the legal system, I'd put my money on me.

Snark aside, you're still treating the existence of "ISO standard" on a document or the possession of a professional license by a witness, or whatever standard/certification/license you care to name, as somehow quasi-magical proof that the testimony offered is (1) accurate, (2) truthful, and (3) means what the side offering it into evidence wants it to mean.

That's now how evidence works, I'm afraid. As I've said at least three times in this thread, courtrooms are not laboratories. Nor are they construction sites or factories. The things that industry uses to determine creditworthiness and trustworthiness (in the epistemic sense) are not relevant in the courtroom, nor should they be. They encompass leaps of faith and ambiguities which it is entirely reasonable to make in the practical world but which lend themselves to significant abuses and injustice in legal proceedings.

You say that ISO paperwork establishes that standards were followed. I'm here to tell you today that it does not. It can certainly serve as persuasive evidence that standards were followed, but that's not the same thing as proving it. Someone could have cut corners. Someone could have made a mistake. Someone could have made the whole thing up. The jury has to evaluate those questions whenever it's presented with expert testimony.

So your problem here:

If you want to say that someone's brief recollection of a face they glimpsed in marginal lighting is even remotely comparable to the result of a piece of physical evidence gathered and documented under existing chain of evidence rules and tested by procedures agreed upon and validated by national and international standards bodies and performed by regularly inspected and certified labs, well as someone once said f*ck that.

...is the assumption that the standards, certifications, and audits you're talking about serve as definitive, unquestionable proof that an exhibit offered into evidence (1) was actually gathered from the scene of the crime, (2) actually followed the chain of evidence on the supporting paperwork, (3) was actually subjected to the procedures indicated, (4) was analyzed according to indicated standards, (5) was handled in labs that were actually inspected/certified the way they claim to have been, and (5) actually follows from the results of said procedures. I hate to break it to you, but they don't. Not by a long shot.

Juries, judges, and attorneys don't look at certifications and say "Well, I guess that settles that then! We can all go home!" In short, and as I've stated numerous times so far, the only thing the jury has to go on for any of this is the word of the person testifying. At best, such documentation can serve as extrinsic evidence in support of the proffered testimony, i.e., the word of the witness. Which makes it no different from eyewitness testimony on that most fundamental epistemic point. A jury might do well to credit such testimony, but that is essentially no different from crediting eyewitness testimony: the decision to believe that someone is telling the truth, that their testimony is accurate, and that their interpretation of the facts is the best one given all of the evidence. The existence of standards, certifications, licenses, and advanced degrees can certainly be persuasive, but all they do is serve to bolster the credibility of the person testifying. They are not, in and of themselves, indicative of anything in particular, and a jury is entirely within its rights to refuse to credit such testimony if they believe the evidence supports a different conclusion.

I'm not sure how much clearer I can make it. You say that these sorts of factors make scientific and/or expert testimony more reliable than eyewitness testimony. I say that there are 20,000 people sitting in jail in Massachusetts alone who were convicted on the basis of a single tech fabricating forensic evidence over a period of, what, the better part of a decade? My conclusion is that evidence is a huge problem and that there isn't any obvious solution to it, largely because precisely the sorts of things that work to improve the usefulness of objects and data in the practical world don't actually matter in the courtroom.

Your conclusion, on the other hand, seems to be that standards, licenses, and certifications do, or at least can, make expert witnesses inherently reliable sources of evidence.

If you think that, well, I'm sorry, but that opinion in this context probably counts as "not even wrong," as it reflects such a basic misunderstanding about the nature of legal evidence that it simply doesn't apply.
posted by valkyryn at 10:27 AM on September 4, 2013 [1 favorite]

In other matters, the particular warning klaxon is "recovered memories," especially memories recovered under therapy.

For what it's worth, this sort of testimony is viewed with deep suspicion by most courts these days. It's not automatically excluded in any jurisdiction that I'm aware of, but it's really hard to get into evidence. Indeed, several practitioners who have helped their clients "discover" repressed memories have been subjected to successful malpractice suits.

Basically, this sort of thing had its heyday back in the 1980s, but by the end of the 1990s it was largely discredited, and it doesn't really happen at all today.
posted by valkyryn at 10:32 AM on September 4, 2013 [1 favorite]

In courts or elsewhere, there is implicit trust in certified expertise and it's not infinite trust but it's different than trust in eyewitness testimony. It's different for 3 reasons:

1. An expert has more to lose - he invested time and effort in gaining experience and certification and he can lose certification and his job. Similarly the person who certified him can lose his job if it's shown that he is bribed to certify people.

2. An expert's opinion can be challenged by another expert on objective grounds. If this happens often enough his reputation suffers, and potentially he may lose his certification and so on. And the expert knows this, and the opposing side, whether it's prosecution or defense, also knows this, and jury knows this too and it's a part of their calculation.

3. We know from experience that systems based on certified experience work reasonably well. Airplane is not likely to crash and a passenger knows this not because he personally interviewed people who certified engine parts and flight controller's training but because he knows that systems of certifications are in place and all people involved stand to lose a whole lot if there are serious lapses in certification.
posted by rainy at 10:44 AM on September 4, 2013

There are a three things about this case which people should remember:
1. The investigator (Brion Hanley) and the prosecutor (Clay Campbell) made the arrest of Jack based on only what Jeanne and Janet Tessier said, and based on the photo identification of Kathy Chapman. At the time of the arrest, that was the only evidence. During the arrest, they questioned Jack for six hours, sat him down for a polygraph test, and had the best and the brightest from Seattle an Illinois grilling him. After six hours they came up with nothing. I have the original arrest warrant on my blog, there is really nothing to it when you ask the first follow up question.
2. The photo identification of Kathy Chapman is a joke. She had her memory refreshed by Brion Hanly eight days before she was shown the photo line up based on her ORIGINAL descriptions of "Johnny" and then she was presented with Jack's picture and five other guys that did not match the original description. She could not have gotten it wrong. I have a complete write up on this on my blog.
3. The prosecution had no direct evidence Jack was involved with the crime, yet they had arrested him. They then spent the next year making him look as bad as they possibly could, with the help of Jeanne and Janet Tessier. No one ever questions what the two sisters say, and just judge Jack by their stories. The prosecution had a lot to loose in this case, and they were very smart in what they did, they painted Jack to be as evil as possible and were successful. Even now, do people talk about the murder, and evidence, or just that Jack is so evil he should be in prison. People should really fear this, having a State spend literially tens of thousand of dollars digging dirt on a person, and destroying a persons character to cover up a false arrest.
~Casey
Jackdmccullough.wordpress
posted by casey1167 at 11:11 AM on September 4, 2013

1. An expert has more to lose

Any witness who perjures himself can go to jail.

2. An expert's opinion can be challenged by another expert on objective grounds.

So can an eyewitness's testimony.

3. We know from experience that systems based on certified experience work reasonably well.

Yes, we do. But "reasonably sure" is not the standard of proof in criminal cases. It's "beyond a reasonable doubt." A jury that thinks that it is more likely than not that the defendant committed the charged offense but that there is a reasonable chance that he didn't, however small, is supposed to turn in an acquittal.

"Systems based on certified experience" do not provide proof beyond a reasonable doubt in and of themselves, though they can certainly contribute to such a conclusion. Indeed, some cases basically require such evidence. But the mere existence of such evidence is not sufficient to eliminate reasonable doubt. It has to be evaluated and weighed just like any other kind of evidence.
posted by valkyryn at 11:35 AM on September 4, 2013 [1 favorite]

That's now how evidence works, I'm afraid. As I've said at least three times in this thread, courtrooms are not laboratories. Nor are they construction sites or factories. The things that industry uses to determine creditworthiness and trustworthiness (in the epistemic sense) are not relevant in the courtroom, nor should they be. They encompass leaps of faith and ambiguities which it is entirely reasonable to make in the practical world but which lend themselves to significant abuses and injustice in legal proceedings.

I'm not going to keep pounding on this, because I suppose you are right that the legal system really does work in this different, archaic, and woefully inadequate manner.

What you are saying here is that the methods and trust systems which have proven adequate to create unbelievably complex industrial systems such as space launch systems and satellites, power and energy distribution systems, and which assure that the chemicals, foods, and drugs you consume are correctly made, are somehow not good enough for the ancient and holy rite of the courtroom, where those things have to justify themselves on each use as if they are strangers who walked in the door without an introduction.

You are also saying that the observations and impressions of often random individual people, exactly the thing which industry has PROVEN with years of research not to be reliable enough for its own purposes, are actually superior and are to be preferred.

All I can say is that it's no wonder that industry using its techniques has given us international communications and the computers we are writing this on while the justice system has given us 142 innocent people on death row. And counting. God help us all.

I say that there are 20,000 people sitting in jail in Massachusetts alone who were convicted on the basis of a single tech fabricating forensic evidence over a period of, what, the better part of a decade?

I am not saying that this is done right now. I can tell you that there is no industrial plant in the world which would ship 20,000 units as valuable as years of a man's life without noticing that something is wrong. That sort of thing did once regularly happen, but the people facing all that liability and cost figured out how to fix it. That is avoidable. The kind of thing standards and precautions I am talking about are not being done by the justice system and that is the problem.

The infuriating thing about this is that you are acting like the problem is simply eternal and inevitable. I am telling you that there would be no communication satellites or computer chips with a billion transistors on them if that was the case. You may know the courtroom, but I know this about the world, and it is the justice system that is failing to see reality here.
posted by localroger at 11:48 AM on September 4, 2013

1. An expert has more to lose

Any witness who perjures himself can go to jail.


I could be wrong, but my understanding is that it's a high standard that has to be met to prove a deliberately false witness account was given, rather than a human mistake. An expert has more to lose because beyond perjuring himself and going to jail, he can have professional reprecussions that meet lower standard than conviction as a false witness.


2. An expert's opinion can be challenged by another expert on objective grounds.

So can an eyewitness's testimony.

If an expert testifies on a result of a dna test and another expert repeats the test and gets an entirely different result, one of them is incompetent. If a person A identifies person B but another witness testifies they saw B at another place at the same time, there's a lot of fuzzyness where you can say, well, it was dark, he saw someone who looks like B, he only saw him for a few seconds from afar.

That's a pretty significant difference.


3. We know from experience that systems based on certified experience work reasonably well.

Yes, we do. But "reasonably sure" is not the standard of proof in criminal cases. It's "beyond a reasonable doubt." A jury that thinks that it is more likely than not that the defendant committed the charged offense but that there is a reasonable chance that he didn't, however small, is supposed to turn in an acquittal.

Ok, if we go back to the airplane analogy, is a person boarding sure that he'll be fine beyond a reasonable doubt? Is he 98.9% sure, 99.8%? What is reasonable in percentage points?

It's not that the word 'certification' is in itself magical, it's what stands behind it - large number of people who invested time and energy in it and how much they stand to lose if shown to be frauds.

I am only arguing an eyewitness and an expert witness are markedly different.
posted by rainy at 12:56 PM on September 4, 2013

If an expert testifies on a result of a dna test and another expert repeats the test and gets an entirely different result, one of them is incompetent.

In that limited circumstance, perhaps. But you do realize that in just about every case involving expert witnesses, both sides have experts, and the experts contradict each other, yes? With no obvious way for the jury to tell whether either of them is competent?

I am only arguing an eyewitness and an expert witness are markedly different.

Again: in the practical world, they are. Most definitely. But in the context of the courtroom, the difference is not nearly as significant as you and others in this thread seem to want to believe.
posted by valkyryn at 1:06 PM on September 4, 2013

Hello, casey1167. Welcome to Metafilter.
posted by zarq at 1:10 PM on September 4, 2013

where those things have to justify themselves on each use as if they are strangers who walked in the door without an introduction.

They have to justify themselves that way because that is precisely what they are doing. The jury has never met any of these people before. They have no reason to know that the experts have not simply made it all up. They can choose to believe that an expert has done the things he says he's done, and may well be justified in doing so, but all they have to go on is that expert's word.

there is no industrial plant in the world which would ship 20,000 units as valuable as years of a man's life without noticing that something is wrong.

So the BP oil spill never happened? And the Mars Climate Orbiter continues to give us good data? And fracking is an uncontroversial and entirely safe oil extraction technology? And no one ever gets sick or died from foodborne pathogens? And no one has been harmed by a defective product in decades? And tobacco products have been completely removed from shelves?

Seriously, man, listen to what you're saying.

And what, if anything, do you make of the fact that experts are almost always paid for their opinions? There's plenty of motive to come up with opinions that will help the person paying you. Not to mention the possibility that the attorney who hired you didn't give you all the relevant evidence. I've seen that happen too many times to count.

You may know the courtroom, but I know this about the world, and it is the justice system that is failing to see reality here.

Not really. The problem here is that law is not a science. Some dead white guys tried to make it that way in the late nineteenth century, but the project failed miserably. The techniques which have proven so dramatically successful in the world of physics and engineering do not necessarily transfer beyond those contexts in ways that preserve that same level of confidence. Indeed, most of the last three centuries have seen just about every discipline out there, from law, to linguistics, to economics, to the social sciences, try to emulate physics' fantastic success at describing the natural world and enabling us to exert control over it. Those attempts have been uniform failures. The kind of certainty we can achieve in the realm of physics and engineering is simply not possible in law, or in almost anything else, for that matter.

The legal system is part of the political system, and is thus primarily about mediating social relationships, both between individuals and each other and individuals and society as a whole. Neither of those relationships is even remotely scientific, and whatever confidence that scientific methods may have in their own context necessarily doesn't transfer very well to the messy social and political context that is the legal system.
posted by valkyryn at 1:36 PM on September 4, 2013 [3 favorites]

I think we're talking about slightly different things here. There can be a case where an expert is answering a question that is clear, measurable, testable (repeatably so). It's another thing where an expert has to make a judgement call, an arguable determination somewhere on the margins of science, more or less an opinion informed by expertise. Then, you can easily have disagreement, like with any judgement call.

A reasonable person sitting on the jury will be able to distinguish between vastly different degrees of certainty of expert testimony. A weaker link in the chain may be getting enough reasonable people on the jury.
posted by rainy at 2:53 PM on September 4, 2013

A reasonable person sitting on the jury will be able to distinguish between vastly different degrees of certainty of expert testimony.

I would like that to be true. In practice, it's not.
posted by valkyryn at 3:30 PM on September 4, 2013

We are actually talking about several different things that sound alike, so you keep saying (and maybe even thinking, who knows) I'm saying things I'm not saying.

Item 1: Traditional Witness Testimony

It varies, but particularly when witnesses have only a short time to observe and are under emotional stress, eyewitness testimony is almost worthless. It is so bad that the going theory in neurological circles is that most of what we think we remember is actually reconstructed from a much thinner set of actual recorded data than we realize. It is possible for us to very easily forget or fail to record critical data points and to modify them through overthinking or suggestion until we have what seems like a very clear vivid memory of something that completely did not happen. This is not the exception, it is the norm.

This can be mitigated, and it's certainly much more credible when someone who had a relationship or spent time with a person testifies as to their identity. But chance encounters in the heat of a threatening situation? Humans don't do those well. Until it recognizes that the court system will continue to put unacceptable numbers of innocent people in jail.

Now it is this observation that attracted your attention with the observation that

But you're attacking two of the mainstays of evidence in criminal trials, i.e., eyewitness testimony and defendants turning states' evidence.

...after heading off the most obvious answer in the previous paragraph...

"Science" is not the answer here, as it doesn't provide nearly the level of certainty that people seem to think that it does, and it's only ever as good as the lab that produces it. My emphasis, note for later.)

And just to make sure I understood the importance...

So really, I'm asking, what are we supposed to do? Just stop prosecuting crimes? Well we're likely to have to think about that simply because we can't afford to pay for all the inmate-days that the criminal justice system generates. But there really are criminals out there, who really do commit acts which everyone agrees are wrong (e.g., assaults, robberies, thefts, etc.). We cannot function as a society if we do not do something significant about that.

So there's my choice, a few innocent people going to jail or ANARCHY!!!! Hats off and over the wall.

Look, I know we will always have to rely on traditional witnesses. But we need to do so as little as possible and recognize their limitations. It's much more convincing if a group's descriptions converge or are backed up by other evidence. I would be very skeptical of the wisdom of ever putting somone in jail on the word of a single fleeting eyewitness, though. The human mind has simply proven too unreliable for that, whether you like it or not.

Item 2: Traditional Expert Testimony

Here we are on firmer ground with respect to the expert's ability to exercise careful consideration but shakier ground with respect to whether his credentials and methods command a certain level of trust. There have been quackpot experts who put on a good show and gotten a court hearing they did not deserve and people have gone to jail, and in one case I would easily believe the death chamber, because of them. It is kind of ridiculous that lay people have to sort this out at EVERY SINGLE TRIAL, which is an absurdity I'll get to shortly.

Item 3: Lab Tests as Done Now

Here we are basically having the expert witness who did the test ad hoc testifying as to their education and so on before giving their results. This is akin to the way industrial chemistry was done before WWI. It hasn't been done that way for a long, long time and for good reason.

I can see why someone looking at this system would consider it functionally equivalent to #2 and subject to the same problems. However, #2 could be signifcantly improved and #3 can for practical purposes be completely fixed, by something that doesn't exist yet but sure the hell should.

Item 4: The as yet nonexistent Forensic Standards Institute

You cannot simply build a scale and put it in your store window for sale, unless it meets certain legal requirements. You should not be able to offer expert testimony either.

In the mushier world of expert testimony, as opposed to lab tests, where two experts may legitimately arrive at different conclusions, their methods can still be compared to standards. New methods should be disallowed until they are evaluated.

Now I can hear you saying "evaluated by who? Who watches the watchers! It's all just tesetimony!!!" except the reason it isn't is the government forms up the standards body and the states pass a law directing the courts that methods conforming to their publications are acceptable. Just as they did long ago for scales. It's been done and it works. Testimony can be evaluated after the fact by the standards committee and such an evaluation should be a valid basis for appeal.

For lab tests, the issue is much clearer as if two labs give different results one of them screwed up. Lab tests are not opinions, they are measurements, and their accuracy can be assessed. This is an objective testable fact and one easily tested IRL by a properly designed quality system. We know how to build such quality systems and we know that they work a lot better than not using one. They work a hell of a lot better than depending on the ad hoc skills and observations of individual people who are then evaluated by laypeople who aren't even familiar with the body of ideas they are evaluating.

So here you branch off to...

So the BP oil spill never happened? And the Mars Climate Orbiter continues to give us good data? And fracking is an uncontroversial and entirely safe oil extraction technology? And no one ever gets sick or died from foodborne pathogens? And no one has been harmed by a defective product in decades? And tobacco products have been completely removed from shelves

Seriously man, listen to yourself. The BP spill and MCO both represent the failure of quality systems. Both had great blowback to avoid a relapse. Do you realize how many things have to go right to drill an oil well or put an orbiter around Mars? If those systems had a failure rate comparable to what the Innocence Project is uncovering we would still be burning trees and flying unsafe propeller based aircraft, if that. The number of food contamination outbreaks which reach the outside world is incredibly tiny, and there is a vast infrastructure in place to track the few that happen back to their source and prevent recurrences. Again, if that system had a failure rate comparable to the justice system there would be mass poisonings every week.

Failure is part of any system but the key to a modern quality system is that you have mechanisms to detectit and avoid recurrences. Before that asshole lab tech was caught the importance might not have been realized of using double-blind techniques in DNA testing. Now we know it's important and the standard (if it had ever existed at all) should be revised to require them.

One of the things you internalize as you move to ISO 17025 is that there is no such thing as perfection. There is only better and worse, both of which must be (and in cases like test labs can be) quantified. The law is about words but science is about numbers, and when you look at numbers the things you have been saying about the usefulness of scientific techniques toward plugging the gap left when we admit eyewitness testimony is mostly garbage are mostly unjustifiable. It's not good enough the way we are doing it now, I admit. The way we are doing it now is horrible, as anyone with an industrial QA background would instantly recognize.

This CAN be fixed. The states can send a guy out to gas stations to make sure they dispense an honest gallon but they can't be arsed to make sure the labs whose results will steal a person's entire life are being run correctly? What kind of priority is that?

A person on a jury should not have to decide whether a scientific technique is in accord with accepted standards or whether a lab technician seems competent and trustworthy. We have elaborate systems (or in some cases easily could have them if we wanted) which make that unnecessary. By all means let the jury judge the believability of those eyewitnesses we must unavoidably use and experts who work in fields like psychology where honest disagreement is possible. But don't act like we can't get measurement right. We do that far better than you realize.
posted by localroger at 3:47 PM on September 4, 2013

Oh, another thing that occurs to me on re-reading this V.: You seem to think that there are only two frequencies of failure, "zero" and "not-zero" and that systems whose failure frequencies are not-zero are functionally equivalent. This is of course bollocks, and the difference between 1% and .001% is that between airplanes flown by daredevils and those flown by commuting passengers.

The US has a bit over 3,000 death row inmates according to Wikipedia, after 140 have been exonerated. That is a minimum failure rate of nearly FIVE PERCENT. Do you have any idea what a chaotic mess the world would be if any commonly used industrial process had a failure rate that high?

And it's probably WORSE in the justice system because we don't know how many innocent men simply don't have evidence to retest, or how many more who aren't on death row took plea deals to avoid a worse fate.

If science did not do far better than this every single day we would be wearing animal skins and hunting with clubs.
posted by localroger at 4:25 PM on September 4, 2013

except the reason it isn't is the government forms up the standards body and the states pass a law directing the courts that methods conforming to their publications are acceptable.

You may think science works that way, but (1) it doesn't, and (2) even if it did, the law doesn't.

Which is what I've been saying the whole time.

The BP spill and MCO both represent the failure of quality systems.

Precisely. Quality systems fail. Indeed, many of the lawsuits I've litigated have precisely to do with the failure of quality systems. Which are only ever as good as the people running them. And to figure out how good those people actually are

don't act like we can't get measurement right. We do that far better than you realize.

No, I'm pretty sure we don't. Measurement, as you suggest, is pretty easy.

The problem is that measurement isn't very useful. You seem to think that expert testimony is mostly a function of running a few entirely standardized tests and reporting the results.

That, like, never happens.

Only a tiny fraction of the expert testimony that gets proffered in court, civil or criminal, is the kind of thing you seem to think can be completely fixed. It totally can't, but stipulating for the purposes of argument that it could, it wouldn't make a lick of difference. The number of cases that turn on that sort of evidence is tiny. Really tiny. Indeed, if the only question at issue were measurement, we wouldn't need experts, because, as you say, measurement is something we can get right a lot of the time with a minimum of fuss.

In reality, most expert testimony is about things which are either hard to measure, or can't be measured at all. We've very little interest in knowing how much a particular object weighed in the vast majority of cases. Even if we did, it's the kind of thing that's so easy to establish that the court might actually take judicial notice of it without any need for expert testimony.

No, we want to know things like whether a physician's actions in a particular case met the standard of professional care when there were three generally accepted courses of treatment and an unclear diagnosis. That's not something you can measure.

We want to know whether a plaintiff's neck pain is due to arthritis or the subject car accident. That's not something you can measure, at least not after the fact, which is what we need.

We want to know whether a defendant is competent to stand trial. That's not something you can measure.

We want to know whether a defendant is suffering from a mental disorder such that he cannot understand the nature of the charges or assist his attorney is his defense. That's not something you can measure.

We want to know how fast the defendant was going when he T-boned the plaintiff's car. Hey, that's something you can measure! Except that it isn't, because we don't have most of the data points we need to actually make that measurement. We have to make assumptions to fill in the holes, and one set of assumptions works in favor of the defendant and the other in favor of the plaintiff, and wouldn't you know it, the jury has to decide which set of measurements is more plausible.

We want to know whether the defendant's wooden telephone pole failed because it was rotten or because the plaintiff hit it while speeding. Hey, we can measure that right? Well, we might have been able to, except that we don't know how fast the plaintiff was actually going (see above), and the relevant section of the telephone pole was blown into about a million pieces when the power line it was holding up came down. So the experts can give us a range of possible values based on a range of possible assumptions, and once again, the jury has to decide which is the most plausible.

We want to know about the projected lifetime earnings of the plaintiff before the accident. Which, again, is something like a measurement, only you have to make up basically all of the input numbers, and the output is only as good as the input, so each side is going to use different figures. Hardly counts as a measurement when put that way.

We want to know whether the plaintiff's BAC was 0.09% or 0.10%, the difference between six months of probation and six months in jail. Again, something you can measure, right? Yes, except that the measurement we've got is from a hospital which used a serum test rather than a whole blood test, which is known to give results up to 15% higher than either a whole blood or a breathalyzer test, the latter of which also has a margin of error of 0.015% even when done properly and can be easily manipulated by cops to give results up to 0.05% higher than appropriate. So that 0.10% measurement that the prosecution wants entered is, on closer analysis, possibly as low as a 0.045%, changing the result from six months in jail to no offense at all. But BAC is just a simple measurement that we can render objectively reliable by the enactment of appropriate laws and the promulgation of appropriate regulations, right? Right?

Yeah, not so much.

You believe things about science and its uses which simply aren't true, even outside the courtroom. Science does not actually give us much information as you think it does. It's enabled us to make monumental advances in our quality of life, no question. But the kinds of questions that we need to have answered in the legal and political system in general and the courtroom in particular are almost never answered solely on the basis of a straightforward, uncomplicated, well-documented scientific test. They are, at best, tangentially informed by such. More often than not, they're simply not related at all.

And with that, I think I'm done. I've said the same thing about half a dozen times now, and you just keep going further and further down the Modernist science-has-all-the-answers path that I thought most people had abandoned thirty or forty years ago. I've no interest in retracing that route. And fortunately for me, the courts do have at least some inkling of what's going on here, even if they give experts far too much creedence most of the time.
posted by valkyryn at 5:57 PM on September 4, 2013 [1 favorite]

Well I guess I'm done too because you seem intent on defiantly interpreting everything I've said on your own terms so as to make it sound ridiculous.

I know how science and technology work. You clearly don't. You persist in conflating wildly divergent error opportunities as if they are all the same thing. There is a lot of math behind this which says why these situations are different, and which you obviously don't know.

I will admit that you know the system of the law better than I do. Apparently your system doesn't do math very well either.

Your system's error rate is five percent. My system would work better, if it were applied to your system where it is appropriate. But that's math, and I guess a system built on words can't understand that.
posted by localroger at 6:24 PM on September 4, 2013

BTW to roll past all this ridiculous digression about neck pain and telephone poles, we were talking about DNA testing. DNA testing is a pure measurement and many, many cases have turned on it. Your original supposition was that because of a rogue operator that DNA testing is no better than eyewitness testimony.

You are perfectly and quantifiably wrong about that. And if your system upon which we depend so much can't get that, we are fucked.
posted by localroger at 6:39 PM on September 4, 2013

you seem intent on defiantly interpreting everything I've said on your own terms so as to make it sound ridiculous.

Or, rather, you seem to refuse to concede that there is more than one plausible interpretation for your words, and that your preferred interpretation might not, in fact, be the one that prevails in practice. I can concede that your interpretation has its place while still insisting that mine does too. You have to argue that nothing I say is remotely true. Which of us has the harder task? And which of us has engaged it?

I know how science and technology work. You clearly don't.

Or, rather, you've adopted an almost pathetically naive and credulous epistemology and philosophy of science which attributes unjustified epistemic certainty to a system which is based on precisely the sort of assumptions and postulates that the legal and political systems cannot make.

Law is not science. Law is not technology. Whatever you think you know about science and/or technology, the idea that those same principles work in the legal system is ridiculous.

Also note that you've completely ignored my argument that even if what you're saying about science and measurement was true, it would only matter in a tiny fraction of cases. If this were litigation, it'd be time for me to file a Rule 56 motion for summary judgment. Even if you're right about every fact you've posited, I still win. Expert testimony is not key in the vast majority of cases, and where it is, it's not the kind of testimony that would be even theoretically aided by your preferred correctives. Ergo, the problems with expert testimony cannot be even marginally improved by your proposals. Q.E.D.

Your system's error rate is five percent. My system would work better, if it were applied to your system where it is appropriate.

I highly doubt that. Your system would bring science and technology crashing to the ground. The last time human society experimented with government-authorized systems of credit and trust was primogeniture. And that worked so well. Replacing that with "science" would be no improvement. That's simply not how science works. Innovation and scientific advances are not possible where there are legally-defined boundaries for acceptable scientific knowledge.

Further, as stated above, applying it to "my system where appropriate" would mean applying it. . . basically never. So, again, Q.E.D.

You are perfectly and quantifiably wrong about that.

If that were true, someone would have done it by now. Which suggests to me that you're wrong.

But whatever. You go on fighting the good Modernist fight. The rest of us will continue to move on from what has proven to be an intellectually bankrupt tradition where anything but science and engineering are concerned.
posted by valkyryn at 5:05 AM on September 5, 2013 [1 favorite]

Well, you have convinced me of something I never thought I'd consider. It occurs to me that I've been subjected to what passes for a "closing argument" in court -- a word salad where math is ignored and dissimilar things are conflated in order to create an impression that makes the appropriate persuasive effect.

And this means we might have to consider burning the whole thing down and starting from scratch. As a way of making life and death decisions this is only a couple of notches up from divination.

You think I'm naive because I don't understand that the law exists in some fantastical place that is inaccessible to the methods of science. I think you're an idiot because you think such a place exists in the material world. We are doomed to talk past one another.

The thing is, a part of your system that is designed to kill people has been proven to kill five percent of them in error. Any other system that failed so spectacularly and so often would be shut down immediately.

In your first comment where you reacted in such horror to the idea that we might have to reconsider eyewitness testimony, after dismissing science as a possible answer you mused that we might not like the police state that could be an answer. Here's the thing: We already live in a panopticon of nearly universal surveillance, location tracking, and where DNA (presumably done right) can make it almost impossible to visit a place or touch a thing without leaving evidence of our passing. Like the fact that eyewitness testimony is crap this is the case whether we like it or not, and it is a state the people who laid the foundations of common law a thousand years ago could not have envisioned.

The world has changed. The law can follow, or one day everyone will realize it has become obsolete.
posted by localroger at 5:30 AM on September 5, 2013

On the staircase...

I have been wondering now for a couple of days how someone so obviously intelligent and literate could pour so much energy into an argument that is so obviously stupid. Your skill with words is admirable, but you use it here maliciously and disingenuously. It's not like anybody over the age of 8 born after 1930 couldn't see that a technological measurement can in fact be superior to human recollection; a similar debate went on with the introduction of fingerprinting and while the experts still have to testify, the technological advance was accepted.

All your flights about how the law is somehow in la-la-land, and how *this* whatever *this* is that isn't whatever la-la-land thing you're on to has been just the same as fascism or whatever, you built that crap for a reason. You're so good at it you have to know better.

Finally, today, it hit me. You're a defense attorney. Duh.

Yeah, every human being someone like me takes out of that witness stand is a hammer removed from your toolbox. A little fact comes in, whether it's NIST certified or created by a corrupt technician whose results have never been checked, doesn't matter. If he testifies in your favor you will coddle him. And if he testifies against you, even if he is telling the truth, you will use all the same tools you have used in this thread to discredit him, make him look like an idiot, try to trip him up, make claims that sound reasonable to the jury but are really so outrageous that the expert is left mouth gaping in astonishment as to how to respond, to benefit yourself.

It won't always work of course, maybe not even often, but it's a card you have up your sleeve and you don't want to lose it. Judging from your performance here it may even be your ace, your primary skill. Actually figuring out the truth is a secondary thing to winning the game when the score determines whether you'll get more clients, or re-elected.

I guess the idea that science could introduce things in to the court that you can't challenge by attacking the person who has to testify to them must terrify you. Instead of having a technician you can badger and hope to confuse and maneuver into a self-contradiction, a simple edict that the court recognizes the result of a test as having passed standards and should be taken as fact by the jury -- oh that must make your blood run cold. I guess if I was you I would argue against it with all the tools in my box too.

Would that my tools did not make me look like such an atavistic fool.
posted by localroger at 7:37 PM on September 6, 2013

Localroger: I think you're arguing past each other. Let's say that a case comes down to whether a scale was properly configured, and you get called as an expert witness. You provide your credentials, and testify that you examined the QC sticker which links to a test diary (or however you do it over there) and from there to a set of calibrated weights and all the way back to the standard kilogram. The court still needs to judge whether they believe that you checked those things, and whether you're telling the truth about it being the proper way to examine a scale, and whether there might be other things that would have confounded the measurement that you haven't considered. So they're really examining you as a witness, even though you're testifying to something that is about as objective as anything could possibly be.

In contrast, when they examine the testimony of an eye witness to a kidnapping they're considering testimony about an event that took place a long time ago, took place very quickly, at a time of emotional stress, and which was observed by an unskilled witness. None the less, this act of hearing evidence is the same thing in both cases. On the one hand we have a professional account of a highly reliable procedure. On the other we have what must now be an unreliable memory. They're both pieces of evidence, even though one is less reliable than the other. The court procedure and the court's duty is the same in both cases: to determine the truth.
posted by Joe in Australia at 10:26 PM on September 7, 2013 [1 favorite]

Joe, Valkyryn has moved the goalposts so many times that you really need to go back to his first comment complaining about my complaint about the value of eyewitness testimony.

Eyewitness testimony is given far more credence than it should in court, full stop. Eyewitnesses are extremely unreliably in verifiable and proven ways. V complains, however, that without this kind of testimony there is nothing left but anarchy.

And to cut me off at the pass, he raises two possible answers and objections to them; (1) "science" is not the answer because hey, some lab technician went rogue once, and (2) a total police state of universal surveillance might do the trick, but it would be unpleasant.

This whole discussion has turned on (1), because of course science can be an answer, but you have to do it right, which admittedly is not the case in forensic science today. And even done wrong to claim that DNA evidence is no different from the recollection of an eyewitness is just insane.

The problems with DNA evidence as it is currently processed can be fixed, as metrologists fixed those same problems hundreds of years ago. The problems with eyewitness testimony cannot be fixed.

Here's where you are wrong:

The court still needs to judge whether they believe that you checked those things.

No, unless there is some positive evidence to the contrary, such as a failed inspection, the court does not need to judge those things because the quality system of inspections and certifications has already done that. Both of the cases where I testified involved extraordinary failures where there was conflicting evidence, which is not what we are talking about here; what we are talking about here is a simple declarative measurement, such as "this DNA sample found in the victim's vagina matches the defendant." That measurement, like the ticket from a certified scale, is a very different thing from "I recognize the defendant as being the guy who jumped up and ran away when I surprised him in the alley." It can of course be wrong, just as a scale can defelop a fault, but it's much less likely to be wrong than the testimony of a person who got a brief glimpse of a stranger in poor lighting.

Now, if you have conflicting evidence, such as the DNA not matching the fingerprints or the defendant having a solid alibi, then you have to sort it out the old fashioned way, but that also means something has gone horribly wrong. And things certainly have gone horribly wrong in the justice system because lab quality systems are not as mature as they should be. Again, this can be fixed.

Valkyryn has been arguing very strenuously that the DNA measurement is not only the same as the eyewitness account, but that it is somehow inferior because hey rogue technician and Deepwater Horizon and NASA once drove a spacecraft into Mars so you can't trust that silly crap just because it says you should and it is basically exactly the same as seeing the black guy chase the white guy into the classroom and stick him up with a gun.
posted by localroger at 6:49 AM on September 8, 2013