Think You Can Rely On Your Local Crime Lab For The Unvarnished Truth? Think Again

By Mark Esposito, Weekend Contributor

Forensic-Science-S_2132330bA 2009 report by the National Research Council (NRC) passed quietly into the night (except in legal and forensic circles) while barely garnering more than a ripple in the public’s psyche. It should have been a tidal wave given news last December  that a  48-year-old New Jersey man, Gerard Henderson, who spent 19 years in prison for a murder he didn’t commit, was done in by faulty crime lab work. Henderson was convicted largely on “bite mark” evidence. Bite mark evidence is a process used to exam indentations and anomalies on a victim’s body and  ostensibly made by human teeth which are then matched to a defendant’s dentures in an effort to prove that he/she was the perpetrator of the crime. Convicted in 1995, Henderson proved that state testing of the bite marks on the back of 19-year-old victim, Monica Reyes, was deeply flawed and conducted without sufficient safeguards to insure its reliability.

Independent forensic scientists working for Project Innocence could not reproduce findings by the state crime lab which is the gold standard for scientific verifiability. Henderson became one of the more than two dozen people wrongfully convicted of rape or murder  since 2000 as a direct result of flawed bite mark evidence analysis all duly attested to as accurate by the local crime lab.

It should have been obvious to the government  that something had to be done after the 2009 NRC report which excoriated state crime labs. ” According to the report, nearly every analytical technique, from hair-sampling methods to those used in arson investigation, is unreliable, with too much variability in test results. Only DNA evidence escaped condemnation.”  The report documented scores of problems from funding to lab protocols to evidence gathering which insured scientifically unreliable results. In its summary, the council found that:

With the exception of nuclear DNA analysis, however,  no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.

What that little bit of scientific obfuscation means is that no evidence outside of properly controlled DNA analysis can match a person to the crime scene that passes the same scientific muster that we apply to introducing new drugs to the market or even to safety testing of new  automobiles. No analysis for hair samples nor bite marks nor even arson investigations are reliable enough to be deemed “scientific.”  And the fault lies not  just in the  hands of the lab technicians crippled by a lack of standardized methodology  but in the prevailing methodology itself and the utter lack of peer reviewed studies establishing a link between the evidence and the ability of the testing to individualize the depositor of the evidence. This flawed methodology leads to inconsistent results and a fragmented system where justice is merely a hope.

And the problem has real human costs. On February 17, 2004, Texas resident Cameron Todd Willingham was executed by lethal injection for the arson death of his three daughters at their home in Iraan, Texas. The main evidence against him was the expert opinion by law enforcement officials that the fire had spread by means of a liquid accelerant. Proof of the accelerant, the arson investigators said, were “char patterns” in the floor in the shape of “puddles”,  and a finding of multiple starting points of the fire, that had burned “fast and hot.”  Willingham denied the charge to his dying day and no motive was ever established. But Deputy State Fire Marshal Manny Vasquez and others concluded that burn patterns clearly established the use of an accelerant and testified that human agency started  the fire.

But how reliable was Vasquez’ opinion?  Not very much said Craig Beyler, who holds a Ph.D.  in Engineering Science from Harvard, and who prepared a written report at the request of the Texas Forensic Science Commission. Writing five years after the execution and in the same year as the release of the NRC study, Dr. Beyler  concluded that investigators ignored the scientific method for analyzing fires described in NFPA 921, Guide for Fire and Explosion Investigations and relied on “folklore” and “myths”.  Citing many of the same problems that the NRC would cite a month later, Beyler made this chilling assessment about the “methodology” employed to convict a man of a capital crime:

NFPA 921 provides a core methodology, methods for planning and conducting the investigation, and methods for collecting, interpreting, and documenting evidence. Most modern fire investigations texts mirror or amplify upon NFPA 921 (e.g., Icove and DeHaan (2004), DeHaan (2002), Lentini (2006)). The core of the 921 methodology is the application of the scientific method to fire investigation. In the context of fire investigation this involves the collection of data, the formulation of hypotheses from the data, and testing of the hypotheses. Conclusions can only be drawn when only a single hypothesis survives the testing process. None of the investigators employed this methodology. Indeed, in no case was any methodology identified. The testifying investigators admitted on the stand that there were possible alternate hypotheses that were consistent with the facts of the case. In no instance did this cause the testifying investigator to alter his opinions in the least. The overall standard that seems to be in use by the investigator is that his professional opinion with regard to cause was simply the explanation of the case facts that the investigator was personally most comfortable with.

Dr. Beyler found that “a finding of arson could not be sustained” and that key testimony from the fire marshal at Willingham’s trial was “hardly consistent with a scientific mind-set and is more characteristic of mystics or psychics”.

Did Texas execute an innocent man on flawed evidence? In 2010, Judge Charlie Baird thought so but a crafted motion for recusal by the prosecutor prevailed and stopped entry of an order which found, “overwhelming, credible, and reliable evidence” that Willingham was wrongfully convicted of murdering his daughters. Citing a report by fire investigator Gerald Hurst and the Arson Review Committee impaneled by Texas Innocence Project , Baird concluded that “every indicator relied upon since [by the prosecution’s experts] has been scientifically proven to be invalid.”

Against this backdrop and the outcry over the execution of a likely innocent man on flawed scientific evidence, the US Department of Justice and the National Institute of Standards and Technology (NIST) has now created the first US National Commission on Forensic Science. “The panel of 37 scientists, lawyers, forensics practitioners and law-enforcement officials met for the first time this week in Washington DC, and aim to advise on government policies such as training and certification standards. In March, NIST will begin to set up a parallel panel, a forensic-science standards board that will set specific standards for the methods used in crime labs.”

The goal is to put some scientific method into forensic science which for too long has enjoyed the undeserved status of infallibility in assessing guilt and innocence. “The fundamental issues with forensic science can be solved by fixing the science,” says Suzanne Bell, a forensic chemist at West Virginia University in Morgantown.

The question is now should courts have swallowed the forensic Kool-ade so completely and admitted into evidence what  can only be characterized as “junk science” at the behest of prosecutors? Maybe defense attorneys were right to have complained that modern forensic techniques displaced the deliberative role of the jury in determining guilty or innocence since the reliability of the testing was clearly oversold. And lest you think it’s only the new-fangled techniques in question, even fingerprint analysis, the hoariest and most famous technique we have,  has shown some flaws despite the most rigorous protocols.  “A 2011 study found that professional examiners matched two finger­prints incorrectly once in every 1,000 times, and missed a correct match 7.5% of the time .”

The truth now evident, judges must carefully screen all forensic evidence in light of the requirements of  Daubert to insure both materiality and reliability. What was once the worry of creative criminal defense lawyers  seeking to pioneer new scientific theories seems to have now shifted to the prosecutor to establish reliability for things we took for granted for so long.

Did we buy the forensic scientist’s snake oil to the detriment of innocent men? Gerard Richardson and Cameron Todd Willingham seem to suggest that we did. Now what can we do about it?

Source: Journal Nature

~Mark Esposito, Weekend Contributor

32 thoughts on “Think You Can Rely On Your Local Crime Lab For The Unvarnished Truth? Think Again

  1. Not surprised. This is one of several reasons I am not in favor of the death penalty. Way too many innocents on death row and once that punishment is administered there is no going back.

    I’m a retired chemist who has worked in EPA certified laboratories, not forensic labs, all my adult life. I have seen cases where technicians report what they want to find and not what they did find—they were promptly dismissed. These errors of judgment are discovered by proper lab protocols and quality control review. Job pressure, time constraints, people demanding immediate results lead to inaccurate data….and when this involves forensic work someone’s life is in jeopardy.

  2. Mespo,
    I have been involved in a number of cases where there were experts who were questionable at best and outright frauds at worst. For example there was the late unlamented Fred Zain who should have gone to prison himself.

    One I ran across all the time in Mississippi was Dr. Steven Hayne, who presented himself to the courts as a forensic pathologist. I have another less flattering view of him.

  3. Lawyers in the defense bar are not doing enough to educate ourselves in how to defeat these bogus testing and testors. We need a mandatory seminar in each state which all members of the defense bar should be required to attend. There we need some experts to demonstrate how the labs in our state are faulty and fraudulent.
    The story is too lenient on DNA evidence.
    One of the big failures of defense lawyers is not putting the prosecution and the trial judges to proper foundation before evidence is introduced. Did JoeBob wear gloves when he handled the blood sample found on the dead lady’s hat just after he touched the defendant’s drivers license handed over to him by the defendant? The cross contamination issues arise at every crime scene.

    The results of the study here on topic need to be published in every state bar magazine next month. This blog ought to be required reading for every criminal defense lawyer in America.

  4. Part of the problem of these experts is not whether they can pass a Daubert challenge. They can often pass a Daubert challenge because they are willing to lie. Perjury is a hard thing to defend against when it blindsides the attorney. True preparation for such an expert is to hire an expert who is able to prepare the attorney for a scorched earth cross examination. My problem in the past has come when I try to prepare an attorney and his or her eyes start to glass over.

  5. This is quite encouraging. If proper, standardized tests can be uniform and subject to national scientific scrutiny it would promise a great inprovement not only in the quality but the credibility of current and future tests recognized. The precision of NIST standards are revised when needed and effort is made to achieve the most reliable and accurate standards.

  6. mespo,
    You’re welcome.
    A few years ago when living in San Francisco we saw a stage play “The Exonerated” with Brian Dennehy, Stockard Channing and others. A chilling account of real people on death row who had been exonerated of their crimes. How anyone can support the death penalty after seeing this play is beyond me. This play had a visceral impact difficult to describe and it is well worth seeing if you have an opportunity.

  7. Darren,
    Excellent point on the standards. However, even when procedures are standardized there are still rogue lab people. How useful is a standardized test when somebody like Fred Zain throws the blood sample away, and simply fabricates a report?

    The story about the Records Center employees in St. Louis is instructive. No matter how standardized the procedures, how do we protect ourselves from people who are careless, lazy, ignorant, or all of the above?

  8. Chuck:

    Maybe it should be in the case of lab workers where better standardization of lab procedures might help but it does come down to oversight and in the end there is always the malevolent mind that should be removed from the situation, preferably beforehand.

    One thing that might be of use, and I am really shooting in the dark because I haven’t been within or reviewed the processes of crime labs, is to apply some form of balance system as is used in research laboratories to ensure accuracy of data. A harmonic of this would be double blind studies where the human element is minimized such as placebo effect or “confirming the evidence via the presumption”.

    There is a industrial design method called Poka-Yoke, which translates from the Japanese to mean Mistake-proofing. In a production environment it is a system where human mistakes are minimized by preventing or drawing attention to defects made by human operators or machines. A Poka-Yoke itself is thing or a procedure that prevents a mistake thereby halting an undesirable outcome. A common Poka-yoke is a large nozzel for diesel fuel pumps that will not fit into a gasoline based car’s fuel tank.

    If the methodoligy at the tech level has more intrinsic checks and balances on the human side and interlocking standards on the procedural side I am certain it would be of benefit.

    For a very interesting look at NIST standards one can look to time standards. The precision that has become time standards is astronomical, literally, as it relies on celestial standards such as quasars, the moon, the sun, variations in rotation of the earth and many other factors. The public side of this, radio station WWV, is in itself quite interesting, I still enjoy listening to it once in a while. Reminds me of my early teens in the radio room.

  9. We see on TV numerous CIS serial programs, none of which have addressed the dubious nature of what they do. Perhaps it’s time they addressed this problem. What would Abby do if her work lead to the death of an innocent person? She’d hang herself in her own lab!

  10. With the exception of nuclear DNA analysis, however, no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.” – 2009 NRC report

    That may change in future reports:

    From biology class to “C.S.I.,” we are told again and again that our genome is at the heart of our identity. Read the sequences in the chromosomes of a single cell, and learn everything about a person’s genetic information — or, as 23andme, a prominent genetic testing company, says on its Web site, “The more you know about your DNA, the more you know about yourself.”

    But scientists are discovering that — to a surprising degree — we contain genetic multitudes. Not long ago, researchers had thought it was rare for the cells in a single healthy person to differ genetically in a significant way. But scientists are finding that it’s quite common for an individual to have multiple genomes. Some people, for example, have groups of cells with mutations that are not found in the rest of the body. Some have genomes that came from other people.

    Medical researchers aren’t the only scientists interested in our multitudes of personal genomes. So are forensic scientists. When they attempt to identify criminals or murder victims by matching DNA, they want to avoid being misled by the variety of genomes inside a single person.

    Last year, for example, forensic scientists at the Washington State Patrol Crime Laboratory Division described how a saliva sample and a sperm sample from the same suspect in a sexual assault case didn’t match.

    (The “It’s In Your Genes” Myth – 2).

    Very informative but distressing post Mark.

    One would hope that professionals who are paid well would be more careful with people’s lives.

    But the history of Eugenics dampens hope to some degree, because sometimes professionals in these forensic fields get deep into before they realize a wrong turn has been made.

  11. I usually refer to this as the “CSI Effect,” but one has to wonder, have television shows like CSI contributed to the problem, or are they based on the problem…?

    I fully understand that television, for storytelling purposes, often gloss over the boring details — pop the sample into the machine, it goes “ding” and we have an answer — which simply doesn’t work in the real world.

    But, just how much are juries swayed by that idea, and how much are juries swayed by the overall system? We already pretty commonly see the attitude that, if someone has been arrested, they must be guilty of something. I think we are here seeing something similar — if there is evidence, then it must show the defendant to be guilty.

    What’s to be done? It will have to fall on the judges, I’m afraid — and that doesn’t always look good when it comes to the protection of a defendant’s rights…

  12. Excellent article Mark. Until we hold posecutors and police accountable for sending innocent people to prison and to death, these injustices will continue. Standards are great, but if perjury isn’t prosecuted, innocents will continue to die.

  13. In my opinion television shows that deal with criminal investigation have sold the public, that is the next jury pool, on the infallibility of “CSI” evidence as well as the appropriate level of rule breaking in which police and prosecutors can engage to get the perpetrator. These same shows rarely deal with the fall out from police abuse or lab mistakes. Add to this the new crop of vigilante type “crime” fighters who appoint themselves judge, jury and executioner ant you have a pretty dangerous mix for all of us.

  14. Rafflaw, lets not forget Judges, expert witnesses and baited witnesses as I like to call them. People that are willing to fraudulently testify for reduced sentences or immunity.

    This is a very depressing blog at times but keep up the great work everyone.

  15. It would be interesting to take one of these cases, get all the evidence and court testimony, have every one on here evaluate it and make recommendations to the State(s) or Federal Prosecutors Office of jurisdiction.

  16. hskiprob:

    “This is a very depressing blog at times but keep up the great work everyone.”


    Like we say around here: Eventually, even the truth will make you happy.

  17. One concern is LCN DNA testing – it is accurate in terms of establishing a match, but because of its extreme sensitivity, it is much more sensitive to cross contamination, both in the lab and in collection. The result may be not so much as false positive, but a positive that lends itself to multiple explanations.

  18. I just came away from a matter where a neutral, tribunal appointed expert clearly decided to take the money and do a sloppy and lazy job, confirmation bias all the way – her report ignored key facts, instead she replaced them with assumptions. On examination she waffled and dodged every question, including questions from members of the tribunal – who were, I think, disgusted. I needed her report thrown out, and other than her reversing herself, sounding ridiculous was good enough for me. Stonewalling on simple fair questions destroyed her report, especially when they came from the tribunal that proposed and appointed her.

    Sad really, I had high hopes of the independent expert approach – I suppose we agreed if the wrong person.

  19. I worked arson cases both in the criminal and civil justice system. I have seen the craft[I never considered it science] evolve over the last 35 years. As w/ any craft, the result is determined by the skill of the craftsman. Hire a ham n’ egger carpenter, you get a ham n’ egger house. Hire a skilled craftsman, you get not only a structurally sound house, but every detail is perfect.

    The “science” of arson investigation has been under attack in the past 5-7 years. Some of it is warranted. The local Madison paper did a 5 segment expose on arson cases. Most of the cases they analyzed were criminal cases. I was aware of one of them but did not work it. They made me skeptical ofr the entire piece by the way they covered a civil case I worked involving an arson for profit case.

    There was a very good arson investigator on this case American Family Insurance v Plucinski. In arson for profit cases, I do the background and statements. Having worked criminal arson cases I know just how DIFFICULT they are to prove. Civilly, the burden of proof is the lower preponderance. I won’t give you the blow by blow on this civil case but the evidence was OVERWHELMING! The couple were in dire financial straits. The red flags of personal effects and pets were removed in this incredibly fast, fully involved fire. I said to the attorney who hired me we could prove this case criminally. It went to trial. It was a 5 day one and the jury came back in an hour for us.

    Again, I have NEVER considered arson investigation science, but craft. There are ham n’ eggers doing these investigations and most of them are government investigators. You don’t last long in private practice if you’re not good. There’s too much money @ stake. Weed out the bad investigators. The shame is the vast majority are government employees, protected by civil service, working CRIMINAL cases.

  20. Darren,
    It all boils down to a matter of money. Most private pay litigants, both civil and criminal, don’t have money to pay for the level of competence required. Public defender offices operate on a shoestring.

    I am working on a project involving the issue of out of state experts being unable to travel to states where they may not be licensed. State licensing boards throw up unbelievable roadblocks for lawyers needing to bring in the best experts, because the experts fear prosecution for practicing without a license.

  21. This is interesting in that willinghham was rushed to death….. Perry would not stay the execution even based upon the information presented…. The legislature passed a bill for a special commission…. It did its investigation….. On the day that the results were to be released…. The governor called for the resignation of those he appointed to the board…. There was not a majority and the report was delayed in release….. Presumably because of his bid for president….. He did not want scathing evidence that showed the man was innocent….. Another bone of contention I have with Texas and its final analysis…. Is that the coroner does not have to be anyone familiar with the medical profession or how the human body works….

  22. Charlton,

    A number of state limit the amount of money that a public defender can spend on a case…… That’s under review in a couple of states I understand…..

  23. Theater Goon:

    Yes there are unrealistic expectations on what a lab can do for investigation purposes. The public, largely, is convinced just as you mentioned. Since the CSI program became popular frequently it was expected in crime investigation by victims especially anything could be solved and it led to much disappointment by them by police and other investigators.

  24. In these days of modern times, when you can’t tell the difference between the AC’s and the DC’s, you should only be surprised whenever you encounter someone who IS competent in their work. IS honest, and IS NOT lazy.

  25. Forensic science is an essential part of our crime units. Hope with the continuing advancement in technology and knowledge, we can eliminate mistakes from occurring.

Comments are closed.