A now forgotten turn-of-the-20th-century song, “Do You Believe Your Baby or Your Eyes?,” apparently gave birth to one of that century’s most famous witticisms. If you are caught red-handed in an undeniably awkward, embarrassing, or illicit situation, one way out is to beg for trust. No need to confess or apologize or be humiliated.
Instead, be audacious. Deny. Ask rhetorically: “Who you gonna believe, me or your lying eyes?”
Variations of that classic line can be found in the Marx Brothers’ nihilistic comedy Duck Soup, a 1948 column by the syndicated advice giver Dorothy Dix, and more recently by Richard Pryor and Cher.
Much of what people claim to know rests on trust and deference. Most of us rely heavily on those who claim special expertise, whether the topic is COVID or climate change.
But in our age of disinformation, photo, video, and audio manipulation, doxxing, and big lies, how can we possibly be sure that anything is unambiguously true? Given the inaccurate, ever-shifting advice provided by the CDC and the World Health Organization over the pandemic’s course, it’s easy to conclude that Nietzsche right when he wrote in his notebook around 1886: “There are no facts, only interpretations”?
The comedian, television host, and political commentator Stephen Colbert coined the word truthiness to describe the public’s tendency to confuse wishful thought and opinions rooted in fantasy as accurate, irrespective of logic or factual evidence. On many issues, our identity determines our personal truth. As one newspaper headline put it: “Belonging is stronger than facts.”
The 2016 presidential campus touched off an explosion of books and articles on disinformation, as well as widespread calls to combat misinformation and the supposedly growing contempt for verified facts in today’s post-truth moment. Less than a year ago, former US Secretary of Education Margaret Spelling and a collaborator declared: “Just as young people should be taught coding, they must be taught the decoding of news and information as a prerequisite of informed citizenship.”
Of all the recent books on propaganda, disinformation, fake news, and misinformation, two examples stand out: Frederick Schauer’s The Proof: Uses of Evidence in Law, Politics, and Everything Else and Jerry Z. Muller’s The Tyranny of Metrics.
Muller’s book looks at the use and abuse of quantitative measures by business, medicine, education, and government, and shows how the growing fixation upon data analytics, while sometimes beneficial, can also result in harm. You’ll recall the textbook example: Body counts during the Vietnam war. In his classic account of the follies of American leadership in Vietnam, David Halberstam recounts an occasion when former Supreme Court Justice Arthur Goldberg asked a Pentagon spokesman the number of Viet Cong and North Vietnamese deaths during the 1968 Tet Offensive and the ratio of wounded to killed (which was about 4.45 to 1). If the Pentagon body counts were correct, then all of the enemy’s troops were casualties. The absurdity of the relying on body counts to measure battlefield success was laid bare.
The Tyranny of Metrics offers a number of striking examples about how quantitative measures, stripped of context and interpreted without critical and ethical judgment, can not only mislead but inflict harm, distorting behavior, squandering time, promoting a short-term focus, discouraging innovation, and degrading performance. Here are some of Muller’s examples:
- Surgical scorecards may actually increase patient deaths by encouraging surgeons to refuse to operate on high-risk patients.
- Medical checklists can lead physicians to substitute a list of procedures for experience-informed judgment.
- The police sometimes focus on petty crimes, often drug-related, to beef up statistics in the war on crime.
While it’s true as the management guru Peter Drucker declared that “You can't improve what you don't measure,” even accurate data can lead us astray. Statistics can be cherry picked. Books can be cooked. Numbers can be gamed and fudged and manipulated. Numbers can lie. Teachers can teach to the test, or, believe it or not, lower standards to meet predetermined educational goals.
Muller’s book seeks to refute supports several well-known axioms:
- Goodhart's and Campbell’s law: "When a measure becomes a target, it ceases to be a good measure".
- The False Cause Fallacy: Confusing correlation with causation.
- McNamara Fallacy: Relying solely on metrics in complex situations leads to losing sight of the big picture.
- Anscombe's quartet: Very different data distributions can produce identical descriptive statistics.
As we know from the college rankings, higher education isn’t immune from the tyranny of metrics, and the incentives to distort or omit data. Some law schools admit less qualified students on a part-time or probationary status to exclude them from the rankings that look exclusively at full-time students. 4-year institutions do something similar when they enroll “weaker” students in a general studies unit or require them to enter during the Spring or Summer terms. Here are some of Muller’s other arguments:
- When productivity or merit is measured in terms of the number of publications or citations, faculty members will publish more, but quality may diminish.
- When learning is assessed largely through multiple choice tests, “students too often learn test-taking strategies rather than substantive knowledge” and frequently fail to achieve conceptual understanding.
- When more students earn bachelor’s degrees, the value of the degree as a signaling device declines.
- When colleges are evaluated based on enrollment or graduation rates, the institutions respond rationally, by reducing admissions or grading standards; when, in contrast, colleges are assessed in terms of value added, the most selective and richly resourced institutions or those focused most heavily on engineering and business, predominate.
Let’s take Muller’s message to heart:
“There are things that can be measured. There are things that are worth measuring. But what can be measured is not always what is worth measuring; what gets measured may have no relationship to what we really want to know. The costs of measuring may be greater than the benefits. The things that get measured may draw effort away from the things we really care about. And measurement may provide us with distorted knowledge—knowledge that seems solid but is actually deceptive."
In other words, don’t use data mindlessly, and always remember, measurement isn’t the same thing as understanding.
Perhaps you’ve heard of “Newton’s Flaming Laser Sword, a concept devised by the Australian mathematician Michael Adler. Put far too simply, it says that if a claim has no observable consequences, it should be dismissed. I don’t think Adler’s Razor can be readily applied to teaching, because much of what’s most valuable in an instructor isn’t easily measured. Passion for a subject, Infectious enthusiasm. A connecting with students. Dedication to bringing students to success.
Let’s turn next to Frederick Schauer’s thoughtful reflections on:
- What constitutes a fact, which can be empirical and therefore verifiable or evaluative and likely contested.
- How laypersons can be assess the validity of a particular truth claim that is outside their area of expertise.
- The extent to which non-specialists can rely on an expert consensus.
- How to decide what policy decisions or prescriptions should be drawn from science or data
As a law professor and a former defense attorney, it is not surprising that Schauer approaches issues of evidence through a legal lens, focusing on questions of inference, relevance, reliability, probability, burden of proof, and degrees of guilt. The book devotes several chapters to the devices that lawyers use to evaluate and challenge the weight and strength of evidence and testimony, including the testimony from dueling experts. His essential argument is that in assessing evidence, we need, first of all, to recognize that evidence comes in degrees (from weak to strong, from extraneous to relevant) and that probability, the likelihood that the evidence or testimony is accurate, matters.
The book tackles some extraordinarily controversial topics:
- What should we think when a work of art that had been authenticated by various tests and museum experts and art historians, turns out to be the work of a self-confessed forger?
- What practical difference does it make between applying a “preponderance of the evidence” standard of proof or a “clear and convincing” in sexual disciplinary proceedings against those accused of sexual violence?
- When scientists disagree, for example, about the dangers of Genetically Modified Organisms, who should non-scientists trust and on what basis?
Schauer likens his approach to Bayesian statistics. He is interested in whether a particular fact, finding, or piece of evidence tends to support or undercut a particular hypothesis. Two phrases sum up his argument: “Uncertainty in factual judgments is an inevitable aspect of the human condition” and “all evidence involves statistical inference.”
But, an opposing lawyer might object, observing that the courts are averse to the use of statistical evidence, at least when applied to individual defendants, as opposed to cases involving corporations or municipalities or states accused of discrimination. Schauer suggests that the courts’ resistance to the use of statistical generalizations seeks to compel prosecutors to come up with stronger evidence.
Much of our knowledge is secondhand. We rely heavily on various kinds of testimony (for example, about when and where we were born), and we need to learn, Schauer argues, to do what juries do: calibrate the credibility and trustworthiness of that testimony. This requires us to understand whether testimony is shaded, embellished, fudged, or the product of an erroneous perception or an honest mistake.
Schauer has fascinating things to say about the reliability of eyewitnesses, hearsay, and lie detectors, the efficacy of honor codes and courtroom oathtaking, and the trustworthiness of letters of recommendation. Readers will also learn a great deal about paltering, that is, attempting to decisive without saying anything literally false, and the debate over groupthink versus the collective wisdom of crowds. Especially interesting is his discussion of the precautionary principle, a legal concept popular in Western Europe that holds that when there is evidence that a practice or substance presents a plausible, but uncertain, risk, it should be banned.
My takeaway from the Schauer and Muller books is that everyone need to think more like – ta-da -- a humanist. We need to learn how to weigh evidence, draw inferences, and interrogate claims. We also need to become more mindful, alert, observant, and emotionally intelligent. As we navigate postmodern environments laced with uncertainty, ambiguity, and indeterminacy, our most reliable sextants and compasses are informed judgment and discernment, the very qualities that the humanities nurtures.
Steven Mintz is professor of history at the University of Texas at Austin.
from Inside Higher Ed https://ift.tt/RGzylwrvia IFTTT
Comments
Post a Comment