The UFO Iconoclast(s)

Friday, December 10, 2010

UFO Truth? Not so fast....

ny12-13.jpg

The December 13th, 2010 Issue of The New Yorker has an article that all UFO mavens would do well to absorb.

It’s “The Truth Wears Off” by Jonah Lehrer [Page 52 ff.].

Lehrer recounts how science (and researchers) get snookered by “The Decline Effect” –the measurable drop off of one-time observations and data that, when first obtained, seemed invincible as proof of various phenomenon, but end up on second (or third) observations to be considerably less that originally measured.

Rhine’s studies, at Duke, of ESP are mentioned, as are pharmacological, ecological, psychological, biological studies.

jbrhine.jpg

The gist of the article is that studies are intrinsically flawed but why is still open to question.

Some relevant quotes from the piece, which UFO debaters should heed, include:

“Most of the time, scientists know what results they want and that can influence the results they get.” [Page 52]

“Asking people to put their perceptions into words led to dramatic decreases in performance.” [Page 53]

(Re: Rhine’s studies/experiments) “What he wanted to know was whether the images [from Zener card experiments] that got a second showing were more likely to have been identified the first time around. Could subsequent exposure have somehow influenced the initial results? Could the effect be the cause?” [Page 54]

zener.jpg

“The extrasensory powers of…subjects didn’t decline – they were simply an illusion that vanished over time.” [Page 54]

“…the ‘decline effect’ deserves more attention: its ubiquity seems to violate the laws of statistics.” [Page 54]

“…[an] author might publish several critical papers, which distort his analysis.” [Page 54]

“…some – perhaps many – cherished generalities are at best exaggerated in their…significance and at worst a collective illusion nurtured by strong a-priori beliefs [are] often repeated.” [Page 55]

“…the problem seems to be one of subtle omissions and unconscious misperceptions, as researchers struggle to make sense of their results.” [Page 55]

“…act[s] of measurement [are] going to be vulnerable to all sorts of perception biases.” [Page 56]

“The problem of selective reporting is rooted in a fundamental cognitive flaw, which is that we like proving ourselves right an hate being wrong.” [Page 56]

“…after a claim has been systematically disproven…you still see some stubborn researchers citing the first few studies that show a strong effect. They really want to believe that it’s true.” [Page 56]

“Every researcher should have to spell out, in advance, how many subjects they’re going to use, and what exactly they’re testing, and what constitutes a sufficient level of proof. We have the tools to be much more transparent about our experiments.” [Page 56]

“Although…reforms would mitigate the dangers of publication bias and selective reporting, they still wouldn’t erase the decline effect. This is largely because scientific research will always be shadowed by a force that can’t be curbed, only contained: sheer randomness.” [Page 56]

“…a lot of extraordinary scientific data are nothing but noise.” [Page 57]

“…dramatic findings are also the most likely to get published in prestigious journals…” [Page 57]

“…the decline effect is actually a decline of illusion.” [Page 57]

“While Karl Popper imagined falsification occurring with a single, definitive experiment – Galileo refuted Aristotelian mechanics in an afternoon – the process turns out to be much messier than that. Many scientific theories continue to be considered true even after failing numerous experimental tests.” [Page 57]

galileo10.jpg

“Even the law of gravity hasn’t been perfect at predicting real-world phenomena. In one test, physicists measuring gravity…in the Nevada desert found a two-and-a-half-per-cent
discrepancy between the theoretical predictions and the actual data. Despite these findings…The law of gravity remains the same.” [Page 57]

“Such anomalies demonstrate the slipperiness of empiricism.” [Page 57]

“Although many scientific ideas generate conflicting results and suffer from falling effect sizes, they continue to get cited in textbooks…Why? Because these ideas seem true. Because they make sense. Because we can’t bear to let them go. And this is why the decline effect is so troubling. Not because it reveals the human fallibility of science, in which data are tweaked and beliefs shape perceptions…And not because it reveals that many of our most exciting theories are fleeting fads and soon will be rejected.” [Page 57]

The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that is often not the case. Just because an idea is true doesn’t mean it can be proved. An just because an idea can be proved doesn’t mean it’s true.” [Page 57]

“When he experiments are done, we still have to choose what to believe.” [End of article]

Scientists and (some) “ufologists” like to think they have a lock on truth. But as Mr. Lehrer’s New Yorker article shows, truth is as elusive as ever, and nothing is certain.

8 Comments:

  • "The only truth is that there is no truth. Beware even of this truth."

    By Blogger Adam, at Friday, December 10, 2010  

  • I will have to read this article when I get the chance. Good find. I think among those scientific disciplines where observation and measurability are less cut-and-dried, we find answers to be far less certain and harder to attain. Psychology (parapsychology) and certain aspects of physics (astro and quantum) where direct observation and measurements can't be made and instead effects are observed and scientists have to hypothesize backward to the cause are other examples.

    By Blogger Cullan Hudson, at Saturday, December 11, 2010  

  • Cullan:

    I bet you can find the article online at the New Yorker web-site.

    It's revelatory in a number of ways.

    RR

    By Blogger RRRGroup, at Saturday, December 11, 2010  

  • I read the author's article at Wired about the New Yorker article. He quoted the part about the studies of anatomical symmetries and sexual selection.

    "By 1998, when there were twelve additional investigations of fluctuating asymmetry, only a third of them confirmed the theory. Worse still, even the studies that yielded some positive result showed a steadily declining effect size. Between 1992 and 1997, the average effect size shrank by eighty per cent."

    I wonder what the story is now post-1997. Does the decline continue until zero, or like Zeno's arrow, can we parse it into finer bits and never get there?

    Between "1992-1997" did mate selection decline as well? If not, then one can suppose that the decline of selection for symmetry was accompanied by a like increase in selection for asymmetry.

    I wonder if the scientists who are baffled by the decline effect realize that (to quote my wife) "getting there is half the journey".



    Regards,

    Don

    By Blogger Sourcerer, at Saturday, December 11, 2010  

  • Don:

    You raise some interesting questions.

    What the article(s) indicate is that data measurement and observation are a mish-mash no matter how one parses the information.

    The point is, I think, that truth is never a black and white thing.

    We are in a world composed of innate mystery and inherent chaos.

    We call it, euphemistically, The Game of God.

    (And we'll have more to do with that concept after the first of the year.)

    RR

    By Blogger RRRGroup, at Saturday, December 11, 2010  

  • RR: "What the article(s) indicate is that data measurement and observation are a mish-mash no matter how one parses the information."

    Stephen Salant, an economics professor at UM, said he had been researching the Hiss case since high school, but had not published anything until this year...

    "But I did not need to publish my own research on the case to advance professionally and that gave me the luxury of waiting until I had something significant to say."

    Scientists on a career path have to do research and publish, even if they don't have something significant to say. The best one can hope for is that the data sets collected are accurate, so that they can be used in other endevors.

    In the movie Absence of Malice, a reporter recites the interviewee's involvement in a big story. The interviewee doesn't respond.

    "That's true, isn't it?", the reporter asks.

    The interviewee replies, "No. But it's accurate".

    Regards,

    Don

    By Blogger Sourcerer, at Saturday, December 11, 2010  

  • Don:

    The Publish or Perish mantle offers scientists and professors a requirement that encourages slipshod research, something that is endemic to "ufology" without that P or P rubric being in place.

    As an aside, I just this past week mentioned the Hiss/Chambers relationship with Tony Bragalia, as part of another scenario that causes lies and distortions when one is beset by interviewers.

    When one is judged by others for waywardness, they will assume the lesser of two sins (or crimes) to prevent an adjudication against them for the more serious of the two sins afflicting them, as Dostoevsky illuminated, so presciently, in "Crime and Punishment."

    This is grist for some UFO events, in a way.

    RR

    By Blogger RRRGroup, at Saturday, December 11, 2010  

  • @ Rich:

    "I bet you can find the article online at the New Yorker web-site."

    Um, no, not unless you are a paid subscriber--the article is behind a paywall.

    By Blogger steve sawyer, at Tuesday, December 21, 2010  

Post a Comment

<< Home