Skip to main content

Observation selection bias




From the Titelbaum book on Bayesian epistemology I have mentioned here before.  There is a nice explanation in a footnote near the end about "observation selection bias" as a ubiquitous problem in statistics and, really, in the understanding of probability. The explanation is illustrated with a World War II throwback.

US War Department statisticians observed in 1943 that bombers returning to London from their flights over occupied Europe generally had more bullet holes in their fuselage than in their engine. Somebody drew the conclusion that the German fire tended to hit the fuselage, and that there should be extra plating there.  

Such a reinforcement of the fuselage was not a decision to be taken lightly.  Reinforcement adds weight. Heavier aircraft are less maneuverable, have a lesser range, etc. 

So Abraham Wald, a Hungarian Jew known before the war for econometrics research, who at this point was working at Columbia with the Statistical Research Group, consulting with the military, was shown the data on the frequency of fuselage holes. And he argued successfully against any such reinforcement. 

He said: Save the reinforcement for the engines, where the bullet holes aren't. Wald not only noticed the selection bias in this data, he reasoned from it. Wald said that the data shows that the RETURNING planes are those with the holes in the fuselage.  The implication is not that the German fire finds the fuselage more easily but that the planes with holes in the engine don't return. It is an observation about the vulnerability of the engine, and the survivability of hits to the fuselage. 

That turned the matter around neatly. It is a great story.    


Labels:

Comments

  1. Paul D. Van PeltNovember 7, 2024 at 9:23 AM

    Some of my former co-workers were attorneys. A comment, favored in discussions of strategy and procedures, was "as a practical matter". Sounds a bit along the line of where to strengthen armor plating on military aircraft. As a practical matter, it is preferable to have costly hardware return from missions, hopefully with as many surviving crew as possible, so that both ship and crew members may live to fight again and not end up as prisoners of war. For that to occur, engines must work. Strategy says, cut losses, wherever practicable, while maximizing human and equipment resources. The strategist mentioned was going with what made sense.

    ReplyDelete

Post a Comment

Popular posts from this blog

A Story About Coleridge

This is a quote from a memoir by Dorothy Wordsworth, reflecting on a trip she took with two famous poets, her brother, William Wordsworth, and their similarly gifted companion, Samuel Taylor Coleridge.   We sat upon a bench, placed for the sake of one of these views, whence we looked down upon the waterfall, and over the open country ... A lady and gentleman, more expeditious tourists than ourselves, came to the spot; they left us at the seat, and we found them again at another station above the Falls. Coleridge, who is always good-natured enough to enter into conversation with anybody whom he meets in his way, began to talk with the gentleman, who observed that it was a majestic waterfall. Coleridge was delighted with the accuracy of the epithet, particularly as he had been settling in his own mind the precise meaning of the words grand, majestic, sublime, etc., and had discussed the subject with William at some length the day before. “Yes, sir,” says Coleridge, “it is a maj...
9 comments
Read more

Searle: The Chinese Room

John Searle has become the object of accusations of improper conduct. These accusations even have some people in the world of academic philosophy saying that instructors in that world should try to avoid teaching Searle's views. That is an odd contention, and has given rise to heated exchanges in certain corners of the blogosphere.  At Leiter Reports, I encountered a comment from someone describing himself as "grad student drop out." GSDO said: " This is a side question (and not at all an attempt to answer the question BL posed): How important is John Searle's work? Are people still working on speech act theory or is that just another dead end in the history of 20th century philosophy? My impression is that his reputation is somewhat inflated from all of his speaking engagements and NYRoB reviews. The Chinese room argument is a classic, but is there much more to his work than that?" I took it upon myself to answer that on LR. But here I'll tak...
17 comments
Read more

Recent Controversies Involving Nassim Taleb, Part I

I've written about Nassim Taleb on earlier occasions in this blog. I'll let you do the search yourself, dear reader, for the full background. The short answer to the question "who is Taleb?" is this: he is a 57 year old man born in Lebanon, educated in France, who has been both a hedge fund manager and a derivatives trader. He retired from active participation from the financial world sometime between 2004 and 2006, and has been a full-time writer and provocateur ever since. Taleb's writings for the general public began where one might expect -- in the field where he had made his money -- and he explained certain financial issues to a broad audiences in a very dramatic non-technical way. Since then, he has widened has fields of study, writing about just about everything, applying the intellectual tools he honed in that earlier work. As you might have gather from the above, I respect Taleb, though I have sometimes been critical of him when my own writing ab...
4 comments
Read more