From the Titelbaum book on Bayesian epistemology I have mentioned here before. There is a nice explanation in a footnote near the end about "observation selection bias" as a ubiquitous problem in statistics and, really, in the understanding of probability. The explanation is illustrated with a World War II throwback. US War Department statisticians observed in 1943 that bombers returning to London from their flights over occupied Europe generally had more bullet holes in their fuselage than in their engine. Somebody drew the conclusion that the German fire tended to hit the fuselage, and that there should be extra plating there. Such a reinforcement of the fuselage was not a decision to be taken lightly. Reinforcement adds weight. Heavier aircraft are less maneuverable, have a lesser range, etc. So Abraham Wald, a Hungarian Jew known before the war for econometrics research, who at this point was working at Columbia with the Statistical Research Group, cons...
Posts
My random reading in matters mathematical has brought me to Simpson's Paradox, the idea that a clear trend in a group of data can disappear, or even reverse, when different groups of data that taken severally exhibit the same trend are aggregated. It is named for Edward Simpson, who described the effect in "The Interpretation of Interaction in Contingency Tables" in the Journal of the Royal Statistical Society in 1951. Colin Blyth rediscovered this paper and gave the phenomenon the name Simpson's Paradox in a 1972 paper. It was once referenced in an episode of the prime-time cartoon The Simpsons because ... well, the name. Wikipedia has a good discussion. This is not really a "paradox" at all, though the name will probably stick. It wouldn't impress Zeno, or whoever started the thing about the lying Cretan. But the phrase "Simpson's odd-seeming phenomenon" doesn't seem very resonant. Good brief discussion in pp. 70 - 72 of Titel...
I have mentioned Mayo and her book, STATISTICAL INFERENCE AND SEVERE TESTING before. Here I come back to it to provide you with some fruits of my readings therein. Duhem's problem. Pierre Duhem said that experimental falsification doesn't get us very far in physics because the error that explains the surprising experimental result could invariably have come from any of a number of related postulates. The surprising result only tells us that ONE of them is wrong. "The only thing the experiment teaches us is ... there is at least one error; but, where this error lies is just what it does not tell us." [a 1954 book.] Quine said much the same, though he didn't think the idea restricted to physics. Popper replied, in CONJECTURES AND REFUTATIONS (1962). "We can be reasonably successful in attributing our refutations to definite portions of the theoretical maze. (For we are reasonably successful in this -- a fact which remains inexplicable for one who...
I've discussed both Taleb and Pinker in this blog at various times, but I've never brought the two of them together. Yet reality has brought them together for me. One of Pinker's books for a broad popular audience is called THE BETTER ANGELS OF OUR NATURE (2011). It argues that the world is making progress toward peace. Violence still occurs, but it becomes less common century after century, peaceful methods of the resolution of differences advancing. An example: it is almost possible to imagine France and Germany on opposite sides of a war in the next fifty years or so. Yet not TOO long ago as meta-historians reckon time they were on opposite sides of three different wars within about 3/4s of a century. What has happened? See Pinker's title for his answer -- the better angels of our nature are slowly prevailing and giving us a better world. Taleb has been sharply critical of this book. With the help of statistican Pasquale Cirillo, Taleb wrote a couple of es...
Okay, I think I have a fix on this. Suppose we want to test our hypothesis that otherwise comparable firms with high levels of indebtedness cut back on expenditures desired by employees in circumstances where their not-so-indebted cousins would not. (For a further explanation of that hypothesis, review Part I of this discussion from last week.) How do we do it? We'll assume that we haven't found a smoking gun memo in which the company's Treasurer writes to the CEO and says, "we can't afford those darn safety vests any longer. Tell Human Resources to stop buying them so we can make the interest payments!" Assume we're looking at circumstantial evidence. What counts as evidence? What we can't do is simply say: firm X buys safety vests for its employees and is mostly equity financed. Firm Y doesn't and isn't. No matter how many Xs and Ys we find compliant with our hypothesis, we will still have only correlation, not causation. The arrow of...
One of the Dilbert cartoons that came up on my day-to-day desk calendar recently offers a valuable lesson. Correlation does not imply causation. Usually, the pointy-haired boss plays the stooge in his exchanges with Dilbert, but in this case its the boss who brings the point home. Dilbert tells him, "Studies show that companies with a high level of trust in employees also perform the best." Dilbert is presumably hoping that the boss will infer from this correlation of performance and trust that there is a causative link, and that the boss ought to trust Dilbert more, thereby allowing their common employer to reap the rewards. Instead, the boss acutely responds, "If you ever start performing well, I'll trust you, too." Ah, yes, one of the problems with that sort of causal argument (but only one of them) is that the causal arrow is easily reversed. Good one, boss.
Continuing yesterday's train of thought... Strawson (pictured here) observes that there are differences between the young Hume and the mature Hume: the fellow who wrote the TREATISE (1740) and the man who wrote the ENQUIRY (1748). To some extent, at least, the positivistic reading of Hume on causation follows from some dramatic overstatements in the earlier book, overstatements that Hume later regretted. In the latter book he said, "the positive air, which prevails in that book, and which may be imputed to the ardor of youth, so much displeases me, that I have no patience to review it." Strawson also sometimes quotes from DIALOGUES ON NATURAL RELIGION, which was written subsequent to the ENQUIRY by at least another couple of years, and specifically of course the words of Philo, the Humean mouthpiece there. At any rate: Strawson reads Hume, at all stages of his working life, in such a way as to connect the issue of causation/Causation to the issue of the independen...
Greenpeace USA is making the following claim, "The United States now has more people working in the solar industry than in coal mining." This claim is used to bolster the further idea that the U.S. is moving away from carbon-based and earth-extractive sources of energy, i.e. that Greenpeace is on the winning side of history and everyone might as well clamber on board the bandwagon. As for the big claim ... whatever. As for the comparative employment figures from two industries, I'll make two observations here: 1) I haven't looked into it sufficiently to say whether I believe it -- there are different ways of defining who belongs to what "industry" after all, so the question might be complicated. Greenpeace takes its numbers on the coal industry from one source and its numbers for the solar industry from another source, so I have a suspicion that this is an apples-and-oranges thing. 2) Even if true, I'm not sure what it means in term...
"Given how ubiquitous and central the CBO has become to government policy, it's surprising how recently it was created [in 1974]....The CBO is responsible for analyzing the effects of spending and of commitments to spend on the long-term federal budget. As the federal government began to rely on deficit spending in the 1980s, the Congressional Budget Office was anointed the official keeper of budget statistics, along with the OMB. This was formalized in 1985, when the rising public anxiety about mounting deficits -- a foreshadowing of much higher levels of concern after 2008 --le to the passage of a bipartisan bill to keep the deficit in check. "...One of the core functions of the CBO is to estimate how much the government will spend on health care and Social Security. Though the CBO is staffed by nonpartisan economists and accountants, its analysis is based almost entirely on guesstimates about the future rate of inflation, the future rate of growth, and the future...
Last Sunday I made some fatuous comments about the NCAA basketball tournament in an effort to illustrate the statistical concept of regression to the mean. I did so by inventing a basketball concept of regression to the chalk. As a discussion of the statistical point, that post holds up. As a discussion of the NCAA tournament, I am afraid it already looks pathetic. when I made out my own brackets, as the tournament was getting underway, I ended up with a final four of Syracuse, Iowa State, Arizona, and Michigan. You can judge my b-ball predicting prowess from the fact that not one of those teams is in the final four. The actual line-up is: Florida, Wisconsin, Connecticut, and Kentucky. That's not so embarrassing, really. what is worse is that the results thus far conspicuously fail to illustrate the idea of regression to the mean/chalk. The "regression" should certainly have eliminated the Wildcats and Huskies by now, if it were a fact. As the musical version of...
Did you fill out a bracket this year for the NCAA tournament, dear reader? I did. I'll report the following three points about the experience: I wasn't even close to winning the billion dollars; I didn't try to predict any of the "first round" games; and I've decided that tournament results exhibit a definite pattern, which I have come to think of as "regression to the chalk." The first of those three points needs no particular explanation. As to the first round, I'll offer this. I am still sentimentally attached to the idea of a 64-team field. Nowadays, though, the Powers that Be have mandated that 68 teams be involved. So they have four "first round" games on Tuesday and Wednesday of the first week of the tournament to eliminate four of those 68. By Thursday morning, we're back to a 64 team field, and that is the best time to fill out the form, IMHO. The "second round" is exactly what the first round was in d...
P(H/D) = [P(D/H) x P(H)] ÷ P(L) Thomas Bayes, a mid-eighteenth century English mathematician, dead since 1761, performed magic the ramifications of which continue to unfold, and in directions of interest to the world of finance today. But I won't really get into the applications here. I'll just state the main point associated wit...
Today is my birthday so in a life-of-the-party spirit remarkable for a 55 year old geezer like myself, I will tell a joke. A chemist, a biologist, and a statistician are out hunting deer together. The chemist shoots first, and misses the deer five feet to the right. The biologist shoots next, and misses the deer five feet to the left. The statistician says, "I'm not going to shoot at all. On average, we've already bagged that deer." Okay, I didn't promise I'd tell a good joke. If you've got a funnier joke in which a statistician delivers the punchline, let me know.
One of the central elements in modern finance theory is that of a risk-return tradeoff. The idea is simply that investors are risk averse, and accordingly must be paid to incur risk. T here is, then, a constant trade-off in the investment world: safe investments carry low return, high-return investments aren’t so safe. Fortunately, this conforms with almost everyone’s intuitions. What exactly is risk, though? Yes, we have an intuitive idea. The guy jumping out of an airplane is taking a risk that the appreciative audience standing on firm ground below is not. Further, if he has neglected to check his gear properly he is taking an extra, unwarranted, risk. But we can be a good deal more specific about what the word means in the world of investments. It means the size of the standard deviation of return. It means the width of that bell curve we've discussed in earlier posts. Standard Deviation A standard deviation is the “average distance from ...
Koksvik is especially concerned with the argument that interactionist dualism can't work because for it to do so would violation the law of the conservation of energy (supplemented perhaps by the law of the conservation of angular momentum). The idea behind this argument is that IF my thoughts cause me to do something I wouldn't otherwise do, then they must do so by exerting energy. Energy after all is that which does work, that which which moves matter around. Thus, if interactionism is true, energy is leaking into the phenomenal world from some trans-phenomenal source. Surely this would violate that law. If we're going to say, "so much the worse for the law then!" we can do so. Laws sometimes have to be modified and sometimes abandoned. Still, these leaks of energy into the phenomenal world should be in principle detectable, and anyone arguing for interaction should be able to make a case that they exist, that the law is violated, before arguing for ...