Welcome to European Tribune. It's gone a bit quiet around here these days, but it's still going.
Display:
Thanks for the diary - it was very interesting. I had read about Taleb but hadn't read anything of his and the piece in Edge was indeed thought-provoking.

Although he's properly attacking the right beasts, some thing about his epistemology leaves me unconvinced:

- Take for example what he says about the "inverse problem":

"Given a set of observations, plenty of statistical distributions can correspond to the exact same realizations--each would extrapolate differently outside the set of events on which it was derived."

True obviously, but it does seem that he subscribes to the idea that theory building (and a model that isn't grounded on theory is numerology IMHO) is basically fitting curves to data points. Now my prior life as a physicist makes me very hostile to this idea. There is a very lively interplay between theory, model building and producing, reading and interpreting  data. In fact I'm not entirely sure that there exist quantitative data that one can make much sense of without (some sort of) a theory.

This attitude is evident in his opening paragraph. Surely a statistician's hubris is behind the claim that:

Statistical and applied probabilistic knowledge is the core of knowledge

...unless one defines knowledge's "core" in a trivial way. In fact this sentence makes sense only in the light of a conviction that "the empirical method" = "data fitting". This is a fallacy recently expounded in Wired magazine and discussed (quite disapprovingly) in the Edge, and in various physics blogs (i.e. backreaction, cosmic variance.)

Now I would argue that a lot of what Taleb is criticizing is part of this "dataset as theory" mentality, permeating a whole branch of financial analysis. But no one sane, expects this method to predict anything but the routine. Taleb is obviously right that you can't extrapolate from the routine to the extreme like this, but if you define extreme as "very infrequent", astronomers can predict the sun turning into a Red Giant a billion years from now; volcanologists can read the signs that a  volcano is preparing for an explosion sometimes months in advance; meteorologists can spot a hurricane forming even as it is still some disorganized low near Cape Verde. Ditto for markets, some have a more accurate view than others about dangerous policies: meltdowns and crashes (as opposed to cycles possibly) might or might not be intrinsic to capitalism, but it sure as hell seems evident that certain policies make them far more likely than others. So statistical inferences and extreme event distributions aside: the odds of a hurricane in the Gulf Coast in January are vastly smaller than those  of a hurricane in August. The odds that all the molecules in the air around me will spontaneously and catastrophically move themselves to the kitchen next door are practically zero. The probability that an economy built on toxic credit will be going "boom" if left unregulated is pretty much 1. I say this because his Dark Swan territory seems to be indifferent to the types of macroeconomic policies pursued. Are crashes equally (un)likely regardless of policy?

- No dataset is an island. Only a turkey could possibly believe that being fed means that "the human race cares about its welfare". A human has datasets of turkeys past who were fed only to be slaughtered. In fact knowledge of the world beyond the particular turkey would make such a claim improbable. Now I understand that the example is for illustration purposes, but there is a similarity with stock markets: crashing after an extravaganza of free and unregulated markets is not exactly a once in a millenium effect, and catastrophic anthropogenic global warming via feedback mechanisms is unprecedented in the history of this planet, but apparently not at all impossible if you're guided by events and data beyond the average global temperature timeseries.  Even though you can't place ballpark probability values on particular catastrophic outcomes, the information about the particular catastrophe (say a "sudden" increase in CO2(eq) in the atmosphere), is not guided by a single dataset but by a host of data, theories and speculations.

I should read the book though, it is quite possible that I'm missing something or misinterpreting...

The road of excess leads to the palace of wisdom - William Blake

by talos (mihalis at gmail dot com) on Wed Sep 24th, 2008 at 08:49:04 PM EST
Thanks for the compliment.

"Given a set of observations, plenty of statistical distributions can correspond to the exact same realizations--each would extrapolate differently outside the set of events on which it was derived."

True obviously, but it does seem that he subscribes to the idea that theory building (and a model that isn't grounded on theory is numerology IMHO) is basically fitting curves to data points.

I think there's a more technical meaning underlying his viewpoint, which is also relevant to the turkey problem. A fundamental idea in probabilistic finance is the idea of the change of probability measure. This is the crucial idea which allows calculations to be done.

Normally, a probability model/distribution is a kind of encoding of degrees of belief. (There's a famous result due to Cox to the effect that the only consistent way (with conventional logic) of modelling belief with a single number leads to Bayesian probability theory). In particular, the exact values of the probabilities matter a lot.

If two models are only slightly different, then it's possible to mathematically reweigh one model in terms of another (via likelihood ratios, radon-nikodym derivatives, Girsanov's formula...), which means that you can work within one belief framework, and in the end adjust the numerical predictions so that it's as if you had computed entirely within another set of beliefs.

For some kinds of calculations, such as those used when replicating portfolios, the end result only depends on the equivalence class of the beliefs, so it's common for finance people to work in a "risk neutral" belief system. Even though they don't know a good numerical model for the truth, they argue (rightly or wrongly) that the best "true" model belongs to the equivalence class of a simpler model, such as geometric Brownian Motion. So they calculate with BM in the knowledge that, had they worked with the true best model, they would have ended up with the same hedging rules. This is a huge advance, because the true values of the probabilities don't need to be understood exactly. You still need to estimate some parameters, but only "implied" ones rather than true predictive ones. And of course, you can't calculate everything, just those things that are invariant under change of measure.

At this level, what Taleb is getting at I think is that it's still hard to be sure that the true model is in fact equivalent, in the above sense, to the simpler models which allow calculations. It's the structural properties which matter. For example, Brownian motion is a continuous process without jumps, and that's a structural property. As long as the best "true" model doesn't have jumps, it might be equivalent to the BM model, but if you observe rare kinds of events such as a spectacular crash, then the BM model becomes incompatible, and you really should be looking for another family of risk neutral measures to work with. The heavy tails idea is one of those: BM doesn't have heavy tails, although other processes do.

This attitude is evident in his opening paragraph. Surely a statistician's hubris is behind the claim that:
Statistical and applied probabilistic knowledge is the core of knowledge
I agree with you, although I think much of this can be traced back in this case also to Taleb wanting to connect with various other pop science books that his readers would know about and respect. Later in the essay, he mentions Mandelbrot and Barabasi, which brings to mind fractals and networks. I also wouldn't be surprised if his "logic of science" is just a hat tip to Jaynes.

Yet in exhorting the theories of rare events, he misses other approaches. In the 70s, the big thing was catastrophe theory, which is a fully deterministic approach to rare events, although it's now out of style.

The reason I described EVT in detail was that it's one of those things I would have expected to show up somewhere in his essay. It's like when you open a new book and look at the index first, just to be sure that it contains the right mix of keywords.

--
$E(X_t|F_s) = X_s,\quad t > s$

by martingale on Wed Sep 24th, 2008 at 10:51:14 PM EST
[ Parent ]

Display:

Occasional Series