Thursday, July 2, 2009

The Stockdale Paradox (and how it applies to life and trading)

I've been reading Jim C Collins' book Good to Great (Why some companies make the leap and others don't), which is a very well-reasoned book backed by lots of data.
The Stockdale Paradox was coined by Jim Collins, and was named after Vice Admiral James Stockdale, USN, who was the highest ranking US prisoner-of-war in the Vietnam War, and who was held for eight years (from 1965 to 1973) and tortured over twenty times. Most impressively he actually took command within the Hanoi Hilton (together with Brigadier Risner), and never allowed himself to be used by the Vietnamese for propaganda, even slitting his face on purpose.

During an interview with Admiral Stockdale, Jim Collins asked "Who didn't make it out?"

Admiral Stockdale's reply was enlightening:
"Oh that's easy; the optimists. They were the ones who said, 'We're going to be out by Christmas.' And Christmas would come, and Christmas would go.

Then they'd say, ' We're going to be out by Easter!' And Easter would come, and Easter would go.

And then Thanksgiving, and then it would be Christmas again. And they died of a broken heart."

He then ended with this:
This is a very important lesson.You must never confuse faith that you will prevail in the end - which you can never afford to lose - with the discipline to confront the most brutal facts of your current reality, whatever they might be.


The Stockdale Paradox is something that I have thought about many times before. What Admiral Stockdale called the optimists, I call them the unhealthy optimists, guilty of unhealthy optimism. How does unhealthy optimism manifest itself? It very often manifests itself in explicit and outcomes-focused positive self-talk, like "best training", "cutting edge", "best performing", "peerless" and other superlatives. You can see this in certain people (especially, in my experience, inferior managers) who talk about "positive thinking" and "positive self-talk"; arguably what these people are doing instead is positive denial. They refuse to confront reality, instead cling on desperately to their illusion of positivity.

A friend of mine not so recently remarked to me that she had tried positive thinking for a while, but gave up because it made her feel worse. I suspect that her positive thinking is what I call positive denial, and her reaction is exactly what happens when unhealthy optimism meets with brutal reality.

When you believe in your own bullshit, you end up in really deep shit, especially when you are finally confronted with the reality of the situation. This applies to organizations that believe their own propaganda, and to individuals who believe their own self-talk, and to both organizations and individuals who do not face up to reality.

The reverse of unhealthy optimism is healthy optimism. Healthy optimism is implicit: if you're good, you don't need to say it out. In fact, it is often impossible to spell it out without sounding like a broken record. Far more often, you would prefer to not say it out and just focus on what you need to do. This is something that can be trained: arguably the healthy optimism mindset is the basis of all sports training and self-improvement, allowing the mind to actually focus on feedback instead of feeding on emotional fears and internal dialogue.
Personally for me, the crystallizing moment between unhealthy and healthy optimism actually occurred during Basic Military training, during the final 24-km route march. After the fifth time of telling yourself "I can do it!", this self-talk begins to sound like what it really is: an empty promise to yourself. Internally it becomes very clear that there is a very sharp contrast between what your head is saying and what you are actually feeling: your rational voice is saying "I can do it!" but in actual fact you are despairing at the sheer pain.

Instead, what actually worked for me (especially since I developed a huge blister by the 4th kilometer) was to internalize the belief that I can do it, literally by saying to myself, "yes, you can do it, so shut up and do it" and then I just focused on the reality of the situation: that I will only finish this one step at a time. So I literally focused on one step.

Then the next. And so on, for the remaining 20-km with the blister.

It was the confrontation of the reality that allowed me to cope, because ultimately reality is what you experience. Interestingly enough, the memory of this has also given me the confidence to overcome other things that I have experienced before in my life. Healthy optimism begets more healthy optimism.

Specifically, in trading, the Stockdale paradox is the ideal trader's mindset: ideally a trader will never lose faith in his/her ability to learn the markets and to trade profitably, while confronting the realities that manifest in his losing trades.

Unhealthy optimism manifests itself (I think) in some of the following ways:
  1. A trader looks only at the opportunities, and overlooks the risk: a case of positive self-talk ("There is plenty of opportunity in the markets!") without focusing on the reality ("You could lose your pants in the markets too!")
  2. A trader adds to losing trades: again, a case of positive self-talk ("The market will turn! This is good value! I am right!") vs. market reality ("There go your pants again, thank you, O liquidity provider!")
  3. A trader looks perpetually for the perfect indicator, the perfect broker, the perfect market, and the perfect trading teacher who will teach one to be a perfect trader: this is less obvious a manifestation. Underlying this search for perfection is an optimism that such perfection actually exists, and that this panacea will be readily available to the trader looking for such a thing. Like the wishful thinking in the examples above, this is usually an evasion of self-control and self-responsibility. Usually this trader's self-confidence in the perfect indicator/broker/market/trading teacher lasts until the first big loss, after which the trader's mindset turns around from "this indicator/broker/market/teacher is perfect" to "damn this scheisse!!" Again, when wishful thinking meets brutal reality, brutal reality usually wins.
  4. A trader who trades without a plan, only with self-confidence: wishful thinking.
Undoubtedly everyone has gone through some of these flaws at some point or another. What differentiates the (eventual) winners from the losers is their ability to face up to reality, and to take steps to deal with reality. Often these steps are small baby-steps, but these are still very important. Many times, you will dance the Annoying Tango of Frustration ("one step forward, three steps backwards").

But ultimately, my belief is that the person who most readily faces up to reality, and who most actively seeks rational solutions to his/her reality will be the most likely to succeed in the longer term.