Elegance is more than just a frill in life; it is one of the driving criteria behind survival.
- Douglas Hofstadter

It would seem, then, that we need to look beyond the merely 'technical' level if we wish to solve the Prisoner's Dilemma. However, this is not to say that rationality is no use to us but that, rather, our conventional understanding of what it means to be 'rational' is simply too narrow and needs broadening a little. Thus, we come back to the quote at the start of this article. Irrationality, says Douglas Hofstadter, is the 'square root' - ie, the cause - of all evil. Fair enough - but how can we distinguish, once and for all, what is rational from what is irrational? A possible answer lies in Hofstadter's concept of superrationality - that is, looking outside one's own decision and taking into account the decisions of others too, and consequently making the decision that one would hope they would also make. In other words, the 'superrationalist' thinks 'globally' - in the wider interest - rather than 'locally', simply with his/her own interest in mind.

It's my understanding that Hofstadterian superrationality is not generally
accepted within the game theory research community as a valid principle of
decision making. Do you have any information to the contrary, or some
other reason to think that it will be commonly used by transhumans?

You yourself articulated, very precisely, the structure underlying Hofstadterian superrationality: "Expected utility of a course of action is defined as the average of the utility function evaluated on each possible state of the multiverse, weighted by the probability of that state being the actual state if the course was chosen." The key precise phrasing is "weighted by the probability of that state being the actual state if the course was chosen". This view of decisionmaking is applicable to a timeless universe; it provides clear recommendations in the case of, e.g., Newcomb's Paradox.

Superrationality----------------

Not really intuition. I guess I have to go into detail after all.

I'll discuss a "toy model" of rational choice, it is instructive, but clearly not realistic.

The model will be as follows: I imagine two rational beings in two rooms, and there is a button on each wall. If one of the two presses the button, the other is killed. If they don't, they survive. If both of them press the button, one of them is killed at random.

What course of action maximizes each individuals probability of survival?

This seems like a question that has a definite answer, and it is surprising that when you do the analysis, you run into a problem. Well, I won't talk aboutit, I'll do it.

Irrationality is the square root of all evil...
- Douglas Hofstadter

The Mind's I
Douglas Hofstadter, Daniel Dennett
Copyright 1981
ISBN 0553345842

Godel, Escher, Bach: An Eternal Golden Braid
Douglas Hofstadter
Copyright 1979