Is it Rational to be Bayesian Rational?

Most economists and many decision theorists equate the notion of rationality with Bayesian rationality. While the assumption that individuals actually are Bayesian rational has been largely disputed and is now virtually rejected, the conviction that Bayesianism defines the normative standard of rational behavior remains fairly entrenched among economists. However, even the normative relevance of Bayesianism has been questioned. In this post, I briefly survey one particular and interesting kind of argument that has been particularly developed by the decision theorist Itzhak Gilboa with different co-authors in several papers.

First, it is useful to start with a definition of Bayesianism in the context of economic theory: the doctrine according to which it is always rational to behave according to the axioms of Bayesian decision theory. Bayesianism is a broad church with many competing views (e.g. radical subjectivism, objective Bayesianism, imprecise Bayesianism…) but it will be sufficient to retain a generic characterization through the two following principles:

Probabilism: Bayesian rational agents have beliefs that can be characterized through a probability function whose domain is some state space.

Expected Utility Maximization: The choices of Bayesian rational agents can be represented by the maximization of the expectation of a utility function according to some probability function.

Gilboa’s critique of Bayesianism is uniquely concerned with probabilism though some of its aspects could be easily extended to the expected utility maximization principle. Probabilism can itself be characterized as the conjunction of three tenets:

(i) Grand State Space: each atom (“state of nature”) in the state space is assumed to resolve all uncertainty, i.e. everything that is relevant for the modeler is specified, included all causal relationships. Though in Savage’s version of Bayesian decision theory, states of nature where understood as “small worlds” corresponding to some coarse partition of the state space, in practice most economists implicitly interpret states of nature as “large worlds”, i.e. as resulting from the finest partition of the state space.

(ii) Prior Probability: Rational agents have probabilistic beliefs over the state space which are captured by a single probability measure.

(iii) Bayesian updating: In light of new information, rational agents update their prior to a posterior belief according to Bayes’s rule.

While the third tenet may be disputed, included within the realm of Bayesianism (see for instance Jeffrey’s probability kinematics or views entertained by some objective Bayesians), it is the first two that are targeted by Gilboa. More exactly, while each tenet taken separately seems pretty reasonable normatively speaking, problems arise as soon as one decides to combine them.

Consider an arbitrary decision problem where it is assumed (as economists routinely do) that all uncertainty is captured through a Grand State Space. Say, you have to decide between choosing to bet on what is presented to you as a fair coin falling on heads and betting on the fact that the next winner of the US presidential will be a Republican. There seem to be only four obvious states of nature: [Heads, Republican], [Heads, Democrat], [Tail, Republican], [Tail, Democrat]. Depending on your prior beliefs that the coin toss will fall on Heads (maybe a 1:1 odd) and that the next US president will be a Republican (and assuming monotonic preferences in money), your choice will reveal your preference for one of the two bets. Even if ascribing probabilities to some of the events may be difficult, it seems that the requirements of Bayesian rationality cannot be said to be unreasonable here. But matters are actually more complicated because there are many things that may causally affect the likelihood of each event. For instance, while you have been said that the coin is fair, maybe you have reason to doubt this affirmation. This will depend for instance on who has made the statement. Obviously, the result of the next US presidential elections will depend on the many factual and counterfactual events that may happen. To form a belief about the result of the US elections not only you have to form a belief over these events but also over the nature of the causal relationships between them and the result of the US election. Computationally, the task quickly becomes tremendous as the number of states of nature to consider is quite huge. Assuming that a rational agent should be able to assign a prior over all of them is normatively unreasonable.

An obvious answer (at least for economists and behaviorists-minded philosophers) is to remark that prior beliefs need not be “in the mind” of the decision-maker. What matters is that the betting behavior of the decision-maker reveals preferences over prospects that can be represented by a unique probability measure over as larger a state space as needed to make sense of it. There are many things to be said against this standard defense but for the sake of the argument we may momentarily accept it. What happens however of the behavior of the agents fail to reveal the adequate preferences? Must we conclude then that the decision-maker is irrational? A well-known case leading to such questions is Ellsberg’s paradox. Under a plausible interpretation, the latter indicates that most actual agents reveal through their choices an aversion for probabilistic ambiguity which directly led to the violation of the independence axiom of Bayesian decision theory. In this case, the choice behavior of agents cannot be consistently represented by a unique probability measure. Rather than arguing that such a choice behavior is irrational, a solution (which I have already discussed here) is to adopt the Grand State Space approach. It is then possible to show that with an augmented state space there is nothing “paradoxical” in Ellsber’s paradox. The problem however with this strategy is twofold. On the one hand, many choices are “unobservable” by definition, which fits uneasily in the behaviorist interpretation of Bayesian axioms. On the other hand,  it downplays the reasons that explain the choices that actual agents are actually making.

To understand this last point, it must be acknowledged that Bayesianism defines rationality merely in terms of consistency with respect to a set of axioms. As a result, such an approach completely disregards the way agents form their beliefs (as well as their preferences) and – more importantly – abstains from making any normative statement regarding the content of beliefs. “Irrational” beliefs are merely beliefs that fail to qualify for a representation through a unique probability measure. Now, consider whether it is irrational to fail or to refuse to have such beliefs in cases where some alternatives but not others suffer from probabilistic ambiguity. Also, consider whether it is irrational to firmly believe (eventually to degree 1) that smoking presents no risk for health. Standard Bayesianism will answer positively in the first case but negatively in the second. Not only this is unintuitive but it also seems to be pretty unreasonable. Consider the following alternative definition of rationality proposed by Gilboa:

A mode of behavior is irrational for a decision maker, if, when the latter is exposed to the analysis of her choices, she would have liked to change her decision, or to make different choices in similar future circumstances.

This definition of rationality appeals to the reflexive abilities of human agents and, crucially, to our capacity to motivate our choices through reasons. This suggests first that the axioms of Bayesian decision theory can be submitted both as reasons to make specific choices but also has the subject of the normative evaluation. This also indicates that whatever may be thought of these axioms, Bayesianism lacks an adequate account of beliefs formation. In other words, Bayesianism cannot pretend to constitute a normative theory of rationality because it does not offer any justification neither for the way an agent should partition the state space nor for deciding which prior to adopt. The larger the state space is made to capture all the relevant features explaining an agent’s prior, the lesser it seems reasonable to expect rational agents to be able or to be willing to entertain such a prior.

Advertisements

Accounting for Choices in Economics

Economics is sometimes characterized as the “science of rational choices over the allocation of scarce resources” or even more straightforwardly as the “science of choices”. In a recent blog, Chris Dillow makes some interesting remarks about people’s economic behavior. He notes that our behavior is often partially unconscious and/or habit-based. Moreover, the set of available options is quite frequently severely restricted such that there is few room to make voluntary choices. Finally, many decisions are actually more or less random and grounded on social norms, conventions and other factors on which we barely reflect. The conclusion is then that

“when we ask “why did he do that?” we must look beyond “max U” stories about mythical individuals abstracted from society and look at the role of habit, cultural persistence and constraints.”

These are interesting and important remarks because they directly concern the scope of economics as well as the meaning of the key concept of choice. It seems that Dillow is using the choice concept according to its folk meaning. According to the latter, to properly say “she chooses x” requires at least that (a) one has several available options at her disposal to choose between and (b) she opts for one of the available option consciously and voluntarily. However, I would argue that this is not how economists generally use and understand the choice concept. They rather use a concept of choice* in a technical sense. To put it using some jargon, in economics choices* are basically behavioral patterns that correlate with changes in opportunity costs. In other words, when we say that economics is the science of choices*, what is actually meant is that it studies how some particular variable reflecting for instance the consumption level of a given good, changes as the good’s relative price or consumers’ information change. This definition of choice* has at least two noteworthy implications:

1) Economists are not interested in individual choices per se. Economists almost always work at some aggregated level and they do not aim at explaining the choices made by specific individuals or firms. They are rather interested in the properties of aggregate demand and supply.

2) Economists are agnostic regarding the specific mechanisms through which economic agents are making choices. In particular, there is no presumption that these choices are conscious and not habit-based. The U-Max framework only assumes that individual choices are responsive to change in opportunity costs, not how and why they are responsive.

These two implications work in conjunctions. Choices* need not be conscious nor based on any form of complex calculus but they are however intentional: choices (in both the folk and technical meanings) are about something and they are the product of the agents’ intentional states (desires, beliefs, wants…). As philosophers of mind have emphasized, there is nothing paradoxical in the combination of unconsciousness and intentionality. The U-Max framework, as well as decision and game theory as a whole are tools that are particularly well-fitted to study intentional behavior, whether conscious or not. These tools indeed assume that individual choices are responsive to changes in opportunity costs which, in special cases (e.g. addictive behavior), may not be true. However, this is mostly irrelevant as long as responsiveness is preserved at some market level. Gary Becker’s paper “Irrational Behavior and Economic Theory” provides an extreme example of this point. It shows how we can derive “well-behaved” demand and supply functions with individual agents (households and firms) using “irrational” decision rules. This result is by no way a necessity: there are cases where irrational behavior will lead to unconventional demand and supply functions and because of income effects even rational behavior at the individual level can generate upward-slopping demand curves. Generally speaking, institutions matter: the way exchanges are organized will determine the aggregate outcome for a given profile of preferences and production costs.

All of this depends on the claim that economists are not interested in explaining individual choices. Economists with the strongest revealed-preference stance are likely to agree with this claim. But there are many economists who are likely to disagree, considering that accounting for individual choices is necessary to understand aggregate outcomes such as a financial crisis. More generally, I would argue that attempting to explain individual choices can hardly be avoided in the numerous cases where multiple equilibria exist. The point is that to explain why a given equilibrium has been selected, it will most of the time be required to understand how individuals make choices. Here, whether choices are habit- or calculus-based, conscious or automatic, and so on, may matter. For instance, Thomas Schelling famously pointed out in The Strategy of Conflict the important of focal points to account for the way people are able to coordinate without communicating. As Schelling made it clear, focal points are not determined by the mathematical properties of the game nor by purely instrumental considerations. They depend on cultural, social and aesthetic features.

A slightly more complex example but which is even more relevant, especially in industrial organization, is the existence of multiple (Bayesian perfect) equilibria in incomplete information games. In incomplete information games, one player (the “principal”) ignores the other players’ (the “agent”) type. The agent’s choice may sometimes convey an information to the principal and helps him to identify the agent’s type. Such games typically have multiple equilibria with some of them separating and other pooling ones. Which equilibrium is implemented is partially determined by the way the principal interprets the agent’s choice. Under a separating equilibrium, the principal interprets the agent’s choice in such a way that it provides him with an information about the agent’s type. This is not the case under a pooling equilibrium. Of course, since under a pooling equilibrium all agents behave the same way whatever their type, observed behavior cannot serve as a basis to infer agents’ type. But the fact that all agents behave the same is itself a rational response to their own understanding of the way the principal will interpret their choice at the equilibrium.

My point is thus that in strategic interactions where players have to think about how other players are thinking, it is less clear that economists can safely ignore how people make choices. Given the same set of “fundamentals” (preferences, technology, information distribution), different behavioral patterns may arise and these differences are likely to be due to the way individual agents are choosing.

Consequentialism and Formalism in Rational and Social Choice Theory

Rational and social choice theory (RCT and SCT respectively) in economics are broadly consequentialists. Consequentialism can be characterized as the view that all choice alternatives should be evaluated in terms of their consequences and that the best alternatives are those which have the best consequences. This is a very general view which allows for many different approaches and frameworks. In SCT, welfarism is for example a particular form of consequentialism largely dominant in economics and utilitarianism is a specific instance of welfarism. In RCT, expected utility theory and revealed preference theory are two accounts of rational decision-making that assume that choices are made on the basis of their consequences.

Consequentialism is also characterized by a variety of principles or axioms that take different and more or less strong forms depending on the specific domain of application. The most important are the following:

Complete ordering (CO): The element of any set A of alternatives can be completely ordered on the basis of a reflexive and transitive binary relation ≥.

Independence (I): The ranking of any pair of alternatives is unaffected by a change in the likelihood of consequences which are identical across the two alternatives.

Normal/sequential form equivalence (NSE): The ordering of alternatives is the same whether the decision problem is represented in normal form (the alternative is directly associated to a consequence or a probability distribution of consequences) or in sequential form (the alternative is a sequence of actions leading to a terminal node associated to a consequence or a probability distribution of consequences).

Sequential separability (SS): For any decision tree T and any subtree Tn starting at node n of T, the ordering of the subset of consequences accessible in Tn is the same in T than in Tn.

Pareto (P): If two alternatives have the same or equivalent consequences across some set of locations (events, persons), then there must be indifference between the two alternatives.

Independence of irrelevant alternatives (IIA): The ordering of any pair of alternatives is independent of the set of available alternatives.

All these axioms are used either in RCT or in SCT, sometimes in both. CO, I, NSE, SS and IIA are almost always imposed on individual choice as criteria of rationality. CO and IIA, together with P, are generally regarded as conditions that Arrowian social welfare functions must satisfy. I is also sometimes considered as a requirement for social welfare functionals, especially in the context of discussions over utilitarianism and prioritarianism.

It should be noted that they are not completely independent: for instance, CO will generally require the satisfaction of IIA or of NSE. Regarding the former for instance, define a choice function C(.) such that, for any set S of alternatives, C(S) = {x|x ≥ y for all y  S}, i.e. the alternatives that can be chosen are those and only those which are not ranked below any other alternative in terms of their consequences. Consider a set of three alternatives x, y, z and suppose that C(x, y) = {x} but C(x, y, z) = {y, z}. This is a violation of IIA since while x y and (not y x) when S = (x, y), we have y x and (not x y) when S = (x, y, z). Now suppose that C(x, z) = {z}. We have a violation of the transitivity of the negation of binary relation ≥ since while we have (not z y) and (not y x), we nevertheless have z x. However, this is not possible if CO is satisfied.

All these axioms have traditionally been given a normative interpretation. By this, I mean that they are seen as normative criteria of individual and collective rationality: a rational agent should or must have completely ordered preferences over the set of all available alternatives, he cannot on pain of inconsistency violate I or NSE, and so on. Similarly, collective rationality entails that any aggregation of the individuals’ evaluations of the available alternatives generates a complete ordering satisfying P and IIA and possibly I. Understood this way, these axioms characterize consequentialism as a normative doctrine setting constraints on rational and social choices. For instance, in the moral realm, consequentialism rules out various forms of egalitarian accounts which violate I and sometimes P. In the domain of individual choice, it will regard criteria such as minimization of maximum regret or maximin as irrational. Consequentialists have to face however several problems. The first and most evident one is that reasonable individuals regularly fail to meet the criteria of rationality imposed by consequentialism. This has been well-documented in economics, starting with axiom I in Allais’ paradox and Ellsberg’s paradox. A second problem is that the axioms of consequentialism sometimes lead to counterintuitive and disturbing moral implications. It has been suggested that criterion of individual rationality should not apply to collective rationality, especially CO and I (but also P and IIA).

These difficulties have led consequentialists to develop defensive strategies to preserve most of the axioms. Most of these strategies refer to what I will call formalism: in a nutshell, they consist as regarding the axioms as structural or formal constraints for representing, rather than assessing, individual and collective choices. In other words, rather than a normative doctrine, consequentialism is instead best viewed as a methodological and theoretical framework to account for the underlying values that ground individual and collective choices. As this may sound quite abstract, I will discuss two examples, one related to individual rational choice the other to social choice, both concerned with axiom I. The first example is simply the well-known Ellsberg’s paradox. Assume you are presented with two consecutive decision-problems, each time between a pair of alternatives. In the first one, we suppose that an urn contains 30 red balls and 60 other balls which can be either black or yellow. You are presented with two alternatives: alternative A gives you 100$ in case a red ball is drawn and alternative B gives you 100$ in case a black ball is drawn. In the second decision-problem, the content of the urn is assumed to be the same, but this time alternative C gives you 100$ in case you draw either a red or yellow ball and alternative D gives you 100$ in case you draw either a black or yellow ball.

Alternative/event E1: Red ball is drawn E2: Black ball is drawn E3: Yellow ball is drawn
A 100$ 0$ 0$
B 0$ 100$ 0$
Alternative/event E1: Red ball is drawn E2: Black ball is drawn E3: Yellow ball is drawn
C 100$ 0$ 100$
D 0$ 100$ 100$

Axiom I entails that if the decision-maker prefers A to B, then he should prefer C to D. The intuition is that if one prefers A to B, that must mean that the decision-maker ascribes a higher probability to event E1 than to event E2. Since the content of the urn is assumed to be the same in both decision-problems, this should imply that the expected gain of C (measured either in money or in utility) should be higher than D’s. The decision-maker’s ranking of alternatives should be independent of what happen in case event E3 holds, since in each decision-problem the alternatives have the same outcome. However, as Ellsberg’s experiment shows, while most persons prefer A to B, they prefer D to C which is sometimes interpreted as the result of some ambiguity-aversion.

The second example has been suggested by Peter Diamond in a discussion of John Harsanyi’s utilitarian aggregation theorem. Suppose a doctor has two patients waiting for kidney transplantation. Unfortunately, only one kidney is available and it is not expected that another one will be before quite some time. We assume that the doctor, endorsing the social preference of the society, is indifferent between giving the kidney to one or the other patient. The doctor is considering choosing between three allocation mechanisms: mechanism S1 gives the kidney to patient 1 for sure, mechanism S2 gives the kidney to patient 2 for sure, while in mechanism R he tosses a fair coin and gives the kidney to patient 1 if tails but to patient 2 if heads.

Alternative/event E1: Coin toss falls Tails E2: Coin toss falls Heads
S1 Kidney is given to patient 1 Kidney is given to patient 1
S2 Kidney is given to patient 2 Kidney is given to patient 2
R Kidney is given to patient 1 Kidney is given to patient 2

Given that it is assumed that the society (and the doctor) is indifferent between giving the kidney to patient 1 or 2, axiom I implies that the three alternatives should be ranked as indifferent. Most people have the strong intuition however that allocation mechanism R is better because it is fairer.

Instead of giving up axiom I, several consequentialists have suggested instead to reconcile our intuitions with consequentialism through a refinement of the description of outcomes. The basic idea is that, following consequentialism, everything in the individual or collective choice should be featured in the description of outcomes. Consider Ellsberg’s paradox first. If we assume that the violation of I is due to the decision-makers’ aversion to probabilistic ambiguity, then we modify the tables in the following way:

Alternative/event E1: Red ball is drawn E2: Black ball is drawn E3: Yellow ball is drawn
A 100$ + sure to have a 1/3 probability of winning 0$ + sure to have a 1/3 probability of winning 0$ + sure to have a 1/3 probability of winning
B 0$ + unsure of the probability of winning 100$ + unsure of the probability of winning 0$ + unsure of the probability of winning
Alternative/event E1: Red ball is drawn E2: Black ball is drawn E3: Yellow ball is drawn
C 100$ + unsure of the probability of winning 0$ + unsure of the probability of winning 100$ + unsure of the probability of winning
D 0$ + sure to have a 2/3 probability of winning 100$ + sure to have a 2/3 probability of winning 100$ + sure to have a 2/3 probability of winning

The point is simple. If we consider that being unsure of one’s probability of winning the 100$ is something that makes an alternative less desirable everything else equals, then this has to be reflected in the description and valuation of outcomes. It is then easy to see that ranking A over B but D over C no longer entails a violation of I because the outcomes associated to event E3 are no longer the same in each pair of alternatives. A similar logic can be applied to the second example. If it is collectively considered that the fairness of the allocation mechanism is something valuable, then this must be reflected in the description of outcomes. Then, we have

Alternative/event E1: Coin toss falls Tails E2: Coin toss falls Heads
S1 Kidney is given to patient 1 Kidney is given to patient 1
S2 Kidney is given to patient 2 Kidney is given to patient 2
R Kidney is given to patient 1 + both patients are fairly treated Kidney is given to patient 2 + both patients are fairly treated

Once again, this new description allows to rank R strictly above S1 and S2 without violating I. Hence, the consequentialist’s motto in all the cases where one axiom seems to be problematic is simply “get the outcome descriptions right!”.

A natural objection to this strategy is of course that it seems to make things too easy for the consequentialist. On the one hand, it makes the axioms virtually unfalsifiable as any choice behavior can be trivially accounted for by a sufficiently fine grain partition of the outcome space. On the other hand, all moral intuitions and principles can be made compatible with a consequentialist perspective, once again provided that we have the right partition of the outcome space. However, one can argue that this is precisely the point of the formalist strategy. The consequentialist will argue that this is unproblematic as long as consequentialism is not seen as a normative doctrine about rationality and morality, but rather as a methodological and theoretical framework to account for the implications of various values and principles on rational and social choices. More precisely, what can be called formal consequentialism can be seen as a framework to uncover the principles and values underlying our moral and rational behavior and judgments.

Of course, this defense is not completely satisfactory. Indeed, most consequentialists will not be comfortable with the removal of all the normative content from their approach. As a consequentialist, one wants to be able to argue what it is rational to do and to say what morality commends in specific circumstances. If one wants to preserve some normative content, then the only solution is to impose normative constraints on the permissible partitions of the outcome space. This is indeed what John Broome has suggested in several of his writings with the notion of “individuation of outcomes by justifiers”: the partition of the outcome space should distinguish outcomes if and only if they differ in a way that makes it rational to not be indifferent between them. It follows then that theories of rational choice and social choice are in need of a substantive account of rational preferences and goodness. Such an account is notoriously difficult to conceive. A second difficulty is that the formalist strategy will sometimes be implausible or may even lead to some form of inconsistency. For instance, in the context of expected utility theory, Broome’s individuation of outcomes depends on the crucial and implausible assumption that all “constant acts” are available. This leads to a “richness” axiom (made by Savage for instance) according to which all probabilistic distribution of outcomes should figure in the set of available alternatives, including logically or materially impossible alternatives (e.g. being dead and in a good health). In sequential decision-problems, the formalist strategy is bounded to fail as soon as the path taken to reach a given outcome is relevant for the decision-maker. In this case, to include the path taken in the description of outcomes will not be always possible without leading to inconsistent descriptions of what is supposed to be the same outcome.

These difficulties indicate that formalism cannot fully vindicate consequentialism. Still, it remains an interesting perspective both in rational and social choice theory.

Isaac Levi on Rationality, Deliberation and Prediction (1/3)

This is the first of a three-part post on the philosopher Isaac Levi’s account of the relationship between deliberation and prediction in decision theory and which is an essential part of Levi’s more general theory of rationality. Levi’s views potentially have tremendous implications for economists especially regarding the current use of game theory. These views are more particularly developed in several essays collected in his book The Covenant of Reason, especially “Rationality, prediction and autonomous choice”, “Consequentialism and sequential choice” and “Prediction, deliberation and correlated equilibrium”. The first post presents and discusses Levi’s main thesis that “deliberation crowds out prediction”. The next two posts will discuss some implications of this thesis for decision theory and game theory, specifically (i) the equivalence between games in dynamic form and in normal form and (ii) the relevance of the correlated equilibrium concept for Bayesianism in the context of strategic interactions. The three posts are collected under a single pdf file here.

The determination of principles of rational choice is the main subject of decision theory since its early development at the beginning of the 20th century. Since its beginnings, decision theory has pursued two different and somehow conflicting goals: on the one hand, to describe and explain how people actually make choices and, on the other hand, to determine how people should make choices and what choices they should make. While the former goal corresponds to what can be called “positive decision theory”, the latter is constitutive of “normative decision theory”. Most decision theorists, especially the proponents of “Bayesian” decision theory, have agreed that decision theory cannot but be partially normative. Indeed, while today Bayesian decision theory is generally not regarded as an accurate account of how individuals are actually making choices, most decision theorists remain convinced that it is still relevant as a normative theory of rational decision-making. This is in this context that Isaac Levi’s claim that “deliberation crowds out prediction” should be discussed.

In this post, I will confine the discussion to the restrictive framework of Bayesian decision theory though Levi’s account more generally applies to any form of decision theory that adheres to consequentialism. Consequentialism will be more fully discussed in the second post of this series. Consider any decision problem D in which an agent DM has to make a choice over a set of options whose consequences are not necessarily fully known for sure. Bayesians will generally model D as a triple < A, S, C > where A is the set of acts a, S the set of states of nature s and C the set of consequences c. In the most general form of Bayesian decision theory, any a, s and c may be regarded as a proposition to which truth-values might be assigned. In Savage’s specific version of Bayesian decision theory, acts are conceived as functions from states to consequences, i.e. a: S à C or c = a(s). In this framework, it is useful to see acts as DM’s objects of choice, i.e. the elements over which he has a direct control, while states may be interpreted as every features in D over which DM has no direct control. Consequences are simply the result of the combination of an act (chosen by DM) and a state (not chosen by DM). Still following Savage, it is standard to assume that DM has (subjective) beliefs over which state s actually holds. These beliefs are captured by a probability function p(.) with ∑sp(s) = 1 for a finite state space. Moreover, each consequence c is assigned a utility value u representing DM’s preferences over the consequences. A Bayesian DM will then choose the act that maximizes his expected utility given his subjective beliefs and his preferences, i.e.

Maxa Eu(a) = ∑sp(s|a)u(a(s)) =  ∑sp(s|a)u(c).

Two things are worth noting. First, note that the probabilities that enter into the expected utility computation are conditional probabilities of states given acts. We should indeed account for the possibility that the probabilities of states depend on the act performed. The nature of the relationship between states and acts represented by these conditional probabilities is the main subject of conflict between causal and evidential decision theorists. Second, as it is well-known, in Savage’s version of Bayesian decision theory, we start with a full ordering representing DM’s preferences over acts and given a set of axioms, it is shown that we can derive a unique probability function p(.) and a cardinal utility function u(.) unique up to any positive affine transformation. It is indeed important to recognize that Savage’s account is essentially behaviorist because it merely shows that given the fact that DM’s preferences and beliefs satisfy some properties, then his choice can be represented as the maximization of some function with some uniqueness property. Not all Bayesian decision theorists necessarily share Savage’s behaviorist commitment.

I have just stated that in Savage’s account, DM ascribes probabilities to states, utilities to consequences and hence expected utilities to acts. However, if acts, states and consequences are all understood as propositions (as argued by Richard Jeffrey and Levi among others), then there is nothing in principle prohibiting to ascribe utilities to states and probabilities to both consequences and acts. This is this last possibility (ascribing probabilities to acts) that is the focus of Levi’s claim that deliberation crowds out prediction. In particular, does it make sense for DM to have unconditional probabilities over the set A? How having such probabilities could be interpreted from the perspective of DM’s deliberation in D? If we take a third person perspective, ascribing probabilities to DM’s objects of choice seems not particularly contentious. It makes perfect sense for me to say for instance “I believe that you will start again to smoke before the end of the month with probability p”. Ascribing probabilities to others’ choices is an essential part of our daily activity consisting in predicting others’ choices. Moreover, probability ascription may be a way to explain and rationalize others’ behavior. The point of course is that these are my probabilities, not yours. The issue here is whether a deliberating agent has to, or even can ascribe such probabilities to his own actions, acknowledging that such probabilities are in any case not relevant in the expected utility computation.

Levi has been (with Wolfgang Spohn) the most forceful opponent to such a possibility. He basically claims that the principles of rationality that underlie any theory of decision-making (including Bayesian ones) cannot at the same time serve as explanatory and predictive tools and as normative principles guiding rational behavior. In other words, as far as the deliberating agent is using rationality principles to make the best choice, he cannot at the same time use these principles to predict his own behavior at the very moment he is making his choice.* This is the essence of the “deliberation crowds out prediction” slogan. To understand Levi’s position, it is necessary to delve into some technical details underlying the general argument. A paper of philosopher Wlodek Rabinowicz makes a great job in reconstructing this argument (see also this paper by James Joyce). A crucial premise is that, following De Finetti, Levi considers belief ascription as fully constituted by the elicitation of betting rates, i.e. DM’s belief over some event E is determined and corresponds to what DM would consider as the fair price of a gamble where event E pays x$ and event non-E pays y$.** Consider this example: I propose you to pay y$ (the cost or the price of the bet) to participate to the following bet: if Spain win the Olympic gold medal of basketball at Rio this month, I will pay you x$, otherwise I pay you nothing. Therefore, x is the net gain of the bet and x+y is called the stake of the bet. Now, the fair price y*$ of the bet corresponds to the amount for which you are indifferent between taking and not taking the bet. Suppose that x = 100 and that y* = 5. Your betting rate for this gamble is then y*/(x+y*) = 5/105 = 0,048, i.e. you believe that Spain will win with probability less than 0,05. This is the traditional way beliefs are determined in Bayesian decision theory. Now, Levi’s argument is that such a procedure cannot be applied in the case of beliefs over acts on pain of inconsistency. The argument relies on two claims:

(1)       If DM is certain that he will not perform some action a, then a is not regarded as part of the feasible acts by DM.

(2)       If DM assigns probabilities to acts, then he must assign probability 0 to acts he regards as inadmissible, i.e. which do not maximize expected utility.

Clearly, (1) and (2) entail together that only feasible acts (figuring in the set A) are admissible (maximize expected utility), in which case deliberation is unnecessary for DM. If it is the case however, that means that principles of rationality cannot be used as normative principles in the deliberation process. While claim (1) is relatively transparent (even if it is disputable), claim (2) is less straightforward. Consider therefore the following illustration.

DM has a choice between two feasible acts a and b with Eu(a) > Eu(b), i.e. only a is admissible. Suppose that DM assigns probabilities p(a) and p(b) according to the procedure presented above. We present DM with a fair bet B on a where the price is y* and the stake is x+y*. As the bet is fair, y* is the fair price and y*/(x+y*) = p(a) is the betting rate measuring DM’s belief. Now, DM has four feasible options:

Take the bet and choose a (B&a)

Do not take the bet and choose a (notB&a)

Take the bet and choose b (B&b)

Do not take the bet and choose b (notB&b)

As taking the bet and choosing a guarantee a sure gain of x to DM, it is easy to see that B&a strictly dominates notB&a. Similarly, as taking the bet and choosing b guarantee a sure loss of y*, notB&b strictly dominates B&b. The choice is therefore between B&a and notB&b and clearly Eu(a) + x > Eu(b). It follows that the fair price for B is  y* = x + y* and hence p(a) = 1 and p(b) = 1 – p(a) = 0. The inadmissible option b has probability 0 and is thus regarded as unfeasible by DM (claim 1). No deliberation is needed for DM if he predicts his choice since only a is regarded as feasible.

Levi’s argument is by no means undisputable and the papers of Rabinowicz and Joyce referred above make a great job at showing its weaknesses. In the next two posts, I will however take it as granted and discuss some of its implications for decision theory and game theory.

 Notes

* As I will discuss in the second post, Levi considers that there is nothing contradictory or problematic in the assumption that one may be able to predict his future choices.

** A gamble’s fair price is the price at which DM is indifferent accepting to buy the bet and accepting to sell the bet.

Economics, Rational Choice Theory and Utility Maximization

Economist Geoff Hodgson has a short article at Evonomics on the issue of the theoretical and methodological status of the rationality principle in economics. Hodgson sketches an argument that he has more fully developed in his book From Pleasure Machines to Moral Communities. In a nutshell, Hodgson argues that the rationality principle, according to which individuals act such as to maximize their utility, is (i) not a tautology but (ii) is unfalsifiable. Let me take these two points in order.

First, Hodgson argues that the rationality principle is not a tautology because utility maximization “is potentially false”. That is, there may be cases where people’s choices fail to maximize their utility. We may suppose that the non-maximizing behavior may be intentional or not: there are cases where we intend to maximize our utility but we fail, for instance because of weakness of will (a phenomenon known as Akrasia in ancient Greece); there are other cases where reason provides us with good reason to make choices that do not maximize our utility. This latter possibility is at the core of Amartya Sen’s criticism of rational choice theory developed in the 1970’s. Second, Hodgon points out that in spite of the fact that the U-max assumption may be false, we can never know when or at least we can never establish that is false in any specific case. This is due to the fact that it is at least always possible to change the description of some decision problem such as to make the observed behavior compatible with any consistency assumption and thus utility maximization. This is indeed true in all versions of rational choice theory, either under the form of revealed preference theory or of expected utility theory. The basic strategy consists in changing the description of the outcome space of the decision problem such as to make the choice behavior consistent. Unless a behavior is completely random, there should always be in principle a way to rationalize it according to some consistency criterion. U-max only requires transitivity of the underlying preference ordering (plus some reflexivity and continuity conditions). According to Hodgson, these two points make rational choice theory useless as a theory of human behavior. In particular, he rightly note that rational choice theory applies equally well to machines, insects and animals and that as a consequence it cannot tell us anything specific about humans.

I partially agree with Hodgson but his argument requires some qualifications. More specifically, I would argue that (i) actually rational choice theory is a tautology and that (ii) the fact that is unfalsifiable is not necessarily problematic depending on its purpose. Consider the former point first. The only reason Hodgson can validly claim that utility maximization is not a tautology is because he takes utility to be something to be independently measurable. This is of course the way the utility concept was understood by Bentham and by the first marginalists. There is also a bunch of behavioral economists defending a “back to Bentham” paradigmatic move who speak in terms of “experienced utility”, where the latter refers to something akin to happiness or pleasure. Finally, we may also admit that some economists of the Chicago school may have entertained an interpretation of utility as something independently measurable. But all of this is unorthodox. Since the days of Pareto and Samuelson, economic theory (and especially consumer theory) has given up the interpretation of utility as an independently measurable quantity. The ordinalist revolution and Samuelson’s pioneering contribution to revealed preference theory have shown how consumer theory can be formulated without any reference to the utility concept. More exactly, they have established that utility maximization is nothing but a mathematically convenient statement equivalent to the assumption that people make consistent choices and/or have well-ordered preferences. The same is true for expected utility theory, especially Savage’s version which is explicitly behaviorist. Absolutely nothing is assumed regarding what happens “in the head” of the person making some choice. U-max is not an assumption; it is only a descriptive statement of what one is doing. It is a tautology as long as there is always a possibility to rationalize one’s choices in terms of some consistency condition.

Consider now the second point. The fact that the U-max principle is actually a tautology only strengthens Hodgson’s claim that it is unfalsifiable. You cannot falsify a tautology as it is true by definition. Does it make it useless from a scientific perspective? The short answer is clearly “no”. Science is full of useful tautologies, also in economics. Consider only one example coming from biology, one on which Hodgson extensively relies in his work: the Price equation. The Price equation is a highly general mathematical statement of a process of differential replication, i.e. natural selection. The mathematical beauty of the Price equation is that whatever the specificities of the actual process of selection (whether organisms are haploid, haplodiploid, diploid, whether it is cultural or genetic, …), it captures them in a straightforward formula according to which, to simplify matters, the growth rate of some trait in a population can be expressed as the covariance between the trait frequency and the fitness of its bearer. Under the classical meaning of fitness (a measure of reproductive success), Price equation is of course both unfalsifiable and a tautology. But no biologists or behavioral scientists would reject it for this reason. The usefulness of the Price equation comes from its value as a problem-solving device. It gives the scientist a methodological strategy to solve empirical or theoretical problems. As an instance of the latter case, consider for instance how Price equation is useful to derive Hamilton’s rule and to make explicit the assumptions on which the latter rely regarding the property of selection and inclusive fitness.

I would argue that the same is true for rational choice theory in economics. From a Weberian point of view, rational choice theory provides us with a methodological strategy to uncover people’s motivations and reasons for action. Similarly, Don Ross argues it is part of the “intentional stance strategy” through which we are able to understand and predict agents’ behavior. Hodgson is right that rational choice theory is rather weak as an explanatory theory of individual behavior, simply because the theory suffers from an obvious problem of under-determination. But individual behavior is not the right level at which the theory should be applied. It is way more useful for instance to understand how the change in the institutional framework, by modifying people’s incentives and beliefs, may affect their behavior. This strategy is at the core of the applied branch of microeconomics, known as mechanism design. A branch which has enjoyed some empirical successes recently. Of course, there are other reasons to reject the imperialistic claims of the proponents of rational choice theory. I explore some of them in this (version of a) forthcoming paper in the Journal of Economic Methodology.

A Short Note on Newcomb’s and Meta-Newcomb’s Paradoxes

[Update: As I suspected, the original computations were false. This has been corrected with a new and more straightforward result!]

For some reasons, I have been thinking about the famous Newcomb’s paradox and I came with a “solution” which I am unable to see if it has been proposed in the vast literature on the topic. The basic idea is that a consistent Bayesian decision-maker should have a subjective belief over the nature of the “Oracle” that, in the original statement of the paradox, is deemed to predict perfectly your choice of taking either one or two boxes. In particular, one has to set a probability regarding the event that the Oracle is truly omniscient, i.e. he is able to foreseen your choice. Another, more philosophical way to state the problem is for the decision-maker to decide over a probability that Determinism is true (i.e. the Oracle is omniscient) or that the Free Will hypothesis is true (i.e. the Oracle cannot predict your choice).

Consider the following table depicting the decision problem corresponding to Newcomb’s paradox:

Matrice

Here, p denotes the probability that the Oracle will guess that you will pick One Box (and thus put 1 000 000$ in the opaque box), under the assumption that the Free Will hypothesis is true. Of course, as it is traditionally stated, the Newcomb’s paradox normally implies that p is a conditional probability (p = 1 if you choose One Box, p = 0 if you choose two boxes), but this is the case only in the event that Determinism is true. If the Free Will hypothesis is true, then p is an unconditional probability as argued by causal decision theorists.

Denote s the probability for the event “Determinism” and 1-s the resulting probability for the event “Free Will”. It is rational for the Bayesian decision-maker to choose One Box if his expected gain for taking one box g(1B) is higher than his expected gain for taking two boxes g(2B), hence if

s > 1/1000.

Interestingly, One Box is the correct choice even if one puts a very small probability on Determinism being the correct hypothesis. Note that is independent of the value of p. If one has observed a sufficient number of trials where the Oracle has made the correct guess, then one has strong reasons to choose One Box, even if he endorses causal decision theory!

Now consider the less-known “Meta-newcomb’s paradox” proposed by philosopher Nick Bostrom. Bostrom introduces the paradox in the following way:

There are two boxes in front of you and you are asked to choose between taking only box B or taking both box A and box B. Box A contains $ 1,000. Box B will contain either nothing or $ 1,000,000. What B will contain is (or will be) determined by Predictor, who has an excellent track record of predicting your choices. There are two possibilities. Either Predictor has already made his move by predicting your choice and putting a million dollars in B iff he predicted that you will take only B (like in the standard Newcomb problem); or else Predictor has not yet made his move but will wait and observe what box you choose and then put a million dollars in B iff you take only B. In cases like this, Predictor makes his move before the subject roughly half of the time. However, there is a Metapredictor, who has an excellent track record of predicting Predictor’s choices as well as your own. You know all this. Metapredictor informs you of the following truth functional: Either you choose A and B, and Predictor will make his move after you make your choice; or else you choose only B, and Predictor has already made his choice. Now, what do you choose?

Bostrom argues that this lead to a conundrum to the causal decision theorist:

If you think you will choose two boxes then you have reason to think that your choice will causally influence what’s in the boxes, and hence that you ought to take only one box. But if you think you will take only one box then you should think that your choice will not affect the contents, and thus you would be led back to the decision to take both boxes; and so on ad infinitum.

The point is that here if you believe the “Meta-oracle”, by choosing Two Boxes you then have good reasons to think that your choice will causally influence the “guess” of the Oracle (he will not put 1000 000$ in the opaque box) and therefore, by causal decision theory, you have to choose One Box. However, if you believe the “Meta-Oracle”, by choosing One Box you have good reasons to think that your choice will not causally influence the guess of the Oracle. In this case, causal decision theory recommends you to choose Two Boxes, as in the standard Newcomb’s paradox.

The above reasoning seems to work also for the Meta-Newcomb paradox even though the computations are slightly more complicated. The following tree represents the decision problem if the Determinism hypothesis is true:

Newcomb

Here, “Before” and “After” denote the events where the Oracle predicts and observes your choice respectively. The green path and the red path in the three correspond to the truth functional stated by the Meta-oracle. The second tree depicts the decision problem if the Free Will hypothesis is true.

Newcomb 2

It is similar to the first one except for small but important differences: in the case the Oracle predicts your choice (he makes his guess before you choose) your payoff depends on the (subjective) probability p that he makes the right guess; moreover, the Oracle is now an authentic player in an imperfect information game with q the decision-maker’s belief over whether the Oracle has already made his choice or not (note that if Determinism is true, q is irrelevant exactly for the same reason than probability p in Newcomb’s paradox). Here, the green and red paths depict the decision-maker best responses.

Assume in the latter case that q = ½ as suggested in Bostrom’s statement of the problem. Denote s the probability that Determinism is true and thus that the Meta-oracle as well as the Oracle are omniscient. I will spare you the computations but (if I have not made mistakes) it can be shown that it is optimal for the Bayesian decision maker to choose One Box whenever s ≥ 0. Without fixing q, we have s > 1-(999/1000q). Therefore, even if you are a causal decision theorist and you believe strongly in Free Will, you should play as if you believe in Determinism!

What Are Rational Preferences

Scott Sumner has an interesting post on Econlog about the economists’ use of what can be called the “Max U” framework, i.e. the approach consisting in describing and/or explaining people’s behavior as a utility maximization. As he points out, there are many behaviors (offering gifts at Christmas, voting, buying lottery tickets, smoking) that most economists are ready to deem “irrational” while actually they seem amenable to some kind of rationalization. Sumner then argues that the problem is not with the Max U framework but rather lies in the economists’ “lack of imagination” regarding the ways people can derive utility.

Sumner’s post singles out an issue that lies at the heart of economic theory since the “Marginalist revolution”: what is the nature of utility and of the related concept of preferences? I will not return here on the fascinating history of this issue that passes through Pareto’s ordinalist reinterpretation of the utility concept to Samuelson’s revealed preference account whose purpose was to frame the ordinalist framework in purely behaviorist terms. These debates had also much influence on normative economics as they underlie Robbins’ argument for the rejection of interpersonal comparisons of utility that ultimately led to Arrow’s impossibility theorem and the somewhat premature announcement of the “death” of welfare economics. From a more contemporary point of view, this issue is directly relevant for modern economics and in particular for the fashionable behavioral economics research program, especially as it has now taken a normative direction. Richard Thaler’s reaction to Sumner’s post on Twitter is thus no surprise:

<blockquote class=”twitter-tweet” lang=”fr”><p lang=”en” dir=”ltr”>Yes. This version of economics is unfalsifiable. If people can &quot;prefer&quot; $5 to $10 then what are preferences? <a href=”https://t.co/Cn1XQoIzsh”>https://t.co/Cn1XQoIzsh</a></p>&mdash; Richard H Thaler (@R_Thaler) <a href=”https://twitter.com/R_Thaler/status/680831304175202305″>26 Décembre 2015</a></blockquote>

Thaler’s point is clear: if we are to accept that all the examples given by Sumner are actual cases of utility maximization, then virtually all kinds of behaviors can be seen as utility maximization. Equivalently, any behavior can be explained by an appropriate set of “rational” preferences with the required properties of consistency and continuity. This point if of course far from being new: many scholars have already argued that rational choice theory (either formulated in terms of utility functions [decision theory for certain and uncertain decision problems] or of choice functions [revealed preference theory]) is unfalsifiable: it is virtually always possible to change the description of a decision problem such as to make the observed behavior consistent with some set of axioms. In the context of revealed preference theory, this point is wonderfully made by Bhattacharyya et al. on the basis on Amartya Sen’s long-standing critique of the rationality-as-consistency approach. As they point out, revealed preference theory suffers from an underdetermination problem: for any set of inconsistent choices (according to some consistency axiom), it is in practice impossible to know whether the inconsistency is due to “true” and intrinsic irrationality or is just the result of an improper specification of the decision problem. In the context of expected utility theory, John Broome’s discussion of the Allais paradox clearly shows that reconciliation is in principle possible on the basis of a redefinition of the outcome space.

Therefore, the fact that rational choice theory may be unfalsifiable is widely acknowledged. Is this a problem? Not so much if we recognize that falsification is no longer recognized as the undisputed demarcation criterion for defining science (as physicists are currently discovering). But even if we ignore this philosophy of science feature, the answer to the above question also depends on what we consider to be the relevant purpose of rational choice theory (and more generally of economics) and relatedly, what should the scientific meaning of the utility and preference concepts. In particular, a key issue is whether or not a theory of individual rationality should be part of economics. Three positions seem to be possible: The “Not at all” thesis, the “weakly positive” thesis and the “strongly positive” thesis:

A) Not at all thesis: Economics is not concerned with individual rationality and therefore does not need a theory of individual rationality. Preferences and utility are concepts used to describe choices (actually or counterfactually) made by economic agents through formal (mathematical) statements useful to deal with authentic economic issues (e.g. under what conditions an equilibrium with such and such properties exists?).

B) Weakly positive thesis: Economics builds on a theory of individual rationality but this theory is purely formal. It equates rationality with consistency of choices and/or preferences. Therefore, it does not specify the content of rational preferences but it sets minimal formal conditions that the preference relation or the choice function should satisfy. Preferences and utility are more likely (but not necessarily) to be defined in terms of choices.

C) Strongly positive thesis: Economics builds on a theory of individual rationality and actually parts of economics consist in developing such a theory. The theory is substantive: it should state what are rational preferences, not only define consistency properties for the preference relation. Preferences and in particular utility cannot be defined exclusively in terms of choices, they should refer to inner states of mind (e.g. “experienced utility”) which are accessible in principle through psychological and neurological techniques and methods.

Intuitively, I would say that if asked most economists would entertain something like the (B) view. Interestingly however, this is probably the only view that is completely unsustainable after careful inspection! The problem is the one emphasized by Thaler and others: if rational choice theory is a theory of individual rationality, then it is empirically empty. The only way to circumvent the problem is the following: consider any decision problem Di faced by some agent i. Denote T the theory or model used by the economist to describe this decision problem (T can be either formulated in an expected utility framework or in a revealed preference framework). A theory T specifies, for any Di, what are the permissible implications in terms of behavior (i.e. what i can do given the minimal conditions and constraints defined in T). Denote I the set of such implications and S any subset of these implications. Then, a theory T corresponds to a mapping T: D –> I with D the set of all decision problems or, equivalently, T(Di) = S. Suppose that for a theory T and a decision problem Di we observe a behavior b such that b is not in S. This is not exceptional as any behavioral economist will tell you. What can we do? The first solution is the (naïve) Popperian one: discard T and adopt an alternative theory T’. This is the behavioral economists’ solution when they defend cumulative prospect theory against expected utility theory. The other solution is to stipulate that actually i is not facing decision problem Di but rather decision problem Di’, where T(Di’) = S’ and b ∈ S’. If we adopt this solution, then the only way to make T falsifiable is to limit the range of admissible redefinitions of any decision problem. If theory T is not able to account to some implication b under all the range of admissible descriptions, then it will be falsified. However, it is clear that to define such a range of admissible descriptions necessitates making substantive assumptions over what are rationalizable preferences. Hence, this leads one toward view (C)!

Views (A) and (C) are clearly incompatible. The former has been defended by contemporary proponents of variants of revealed preference theory such as Ken Binmore or Gul and Pesendorfer. Don Ross provides the most sophisticated philosophical defense of this view. View (C) is more likely to be endorsed by behavioral economists and also by some heterodox economists. Both have however a major (and problematic for some scholars) implication once the rationality concept is no longer understood positively (are people rational?) but from an evaluative and normative perspective (what is it to be rational?). Indeed, one virtue of view (B) is that it nicely ties together positive and normative economics. In particular, if it appears that people are sufficiently rational, then the consumer’s sovereignty principle will permit to make welfare judgments on the basis of people’s choices. But this is no longer true under views (A) and (C). Under the former, it is not clear why we should grant any normative significance to the fact that economic agent make consistent choices, in particular because these agents have not to be necessarily flesh-and-bones persons (they can be temporal selves). Welfare judgments can still be formally made but they are not grounded on any theory of rationality. A normative account of agency and personality is likely to be required to make any convincing normative claim. View (C) cannot obviously build on the consumer’s sovereignty principle once it is recognized that people do not always choices in their personal interests. Indeed, this is the very point of the so-called “libertarian paternalism” and more broadly of the normative turn of behavioral economics. It has to face however the difficulty that today positive economics does not offer any theory of “substantively rational preferences”. The latter is rather to be found in moral philosophy and possibly in natural sciences. In any case, economics cannot do the job alone.