Experiments in Social Science
Introduction to Social and Economic Science
26 January 2010
The design, difficulties, and results of social science experiments,
Part II.
Each part contains a brief introduction to experimental social
science, and descriptions of some real experiments. The introduction
is the same, but the experiments are different. You must attend two
parts, and in the third you will be participating in a real experiment.
Economics is a science
- In theory, we ask for logical and quantitative analysis.
- We require that theory be verified by empirical (practical) tests.
- The tests must correspond to the models.
- In general, tests can based on history or experiment.
Historical tests
- Does historical data "fit" the model?
- Use statistics:
- Analysis of variance for qualitative models (psychology).
- Regression analysis for quantitative models (economics).
- Data sources:
- Collected for other purposes. E.g., census, trade and, income
data are collected by the government to create election districts
and levy taxes, internal corporate data for management information.
- Designed surveys. Consumer behavior: Michigan Panel Survey,
National Longitudinal Survey.
- Combinations.
Problems of historical tests
- Varying quality of data, especially over time.
- Measured variables do not correspond to theoretical variables.
- Utility can't be measured.
- There is no market for "automobiles" in the sense of the Law of
One Price and market clearing.
- Prices are averages over models and transactions (bargaining).
- Quantities are averages.
- Market clearing is achieved by inventory adjustment.
Experimental tests
- Experiments are designed to fit theory.
- Extraneous variables are controlled.
- Relevant variables match theoretical definitions.
- Relevant variables are recorded.
- Equations are calibrated to match theory.
Problems with experiments
- Framing effects:
- Stakes are small. Real economic behavior is "you bet your
life/company/country"; can't do that with experiments.
- Typical experimental subjects (college students) are not real
decision-makers (housewives, company president, country Prime
Minister).
- Experiments normally restricted to gains, but agents may view
losses differently.
- Not all interesting variables can be measured (e.g., utility).
- Basic assumptions needed for measurement may be violated (e.g.,
expected utility).
- Nevertheless, experiments are useful. (Including an endless source
of graduation research themes!)
Expected Utility
- We suppose that you are offered a chance to win one of three
prizes, A, B, or C. You get utility U(A), U(B), or U(C) depending
on which prize you receive.
- A lottery is a set of probabilities, p(A), p(B), and p(C)
corresponding to each prize such that p(A) + p(B) + p(C) = 1.
- If the probabilities are p(A) of winning A, p(B) of winning B, and
p(C) of winning C, your expected utility of the lottery p is
V(p) = p(A)*U(A) + p(B)*U(B) + p(C)*U(B).
Choice of lottery experiment
- First, you are offered the choice between lotteries
- Option 1: you get A with probability p(A), or B with probability
p(B). Of course p(A) + p(B) = 1.
- Option 2: you get A with probability q(A), or B with probability
q(B). Of course q(A) + q(B) = 1.
Choice of lottery experiment, cont.
- Second, you are offered the choice between the lotteries
- Option 3: you get A with probability rp(A), or B with probability
rp(B), or C with probability 1 - r.
Of course rp(A) + rp(B) + (1 - r) = 1.
- Option 4: you get A with probability rq(A), or B with probability
rq(B), or C with probability 1 - r.
Of course rq(A) + rq(B) + (1 - r) = 1.
- The p's, q's, and r's are the same in every option.
- According to expected utility theory, you will choose Option 1 in
the first treatment if and only if you choose Option 3 in the second
treatment.
- Homework: Prove it using the expected utility formula.
Preference reversal
- Suppose A = 10,000 yen, B = 12,000 yen, and C = 0 yen.
- Some people who definitely prefer 1 to 2, will also insist they
like 4 better than 3.
- This "violates" expected utility, and so is called a "paradox".
But expected utility is not a law, just an hypothesis, so
"preference reversal" is more accurate.
- Quite rare. Maybe a very few people just don't understand, but the
economy as a whole can be predicted by expected utility.
Allais Paradox
- p, q, and r given as before, all positive. r is "big".
- Treatment 1: A = 10,000 yen, B = 12,000 yen, and C = 0 yen.
- Option 1: you get A with probability rp(A), or B with probability
rp(B), or C with probability 1 - r.
- Option 2: you get A with probability rq(A), or B with probability
rq(B), or C with probability 1 - r.
- Treatment 2: A = 10,000, B = 12,000, and C = 8000 yen.
- Option 3: you get A with probability rp(A), or B with probability
rp(B), or C with probability 1 - r.
- Option 4: you get A with probability rq(A), or B with probability
rq(B), or C with probability 1 - r.
- Preference reversal is quite common.
- Logic: when C is big, base level is secure, avoid too much risk.
When C is small, not much chance of positive prize, so choose best
chance of getting B (the biggest prize).
Ultimatum game: Roth experiment
- Economists A. Roth (US), M. Okuno-Fujiwara (Japan), M. Maschler
(Israel)
- Ultimatum game:
- Total prize of 100 dollars (10,000 yen).
- Player 1 proposes his share, from 0% to 100% of prize.
- Player 2 responds "yes" or "no".
- If "yes", split according to Player 1's proposal; if "no", both
players get zero.
- Equilibrium: Player 1 demands 99%, player 2 says "yes".
- Control conditions: players are in individual rooms, never meet
other players, play only once.
Roth game results
- Japan: great majority propose 50%, Player 2 says "yes". Average
demand about 55%, almost all player 2 role say "yes".
- US: near majority propose 50%, Player 2 says "yes". Average
demand about 65%, most player 2 role say "yes", but disagreement is
not uncommon.
- Israel: about 1/3 propose 50%, Player 2 says "yes". Average
demand about 85%, demands over 90% almost always refused.
Ohio State University experiment
- Another ultimatum game. Differences:
- Multi-player: One player 1, many player 2 all with veto (if one
"no", all get zero).
- Players are game theorists and mathematical economists (Roth
experiment, college students).
- Controls poor (the participants know each other very well).
- Result: Player 1 demands $990, offering $1 to each of the others.
5 say yes, 5 say no.
Market games
- Market games are usually constructed on the reservation price
principle.
- Each buyer may buy one unit per market. If he does, he sells it
to the experimenter for a fixed price, set in advance. Buyers
want to pay a lot less than the fixed price, but make some profit
as long as they do pay less. The fixed price is that buyer's
reservation price.
- Each seller may sell one unit per market. If he does not, the
experimenter will pay a fixed price (the seller's reservation
price) for the left over good. Sellers want to receive a higher
price than their reservation price, of course.
Market games, cont.
- Controls: Demand and supply curve defined by number of buyers
(respectively, sellers) with each reservation price.
- No inventory; market must clear.
- Need market price setting.
Oral Double auction
- Auctioneer controls.
- Either buyer or seller may speak with permission, to say "buy at
<price>" or "sell at <price>", or "accept". Offers are publicly
posted.
- "Buy at" and "sell at" are binding offers; if accepted you must
trade.
- The offer accepted is the most advantageous for the trader who
accepts.
- All offers are cleared after each trade, and the process continues
in the same way until no traders will accept.
Designing a Market: The Oil Market Experiment
- What is (economically) special about oil?
- It is a pure exhaustible resource.
- Consumers are perfectly competitive.
- Producers are an oligopoly.
Pure Exhaustible Resources
- Cannot be produced (unlike cars) and are not self-reproducing
(unlike tuna).
- Sustainable use is impossible.
- Well-developed theory of optimal use and market behavior because
constraints are simple.
Consumer Behavior
- Everybody needs energy, and uses oil energy in many forms: gasoline,
heating oil, electricity, jet fuel.
- Also use in products as a base for many plastics and as a lubricant.
- Compared to market demand, consumers are small and many. These are
conditions that lead to perfect competition (price-taking
behavior).
- We model consumers as a demand curve.
- For simplicity, we simply take the same demand curve in each
period.
Seller Behavior
- Producers are few and large.
- We model producers as an oligopoly. They behave
strategically, that is
- They take the effect of their own behavior on price into account.
- They take the effect of others' behavior on price into account,
and furthermore know that others will behave strategically.
- Strategic behavior is studied in game theory. Unlike the
competitive market, economists and other social scientists still
disagree about what the best theory of strategic behavior is.
Seller Actions in the Oil Market
- Sellers in the oil market must possess a stock to be able to sell;
it is not possible to produce new oil for sale.
- Quantities of oil (or the right to extract that oil from the ground)
are bought and sold. Thus, sellers may invest in purchasing
stocks as well as draw down the existing stock to sell to
consumers (or investors).
- In our market, we allow traders to buy and sell. Of course, they
buy in hope of selling at a higher price later.
The Planning Horizon for an Exhaustible Resource
- If we have a stock of X barrels of oil (perhaps still in the
ground), and we use Q barrels per year, then we will run out of
oil in T = X/Q years.
- But we will still need energy in T+1 years. There is always a
"tomorrow" we must consider when planning to use a an exhaustible
resource today.
- In theoretical models, we assume that decision-makers plan for an
infinite horizon.
- In an experiment, however, we can't wait for infinity, even if we
pretend that a year passes every five minutes.
Discounting in Dynamic Problems
- An important aspect of long-term planning is discounting: people
give less weight to rewards received in the future.
- In theory, we model this with discounted utility: V(c1,c2,c3...) =
U(c1) + d x U(c2) + d x d x U(c3) + ..., where c1 is received today,
c2 is received next year, c3 the year after that, and so on. 0
< d < 1.
- However, in an experiment, the reward is received all at once at
the end.
Solving the Horizon and Discounting Problems
- Both problems can be solved in the same way: by randomly stopping
the game with probability 1 - d (continuing with probability d).
- Randomly stopping allows the game to end in finite time.
- If R is revenue to the subject, the probability of getting the
prize is proportional to the revenue at some factor a.
- With random stopping by the above rule, the probability of getting
the prize is p(A) = a x (R1 + d x R2 + d x d x R3 + ...).
- But with a lottery, p(A) is equivalent to utility!