Experiments in Social Science
Introduction to Social and Economic Science
12 January 2010
The design, difficulties, and results of social science experiments,
Part I.
Each part contains a brief introduction to experimental social
science, and descriptions of some real experiments. The introduction
is the same, but the experiments are different. You must attend two
parts, and in the third you will be participating in a real experiment.
Economics is a science
- In theory, we ask for logical and quantitative analysis.
- We require that theory be verified by empirical (practical) tests.
- The tests must correspond to the models.
- In general, tests can based on history or experiment.
Historical tests
- Does historical data "fit" the model?
- Use statistics:
- Analysis of variance for qualitative models (psychology).
- Regression analysis for quantitative models (economics).
- Data sources:
- Collected for other purposes. E.g., census, trade and, income
data are collected by the government to create election districts
and levy taxes, internal corporate data for management information.
- Designed surveys. Consumer behavior: Michigan Panel Survey,
National Longitudinal Survey.
- Combinations.
Problems of historical tests
- Varying quality of data, especially over time.
- Measured variables do not correspond to theoretical variables.
- Utility can't be measured.
- There is no market for "automobiles" in the sense of the Law of
One Price and market clearing.
- Prices are averages over models and transactions (bargaining).
- Quantities are averages.
- Market clearing is achieved by inventory adjustment.
Experimental tests
- Experiments are designed to fit theory.
- Extraneous variables are controlled.
- Relevant variables match theoretical definitions.
- Relevant variables are recorded.
- Equations are calibrated to match theory.
Problems with experiments
- Framing effects:
- Stakes are small. Real economic behavior is "you bet your
life/company/country"; can't do that with experiments.
- Typical experimental subjects (college students) are not real
decision-makers (housewives, company president, country Prime
Minister).
- Experiments normally restricted to gains, but agents may view
losses differently.
- Not all interesting variables can be measured (e.g., utility).
- Basic assumptions needed for measurement may be violated (e.g.,
expected utility).
- Nevertheless, experiments are useful. (Including an endless source
of graduation research themes!)
Expected Utility
- We suppose that you are offered a chance to win one of three
prizes, A, B, or C. You get utility U(A), U(B), or U(C) depending
on which prize you receive.
- A lottery is a set of probabilities, p(A), p(B), and p(C)
corresponding to each prize such that p(A) + p(B) + p(C) = 1.
- If the probabilities are p(A) of winning A, p(B) of winning B, and
p(C) of winning C, your expected utility of the lottery p is
V(p) = p(A)*U(A) + p(B)*U(B) + p(C)*U(B).
Choice of lottery experiment
- First, you are offered the choice between lotteries
- Option 1: you get A with probability p(A), or B with probability
p(B). Of course p(A) + p(B) = 1.
- Option 2: you get A with probability q(A), or B with probability
q(B). Of course q(A) + q(B) = 1.
Choice of lottery experiment, cont.
- Second, you are offered the choice between the lotteries
- Option 3: you get A with probability rp(A), or B with probability
rp(B), or C with probability 1 - r.
Of course rp(A) + rp(B) + (1 - r) = 1.
- Option 4: you get A with probability rq(A), or B with probability
rq(B), or C with probability 1 - r.
Of course rq(A) + rq(B) + (1 - r) = 1.
- The p's, q's, and r's are the same in every option.
- According to expected utility theory, you will choose Option 1 in
the first treatment if and only if you choose Option 3 in the second
treatment.
- Homework: Prove it using the expected utility formula.
Preference reversal
- Suppose A = 10,000 yen, B = 12,000 yen, and C = 0 yen.
- Some people who definitely prefer 1 to 2, will also insist they
like 4 better than 3.
- This "violates" expected utility, and so is called a "paradox".
But expected utility is not a law, just an hypothesis, so
"preference reversal" is more accurate.
- Quite rare. Maybe a very few people just don't understand, but the
economy as a whole can be predicted by expected utility.
Allais Paradox
- p, q, and r given as before, all positive. r is "big".
- Treatment 1: A = 10,000 yen, B = 12,000 yen, and C = 0 yen.
- Option 1: you get A with probability rp(A), or B with probability
rp(B), or C with probability 1 - r.
- Option 2: you get A with probability rq(A), or B with probability
rq(B), or C with probability 1 - r.
- Treatment 2: A = 10,000, B = 12,000, and C = 8000 yen.
- Option 3: you get A with probability rp(A), or B with probability
rp(B), or C with probability 1 - r.
- Option 4: you get A with probability rq(A), or B with probability
rq(B), or C with probability 1 - r.
- Preference reversal is quite common.
- Logic: when C is big, base level is secure, avoid too much risk.
When C is small, not much chance of positive prize, so choose best
chance of getting B (the biggest prize).
Ultimatum game: Roth experiment
- Economists A. Roth (US), M. Okuno-Fujiwara (Japan), M. Maschler
(Israel)
- Ultimatum game:
- Total prize of 100 dollars (10,000 yen).
- Player 1 proposes his share, from 0% to 100% of prize.
- Player 2 responds "yes" or "no".
- If "yes", split according to Player 1's proposal; if "no", both
players get zero.
- Equilibrium: Player 1 demands 99%, player 2 says "yes".
- Control conditions: players are in individual rooms, never meet
other players, play only once.
Roth game results
- Japan: great majority propose 50%, Player 2 says "yes". Average
demand about 55%, almost all player 2 role say "yes".
- US: near majority propose 50%, Player 2 says "yes". Average
demand about 65%, most player 2 role say "yes", but disagreement is
not uncommon.
- Israel: about 1/3 propose 50%, Player 2 says "yes". Average
demand about 85%, demands over 90% almost always refused.
Ohio State University experiment
- Another ultimatum game. Differences:
- Multi-player: One player 1, many player 2 all with veto (if one
"no", all get zero).
- Players are game theorists and mathematical economists (Roth
experiment, college students).
- Controls poor (the participants know each other very well).
- Result: Player 1 demands $990, offering $1 to each of the others.
5 say yes, 5 say no.
Market games
- Market games are usually constructed on the reservation price
principle.
- Each buyer may buy one unit per market. If he does, he sells it
to the experimenter for a fixed price, set in advance. Buyers
want to pay a lot less than the fixed price, but make some profit
as long as they do pay less. The fixed price is that buyer's
reservation price.
- Each seller may sell one unit per market. If he does not, the
experimenter will pay a fixed price (the seller's reservation
price) for the left over good. Sellers want to receive a higher
price than their reservation price, of course.
Market games, cont.
- Controls: Demand and supply curve defined by number of buyers
(respectively, sellers) with each reservation price.
- No inventory; market must clear.
- Need market price setting.
Oral Double auction
- Auctioneer controls.
- Either buyer or seller may speak with permission, to say "buy at
<price>" or "sell at <price>", or "accept". Offers are publicly
posted.
- "Buy at" and "sell at" are binding offers; if accepted you must
trade.
- The offer accepted is the most advantageous for the trader who
accepts.
- All offers are cleared after each trade, and the process continues
in the same way until no traders will accept.