Identification for Prediction and Decision

This book is a full-scale exposition of Charles Manski's new methodology for analyzing empirical questions in the social sciences. He recommends that researchers first ask what can be learned from data alone, and then ask what can be learned when data are combined with credible weak assumptions. Inferences predicated on weak assumptions, he argues, can achieve wide consensus, while ones that require strong assumptions almost inevitably are subject to sharp disagreements.

Building on the foundation laid in the author's Identification Problems in the Social Sciences (Harvard, 1995), the book's fifteen chapters are organized in three parts. Part I studies prediction with missing or otherwise incomplete data. Part II concerns the analysis of treatment response, which aims to predict outcomes when alternative treatment rules are applied to a population. Part III studies prediction of choice behavior.

Each chapter juxtaposes developments of methodology with empirical or numerical illustrations. The book employs a simple notation and mathematical apparatus, using only basic elements of probability theory.

More than anyone else, Charles Manski has changed the way we think about identification. This book contains the most comprehensive discussion of his work in this area. It is a must-read for everybody interested in identification, and there isn't an empirical economist or econometrician who can afford not to be. -- Gerald M. Howard, Executive Vice President and Chief Executive Officer, National Association of Home Builders

Charles Manski is a highly original and influential voice in econometrics. His work on partial identification and nonparametric bounds now holds a central position in many areas of theoretical and applied research. This comprehensive yet accessible text brings together the author's research on incomplete data, on treatment response and on choice behavior. It is an important contribution to our knowledge and will stand as a key reference for students and researchers for years to come. -- Guido Imbens, Harvard University

This book is a full-scale exposition of Manski's new methodology for analyzing empirical questions in the social sciences. He recommends that researchers ask first what can be learned from data alone, and then what can be learned when data are combined with credible weak assumptions. Each chapter juxtaposes developments of methodology with empirical or numerical illustrations.

Charles F. Manski is Board of Trustees Professor of Economics at Northwestern University.

Board of Trustees Professor of Economics, Northwestern University

Preface

Introduction

• The Reflection Problem
• The Law of Decreasing Credibility
• Identification and Statistical Inference
• Prediction and Decisions
• Coping with Ambiguity
• Organization of the Book
• The Developing Literature on Partial Identification

PART I: PREDICTION WITH INCOMPLETE DATA

1. Conditional Prediction

1.1. Predicting Criminality

1.2. Probabilistic Prediction

• Conditional Distributions
• Best Predictors
• Specifying a Loss Function

1.3. Estimation of Best Predictors from Random Samples

• Covariates with Positive Probability
• Covariates with Zero Probability but on the Support
• Covariates off the Support

1.4. Extrapolation

• Invariance Assumptions and Shape Restrictions
• Testing and Using Theories

1.5. Predicting High School Graduation

Complement 1A. Best Predictors Under Square and Absolute Loss

• Square Loss
• Absolute Loss

Complement 1B. Nonparametric Regression Analysis

• Consistency of the Local-Average Estimate
• Choosing an Estimate

Complement 1C. Word Problems

2. Missing Outcomes

2.1. Anatomy of the Problem

• Identification of Event Probabilities
• Identification of Quantiles

2.2. Bounding the Probability of Exiting Homelessness

• Is the Cup Part Empty or Part Full?

2.3. Means of Functions of the Outcome

• Bounded Random Variables
• Unbounded Random Variables

2.4. Parameters that Respect Stochastic Dominance

2.5. Distributional Assumptions

• Missingness at Random
• Refutable and Non-refutable Assumptions
• Refutability and Credibility

2.6. Wage Regressions and the Reservation Wage Model of Labor Supply

• Homogeneous Reservation Wages
• Other Cases of Missingness by Choice

2.7. Statistical Inference

• Sample Analogs of Identification Regions
• Confidence Sets
• Testing Refutable Assumptions

Complement 2A. Interval Measurement of Outcomes

• Measurement Devices with Bounded Range

Complement 2B. Jointly Missing Outcomes and Covariates

• Conditioning on a Subset of the Outcomes
• Illustration: Bounding the Probability of Employment and the Unemployment Rate

Complement 2C. Convergence of Sets to Sets

3. Instrumental Variables

3.1. Distributional Assumptions and Credible Inference

• Assumptions Using Instrumental Variables

3.2. Missingness at Random

• Conditioning is not Controlling

3.3. Statistical Independence

• Binary Outcomes
• Identifying Power
• Combining Multiple Surveys

3.4. Equality of Means

• Means Missing at Random
• Mean Independence

3.5. Inequality of Means

• Means Missing Monotonically
• Monotone Regressions

Complement 3A. Imputations and Nonresponse Weights

• Imputations
• Nonresponse Weights

Complement 3B. Conditioning on the Propensity Score

Complement 3C. Word Problems

4. Parametric Prediction

4.1. The Normal-Linear Model of Market and Reservation Wages

4.2. Selection Models

• A Semiparametric Model

4.3. Parametric Models for Best Predictors

• Identification of the Parameters and the Best Predictor
• Linear-Index Models
• Statistical Inference

Complement 4A. Minimum-Distance Estimation of Partially Identified Models

5. Decomposition of Mixtures

5.1. The Inferential Problem and Some Manifestations

• The Problem in Abstraction
• Ecological Inference
• Contaminated Sampling
• The Task Ahead

5.2. Binary Mixing Covariates

• Inference on One Component Distribution
• Event Probabilities
• Parameters that Respect Stochastic Dominance

5.3. Contamination Through Imputation

• Income Distribution in the United States
• Corrupted Sampling

5.4. Instrumental Variables

• The Identification Region

Complement 5A. Sharp Bounds on Parameters that Respect Stochastic Dominance

6. Response-Based Sampling

6.1. The Odds Ratio and Public Health

• Relative and Attributable Risk
• The Rare-Disease Assumption

6.2. Bounds on Relative and Attributable Risk

• Relative Risk
• Attributable Risk

6.3. Information on Marginal Distributions

6.4. Sampling from One Response Stratum

• Using Administrative Records to Infer AFDC Transition Rates

6.5. General Binary Stratifications

• Sampling from Both Strata
• Sampling from One Stratum

PART II: ANALYSIS OF TREATMENT RESPONSE

7. The Selection Problem

7.1. Anatomy of the Problem

• Prediction Using the Empirical Evidence Alone
• Comparing Treatments
• Average Treatment Effects
• Distributional Assumptions

7.2. Sentencing and Recidivism

7.3. Randomized Experiments

• Experiments in Practice

7.4. Compliance with Treatment Assignment

• Experiments without Crossover
• Experiments with Crossover
• Point Identification with Partial Compliance
• Intention to Treat
• The Effect of Treatment on Compliers

7.5. Treatment by Choice

• Outcome Optimization
• Parametric Selection Models

7.6. Treatment at Random in Non-Experimental Settings

• Association and Causation
• Sensitivity Analysis

7.7. Homogeneous Linear Response

• "The" Instrumental Variables Estimator
• Mean Independence and Over-identification

Complement 7A. Perspectives on Treatment Comparison

• Differences in Outcome Distributions or Distributions of Outcome Differences
• The Population to be Treated or the Sub-population of the Treated

Complement 7B. Word Problems

8. Linear Simultaneous Equations

8.1. Simultaneity in Competitive Markets

• "The" Identification Problem in Econometrics
• Simultaneity is Selection

8.2. The Linear Market Model

• Credibility of the Assumptions
• Analysis of the Reduced Form

8.3. Games with Linear Reaction Functions

• Ehrlich, the Supreme Court, and the National Research Council

8.4. The Reflection Problem

• Endogenous, Contextual, and Correlated Effects
• The Linear-in-Means Model
• Identification of the Parameters
• Inferring the Composition of Reference Groups

9. Monotone Treatment Response

9.1. Shape Restrictions

• Downward Sloping Demand
• Production Analysis

9.2. Bounds on Parameters that Respect Stochastic Dominance

• The General Result
• Means of Increasing Functions of the Outcome
• Upper Tail Probabilities

9.3. Bounds on Treatment Effects

• Average Treatment Effects

9.4. Monotone Response and Selection

• Interpreting the Statement "Wage Increases with Schooling"
• Bounds on Mean Outcomes and Average Treatment Effects

9.5. Bounding the Returns to Schooling

• Data
• Statistical Considerations
• Findings

10. The Mixing Problem

10.1. Extrapolation from Experiments to Rules With Treatment Variation

• From Marginals to Mixtures

10.2. Extrapolation from the Perry Preschool Experiment

• Prediction with the Experimental Evidence Alone
• Prediction with Assumptions

10.3. Identification of Event Probabilities with the Experimental Evidence Alone

10.4. Treatment-Response Assumptions

• Statistically Independent Outcomes
• Monotone Treatment Response

10.5. Treatment-Rule Assumptions

• Treatment at Random
• Outcome Optimization
• Known Treatment Shares

10.6. Combining Assumptions

11. Planning Under Ambiguity

11.1. Studying Treatment Response to Inform Treatment Choice

• Partial Identification and Ambiguity

11.2. Criteria for Choice Under Ambiguity

• Dominance
• Bayes Rules
• The Maximin Criterion
• The Minimax-Regret Criterion

11.3. Treatment Using Data from an Experiment with Partial Compliance

• The Illinois UI Experiment

11.4. An Additive Planning Problem

• The Choice Set
• The Objective Function and the Optimal Treatment Rule
• The Value of Covariate Information
• Non-Separable Planning Problems

11.5. Planning with Partial Knowledge of Treatment Response

• The Study Population and the Treatment Population
• Planning Under Ambiguity

11.6. Planning and the Selection Problem

• Bayes Rules
• The Maximin Criterion
• The Minimax-Regret Rule
• Sentencing Juvenile Offenders

11.7. The Ethics of Fractional Treatment Rules

• Choosing Treatments for X-Pox

11.8. Decentralized Treatment Choice

• The Informational Argument for Decentralization
• Decentralized Treatment of X-Pox

Complement 11A. Minimax-Regret Rules for Two Treatments are Fractional

Complement 11B. Reporting Observable Variation in Treatment Response

Complement 11C. Word Problems

12. Planning with Sample Data

12.1. Statistical Induction

12.2. Wald's Development of Statistical Decision Theory

• The Expected Welfare of a Statistical Treatment Rule
• The States of Nature
• Admissibility
• Implementable Criteria for Treatment Choice
• Unification of Identification, Statistical Inference, and Sample Design

12.3. Using a Randomized Experiment to Evaluate an Innovation

• The Setting
• The Admissible Treatment Rules
• Some Monotone Rules
• Savage on the Maximin and Minimax-Regret Criteria

PART III: PREDICTING CHOICE BEHAVIOR

13. Revealed Preference Analysis

13.1. Revealing the Preferences of an Individual

• Observation of One Choice Setting
• Observation of Multiple Choice Settings
• Application to General Choice Problems
• Thought Experiment or Practical Prescription for Prediction?

13.2. Random Utility Models of Population Choice Behavior

• Consistency with Utility Theory
• Prediction using Attributes of Alternatives and Decision Makers
• Incomplete Data and Conditional Choice Probabilities
• Practicality Through the Conditional Logit Model
• Other Distributional Assumptions
• Extrapolation

13.3. College Choice in America

• An Idealized Binary Choice Setting
• Predicting the Enrollment Effects of Student Aid Policy
• Power and Price of the Analysis

13.4. Random Expected-Utility Models

• Identification of the Decision Rules of Proposers in Ultimatum Games
• Rational Expectations Assumptions
• How do Youth Infer the Returns to Schooling?

Complement 13A. Prediction Assuming Strict Preferences

Complement 13B. Axiomatic Decision Theory

14. Measuring Expectations

14.1. Elicitation of Expectations from Survey Respondents

• Attitudinal Research
• Probabilistic Expectations in Cognitive Psychology
• Probabilistic Expectations in Economics

14.2. Illustrative Findings

• Response Rates and Use of the Percent-Chance Scale
• One-year-ahead Income Expectations
• Social Security Expectations

14.3. Using Expectations Data to Predict Choice Behavior

• Choice Expectations
• Using Expectations and Choice Data to Estimate Random Expected-Utility Models

14.4. Measuring Ambiguity

Complement 14A. The Predictive Power of Intentions Data: A Best-Case Analysis

• Rational Expectations Responses to Intentions Questions
• Prediction of Behavior Conditional on Intentions
• Prediction Not Conditioning on Intentions
• Interpreting Fertility Intentions

Complement 14B. Measuring Expectations of Facts

• Anchoring

15. Studying Human Decision Processes

15.1. As-If Rationality and Bounded Rationality

• The As-If Argument of Friedman and Savage
• Simon and Bounded Rationality

15.2. Choice Experiments

• Heuristics and Biases
• Widespread Irrationality or Occasional Cognitive Illusions?

15.3. Prospects for a Neuroscientific Synthesis

References