Economics problem
- Impact Evaluation
Gansu is the poorest province in China. Its population is primarily rural, and a high percentage of adults migrate to cities, leaving their children at home under the care of grandparents. Government officials are concerned that these left-behind children perform poorly in school because they don’t have support from their parents who have migrated. In an effort to improve the educational outcomes of high school children, the provincial governor implemented a free, after school tutoring program in all high schools. All children attending high school in Gansu are eligible to participate in the tutoring program.
Jiting is a famous Chinese economist hired by the Ministry of Education of Gansu to evaluate the impact of the tutoring program on educational performance. The outcome indicator Jiting identified is the score of high school seniors on the Gao Kao, which is China’s national, standardized college entrance exam. In order to measure the impact, Jiting selected a sample of 3 high school seniors who participated in the program and 3 high school seniors who did not participate in the program. She then acquired these students’ scores on the Gao Kao and calculated the Naïve Average Treatment Effect.
The table below provides Jiting’s data set. Column B takes value 1 if the student participated in the tutoring program and 0 if not. The students’ scores on the Gao Kao test are reported in column C.
| Table 1. Survey Data for Jiting’s Impact Evaluation | ||
| (A)
Student # |
(B)
Participated in Tutoring Program |
(C)
Observed Gao Kao Test Score |
| 1 | 1 | 90 |
| 2 | 1 | 70 |
| 3 | 1 | 80 |
| 4 | 0 | 100 |
| 5 | 0 | 80 |
| 6 | 0 | 90 |
- Calculate and report in the box below the Naïve Average Treatment Effect.
- Write down the equation that defines selection bias and provide a clear definition of the concept of selection bias in the context of this example. Your definition must show that you have a clear understanding of the concept of selection bias. Do NOT calculate a numerical value of selection bias or discuss whether or not you think selection bias exists. You will do that in later questions.
- Do you think that the Naïve Average Treatment Effect you calculated in part A suffers from selection bias? If no, justify your answer. If yes, is the selection bias positive or negative? Justify your answer.
- Assume you can “play god” and actually know the counterfactual test scores for each student. Fill in column D in the table below by picking values of the counterfactual test scores that are consistent with your answer to question C.
| Table 2. Survey Data plus Counterfactual Data | |||
| (A)
Student # |
(B)
Participated in Tutoring Program |
(C)
Observed Gao Kao Test Score |
(D)
Counterfactual Gao Kao Test Score |
| 1 | 1 | 90 | |
| 2 | 1 | 70 | |
| 3 | 1 | 80 | |
| 4 | 0 | 100 | |
| 5 | 0 | 80 | |
| 6 | 0 | 90 | |
- Using the values in Table 2 above, compute the true Average Treatment Effect.
- Using the values in Table 2 above, compute the value of Selection Bias.
