Home PCs lower education results?
A new study with very strong credentials from Ofer Malamud (University of Chicago and NBER) and Cristian Pop-Eleches (Columbia University, Harvard University’s BREAD, and NBER) suggests that home PC ownership lowers educational outcomes.
"This result is large and statistically significant … To summarize, the evidence on GPA, college plans and the school behavior grade presented in Table 5 suggests that, if anything, computer ownership has a negative impact on child academic and behavioral outcomes. …"
This study uses regression discontinuity analysis. The Romanian government offered vouchers to low income families to buy computers. The idea is that for the two communities, those above the qualification income and those below, two regression lines can be drawn. At the income breakpoint, the two line ends represent two identical families in all respects except access to the vouchers.
If you examine the graphs at the end of the report, you will see a serious problem with the use of regression discontinuity which I believe makes the study invalid. The two regression lines for these 3 indicators have opposite slopes.
For regression discontinuity to be valid for straight line regression lines then the underlying distribution, pre-treatment, should be linear:
"The conclusion is a simple one. If the true model is curved and we fit only straight-lines, we are likely to conclude wrongly that the treatment made a difference when it did not. This is a specific instance of the more general problem of model specification."
http://www.socialresearchmethods.net/kb/statrd.php
This is how the data should look if linear regression discontinuity is to be used:
And here, roughly, is how the study's graphs of GPA, college plans and the school behavior grade look:
And to show how the discontinuity in the two regression lines does not mean anything for the two populations, here is the underlying data for the two lines above, a continuous curve.
On this basis, I believe the statistical technique used in the study is not valid.
Read more about the implications of the study at http://radian.org/notebook/distraction-machine
and
http://www.olpcnews.com/use_cases/education/constructive_or_distractive.html
"This result is large and statistically significant … To summarize, the evidence on GPA, college plans and the school behavior grade presented in Table 5 suggests that, if anything, computer ownership has a negative impact on child academic and behavioral outcomes. …"
This study uses regression discontinuity analysis. The Romanian government offered vouchers to low income families to buy computers. The idea is that for the two communities, those above the qualification income and those below, two regression lines can be drawn. At the income breakpoint, the two line ends represent two identical families in all respects except access to the vouchers.
If you examine the graphs at the end of the report, you will see a serious problem with the use of regression discontinuity which I believe makes the study invalid. The two regression lines for these 3 indicators have opposite slopes.
For regression discontinuity to be valid for straight line regression lines then the underlying distribution, pre-treatment, should be linear:
"The conclusion is a simple one. If the true model is curved and we fit only straight-lines, we are likely to conclude wrongly that the treatment made a difference when it did not. This is a specific instance of the more general problem of model specification."
http://www.socialresearchmethods.net/kb/statrd.php
This is how the data should look if linear regression discontinuity is to be used:
And here, roughly, is how the study's graphs of GPA, college plans and the school behavior grade look:
And to show how the discontinuity in the two regression lines does not mean anything for the two populations, here is the underlying data for the two lines above, a continuous curve.
On this basis, I believe the statistical technique used in the study is not valid.Read more about the implications of the study at http://radian.org/notebook/distraction-machine
and
http://www.olpcnews.com/use_cases/education/constructive_or_distractive.html
posted by Unknown at 12:39 PM
3 Comments:
OK, the study's results are invalid. But if that's the real data, it doesn't look as if things are too noisy. So a curve fit would not be overfitting, and, just by eye, you can tell that with two curve fits, you'd find no effect.
So, the valid result would be that the computers don't have harmful effects, but rather have no effect. Which is still, generally speaking, an argument against spending lots of money on computers.
Nevertheless, this is important. Have you written to the journal and the study authors?
Hi Anonymous
No, this is not the real data in my blog, just test data to demonstrate that two lines that do not meet do not indicate a disconuity in the data.
I don't think any conclusions can be drawn from the study.
I have not contacted the journal or the authors.
Post a Comment
<< Home