Michael Trucano wrote a long article on the topic of how to evaluate technology use in education. He tells us that utilizing randomized control trials (RCTs) “are consider a sort of ‘gold standard’ by many in the education research community.  A working paper from the the Inter-American Development Bank (DB), Technology and Child Development: Evidence from One Laptop per Child Program in Peru, has been a notable exception to many of the research reports about educational technology initiatives in developing countries that have employed methodologies that haven’t been particularly rigorous, and which have often focused on things like changes in attitudes, and perceptions of change, among groups which have been involved in various ways with such projects. There is nothing necessarily wrong with evaluations that make extensive use of things like self-reported data, of course, but the limitations of such efforts should be pretty clear as well. RCTs can be quite expensive, and difficult to do, and, depending on the circumstances at hand, there may well be other research approaches that are more viable and useful. That said, by attempting to set up and assess the results from an RCT, the IDB working paper exploring some of the potential impacts of the OLPC project in Peru has been widely cited by many researchers with serious interests in this area, and has in many ways helped raise the bar for what is expected by many international funders interested in supporting other rigorous research efforts of this sort.”

Trucano adds: “For those not familiar with the writing style and notation conventions of academic works of these sorts, these working papers may make for tough reading at times. In some cases, the papers may raise more questions than they perhaps answer, and the careful language with which conclusions are presented may frustrate policymakers looking for clear, unambigous insight into what the ‘impact’ of various types of interventions may be. (The reality of work in this area is typically much messier than what is portrayed in the marketing brochures produced by both vendors and governments alike.)  That said, many people who make decisions about large scale investments into the use of ICTs in education would do well to take some time to read through the studies and familiarize themselves with the approaches and language which characterize reports and analysis resulting from the use of research methodologies which utilize randomized control trials.”