Learning and enrollment gains of the world’s first Development Impact Bond
Client: Children’s Investment Fund Foundation (CIFF)
Partners: Educate Girls, Instiglio, UBS Optimus Foundation
Dates of service: 2012 – Present
IDinsight service: Randomized evaluation
IDinsight contacts: Jeffery McManus, Kate Sturla
Ramavatar Dhonli, a surveyor, records a child’s responses to the ASER learning assessment on a mobile phone during piloting for the EG DIB in September 2015. (Photo credit: Kate Sturla)
Nearly three million girls, aged 6 to 13, are out of school in India. In the state of Rajasthan, 40% of girls leave primary school before class 5 and 50% of girls are married before the legal age. Educate Girls is a Rajasthan-based NGO that trains community volunteers to encourage families to enroll their girls in school (by making door-to-door household visits) and to deliver a child-friendly supplementary curriculum in classrooms to improve basic reading and math skills. Educate Girls currently operates in over 8,000 villages and 12,000 primary schools across Rajasthan.
A development impact bond (DIB) has been set up to finance a portion of Educate Girls’s services. The DIB, which is the first in the world to be implemented, ties learning outcomes to financial returns for a private-sector investor. The lessons from this DIB will inform the viability and structure of future social impact opportunities for donors, investors, and service providers.
To determine the financial returns of the DIB and estimate the impact of Educate Girls’s program on improving learning outcomes, IDinsight designed and is currently implementing a clustered randomized evaluation. The evaluation involves a sample of over 4,000 students in 396 schools in the region where Educate Girls is working. Half of these schools have been randomly assigned to receive Educate Girls’s program, while the other half are the comparison group (schools not receiving Educate Girls’s program). IDinsight is measuring two outcomes – learning gains and enrollment of out-of-school girls – which will form the basis of the DIB payments at the end of the project.
How we’re measuring learning gains
1. During the Year 1 baseline in September 2015, IDinsight visited all treatment and control schools and administered a version of the ASER assessment to a randomly selected subset of students in grades 1-5. IDinsight teams visited both treatment and control schools in the same geographic cluster on the same day. Students received a numeric score on each of the three sections of the assessment (English, Hindi, and Math), and their overall baseline score was the sum of these grades (possible scores: 3-16).
2. During the Year 1 endline in February 2016, IDinsight administered a new assessment to students in grades 3-5 who were previously assessed during the baseline. If students were not at school, they were visited at their homes for a follow-up rate of 98.8%.
3. IDinsight also surveyed a subset of newly-enrolled students during the endline. These students were not assessed at baseline, so their baseline scores were imputed.
4. Steps (2) and (3) will be repeated during the Year 2 and Year 3 endlines in February 2017 and 2018, respectively. The estimates in learning gains from the Year 3 endline will be used to determine the financial return on the bond.
How we’re measuring enrollment
To determine payments for the DIB based on the enrollment of out-of-school girls, IDinsight is conducting a simple pre-post evaluation in the 169 villages where treatment schools are located. This is not a randomized controlled trial, and thus these results cannot be taken as an estimate of the impact of Educate Girls’s program on enrollment.
1. In August 2015, Educate Girls compiled a census of all out-of-school girls ages 7-14 living in treatment villages. IDinsight validated that list by visiting a sample of girls in the area and verifying their eligibility.
2. Before the Year 1 endline, Educate Girls submitted a list of girls they had successfully enrolled in school. IDinsight checked their enrollment against school records.
3. In Years 2 and 3 of the evaluation, Educate Girls will submit a list of additions to the out-of-girl census each August for IDinsight (e.g., girls who turned seven the previous year and didn’t enroll in school). IDinsight will verify any enrollments Educate Girls reports making off of that list during the Year 2 and Year 3 endlines.
IDinsight found positive learning and enrollment gains after Year 1.
1. Students in treatment schools had 30% greater increase in scores on the ASER assessment than students in control schools, which corresponds to 23% of the learning gains target stipulated in the DIB contract.
2. Educate Girls enrolled 44% of out-of-school girls identified in treatment villages.
3. For lead investor UBS, about 40% of their initial $267,000 investments has been recovered.
To read more, click on the below links:
About the Development Impact Bond: http://bit.ly/2a0YFSy
About IDinsight’s evaluation design and process: http://bit.ly/2badJQe
SSIR’s “Lessons from a Development Impact Bond”: http://bit.ly/2b5LjVM
ODI event convening: http://bit.ly/2a6o7d2
The Brookings Webinar: http://brook.gs/2ag5Wk7