This year’s Nobel laureates Abhijit Banerjee, Esther Duflo and Michael Kremer are amongst the pioneers of Randomized Controlled Trials (RCTs), an ‘experimental approach to alleviate global poverty’. Essentially, it transports societal economics from desks and theories and brings it on-ground, where it is ideally expected to make an impact. In fact, the husband-wife duo of Abhijit and Esther practiced their study in Indian hinterlands, collaborating with non-profit organizations like Seva Mandir and Pratham, and conducted RCTs to scientifically evaluate various targeted anti-poverty programmes in India.
This radically new approach sparked Educational Initiatives’ implementation of the Mindspark program in Rajasthan, associating with the state government to test the impact of classroom-based ICT program amongst the most underserved and resource bereft children of rural India.
The implementation of the program was sponsored by Global Innovation Fund – where Michael Kremer and Esther Duflo the winners of Nobel Sveriges Riksbank Prize in Economic Sciences were among the founding Board Members. This grant was on the basis of the Mindspark program demonstrating strong results in a J-PAL RCT of the program in Delhi urban slums. The proposal was to do a pilot in government schools to show the blueprint of how EdTech can be integrated into the govt school system to improve learning outcomes at scale.
The ground reality
Despite high enrolment rates in recent years, improvement in reading outcomes and arithmetic ability remain alarmingly low. The Annual Status of Education Report (ASER) 2018 reveals 56% of students in Class VIII struggle to solve simple numerical division problems, whereas 72% of students in Class V do not even know how to divide. The report, released by NGO Pratham earlier this year, clearly indicate that learning levels of students in India are well below age-appropriate levels.
In response to this widespread learning crisis, the Government of India has allocated a great deal of funding to support classroom-based ICT programs that promote computer literacy and develop innovative curricula in electronic formats. Yet, there is little evidence to date that suggests a strong correlation between public investment in education and learning outcomes.
EI’s primary objective in implementing the Mindspark program was twinfold- to understand the roadblocks that have created such a scenario, and attain practical insights acquired from real-life implementations of EdTech-based solutions at a scale that imitates real situations.
What is Mindspark?
Mindspark is an AI-powered personalized adaptive learning tool that curates a tailored learning path for students, based on the information generated by an individual student’s responses to questions and activities. It then adjusts the type and difficulty of content delivered as per the child’s need, style and pace of learning. The dynamic platform culls relevant and customized intelligence from over a billion data points and delivers content in the form of questions, activities, games and videos to test students and provide explanations, feedback and learning inputs that are useful to students as well as teachers.
Benefits
The Mindspark software assists children in learning the fundamentals of the topics before calibrating to the next logical step that he/she should take in mastering the particular topic. The teacher can, therefore, focus on common misconceptions or learning gaps that the group as a whole faces, whereas Mindspark can address and fine-tune specific learning challenges of individual students.
How it happened?
In October 2017, EI set up Mindspark Labs in 40 government-run Adarsh schools across four districts of Rajasthan, namely Churu, Jhunjhunu, Dungarpur, and Udaipur. To ensure there is buy-in among all school stakeholders, EI organised several orientation and training workshops – both centrally and at school. Seven such district-level teacher workshops were organized in 2019 itself, with the aim to orient teachers about the program and provide pedagogical and logistical insights about EdTech. In addition, a mobile monitoring dashboard was created along with unique login IDs and passwords for principals of schools and government officials, enabling them to view Mindspark data on any mobile phone. The program included 6677 and 6276 students from October 2018 to April 2019 (part of Academic Year 2018-19) and July 2019 to September 2019 (part of Academic Year 2019-20)respectively.
Initiatives for teachers
The EI team conducted several focused group discussions and personal interviews with subject teachers to help them integrate Mindspark into the school eco-system. On soliciting teachers’ feedback regarding how Mindspark can help them complete their syllabus and achieve a better school result, EI introduced a “Textbook Module Feature” called “Worksheets” from February 2019. These worksheets were made available for Grades 1-8 for all topics recommended by the teachers, enabling them to practice the topics already covered in class and take unit tests to know students’ understanding. Moreover, EI’s team designed an academic calendar for all subject teachers (Hindi and Math) of Grades 1-8.
Student engagement initiatives
EI’s project management appointed two students each from Grades 4 – 8 as Mindspark Monitors and trained them on how to start the server, use a Chromebook, handle connectivity errors, sync the data, and navigate Mindspark. It was a moment of great excitement for the kids, and an instant of empowerment as well. Additionally, the team participated in the ‘Balsabha’ meeting held every month by the government to recognize students’ performance on Mindspark. The EI team presented the monthly school report at the meeting and, along with the respective school principal and teacher, felicitated the top student performers, called Sparkie students, from each school. Sparkie students are those who receive the highest number of Sparkie points on Mindspark. These Sparkie points are awarded when a student correctly answers three questions in a row.
Outcomes
The Abdul Latif Jameel Poverty Action Lab (J-PAL) was formed by Abhijit Banerjee and Esther Duflo. The Mindspark program improved learning levels across all groups of students and proved to be cost-effective compared to other instruction types according to the J-PAL RCT led by Prof. Karthik Muralidharan which did the third-party evaluation of the Mindspark program in the 40 schools across Rajasthan. In May 2019, J-PAL shared preliminary results from its Year 2 randomized control trial (RCT) of Mindspark at the RISE Conference, Washington DC. Prior to the program, students in the sample were many grade levels behind, and learning deficits increased with each grade.
Between start of Mindspark instruction in November 2017 and endline testing in February 2019, students’ performance in both math and Hindi were improved across multiple grade levels. Mindspark students scored 0.2 standard deviations more compared to a control group in both the subjects. In other words, learning gains for students that received Mindspark was twice as much as students in the comparison group (that didn’t receive Mindspark).
It’s estimated that Year 3 results would be more prominent, given the implementation is now stable. Based on these results, Dr. Shawn Cole, who is the John G McLean Professor of Business Administration at Harvard Business School (Executive Committee of J-PAL) also wrote a case study that is now taught to students of Harvard Business School and Harvard Kennedy School of Government.
Given a growing cognizance of the skills and competencies required, as well as the push on digital technologies, India has seen increased interest from all stakeholders ranging from the Prime Minister and MHRD to parents towards improving the learning outcomes of students, especially in public school setups. With this shift in mindset, along with the revised NEP and appropriate line items for operational expenses, it is safe to say that the technology behind personalized adaptive learning will be implementable, scalable and one that promises limitless improvement in the way India learns today. It only needs individuals and organizations with vision and good intentions to act on them, and prove the effectiveness and viability of such structures in our current educational framework.
Speaking on Mindspark success in Rajasthan, Srini Raghavan, Co-Founder and CEO, Educational Initiatives commented, “EI was conceived with the goal to help children everywhere learn with understanding. Our work in Rajasthan has shown that Mindspark, which integrates latest developments in pedagogy and technology through continuous research, has helped us make progress towards this goal. We are thrilled that the JPAL study has validated this.”
Furthermore, Karthik Muralidharan, Tata chancellor’s professor of economics University of California, Sandiego Education sector, Co Chair, JPAL, Honorary adviser(education and social), NITI Aayog said, “In 15 years of education research, I’ve never seen something that has had such a large effect in such a short amount of time. The reason why this is so effective is that you are getting complete customisation in a setting where the vast majority of children are so far behind the textbook and the syllabus that is taught in their class.”
