Authors: Krishnamurthy J and Nagaraja N S. Bangalore. 12th May 2018.
The latest advances in Neurosciences research positively affirms that cognitive abilities are influenced by training (Astle et al., 2015; Paul et al., 2017). With the rise in popularity of brain training, the experts recommend that these programs undergo systematic evaluation before they are made available in the market (Spaulding., 2010; Stanford report., 2014). Recognizing the situation, Brighter Minds incorporated a strong research framework, from the time of inception. The present article sheds light on results emanating from the research at different stages.
In the initial exploration, the research set out to find answers to the following questions:
Is Brighter Minds as a ‘package of interventions’ effective?
What additional changes are observed in children going through the program?
Is the program efficient?
Each component of Brighter Minds was picked up after a thorough review of scientific literature. But then, we wanted to understand if Brighter Minds as a whole (integrated package of tools) is effective in producing the change that was expected.
Since the primary focus of Brighter Minds was to improve observation ability in children, that became the key yardstick for measuring effectiveness. We reviewed standard definitions for ‘observation’ and finally defined it in our context as ‘the ability of the child to pause and feel intuitively, taking support of senses other than those involved in vision’. Blindfolded tests were done to assess changes in observation abilities. We also checked for additional changes i.e. ‘calmness’, and ‘joy’ and observed that a high proportion of children were showing positive results for all 3 outcomes. The initial exploration was based on the subjective feedback of the parents and trainers, and hence we followed up with a systematically designed qualitative research to understand the factors influencing the changes we witnessed. We were also curious to know what additional changes were noticed by parents and trainers. The study revealed the following changes i.e. focus, retention, comprehension, empathy, self-confidence and emotional stability. We also learnt that effective facilitator-child engagement and parental motivation were strong triggers.
The efficiency of the program was dependent on how simple and user-friendly were the tools used by facilitators. Critical reviews with experts, splitting the program into modules, and enabling technology helped greatly. A well-crafted trainer’s manual and a ‘train the trainer’ capsule improved efficiency greatly among the facilitators who were from diverse backgrounds, not necessarily from education.
The program engaged a child health expert during the pilot to rule out any medical or neurological disorders among children as the intention was to offer the program to healthy children. This helped not only to understand the circumstances under which the program is most effective and efficient but also allayed the safety concerns that naturally arise in interventions dealing with children.
In the next phase, we invited external researchers to take up a more objective evaluation. We asked the following questions:
Is the program effective on a large scale?
Are the changes widespread or generalizable?
Are changes reproducible in different contexts?
How much change is possible?
The effectiveness of any new innovation is judged by how well is it reproducible in other contexts and if the results are widespread and generalizable. While the results from the initial phase were positive and encouraging, they were primarily observed in children of parents that were already a motivated lot; they from tier 1 metro-cities (Bangalore, Chennai, etc.) and from upper socio-economic strata. In the next phase, the program was scaled up in children from more disadvantaged and rural backgrounds. During the year 2017, the Mahbubnagar district administration (Telangana state, India) trained about 2500 children across 12 residential schools in the district. This was a unique opportunity in terms of scale and reach which was unprecedented for Brighter Minds. External researchers were engaged to evaluate the effectiveness of the program in Mahbubnagar. A multidisciplinary team with backgrounds in public health and evaluation, psychiatry and psychology came together, developed standard and objective tools and metrics, administered cross-sectional surveys before and after the children underwent training.
The attention ability improved significantly by 48% (61% to 90%) in 8 weeks; the time taken to achieve the results improved by 21% in the end line (121 sec) when compared to baseline (157 sec). The observation ability improved by 43%. It not only improved quantitatively, but also qualitatively as the children reported deeper aspects of the test object such as emotions, feelings, and in relation to one’s own personal experiences. The memory tests revealed positive changes in terms of both immediate and delayed recall, by 13% and 27%, respectively.
In addition to the above changes, the teachers reported during focus group discussions that the children: were more calm, expressive and participatory in classes; showed improved attention span and concentration, and reduced distraction; showed less hesitancy in talking to teachers in class, reflecting improved self-confidence; showed improved memory and self-discipline. The teachers also expressed that the program should start at the beginning of the academic year so that the students benefit more in academics.
The independent evaluation not only showed a positive impact on the cognition and performance of children but also showed high levels of acceptability by the teachers. In a relatively short time of less than a year, close to 2500 children were reached out through institutionalized and sustainable ways of training.
We then elicited customer (parents) feedback through a survey. We specifically explored:
Do parents agree with the changes observed and if yes, how much?
Are these changes sustained for long periods?
Are parents as customers, satisfied at all?
In any continuous quality improvement initiative, knowing whether the customers are satisfied is critical. The customer’s experience and feedback is also vital for further program refinement. An online survey was undertaken in mid-2017 to understand these aspects. About 100 parents were interviewed to know how much they agreed on the changes, over a 5-point scale i.e. ‘totally agree’, ‘partially’, ‘neither agree nor disagree’, ‘partially disagree’ and ‘totally disagree’. In addition, we asked questions related to frequency and regularity of practice, challenges parents faced and if they were satisfied. 71 parents responded; 54% were from tier 1 and the remaining were from tier 2 cities; 39 were boys; 40 children were under 10 years; 46 were trained in 2016. More than 85% of parents agreed that they observed changes in all outcomes. While about 56 totally agreed on different changes, 31% admitted that they partially agreed. Thus a total of 87% were in agreement that they observed changes in their children.
In addition, the study shed light on interesting insights. The program was equally effective in both tier 1 and tier 2 cities. The changes were equally observed in all age groups and both genders with exception of two parameters. The elder children (>10 years) show greater changes in ‘emotional stability’ and ‘self-confidence’ than the younger ones; the girls showed higher changes in ‘preparation and planning’ as compared to boys. The ‘expression of feelings’ was reported more among those that were practising for a long time and regularly.
87% of parents reported high satisfaction levels. 62% reported that they are ‘extremely likely' to recommend to others and 31%, ‘very likely' to recommend. The parents that reported high satisfaction with the program also reported the highest improvements in their children. It is possible that the changes in the children had resulted in higher levels of satisfaction among the parents; it is also possible that these parents were already a motivated lot, who ensured adequate practice and interest among the children that resulted in the changes. Thus the parent’s motivation to ensure child’s interest and compliance seem to be a critical component. Few parents also indicated that they faced challenges in ensuring practice at homes which seemed like a deterrent against the result. This suggested program deliverers to support parents in engaging with children at home.
Summarizing our learnings so far: and next steps …...
The Brighter Minds program has successfully demonstrated results in different contexts. Sustained changes in cognitive and emotional traits are dependent on motivation among parents’/caregivers. High levels of acceptability and satisfaction, both by parents and trainers are noteworthy. The Mahbubnagar experience proves that these programs are scalable and replicable in large institutional settings in a fairly short span of time. Thus Brighter Minds thus presents itself as a unique and innovative opportunity for parents and educationists to make a difference in the cognitive abilities and lives of children.
As we take stock and consolidate our learnings thus far, new questions emerge for future research. An experimental study design is a gold standard to test the effectiveness of any new intervention. We are currently undertaking this venture through a new collaboration. There is also scope to incorporate radio-imaging and other neuro diagnostics to understand what changes are actually taking place in the brains during the interventions. In recent years, we see an explosion of knowledge related to brain changes in relation to cognitive abilities that we are studying (Astle., 2015). There is a dearth of knowledge related to the long term impact of these interventions, which can be another exciting study for the future.
Answering these research questions incurs resources and high-level technical expertise; we are exploring opportunities to collaborate with experts and institutions to answer these questions in future. If this area excites you and you want to work with us in any capacity, or if you happen to know of others who are interested in this area and may like to collaborate with us, please feel free to get in touch with us. Your support in any possible way will be highly appreciated.
Comments