We’ve been running the Gender Balance in Computing programme of research since 2019, as part of the National Centre for Computing Education (NCCE) and with various partners. It’s a £2.4 million research programme funded by the Department for Education in England that aims to identify ways to encourage more girls and young women to engage with Computing and choose to study it further. The programme is made up of four separate areas of research, in which we are running a number of interventions.
The first independent evaluation report from the Behavioural Insights Team (BIT) on our series of interventions has now been published. It relates to an intervention within the research area ‘Teaching Approach’, evaluating our pilot study of teaching computing to Key Stage 1 children using a storytelling approach. The evaluators from BIT found that this pilot study produced evidence of promise for the storytelling approach. They recommend conducting a full-size trial to test how effective this approach is for engaging female pupils with Computing.
Teaching computing through storytelling
Like many Computing curricula around the world, the English National Curriculum emphasises the importance of teaching Computing through a range of content so that pupils can express themselves and develop their ideas using digital tools. Our ‘Teaching Approach’ project builds on research grounded in sociocultural learning theories that suggest teaching approaches that encourage collaboration and use a variety of contexts can make Computing a more inclusive subject for all learners. Within this project, we are running three different interventions, each with learners of different ages.
Evidence indicates that gender stereotypes around Computing develop early (1). Therefore we designed a trial — the first of its kind in England — to explore a storytelling approach for teaching Computing with younger children (6- to 7-year-olds). A small body of research suggests that using storytelling as a learning context for Computing can be engaging for both boys and girls. Research results indicate that:
- Teaching computing through storytelling and story-writing is effective for motivating 11- to 14-year-old girls to learn programming (2)
- Children who write computer programs to tell stories see Computing as a subject that is equally as easy or difficult for both boys and girls (3)
- In a non-formal learning space, primary-aged girls are more likely to choose a storybook beginner electronics activity rather than open-ended beginner electronics free play (4)
The pilot study and the evaluation methods
As combining evidence from research with older students and in non-formal education is experimental, we designed this storytelling trial as a small pilot study. Our aim was to generate early evidence as to how feasible a teaching approach that uses storytelling might be in the primary Computing classroom.
We recruited 53 schools to take part in the pilot study, which ran from April to July 2021. Many schools were still facing challenges due to the ongoing coronavirus pandemic, and we are very grateful to the teachers and learners who have taken part for their contribution to this important research.
To conduct the study, we created a free online training course, and a scheme of work, for schools to teach Computing concepts to 6- and 7-year olds using a storytelling approach. Over a sequence of the 12 lessons in the scheme of work, pupils used the ScratchJr programming environment to animate their own digital stories and learn about Computing concepts, such as sequence and repetition, linked to elements of stories, such as structure, rhyme, and speech.
To enable the independent evaluation of the effectiveness of the storytelling approach by BIT, schools were allocated either to an intervention group, which used the training course and the storytelling scheme of work, or to a control group, which taught Computing in their usual way and was not made aware that the approach being trialled involved storytelling. For their evaluation, BIT gathered data from both groups to compare them:
- They conducted surveys measuring learners’ attitudes toward computing and their intentions to study it in the future
- They carried out observations of lessons, interviews with teachers, and discussions with learners
- They ran a survey to gather feedback about the trial from teachers
The gathered data was assessed against five categories: evidence of promise, fidelity, acceptability, feasibility, and readiness for trial.
Main findings of the evaluation team
After analysing the data collected from observations, interviews, learner discussions, pupil surveys, and teacher surveys, the key finding of the independent evaluators was that the storytelling teaching approach had evidence of promise, and that it is worthwhile scaling up our intervention for a larger trial with more schools.
The evaluators’ teacher interviews confirmed the early development of gender stereotypes in the classroom. This highlights the importance of introducing Computing to young learners in a way that engages both boys and girls.
“I’ve really noticed how there’s already differences in views of what’s a boy, what’s a girl, the boys are getting in front of me, like, ‘I want a boy car, I don’t want a girl car’. Then we’ve got the other side where we’ve got fairy tales and princesses and, ‘Oh, I’m a bunny. Do you want to play with me?’”
Teacher (evaluation report, p. 22)
Teachers told the evaluators that pupils enjoyed personalising their stories in ScratchJr, and that they themselves felt positive about the use of storytelling to teach computing.
“I think [the storytelling aspect] gives them something real to work through, so it’s not… abstract… I think through the storytelling, they’re able to make it as funny or whatever they want, and it’s also their own interest. [Female student], she dotes on animals, so she’s always having giraffes and all of that, so it’s something that they can make their own connections too… Yes, I did really like the storytelling.”
Teacher (evaluation report, p. 26)
Teacher feedback provided some evidence that the storytelling lessons had equally increased both male and female pupils’ interest, confidence, and skills.
The independent evaluation team advised caution when interpreting the quantitative data from the pupil surveys, due to the small sample size in this pilot study and the high attrition rates caused by coronavirus-related disruptions. We ourselves would like to add that the study raises questions about the reliability of quantitative survey data collected from very young children using Likert scales, BIT’s chosen survey format for this evaluation. Although the evaluators have made some positive steps in creating a new survey suitable for young children, this research instrument may need further testing; the survey results would need to be interpreted in this light, and more research in this area would be recommended.
You can read the full evaluation report on the NCCE website.
This intervention was based on one of the teaching approaches for which there was only early evidence of effectiveness, so it is a good outcome to have a larger trial recommended based on our pilot study. It’s often said that research ends up recommending more research, but in this case our small pilot project really does give robust evidence that we should trial the storytelling approach with more schools.
The independent evaluators collected feedback from both teachers and pupils that confirms the storytelling intervention we designed is feasible in the classroom. The feedback also indicates where we can make small adjustments that will refine and develop the training and scheme of work for a larger-scale study (evaluation report, p. 35), and we will consider this feedback carefully. While some teachers suggested that the training be shortened, less experienced teachers highlighted the need to ensure the training introduces teachers to all of the content covered in the lessons. This feedback helps us to better understand how Computing is taught in primary schools, and how this is influenced by the wide variety of experience and subject knowledge that teachers have. Interestingly, in the control group, some of the teachers reported that they also introduced coding to their learners by having them create stories. We would like to conduct further research into how schools introduce young learners to programming, and we’ll be continuing to reflect on how best to offer flexible content for teacher training related to our research studies.
We’re now looking at how to continue to investigate the effectiveness of the storytelling approach through a larger trial, alongside other projects in which we’re exploring female engagement in computing education through our recently established Raspberry Pi Computing Education Research Centre.
More evaluations are on the way for our other studies in the Gender Balance in Computing programme, including:
- Two other trials of teaching approaches
- Interventions in non-formal education contexts
- Trials of approaches to building a sense of belonging in Computing
- Research into the impact of timetabling and options evenings
If you would like to stay up-to-date with the research programme, you can sign up to the Gender Balance in Computing newsletter. We will also post our reflections on the projects on this blog when the evaluations are completed.
1 Mulvey, K. L. and Irvin, M. J. (2018). Judgments and reasoning about exclusion from counter-stereotypic STEM career choices in early childhood. Early Child. Res. Q. 44, 220–230. https://doi.org/10.1016/j.ecresq.2018.03.016
2 Kelleher, C., Pausch, R. and Kiesler, S. (2007). Storytelling alice motivates middle school girls to learn computer programming. In CHI ’07: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1455–1464. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1240624.1240844
3 Zaidi, R., Freihofer, I. and Childress Townsend, G. (2017). Using Scratch and Female Role Models while Storytelling Improves Fifth-Grade Students’ Attitudes toward Computing. In SIGCSE ’17: Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education, 791–792. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3017680.3022451
4 McLean, M., & Harlow, D. (2017). Designing inclusive STEM activities: A comparison of playful interactive experiences across gender. In IDC ’17: Proceedings of the 2017 Conference on Interaction Design and Children, 567–574. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3078072.3084326