Tag Archives: National Centre for Computing Education

Gender Balance in Computing — the big picture

Post Syndicated from Sue Sentance original https://www.raspberrypi.org/blog/gender-balance-in-computing-big-picture/

Improving gender balance in computing is part of our work to ensure equitable learning opportunities for all young people. Our Gender Balance in Computing (GBIC) research programme has been the largest effort to date to explore ways to encourage more girls and young women to engage with Computing.

A girl in a university computing classroom.

Commissioned by the Department for Education in England and led by the Raspberry Pi Foundation as part of our National Centre for Computing Education work, the GBIC programme was a collaborative effort involving the Behavioural Insights Team, Apps for Good, and the WISE Campaign.

Gender Balance in Computing ran from 2019 to 2022 and comprised seven studies relating to five different research areas:

  • Teaching Approach:
  • Belonging: Supporting learners to feel that they “belong” in computer science
  • Non-formal Learning: Establishing the connections between in-school and out-of-school computing
  • Relevance: Making computing relatable to everyday life
  • Subject Choice: How computer science is presented to young people as a subject choice 

In December we published the last of seven reports describing the results of the programme. In this blog post I summarise our overall findings and reflect on what we’ve learned through doing this research.

Gender balance in computing is not a new problem

I was fascinated to read a paper by Deborah Butler from 2000 which starts by summarising themes from research into gender balance in computing from the 1980s and 1990s, for example that boys may have access to more role models in computing and may receive more encouragement to pursue the subject, and that software may be developed with a bias towards interests traditionally considered to be male. Butler’s paper summarises research from at least two decades ago — have we really made progress?

A computing classroom filled with learners.

In England, it’s true that making Computing a mandatory subject from age 5 means we have taken great strides forward; the need for young people to make a choice about studying the subject only arises at age 14. However, statistics for England’s externally assessed high-stakes Computer Science courses taken at ages 14–16 (GCSE) and 16–18 (A level) clearly show that, although there is a small upwards trend in the proportion of female students, particularly for A level, gender balance among the students achieving GCSE/A level qualifications remains an issue:

Computer Science qualification (England): In 2018: In 2021: In 2022:
GCSE (age 16) 20.41% 20.77% 21.37%
A level (age 18) 11.74% 14.71% 15.17%
Percentage of girls among the students achieving Computer Science qualifications in England’s secondary schools

What did we do in the Gender Balance in Computing programme?

In GBIC, we carried out a range of research studies involving more than 14,500 pupils and 725 teachers in England. Implementation teams came from the Foundation, Apps For Good, the WISE Campaign, and the Behavioural Insights Team (BIT). A separate team at BIT acted as the independent evaluators of all the studies.

In total we conducted the following studies:

  • Two feasibility studies: Storytelling; Relevance, which led to a full randomised controlled trial (RCT)
  • Five RCTs: Belonging; Peer Instruction; Pair Programming; Relevance, which was preceded by a feasibility study; Non-formal Learning (primary)
  • One quasi-experimental study: Non-formal Learning (secondary)
  • One exploratory research study: Subject Choice (Subject choice evenings and option booklets)

Each study (apart from the exploratory research study) involved a 12-week intervention in schools. Bespoke materials were developed for all the studies, and teachers received training on how to deliver the intervention they were a part of. For the RCTs, randomisation was done at school level: schools were randomly divided into treatment and control groups. The independent evaluators collected both quantitative and qualitative data to ensure that we gained comprehensive insights from the schools’ experiences of the interventions. The evaluators’ reports and our associated blog posts give full details of each study.

The impact of the pandemic

The research programme ran from 2019 to 2022, and as it was based in schools, we faced a lot of challenges due to the coronavirus pandemic. Many research programmes meant to take place in school were cancelled as soon as schools shut during the pandemic.

A learner and a teacher in a computing classroom.

Although we were fortunate that GBIC was allowed to continue, we were not allowed to extend the end date of the programme. Thus our studies were compressed into the period after schools reopened and primarily delivered in the academic year 2021/2022. When schools were open again, the implementation of the studies was affected by teacher and pupil absences, and by schools necessarily focusing on making up some of the lost time for learning.

The overall results of Gender Balance in Computing

Quantitatively, none of the RCTs showed a statistically significant impact on the primary outcome measured, which was different in different trials but related to either learners’ attitudes to computer science or their intention to study computer science. Most of the RCTs showed a positive impact that fell just short of statistical significance. The evaluators went to great lengths to control for pandemic-related attrition, and the implementation teams worked hard to support teachers in still delivering the interventions as designed, but attrition and disruptions due to the pandemic may have played a part in the results.

Woman teacher and female students at a computer

The qualitative research results were more encouraging. Teachers were enthusiastic about the approaches we had chosen in order to address known barriers to gender balance, and the qualitative data indicated that pupils reacted positively to the interventions. One key theme across the Teaching Approach (and other) studies was that girls valued collaboration and teamwork. The data also offered insights that enable us to improve on the interventions.

We designed the studies so they could act as pilots that may be rolled out at a national scale. While we have gained sufficient understanding of what works to be able to run the interventions at a larger scale, two particular learnings shape our view of what a large-scale study should look like:

1. A single intervention may not be enough to have an impact

The GBIC results highlight that there is no quick fix and suggest that we should combine some of the approaches we’ve been trialling to provide a more holistic approach to teaching Computing in an equitable way. We would recommend that schools adopt several of the approaches we’ve tested; the materials associated with each intervention are freely available (see our blog posts for links).

2. Age matters

One of the very interesting overall findings from this research programme was the difference in intent to study Computing between primary school and secondary school learners; fewer secondary school learners reported intent to study the subject further. This difference was observed for both girls and boys, but was more marked for girls, as shown in the graph below. This suggests that we need to double down on supporting children, especially girls, to maintain their interest in Computing as they enter secondary school at age 11. It also points to a need for more longitudinal research to understand more about the transition period from primary to secondary school and how it impacts children’s engagement with computer science and technology in general.

Bar graph showing that in the Gender Balance in Computing research programme, learners intent to continue studying computing was lower in secondary school than primary school, and that this difference is more pronounced for girls.
Compared to primary school age girls, girls aged 12 to 13 show dramatically reduced intent to continue studying computing.

What’s next?

We think that more time (in excess of 12 weeks) is needed to both deliver the interventions and measure their outcome, as the change in learners’ attitudes may be slow to appear, and we’re hoping to engage in more longitudinal research moving forward.

In a computing classroom, a girl looks at a computer screen.

We know that an understanding of computer science can improve young people’s access to highly skilled jobs involving technology and their understanding of societal issues, and we need that to be available to all. However, gender balance relating to computing and technology is a deeply structural issue that has existed for decades throughout the computing education and workplace ecosystem. That’s why we intend to pursue more work around a holistic approach to improving gender balance, aligning with our ongoing research into making computing education culturally relevant.

Stay in touch

We are very keen to continue to build on our research on gender balance in computing. If you’d like to support us in any way, we’d love to hear from you. To explore the research projects we’re currently involved in, check out our research pages and visit the website of the Raspberry Pi Computing Education Research Centre at the University of Cambridge.

The post Gender Balance in Computing — the big picture appeared first on Raspberry Pi.

Using relevant contexts to engage girls in the Computing classroom: Study results

Post Syndicated from Katharine Childs original https://www.raspberrypi.org/blog/gender-balance-in-computing-relevance/

Today we are sharing an evaluation report on another study that’s part of our Gender Balance in Computing research programme. In this study, we investigated the impact of using relevant contexts in classroom programming activities for 12- to 13-year-olds on girls’ and boys’ attitudes towards Computing.

Two female learners code at a computer together.

We have been working on Gender Balance in Computing since 2018, together with partner organisations Behavioural Insights Team, Apps for Good, and WISE, to conduct research studies exploring ways to encourage more girls and young women to engage with Computing in school. The research programme has been funded by the Department for Education, and we deliver it as part of the National Centre for Computing Education. The report we share today is about the penultimate study in the programme.

Components of a Computing curriculum

A typical Computing curriculum is built around content: a list of concepts, knowledge, and skills that will be covered during the course. For some learners, that list will be enough to motivate and engage them in Computing. But other learners require more to engage with the subject, such as context about how they can use the computing skills they learn in the real world. Crucially, this difference between learners is often gendered. Research has shown that many boys become absorbed by the content in Computing courses, whereas for many girls the context for using computing skills is more important, and this context needs to relate to a variety of relevant scenarios where computing can solve problems.

In a computing classroom, a girl laughs at what she sees on the screen.

Developing teaching materials to highlight the relevance of Computing

In the Relevance study, we worked together with colleagues from Apps for Good to create teaching materials that present Computing in contexts that were relevant to pupils’ own interests. To do this, we drew on a research concept called identification. This states that when learners become interested in a topic because it relates to part of their own identity, that makes the subject more personally meaningful to them, which in turn means that they are more likely to continue studying it. In the materials we created, we drew on learners’ identities based on the communities that they belonged to (see image below). The materials asked them to identify the connections they had to their own communities, and to then use this as the context to design and create a mobile phone app.

A slide from a Computing lesson inviting learners to identify the communities they are part of based on their family, beliefs, school, interests, etc.
The intervention materials asked learners to think about the communities they belong to.

“I feel a sense of achievement in Computing when making your ideas a reality makes you proud of your creation, which is rewarding.” (Female learner, Relevance study evaluation report p. 57)

The Relevance research study

Between January 2022 and April 2022, more than 95 secondary schools were part of our study investigating the effect that learning with these resources might have on the attitudes of Year 8 pupils (aged 12–13) towards Computing. We are very grateful to all the schools, pupils, and teachers who took part in this study.

To enable evaluation of the study as a randomised controlled trial, the schools were randomly divided into two groups: a ‘control’ group that taught standard Computing lessons, and a ‘treatment’ group that delivered the intervention materials we had developed. The impact of the intervention was independently evaluated by the Behavioural Insights Team using data collected from pupils via surveys at the start and end of the intervention. The evaluators also collected data while conducting lesson observations, pupil group discussions, teacher interviews, and teacher surveys to understand how the intervention was delivered.

The girls who took part in the intervention chose an interesting range of contexts for their apps, including: 

  • Solving problems in the school community, such as homework timetabling and public transport
  • Interest-based communities, such as melody-making and interior design 
  • Issues in wider communities, such as sea life population and mental health

“I feel like it’s an important subject, and I feel like sea life is at risk right now, and I want to help people realise that.” (Female learner, Relevance study evaluation report p. 60)

“I feel like computing can create apps to do with solving mental health problems, which I think are very important and personally need a lot of improvement on the way we can cope with mental health.” (Female learner, Relevance study evaluation report p. 60)

What we learned from the Relevance study

The start of this blog refers to the core components of a Computing curriculum: concepts, knowledge, and skills. One way of building a curriculum is to list these components and develop a scheme of work which covers them all. However, in a recent computing education paper, researchers present an alternative way: developing curricula around the possible endpoints of learners. For computing, one endpoint could be the economic opportunities of a programming career, but equally, another could be using digital technologies for creative expression. The researchers argue that when learners have the opportunity to use computing as a tool related to personally meaningful contexts, a more diverse group of learners can become engaged in the subject.

A group of young people in a computer science classroom pose for a group photo.

Girls who took part in our Relevance study expressed the importance of creativity. “I think last term we had instructions and you follow them, whereas now it’s like your own ideas and your own creativity and whatever you make,” said one female learner (report, p. 56). The series of lessons where learners designed a prototype of their app was particularly popular among girls because this activity included creative expression. Girls who see themselves as creative align their interests with subjects that allow them to express this part of their identity.

A slide from a Computing lesson inviting learners to design a mobile phone app on paper.
With the intervention materials, learners developed a paper prototype of their app before going on to create code for it.

Based on learner responses to a ‘yes/no’ question about whether they were likely to choose GCSE Computer Science, the evaluators of the study found no statistically significant differences between the students who were part of the treatment and control groups. However, when learners were asked instead to select from a list which subjects they were likely to choose at GCSE, there was a statistically significant difference in the results: girls from schools in the treatment group were more likely to choose GCSE Computer Science as one of their options than girls in the control group. This finding suggests that it would be beneficial to gender balance in Computing if educators who design Computing curricula consider multiple endpoints for learners and include personally meaningful contexts to create learning experiences that are relevant to diverse groups of learners.

Find out more about making computing relevant for your learners

This is the penultimate report to be published about the studies that are part of the Gender Balance in Computing programme. If you would like to stay up-to-date with the programme, you can sign up to our newsletter. Our final report is about a study that explored the role that options booklets and evenings play in students’ subject choice.

The post Using relevant contexts to engage girls in the Computing classroom: Study results appeared first on Raspberry Pi.

Non-formal learning activities: What do we know and how do we apply it to computing?

Post Syndicated from Katharine Childs original https://www.raspberrypi.org/blog/gender-balance-in-computing-non-formal-learning/

At the Raspberry Pi Foundation, we engage young people in learning about computing and creating with digital technologies. We do this not only by developing curricula for formal education and introducing tens of thousands of children around the world to coding at home, but also through supporting non-formal learning activities such as Code Club and CoderDojo.

A teacher watches two female learners code in Code Club session in the classroom.
Code Clubs are after-school coding clubs.

To find out what works in non-formal computing learning, we’ve conducted two research projects recently: a systematic literature review, and a set of two interventions that were applied and evaluated as part of our Gender Balance in Computing programme. In this blog, we outline these two research projects.

What is non-formal learning?

When you think of young people learning computing, do you think of schools, classrooms, and curricula? You’d be right that lots of computing education for young people takes place in classrooms as part of national curricula. However, a lot of learning can take place outside of formal schooling. When we talk about non-formal computing education, we mean structured or semi-structured learning environments such as clubs or community groups, often set up by volunteers. These may take place in a school, library, or community venue; but we’ve also heard of some of our communities running non-formal learning activities on buses, in fire stations, or at football grounds  — there really is no limit to where learning can happen.

A CoderDojo coding session for young people.
CoderDojos are community-based coding clubs and some take place in offices.

It’s harder to assess the impact and effectiveness of non-formal computing activities than formal computing education: we have to think outside of the traditional measures such as grades and formal exams or assessments. Instead, we estimate outcomes according to measures such as level of participant engagement, attendance, attrition rates, and changes in participants’ attitudes towards computing. We have previously also piloted non-formal assessments such as quizzes and found that these were well-received by adult facilitators and children alike. 

Project 1: Researching the impact of non-formal computing education

Earlier this year, we conducted a systematic literature review into computing education for K–12 learners in non-formal settings. We identified 88 relevant research studies, which we read, compared, and synthesised to provide an overview of what is already known about the effectiveness of non-formal computing activities and to identify opportunities for further research. 

Our analysis looked for common themes within existing studies and suggested some benefits that non-formal learning offers, including: 

  • Access to advanced and innovative topics
  • Awareness about computing careers 
  • The chance to personalise projects according to learner interests
  • The opportunity for learners to progress at their own pace
  • The chance for learners to develop a sense of community through peers and role models

We presented this research at an international computing education conference called ICER 2022, and you can read about it in our open-access paper in the ICER conference proceedings.

A tweet about a presentation about non-formal learning at the ICER 2022 conference.

Project 2: Making links between non-formal learning and formal computing study skills 

One particularly interesting characteristic of non-formal learning is that it tends to attract a broader range of learners than formal computing lessons. For example, a 2019 survey found that about 40% of the young people who attend Code Clubs were female. This is a high percentage compared with the proportion of girls among the learners choosing Computer Science GCSE in England, which is currently around 20%. We believe this points to an opportunity to capitalise on girls’ interest in learning activities outside of the classroom, and we hope to use non-formal activities to encourage more girls to take an interest in formal computer science education.

Two learners from Code Club at Hillside School.
Code Clubs are well-attended by girls.

As part of our Gender Balance in Computing research programme in England, we worked with Apps for Good and the Behavioual Insights Team (BIT) to run two interventions in school-based non-formal settings, for which we adapted non-formal resources and used behavioural science concepts to strengthen the links the resources make between non-formal learning and studying computing more formally. One intervention ran in secondary schools for learners aged 13–14 years old, who used an adapted Apps for Good course, and the other ran in primary school for learners aged 8–11 year olds, who took part in Code Clubs using adapted versions of our projects.

A tweet from a school participating in a research project related to non-formal learning.

The interventions were evaluated independently by a separate team from BIT, based on data from surveys completed by learners before and after the interventions, and interviews with teachers and learners. This data was analysed by the independent team to explore the impact the interventions had on learners’ attitudes towards computing and intention to study the subject in the future. 

What did we learn from these research projects? 

Our literature review concluded that future research in this area would benefit from experimenting with a variety of approaches to designing, and measuring the impact of, computing activities in a non-formal setting. For example, this could include comparing the short-term and long-term impact of specific interventions, aiming to cater for different types of participants, and offering different types of learning experiences.

A girl codes at a laptop while a woman looks on during a Code Club session.

In these two Gender Balance in Computing interventions, there was limited statistical evidence of an improvement in participants’ attitude towards computing or in their stated intention to study computer programming in the future. The independent evaluators recommended that the learning content that was created for the interventions could be adapted further to make the link between non-formal and formal learning even more salient. On the other hand, as is often the case with research, some interesting themes — ones that we weren’t looking for — emerged from the data, including: 

  • In the secondary school intervention, there was a small, positive change in girls’ attitudes toward computing when they saw that it was relevant to real-world problems
  • In the primary school intervention, some teachers also reported an increased confidence to pursue computing among girls who had used the adapted Code Club resources, and they highlighted the importance of positive female role models in computing

In both projects, the findings suggest that it is beneficial for learners to participate in non-formal learning activities that link to real-world situations, and that this could be particularly beneficial for girls to help them see computing as a subject that is relevant to their own interests and goals. Another common theme in both projects is that non-formal learning activities play an important role in showing what a “computer person” looks like and who belongs in computing. This suggests there’s a need for a diverse range of volunteers to run non-formal computing activities, and that we should make sure that non-formal learning resources include representations of a diverse range of learners.

Computing classroom with woman teacher and young students at laptops doing Scratch coding.

Undertaking these research projects has provided evidence that the work the Foundation does is on the right track and suggested opportunities to use these themes in our future non-formal work and resources. 

Find out more about our work on non-formal computing education

More information about research projects at the Raspberry Pi Foundation and our newly launched Raspberry Pi Computing Education Research Centre can be found on our research pages and on the Research Centre’s website.

The post Non-formal learning activities: What do we know and how do we apply it to computing? appeared first on Raspberry Pi.

A peer instruction approach for engaging girls in the Computing classroom: Study results

Post Syndicated from Katharine Childs original https://www.raspberrypi.org/blog/gender-balance-in-computing-peer-instruction-approach-engaging-girls/

Today, we are publishing the third report of our findings from our Gender Balance in Computing research programme. This report shares the outcomes from the Peer Instruction project, which is the last in our set of three interventions that has explored teaching approaches to engage more girls in computing.

In a computing classroom, a smiling girl raises her hand.

The premise of the teaching approach research is that the way Computing is taught may not always match the teaching approaches to which girls are most likely to respond positively [1]. As with the Storytelling project and the Pair Programming project, this project aimed to find new contexts and approaches to help increase the number of girls choosing to study and work in computing. 

What is peer instruction? 

Peer instruction is a structured, collaborative teaching approach. It has been shown to be an effective pedagogy for novice programmers and those studying computer science at a university level because the interactive, cooperative activities help learners to perceive the topics as less stressful and less difficult [2]. 

Multiple-choice questions (MCQs) and peer conversations about the question answers are at the core of the peer instruction approach. Through talking to each other about MCQs, pupils can deepen their understanding about why a particular concept or fact is correct, and correct any misconceptions.

A diagram showing The five stages of the peer instruction teaching approach covered in a computing lesson: based on a misconception focused multiple-choice question, stage 1 is solo response, stage 2 is peer discussion, stage 3 is peer response, stage 4 is sharing results, stage 5 is class discussion. Optional steps are pre-instruction and follow-up multiple-choice question.
The five stages of the peer instruction teaching approach covered in a Computing lesson.

In England, the Computing curriculum at Key Stage 3 (ages 11–14) introduces learners to some new concepts, such as data representation, and moves learners to text-based programming languages. Towards the end of this Key Stage, learners will make choices about the subjects that they go onto study for GCSEs. These choices are influenced by learners’ attitudes towards the subject, and so we decided to trial whether the peer instruction teaching approach might lead to more positive attitudes towards Computing among girls.

The Peer Instruction intervention

The initial pilot of this trial ran from January to March 2020 with 15 secondary schools. We then used teacher feedback to develop resources to use in a full randomised controlled trial which ran from October 2021 to February 2022 in more than 60 secondary schools in England. Due to the COVID-19 pandemic, we changed our original plan to run face-to-face training and instead developed an online course to train teachers in the peer instruction approach. After taking part in the training, the teachers delivered 12 weeks of Computing lessons in data representation and Python programming. The two six-week units of work covered computing concepts for Key Stage 3 learners such as: 

  • Understanding how numbers can be represented in binary format
  • Understanding how data of various types can be represented and manipulated digitally in the form of binary digits
  • Using a text-based programming language to solve a variety of computational problems 

The study was run as a randomised controlled trial where participating schools were randomly divided into two groups. Schools in the treatment group used the peer instruction resources, and schools in the control group taught their normal Computing lessons. The independent evaluators from the Behavioural Insights Team used pupil surveys to measure the impact of the resources and supported this with lesson observations and teacher interviews to better understand the  themes emerging from the data. 

“I think peer instruction lessons are actually better than the normal lessons because you can ask other people around you to help more.”

– Female pupil who took part in the peer instruction lessons (report, p. 45)

Findings from the evaluation

The outcome measures of the peer instruction approach evaluation were quantitative data obtained from Year 8 pupils (aged 12 to 13) via pre- and post-surveys about the pupils’ stated intent to select Computer Science as a GCSE subject, and attitudes towards Computing as captured by the Student Computer Science Attitude Survey (SCSAS). When compared with the control group, the treatment group did not show a statistically significant increase in stated intent or positive attitudes towards Computing. This is a really valuable finding to help us build our understanding of what works in computing education. 

The evaluation report contains some useful suggestions on how peer instruction methods could be improved in the secondary classroom: 

  • Emphasise the importance of the stages of the peer instruction approach throughout the supporting materials. Our support for teachers changed from an in-person training day in stage one to an online course in stage two, and this impacted how much we could model the peer instruction steps that involve pupil discussion. This teaching approach differs from the traditional approach of asking learners to put their hands up to answer questions, and we believe that face-to-face training for teachers would be the best way to explore stage two of peer instruction. The importance of the discussion steps in peer instruction were further emphasised in the report: “The interviewed girls similarly reported that they preferred working in a group (as opposed to answering questions individually) as they were able to hear from people who had different ideas to them and check their answers.” So the discussion steps in peer instruction need careful thought when being delivered.
  • It may be useful to combine the peer instruction approach with other strategies. Although only a small number of girls taking part were interviewed, their feedback about the peer instruction lessons was very positive. The evaluation suggests that a multi-faceted approach to addressing gender balance is needed, given that the lack of girls in computing is indicative of a substantive societal issue, which decades of initiatives and research have attempted to address. The evaluators suggested that combining this pedagogy with other strategies, such as linking Computing to real-world problem-solving (another topic we explored in the Gender Balance in Computing programme), may have a cumulatively positive effect. 

“Year 8 is too late” 

At the start of both the Pair Programming and Peer Instruction projects, pupils were asked the same set of questions about their attitudes towards Computing via the Student Computer Science Attitude Survey (SCSAS). The mean scores from the survey results suggest that there is a small gender gap in attitudes at primary school. Boys feel slightly more confident and interested in Computing than girls. By secondary school, this gap has widened, as shown in the graph below:

Graph of the SCSAS scores to show the differences between boys’ and girls’ mean scores (out of 4) when asked about their attitudes towards computing at Year 4/6 and at year 8. Boys state a more positive attitude on average, and the difference between girls' and boys' attitudes in larger in Year 8.
Graph of the SCSAS scores to show the differences between boys’ and girls’ mean scores (out of 4) when asked about their attitudes towards Computing at Year 4/6 and at year 8.

In both projects, pupils were also asked about their intentions to continue studying Computing. In the Pair Programming project, the participating pupils (in Year 4/6) were asked whether they wanted to study Computing in the future, whereas the Year 8 pupils taking part in the Peer Instruction project were asked whether they intended to choose Computer Science as a GCSE subject. We cannot compare these two sets of answers directly, but there is general indication that as girls progress through stages of education, they begin to decide that Computing is not a subject for them. The independent evaluators commented that “it is striking that the gap between genders widens to such an extent over this 2- to 4-year time period, and that the overall proportions of pupils intending to continue to study Computing decreases to such an extent” (p. 15 of the report).  

“These findings show a clear difference in attitudes towards learning Computing between primary and secondary learners. It really makes the adage ‘Year 8 is too late’ very true, and it is important to think carefully about what happens between Year 6 and Year 8 to make sure that Computing is a subject which engages all learners.”

– Sue Sentance, Chief Learning Officer, Raspberry Pi Foundation

Want to find out more about peer instruction?  

  • Download our Big Book of Computing Pedagogy (available as a free online download) and find out more about peer instruction on pages 56 and 57.
  • Read the evaluation report of the peer instruction intervention.
  • Try the free training course on peer instruction used in this project. This course links to our research materials used by teachers as part of the intervention. 

We are very grateful to all the schools, pupils, and teachers who took part in this project. If you would like to stay up-to-date with the Gender Balance in Computing programme, you can sign up to our newsletter. We will also share reports on the other projects within the programme that have explored: 

  • Pupils’ sense of belonging in Computing 
  • The links between non-formal and formal Computing 
  • The impact of using Computing to solve real-world problems

[1] Goode, J., Estrella, R., & Margolis, J. (2008). Lost in Translation: Gender and High School Computer Science. In Cohoon, J, & Aspray, W. (Eds.) Women and Information Technology. Cambridge, MA: The MIT Press. https://doi.org/https://doi.org/10.7551/mitpress/7272.003.0005

[2] Herman, G. L., & Azad, S. (2020, February). A comparison of peer instruction and collaborative problem solving in a computer architecture course. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. Association for Computing Machinery, New York, NY, USA. pp. 461–467. https://dl.acm.org/doi/10.1145/3328778.3366819

The post A peer instruction approach for engaging girls in the Computing classroom: Study results appeared first on Raspberry Pi.

A pair programming approach for engaging girls in the Computing classroom: Study results

Post Syndicated from Katharine Childs original https://www.raspberrypi.org/blog/gender-balance-in-computing-pair-programming-approach-engaging-girls/

Today we share the second report in our series of findings from the Gender Balance in Computing research programme, which we’ve been running as part of the National Centre for Computing Education and with various partners. In this £2.4 million research programme, funded by the Department for Education in England, we aim to identify ways to encourage more female learners to engage with Computing and choose to study it further.

A teacher encourages a learner in the computing classroom.

Previously, we shared the evaluation report about our pilot study of using a storytelling approach with very young computing learners. This new report, again coming from the Behavioural Insights Team (BIT) which acts as the programme’s independent evaluator, describes our study of another teaching approach.

Existing research suggests that computing is not always taught in a way that is engaging for girls in particular [1], and that we can improve this. With the intervention at hand, we wanted to explore the effects of using a pair programming teaching approach with primary school learners aged 8 to 11. We have critically and carefully examined the findings, which show mixed outcomes regarding the effectiveness of the approach, and we believe that the research provides insights that increase our shared understanding of how to teach computing effectively to young learners. 

Computing education through a collaborative lens

Many people think that writing computer programs is a task carried out by people working individually. A 2017 study of 8- and 9-year-olds [2] confirms this: when asked to draw a picture of a computer scientist doing work, 90% of the children drew a picture of one person working alone. This stereotype is present in teaching and learning about computing and computer science; many computer programming lessons take place in a way that promotes solitary working, with individual students sitting in front of separate computers, working on their own code and debugging their own errors.

A girl codes at a laptop while a woman looks on during a Code Club session.

Professional software development rarely happens like this. For example, at the Raspberry Pi Foundation, our software engineers work collaboratively on design and often pair up to solve problems. Computing education research also has identified the importance of looking at computer programming through a collaborative lens. This viewpoint allows us to see computing as a subject with scope for collaborative group work in which students create useful applications together and are part of a community where programming has a shared social context [3]. 

Researching collaborative learning in the primary computing classroom 

One teaching approach in computing that promotes collaborative learning is pair programming (a practice also used in industry). This is a structured way of working on programming tasks  where learners are paired up and take turns acting as the driver or the navigator. The driver controls the keyboard and mouse and types the code. The navigator reads the instructions, supports the driver by watching out for errors in the code, and thinks strategically about next steps and solutions to problems. Learners swap roles every 5 to 10 minutes, to ensure that both partners can contribute equally and actively to the collaborative learning.

Two female learners code at a computer together.

As one part of the Gender Balance in Computing programme, we designed a project to explore the effect of pair programming on girls’ attitudes towards computing. This project builds on research from the USA which suggests that solving problems collaboratively increases girls’ persistence when they encounter difficulties in programming tasks [4].

In the Pair Programming project, we worked with teachers of Year 4 (ages 8–9) and Year 6 (ages 10–11) in schools in England. From January to March 2020, we ran a pilot study with 10 schools and used the resulting teacher feedback to finalise the training and teaching materials for a full randomised controlled trial. Due to the coronavirus pandemic, we trained teachers in the pair programming approach using an online course instead of face-to-face training.

A tweet from a school about taking part in the pair programming intervention of the Gender Balance in Computing research programme.
A tweet from a school about taking part in the pair programming study.

The randomised controlled trial ran from September to December 2021 with 97 schools. Schools were randomly allocated to either the intervention group and used the pair programming training and the scheme of work we designed, or to the control group and taught Computing in their usual way, not aware that we were investigating the effects of pair programming. Due to the coronavirus pandemic, our training of teachers in the pair programming approach had to take place via an online course instead of face to face.

Teachers in both groups delivered 12 weeks of Computing lessons, in which learners used Scratch programming to draw shapes and create animations. The lessons covered computing concepts from Key Stage 2 (ages 7–11), such as using sequences, selection, and repetition in programs, as well as digital literacy skills such as using technology respectfully.

What can we learn about pair programming from the study? 

The findings about this particular intervention were limited by the amount of data the independent evaluators at BIT were able to collect amongst learners and teachers given the ongoing pandemic. BIT’s evaluation was primarily based on quantitative data collected from learners at the start and the end of the intervention. To collect the data, they used a validated instrument called the Student Computer Science Attitude Survey (SCSAS), which asks learners about their attitudes towards Computing. The evaluators compared the datasets gathered from the intervention group (who took part in pair programming lessons) and the control group (who took part in Computing lessons taught with a ‘business as usual’ model).

A teacher watches two female learners code in Code Club session in the classroom.

The evaluators’ data analysis found no statistically significant evidence that the pair programming approach positively affected girls’ attitudes towards computing or their intention to study computing in the future. The lack of statistically significant results, called a null result in research projects, can appear disappointing at first. But our work involves careful reflection and critical thinking about all outcomes of our research, and the result of this project is no exception. These are factors that may have contributed towards the result: 

  • The independent evaluators suggested that the intervention may lead to different findings if it were implemented again without the disruptions caused by the pandemic. One of their recommendations was to revert to our original planned model of providing face-to-face training to teachers delivering the pair programming approach, and we believe this would embed a deeper understanding of the approach. 
  • Our research built upon a prior study [4] that suggested a connection between pair programming and increased confidence about problem-solving in girls of a similar age. That study took place in a non-formal setting in an all-girls group, whereas our research was situated in formal education in mixed gender groups. It may be that these differences are significant. 
  • It may be that there is no causal link between using the pair programming approach and an increase in girls’ attitudes towards computing, or that the link may only become apparent over a longer time-scale, or that the pair programming approach needs to be combined with other strategies to achieve a positive effect. 

The evaluators also gathered qualitative data by running teacher and learner interviews, and we were pleased that this data provided some rich insights into the benefits of using a pair programming approach in the primary classroom, and gave some promising indications of possible benefits for female learners in particular. 

  1. Teachers spoke positively about the use of paired activities, and felt that having the defined roles of driver and navigator helped both partners to contribute equally to the programming tasks. Learners said that they enjoyed working in pairs, even though there could be some moments of frustration. Some of the teachers were even planning to integrate pair programming into future lessons. This suggests that the approach was effective both in engaging and motivating learners, as well as in facilitating the planned learning outcomes of the lessons,  and that it can be used more widely in primary computing teaching.

“I don’t know why I’ve never thought to do computing like that, actually, because it’s a really good vehicle for the fact that there are two roles, clearly defined. There’s all your conversation, and knowledge comes through that, and then they’re both equally having a turn.” — Primary school teacher (report, p. 38)

“I like working with both [both as a partner and by yourself] because when you do pair programming, you’re collaborating with your partner, making links, and you have to tell them what to do. But if you have a really good idea and then they put the wrong thing in the wrong place, it’s quite annoying.” — Female learner (report, p. 40)

  1. Both teachers and learners felt that having the support of a partner boosted learners’ confidence, which echoes previous research in the field [5, 6]. In computing, boys more accurately assess their capabilities, whereas girls tend to underestimate their performance [7]. When learners feel a positive emotion such as confidence towards a subject, combined with a belief that they can succeed in tasks related to that subject, this shows self-efficacy [8]. Our findings suggest that, through the use of the pair programming approach, both boys and girls improved their sense of self-efficacy towards Computing, which is corroborated by quotes from learners themselves. This is interesting because a sense of self-efficacy in Computing is linked to the decisions to pursue further study in the subject [9]. More research could build on this observation. 

“I do think that having that equal time to have a go at both, thinking of the girls I’ve got, will have helped my girls, because they lack a bit of confidence. They were learning very quickly that, ‘Actually, yes, we are sure. We can do this.’” — Primary teacher (report, p. 44)

“It might be easier to do pair programming [compared to ‘normal’ lessons] because if you’re stuck, your partner can be helpful.” — Female learner (report, p. 43)

Find out more about pair programming 

  • Download our Big Book of Computing Pedagogy a free PDF and read about pair programming on pages 58 and 59.
  • Watch this short video that shows pair programming being used in a primary classroom. 
  • Read the evaluation report of the pair programming intervention, where you’ll also find more quotes from teachers and learners.
  • Try the free training course on pair programming we designed and used for this project. It also includes links to the lesson plans that teachers worked with. 

Collaboration in our research

We will continue to publish evaluation reports and our reflections on the other projects in the Gender Balance in Computing programme. If you would like to stay up-to-date with the programme, you can sign up to the newsletter.

Two learners at a desktop computer doing coding.

The insights gained from this trial will feed forwards into our future work. Through the process of working with schools on this project, we have increased our understanding of the process of research in educational settings in many ways. We are very grateful for the input from teachers who took part in the first stage of the trial, with whom we developed an effective co-production model for developing resources, a model we will use in future research projects. Teachers who took part in the second stage of the project told us that the resources we provided were of good quality, which demonstrates the success of this co-production approach to developing resources. 

In our new Raspberry Pi Computing Education Research Centre, created with the University of Cambridge Department of Computer Science and Technology, we will collaborate closely with teachers and schools when implementing and evaluating research projects. You are invited to the free in-person launch event of the Centre on 20 July in Cambridge, UK, where we hope to meet many teachers, researchers, and other education practitioners to strengthen a collaborative community around computing education research.

References

[1] Goode, J., Estrella, R., & Margolis, J. (2018). Lost in Translation: Gender and High School Computer Science. In Women and Information Technology. https://doi.org/10.7551/mitpress/7272.003.0005

[2] Alexandria K. Hansen, Hilary A. Dwyer, Ashley Iveland, Mia Talesfore, Lacy Wright, Danielle B. Harlow, and Diana Franklin. 2017. Assessing Children’s Understanding of the Work of Computer Scientists: The Draw-a-Computer-Scientist Test. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE ’17). Association for Computing Machinery, New York, NY, USA, 279–284. https://doi.org/10.1145/3017680.3017769

[3] Yasmin B. Kafai and Quinn Burke. 2013. The social turn in K-12 programming: moving from computational thinking to computational participation. In Proceeding of the 44th ACM technical symposium on Computer science education (SIGCSE ’13). Association for Computing Machinery, New York, NY, USA, 603–608. https://doi.org/10.1145/2445196.2445373

[4] Linda Werner & Jill Denning (2009) Pair Programming in Middle School, Journal of Research on Technology in Education, 42:1, 29-49. https://doi.org/10.1080/15391523.2009.10782540

[5] Charlie McDowell, Linda Werner, Heather E. Bullock, and Julian Fernald. 2006. Pair programming improves student retention, confidence, and program quality. Commun. ACM 49, 8 (August 2006), 90–95. https://doi.org/10.1145/1145287.1145293

[6] Denner, J., Werner, L., Campe, S., & Ortiz, E. (2014). Pair programming: Under what conditions is it advantageous for middle school students? Journal of Research on Technology in Education, 46(3), 277–296. https://doi.org/10.1080/15391523.2014.888272

[7] Maria Kallia and Sue Sentance. 2018. Are boys more confident than girls? the role of calibration and students’ self-efficacy in programming tasks and computer science. In Proceedings of the 13th Workshop in Primary and Secondary Computing Education (WiPSCE ’18). Association for Computing Machinery, New York, NY, USA, Article 16, 1–4. https://doi.org/10.1145/3265757.3265773

[8] Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. https://doi.org/10.1037/0033-295X.84.2.191

[9] Allison Mishkin. 2019. Applying Self-Determination Theory towards Motivating Young Women in Computer Science. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE ’19). Association for Computing Machinery, New York, NY, USA, 1025–1031. https://doi.org/10.1145/3287324.3287389

The post A pair programming approach for engaging girls in the Computing classroom: Study results appeared first on Raspberry Pi.

I belong in computer science

Post Syndicated from Janina Ander original https://www.raspberrypi.org/blog/i-belong-in-computer-science-isaac-computer-science/

At the Raspberry Pi Foundation, we believe everyone belongs in computer science, and that it is a much more varied field than is commonly assumed. One of the ways we want to promote inclusivity and highlight the variety of skills and interests needed in computer science is through our ‘I belong’ campaign. We do this because the tech sector lacks diversity. Similarly, in schools, there is underrepresentation of students in computing along the axes of gender, ethnicity, and economic situation. (See how researchers describe data from England, and data from the USA.)

Woman teacher and female students at a computer

The ‘I belong’ campaign is part of our work on Isaac Computer Science, our free online learning platform for GCSE and A level students (ages 14 to 18) and their teachers, funded by the Department for Education. The campaign celebrates young computer scientists and how they came to love the subject, what their career journey has been so far, and what their thoughts are about inclusivity and belonging in their chosen field.

These people are role models who demonstrate that everyone belongs in computer science, and that everyone can bring their interests and skills to bear in the field. In this way, we want to show young people that they can do much more with computing than they might think, and to inspire them to consider how computing could be part of their own life and career path.

Meet Salome

Salome is studying Computer Science with Digital Technology Solutions at the University of Leeds and doing a degree apprenticeship with PricewaterhouseCoopers (PwC).

Salome smiling. The text says I belong in computer science.

“I was quite lucky, as growing up I saw a lot about women in STEM which inspired me to take this path. I think to improve the online community, we need to keep challenging stereotypes and getting more and more people to join, thereby improving the diversity. This way, a larger number of people can have role models and identify themselves with someone currently there.”

“Another thing is the assumption that computer science is just coding and not a wide and diverse field. I still have to explain to my friends what computer science involves and can become, and then they will say, ‘Wow, that’s really interesting, I didn’t know that.’”

Read Salome’s full interview

Meet Devyani

Devyani is a third-year degree apprentice at Cisco. 

Devyani smiling. The text says I belong in computer science.

“It was at A level where I developed my programming skills, and it was more practical rather than theoretical. I managed to complete a programming project where I utilised PHP, JavaScript, and phpMyAdmin (which is a database). It was after this that I started looking around and applying for degree apprenticeships. I thought that university wasn’t for me, because I wanted a more practical and hands-on approach, as I learn better that way.”

“At the moment, I’m currently doing a product owner role, which is where I hope to graduate into. It’s a mix between both a business role and a technical role. I have to stay up to speed with the current technologies we are using and developing for our clients and customers, but also I have to understand business needs and ensure that the team is able to develop and deliver on time to meet those needs.”

Read Devyani’s full interview

Meet Omar

Omar is a Mexican palaeontologist who uses computer science to study dinosaur bones.

Omar. The text says I belong in computer science.

“I try to bring aspects that are very well developed in computer science and apply them in palaeontology. For instance, when digitising the vertebrae, I use a lot of information theory. I also use a lot of data science and integrity to make sure that what we have captured is comparable with what other people have found.”

“What drove me to computers was the fact you are always learning. That’s what keeps me interested in science: that I can keep growing, learn from others, and I can teach people. That’s the other thing that makes me feel like I belong, which is when I am able to communicate the things I know to someone else and I can see the face of the other person when they start to grasp a theory.”

Read Omar’s full interview

Meet Tasnima

Tasnima is a computer science graduate from Queen Mary University of London, and is currently working as a software engineer at Credit Suisse.

Tasnima smiling. The text says I belong in computer science.

“During the pandemic, one of the good things to come out of it is that I could work from home, and that means working with people all over the world, bringing together every race, religion, gender, etc. Even though we are all very different, the one thing we all have in common is that we’re passionate about technology and computer science. Another thing is being able to work in technology in the real world. It has allowed me to work in an environment that is highly collaborative. I always feel like you’re involved from the get-go.”

“I think we need to also break the image that computer science is all about coding. I’ve had friends that have stayed away from any tech jobs because they think that they don’t want to code, but there’s so many other roles within technology and jobs that actually require no coding whatsoever.”

Read Tasnima’s whole interview

Meet Aleena

Aleena is a software engineer who works at a health tech startup in London and is also studying for a master’s degree in AI ethics at the University of Cambridge.

Aleena smiling. The text says I belong in computer science.

“I do quite a lot of different things as an engineer. It’s not just coding, which is part of it but it is a relatively small percentage, compared to a lot of other things. […] There’s a lot of collaborative time and I would say a quarter or third of the week is me by myself writing code. The other time is spent collaborating and working with other people and making sure that we’re all aligned on what we are working on.”

“I think it’s actually a very diverse field of tech to work in, once you actually end up in the industry. When studying STEM subjects at a college or university level it is often not very diverse. The industry is the opposite. A lot of people come from self-taught or bootcamp backgrounds, there’s a lot of ways to get into tech and software engineering, and I really like that aspect of it. Computer science isn’t the only way to go about it.”

Read Aleena’s full interview

Meet Alice

Alice is a final-year undergraduate student of Computer Science with Artificial Intelligence at the University of Brighton. She is also the winner of the Global Challenges COVID-19 Research Scholarship offered by Santander Universities.

Alice wearing a mask over her face and mouth. The text says I belong in computer science.

“[W]e need to advertise computer science as more than just a room full of computers, and to advertise computer sciences as highly collaborative. It’s very creative. If you’re on a team of developers, there’s a lot of communication involved.”

“There’s something about computer science that I think is so special: the fact that it is a skill anybody can learn, regardless of who they are. With the right idea, anybody can build anything.”

Read Alice’s full interview

Share these stories to inspire

Help us spread the message that everyone belongs in computer science: share this blog with schools, teachers, STEM clubs, parents, and young people you want to inspire.

You can learn computer science with us

Whether you’re studying or teaching computer science GCSE or A levels in the UK (or thinking about doing so!), or you’re a teacher or student in another part of the world, Isaac Computer Science is here to help you achieve your computer science goals. Our high-quality learning platform is free to use and open to all. As a student, you can register to keep track of your progress. As a teacher, you can sign up to guide your students’ learning.

Two teenage boys do coding at a shared computer during a computer science lesson while their woman teacher observes them.

And for younger learners, we have lots of fun project guides to try out coding and creating with digital technologies.

Three teenage girls at a laptop

The post I belong in computer science appeared first on Raspberry Pi.

A storytelling approach for engaging girls in the Computing classroom: Pilot study results

Post Syndicated from Katharine Childs original https://www.raspberrypi.org/blog/gender-balance-in-computing-storytelling-approach-engaging-girls/

We’ve been running the Gender Balance in Computing programme of research since 2019, as part of the National Centre for Computing Education (NCCE) and with various partners. It’s a £2.4 million research programme funded by the Department for Education in England that aims to identify ways to encourage more girls and young women to engage with Computing and choose to study it further. The programme is made up of four separate areas of research, in which we are running a number of interventions.

Teenage students and a teacher do coding during a computer science lesson.

The first independent evaluation report from the Behavioural Insights Team (BIT) on our series of interventions has now been published. It relates to an intervention within the research area ‘Teaching Approach’, evaluating our pilot study of teaching computing to Key Stage 1 children using a storytelling approach. The evaluators from BIT found that this pilot study produced evidence of promise for the storytelling approach. They recommend conducting a full-size trial to test how effective this approach is for engaging female pupils with Computing.

Teaching computing through storytelling

Like many Computing curricula around the world, the English National Curriculum emphasises the importance of teaching Computing through a range of content so that pupils can express themselves and develop their ideas using digital tools. Our ‘Teaching Approach’ project builds on research grounded in sociocultural learning theories that suggest teaching approaches that encourage collaboration and use a variety of contexts can make Computing a more inclusive subject for all learners. Within this project, we are running three different interventions, each with learners of different ages.

In a computing classroom, a girl looks at a computer screen.

Evidence indicates that gender stereotypes around Computing develop early (1). Therefore we designed a trial — the first of its kind in England — to explore a storytelling approach for teaching Computing with younger children (6- to 7-year-olds). A small body of research suggests that using storytelling as a learning context for Computing can be engaging for both boys and girls. Research results indicate that:

  • Teaching computing through storytelling and story-writing is effective for motivating 11- to 14-year-old girls to learn programming (2)
  • Children who write computer programs to tell stories see Computing as a subject that is equally as easy or difficult for both boys and girls (3)
  • In a non-formal learning space, primary-aged girls are more likely to choose a storybook beginner electronics activity rather than open-ended beginner electronics free play (4)

The pilot study and the evaluation methods

As combining evidence from research with older students and in non-formal education is experimental, we designed this storytelling trial as a small pilot study. Our aim was to generate early evidence as to how feasible a teaching approach that uses storytelling might be in the primary Computing classroom.

We recruited 53 schools to take part in the pilot study, which ran from April to July 2021. Many schools were still facing challenges due to the ongoing coronavirus pandemic, and we are very grateful to the teachers and learners who have taken part for their contribution to this important research.

In a computing classroom, a girl looks at a computer screen.

To conduct the study, we created a free online training course, and a scheme of work, for schools to teach Computing concepts to 6- and 7-year olds using a storytelling approach. Over a sequence of the 12 lessons in the scheme of work, pupils used the ScratchJr programming environment to animate their own digital stories and learn about Computing concepts, such as sequence and repetition, linked to elements of stories, such as structure, rhyme, and speech. 

To enable the independent evaluation of the effectiveness of the storytelling approach by BIT, schools were allocated either to an intervention group, which used the training course and the storytelling scheme of work, or to a control group, which taught Computing in their usual way and was not made aware that the approach being trialled involved storytelling. For their evaluation, BIT gathered data from both groups to compare them:

  • They conducted surveys measuring learners’ attitudes toward computing and their intentions to study it in the future
  • They carried out observations of lessons, interviews with teachers, and discussions with learners
  • They ran a survey to gather feedback about the trial from teachers

The gathered data was assessed against five categories: evidence of promise, fidelity, acceptability, feasibility, and readiness for trial.

Main findings of the evaluation team

After analysing the data collected from observations, interviews, learner discussions, pupil surveys, and teacher surveys, the key finding of the independent evaluators was that the storytelling teaching approach had evidence of promise, and that it is worthwhile scaling up our intervention for a larger trial with more schools.

The evaluators’ teacher interviews confirmed the early development of gender stereotypes in the classroom. This highlights the importance of introducing Computing to young learners in a way that engages both boys and girls. 

“I’ve really noticed how there’s already differences in views of what’s a boy, what’s a girl, the boys are getting in front of me, like, ‘I want a boy car, I don’t want a girl car’. Then we’ve got the other side where we’ve got fairy tales and princesses and, ‘Oh, I’m a bunny. Do you want to play with me?’”

Teacher (evaluation report, p. 22)

Teachers told the evaluators that pupils enjoyed personalising their stories in ScratchJr, and that they themselves felt positive about the use of storytelling to teach computing. 

“I think [the storytelling aspect] gives them something real to work through, so it’s not… abstract… I think through the storytelling, they’re able to make it as funny or whatever they want, and it’s also their own interest. [Female student], she dotes on animals, so she’s always having giraffes and all of that, so it’s something that they can make their own connections too… Yes, I did really like the storytelling.”

Teacher (evaluation report, p. 26)

Teacher feedback provided some evidence that the storytelling lessons had equally increased both male and female pupils’ interest, confidence, and skills.

Young learners at computers in a classroom.

The independent evaluation team advised caution when interpreting the quantitative data from the pupil surveys, due to the small sample size in this pilot study and the high attrition rates caused by coronavirus-related disruptions. We ourselves would like to add that the study raises questions about the reliability of quantitative survey data collected from very young children using Likert scales, BIT’s chosen survey format for this evaluation. Although the evaluators have made some positive steps in creating a new survey suitable for young children, this research instrument may need further testing; the survey results would need to be interpreted in this light, and more research in this area would be recommended.

You can read the full evaluation report on the NCCE website.

Future directions

This intervention was based on one of the teaching approaches for which there was only early evidence of effectiveness, so it is a good outcome to have a larger trial recommended based on our pilot study. It’s often said that research ends up recommending more research, but in this case our small pilot project really does give robust evidence that we should trial the storytelling approach with more schools.

In a computing classroom, a girl looks at a computer screen.

The independent evaluators collected feedback from both teachers and pupils that confirms the storytelling intervention we designed is feasible in the classroom. The feedback also indicates where we can make small adjustments that will refine and develop the training and scheme of work for a larger-scale study (evaluation report, p. 35), and we will consider this feedback carefully. While some teachers suggested that the training be shortened, less experienced teachers highlighted the need to ensure the training introduces teachers to all of the content covered in the lessons. This feedback helps us to better understand how Computing is taught in primary schools, and how this is influenced by the wide variety of experience and subject knowledge that teachers have. Interestingly, in the control group, some of the teachers reported that they also introduced coding to their learners by having them create stories. We would like to conduct further research into how schools introduce young learners to programming, and we’ll be continuing to reflect on how best to offer flexible content for teacher training related to our research studies.

We’re now looking at how to continue to investigate the effectiveness of the storytelling approach through a larger trial, alongside other projects in which we’re exploring female engagement in computing education through our recently established Raspberry Pi Computing Education Research Centre.

More evaluations are on the way for our other studies in the Gender Balance in Computing programme, including:

  • Two other trials of teaching approaches
  • Interventions in non-formal education contexts
  • Trials of approaches to building a sense of belonging in Computing
  • Research into the impact of timetabling and options evenings

If you would like to stay up-to-date with the research programme, you can sign up to the Gender Balance in Computing newsletter. We will also post our reflections on the projects on this blog when the evaluations are completed.

1 Mulvey, K. L. and Irvin, M. J. (2018). Judgments and reasoning about exclusion from counter-stereotypic STEM career choices in early childhood. Early Child. Res. Q. 44, 220–230. https://doi.org/10.1016/j.ecresq.2018.03.016

2 Kelleher, C., Pausch, R. and Kiesler, S. (2007). Storytelling alice motivates middle school girls to learn computer programming. In CHI ’07: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1455–1464. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1240624.1240844

3 Zaidi, R., Freihofer, I. and Childress Townsend, G. (2017). Using Scratch and Female Role Models while Storytelling Improves Fifth-Grade Students’ Attitudes toward Computing. In SIGCSE ’17: Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education, 791–792. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3017680.3022451

4 McLean, M., & Harlow, D. (2017). Designing inclusive STEM activities: A comparison of playful interactive experiences across gender. In IDC ’17: Proceedings of the 2017 Conference on Interaction Design and Children, 567–574. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3078072.3084326

The post A storytelling approach for engaging girls in the Computing classroom: Pilot study results appeared first on Raspberry Pi.

Creating better online multiple choice questions

Post Syndicated from Jane Waite original https://www.raspberrypi.org/blog/better-online-multiple-choice-questions-education-edtech/

In this blog post we explore good practices around creating online computing questions, specifically multiple choice questions (MCQs). Multiple choice questions are a popular way to help teachers and learners work out the next steps in learning, and to assess learning in examinations. As a case study, we look at some data related to learner responses to computing questions on the Oak National Academy platform.

Someone fills in a standardised test with multiple choice questions using a pencil.

The case study illustrates the many things MCQ authors have to think about while designing questions, and that there is much more research needed to understand how to get an MCQ “just right”.

Uses of multiple choice questions

Online auto-marked MCQs are now being integrated into classroom activities, set as homework, and used in self-led learning at home. Software products involving MCQs, such as Kahoot and Socratic, are easy to use for many, and have become popular in some learning contexts. MCQ may have become more prevalent due to increased online teaching and the availability of whole curricula through platforms such as the Oak National Academy.

A girl does school work at a laptop at home.

An international group of researchers from China, Spain, Singapore, and the UK recently looked into the reasons why MCQ-based testing might improve learning. Chunliang Yang and his co-authors concluded that there are three main ways that MCQ tests help learners learn:

  • They provide learners with additional exposure to learning content
  • They provide learners with content in the same format that they will be later assessed in 
  • They motivate learners, e.g. to prompt them to commit more effort to learn in general

What does the research say about creating multiple choice questions?

In recent research reviewing the use of MCQs, Andrew Butler from Washington University in St Louis looked at the effectiveness of MCQs in relation to learning, rather than assessment. Andrew gives the following advice for educators creating MCQs for learning:

  • Think about the thinking processes the learner will use when answering the question, and make sure the processes are productive for their learning
  • Don’t make the question super easy or too difficult, but make it challenging — the difficulty needs to be “just right”
  • Keep the phrasing of the question simple 
  • Ensure that all answers are plausible; providing three or four answers is usually a good idea
  • Be aware that if learners pick the wrong answer, this can reinforce the wrong thinking
  • Provide corrective feedback to learners who pick the wrong answer

What I find particularly interesting about Andrew’s advice is the need to make the difficulty of the MCQ “just right” for learners. But what does “just right” look like in practice? More research is needed to work this out.

The anatomy of a multiple choice question

When talking about MCQs, there are technical terms to describe question features, e.g.:

  • Incorrect answers are called distractors (or lures)
  • A distractor is defined as plausible if it’s an answer a layperson would see as a reasonable answer
  • Plausible distractors are called working distractors

Here at the Foundation, we created MCQs for the Oak National Academy when we adapted our Teach Computing Curriculum classroom materials into video lessons and accompanying home learning content to support learners and teachers during school closures. Data about what questions are attempted on the Oak platform, and what answer options are chosen, is stored securely by Oak National Academy. The Oak team kindly provided us with four months of anonymous data related to responses to the MCQs in the ‘GCSE Computer Science – Data representations’ unit.

Over this period of four months, learners on the platform made more than 29,000 question attempts on the thirty-five questions across the nine lessons that make up this data representation unit. Here is a breakdown of the questions by topic area:

Data about responses to a set of multiple choice questions on the Oak Academy platform.of a multiple choice question on the Oak Academy platform.
Responses to MCQs in the GCSE Computer Science data representation unit on Oak National Academy, data from February 2021 to end of May 2021 (click to enlarge)

As shown in the table, more questions relate to binary arithmetic than to any other topic area. This was a specific design decision, as it is well-known that learners need lots of practice of the processes involved in answering binary arithmetic questions.

Part of the graph of learning objectives for the Teach Computing Curriculum unit GCSE Computer Science data representation.
Part of the graph of learning objectives for the Teach Computing Curriculum unit GCSE Computer Science — Data representations (click to enlarge)

Let’s look at an example question from the binary arithmetic topic area, with one correct answer and two distractors. The learning objective being addressed with this question is ‘Perform addition in binary on two binary numbers’.

Screenshot of a multiple choice question on the Oak Academy platform.
One of the MCQs in the GCSE Computer Science data representation unit on the Oak National Academy, as displayed on the online platform

As shown in the table below, in four months, 1170 attempts were made to answer the example question. 65% of the attempts were correct responses, and 35% were not, with 21% of responses being distractor b, and 14% distractor c. These distractors appear to be working distractors, as they were chosen by more than 5% of learners, which has been suggested as a rule-of-thumb threshold that distractors have to clear to be classed as working.

Data about responses to a multiple choice question on the Oak Academy platform.
Example MCQ in the GCSE Computer Science data representation unit on the Oak National Academy, plus response data from February 2021 to end of May 2021 (click to enlarge)

However, because of the lack of research into MCQs, we cannot say for certain that this question is “just right” — it may be too hard. We need to do further research to find this out.

Creating multiple choice questions is not easy

The process of creating good MCQs is not an easy task, because question authors need to think about many things, including:

  • What learning objectives are to be addressed
  • What plausible distractors can be used
  • What level of difficulty is right for learners
  • What type of thinking the questions are encouraging, and how this is useful for learners

In order for MCQs to be useful for learners and teachers, much more research is needed in this area to show how to reliably produce MCQs that are “just right” and encourage productive thinking processes. We are very much looking forward to looking at this topic in our research work.

To find out more about the computing education research we are doing, you can browse our website, take part in our monthly seminars, and read our publications.

The post Creating better online multiple choice questions appeared first on Raspberry Pi.

Supporting teachers and students with remote learning through free video lessons

Post Syndicated from original https://www.raspberrypi.org/blog/supporting-teachers-students-remote-learning-free-video-lessons/

Working with Oak National Academy, we’ve turned the materials from our Teach Computing Curriculum into more than 300 free, curriculum-mapped video lessons for remote learning.

A girl in a hijab learning at home at a laptop

A comprehensive set of free classroom materials

One of our biggest projects for teachers that we’ve worked on over the past two years is the Teach Computing Curriculum: a comprehensive set of free computing classroom materials for key stages 1 to 4 (learners aged 5 to 16). The materials comprise lesson plans, homework, progression mapping, and assessment materials. We’ve created these as part of the National Centre for Computing Education, but they are freely available for educators all over the world to download and use.

More than 300 free, curriculum-mapped video lessons

In the second half of 2020, in response to school closures, our team of experienced teachers produced over 100 hours of video to transform Teach Computing Curriculum materials into video lessons for learning at home. They are freely available for parents, educators, and learners to continue learning computing at home, wherever you are in the world.

Here’s the start of lesson 2 in the Year 8 ‘Computer systems’ unit

You’ll find our videos for more than 300 hour-long lessons on the Oak National Academy website. The progression of the lessons is mapped out clearly, and the videos cover England’s computing national curriculum. There are video lessons for:

  • Years 5 and 6 at key stage 2 (ages 7 to 11)
  • Years 7, 8, and 9 at key stage 3 (ages 11 to 14)
  • Examined (GCSE) as well as non-examined (Digital Literacy) at key stage 4 (ages 14 to 16)

To access the full set of classroom materials for teaching, visit the National Centre for Computing Education website.

The post Supporting teachers and students with remote learning through free video lessons appeared first on Raspberry Pi.

How teachers train in Computing with our free online courses

Post Syndicated from Michael Conterio original https://www.raspberrypi.org/blog/how-teachers-train-computing-free-online-courses/

Since 2017 we’ve been training Computing educators in England and around the world through our suite of free online courses on FutureLearn. Thanks to support from Google and the National Centre for Computing Education (NCCE), all of these courses are free for anyone to take, whether you are a teacher or not!

An illustration of a bootcamp for computing teachers

We’re excited that Computer Science educators at all stages in their computing journey have embraced our courses — from teachers just moving into the field to experienced educators looking for a refresher so that they can better support their colleagues.

Hear from two teachers about their experience of training with our courses and how they are benefitting!

Moving from Languages to IT to Computing

Rebecca Connell started out as a Modern Foreign Languages teacher, but now she is Head of Computing at The Cowplain School, a 11–16 secondary school in Hampshire.

Computing teacher Rebecca Connell
Computing teacher Rebecca finds our courses “really useful in building confidence and taking [her] skills further”.

Although she had plenty of experience with Microsoft Office and was happy teaching IT, at first she was daunted by the technical nature of Computing:

“The biggest challenge for me has been the move away from an IT to a Computing curriculum. To say this has been a steep learning curve is an understatement!”

However, Rebecca has worked with our courses to improve her coding knowledge, especially in Python:

“Initially, I undertook some one-day programming courses in Python. Recently, I have found the Raspberry Pi courses to be really useful in building confidence and taking my skills further. So far, I have completed Programming 101 — great for revision and teaching ideas — and am now into Programming 102.”

GCSE Computing is more than just programming, and our courses are helping Rebecca develop the rest of her Computing knowledge too:

“I am now taking some online Raspberry Pi courses on computer systems and networks to firm up my knowledge — my greatest fear is saying something that’s not strictly accurate! These courses have some good ideas to help explain complex concepts to students.”

She also highly rates the new free Teach Computing Curriculum resources we have developed for the NCCE:

“I really like the new resources and supporting materials from Raspberry Pi — these have really helped me to look again at our curriculum. They are easy to follow and include everything you need to take students forward, including lesson plans.”

And Rebecca’s not the only one in her department who is benefitting from our courses and resources:

“Our department is supported by an excellent PE teacher who delivers lessons in Years 7, 8, and 9. She has enjoyed completing some of the Raspberry Pi courses to help her to deliver the new curriculum and is also enjoying her learning journey.”

Refreshing and sharing your knowledge

Julie Price, a CAS Master Teacher and NCCE Computer Science Champion, has been “engaging with the NCCE’s Computer Science Accelerator programme, [to] be in a better position to appreciate and help to resolve any issues raised by fellow participants.”

Computing teacher Julie Price
Computer science teacher Julie Price says she is “becoming addicted” to our online courses!

“I have encountered new learning for myself and also expressions of very familiar content which I have found to be seriously impressive and, in some cases, just amazing. I must say that I am becoming addicted to the Raspberry Pi Foundation’s online courses!”

She’s been appreciating the open nature of the courses, as we make all of the materials free to use under the Open Government Licence:

“Already I have made very good use of a wide range of the videos, animations, images, and ideas from the Foundation’s courses.”

Julie particularly recommends the Programming Pedagogy in Secondary Schools: Inspiring Computing Teaching course, describing it as “a ‘must’ for anyone wishing to strengthen their key stage 3 programming curriculum.”

Join in and train with us

Rebecca and Julie are just 2 of more than 140,000 active participants we have had on our online courses so far!

I want to browse the free courses

With 29 courses to choose from (and more on the way!), from Introduction to Web Development to Robotics with Raspberry Pi, we have something for everyone — whether you’re a complete beginner or an experienced computer science teacher. All of our courses are free to take, so find one that inspires you, and let us support you on your computing journey, along with Google and the NCCE.

If you’re a teacher in England, you are eligible for free course certification from FutureLearn via the NCCE.

The post How teachers train in Computing with our free online courses appeared first on Raspberry Pi.