Tag Archives: Shuchi Grover

Formative assessment in the computer science classroom

Post Syndicated from Sue Sentance original https://www.raspberrypi.org/blog/research-seminar-formative-assessment-computer-science-classroom/

In computing education research, considerable focus has been put on the design of teaching materials and learning resources, and investigating how young people learn computing concepts. But there has been less focus on assessment, particularly assessment for learning, which is called formative assessment. As classroom teachers are engaged in assessment activities all the time, it’s pretty strange that researchers in the area of computing and computer science in school have not put a lot of focus on this.

Shuchi Grover

That’s why in our most recent seminar, we were delighted to hear about formative assessment — assessment for learning — from Dr Shuchi Grover, of Looking Glass Ventures and Stanford University in the USA. Shuchi has a long track record of work in the learning sciences (called education research in the UK), and her contribution in the area of computational thinking has been hugely influential and widely drawn on in subsequent research.

Two types of assessment

Assessment is typically divided into two types:

  1. Summative assessment (i.e. assessing what has been learned), which typically takes place through examinations, final coursework, projects, etcetera.
  2. Formative assessment (i.e. assessment for learning), which is not aimed at giving grades and typically takes place through questioning, observation, plenary classroom activities, and dialogue with students.

Through formative assessment, teachers seek to find out where students are at, in order to use that information both to direct their preparation for the next teaching activities and to give students useful feedback to help them progress. Formative assessment can be used to surface misconceptions (or alternate conceptions) and for diagnosis of student difficulties.

Venn diagram of how formative assessment practices intersect with teacher knowledge and skills
Click to enlarge

As Shuchi outlined in her talk, a variety of activities can be used for formative assessment, for example:

  • Self- and peer-assessment activities (commonly used in schools).
  • Different forms of questioning and quizzes to support learning (not graded tests).
  • Rubrics and self-explanations (for assessing projects).

A framework for formative assessment

Shuchi described her own research in this topic, including a framework she has developed for formative assessment. This comprises three pillars:

  1. Assessment design.
  2. Teacher or classroom practice.
  3. The role of the community in furthering assessment practice.
Shuchi Grover's framework for formative assessment
Click to enlarge

Shuchi’s presentation then focused on part of the first pillar in the framework: types of assessments, and particularly types of multiple-choice questions that can be automatically marked or graded using software tools. Tools obviously don’t replace teachers, but they can be really useful for providing timely and short-turnaround feedback for students.

As part of formative assessment, carefully chosen questions can also be used to reveal students’ misconceptions about the subject matter — these are called diagnostic questions. Shuchi discussed how in a classroom setting, teachers can employ this kind of question to help them decide what to focus on in future lessons, and to understand their students’ alternate or different conceptions of a topic. 

Formative assessment of programming skills

The remainder of the seminar focused on the formative assessment of programming skills. There are many ways of assessing developing programming skills (see Shuchi’s slides), including Parsons problems, microworlds, hotspot items, rubrics (for artifacts), and multiple-choice questions. As an MCQ example, in the figure below you can see some snippets of block-based code, which students need to read and work out what the outcome of running the snippets will be. 

Click to enlarge

Questions such as this highlight that it’s important for learners to engage in code comprehension and code reading activities when learning to program. This really underlines the fact that such assessment exercises can be used to support learning just as much as to monitor progress.

Formative assessment: our support for teachers

Interestingly, Shuchi commented that in her experience, teachers in the UK are more used to using code reading activities than US teachers. This may be because code comprehension activities are embedded into the curriculum materials and support for pedagogy, both of which the Raspberry Pi Foundation developed as part of the National Centre for Computing Education in England. We explicitly share approaches to teaching programming that incorporate code reading, for example the PRIMM approach. Moreover, our work in the Raspberry Pi Foundation includes the Isaac Computer Science online learning platform for A level computer science students and teachers, which is centered around different types of questions designed as tools for learning.

All these materials are freely available to teachers wherever they are based.

Further work on formative assessment

Based on her work in US classrooms researching this topic, Shuchi’s call to action for teachers was to pay attention to formative assessment in computer science classrooms and to investigate what useful tools can support them to give feedback to students about their learning. 

Advice from Shuchi Grover on how to embed formative assessment in classroom practice
Click to enlarge

Shuchi is currently involved in an NSF-funded research project called CS Assess to further develop formative assessment in computer science via a community of educators. For further reading, there are two chapters related to formative assessment in computer science classrooms in the recently published book Computer Science in K-12 edited by Shuchi.

There was much to take away from this seminar, and we are really grateful to Shuchi for her input and look forward to hearing more about her developing project.

Join our next seminar

If you missed the seminar, you can find the presentation slides and a recording of the Shuchi’s talk on our seminars page.

In our next seminar on Tuesday 3 November at 17:00–18:30 BST / 12:00–13:30 EDT / 9:00–10:30 PT / 18:00–19:30 CEST, I will be presenting my work on PRIMM, particularly focusing on language and talk in programming lessons. To join, simply sign up with your name and email address.

Once you’ve signed up, we’ll email you the seminar meeting link and instructions for joining. If you attended this past seminar, the link remains the same.

The post Formative assessment in the computer science classroom appeared first on Raspberry Pi.

“Tinkering is an equity issue” | Hello World #14

Post Syndicated from Sian Williams Page original https://www.raspberrypi.org/blog/tinkering-is-an-equity-issue-shuchi-grover-hello-world-14/

In the brand-new issue of Hello World magazine, Shuchi Grover tells us about the limits of constructionism, the value of formative assessment, and why programming can be a source of both joy and angst.

How much open-ended exploration should there be in computing lessons?

This is a question at the heart of computer science education and one which Shuchi Grover is delicately diplomatic about in the preface to her new book, Computer Science in K-12: An A-to-Z Handbook on Teaching Programming. The book’s chapters are written by 40 teachers and researchers in computing pedagogy, and Grover openly acknowledges the varying views around discovery-based learning among her diverse range of international authors.

“I wonder if I want to wade there,” she laughs. “The act of creating a program is in itself an act of creation. So there is hands-on learning quite naturally in the computer science classroom, and mistakes are made quite naturally. There are some things that are so great about computer science education. It lends itself so easily to being hands-on and to celebrating mistakes; debugging is par for the course, and that’s not the way it is in other subjects. The kids can actually develop some very nice mindsets that they can take to other classrooms.”

Shuchi Grover showing children something on a laptop screen

Grover is a software engineer by training, turned researcher in computer science education. She holds a PhD in learning sciences and technology design from Stanford University, where she remains a visiting scholar. She explains how the beginning of her research career coincided with the advent of the block-based programming language Scratch, now widely used as an introductory programming language for children.

“Almost two decades ago, I went to Harvard to study for a master’s called technology innovation and education, and it was around that time that I volunteered for robotics workshops at the MIT Media Lab and MIT Museum. Those were pretty transformative for me: I started after-school clubs and facilitated robotics and digital storytelling clubs. In the early 2000s, I was an educational technology consultant, working with teachers on integrating technology. Then Scratch came out, and I started working with teachers on integrating Scratch into languages, arts, and science, all the things that we are doing today.”

A girl with her Scratch project
Student Joyce codes in Scratch at her Code Club in Nunavut

Do her formative experiences at MIT, the birthplace of constructionist theory of student-centred, discovery-based learning, lead her to lean one way or another in the tinkering versus direct instruction debate? “The learning in informal spaces is, of course, very interest-driven. There is no measurement. Children are invited to a space to spend some time after school and do whatever they feel like. There would be kids who would be chatting away while a couple of them designed a robot, and then they would hand over the robot to some others and say, ‘OK, now you go ahead and program it,’ and there were some kids who would just like to hang about.

“When it comes to formal education, there needs to be more accountability, you want to do right by every child. You have to be more intentional. I do feel that while tinkering and constructionism was a great way to introduce interest-driven projects for informal learning, and there’s a lot to learn from there and bring to the formal learning context, I don’t think it can only be tinkering.”

“There needs to be more accountability to do right by every child.”

“Everybody knows that engagement is very important for learning — and this is something that we are learning more about: it’s not just interest, it’s also culture, communities, and backgrounds — but all of this is to say that there is a personal element to the learning process and so engagement is necessary, but it’s not a sufficient condition. You have to go beyond engagement, to also make sure that they are also engaging with the concepts. You want at some point for students to engage with the concept in a way that reveals what their misconceptions might be, and then they end up learning and understanding these things more deeply.

“You want a robust foundation — after all, our goal for teaching children anything at school is to build a foundation on which they build their college education and career and anything beyond that. If we take programming as a skill, you want them to have a good understanding of it, and so the personal connections are important, but so is the scaffolding.

“How much scaffolding needs to be done varies from context to context. Even in the same classroom, children may need different levels of scaffolding. It’s a sweet spot; within a classroom a teacher has to juggle so much. And therein lies the challenge of teaching: 30 kids at a time, and every child is different and every child is unique.

“It’s an equity issue. Some children don’t have the prior experience that sets them up to tinker constructively. After all, tinkering is meant to be purposeful exploration. And so it becomes an issue of who are you privileging with the pedagogy.”

She points out that each chapter in her book that comes from a more constructionist viewpoint clearly speaks of the need for scaffolding. And conversely, the chapters that take a more structured approach to computing education include elements of student engagement and children creating their own programs. “Frameworks such as Use-Modify-Create and PRIMM just push that open-ended creation a little farther down, making sure that the initial experiences have more guide rails.”

Approaches to assessment

Grover is a senior research scientist at Looking Glass Ventures, which in 2018 received a National Science Foundation grant to create Edfinity, a tool to enable affordable access to high-quality assessments for schools and universities.

In her book, she argues that asking students to write programs as a means of formative assessment has several pitfalls. It is time-consuming for both students and teachers, scoring is subjective, and it’s difficult to get a picture of how much understanding a student has of their code. Did they get their program to work through trial and error? Did they lift code from another student?

“Formative assessments that give quick feedback are much better. They focus on aspects of the conceptual learning that you want children to have. Multiple-choice questions on code force both the teachers and the children to experience code reading and code comprehension, which are just so important. Just giving children a snippet of code and saying: ‘What does this do? What will be the value of the variable? How many times will this be executed?’ — it goes down to the idea of code tracing and program comprehension.

“Research has also shown that anything you do in a classroom, the children take as a signal. Going back to the constructionist thing, when you foreground personal interest, there’s a different kind of environment in the classroom, where they’re able to have a voice, they have agency. That’s one of the good things about constructionism.

“Formative assessment signals to the student what it is that you’re valuing in the learning process. They don’t always understand what it is that they’re expected to learn in programming. Is the goal creating a program that runs? Or is it something else? And so when you administer these little check-ins, they bring more alignment between a teacher’s goals for the learners and the learners’ understanding of those goals. That alignment is important and it can get lost.”

Grover will present her latest research into assessment at our research seminar series next Tuesday 6 October — sign up to attend and join the discussion.

The joy and angst of programming

The title of Grover’s book, which could be thought to imply that computer science education consists solely of teaching students to program, may cause some raised eyebrows.

What about building robots or devices that interact with the world, computing topics like binary, or the societal impacts of technology? “I completely agree with the statement and the belief that computer science is not just about programming. I myself have been a proponent of this. But in this book I wanted to focus on programming for a couple of reasons. Programming is a central part of the computer science curriculum, at least here in the US, and it is also the part that teachers struggle with the most.

“I want to show where children struggle and how to help them.”

“As topics go, programming carries a lot of joy and angst. There is joy in computing, joy when you get it. But when a teacher is encountering this topic for the first time there is a lot of angst, because they themselves may not be understanding things, and they don’t know what it is that the children are not understanding. And there is this entire body of research on novice programming. There are the concepts, the practices, the pedagogies, and the issues of assessment. So I wanted to give the teachers all of that: everything we know about children and programming, the topics to be learnt, where they struggle, how to help them.”

Computer Science in K-12: An A-to-Z Handbook on Teaching Programming (reviewed in this issue of Hello World) is edited by Shuchi Grover and available now.

Hear more from Shuchi Grover, and subscribe to Hello World

We will host Grover at our next research seminar, Tuesday 6 October at 17:00–18:30 BST, where she will present her work on formative assessment.

Hello World is our magazine about all things computing education. It is free to download in PDF format, or you can subscribe and we will send you each new issue straight to your home.

In issue 14 of Hello World, we have gathered some inspiring stories to help your learners connect with nature. From counting penguins in Antarctica to orienteering with a GPS twist, great things can happen when young people get creative with technology outdoors. You’ll find all this and more in the new issue!

Educators based in the UK can subscribe to receive print copies for free!

The post “Tinkering is an equity issue” | Hello World #14 appeared first on Raspberry Pi.