Replicate the physics of barrel rolling – straight out of the classic Donkey Kong. Mark Vanstone shows you how.
Released in 1981, Donkey Kong was one of the most important games in Nintendo’s history.
Nintendo’s Donkey Kong
Donkey Kong first appeared in arcades in 1981, and starred not only the titular angry ape, but also a bouncing, climbing character called Jumpman – who later went on to star in Nintendo’s little-known series of Super Mario games. Donkey Kong featured four screens per level, and the goal in each was to avoid obstacles and guide Mario (sorry, Jumpman) to the top of the screen to rescue the hapless Pauline. Partly because the game was so ferociously difficult from the beginning, Donkey Kong’s first screen is arguably the most recognisable today: Kong lobs an endless stream of barrels, which roll down a network of crooked girders and threaten to knock Jumpman flat.
Barrels in Pygame Zero
Donkey Kong may have been a relentlessly tough game, but we can recreate one of its most recognisable elements with relative ease. We can get a bit of code running with Pygame Zero – and a couple of functions borrowed from Pygame – to make barrels react to the platforms they’re on, roll down in the direction of a slope, and fall off the end onto the next platform. It’s a very simple physics simulation using an invisible bitmap to test where the platforms are and which way they’re sloping. We also have some ladders which the barrels randomly either roll past or sometimes use to descend to the next platform below.
Our Donkey Kong tribute up and running in Pygame Zero. The barrels roll down the platforms and sometimes the ladders.
Once we’ve created a barrel as an Actor, the code does three tests for its platform position on each update: one to the bottom-left of the barrel, one bottom-centre, and one bottom-right. It samples three pixels and calculates how much red is in those pixels. That tells us how much platform is under the barrel in each position. If the platform is tilted right, the number will be higher on the left, and the barrel must move to the right. If tilted left, the number will be higher on the right, and the barrel must move left. If there is no red under the centre point, the barrel is in the air and must fall downward.
There are just three frames of animation for the barrel rolling (you could add more for a smoother look): for rolling right, we increase the frame number stored with the barrel Actor; for rolling to the left, we decrease the frame number; and if the barrel’s going down a ladder, we use the front-facing images for the animation. The movement down a ladder is triggered by another test for the blue component of a pixel below the barrel. The code then chooses randomly whether to send the barrel down the ladder.
The whole routine will keep producing more barrels and moving them down the platforms until they reach the bottom. Again, this is a very simple physics system, but it demonstrates how those rolling barrels can be recreated in just a few lines of code. All we need now is a jumping player character (which could use the same invisible map to navigate up the screen) and a big ape to sit at the top throwing barrels, then you’ll have the makings of your own fully featured Donkey Kong tribute.
You can read more features like this one in Wireframe issue 24, available now at Tesco, WHSmith, all good independent UK newsagents, and the Raspberry Pi Store, Cambridge.
Or you can buy Wireframe directly from Raspberry Pi Press — delivery is available worldwide. And if you’d like a handy digital version of the magazine, you can also download issue 24 for free in PDF format.
Make sure to follow Wireframe on Twitter and Facebook for updates and exclusive offers and giveaways. Subscribe on the Wireframe website to save up to 49% compared to newsstand pricing!
Over the autumn term, we’ll be launching three brand-new, free online courses on the FutureLearn platform. Wherever you are in the world, you can learn with us for free!
The course presenters are Pi Towers residents Mark, Janina, and Eirini
Design and Prototype Embedded Computer Systems
The first new course is Design and Prototype Embedded Computer Systems, which will start on 28 October. In this course, you will discover the product design life cycle as you design your own embedded system!
You’ll investigate how the purpose of the system affects the design of the system, from choosing its components to the final product, and you’ll find out more about the design of an algorithm. You will also explore how embedded systems are used in the world around us. Book your place today!
Programming 103: Saving and Structuring Data
What else would you expect us to call the sequel to Programming 101 and Programming 102? That’s right — we’ve made Programming 103: Saving and Structuring Data! The course will begin on 4 November, and you can reserve your place now.
Programming 103 explores how to use data across multiple runs of your program. You’ll learn how to save text and binary files, and how structuring data is necessary for programs to “understand” the data that they load. You’ll look at common types of structured files such as CSV and JSON files, as well as how you can connect to a SQL database to use it in your Python programs.
Introduction to Encryption and Cryptography
The third course, Introduction to Encryption and Cryptography, is currently in development, and therefore coming soon. In this course, you’ll learn what encryption is and how it was used in the past, and you’ll use the Caesar and Vigenère ciphers.
The Caesar cipher is a type of substitution cipher
You’ll also look at modern encryption and investigate both symmetric and asymmetric encryption schemes. The course also shows you the future of encryption, and it includes several practical encryption activities, which can be used in the classroom too.
When we invited Estefannie Explains It All to present at Coolest Projects International, she decided to make something cool with a Raspberry Pi to bring along. But being Estefannie, she didn’t just make something a little bit cool. She went ahead and made Raspberry Pi Zero-powered Jurassic Park goggles, or, as she calls them, the world’s first globally triggered, mass broadcasting, photon-emitting and -collecting head unit.
Is it heavy? Yes. But these goggles are not expensive. Follow along as I make the classic Jurassic Park Goggles from scratch!! The 3D Models: https://www.thingiverse.com/thing:3732889 My code: https://github.com/estefanniegg/estefannieExplainsItAll/blob/master/makes/JurassicGoggles/jurassic_park.py Thank you Coolest Projects for bringing me over to speak in Ireland!! https://coolestprojects.org/ Thank you Polymaker for sending me the Polysher and the PolySmooth filament!!!!
3D-printing, sanding, and sanding
Estefannie’s starting point was the set of excellent 3D models of the iconic goggles that Jurassicpaul has kindly made available on Thingiverse. There followed several 3D printing attempts and lots of sanding, sanding, sanding, spray painting, and sanding, then some more printing with special Polymaker filament that can be ethanol polished.
Adding the electronics and assembling the goggles
Estefannie soldered rings of addressable LEDs and created custom models for 3D-printable pieces to fit both them and the goggles. She added a Raspberry Pi Zero, some more LEDs and buttons, an adjustable headgear part from a welding mask, and – importantly – four circles of green acetate. After quite a lot of gluing, soldering, and wiring, she ended up with an entirely magnificent set of goggles.
Here, they’re modelled magnificently by Raspberry Pi videographer Brian. I think you’ll agree he cuts quite a dash.
Coding and LED user interface
Estefannie wrote a Python script to interact with Twitter, take photos, and provide information about the goggles’ current status via the LED rings. When Estefannie powers up the Raspberry Pi, it runs a script on startup and connects to her phone’s wireless hotspot. A red LED on the front of the goggles indicates that the script is up and running.
Once it’s running, pressing a button at the back of the head unit makes the Raspberry Pi search Twitter for mentions of @JurassicPi. The LEDs light up green while it searches, just like you remember from the film. If Estefannie’s script finds a mention, the LEDs flash white and the Raspberry Pi camera module takes a photo. Then they light up blue while the script tweets the photo.
All the code is available on Estefannie’s GitHub. I love this project – I love the super clear, simple user experience provided by the LED rings, and there’s something I really appealing about the asynchronous Twitter interaction, where you mention @JurassicPi and then get an image later, the next time googles are next turned on.
Extra bonus Coolest Projects
If you read the beginning of this post and thought, “wait, what’s Coolest Projects?” then be sure to watch to the end of Estefannie’s video to catch her excellentCoolest Projects mini vlog. And then sign up for updates about Coolest Projects events near you, so you can join in next year, or help a team of young people to join in.
What happens when you give two linguists jobs at Raspberry Pi? They start thinking they can do digital making, even though they have zero coding skills! Because if you don’t feel inspired to step out of your comfort zone here — surrounded by all the creativity, making, and technology — then there is no hope you’ll be motivated to do it anywhere else.
Maja and Nina, our translation team, and coding beginners
Maja and I support the community of Raspberry Pi translation volunteers, and we wanted to build something to celebrate them and the amazing work they do! Our educational content is already available in 26 languages, with more than 400 translations on our projects website. But our volunteer community is always translating more content, and so off we went, on an ambitious (by our standards!) mission to create a Raspberry Pi–powered translation notification system. This is a Raspberry Pi that pulls GitHub data to display a message on a Sense HAT and play a tune whenever we add fresh translated content to the Raspberry Pi projects website!
Breaking it down
There were three parts to the project: two of them were pretty easy (displaying a message on a Sense HAT and playing a tune), and one more challenging (pulling information about new translated content added to our repositories on GitHub). We worked on each part separately and then put all of the code together.
Mandatory for coding: baked goods and tea
Displaying a message on Sense HAT and playing a sound
At first we wanted the Sense HAT to display fireworks, but we soon realised how bad we both are at designing animations, so we moved on to displaying a less creative but still satisfying smiley face, followed by a message saying “Hooray! Another translation!” and another smiley face.
We used the sense_hat and time modules, and wrote a function that can be easily used in the main body of the program. You can look at the comments in the code above to see what each line does:
So we could add the fun tune, we learned how to use the Pygame library to play sounds. Using Pygame it’s really simple to create a function that plays a sound: once you have the .wav file in your chosen location, you simply import and initialise the pygame module, create a Sound object, and provide it with the path to your .wav file. You can then play your sound:
We’ve programmed our translation notification system to play the meow sound three times, using the sleep function to create a one-second break between each sound. Because why would you want one meow if you can have three?
Pulling repository information from GitHub
This was the more challenging part for Maja and me, so we asked for help from experienced programmers, including our colleague Ben Nuttall. We explained what we wanted to do: pull information from our GitHub repositories where all the projects available on the Raspberry Pi projects website are kept, and every time a new language directory is found, to execute the sparkles and meow functions to let us and EVERYONE in the office know that we have new translations! Ben did a bit of research and quickly found the PyGithub library, which enables you to manage your GitHub resources using Python scripts.
Check out the comments to see what the code does
The script runs in an infinite loop, checking all repositories in the ‘raspberrypilearning’ organisation for new translations (directories with names in form of xx-XX, eg. fr-CA) every 60 minutes. Any new translation is then printed and preserved in memory. We had some initial issues with the usage of the PyGithub library: calling .get_commits() on an empty repository throws an exception, but the library doesn’t provide any functions to check whether a repo is empty or not. Fortunately, wrapping this logic in a try...except statement solved the problem.
Subscribe to our YouTube channel: http://rpf.io/ytsub Help us reach a wider audience by translating our video content: http://rpf.io/yttranslate Buy a Raspberry Pi from one of our Approved Resellers: http://rpf.io/ytproducts Find out more about the #RaspberryPi Foundation: Raspberry Pi http://rpf.io/ytrpi Code Club UK http://rpf.io/ytccuk Code Club International http://rpf.io/ytcci CoderDojo http://rpf.io/ytcd Check out our free online training courses: http://rpf.io/ytfl Find your local Raspberry Jam event: http://rpf.io/ytjam Work through our free online projects: http://rpf.io/ytprojects Do you have a question about your Raspberry Pi?
Our ideas for further development
We’re pretty proud that the whole Raspberry Pi office now hears a meowing cat whenever new translated content is added to our projects website, but we’ve got plans for further development of our translation notification system. Our existing translated educational resources have already been viewed by over 1 million users around the world, and we want anyone interested in the translations our volunteers make possible to be able to track new translated projects as the go live!
One way to do that is to modify the code to tweet or send an email with the name of the newly added translation together with a link to the project and information on the language in which it was added. Alternatively, we could adapt the system to only execute the sparkles and meow functions when a translation in a particular language is added. Then our more than 1000 volunteers, or any learner using our translations, could set up their own Raspberry Pi and Sense HAT to receive notifications of content in the language that interests them, rather than in all languages.
We need your help
Both ideas pose a pretty big challenge for the inexperienced new coders of the Raspberry Pi translation team, so we’d really appreciate any tips you have for helping us get started or for improving our existing system! Please share your thoughts in the comments below.
Learn how to code a sprinting minigame straight out of Daley Thompson’s Decathlon with Raspberry Pi’s own Rik Cross.
Spurred on by the success of Konami’s Hyper Sports, Daley Thompson’s Decathlon featured a wealth of controller-wrecking minigames.
Daley Thompson’s Decathlon
Released in 1984, Daley Thompson’s Decathlon was a memorable entry in what’s sometimes called the ‘joystick killer’ genre: players competed in sporting events that largely consisted of frantically waggling the controller or battering the keyboard. I’ll show you how to create a sprinting game mechanic in Python and Pygame.
Python sprinting game
There are variables in the Sprinter() class to keep track of the runner’s speed and distance, as well as global constant ACCELERATION and DECELERATION values to determine the player’s changing rate of speed. These numbers are small, as they represent the number of metres per frame that the player accelerates and decelerates.
The player increases the sprinter’s speed by alternately pressing the left and right arrow keys. This input is handled by the sprinter’s isNextKeyPressed() method, which returns True if the correct key (and only the correct key) is being pressed. A lastKeyPressed variable is used to ensure that keys are pressed alternately. The player also decelerates if no key is being pressed, and this rate of deceleration should be sufficiently smaller than the acceleration to allow the player to pick up enough speed.
Press the left and right arrow keys alternately to increase the sprinter’s speed. Objects move across the screen from right to left to give the illusion of sprinter movement.
For the animation, I used a free sprite called ‘The Boy’ from gameart2d.com, and made use of a single idle image and 15 run cycle images. The sprinter starts in the idle state, but switches to the run cycle whenever its speed is greater than 0. This is achieved by using index() to find the name of the current sprinter image in the runFrames list, and setting the current image to the next image in the list (and wrapping back to the first image once the end of the list is reached). We also need the sprinter to move through images in the run cycle at a speed proportional to the sprinter’s speed. This is achieved by keeping track of the number of frames the current image has been displayed for (in a variable called timeOnCurrentFrame).
To give the illusion of movement, I’ve added objects that move past the player: there’s a finish line and three markers to regularly show the distance travelled. These objects are calculated using the sprinter’s x position on the screen along with the distance travelled. However, this means that each object is at most only 100 pixels away from the player and therefore seems to move slowly. This can be fixed by using a SCALE factor, which is the relationship between metres travelled by the sprinter and pixels on the screen. This means that objects are initially drawn way off to the right of the screen but then travel to the left and move past the sprinter more quickly.
Finally, startTime and finishTime variables are used to calculate the race time. Both values are initially set to the current time at the start of the race, with finishTime being updated as long as the distance travelled is less than 100. Using the time module, the race time can simply be calculated by finishTime - startTime.
Here’s Rik’s code, which gets a sprinting game running in Python (no pun intended). To get it working on your system, you’ll first need to install Pygame Zero. Download the code here.
Get your copy of Wireframe issue 23
You can read more features like this one in Wireframe issue 23, available now at Tesco, WHSmith, all good independent UK newsagents, and the Raspberry Pi Store, Cambridge.
Or you can buy Wireframe directly from Raspberry Pi Press — delivery is available worldwide. And if you’d like a handy digital version of the magazine, you can download issue 23 for free in PDF format.
Autonauts is coming to colonise your computers with cuteness. We find out more in Wireframe issue 23.
Make sure to follow Wireframe on Twitter and Facebook for updates and exclusive offers and giveaways. Subscribe on the Wireframe website to save up to 49% compared to newsstand pricing!
Weave through a randomly generated landscape in Mark Vanstone’s homage to the classic arcade game Scramble.
Scramble was developed by Konami and released in arcades in 1981. Players avoid terrain and blast enemy craft.
In the early eighties, arcades and sports halls rang with the sound of a multitude of video games. Because home computers hadn’t yet made it into most households, the only option for the avid video gamer was to go down to their local entertainment establishment and feed the machines with ten pence pieces (which were bigger then). One of these pocket money–hungry machines was Konami’s Scramble — released in 1981, it was one of the earliest side-scrolling shooters with multiple levels.
The Scramble player’s jet aircraft flies across a randomly generated landscape (which sometimes narrows to a cave system), avoiding obstacles and enemy planes, bombing targets on the ground, and trying not to crash. As the game continues, the difficulty increases. The player aircraft can only fly forward, so once a target has been passed, there’s no turning back for a second go.
Code your own scrolling landscape
In this example code, I’ll show you a way to generate a Scramble-style scrolling landscape using Pygame Zero and a couple of additional Pygame functions. On early computers, moving a lot of data around the screen was very slow — until dedicated video hardware like the blitter chip arrived. Scrolling, however, could be achieved either by a quick shuffle of bytes to the left or right in the video memory, or in some cases, by changing the start address of the video memory, which was even quicker.
Avoid the roof and the floor with the arrow keys. Jet graphic courtesy of TheSource4Life at opengameart.org.
For our scrolling, we can use a Pygame surface the same size as the screen. To get the scrolling effect, we just call the scroll() function on the surface to shift everything left by one pixel and then draw a new pixel-wide slice of the terrain. The terrain could just be a single colour, but I’ve included a bit of maths-based RGB tinkering to make it more colourful. We can draw our terrain surface over a background image, as the SRCALPHA flag is set when we create the surface. This is also useful for detecting if the jet has hit the terrain. We can test the pixel from the surface in front of the jet: if it’s not transparent, kaboom!
The jet itself is a Pygame Zero Actor and can be moved up and down with the arrow keys. The left and right arrows increase and decrease the speed. We generate the landscape in the updateLand() and drawLand() functions, where updateLand() first decides whether the landscape is inclining or declining (and the same with the roof), making sure that the roof and floor don’t get too close, and then it scrolls everything left.
Each scroll action moves everything on the terrain surface to the left by one pixel.
The drawLand() function then draws pixels at the right-hand edge of the surface from y coordinates 0 to 600, drawing a thin sliver of roof, open space, and floor. The speed of the jet determines how many times the landscape is updated in each draw cycle, so at faster speeds, many lines of pixels are added to the right-hand side before the display updates.
The use of randint() can be changed to create a more or less jagged landscape, and the gap between roof and floor could also be adjusted for more difficulty. The original game also had enemy aircraft, which you could make with Actors, and fuel tanks on the ground, which could be created on the right-hand side as the terrain comes into view and then moved as the surface scrolls. Scramble sparked a wave of horizontal shooters, from both Konami and rival companies; this short piece of code could give you the basis for making a decent Scramble clone of your own:
You can read more features like this one in Wireframe issue 22, available now at Tesco, WHSmith, and all good independent UK newsagents, and the Raspberry Pi Store, Cambridge.
Or you can buy Wireframe directly from Raspberry Pi Press — delivery is available worldwide. And if you’d like a handy digital version of the magazine, you can also download issue 22 for free in PDF format.
Make sure to follow Wireframe on Twitter and Facebook for updates and exclusive offers and giveaways. Subscribe on the Wireframe website to save up to 49% compared to newsstand pricing!
Making player and computer-controlled cars race round a track isn’t as hard as it sounds. Mark Vanstone explains all.
The original Super Sprint arcade machine had three steering wheels and three accelerator pedals.
From Gran Trak 10 to Super Sprint
Decades before the advent of more realistic racing games such as Sega Rally or Gran Turismo, Atari produced a string of popular arcade racers, beginning with Gran Trak 10 in 1974 and gradually updated via the Sprint series, which appeared regularly through the seventies and eighties. By 1986, Atari’s Super Sprint allowed three players to compete at once, avoiding obstacles and collecting bonuses as they careened around the tracks.
The original arcade machine was controlled with steering wheels and accelerator pedals, and computer-controlled cars added to the racing challenge. Tracks were of varying complexity, with some featuring flyover sections and shortcuts, while oil slicks and tornadoes posed obstacles to avoid. If a competitor crashed really badly, a new car would be airlifted in by helicopter.
Code your own Super Sprint
So how can we make our own Super Sprint-style racing game with Pygame Zero? To keep this example code short and simple, I’ve created a simple track with a few bends. In the original game, the movement of the computer-controlled cars would have followed a set of coordinates round the track, but as computers have much more memory now, I have used a bitmap guide for the cars to follow. This method produces a much less predictable movement for the cars as they turn right and left based on the shade of the track on the guide.
Four Formula One cars race around the track. Collisions between other cars and the sides of the track are detected.
With Pygame Zero, we can write quite a short piece of code to deal with both the player car and the automated ones, but to read pixels from a position on a bitmap, we need to borrow a couple of objects directly from Pygame: we import the Pygame image and Color objects and then load our guide bitmaps. One is for the player to restrict movement to the track, and the other is for guiding the computer-controlled cars around the track.
Three bitmaps are used for the track. One’s visible, and the other two are guides for the cars.
The cars are Pygame Zero Actors, and are drawn after the main track image in the draw() function. Then all the good stuff happens in the update() function. The player’s car is controlled with the up and down arrows for speed, and the left and right arrows to change the direction of movement. We then check to see if any cars have collided with each other. If a crash has happened, we change the direction of the car and make it reverse a bit. We then test the colour of the pixel where the car is trying to move to. If the colour is black or red (the boundaries), the car turns away from the boundary.
The car steering is based on the shade of a pixel’s colour read from the guide bitmap. If it’s light, the car will turn right, if it’s dark, the car will turn left, and if it’s mid-grey, the car continues straight ahead. We could make the cars stick more closely to the centre by making them react quickly, or make them more random by adjusting the steering angle more slowly. A happy medium would be to get the cars mostly sticking to the track but being random enough to make them tricky to overtake.
Our code will need a lot of extra elements to mimic Atari’s original game, but this short snippet shows how easily you can get a top-down racing game working in Pygame Zero:
You can read more features like this one in Wireframe issue 21, available now at Tesco, WHSmith, and all good independent UK newsagents.
Or you can buy Wireframe directly from Raspberry Pi Press — delivery is available worldwide. And if you’d like a handy digital version of the magazine, you can also download issue 21 for free in PDF format.
Make sure to follow Wireframe on Twitter and Facebook for updates and exclusive offers and giveaways. Subscribe on the Wireframe website to save up to 49% compared to news stand pricing!
Raspberry Pi’s own Rik Cross shows you how to hit enemies with your mouse pointer as they move around the screen.
Duck Hunt made effective use of the NES Zapper, and made a star of its sniggering dog, who’d pop up to heckle you between stages.
Clicky Clicky Bang Bang
Shooting galleries have always been a part of gaming, from the Seeburg Ray-O-Lite in the 1930s to the light gun video games of the past 40 years. Nintendo’s Duck Hunt — played with the NES Zapper — was a popular console shooting game in the early eighties, while titles such as Time Crisis and The House of the Dead kept the genre alive in the 1990s and 2000s.
Here, I’ll show you how to use a mouse to fire bullets at moving targets. Code written to instead make use of a light gun and a CRT TV (as with Duck Hunt) would look very different. In these games, pressing the light gun’s trigger would cause the entire screen to go black and an enemy sprite to become bright white. A light sensor at the end of the gun would then check whether the gun is pointed at the white sprite, and if so, would register a hit. If more than one enemy was on the screen when the trigger was pressed, each enemy would flash white for one frame in turn, so that the gun would know which enemy had been hit.
Our simple shooting gallery in Python. You could try adding randomly spawning ducks, a scoreboard, and more.
I’ve used two Pygame Zero event hooks for dealing with mouse input. Firstly, the on_mouse_move() function updates the position of the crosshair sprite whenever the mouse is moved. The on_mouse_down() function reacts to mouse button presses, with the left button being pressed to fire a bullet (if numberofbullets > 0) and the right button to reload (setting numberofbullets to MAXBULLETS).
Each time a bullet is fired, a check is made to see whether any enemy sprites are colliding with the crosshair — a collision means that an enemy has been hit. Luckily, Pygame Zero has a colliderect() function to tell us whether the rectangular boundary around two sprites intersects.
If this helper function wasn’t available, we’d instead need to use sprites’ x and y coordinates, along with width and height data (w and h below) to check whether the two sprites intersect both horizontally and vertically. This is achieved by coding the following algorithm:
Is the left-hand edge of sprite 1 further left than the right-hand edge of sprite 2 (x1 < x2+w2)?
Is the right-hand edge of sprite 1 further right than the left-hand edge of sprite 2 (x1+w1 > x2)?
Is the top edge of sprite 1 higher up than the bottom edge of sprite 2 (y1 < y2+h2)?
Is the bottom edge of sprite 1 lower down than the top edge of sprite 2 (y1+h1 > y2)?
If the answer to the four questions above is True, then the two sprites intersect (see Figure 1). To give visual feedback, hit enemies briefly remain on the screen (in this case, 50 frames). This is achieved by setting a hit variable to True, and then decrementing a timer once this variable has been set. The enemy’s deleted when the timer reaches 0.
Figure 1: A visual representation of a collision algorithm, which checks whether two sprites intersect.
As well as showing an enemy for a short time after being hit, successful shots are also shown. A problem that needs to be overcome is how to modify an enemy sprite to show bullet holes. A hits list for each enemy stores bullet sprites, which are then drawn over enemy sprites.
Storing hits against an enemy allows us to easily stop drawing these hits once the enemy is removed. In the example code, an enemy stops moving once it has been hit.
If you don’t want this behaviour, then you’ll also need to update the position of the bullets in an enemy’s hits list to match the enemy movement pattern.
When decrementing the number of bullets, the max() function is used to ensure that the bullet count never falls below 0. The max() function returns the highest of the numbers passed to it, and as the maximum of 0 and any negative number is 0, the number of bullets always stays within range.
There are a couple of ways in which the example code could be improved. Currently, a hit is registered when the crosshair intersects with an enemy — even if they are barely touching. This means that often part of the bullet is drawn outside of the enemy sprite boundary. This could be solved by creating a clipping mask around an enemy before drawing a bullet. More visual feedback could also be given by drawing missed shots, stored in a separate list.
Raspberry Pi’s Rik Cross shows you how to create game states, and rules for moving between them.
Ninja Gaiden’s dramatic continue screen. Who would be cruel enough to walk away?
The continue screen, while much less common now, was a staple feature of arcade games, providing an opportunity (for a small fee) to reanimate the game’s hero and to pick up where they left off.
Games such as Tecmo’s Ninja Gaiden coin-op (known in some regions as Shadow Warriors) added jeopardy to their continue screen, in an effort to convince us to part with our money.
Often, a continue screen is one of many screens that a player may find themselves on; other possibilities being a title screen or an instruction screen. I’ll show you how you can add multiple screens to a game in a structured way, avoiding a tangle of if…else statements and variables.
A simple way of addressing this problem is to create separate update and draw functions for each of these screens, and then switch between these functions as required. Functions are ‘first-class citizens’ of the Python language, which means that they can be stored and manipulated just like any other object, such as numbers, text, and class instances. They can be stored in variables and other data types such as lists and dictionaries, and passed as parameters to (or returned from) other functions.
SNK’s Fantasy, released in 1981, was the first arcade game to feature a continue screen.
We can take advantage of the first-class nature of Python functions by storing the functions for the current screen in variables, and then calling them in the main update() and draw() functions. In the following example, notice the difference between storing a function in a variable (by using the function name without parentheses) and calling the function (by including parentheses).
[Ed. comment: We have to use an image here because WordPress doesn’t seem to allow code indentation. We know that’s annoying because you can’t copy and paste the code, so if you know a better solution, please leave us a comment.]
The example code above calls currentupdatefunction() and currentdrawfunction(), which each store a reference to separate update() and draw() functions for the continue screen. These continue screen functions could then also include logic for changing which function is called, by updating the function reference stored in currentupdatefunction and currentdrawfunction.
This way of structuring code can be taken a step further by making use of state machines. In a state machine, a system can be in one of a (finite) number of predefined states, and rules determine the conditions under which a system can transition from one state into another.
Rules define conditions that need to be satisfied in order to move between states.
A state machine (in this case a very simplified version) can be implemented by first creating a core State() class. Each game state has its own update() and draw() methods, and a rules dictionary containing state:rule pairs – references to other state objects linked to functions for testing game conditions. As an example, the continuescreen state has two rules:
Transition to the gamescreen state if the SPACE key is pressed;
Transition to the titlescreen state if the frame timer reaches 10.
This is pulled together with a StateMachine() class, which keeps track of the current state. The state machine calls the update() and draw() methods for the current state, and checks the rules for transitioning between states. Each rule in the current state’s rules list is executed, with the state machine updating the reference to its current state if the rule function returns True. I’ve also added a frame counter that is incremented by the state machine’s update() function each time it is run. While not a necessary part of the state machine, it does allow the continue screen to count down from 10, and could have a number of other uses, such as for animating sprites.
Something else to point out is the use of lambda functions when adding rules to states. Lambda functions are small, single-expression anonymous functions that return the result of evaluating its expression when called. Lambda functions have been used in this example simply to make the code a little more concise, as there’s no benefit to naming the functions passed to addrule().
State machines have lots of other potential uses, including the modelling of player states. It’s also possible to extend the state machine in this example by adding onenter() and onexit() functions that can be called when transitioning between states.
You too can recreate the techniques behind a pioneering 3D maze game in Python. Mark Vanstone explains how.
3D Monster Maze, released in 1982 by J.K. Greye software, written by Malcolm Evans.
3D Monster Maze
While 3D games have become more and more realistic, some may forget that 3D games on home computers started in the mists of time on machines like the Sinclair ZX81. One such pioneering game took pride of place in my collection of tapes, took many minutes to load, and required the 16K RAM pack expansion. That game was 3D Monster Maze — perhaps the most popular game released for the ZX81.
The game was released in 1982 by J.K. Greye Software, and written by Malcolm Evans. Although the graphics were incredibly low resolution by today’s standards, it became an instant hit. The idea of the game was to navigate around a randomly generated maze in search of the exit.
The problem was that a Tyrannosaurus rex also inhabited the maze, and would chase you down and have you for dinner if you didn’t escape quickly enough. The maze itself was made of straight corridors on a 16×18 grid, which the player would move around from one block to the next. The shape of the blocks were displayed by using the low-resolution pixels included in the ZX81’s character set, with 2×2 pixels per character on the screen.
The original ZX81 game drew its maze from chunky 2×2 pixel blocks.
Draw imaginary lines
There’s an interesting trick to recreating the original game’s 3D corridor display which, although quite limited, works well for a simplistic rendering of a maze. To do this, we need to draw imaginary lines diagonally from corner to corner in a square viewport: these are our vanishing point perspective guides. Then each corridor block in our view is half the width and half the height of the block nearer to us.
If we draw this out with lines showing the block positions, we get a view that looks like we’re looking down a long corridor with branches leading off left and right. In our Pygame Zero version of the maze, we’re going to use this wireframe as the basis for drawing our block elements. We’ll create graphics for blocks that are near the player, one block away, two, three, and four blocks away. We’ll need to view the blocks from the left-hand side, the right-hand side, and the centre.
The maze display is made by drawing diagonal lines to a central vanishing point.
Once we’ve created our block graphics, we’ll need to make some data to represent the layout of the maze. In this example, the maze is built from a 10×10 list of zeros and ones. We’ll set a starting position for the player and the direction they’re facing (0–3), then we’re all set to render a view of the maze from our player’s perspective.
The display is created from furthest away to nearest, so we look four blocks away from the player (in the direction they’re looking) and draw a block if there’s one indicated by the maze data to the left; we do the same on the right, and finally in the middle. Then we move towards the player by a block and repeat the process (with larger graphics) until we get to the block the player is on.
Each visible block is drawn from the back forward to make the player’s view of the corridors.
That’s all there is to it. To move backwards and forwards, just change the position in the grid the player’s standing on and redraw the display. To turn, change the direction the player’s looking and redraw. This technique’s obviously a little limited, and will only work with corridors viewed at 90-degree angles, but it launched a whole genre of games on home computers. It really was a big deal for many twelve-year-olds — as I was at the time — and laid the path for the vibrant, fast-moving 3D games we enjoy today.
In Hello World issue 9, Raspberry Pi’s own Mac Bowley shares a lesson that introduces students to databases using Python and text files.
In this lesson, students create a library app for their books. This will store information about their book collection and allow them to display, manipulate, and search their collection. You will show students how to use text files in their programs that act as a database.
The project will give your students practical examples of database terminology and hands-on experience working with persistent data. It gives opportunities for students to define and gain concrete experience with key database concepts using a language they are familiar with. The script that accompanies this activity can be adapted to suit your students’ experience and competency.
This ready-to-go software project can be used alongside approaches such as PRIMM or pair programming, or as a worked example to engage your students in programming with persistent data.
What makes a database?
Start by asking the students why we need databases and what they are: do they ever feel unorganised? Life can get complicated, and there is so much to keep track of, the raw data required can be overwhelming. How can we use computing to solve this problem? If only there was a way of organising and accessing data that would let us get it out of our head. Databases are a way of organising the data we care about, so that we can easily access it and use it to make our lives easier.
Then explain that in this lesson the students will create a database, using Python and a text file. The example I show students is a personal library app that keeps track of which books I own and where I keep them. I have also run this lesson and allowed the students pick their own items to keep track of — it just involves a little more planning time at the end. Split the class up into pairs; have each of them discuss and select five pieces of data about a book (or their own item) they would like to track in a database. They should also consider which type of data each of them is. Give them five minutes to discuss and select some data to track.
Databases are organised collections of data, and this allows them to be displayed, maintained, and searched easily. Our database will have one table — effectively just like a spreadsheet table. The headings on each of the columns are the fields: the individual pieces of data we want to store about the books in our collection. The information about a single book are called its attributes and are stored together in one record, which would be a single row in our database table. To make it easier to search and sort our database, we should also select a primary key: one field that will be unique for each book. Sometimes one of the fields we are already storing works for this purpose; if not, then the database will create an ID number that it uses to uniquely identify each record.
Create a library application
Pull the class back together and ask a few groups about the data they selected to track. Make sure they have chosen appropriate data types. Ask some if they can find any of the fields that would be a primary key; the answer will most likely be no. The ISBN could work, but for our simple application, having to type in a 10- or 13-digit number just to use for an ID would be overkill. In our database, we are going to generate our own IDs.
The requirements for our database are that it can do the following things: save data to a file, read data from that file, create new books, display our full database, allow the user to enter a search term, and display a list of relevant results based on that term. We can decompose the problem into the following steps:
Set up our structures
Create a record
Save the data to the database file
Read from the database file
Display the database to the user
Allow the user to search the database
Display the results
Have the class log in and power up Python. If they are doing this locally, have them create a new folder to hold this project. We will be interacting with external files and so having them in the same folder avoids confusion with file locations and paths. They should then load up a new Python file. To start, download the starter file from the link provided. Each student should make a copy of this file. At first, I have them examine the code, and then get them to run it. Using concepts from PRIMM, I get them to print certain messages when a menu option is selected. This can be a great exemplar for making a menu in any application they are developing. This will be the skeleton of our database app: giving them a starter file can help ease some cognitive load from students.
Have them examine the variables and make guesses about what they are used for.
current_ID – a variable to count up as we create records, this will be our primary key
new_additions – a list to hold any new records we make while our code is running, before we save them to the file
filename – the name of the database file we will be using
fields – a list of our fields, so that our dictionaries can be aligned with our text file
data – a list that will hold all of the data from the database, so that we can search and display it without having to read the file every time
Create the first record
We are going to use dictionaries to store our records. They reference their elements using keys instead of indices, which fit our database fields nicely. We are going to generate our own IDs. Each of these must be unique, so a variable is needed that we can add to as we make our records. This is a user-focused application, so let’s make it so our user can input the data for the first book. The strings, in quotes, on the left of the colon, are the keys (the names of our fields) and the data on the right is the stored value, in our case whatever the user inputs in response to our appropriate prompts. We finish this part of by adding the record to the file, incrementing the current ID, and then displaying a useful feedback message to the user to say their record has been created successfully. Your students should now save their code and run it to make sure there aren’t any syntax errors.
You could make use of pair programming, with carefully selected pairs taking it in turns in the driver and navigator roles. You could also offer differing levels of scaffolding: providing some of the code and asking them to modify it based on given requirements.
How to use the code in your class
To complete the project, your students can add functionality to save their data to a CSV file, read from a database file, and allow users to search the database. The code for the whole project is available at helloworld.cc/database.
An example of the code
You may want to give your students the entire piece of code. They can investigate and modify it to their own purpose. You can also lead them through it, having them follow you as you demonstrate how an expert constructs a piece of software. I have done both to great effect. Let me know how your classes get on! Get in touch at [email protected]
Hello World issue 9
The brand-new issue of Hello World is out today, and available right now as a free PDF download from the Hello World website.
Learn how to create your own obedient lemmings that follow any path put in front of them. Raspberry Pi’s own Rik Cross explains how.
The original Lemmings, first released for the Amiga, quickly spread like a virus to just about every computer and console of the day.
Lemmings is a puzzle-platformer, created at DMA Design, and first became available for the Amiga in 1991. The aim is to guide a number of small lemming sprites to safety, navigating traps and difficult terrain along the way. Left to their own devices, the lemmings will simply follow the path in front of them, but additional ‘special powers’ given to lemmings allow them to (among other things) dig, climb, build, and block in order to create a path to freedom (or to the next level, anyway).
Code your own lemmings
I’ll show you a simple way (using Python and Pygame) in which lemmings can be made to follow the terrain in front of them. The first step is to store the level’s terrain information, which I’ve achieved by using a two-dimensional list to store the colour of each pixel in the background ‘level’ image. In my example, I’ve used the ‘Lemcraft’ tileset by Matt Hackett (of Lost Decade Games) – taken from opengameart.org – and used the Tiled software to stitch the tiles together into a level.
The algorithm we then use can be summarised as follows: check the pixels immediately below a lemming. If the colour of those pixels isn’t the same as the background colour, then the lemming is falling. In this case, move the lemming down by one pixel on the y-axis. If the lemming isn’t falling, then it’s walking. In this case, we need to see whether there is a non-ground, background-coloured pixel in front of the lemming for it to move onto.
Sprites cling to the ground below them, navigating uneven terrain, and reversing direction when they hit an impassable obstacle.
If a pixel is found in front of the lemming (determined by its direction) that is low enough to get to (i.e. lower than its climbheight), then the lemming moves forward on the x-axis by one pixel, and upwards on the y-axis to the new ground level. However, if no suitable ground is found to move onto, then the lemming reverses its direction.
The algorithm is stored as a lemming’s update() method, which is executed for each lemming, each frame of the game. The sample level.png file can be edited, or swapped for another image altogether. If using a different image, just remember to update the level’s BACKGROUND_COLOUR in your code, stored as a (red, green, blue, alpha) tuple. You may also need to increase your lemming climbheight if you want them to be able to navigate a climb of more than four pixels.
There are other things you can do to make a full Lemmings clone. You could try replacing the yellow-rectangle lemmings in my example with pixel-art sprites with their own walk cycle animation (see my article in issue #14), or you could give your lemmings some of the special powers they’ll need to get to safety, achieved by creating flags that determine how lemmings interact with the terrain around them.
Learn how to create game objects that follow the path of the main player sprite. Raspberry Pi’s own Rik Cross explains all.
Options first appeared in 1985’s Gradius, but became a mainstay of numerous sequels and spin-offs, including the Salamander and Parodius series of games.
First released by Konami in 1985, Gradius pushed the boundaries of the shoot-’em-up genre with its varied level design, dramatic boss fights, and innovative power-up system.
One of the most memorable of its power-ups was the Option — a small, drone-like blob that followed the player’s ship and effectively doubled its firepower.
By collecting more power-ups, it was possible to gather a cluster of death-dealing Options, which obediently moved wherever the player moved.
Recreate sprite-following in Python
There are a few different ways of recreating Gradius’ sprite-following, but in this article, I’ll show you a simple implementation that uses the player’s ‘position history’ to place other following items on the screen. As always, I’ll be using Python and Pygame to recreate this effect, and I’ll be making use of a spaceship image created by ‘pitrizzo’ from opengameart.org.
The first thing to do is to create a spaceship and a list of ‘power-up’ objects. Storing the power-ups in a list allows us to perform a simple calculation on a power-up to determine its position, as you’ll see later. As we’ll be iterating through the power-ups stored in a list, there’s no need to create a separate variable for each. Instead, we can use list comprehension to create the power-ups:
powerups = [Actor(‘powerup’) for p in range(3)]
The player’s position history will be a list of previous positions, stored as a list of (x,y) tuples. Each time the player’s position changes, the new position is added to the front of the list (as the new first element). We only need to know the spaceship’s recent position history, so the list is also truncated to only contain the 100 most recent positions. Although not necessary, the following code can be added to allow you to see a selection (in this case every fifth) of these previous positions:
for p in previouspositions[::5]:
screen.draw.filled_circle(p, 2, (255,0,0))
Plotting the spaceship’s position history.
Each frame of the game, this position list is used to place each of the power-ups. In our Gradius-like example, we need each of these objects to follow the player’s spaceship in a line, as if moving together in a single-file queue. To achieve this effect, a power-up’s position is determined by its position in the power-ups list, with the first power-up in the list taking up a position nearest to the player. In Python, using enumerate when iterating through a list allows us to get the power-up’s position in the list, which can then be used to determine which position in the player’s position history to use.
newposition = previouspositions[(i+1)*20]
So, the first power-up in the list (element 0 in the list) is placed at the coordinates of the twentieth ((0+1)*20) position in the spaceship’s history, the second power-up at the fourtieth position, and so on. Using this simple calculation, elements are equally spaced along the spaceship’s previous path. The only thing to be careful of here is that you have enough items in the position history for the number of items you want to follow the player!
Power-ups following a player sprite, using the player’s position history.
This leaves one more question to answer; where do we place these power-ups initially, when the spaceship has no position history? There are a few different ways of solving this problem, but the simplest is just to generate a fictitious position history at the beginning of the game. As I want power-ups to be lined up behind the spaceship initially, I again used list comprehension
to generate a list of 100 positions with ever-decreasing x-coordinates.
previouspositions = [(spaceship.x - i*spaceship.speed,spaceship.y) for i in range(100)]
With an initial spaceship position of (400,400) and a spaceship.speed of 4, this means the list will initially contain the following coordinates:
Storing our player’s previous position history has allowed us to create path-following power-ups with very little code. The idea of storing an object’s history can have very powerful applications. For example, a paint program could store previous commands that have been executed, and include an ‘undo’ button that can work backwards through the commands.
Isometric graphics give 2D games the illusion of depth. Mark Vanstone explains how to make an isometric game map of your own.
Published by Quicksilva in 1983, Ant Attack was one of the earliest games to use isometric graphics. And you threw grenades at giant ants. It was brilliant.
Most early arcade games were 2D, but in 1982, a new dimension emerged: isometric projection. The first isometric game to hit arcades was Sega’s pseudo-3D shooter, Zaxxon. The eye-catching format soon caught on, and other isometric titles followed: Q*bert came out the same year, and in 1983 the first isometric game for home computers was published: Ant Attack, written by Sandy White.
Ant Attack was first released on the ZX Spectrum, and the aim of the game was for the player to find and rescue a hostage in a city infested with giant ants. The isometric map has since been used by countless titles, including Ultimate Play The Game’s classics Knight Lore and Alien 8, and my own educational history series ArcVenture.
Let’s look at how an isometric display is created, and code a simple example of how this can be done in Pygame Zero — so let’s start with the basics. The isometric view displays objects as if you’re looking down at 45 degrees onto them, so the top of a cube looks like a diamond shape. The scene is made by drawing cubes on a diagonal grid so that the cubes overlap and create solid-looking structures. Additional layers can be used above them to create the illusion of height.
Blocks are drawn from the back forward, one line at a time and then one layer on top of another until the whole map is drawn.
The cubes are actually two-dimensional bitmaps, which we start printing at the top of the display and move along a diagonal line, drawing cubes as we go. The map is defined by a three-dimensional list (or array). The list is the width of the map by the height of the map, and has as many layers as we want to represent in the upward direction. In our example, we’ll represent the floor as the value 0 and a block as value 1. We’ll make a border around the map and create some arches and pyramids, but you could use any method you like — such as a map editor — to create the map data.
To make things a bit easier on the processor, we only need to draw cubes that are visible in the window, so we can do a check of the coordinates before we draw each cube. Once we’ve looped over the x, y, and z axes of the data list, we should have a 3D map displayed. The whole map doesn’t fit in the window, and in a full game, the map is likely to be many times the size of the screen. To see more of the map, we can add some keyboard controls.
If we detect keyboard presses in the update() function, all we need to do to move the map is change the coordinates we start drawing the map from. If we start drawing further to the left, the right-hand side of the map emerges, and if we draw the map higher, the lower part of the map can be seen.
We now have a basic map made of cubes that we can move around the window. If we want to make this into a game, we can expand the way the data represents the display. We could add differently shaped blocks represented by different numbers in the data, and we could include a player block which gets drawn in the draw() function and can be moved around the map. We could also have some enemies moving around — and before we know it, we’ll have a game a bit like Ant Attack.
When writing games with large isometric maps, an editor will come in handy. You can write your own, but there are several out there that you can use. One very good one is called Tiled and can be downloaded free from mapeditor.org. Tiled allows you to define your own tilesets and export the data in various formats, including JSON, which can be easily read into Python.
Get your copy of Wireframe issue 15
You can read more features like this one in Wireframe issue 15, available now at Tesco, WHSmith, and all good independent UK newsagents.
Effective animation gave Donkey Kong barrels of personality. Raspberry Pi’s own Rik Cross explains how to create a similar walk cycle.
Donkey Kong wasn’t the first game to feature an animated character who could walk and jump, but on its release in 1981, it certainly had more personality than the games that came before it. You only have to compare Donkey Kong to another Nintendo arcade game that came out just two years earlier — the half-forgotten top-down shooter Sheriff — to see how quickly both technology and pixel art moved on in that brief period. Although simple by modern standards, Donkey Kong’s hero Jumpman (later known as Mario) packed movement and personality into just a few frames of animation.
In this article, I’ll show you how to use Python and Pygame to create a character with a simple walk cycle animation like Jumpman’s in Donkey Kong. The code can, however, be adapted for any game object that requires animation, and even for multiple game object animations, as I’ll explain later.
Jumpman’s (aka Mario’s) walk cycle comprised just three frames of animation.
Firstly, we’ll need some images to animate. As this article is focused on the animation code and not the theory behind creating walk cycle images, I grabbed some suitable images created by Kenney Vleugels and available at opengameart.org.
Let’s start by animating the player with a simple walk cycle. The two images to be used in the animation are stored in an images list, and an animationindex variable keeps track of the index of the current image in the list to display. So, for a very simple animation with just two different frames, the images list will contain two different images:
images = [‘walkleft1’,‘walkleft2’
To achieve a looping animation, the animationindex is repeatedly incremented, and is reset to 0 once the end of the images list is reached. Displaying the current image can then be achieved by using the animationindex to reference and draw the appropriate image in the animation cycle:
A list of images along with an index is used to loop through an animation cycle.
The problem with the code described so far is that the animationindex is incremented once per frame, and so the walk cycle will happen way too quickly, and won’t look natural. To solve this problem, we need to tell the player to update its animation every few frames, rather than every frame. To achieve this, we need another couple of variables; I’ll use animationdelay to store the number of frames to skip between displayed images, and animationtimer to store the number of frames since the last image change.
Therefore, the code needed to animate the player becomes:
So we have a player that appears to be walking, but now the problem is that the player walks constantly, and always in the same direction! The rest of this article will show you how to solve these two related problems.
There are a few different ways to approach this problem, but the method I’ll use is to make use of game object states, and then have different animations for each state. This method is a little more complicated, but it’s very adaptable.
The first thing to do is to decide on what the player’s ‘states’ might be — stand, walkleft, and walkright will do as a start. Just as we did with our previous single animation, we can now define a list of images for each of the possible player’s states. Again, there are lots of ways of structuring this data, but I’ve opted for a Python dictionary linking states and image lists:
The correct player state can then be set by getting the keyboard input, setting the player to walkleft if the left arrow key is pressed or walkright if the right arrow key is pressed. If neither key is pressed, the player can be set to a stand state; the image list for which contains a single image of the player facing the camera.
Animation cycles can be linked to player ‘states’.
For simplicity, a maximum of two images are used for each animation cycle; adding more images would create a smoother or more realistic animation.
Using the code above, it would also be possible to easily add additional states for, say, jumping or fighting enemies. You could even take things further by defining an Animation() object for each player state. This way, you could specify the speed and other properties (such as whether or not to loop) for each animation separately, giving you greater flexibility.
Netflix brings delightful customer experiences to homes on a variety of devices that continues to grow each day. The device ecosystem is rich with partners ranging from Silicon-on-Chip (SoC) manufacturers, Original Design Manufacturer (ODM) and Original Equipment Manufacturer (OEM) vendors.
Partners across the globe leverage Netflix device certification process on a continual basis to ensure that quality products and experiences are delivered to their customers. The certification process involves the verification of partner’s implementation of features provided by the Netflix SDK.
The Partner Device Ecosystem organization in Netflix is responsible for ensuring successful integration and testing of the Netflix application on all partner devices. Netflix engineers run a series of tests and benchmarks to validate the device across multiple dimensions including compatibility of the device with the Netflix SDK, device performance, audio-video playback quality, license handling, encryption and security. All this leads to a plethora of test cases, most of them automated, that need to be executed to validate the functionality of a device running Netflix.
With a collection of tests that, by nature, are time consuming to run and sometimes require manual intervention, we need to prioritize and schedule test executions in a way that will expedite detection of test failures. There are several problems efficient test scheduling could help us solve:
Quickly detect a regression in the integration of the Netflix SDK on a consumer electronic or MVPD (multichannel video programming distributor) device.
Detect a regression in a test case. Using the Netflix Reference Application and known good devices, ensure the test case continues to function and tests what is expected.
When code many test cases are dependent on has changed, choose the right test cases among thousands of affected tests to quickly validate the change before committing it and running extensive, and expensive, tests.
Choose the most promising subset of tests out of thousands of test cases available when running continuous integration against a device.
Recommend a set of test cases to execute against the device that would increase the probability of failing the device in real-time.
Solving the above problems could help Netflix and our Partners save time and money during the entire lifecycle of device design, build, test, and certification.
These problems could be solved in several different ways. In our quest to be objective, scientific, and inline with the Netflix philosophy of using data to drive solutions for intriguing problems, we proceeded by leveraging machine learning.
In the case of continuously testing a Netflix SDK integration on a new device, we usually lack relevant data for model training in the early phases of integration. In this situation training an agent is a great fit as it allows us to start with very little input data and let the agent explore and exploit the patterns it learns in the process of SDK integration and regression testing. The agent in reinforcement learning is an entity that performs a decision on what action to take considering the current state of the environment, and gets a reward based on the quality of the action.
We built a system called Lerner that consists of a set of microservices and a python library that allows scalable agent training and inference for test case scheduling. We also provide an API client in Python.
Lerner works in tandem with our continuous integration framework that executes on-device tests using the Netflix Test Studio platform. Tests are run on Netflix Reference Applications (running as containers on Titus), as well as on physical devices.
There were several motivations that led to building a custom solution:
We wanted to keep the APIs and integrations as simple as possible.
We needed a way to run agents and tie the runs to the internal infrastructure for analytics, reporting, and visualizations.
We wanted the to tool be available as a standalone library as well as scalable API service.
Lerner provides ability to setup any number of agents making it the first component in our re-usable reinforcement learning framework for device certification.
Lerner, as a web-service, relies on Amazon Web Services (AWS) and Netflix’s Open Source Software (OSS) tools. We use Spinnaker to deploy instances and host the API containers on Titus — which allows fast deployment times and rapid scalability. Lerner uses AWS services to store binary versions of the agents, agent configurations, and training data. To maintain the quality of Lerner APIs, we are using the server-less paradigm for Lerner’s own integration testing by utilizing AWS Lambda.
The agent training library is written in Python and supports versions 2.7, 3.5, 3.6, and 3.7. The library is available in the artifactory repository for easy installation. It can be used in Python notebooks — allowing for rapid experimentation in isolated environments without a need to perform API calls. The agent training library exposes different types of learning agents that utilize neural networks to approximate action.
The neural network (NN)-based agent uses a deep net with fully connected layers. The NN gets the state of a particular test case (the input) and outputs a continuous value, where a higher number means an earlier position in a test execution schedule. The inputs to the neural network include: general historical features such as the last N executions and several domain specific features that provide meta-information about a test case.
The Lerner APIs are split into three areas:
Storing execution results.
Getting recommendations based on the current state of the environment.
Assign reward to the agent based on the execution result and predicted recommendations.
A process of getting recommendations and rewarding the agent using APIs consists of 4 steps:
Out of all available test cases for a particular job — form a request that can be interpreted by Lerner. This involves aggregation of historical results and additional features.
Lerner returns a recommendation identified with a unique episode id.
A CI system can execute the recommendation and submit the execution results to Lerner based on the episode id.
Call an API to assign a reward based on the agent id and episode id.
Below is a diagram of the services and persistence layers that support the functionality of the Lerner API.
The self-service nature of the tool makes it easy for service owners to integrate with Lerner, create agents, ask agents for recommendations and reward them after execution results are available.
The metrics relevant to the training and recommendation process are reported to Atlas and visualized using Netflix’s Lumen. Users of the service can track the statistics specific to the agents they setup and deploy, which allows them to build their own dashboards.
We have identified some interesting patterns while doing online reinforcement learning.
The recommendation/execution reward cycle can happen without any prior training data.
We can bootstrap several CI jobs that would use agents with different reward functions, and gain additional insight based on agents performance. It could help us design and implement more targeted reward functions.
We can keep a small amount of historical data to train agents. The data can be truncated after each execution and offloaded to a long-term storage for further analysis.
Some of the downsides:
It might take time for an agent to stop exploring and start exploiting the accumulated experience.
As agents stored in a binary format in the database, an update of an agent from multiple jobs could cause a race condition in its state. Handling concurrency in the training process is cumbersome and requires trade offs. We achieved the desired state by relying on the locking mechanisms of the underlying persistence layer that stores and serves agent binaries.
Thus, we have the luxury of training as many agents as we want that could prioritize and recommend test cases based on their unique learning experiences.
We are currently piloting the system and have live agents serving predictions for various CI runs. At the moment we run Lerner-based CIs in parallel with CIs that either execute test cases in random order or use simple heuristics as sorting test cases by time and execute everything that previously failed.
The system was built with simplicity and performance in mind, so the set of APIs are minimal. We developed client libraries that allow seamless, but opinionated, integration with Lerner.
We collect several metrics to evaluate the performance of a recommendation, with main metrics being time taken to first failure and time taken to complete a whole scheduled run.
Lerner-based recommendations are proving to be different and more insightful than random runs, as they allow us to fit a particular time budget and detect patterns such as cases that tend to fail together in a cluster, cases that haven’t been run in a long time, and so on.
The below graphs shows more or less an artificial case when a schedule of 100+ test cases would contain several flaky tests. The Y-axis represents how many minutes it took to complete the schedule or reach a first failed test case. In blue, we have random recommendations with no time budget constraints. In green you can see executions based on Lerner recommendations under a time constraint of 60 minutes. The green spikes represent Lerner exploring the environment, where the wiggly lines around 0 are the executions that failed quickly as Lerner was exploiting its policy.
The next phases of the project will focus on:
Reward functions that are aware of a comprehensive domain context, such as assigning appropriate rewards to states where infrastructure is fragile and test case could not be run appropriately.
Administrative user-interface to manage agents.
More generic, simple, and user-friendly framework for reinforcement learning and agent deployment.
Using Lerner on all available CIs jobs against all SDK versions.
Experiment with different neural network architectures.
If you would like to be a part of our team, come join us.
Unparalleled depth in a 2D game: PyGame Zero extraordinaire Daniel Pope shows you how to recreate a zooming starfield effect straight out of the eighties arcade classic Gyruss.
The crowded, noisy realm of eighties amusement arcades presented something of a challenge for developers of the time: how can you make your game stand out from all the other ones surrounding it? Gyruss, released by Konami in 1983, came up with one solution. Although it was yet another alien blaster — one of a slew of similar shooters that arrived in the wake of Space Invaders, released in 1978 — it differed in one important respect: its zooming starfield created the illusion that the player’s craft was hurtling through space, and that aliens were emerging from the abyss to attack it.
This made Gyruss an entry in the ‘tube shooter’ genre — one that was first defined by Atari’s classic Tempest in 1981. But where Tempest used a vector display to create a 3D environment where enemies clambered up a series of tunnels, Gyruss used more common hardware and conventional sprites to render its aliens on the screen. Gyruss was designed by Yoshiki Okamoto (who would later go on to produce the hit Street Fighter II, among other games, at Capcom), and was born from his affection for Galaga, a 2D shoot-’em-up created by Namco.
Under the surface, Gyruss is still a 2D game like Galaga, but the cunning use of sprite animation and that zooming star effect created a sense of dynamism that its rivals lacked. The tubular design also meant that the player could move in a circle around the edge of the play area, rather than moving left and right at the bottom of the screen, as in Galaga and other fixed-screen shooters like it. Gyruss was one of the most popular arcade games of its period, probably in part because of its attention-grabbing design.
The code sample above, written by Daniel Pope, shows you how a zooming star field can work in PyGame Zero — and how, thanks to modern hardware, we can heighten the sense of movement in a way that Konami’s engineers couldn’t have hoped to achieve about 30 years ago. The code generates a cluster of stars on the screen, and creates the illusion of depth and movement by redrawing them in a new position in a randomly chosen direction each frame.
At the same time, the stars gradually increase their brightness over time, as if they’re getting closer. As a modern twist, Pope has also added an extra warp factor: holding down the Space bar increases the stars’ velocity, making that zoom into space even more exhilarating.
Get your copy of Wireframe issue 13
You can read the rest of the feature in Wireframe issue 13, available now at Tesco, WHSmith, and all good independent UK newsagents.
Note from May 10, 2019: We’ve updated a code sample for accuracy.
Today, AWS Secrets Manager introduced a client-side caching library for Python that improves the availability and latency of accessing and distributing credentials to your applications. It can also help you reduce the cost associated with retrieving secrets. In this post, I’ll walk you through the following topics:
An overview of the Secrets Manager client-side caching library for Python
How to use the Python client-side caching library to retrieve a secret
Here are the key benefits of client-side caching libraries:
Improved availability: You can cache secrets to reduce the impact of network availability issues such as increased response times and temporary loss of network connectivity.
Reduced cost: Retrieving secrets from the cache can reduce the number of API requests made to and billed by Secrets Manager.
Automatic refresh of secrets: The library updates the cache by calling Secrets Manager periodically, ensuring your applications use the most current secret value. This ensures any regularly rotated secrets are automatically retrieved.
Implementation in just two steps: Add the Python library dependency to your application, and then provide the identifier of the secret that you want the library to use.
Using the Secrets Manager client-side caching library for Python
First, I’ll walk you through an example in which I retrieve a secret without using the Python cache. Then I’ll show you how to update your code to use the Python client-side caching library.
Retrieving a secret without using a cache
Using the AWS SDK for Python (Boto3), you can retrieve a secret from Secrets Manager using the API call flow, as shown below.
Figure 1: Diagram showing GetSecretValue API call without the Python cache
The code below demonstrates a GetSecretValue API call to AWS Secrets Manager without using the cache feature. Each time the application makes a call, the AWS Secrets Manager GetSecretValue API will also be called. This increases the secret retrieval latency. Additionally, there is a minor cost associated with an API call made to the AWS Secrets Manager API endpoint.
from botocore.exceptions import ClientError
secret_name = "python-cache-test"
region_name = "us-west-2"
# Create a Secrets Manager client
session = boto3.session.Session()
client = session.client(
# In this sample we only handle the specific exceptions for the 'GetSecretValue' API.
# See https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html
# We rethrow the exception by default.
get_secret_value_response = client.get_secret_value(
except ClientError as e:
if e.response['Error']['Code'] == 'DecryptionFailureException':
# Secrets Manager can't decrypt the protected secret text using the provided KMS key.
# Deal with the exception here, and/or rethrow at your discretion.
# Decrypts secret using the associated KMS CMK.
# Depending on whether the secret is a string or binary, one of these fields will be populated.
if 'SecretString' in get_secret_value_response:
secret = get_secret_value_response['SecretString']
decoded_binary_secret = base64.b64decode(get_secret_value_response['SecretBinary'])
# Your code goes here.
Using the Python client-side caching library to retrieve a secret
Using the Python cache feature, you can now use the cache library to reduce calls to the AWS Secrets Manager API, improving the availability and latency of your application. As shown in the diagram below, when you implement the Python cache, the call to retrieve the secret is routed to the local cache before reaching the AWS Secrets Manager API. If the secret exists in the cache, the application retrieves the secret from the client-side cache. If the secret does not exist in the client-side cache, the request is routed to the AWS Secrets Manager endpoint to retrieve the secret.
Figure 2: Diagram showing GetSecretValue API call using Python client-side cache
In the example below, I’ll implement a Python cache to retrieve the secret from a local cache, and hence avoid calling the AWS Secrets Manager API:
from aws_secretsmanager_caching import SecretCache, SecretCacheConfig
from botocore.exceptions import ClientError
secret_name = "python-cache-test"
region_name = "us-west-2"
# Create a Secrets Manager client
session = boto3.session.Session()
client = session.client(
# Create a cache
cache = SecretCache(SecretCacheConfig(),client)
# Get secret string from the cache
get_secret_value_response = cache.get_secret_string(secret_name)
except ClientError as e:
if e.response['Error']['Code'] == 'DecryptionFailureException':
# Deal with the exception here, and/or rethrow at your discretion.
secret = get_secret_value_response
# Your code goes here.
The cache allows advanced configuration using the SecretCacheConfig library. This library allows you to define cache configuration parameters to help meet your application security, performance, and cost requirements. The SDK enforces the configuration thresholds on maximum cache size, default secret version stage to request, and secret refresh interval between requests. It also allows configuration of various exception thresholds. Further detail on this library is provided in the library.
Based on the secret refresh interval defined in your cache configuration, the cache will check the version of the secret at the defined interval, using the DescribeSecret API to determine if a new version is available. If there is a newer version of the secret, the cache will update to the latest version from AWS Secrets Manager, using the GetSecretValue API. This ensures that an updated version of the secret is available in the cache.
Additionally, the Python client-side cache library allows developers to retrieve secrets from the cache directly, using the secret name through decorator functions. An example of using a decorator function is shown below:
from aws_secretsmanager_caching.decorators import InjectKeywordedSecretString
@InjectKeywordedSecretString('python-cache-test', cache, arg1='secret_key1', arg2='secret_key2')
def my_test_function(arg1, arg2):
test = TestClass()
To delete the secret created in this post, run the command below:
If you have comments about this post, submit them in the Comments section below. If you have questions about anything in this post, start a new thread on the Secrets Manager forum or contact AWS Support.
Want more AWS Security news? Follow us on Twitter.
Rik Cross, Senior Learning Manager here at the Raspberry Pi Foundation, shows you how to recreate the deadly explosions in the classic game, Bomberman.
An early incarnation of Bomberman on the NES; the series is still going strong today under Konami’s wing.
Bomberman was first released in the early 1980s as a tech demo for a BASIC compiler, but soon became a popular series that’s still going today. Bomberman sees players use bombs to destroy enemies and uncover doors behind destructible tiles. In this article, I’ll show you how to recreate the bombs that explode in four directions, destroying parts of the level as well as any players in their path!
The game level is a tilemap stored as a two-dimensional array. Each tile in the map is a Tile object, which contains the tile type, and corresponding image. For simplicity, a tile can be set to one of five types; GROUND, WALL, BRICK, BOMB, or EXPLOSION. In this example code, BRICK and GROUND can be exploded with bombs, but WALL cannot, but of course, this behaviour can be changed.
Each Tile object also has a timer, which is decremented each frame of the game. When a tile’s timer reaches 0, an action is carried out, which is dependent on the tile type. BOMB tiles (and surrounding tiles) turn into EXPLOSION tiles after a short delay, and EXPLOSION tiles eventually turn back into GROUND. At the start of the game, the tilemap for the level is generated, in this case consisting of mostly GROUND, with some WALL and a couple of BRICK tiles. The player starts off in the top-left tile, and moves by using the arrow keys. Pressing the SPACE key will place a bomb in the player’s current tile, which is achieved by setting the Tile at the player’s position to BOMB. The tile’s timer is also set to a small number, and once this timer is decremented to 0, the bomb tile and the tiles around it are set to EXPLOSION.
The bomb explodes outwards in four directions, with a range determined by the RANGE, which in our code is 3. As the bomb explodes out to the right, for example, the tile to the right of the bomb is checked. If such a tile exists (i.e. the position isn’t out of the level bounds) and can be exploded, then the tile’s type is set to EXPLOSION and the next tile to the right is checked. If the explosion moves out of the level bounds, or hits a WALL tile, then the explosion will stop radiating in that direction. This process is then repeated for the other directions.
There’s a nice trick for exploding the bomb without repeating the code four times, and it relies on the sine and cosine values for the four direction angles. The angles are 0° (up), 90° (right), 180° (down) and 270° (left). When exploding to the right (at an angle of 90°), sin(90) is 1 and cos(90) is 0, which corresponds to the offset direction on the x- and y-axis respectively. These values can be multiplied by the tile offset, to explode the bomb in all four directions.
Get your copy of Wireframe issue 12
You can read the rest of the feature in Wireframe issue 12, available now at Tesco, WHSmith, and all good independent UK newsagents.
By Pythonistas at Netflix, coordinated by Amjith Ramanujam and edited by Ellen Livengood
As many of us prepare to go to PyCon, we wanted to share a sampling of how Python is used at Netflix. We use Python through the full content lifecycle, from deciding which content to fund all the way to operating the CDN that serves the final video to 148 million members. We use and contribute to many open-source Python packages, some of which are mentioned below. If any of this interests you, check out the jobs site or find us at PyCon. We have donated a few Netflix Originals posters to the PyLadies Auction and look forward to seeing you all there.
Open Connect is Netflix’s content delivery network (CDN). An easy, though imprecise, way of thinking about Netflix infrastructure is that everything that happens before you press Play on your remote control (e.g., are you logged in? what plan do you have? what have you watched so we can recommend new titles to you? what do you want to watch?) takes place in Amazon Web Services (AWS), whereas everything that happens afterwards (i.e., video streaming) takes place in the Open Connect network. Content is placed on the network of servers in the Open Connect CDN as close to the end user as possible, improving the streaming experience for our customers and reducing costs for both Netflix and our Internet Service Provider (ISP) partners.
Various software systems are needed to design, build, and operate this CDN infrastructure, and a significant number of them are written in Python. The network devices that underlie a large portion of the CDN are mostly managed by Python applications. Such applications track the inventory of our network gear: what devices, of which models, with which hardware components, located in which sites. The configuration of these devices is controlled by several other systems including source of truth, application of configurations to devices, and back up. Device interaction for the collection of health and other operational data is yet another Python application. Python has long been a popular programming language in the networking space because it’s an intuitive language that allows engineers to quickly solve networking problems. Subsequently, many useful libraries get developed, making the language even more desirable to learn and use.
Demand Engineering is responsible for Regional Failovers, Traffic Distribution, Capacity Operations, and Fleet Efficiency of the Netflix cloud. We are proud to say that our team’s tools are built primarily in Python. The service that orchestrates failover uses numpy and scipy to perform numerical analysis, boto3 to make changes to our AWS infrastructure, rq to run asynchronous workloads and we wrap it all up in a thin layer of Flask APIs. The ability to drop into a bpython shell and improvise has saved the day more than once.
We are heavy users of Jupyter Notebooks and nteract to analyze operational data and prototype visualization tools that help us detect capacity regressions.
The CORE team uses Python in our alerting and statistical analytical work. We lean on many of the statistical and mathematical libraries (numpy, scipy, ruptures, pandas) to help automate the analysis of 1000s of related signals when our alerting systems indicate problems. We’ve developed a time series correlation system used both inside and outside the team as well as a distributed worker system to parallelize large amounts of analytical work to deliver results quickly.
Python is also a tool we typically use for automation tasks, data exploration and cleaning, and as a convenient source for visualization work.
Monitoring, alerting and auto-remediation
The Insight Engineering team is responsible for building and operating the tools for operational insight, alerting, diagnostics, and auto-remediation. With the increased popularity of Python, the team now supports Python clients for most of their services. One example is the Spectator Python client library, a library for instrumenting code to record dimensional time series metrics. We build Python libraries to interact with other Netflix platform level services. In addition to libraries, the Winston and Bolt products are also built using Python frameworks (Gunicorn + Flask + Flask-RESTPlus).
The information security team uses Python to accomplish a number of high leverage goals for Netflix: security automation, risk classification, auto-remediation, and vulnerability identification to name a few. We’ve had a number of successful Python open sources, including Security Monkey (our team’s most active open source project). We leverage Python to protect our SSH resources using Bless. Our Infrastructure Security team leverages Python to help with IAM permission tuning using Repokid. We use Python to help generate TLS certificates using Lemur.
Some of our more recent projects include Prism: a batch framework to help security engineers measure paved road adoption, risk factors, and identify vulnerabilities in source code. We currently provide Python and Ruby libraries for Prism. The Diffy forensics triage tool is written entirely in Python. We also use Python to detect sensitive data using Lanius.
We use Python extensively within our broader Personalization Machine Learning Infrastructure to train some of the Machine Learning models for key aspects of the Netflix experience: from our recommendation algorithms to artwork personalization to marketing algorithms. For example, some algorithms use TensorFlow, Keras, and PyTorch to learn Deep Neural Networks, XGBoost and LightGBM to learn Gradient Boosted Decision Trees or the broader scientific stack in Python (e.g. numpy, scipy, sklearn, matplotlib, pandas, cvxpy). Because we’re constantly trying out new approaches, we use Jupyter Notebooks to drive many of our experiments. We have also developed a number of higher-level libraries to help integrate these with the rest of our ecosystem (e.g. data access, fact logging and feature extraction, model evaluation, and publishing).
Machine Learning Infrastructure
Besides personalization, Netflix applies machine learning to hundreds of use cases across the company. Many of these applications are powered by Metaflow, a Python framework that makes it easy to execute ML projects from the prototype stage to production.
Metaflow pushes the limits of Python: We leverage well parallelized and optimized Python code to fetch data at 10Gbps, handle hundreds of millions of data points in memory, and orchestrate computation over tens of thousands of CPU cores.
But Python plays a huge role in how we provide those services. Python is a primary language when we need to develop, debug, explore, and prototype different interactions with the Jupyter ecosystem. We use Python to build custom extensions to the Jupyter server that allows us to manage tasks like logging, archiving, publishing, and cloning notebooks on behalf of our users. We provide many flavors of Python to our users via different Jupyter kernels, and manage the deployment of those kernel specifications using Python.
The Big Data Orchestration team is responsible for providing all of the services and tooling to schedule and execute ETL and Adhoc pipelines.
Many of the components of the orchestration service are written in Python. Starting with our scheduler, which uses Jupyter Notebooks with papermill to provide templatized job types (Spark, Presto, …). This allows our users to have a standardized and easy way to express work that needs to be executed. You can see some deeper details on the subject here. We have been using notebooks as real runbooks for situations where human intervention is required — for example: to restart everything that has failed in the last hour.
Internally, we also built an event-driven platform that is fully written in Python. We have created streams of events from a number of systems that get unified into a single tool. This allows us to define conditions to filter events, and actions to react or route them. As a result of this, we have been able to decouple microservices and get visibility into everything that happens on the data platform.
Our team also built the pygenie client which interfaces with Genie, a federated job execution service. Internally, we have additional extensions to this library that apply business conventions and integrate with the Netflix platform. These libraries are the primary way users interface programmatically with work in the Big Data platform.
Finally, it’s been our team’s commitment to contribute to papermill and scrapbook open source projects. Our work there has been both for our own and external use cases. These efforts have been gaining a lot of traction in the open source community and we’re glad to be able to contribute to these shared projects.
The scientific computing team for experimentation is creating a platform for scientists and engineers to analyze AB tests and other experiments. Scientists and engineers can contribute new innovations on three fronts, data, statistics, and visualizations.
The Metrics Repo is a Python framework based on PyPika that allows contributors to write reusable parameterized SQL queries. It serves as an entry point into any new analysis.
The Causal Models library is a Python & R framework for scientists to contribute new models for causal inference. It leverages PyArrow and RPy2 so that statistics can be calculated seamlessly in either language.
The Visualizations library is based on Plotly. Since Plotly is a widely adopted visualization spec, there are a variety of tools that allow contributors to produce an output that is consumable by our platforms.
The Partner Ecosystem group is expanding its use of Python for testing Netflix applications on devices. Python is forming the core of a new CI infrastructure, including controlling our orchestration servers, controlling Spinnaker, test case querying and filtering, and scheduling test runs on devices and containers. Additional post-run analysis is being done in Python using TensorFlow to determine which tests are most likely to show problems on which devices.
Video Encoding and Media Cloud Engineering
Our team takes care of encoding (and re-encoding) the Netflix catalog, as well as leveraging machine learning for insights into that catalog. We use Python for ~50 projects such as vmaf and mezzfs, we build computer vision solutions using a media map-reduce platform called Archer, and we use Python for many internal projects. We have also open sourced a few tools to ease development/distribution of Python projects, like setupmeta and pickley.
Netflix Animation and NVFX
Python is the industry standard for all of the major applications we use to create Animated and VFX content, so it goes without saying that we are using it very heavily. All of our integrations with Maya and Nuke are in Python, and the bulk of our Shotgun tools are also in Python. We’re just getting started on getting our tooling in the cloud, and anticipate deploying many of our own custom Python AMIs/containers.
Content Machine Learning, Science & Analytics
The Content Machine Learning team uses Python extensively for the development of machine learning models that are the core of forecasting audience size, viewership, and other demand metrics for all content.
The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.