Tag Archives: python

Watch Game of Thrones with a Raspberry Pi-powered Drogon

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/watch-game-of-thrones-with-raspberry-pi-powered-drogon/

Channel your inner Targaryen by building this voice-activated, colour-changing, 3D-printed Drogon before watching the next episode of Game of Thrones.

Winter has come

This is a spoiler-free zone! I’ve already seen the new episode of season 8, but I won’t ruin anything, I promise.

Even if you’ve never watched an episode of Game of Thrones (if so, that’s fine, I don’t judge you), you’re probably aware that the final season has started.

And you might also know that the show has dragons in it — big, hulking, scaley dragons called Rhaegal, Viserion, and Drogon. They look a little something like this:Daenerys-Targaryen-game-of-thrones

Well, not anymore. They look like this now:

04_15_GameOfThrones_S07-920x584

Raspberry Pi voice-responsive dragon!

The creator of this project goes by the moniker Botmation. To begin with, they 3D printed modified a Drogon model they found on Thingiverse. Then, with Dremel in hand, they modified the print, to replace its eyes with RGB LEDs. Before drawing the LEDs through the hollowed-out body of the model, they soldered them to wires connected to a Raspberry Pi Zero W‘s GPIO pins.

Located in the tin beneath Drogon, the Pi Zero W is also equipped with a microphone and runs the Python code for the project. And thanks to Google’s Speech to Text API, Drogon’s eyes change colour whenever a GoT character repeats one of two keywords: white turns the eyes blue, while fire turns them red.

If you’d like more information about building your own interactive Drogon, here’s a handy video. At the end, Botmation asks viewers to help improve their code for a cleaner voice-activation experience.

3D printed Drogon with LED eyes for Game of Thrones

Going into the final season of Game of Thrones with your very own 3D printed Drogron dragon! The eyes are made of LEDs that changes between Red and Blue depending on what happens in the show. When you’re watching the show, Drogon will watch the show with you and listen for cues to change the eye color.

Drogon for the throne!

I’ve managed to bag two of the three dragons in the Pi Towers Game of Thrones fantasy league, so I reckon my chances of winning are pretty good thanks to all the points I’ll rack up by killing White Walker.

Wait — does killing a White Walker count as a kill, since they’re already dead?

Ah, crud.

The post Watch Game of Thrones with a Raspberry Pi-powered Drogon appeared first on Raspberry Pi.

Raspberry Pi-controlled brass bell for ultimate the wake-up call

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/raspberry-pi-controlled-brass-bell-for-ultimate-the-wake-up-call/

Not one for rising with the sun, and getting more and more skilled at throwing their watch across the room to snooze their alarm, Reddit user ravenspired decided to hook up a physical bell to a Raspberry Pi and servo motor to create the ultimate morning wake-up call.

DIY RASPBERRY PI BELL RINGING ALARM CLOCK!

This has to be the harshest thing to wake up to EVER!

Wake up, Boo

“I have difficulty waking up in the morning” admits ravenspired, who goes by the name Darks Pi on YouTube. “My watch isn’t doing its job.”

Therefore, ravenspired attached a bell to a servo motor, and the servo motor to a Raspberry Pi. Then they wrote Python code in Raspbian’s free IDE software Thonny that rings the bell when it’s time to get up.

“A while loop searches for what time it is and checks it against my alarm time. When the alarm is active, it sends commands to the servo to move.”

Ouch!

While I’d be concerned about how securely attached the heavy brass bell above my head is, this is still a fun project, and an inventive way to address a common problem.

And it’s a lot less painful than this…

The Wake-up Machine TAKE #2

I built an alarm clock that slapped me in the face with a rubber arm to wake me up.I built an alarm clock that wakes me up in the morning by slapping me in the face with a rubber arm.

Have you created a completely over-engineered solution for a common problem? Then we want to see it!

The post Raspberry Pi-controlled brass bell for ultimate the wake-up call appeared first on Raspberry Pi.

Coding Breakout’s brick-breaking action | Wireframe #11

Post Syndicated from Ryan Lambie original https://www.raspberrypi.org/blog/coding-breakouts-brick-breaking-action-wireframe-11/

Atari’s Breakout was one of the earliest video game blockbusters. Here’s how to recreate it in Python.

The original Breakout, designed by Nolan Bushnell and Steve Bristow, and famously built by a young Steve Wozniak.

Atari Breakout

The games industry owes a lot to the humble bat and ball. Designed by Allan Alcorn in 1972, Pong was a simplified version of table tennis, where the player moved a bat and scored points by ricocheting a ball past their opponent. About four years later, Atari’s Nolan Bushnell and Steve Bristow figured out a way of making Pong into a single-player game. The result was 1976’s Breakout, which rotated Pong’s action 90 degrees and replaced the second player with a wall of bricks.

Points were scored by deflecting the ball off the bat and destroying the bricks; as in Pong, the player would lose the game if the ball left the play area. Breakout was a hit for Atari, and remains one of those game ideas that has never quite faded from view; in the 1980s, Taito’s Arkanoid updated the action with collectible power-ups, multiple stages with different layouts of bricks, and enemies that disrupted the trajectory of the player’s ball.

Breakout had an impact on other genres too: game designer Tomohiro Nishikado came up with the idea for Space Invaders by switching Breakout’s bat with a base that shot bullets, while Breakout’s bricks became aliens that moved and fired back at the player.

Courtesy of Daniel Pope, here’s a simple Breakout game written in Python. To get it running on your system, you’ll first need to install Pygame Zero. And download the code for Breakout here.

Bricks and balls in Python

The code above, written by Daniel Pope, shows you just how easy it is to get a basic version of Breakout up and running in Python, using the Pygame Zero library. Like Atari’s original, this version draws a wall of blocks on the screen, sets a ball bouncing around, and gives the player a paddle, which can be controlled by moving the mouse left and right. The ball physics are simple to grasp too. The ball has a velocity, vel – which is a vector, or a pair of numbers: vx for the x direction and vy for the y direction.

The program loop checks the position of the ball and whether it’s collided with a brick or the edge of the play area. If the ball hits the left side of the play area, the ball’s x velocity vx is set to positive, thus sending it bouncing to the right. If the ball hits the right side, vx is set to a negative number, so the ball moves left. Likewise, when the ball hits the top or bottom of a brick, we set the sign of the y velocity vy, and so on for the collisions with the bat and the top of the play area and the sides of bricks. Collisions set the sign of vx and vy but never change the magnitude. This is called a perfectly elastic collision.

To this basic framework, you could add all kinds of additional features: a 2012 talk by developers Martin Jonasson and Petri Purho, which you can watch on YouTube here, shows how the Breakout concept can be given new life with the addition of a few modern design ideas.

You can read this feature and more besides in Wireframe issue 11, available now in Tesco, WHSmith, and all good independent UK newsagents.

Or you can buy Wireframe directly from us – worldwide delivery is available. And if you’d like to own a handy digital version of the magazine, you can also download a free PDF.

Make sure to follow Wireframe on Twitter and Facebook for updates and exclusives, and for subscriptions, visit the Wireframe website to save 49% compared to newsstand pricing!

The post Coding Breakout’s brick-breaking action | Wireframe #11 appeared first on Raspberry Pi.

Bind MIDI inputs to LED lights using a Raspberry Pi

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/midi-controlled-led-lights-raspberry-pi/

Blinky lights and music created using a Raspberry Pi? Count us in! When Aaron Chambers shared his latest project, Py-Lights, on Reddit, we were quick to ask for more information. And here it is:

Controlling lights with MIDI commands

Tentatively titled Py-Lights, Aaron’s project allows users to assign light patterns to MIDI actions, creating a rather lovely blinky light display.

For his example, Aaron connected a MIDI keyboard to a strip of RGB LEDs via a Raspberry Pi that ran his custom Python code.

Aaron explains on Reddit:

The program I made lets me bind “actions” (strobe white, flash blue, disable all colors, etc.) to any input and any input type (hold, knob, trigger, etc.). And each action type has a set of parameters that I bind to the input. For example, I have a knob that changes a strobe’s intensity, and another knob that changes its speed.

The program updates each action, pulls its resulting color, and adds them together, then sends that to the LEDs. I’m using rtmidi for reading the midi device and pigpio for handling the LED output.

Aaron has updated the Py-Lights GitHub repo for the project to include a handy readme file and a more stable build.

The post Bind MIDI inputs to LED lights using a Raspberry Pi appeared first on Raspberry Pi.

Coding Pang’s sprite spawning mechanic | Wireframe #10

Post Syndicated from Ryan Lambie original https://www.raspberrypi.org/blog/pang-sprite-spawning-wireframe-10/

Rik Cross, Senior Learning Manager here at Raspberry Pi, shows you how to recreate the spawning of objects found in the balloon-bursting arcade gem Pang.

Pang: bringing balloon-hating to the masses since 1989.

Capcom’s Pang

Programmed by Mitchell and distributed by Capcom, Pang was first released as an arcade game in 1989, but was later ported to a whole host of home computers, including the ZX Spectrum, Amiga, and Commodore 64. The aim in Pang is to destroy balloons as they bounce around the screen, either alone or working together with another player, in increasingly elaborate levels. Destroying a balloon can sometimes also spawn a power-up, freezing all balloons for a short time or giving the player a better weapon with which to destroy balloons.

Initially, the player is faced with the task of destroying a small number of large balloons. However, destroying a large balloon spawns two smaller balloons, which in turn spawns two smaller balloons, and so on. Each level is only complete once all balloons have been broken up and completely destroyed. To add challenge to the game, different-sized balloons have different attributes – smaller balloons move faster and don’t bounce as high, making them more difficult to destroy.

Rik’s spawning balloons, up and running in Pygame Zero. Hit space to divide them into smaller balloons.

Spawning balloons

There are a few different ways to achieve this game mechanic, but the approach I’ll take in my example is to use various features of object orientation (as usual, my example code has been written in Python, using the Pygame Zero library). It’s also worth mentioning that for brevity, the example code only deals with simple spawning and destroying of objects, and doesn’t handle balloon movement or collision detection.

The base Enemy class is simply a subclass of Pygame Zero’s Actor class, including a static enemies list to keep track of all enemies that exist within a level. The Enemy subclass also includes a destroy() method, which removes an enemy from the enemies list and deletes the object.

There are then three further subclasses of the Enemy class, called LargeEnemy, MediumEnemy, and SmallEnemy. Each of these subclasses are instantiated with a specific image, and also include a destroy() method. This method simply calls the same destroy() method of its parent Enemy class, but additionally creates two more objects nearby — with large enemies spawning two medium enemies, and medium enemies spawning two small enemies.

Wireframe 10 Pang

Here’s Rik’s example code, which recreates Pang’s spawning balloons in Python. To get it running on your system, you’ll first need to install Pygame Zero – you can find full instructions here. And you can download the code here.

In the example code, initially two LargeEnemy objects are created, with the first object in the enemies list having its destroy() method called each time the Space key is pressed. If you run this code, you’ll see that the first large enemy is destroyed and two medium-sized enemies are created. This chain reaction of destroying and creating enemies continues until all SmallEnemy objects are destroyed (small enemies don’t create any other enemies when destroyed).

As I mentioned earlier, this isn’t the only way of achieving this behaviour, and there are advantages and disadvantages to this approach. Using subclasses for each size of enemy allows for a lot of customisation, but could get unwieldy if much more than three enemy sizes are required. One alternative is to simply have a single Enemy class, with a size attribute. The enemy’s image, the entities it creates when destroyed, and even the movement speed and bounce height could all depend on the value of the enemy size.

You can read the rest of the feature in Wireframe issue 10, available now in Tesco, WHSmith, and all good independent UK newsagents.

Or you can buy Wireframe directly from us – worldwide delivery is available. And if you’d like to own a handy digital version of the magazine, you can also download a free PDF.

Make sure to follow Wireframe on Twitter and Facebook for updates and exclusives, and for subscriptions, visit the Wireframe website to save 49% compared to newsstand pricing!

The post Coding Pang’s sprite spawning mechanic | Wireframe #10 appeared first on Raspberry Pi.

The Junk Drum Machine

Post Syndicated from Liz Upton original https://www.raspberrypi.org/blog/junk-drum-machine/

I do not really have any spare time. (Toddler, job, very demanding cat, lots of LEGO to tidy up.) If I did, I like to imagine that I’d come up with something like this to do with it.

junk drum machine

Want to see this collection of junk animate? Scroll down for video.

From someone calling themselves Banjowise (let me know what your real name is in the comments, please, so I can credit you properly here!), here is a pile of junk turned into a weirdly compelling drum machine.

Mechanically speaking, this isn’t too complicated: just a set of solenoids triggered by a Raspberry Pi. The real clever is in the beauteous, browser-based step sequencer Banjowise has built to program the solenoids to wallop things in beautiful rhythm. And in the beauteous, skip-sourced tchotchkes that Banjowise has found for them to wallop. Generously, they’ve made full instructions on making your own available on Instructables. Use any bits and bobs you can get your hands on if old piano hammers and crocodile castanets are not part of the detritus kicking around your house.

Warning: this video is weirdly compelling.

Automabeat – A Raspberry Pi Mechanical Robotic Junk Drum Machine

My Raspberry Pi based drum / percussion machine. Consisting of 8 12v solenoids, a relay, wooden spoons, a Fullers beer bottle, a crocodile maraca and a few other things. An Instructable on how to build your own is here: https://www.instructables.com/id/A-Raspberry-Pi-Powered-Junk-Drum-Machine/, or take a look at: http://www.banjowise.com/post/automabeat/

The sequencer is lovely: a gorgeously simple user interface that you can run on a tablet, your phone, or anything else with a browser (and it’s very easily adaptable to other projects). The web interface lets Python trigger the GPIO pins over web sockets. There’s a precompiled version available for people who’ve followed Banjowise’s comprehensive wiring instructions, but you can also get the source code from GitHub.

Sequencer UI

I think I’m getting good, but I can handle criticism.

We love it. Now please excuse me. I need a little while to search online for crocodile castanets.

The post The Junk Drum Machine appeared first on Raspberry Pi.

Coding Space Invaders’ disintegrating shields | Wireframe #9

Post Syndicated from Ryan Lambie original https://www.raspberrypi.org/blog/coding-space-invaders-disintegrating-shields-wireframe-9/

They add strategy to a genre-defining shooter. Andrew Gillett lifts the lid on Space Invaders’ disintegrating shields.

Wireframe 9 Space Invaders

Released in 1978, Space Invaders introduced ideas so fundamental to video games that it’s hard to imagine a time before them. And it did this using custom-made hardware which by today’s standards is unimaginably slow.

Space Invaders ran on an Intel 8080 CPU operating at 2MHz. With such meagre processing power, merely moving sprites around the screen was a struggle. In modern 2D games, at the start of each frame the entire screen is reset, then all objects are displayed.

For Space Invaders’ hardware, this process would have been too slow. Instead, each time a sprite needs to move, the game first erases the sprite from the screen, then redraws it in the new position. The game also updates only one alien per frame — which leads to the effect of the aliens moving faster when there are fewer of them. These techniques cut down the number of pixels which need to be updated each frame, from nearly 60,000 to around a hundred.

Wireframe 9 Space Invaders

One of Space Invaders’ most notable features is its four shields. These provide shelter from enemy fire, but deteriorate after repeated hits. The player can take advantage of the shields’ destructible nature — by repeatedly firing at the same place on a shield’s underside, a narrow gap can be created which can then be used to take out enemies. (Of course, the player can also be shot through the same gap.)

The system of updating only the minimum necessary number of pixels works well as long as there’s no need for objects to overlap. In the case of the shields, though, what happens when objects do overlap is fundamental to how they work. Whenever a shot hits something, it’s replaced by an explosion sprite. A few frames later, the explosion sprite is deleted from the screen. If the explosion sprite overlapped with a shield, that part of the shield is also deleted.

Wireframe 9 Space Invaders

Here’s a code snippet that shows Andrew’s Space Invaders-style disintegrating shields working in Python. To get it running on your system, you’ll need to install Pygame Zero — you can find full instructions here. And download the above code here.

The code to the right displays four shields, and then bombards them with a series of shots which explode on impact. I’m using sprites which have been scaled up by ten, to make it easier to see what’s going on.

We first create two empty lists — one to hold details of any shots on screen, as well as explosions. These will be displayed on the screen every frame. Each entry in the shots list will be a dictionary data structure containing three values: a position, the sprite to be displayed, and whether the shot is in ‘exploding’ mode — in which case it’s displayed in the same position for a few frames before being deleted.

The second list, to_delete, is for sprites which need to be deleted from the screen. For simplicity, I’m using separate copies of the shot and explosion sprites where the white pixels have been changed to black (the other pixels in these sprites are set as transparent).

The function create_random_shot is called every half-second. The combination of dividing the maximum value by ten, choosing a random whole number between zero and the maximum value, and then multiplying the resulting random number by ten, ensures that the chosen X coordinate is a multiple of ten.


Wireframe 9 Space Invaders
Wireframe 9 Space Invaders

Andrew’s Space Invaders shields up and running in Pygame Zero.

In the draw function, we first check to see if it’s the first frame, as we only want to display the shields on that frame. The screen.blit method is used to display sprites, and Pygame Zero’s images object is used to specify which sprite should be displayed. We then display all sprites in the to_delete list, after which we reset it to being an empty list. Finally we display all sprites in the shots list.

Wireframe 9 Space Invaders

In the update function, we go through all sprites in the shots list, in reverse order. Going through the list backwards avoids problems that can occur when deleting items from a list inside a for loop. For each shot, we first check to see if it’s in ‘exploding’ mode. If so, its timer is reduced each frame — when it hits zero we add the shot to the to_delete list, then delete it from shots.

If the item is a normal shot rather than an explosion, we add its current position to to_delete, then update the shot’s position to move the sprite down the screen. We next check to see if the sprite has either gone off the bottom of the screen or collided with something. Pygame’s get_at method gives us the colour of a pixel at a given position. If a collision occurs, we switch the shot into ‘exploding’ mode — the explosion sprite will be displayed for five frames.

You can read the rest of the feature in Wireframe issue 9, available now in Tesco, WHSmith, and all good independent UK newsagents.

Or you can buy Wireframe directly from us – worldwide delivery is available. And if you’d like to own a handy digital version of the magazine, you can also download a free PDF.

Make sure to follow Wireframe on Twitter and Facebook for updates and exclusives, and for subscriptions, visit the Wireframe website to save 49% compared to newsstand pricing!

The post Coding Space Invaders’ disintegrating shields | Wireframe #9 appeared first on Raspberry Pi.

GPIO Zero v1.5 is here!

Post Syndicated from Ben Nuttall original https://www.raspberrypi.org/blog/gpio-zero-v1-5/

GPIO Zero is a zero-boilerplate Python library that makes physical computing with Python more accessible and helps people progress from zero to hero.

Today, I’m pleased to announce the release of GPIO Zero v1.5.0. It’s packed full of updates, including new features, bug fixes, and lots of improvements to the documentation.

Guido, the creator of Python, happened across the library recently, and he seemed to like it:

Guido van Rossum on Twitter

GPIOzero I love you! https://t.co/w3CnUGx3yO

Pin factories – take your pick

GPIO Zero started out as a friendly API on top of the RPi.GPIO library, but later we extended it to allow other pin libraries to be used. The pigpio library is supported, and that includes the ability to remotely control GPIO pins over the network, or on a Pi Zero over USB.

This also gave us the opportunity to create a “mock” pin factory, so that we could emulate the effect of pin changes without using real Raspberry Pi hardware. This is useful for prototyping without hardware, and for testing. Try it yourself!

As well as the pin factories we provide with the library (RPi.GPIO, pigpio, RPIO, and native), it’s also possible to write your own. So far, I’m aware of only one custom pin factory, and that has been written by the AIY team at Google, who created their own pin factory for the pins on the AIY Vision Kit. This means that you can connect devices to these pins, and use GPIO Zero to program them, despite the fact they’re not connected to the Pi’s own pins.

If you have lots of experience with RPi.GPIO, you might find this guide on migrating from RPi.GPIO to GPIO Zero handy.

Ultrasonic distance sensor

We had identified some issues with the results from the DistanceSensor class, and we dealt with them in two ways. Firstly, GPIO Zero co-author Dave Jones did some work under the hood of the pins API to use timing information provided by underlying drivers, so that timing events from pins will be considerably more accurate (see #655). Secondly, Dave found that RPi.GPIO would often miss edges during callbacks, which threw off the timing, so we now drop missed edges and get better accuracy as a result (see #719).

The best DistanceSensor results come when using pigpio as your pin factory, so we recommend changing to this if you want more accuracy, especially if you’re using (or deploying to) a Pi 1 or Pi Zero.

Connecting devices

A really neat feature of GPIO Zero is the ability to connect devices together easily. One way to do this is to use callback functions:

button.when_pressed = led.on
button.when_released = led.off

Another way is to set the source of one device to the values of another device:

led.source = button.values

In GPIO Zero v1.5, we’ve made connecting devices even easier. You can now use the following method to pair devices together:

led.source = button

Read more about this declarative style of programming in the source/values page in the docs. There are plenty of great examples of how you can create projects with these simple connections:

Testing

An important part of software development is automated testing. You write tests to check your code does what you want it to do, especially checking the edge cases. Then you write the code to implement the features you’ve written tests for. Then after every change you make, you run your old tests to make sure nothing got broken. We have tools for automating this (thanks pytest, tox, coverage, and Travis CI).

But how do you test a GPIO library? Well, most of the GPIO parts of our test suite use the mock pins interface, so we can test our API works as intended, abstracted from how the pins behave. And while Travis CI only runs tests with mock pins, we also do real testing on Raspberry Pi: there are additional tests that ensure the pins do what they’re supposed to. See the docs chapter on development to learn more about this process, and try it for yourself.

pinout

You may remember that the last major GPIO Zero release introduced the pinout command line tool. We’ve added some new art for the Pi 3A+ and 3B+:

pinout also now supports the -x (or --xyz) option, which opens the website pinout.xyz in your web browser.

Zero boilerplate for hardware

The goal of all this is to remove obstacles to physical computing, and Rachel Rayns has designed a wonderful board that makes a great companion to GPIO Zero for people who are learning. Available from The Pi Hut, the PLAY board provides croc-clip connectors for four GPIO pins, GND, and 3V3, along with a set of compatible components:

Since the board simply breaks out GPIO pins, there’s no special software required. You can use Scratch or Python (or anything else).

New contributors

This release welcomed seven new contributors to the project, including Claire Pollard from PiBorg and ModMyPi, who provided implementations for TonalBuzzer, PumpkinPi, and the JamHat. We also passed 1000 commits!

Watch your tone

As part of the work Claire did to add support for the Jam HAT, she created a new class for working with its buzzer, which works by setting the PWM frequency to emit a particular tone. I took what Claire provided and added some maths to it, then Dave created a whole Tones module to provide a musical API. You can play buzzy jingles, or you can build a theremin:

GPIO Zero theremin

from gpiozero import TonalBuzzer, DistanceSensor buzzer = TonalBuzzer(20) ds = DistanceSensor(14, 26) buzzer.source = ds

…or you can make a siren:

GPIO Zero TonalBuzzer sine wave

from gpiozero import TonalBuzzer from gpiozero.tools import sin_values buzzer = TonalBuzzer(20) buzzer.source = sin_values()

The Tones API is a really neat way of creating particular buzzer sounds and chaining them together to make tunes, using a variety of musical notations:

>>> from gpiozero.tones import Tone
>>> Tone(440.0)
>>> Tone(69)
>>> Tone('A4')

We all make mistakes

One of the important things about writing a library to help beginners is knowing when to expect mistakes, and providing help when you can. For example, if a user mistypes an attribute or just gets it wrong – for example, if they type button.pressed = foo instead of button.when_pressed = foo – they wouldn’t usually get an error; it would just set a new attribute. In GPIO Zero, though, we prevent new attributes from being created, so you’d get an error if you tried doing this. We provide an FAQ about this, and explain how to get around it if you really need to.

Similarly, it’s common to see people type button.when_pressed = foo() and actually call the function, which isn’t correct, and will usually have the effect of unsetting the callback (as the function returns None). Because this is valid, the user won’t get an error to call their attention to the mistake.

In this release, we’ve added a warning that you’ll see if you set a callback to None when it was previously None. Hopefully that will be useful to people who make this mistake, helping them quickly notice and rectify it.

Update now

Update your Raspberry Pi now to get the latest and greatest GPIO Zero goodness in your (operating) system:

sudo apt update
sudo apt install python3-gpiozero python-gpiozero

Note: it’s currently syncing with the Raspbian repo, so if it’s not available for you yet, it will be soon.

What’s next?

We have plenty more suggestions to be working on. This year we’ll be working on SPI and I2C interfaces, including I2C expander chips. If you’d like to make more suggestions, or contribute yourself, find us over on GitHub.

The post GPIO Zero v1.5 is here! appeared first on Raspberry Pi.

Building a text adventure | Wireframe #6

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/making-text-adventure-wireframe-6/

Game developer Andrew Gillett explains how to make a simple text adventure in Python — and what pitfalls to avoid while doing so — in the latest issue of Wireframe magazine, out now.

Writing games in BASIC

The first game I ever wrote was named Pooh. It had nothing to do with the bear. In September 1982, I was four years old, and the ZX Spectrum home computer had just been released. It was incredible enough that the Spectrum let you play games on the TV, but like most home computers of the time, it also came with a built-in language called BASIC, and a manual which explained how to program it. In my first game, Pooh (the title was a misspelling), the player controlled a baby, represented by a pound sign, and had to guide it to a potty, represented by the letter O. There were no obstacles, no enemies, and if you tried to walk off the screen, the program would stop with an error message. I didn’t have any idea how to create a graphical game more complex than Pooh. I didn’t even know how to display a sprite on the screen.

The Hobbit, released in 1982, was widely praised for its intuitive parser.

Text adventures

Instead, I focused on writing text adventures, where the game describes scenes to the player (“You are in a comfortable, tunnel-like hall. You can see a door,” from 1982’s The Hobbit) and the player enters commands such as “Go through door” or “Kill goblin with sword.” Although this type of game is comparatively easy to write, I implemented it in the worst way possible. The code was essentially a huge list of IF statements. Each room had its own set of code, which would print out a description of the room and then check to see what the player typed. This ‘hard-coding’ led to the code being much longer than necessary, and more difficult to maintain.

The correct way would have been to separate my code and data. Each room would have had several pieces of data associated with it, such as an ID number, the description of the room (“You are in a small cave”), an array of objects which can be found in the room, and an array of room numbers indicating where the player should end up if they try to move in a particular direction – for example, the first number could indicate which room to go to if the player enters ‘NORTH’. You’d then have the main game code which keeps track of the room the player is currently in, and looks up the data for that room. With that data, it can then take the appropriate action based on the command the player typed.

Getting it right

The code below shows how to implement the beginnings of a text adventure game in Python. Instead of numeric IDs and arrays, the code uses string IDs and dictionary data structures, where each piece of data is associated with an ID or ‘key’. This is a more convenient option which wasn’t available in Spectrum BASIC. We first create a list of directions in which the player can potentially move. We then create the class Location which specifies each location’s properties. We store a name, a description, and a dictionary data structure which stores the other locations that the current location is linked to. For example, if you go north from the woods, you’ll reach the lake. The class includes a method named addLink, which adds entries to the linked locations dictionary after checking that the specified direction and destination exist.

Following the class definition, we then create a dictionary named locations. This has two entries, with the keys being woods and lake, and the values being instances of the Location class. Next, we call the addLink method on each of the locations we’ve just created, so that the player will be able to walk between them. The final step of the setup phase is to create the variable currentLocation, specifying where the player will start the game.

We then reach the main game loop, which will repeat indefinitely. We first display the description of the current location, along with the available directions in which the player can move. Then we wait for the player to input a command. In this version of the code, the only valid commands are directions: for example, type ‘north’ at the starting location to go to the lake. When a direction is entered, we check to make sure it’s a valid direction from the current location, then update currentLocation to the new location. When the main loop restarts, the description of the new location is displayed.

I moved on from the ZX Spectrum eight years after my dad first unpacked it. Despite the poor design of my code, I’d learned the essentials of programming. Ten years later, I was a game developer.

Further reading

If you’re keen to learn more about making a text adventure in Python, you could check out Phillip Johnson’s guide to the subject, Make Your Own Python Text Adventure. The author has also written a condensed version of the same guide.

You may also be interested in our free online course Object-oriented Programming in Python: Create Your Own Adventure Game.

More from Wireframe

You can discover more tutorials, alongside great reviews, articles and advice, in Wireframe issue 6, out now and available in Tesco, WHSmith, and all good independent UK newsagents.

Or you can buy Wireframe directly from the Raspberry Pi Press store — worldwide delivery is available. And if you’d like to own a handy digital version of the magazine, you can also download the PDF for free.

The post Building a text adventure | Wireframe #6 appeared first on Raspberry Pi.

MagPi 77: Make with code

Post Syndicated from Rob Zwetsloot original https://www.raspberrypi.org/blog/magpi-77/

Hey folks, Rob from The MagPi here! Before I head off on my Christmas holidays, I want to introduce you to The MagPi 77, where we teach you how to make with code.

Making made fun! See what we did there?

What do we mean by that? Well, using code to make things – whether that’s scripts, programs, or games on your Pi, or whether you’re controlling LEDs with code, or robots, or massive Rube Goldberg machines. In this feature, we show new Pi users how to get started making practical applications with Python, and hopefully you’ll be inspired to go on and do something special.

Can you make… with code?

Accessories make the Pi

Want to power up your Raspberry Pi with a few extras? We’ve put together a guide to the 20 best Raspberry Pi accessories, covering IoT, robots, media, power solutions, and even industrial add-ons. There’s a lot of stuff you can do with your Pi, and even more if you’ve got the right tool to help.

We have the best accessories for you

More, you say?

Still need more reasons to grab a copy? Well, we have a tutorial on how to make a smart door, we continue developing Pac-Man while checking out the Picade Console, and we have plenty of amazing project showcases like the SelfieBot!


Get The MagPi 77

You can get The MagPi 77 from WHSmith, Tesco, Sainsbury’s, and Asda. If you live in the US, head over to your local Barnes & Noble or Micro Center in the next few days for a print copy. You can also get the issue online: check it out on our store, or digitally via our Android or iOS apps. And don’t forget, there’s always the free PDF.

Free Raspberry Pi 3A+ offer!

We’re still running our super special Raspberry Pi 3A+ subscription offer! If you subscribe to twelve months of The MagPi, you’ll get a Raspberry Pi 3A+ completely free while stocks last. Make sure to check out our other subs offers while you’re there, like three issues for £5, and our rolling monthly sub.

Get a 3A+ completely free while stocks last!

Right, happy holidays, folks! See you all in the New Year!

The post MagPi 77: Make with code appeared first on Raspberry Pi.

New – AWS Toolkits for PyCharm, IntelliJ (Preview), and Visual Studio Code (Preview)

Post Syndicated from Danilo Poccia original https://aws.amazon.com/blogs/aws/new-aws-toolkits-for-pycharm-intellij-preview-and-visual-studio-code-preview/

Software developers have their own preferred tools. Some use powerful editors, others Integrated Development Environments (IDEs) that are tailored for specific languages and platforms. In 2014 I created my first AWS Lambda function using the editor in the Lambda console. Now, you can choose from a rich set of tools to build and deploy serverless applications. For example, the editor in the Lambda console has been greatly enhanced last year when AWS Cloud9 was released. For .NET applications, you can use the AWS Toolkit for Visual Studio and AWS Tools for Visual Studio Team Services.

AWS Toolkits for PyCharm, IntelliJ, and Visual Studio Code

Today, we are announcing the general availability of the AWS Toolkit for PyCharm. We are also announcing the developer preview of the AWS Toolkits for IntelliJ and Visual Studio Code, which are under active development in GitHub. These open source toolkits will enable you to easily develop serverless applications, including a full create, step-through debug, and deploy experience in the IDE and language of your choice, be it Python, Java, Node.js, or .NET.

For example, using the AWS Toolkit for PyCharm you can:

These toolkits are distributed under the open source Apache License, Version 2.0.

Installation

Some features use the AWS Serverless Application Model (SAM) CLI. You can find installation instructions for your system here.

The AWS Toolkit for PyCharm is available via the IDEA Plugin Repository. To install it, in the Settings/Preferences dialog, click Plugins, search for “AWS Toolkit”, use the checkbox to enable it, and click the Install button. You will need to restart your IDE for the changes to take effect.

The AWS Toolkit for IntelliJ and Visual Studio Code are currently in developer preview and under active development. You are welcome to build and install these from the GitHub repositories:

Building a Serverless application with PyCharm

After installing AWS SAM CLI and AWS Toolkit, I create a new project in PyCharm and choose SAM on the left to create a serverless application using the AWS Serverless Application Model. I call my project hello-world in the Location field. Expanding More Settings, I choose which SAM template to use as the starting point for my project. For this walkthrough, I select the “AWS SAM Hello World”.

In PyCharm you can use credentials and profiles from your AWS Command Line Interface (CLI) configuration. You can change AWS region quickly if you have multiple environments.
The AWS Explorer shows Lambda functions and AWS CloudFormation stacks in the selected AWS region. Starting from a CloudFormation stack, you can see which Lambda functions are part of it.

The function handler is in the app.py file. After I open the file, I click on the Lambda icon on the left of the function declaration to have the option to run the function locally or start a local step-by-step debugging session.

First, I run the function locally. I can configure the payload of the event that is provided in input for the local invocation, starting from the event templates provided for most services, such as the Amazon API Gateway, Amazon Simple Notification Service (SNS), Amazon Simple Queue Service (SQS), and so on. You can use a file for the payload, or select the share checkbox to make it available to other team members. The function is executed locally, but here you can choose the credentials and the region to be used if the function is calling other AWS services, such as Amazon Simple Storage Service (S3) or Amazon DynamoDB.

A local container is used to emulate the Lambda execution environment. This function is implementing a basic web API, and I can check that the result is in the format expected by the API Gateway.

After that, I want to get more information on what my code is doing. I set a breakpoint and start a local debugging session. I use the same input event as before. Again, you can choose the credentials and region for the AWS services used by the function.

I step over the HTTP request in the code to inspect the response in the Variables tab. Here you have access to all local variables, including the event and the context provided in input to the function.

After that, I resume the program to reach the end of the debugging session.

Now I am confident enough to deploy the serverless application right-clicking on the project (or the SAM template file). I can create a new CloudFormation stack, or update an existing one. For now, I create a new stack called hello-world-prod. For example, you can have a stack for production, and one for testing. I select an S3 bucket in the region to store the package used for the deployment. If your template has parameters, here you can set up the values used by this deployment.

After a few minutes, the stack creation is complete and I can run the function in the cloud with a right-click in the AWS Explorer. Here there is also the option to jump to the source code of the function.

As expected, the result of the remote invocation is the same as the local execution. My serverless application is in production!

Using these toolkits, developers can test locally to find problems before deployment, change the code of their application or the resources they need in the SAM template, and update an existing stack, quickly iterating until they reach their goal. For example, they can add an S3 bucket to store images or documents, or a DynamoDB table to store your users, or change the permissions used by their functions.

I am really excited by how much faster and easier it is to build your ideas on AWS. Now you can use your preferred environment to accelerate even further. I look forward to seeing what you will do with these new tools!

Malcom – Malware Communication Analyzer

Post Syndicated from Darknet original https://www.darknet.org.uk/2018/11/malcom-malware-communication-analyzer/?utm_source=rss&utm_medium=social&utm_campaign=darknetfeed

Malcom – Malware Communication Analyzer

Malcom is a Malware Communication Analyzer designed to analyze a system’s network communication using graphical representations of network traffic, and cross-reference them with known malware sources.

This comes handy when analyzing how certain malware species try to communicate with the outside world.

Malcom Malware Communication Analyzer Features

Malcom can help you:

  • Detect central command and control (C&C) servers
  • Understand peer-to-peer networks
  • Observe DNS fast-flux infrastructures
  • Quickly determine if a network artifact is ‘known-bad’

The aim of Malcom is to make malware analysis and intel gathering faster by providing a human-readable version of network traffic originating from a given host or network.

Read the rest of Malcom – Malware Communication Analyzer now! Only available at Darknet.

Python 3.7 runtime now available in AWS Lambda

Post Syndicated from Chris Munns original https://aws.amazon.com/blogs/compute/python-3-7-runtime-now-available-in-aws-lambda/

This post is courtesy of Shivansh Singh, Partner Solutions Architect – AWS

We are excited to announce that you can now develop your AWS Lambda functions using the Python 3.7 runtime. Start using this new version today by specifying a runtime parameter value of “python3.7″ when creating or updating functions. AWS continues to support creating new Lambda functions on Python 3.6 and Python 2.7.

Here’s a quick primer on some of the major new features in Python 3.7:

  • Data classes
  • Customization of access to module attributes
  • Typing enhancements
  • Time functions with nanosecond resolution

Data classes

In object-oriented programming, if you have to create a class, it looks like the following in Python 3.6:

class Employee:
    def __init__(self, name: str, dept: int) -> None:
        self.name = name
        self.dept = dept
        
    def is_fulltime(self) -> bool:
        """Return True if employee is a Full-time employee, else False"""
        return self.dept > 25

The __init__ method receives multiple arguments to initialize a call. These arguments are set as class instance attributes.

With Python 3.7, you have dataclasses, which make class declarations easier and more readable. You can use the @dataclass decorator on the class declaration and self-assignment is taken care of automatically. It generates __init__, __repr__, __eq__, __hash__, and other special methods. In Python 3.7, the Employee class defined earlier looks like the following:

@dataclass
Class Employee:
    name: str
    dept: int
    
    def is_fulltime(self) -> bool:
        """Return True if employee is a Full-time employee, else False"""
        return self.dept > 25

Customization of access to module attributes

Attributes are widely used in Python. Most commonly, they used in classes. However, attributes can be put on functions and modules as well. Attributes are retrieved using the dot notation: something.attribute. You can also get attributes that are named at runtime using the getattr() function.

For classes, something.attr first looks for attr defined on something. If it’s not found, then the special method something.__getattr__(“attr”) is called. The .getattr__() function can be used to customize access to attributes on objects. This customization is not easily available for module attributes, until Python 3.7. But PEP 562 provides __getattr__() on modules, along with a corresponding __dir__() function.

Typing enhancements

Type annotations are commonly used for code hints. However, there were two common issues with using type hints extensively in the code:

  • Annotations could only use names that were already available in the current scope. In other words, they didn’t support forward references.
  • Annotating source code had adverse effects on the startup time of Python programs.

Both of these issues are fixed in Python 3.7, by postponing the evaluation of annotations. Instead of compiling code that executes expressions in annotations at their definition time, the compiler stores the annotation in a string form equivalent to the AST of the expression in question.

For example, the following code fails, as spouse cannot be defined as type Employee, given that Employee is not defined yet.

class Employee:
    def __init__(self, name: str, spouse: Employee) --> None
        pass

In Python 3.7, the evaluation of annotation is postponed. It gets stored as a string and optionally evaluated as needed. You do need to import __future__, which breaks the backward compatibility of the code with previous versions of Python.

from __future__ import annotations

class Employee:
    def __init__(self, name: str, spouse: Employee) --> None
        pass

Time functions with nanosecond resolution

The time module gets some new functions in Python 3.7. The clock resolution can exceed the limited precision of a floating point number returned by the time.time() function and its variants. The following new functions are being added:

  • clock_gettime_ns()
  • clock_settime_ns()
  • monotonic_ns()
  • perf_counter_ns()
  • process_time_ns()
  • time_ns()

These functions are similar to already existing functions without the _ns suffix. The difference is that the above functions return a number of nanoseconds as an int instead of a number of seconds as a float.

For more information, see the AWS Lambda Developer Guide.

Hope you enjoy… go build with Python 3.7!

CTFR – Abuse Certificate Transparency Logs For HTTPS Subdomains

Post Syndicated from Darknet original https://www.darknet.org.uk/2018/10/ctfr-abuse-certificate-transparency-logs-for-https-subdomains/?utm_source=rss&utm_medium=social&utm_campaign=darknetfeed

CTFR – Abuse Certificate Transparency Logs For HTTPS Subdomains

CTFR is a Python-based tool to Abuse Certificate Transparency Logs to get subdomains from a HTTPS website in a few seconds.

You missed AXFR technique didn’t you? (Open DNS zone transfers), so how does it work? CTFR does not use dictionary attack or brute-force attacks, it just helps you to abuse Certificate Transparency Logs.

What is Certificate Transparency?

Google’s Certificate Transparency project fixes several structural flaws in the SSL certificate system, which is the main cryptographic system that underlies all HTTPS connections.

Read the rest of CTFR – Abuse Certificate Transparency Logs For HTTPS Subdomains now! Only available at Darknet.

MagPi 74: Build a Raspberry Pi laptop!

Post Syndicated from Rob Zwetsloot original https://www.raspberrypi.org/blog/magpi-74-build-a-raspberry-pi-laptop/

Hey folks! Rob from The MagPi here with the good news that a brand new issue is out today, with a slightly new look. The MagPi 74 shows you how to build a Pi‑powered laptop, and gives tips on how to recycle an old laptop to use with Pi.

magpi 74

The laptop is not spooky, but the Halloween projects definitely are

We’ve got a pretty simple, tiny laptop build that you can follow along with, which will easily slip into your pocket once it’s completed. We also cover the basic Raspberry Pi Desktop experience, in case you fancy installing the x86 version to bring new life to an old laptop.

Welcome, foolish mortals…

I’m also very happy to announce that The MagPi Halloween projects feature is back this year! Put together by yours truly, Haunted Halloween Hacks should get you in the mood for the spookiest time of the year. October is the only month of the year that I’m allowed to make puns, so prepare yourself for some ghastly groaners.

magpi 74

Rob has unleashed his awful alliteration skills this issue, with some putrid puns

Still want more?

On top of all that, you can find more fantastic guides on making games in Python and in C/C++, along with our brand new Quickstart guide, a review of the latest Picade, and more inspiring projects than you can shake a Pi Zero at.

Qwerty the fish keeps this garden growing

magpi 74

Start making a Space Invaders clone with Pygame!

Get The MagPi 74

You can get The MagPi 74 today from WHSmith, Tesco, Sainsbury’s, and Asda. If you live in the US, head over to your local Barnes & Noble or Micro Center in the next few days for a print copy. You can also get the new issue online from our store, or digitally via our Android or iOS apps. And don’t forget, there’s always the free PDF as well.

Rolling subscription offer!

Want to support the Raspberry Pi Foundation and the magazine? You can now take out a monthly £5 subscription to the magazine, effectively creating a rolling pre‑order system that saves you money on each issue.

The MagPi subscription offer — The MagPi 74

You can also take out a twelve-month print subscription and get a Pi Zero W plus case and adapter cables absolutely free! This offer does not currently have an end date.

We need you!

Issue 75 is next month, and we’re planning to showcase 75 amazing Raspberry Pi projects! We need your help to vote for the top 50, so please head to the voting page and choose your favourite project. Click on a project name to cast your vote for that project.

That’s it for now! Oh, and if you make any Raspberry Pi Halloween projects this year, send them to us on Twitter or via email.

The post MagPi 74: Build a Raspberry Pi laptop! appeared first on Raspberry Pi.

Networked knitting machine: not your average knit one, purl one

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/knitting-network-printer/

The moment we saw Sarah Spencer‘s knitted Stargazing tapestry, we knew we needed to know more. A couple of emails later, and here’s Sarah with a guest blog post telling you all you need to know about her hacking adventure with a 1980s knitting machine and a Raspberry Pi.

Knitting Printer! (slowest speed)

Printing a scarf on a Brother KM950i knitting machine from the 1980’s. To do this I have a Brother Motor arm to push the carriage back and forth and a homemade colour changer that automatically selects the colour on the left (the white and purple device with the LED).

Here’s Sarah…

Raspberry Pi: what’s there not to like? It’s powerful, compact, and oh so affordable! I used one as a portable media box attached to a pico projector for years. Setting one up as a media box is one of the most popular uses for them, but there’s so much more you can do.

Cue a 1980s Brother domestic knitting machine. Yep, you read that right. A knitting machine – to knit jumpers, hats, scarves, you name it. They don’t make domestic knitting machines any more, so a machine from the 1980s is about as modern as you can get. It comes with an onboard scanner to scan knitting patterns and a floppy drive port to back up your scans to an old floppy disk. Aah, the eighties – what a time to be alive!

Building a networked knitting machine

But this is an article about Raspberry Pi, right? So what does a 30-year-old knitting machine have to do with that? Well, I hacked my domestic knitting machine and turned it into a network printer with the help of a Raspberry Pi. By using a floppy drive emulator written in Python and a web interface, I can send an image to the Raspberry Pi over the network, preview it in a knitting grid, and tell it to send the knitting pattern to the knitting machine via the floppy drive port.

Sarah Spencer Networked knitting machine

OctoKnit

I call this set-up OctoKnit in honour of a more famous and widely used tool, OctoPrint for 3D printers, another popular application for Raspberry Pi.

Sarah Spencer Knitting Network Printer

I’ve made the OctoKnit web interface open source. You can find it on GitHub.

This project has been in the works for several years, and there’s been a few modifications to the knitting machine over that time. With the addition of a motor arm and an automatic colour changer, my knitting is getting very close to being hands-free. Here’s a photo of the knitting machine today, although the Raspberry Pi is hiding behind the machine in this shot:

Sarah Spencer Networked knitting machine

I’ve specialised in knitting multicolour work using a double-layered technique called double Jacquard, which requires two beds of needles. Hence the reason the machine has doubled in size from when I first started.

Knitting for Etsy

I made a thing that can make things, so I need to make something with it, right? Here are a few custom orders I’ve completed through my Etsy store:

Sarah Spencer Networked knitting machine

Stargazing

However, none of my previous works quite compares to my latest piece, Stargazing: a knitted tapestry. Knitted in seven panels stitched together by hand, the pattern on the Raspberry Pi is 21 times bigger than the memory available on the vintage knitting machine, so it’s knitted in 21 separate but seamless file transfers. It took over 100 hours of work and weighs 15kg.

Sarah Spencer Networked knitting machine

Stargazing is a celestial map of the night sky, featuring all 88 constellations across both Northern and Southern hemispheres. The line through the center is the Earth’s equator, projected out into space, with the sun, moon and planets of our solar system featured along it. The grey cloud is a representation of our galaxy, the Milky Way.

Heart of Pluto on Twitter

Happy 6pm, Fri 31st Aug 2018 😊 The tapestry is installed and the planets in the sky have now aligned with those in the knitting

When I first picked up a Raspberry Pi and turned it over in my hand, marvelling at the computing power in such a small, affordable unit, I never imagined in my wildest dreams what I’d end up doing with it.

What will you do with your Raspberry Pi?

The post Networked knitting machine: not your average knit one, purl one appeared first on Raspberry Pi.

MagPi 73: make a video game!

Post Syndicated from Rob Zwetsloot original https://www.raspberrypi.org/blog/magpi-73-make-video-game/

Hi folks, Rob from The MagPi here! As far back as I can remember, I always wanted to learn to code to make a video game. I’m technically working on one right now! It’s wildly behind my self-imposed schedule, though. If you too wish to learn how to make games, then check out issue 73 of The MagPi, out today!

The MagPi 73

Make video games in the latest issue of The MagPi!

Let’s play a game

There are many classifications of video games these days, and many tools to help make it easy. We take you through making a purely narrative experience on Twine, up to programming a simple 8-bit game for Pico-8 in this month’s main feature. Don’t forget our ongoing series on how to make games in C/C++ and Pygame as well!

The MagPi 73

Make games today on your Pi!

Boost your home security

If making games aren’t quite your thing, then we also have a feature for our more serious-sided readers on how to secure your home using a Raspberry Pi. We show you how to set up a CCTV camera, an IoT doorbell, and a door security monitor too.

Home security made easy with a Raspberry Pi

Maker Faire Tokyo

We also have a bumper five pages on Maker Faire Tokyo and the Japanese Raspberry Pi community! I went out there earlier this month and managed to drag myself away from the Gundam Base and the Mandarake in Akihabara long enough to see some of the incredible and inventive things Japanese makers had created.

The MagPi 73

See our report from Maker Faire Tokyo!

All of this along with our usual selection of tutorials, projects, and reviews? We spoil you.

The MagPi 73

Amazing projects to inspire!

Get The MagPi 73

You can get The MagPi 72 today from WHSmith, Tesco, Sainsbury’s, and Asda. If you live in the US, head over to your local Barnes & Noble or Micro Center in the next few days for a print copy. You can also get the new issue online from our store, or digitally via our Android or iOS apps. And don’t forget, there’s always the free PDF as well.

Rolling subscription offer!

Want to support the Raspberry Pi Foundation and the magazine? You can now take out a monthly £5 subscription to the magazine, effectively creating a rolling pre-order system that saves you money on each issue.

The MagPi subscription offer — The MagPi 73

You can also take out a twelve-month print subscription and get a Pi Zero W plus case and adapter cables absolutely free! This offer does not currently have an end date.

That’s it for now, see ya real soon!

Edit: I’m sure he’ll run out of Star Trek GIFs eventually – Alex

The post MagPi 73: make a video game! appeared first on Raspberry Pi.

How to Efficiently Extract and Query Tagged Resources Using the AWS Resource Tagging API and S3 Select (SQL)

Post Syndicated from Marcilio Mendonca original https://aws.amazon.com/blogs/architecture/how-to-efficiently-extract-and-query-tagged-resources-using-the-aws-resource-tagging-api-and-s3-select-sql/

AWS customers can use tags to assign metadata to their AWS resources. Each tag is a simple label consisting of a customer-defined key and an optional value that can make it easier to manage, search for, and filter resources. Although there are no inherent types of tags, they enable customers to categorize resources by multiple criteria such as purpose, owner and, environment.

Once a tagging strategy is defined and enforced, customers can use the AWS Tag Editor to view and manage tags on their AWS resources, regardless of service or region. They can use the tag editor to search for resources by resource type, region, or tag, and then manage the tags applied to those resources.

However, customers have asked for guidance on how to build custom automation mechanisms to extract and query tagged resources so that they can extend the built-in functionalities of the Tag Editor. For instance, customers can build automation to generate custom CSV files for tagged resources and perhaps use SQL to query those resources. In addition, automation allows customers to add validation checks to their CI/CD deployment pipelines, for instance, to check whether resources have been properly tagged.

In this blog post, we introduce a simple yet efficient AWS architecture for extracting and querying tagged resources based on AWS cloud-native features such as the Resource Tagging API and S3 Select. We provide sample code for the architecture discussed that can help customers to customize and/or extend the architecture for their own purpose. By relying on AWS cloud-native features, customers can save time and reduce costs while still being able to do customizations.

For customers unfamiliar with the Resource Tagging API and the S3 Select features, below is a very brief introduction.

Resource Tagging API
AWS customers can use the Resource Tagging API to programatically access the same resource group operations that had been accessible only from the AWS Management Console by now using the AWS SDKs or the AWS Command Line Interface (CLI). By doing so, customers can build automation that fits their need, e.g., code that extract, export, and queries tagged resources.

For further details, please read Resource Groups Tagging – Reference

S3 Select
S3 Select enables applications to retrieve only a subset of data from an object by using simple SQL expressions. By using S3 Select to retrieve only the data needed by the application, customers can achieve drastic performance increases – in many cases you can get as much as a 400% improvement.

For further details, please read:

The Overall Solution Architecture

The figure above depict the overall architecture discussed in this post. It is a simple yet efficient architecture for extracting and querying tagged resources based on AWS cloud-native features. The Resource Tagging API is used to extract tagged resources from one or more AWS accounts via the Python AWS SDK, then a custom CSV file is generated and pushed to S3. Once in S3, the tagged resources file can now be efficiently queried via S3 Select also using Python AWS SDK. By leveraging S3 Select, we can now use SQL to query tagged resources and save on S3 data transfer costs since only the filtered results will be returned directly from S3. Pretty neat, eh?

The Extract Process
The extract process was built using Python 3 and relies on the Resource Tagging API to fetch pages of tagged resources and export them to CSV using the csv Python library.

We start importing the required libraries (boto3 is the AWS SDK for Python, argparse helps managing input parameters, and csv supports building valid CSV files):

import boto3
import argparse
import csv

Then, we define the header columns to use when generating the CSV files containing all tagged resources and the writeToCsv function:

field_names = ['ResourceArn', 'TagKey', 'TagValue']

def writeToCsv(writer, args, tag_list):
    for resource in tag_list:
        print("Extracting tags for resource: " +
              resource['ResourceARN'] + "...")
        for tag in resource['Tags']:
            row = dict(
                ResourceArn=resource['ResourceARN'], TagKey=tag['Key'], TagValue=tag['Value'])
            writer.writerow(row)

We take the CSV output file path as a required parameter so that users can specificy the desired output file name using the argparse library:

def input_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("--output", required=True,
                        help="Output CSV file (eg, /tmp/tagged-resources.csv)")
    return parser.parse_args()

And then, we implement the main extract logic that uses the Resource Tagging API (see boto3.client(‘resourcegroupstaggingapi’) in the code below). Note that we fetch 50 resources at a time and write them to the CSV output file until no more resources are found.

def main():
    args = input_args()
    restag = boto3.client('resourcegroupstaggingapi')
    with open(args.output, 'w') as csvfile:
        writer = csv.DictWriter(csvfile, quoting=csv.QUOTE_ALL,
                                delimiter=',', dialect='excel', fieldnames=field_names)
        writer.writeheader()
        response = restag.get_resources(ResourcesPerPage=50)
        writeToCsv(writer, args, response['ResourceTagMappingList'])
        while 'PaginationToken' in response and response['PaginationToken']:
            token = response['PaginationToken']
            response = restag.get_resources(
                ResourcesPerPage=50, PaginationToken=token)
            writeToCsv(writer, args, response['ResourceTagMappingList'])

if __name__ == '__main__':
    main()

The extract procedure is pretty simple and illustrates well how to use the Resource Tagging API to customize the output. It will also use the default credentials in your account.

Here is how the extract process can be triggered for the QA account (assuming the python source file is named aws-tagged-resources-extractor.py and that there is a QA_AWS_ACCOUNT AWS profile defined in your ~/.aws/credentials file).

export AWS_PROFILE=QA_AWS_ACCOUNT
python aws-tagged-resources-extractor.py --output /tmp/qa-tagged-resources.csv

The extract procedure can be applied to other AWS accounts by updating the AWS_PROFILE environment variable accordingly.

The extract procedure can be applied to other AWS accounts by updating the AWS_PROFILE environment variable accordingly.

The ‘Upload to S3’ Process
Once file /tmp/qa-tagged-resources.csv is generated, it can be upload to an S3 bucket using the AWS CLI (or one could extend the extract sample code above to do so):

aws s3 cp /tmp/qa-tagged-resources.csv s3://[REPLACE-WITH-YOUR-S3-BUCKET]

The Query Process
Once the CSV files containing tagged resources for different AWS accounts are uploaded to S3, we can now use S3 Select to perform familiar SQL queries against these files. Another advantage of using S3 Select is that it reduces the amount of data transferred from S3 which is especially relevant in our case when accounts have a very large number of tagged resources.

We again use the boto3 and argparse libraries (Python 3). Required input parameters include the S3 bucket (–bucket) and the S3 key (–key). The SQL query parameter (–query) is optional and will return all results if not provided.

import boto3
import argparse

def input_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("--bucket", required=True, help="SQL query to filter tagged resources output")
    parser.add_argument("--key", required=True, help="SQL query to filter tagged resources output")
    parser.add_argument("--query", default="select * from s3object", help="SQL query to filter tagged resources output")
    return parser.parse_args()

The main query logic is shown below. It uses the boto3.client(‘s3’) to initialize an s3 client that is later used to query the tagged resources CSV file in S3 via the select_object_content() function. This function takes the S3 bucket name, S3 key, and query as parameters. Check the [Boto3] (http://boto3.readthedocs.io/en/latest/reference/services/s3.html) API reference for details on this function and its inputs and outputs.

def main():
    args = input_args()
    s3 = boto3.client('s3')
    response = s3.select_object_content(
        Bucket=args.bucket,
        Key=args.key,
        ExpressionType='SQL',
        Expression=args.query,
        InputSerialization = {'CSV': {"FileHeaderInfo": "Use"}},
        OutputSerialization = {'CSV': {}},
    )

    for event in response['Payload']:
        if 'Records' in event:
            records = event['Records']['Payload'].decode('utf-8')
            print(records)
            
if __name__ == '__main__':
    main()

Here’s a few examples of how to trigger the query procedure against the CSV files stored in S3 (assuming the Python source file for the query procedure is called aws-tagged-resources-querier). We assume that the S3 bucket is located in a single account referenced by profile CENTRAL_AWS_ACCOUNT.

Return the resource ARNs of all route tables containing a tag named ‘aws:cloudformation:stack-name’ in the QA AWS account

export AWS_PROFILE= CENTRAL_AWS_ACCOUNT
python aws-tagged-resources-querier \
     --bucket [REPLACE-WITH-YOUR-S3-BUCKET] \
     --key qa-tagged-resources.csv \
     --query "select ResourceArn from s3object s \
              where s.ResourceArn like 'arn:aws:ec2%route-table%' \
                and s.TagKey='aws:cloudformation:stack-name'"

We invite readers to build more sophisticated SQL queries.

Summary
In this blog post, we introduced a simple yet efficient AWS architecture for extracting and querying tagged resources based on AWS cloud-native features such as the Resource Tagging API and S3 Select. We provided sample code that can help customers to customize and/or extend the architecture for their own purpose. By relying on AWS cloud-native features, customers can save time and reduce costs while still being able to do customizations.

The “extract” process discussed above is available in the AWS Serverless Repository under an application called aws-tag-explorer. Check it out!

Happy Resource Tagging!

About the Author

Marcilio Mendonca is a Sr. Consultant in the Global DevOps Team at AWS Professional Services. In the past years, he has been helping AWS customers to design, build and deploy best-in-class cloud-native AWS applications using VMs, containers and serverless architectures. Prior to joining AWS, Marcilio was a Software Development Engineer with Amazon. Marcilio also holds a PhD in Computer Science.

 

 

Cangibrina – Admin Dashboard Finder Tool

Post Syndicated from Darknet original https://www.darknet.org.uk/2018/08/cangibrina-admin-dashboard-finder-tool/?utm_source=rss&utm_medium=social&utm_campaign=darknetfeed

Cangibrina – Admin Dashboard Finder Tool

Cangibrina is a Python-based multi platform admin dashboard finder tool which aims to obtain the location of website dashboards by using brute-force, wordlists, Google, Nmap and robots.txt.

It is multi-threaded, supports modifying your user agent, using a TOR proxy, custom dorks, Nmap integration and can use both DuckDuckGo and Google.

Cangibrina Admin Dashboard Finder Requirements

  • Python 2.7
  • mechanize
  • PySocks
  • beautifulsoup4
  • html5lib
  • Nmap
  • TOR

Cangibrina Usage to Find Admin Dashboards

usage: cangibrina.py [-h] -u U [-w W] [-t T] [-v] [–ext EXT] [–user-agent]
[–tor] [–search] [–dork DORK] [–nmap [NMAP]]

Fast and powerful admin finder

optional arguments:
-h, –help show this help message and exit
-u U target site
-w W set wordlist (default: wl_medium)
-t T set threads number (default: 5)
-v enable verbose
–ext EXT filter path by target extension
–user-agent modify user-agent
–sub-domain search for sub domains instead of directories
–tor set TOR proxy
–search use google and duckduckgo to search
–dork DORK set custom dork
–nmap [NMAP] use nmap to scan ports and services

There are other specific tools in this area like WPScan for WordPress and DruPwn for Drupal – and in those cases the dashboard URLs are already known.

Read the rest of Cangibrina – Admin Dashboard Finder Tool now! Only available at Darknet.

Mu, a new Python IDE for beginners

Post Syndicated from Martin O'Hanlon original https://www.raspberrypi.org/blog/mu-python-ide/

Mu is a very simple-to-use Python editor and IDE (integrated development environment) and this week, version 1.0 was released!

Mu Python IDE for beginners Raspberry Pi

New Mu

Mu is designed to be as user-friendly and as helpful as possible for new Python programmers, presenting just the tools that are useful, such as:

  • Syntax highlighting
  • Automatic indentation
  • In-built help
  • Code checking
  • Debugging

Great for new programmers

Mu is intended to be not the only Python IDE you’ll ever need, but the first one — the editor that helps you start your coding journey, but not necessarily the one you finish it with. So when you’re ready, you will have the skills and confidence to move on to using a more advanced Python IDE.

You can use Mu in a number of modes; modes make working with Mu easier by only presenting the options most relevant to what you’re using Mu for:

Mu Python IDE for beginners Raspberry Pi

Available now

Mu version 1.0 is available now for Windows, macOS, Linux, and the Raspberry Pi’s official operating system Raspbian! And to help new Python programmers get started, we have created a guide to Getting Started with Mu for all these operating systems.

Mu Python IDE for beginners Raspberry Pi

Mu is the brainchild of Nicholas Tollervey, who has worked tirelessly to create Mu. I recently met up with him and some of the Mu team at the world’s first Mu-“moot” to celebrate this release:

Nicholas Tollervey on Twitter

World’s first Mu-moot. 🙁

One of the inspirations for Mu was the keynote presentation at EuroPython 2015 given by Raspberry Pi’s Carrie Anne Philbin. She talked about the barriers to children getting started with Python, including the lack of an suitably easy-to-use IDE:

Carrie Anne Philbin – Keynote: Designed for Education: A Python Solution

Carrie Anne Philbin – Keynote: Designed for Education: A Python Solution [EuroPython 2015] [23 July 2015] [Bilbao, Euskadi, Spain] The problem of introducing children to programming and computer science has seen growing attention in the past few years. Initiatives like Raspberry Pi, Code Club, code.org, (and many more) have been created to help solve this problem.

Raspberry Pi has provided support for the project, helping to take Mu from its first implementation as a micro:bit programming tool to a general-purpose and simple-to-use Python editor and IDE!

You can find installation instructions as well as tutorials on Mu’s website.

The post Mu, a new Python IDE for beginners appeared first on Raspberry Pi.