Tag Archives: python

MagPi 77: Make with code

Post Syndicated from Rob Zwetsloot original https://www.raspberrypi.org/blog/magpi-77/

Hey folks, Rob from The MagPi here! Before I head off on my Christmas holidays, I want to introduce you to The MagPi 77, where we teach you how to make with code.

Making made fun! See what we did there?

What do we mean by that? Well, using code to make things – whether that’s scripts, programs, or games on your Pi, or whether you’re controlling LEDs with code, or robots, or massive Rube Goldberg machines. In this feature, we show new Pi users how to get started making practical applications with Python, and hopefully you’ll be inspired to go on and do something special.

Can you make… with code?

Accessories make the Pi

Want to power up your Raspberry Pi with a few extras? We’ve put together a guide to the 20 best Raspberry Pi accessories, covering IoT, robots, media, power solutions, and even industrial add-ons. There’s a lot of stuff you can do with your Pi, and even more if you’ve got the right tool to help.

We have the best accessories for you

More, you say?

Still need more reasons to grab a copy? Well, we have a tutorial on how to make a smart door, we continue developing Pac-Man while checking out the Picade Console, and we have plenty of amazing project showcases like the SelfieBot!

Get The MagPi 77

You can get The MagPi 77 from WHSmith, Tesco, Sainsbury’s, and Asda. If you live in the US, head over to your local Barnes & Noble or Micro Center in the next few days for a print copy. You can also get the issue online: check it out on our store, or digitally via our Android or iOS apps. And don’t forget, there’s always the free PDF.

Free Raspberry Pi 3A+ offer!

We’re still running our super special Raspberry Pi 3A+ subscription offer! If you subscribe to twelve months of The MagPi, you’ll get a Raspberry Pi 3A+ completely free while stocks last. Make sure to check out our other subs offers while you’re there, like three issues for £5, and our rolling monthly sub.

Get a 3A+ completely free while stocks last!

Right, happy holidays, folks! See you all in the New Year!

The post MagPi 77: Make with code appeared first on Raspberry Pi.

New – AWS Toolkits for PyCharm, IntelliJ (Preview), and Visual Studio Code (Preview)

Post Syndicated from Danilo Poccia original https://aws.amazon.com/blogs/aws/new-aws-toolkits-for-pycharm-intellij-preview-and-visual-studio-code-preview/

Software developers have their own preferred tools. Some use powerful editors, others Integrated Development Environments (IDEs) that are tailored for specific languages and platforms. In 2014 I created my first AWS Lambda function using the editor in the Lambda console. Now, you can choose from a rich set of tools to build and deploy serverless applications. For example, the editor in the Lambda console has been greatly enhanced last year when AWS Cloud9 was released. For .NET applications, you can use the AWS Toolkit for Visual Studio and AWS Tools for Visual Studio Team Services.

AWS Toolkits for PyCharm, IntelliJ, and Visual Studio Code

Today, we are announcing the general availability of the AWS Toolkit for PyCharm. We are also announcing the developer preview of the AWS Toolkits for IntelliJ and Visual Studio Code, which are under active development in GitHub. These open source toolkits will enable you to easily develop serverless applications, including a full create, step-through debug, and deploy experience in the IDE and language of your choice, be it Python, Java, Node.js, or .NET.

For example, using the AWS Toolkit for PyCharm you can:

These toolkits are distributed under the open source Apache License, Version 2.0.


Some features use the AWS Serverless Application Model (SAM) CLI. You can find installation instructions for your system here.

The AWS Toolkit for PyCharm is available via the IDEA Plugin Repository. To install it, in the Settings/Preferences dialog, click Plugins, search for “AWS Toolkit”, use the checkbox to enable it, and click the Install button. You will need to restart your IDE for the changes to take effect.

The AWS Toolkit for IntelliJ and Visual Studio Code are currently in developer preview and under active development. You are welcome to build and install these from the GitHub repositories:

Building a Serverless application with PyCharm

After installing AWS SAM CLI and AWS Toolkit, I create a new project in PyCharm and choose SAM on the left to create a serverless application using the AWS Serverless Application Model. I call my project hello-world in the Location field. Expanding More Settings, I choose which SAM template to use as the starting point for my project. For this walkthrough, I select the “AWS SAM Hello World”.

In PyCharm you can use credentials and profiles from your AWS Command Line Interface (CLI) configuration. You can change AWS region quickly if you have multiple environments.
The AWS Explorer shows Lambda functions and AWS CloudFormation stacks in the selected AWS region. Starting from a CloudFormation stack, you can see which Lambda functions are part of it.

The function handler is in the app.py file. After I open the file, I click on the Lambda icon on the left of the function declaration to have the option to run the function locally or start a local step-by-step debugging session.

First, I run the function locally. I can configure the payload of the event that is provided in input for the local invocation, starting from the event templates provided for most services, such as the Amazon API Gateway, Amazon Simple Notification Service (SNS), Amazon Simple Queue Service (SQS), and so on. You can use a file for the payload, or select the share checkbox to make it available to other team members. The function is executed locally, but here you can choose the credentials and the region to be used if the function is calling other AWS services, such as Amazon Simple Storage Service (S3) or Amazon DynamoDB.

A local container is used to emulate the Lambda execution environment. This function is implementing a basic web API, and I can check that the result is in the format expected by the API Gateway.

After that, I want to get more information on what my code is doing. I set a breakpoint and start a local debugging session. I use the same input event as before. Again, you can choose the credentials and region for the AWS services used by the function.

I step over the HTTP request in the code to inspect the response in the Variables tab. Here you have access to all local variables, including the event and the context provided in input to the function.

After that, I resume the program to reach the end of the debugging session.

Now I am confident enough to deploy the serverless application right-clicking on the project (or the SAM template file). I can create a new CloudFormation stack, or update an existing one. For now, I create a new stack called hello-world-prod. For example, you can have a stack for production, and one for testing. I select an S3 bucket in the region to store the package used for the deployment. If your template has parameters, here you can set up the values used by this deployment.

After a few minutes, the stack creation is complete and I can run the function in the cloud with a right-click in the AWS Explorer. Here there is also the option to jump to the source code of the function.

As expected, the result of the remote invocation is the same as the local execution. My serverless application is in production!

Using these toolkits, developers can test locally to find problems before deployment, change the code of their application or the resources they need in the SAM template, and update an existing stack, quickly iterating until they reach their goal. For example, they can add an S3 bucket to store images or documents, or a DynamoDB table to store your users, or change the permissions used by their functions.

I am really excited by how much faster and easier it is to build your ideas on AWS. Now you can use your preferred environment to accelerate even further. I look forward to seeing what you will do with these new tools!

Malcom – Malware Communication Analyzer

Post Syndicated from Darknet original https://www.darknet.org.uk/2018/11/malcom-malware-communication-analyzer/?utm_source=rss&utm_medium=social&utm_campaign=darknetfeed

Malcom – Malware Communication Analyzer

Malcom is a Malware Communication Analyzer designed to analyze a system’s network communication using graphical representations of network traffic, and cross-reference them with known malware sources.

This comes handy when analyzing how certain malware species try to communicate with the outside world.

Malcom Malware Communication Analyzer Features

Malcom can help you:

  • Detect central command and control (C&C) servers
  • Understand peer-to-peer networks
  • Observe DNS fast-flux infrastructures
  • Quickly determine if a network artifact is ‘known-bad’

The aim of Malcom is to make malware analysis and intel gathering faster by providing a human-readable version of network traffic originating from a given host or network.

Read the rest of Malcom – Malware Communication Analyzer now! Only available at Darknet.

Python 3.7 runtime now available in AWS Lambda

Post Syndicated from Chris Munns original https://aws.amazon.com/blogs/compute/python-3-7-runtime-now-available-in-aws-lambda/

This post is courtesy of Shivansh Singh, Partner Solutions Architect – AWS

We are excited to announce that you can now develop your AWS Lambda functions using the Python 3.7 runtime. Start using this new version today by specifying a runtime parameter value of “python3.7″ when creating or updating functions. AWS continues to support creating new Lambda functions on Python 3.6 and Python 2.7.

Here’s a quick primer on some of the major new features in Python 3.7:

  • Data classes
  • Customization of access to module attributes
  • Typing enhancements
  • Time functions with nanosecond resolution

Data classes

In object-oriented programming, if you have to create a class, it looks like the following in Python 3.6:

class Employee:
    def __init__(self, name: str, dept: int) -> None:
        self.name = name
        self.dept = dept
    def is_fulltime(self) -> bool:
        """Return True if employee is a Full-time employee, else False"""
        return self.dept > 25

The __init__ method receives multiple arguments to initialize a call. These arguments are set as class instance attributes.

With Python 3.7, you have dataclasses, which make class declarations easier and more readable. You can use the @dataclass decorator on the class declaration and self-assignment is taken care of automatically. It generates __init__, __repr__, __eq__, __hash__, and other special methods. In Python 3.7, the Employee class defined earlier looks like the following:

Class Employee:
    name: str
    dept: int
    def is_fulltime(self) -> bool:
        """Return True if employee is a Full-time employee, else False"""
        return self.dept > 25

Customization of access to module attributes

Attributes are widely used in Python. Most commonly, they used in classes. However, attributes can be put on functions and modules as well. Attributes are retrieved using the dot notation: something.attribute. You can also get attributes that are named at runtime using the getattr() function.

For classes, something.attr first looks for attr defined on something. If it’s not found, then the special method something.__getattr__(“attr”) is called. The .getattr__() function can be used to customize access to attributes on objects. This customization is not easily available for module attributes, until Python 3.7. But PEP 562 provides __getattr__() on modules, along with a corresponding __dir__() function.

Typing enhancements

Type annotations are commonly used for code hints. However, there were two common issues with using type hints extensively in the code:

  • Annotations could only use names that were already available in the current scope. In other words, they didn’t support forward references.
  • Annotating source code had adverse effects on the startup time of Python programs.

Both of these issues are fixed in Python 3.7, by postponing the evaluation of annotations. Instead of compiling code that executes expressions in annotations at their definition time, the compiler stores the annotation in a string form equivalent to the AST of the expression in question.

For example, the following code fails, as spouse cannot be defined as type Employee, given that Employee is not defined yet.

class Employee:
    def __init__(self, name: str, spouse: Employee) --> None

In Python 3.7, the evaluation of annotation is postponed. It gets stored as a string and optionally evaluated as needed. You do need to import __future__, which breaks the backward compatibility of the code with previous versions of Python.

from __future__ import annotations

class Employee:
    def __init__(self, name: str, spouse: Employee) --> None

Time functions with nanosecond resolution

The time module gets some new functions in Python 3.7. The clock resolution can exceed the limited precision of a floating point number returned by the time.time() function and its variants. The following new functions are being added:

  • clock_gettime_ns()
  • clock_settime_ns()
  • monotonic_ns()
  • perf_counter_ns()
  • process_time_ns()
  • time_ns()

These functions are similar to already existing functions without the _ns suffix. The difference is that the above functions return a number of nanoseconds as an int instead of a number of seconds as a float.

For more information, see the AWS Lambda Developer Guide.

Hope you enjoy… go build with Python 3.7!

CTFR – Abuse Certificate Transparency Logs For HTTPS Subdomains

Post Syndicated from Darknet original https://www.darknet.org.uk/2018/10/ctfr-abuse-certificate-transparency-logs-for-https-subdomains/?utm_source=rss&utm_medium=social&utm_campaign=darknetfeed

CTFR – Abuse Certificate Transparency Logs For HTTPS Subdomains

CTFR is a Python-based tool to Abuse Certificate Transparency Logs to get subdomains from a HTTPS website in a few seconds.

You missed AXFR technique didn’t you? (Open DNS zone transfers), so how does it work? CTFR does not use dictionary attack or brute-force attacks, it just helps you to abuse Certificate Transparency Logs.

What is Certificate Transparency?

Google’s Certificate Transparency project fixes several structural flaws in the SSL certificate system, which is the main cryptographic system that underlies all HTTPS connections.

Read the rest of CTFR – Abuse Certificate Transparency Logs For HTTPS Subdomains now! Only available at Darknet.

MagPi 74: Build a Raspberry Pi laptop!

Post Syndicated from Rob Zwetsloot original https://www.raspberrypi.org/blog/magpi-74-build-a-raspberry-pi-laptop/

Hey folks! Rob from The MagPi here with the good news that a brand new issue is out today, with a slightly new look. The MagPi 74 shows you how to build a Pi‑powered laptop, and gives tips on how to recycle an old laptop to use with Pi.

magpi 74

The laptop is not spooky, but the Halloween projects definitely are

We’ve got a pretty simple, tiny laptop build that you can follow along with, which will easily slip into your pocket once it’s completed. We also cover the basic Raspberry Pi Desktop experience, in case you fancy installing the x86 version to bring new life to an old laptop.

Welcome, foolish mortals…

I’m also very happy to announce that The MagPi Halloween projects feature is back this year! Put together by yours truly, Haunted Halloween Hacks should get you in the mood for the spookiest time of the year. October is the only month of the year that I’m allowed to make puns, so prepare yourself for some ghastly groaners.

magpi 74

Rob has unleashed his awful alliteration skills this issue, with some putrid puns

Still want more?

On top of all that, you can find more fantastic guides on making games in Python and in C/C++, along with our brand new Quickstart guide, a review of the latest Picade, and more inspiring projects than you can shake a Pi Zero at.

Qwerty the fish keeps this garden growing

magpi 74

Start making a Space Invaders clone with Pygame!

Get The MagPi 74

You can get The MagPi 74 today from WHSmith, Tesco, Sainsbury’s, and Asda. If you live in the US, head over to your local Barnes & Noble or Micro Center in the next few days for a print copy. You can also get the new issue online from our store, or digitally via our Android or iOS apps. And don’t forget, there’s always the free PDF as well.

Rolling subscription offer!

Want to support the Raspberry Pi Foundation and the magazine? You can now take out a monthly £5 subscription to the magazine, effectively creating a rolling pre‑order system that saves you money on each issue.

The MagPi subscription offer — The MagPi 74

You can also take out a twelve-month print subscription and get a Pi Zero W plus case and adapter cables absolutely free! This offer does not currently have an end date.

We need you!

Issue 75 is next month, and we’re planning to showcase 75 amazing Raspberry Pi projects! We need your help to vote for the top 50, so please head to the voting page and choose your favourite project. Click on a project name to cast your vote for that project.

That’s it for now! Oh, and if you make any Raspberry Pi Halloween projects this year, send them to us on Twitter or via email.

The post MagPi 74: Build a Raspberry Pi laptop! appeared first on Raspberry Pi.

Networked knitting machine: not your average knit one, purl one

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/knitting-network-printer/

The moment we saw Sarah Spencer‘s knitted Stargazing tapestry, we knew we needed to know more. A couple of emails later, and here’s Sarah with a guest blog post telling you all you need to know about her hacking adventure with a 1980s knitting machine and a Raspberry Pi.

Knitting Printer! (slowest speed)

Printing a scarf on a Brother KM950i knitting machine from the 1980’s. To do this I have a Brother Motor arm to push the carriage back and forth and a homemade colour changer that automatically selects the colour on the left (the white and purple device with the LED).

Here’s Sarah…

Raspberry Pi: what’s there not to like? It’s powerful, compact, and oh so affordable! I used one as a portable media box attached to a pico projector for years. Setting one up as a media box is one of the most popular uses for them, but there’s so much more you can do.

Cue a 1980s Brother domestic knitting machine. Yep, you read that right. A knitting machine – to knit jumpers, hats, scarves, you name it. They don’t make domestic knitting machines any more, so a machine from the 1980s is about as modern as you can get. It comes with an onboard scanner to scan knitting patterns and a floppy drive port to back up your scans to an old floppy disk. Aah, the eighties – what a time to be alive!

Building a networked knitting machine

But this is an article about Raspberry Pi, right? So what does a 30-year-old knitting machine have to do with that? Well, I hacked my domestic knitting machine and turned it into a network printer with the help of a Raspberry Pi. By using a floppy drive emulator written in Python and a web interface, I can send an image to the Raspberry Pi over the network, preview it in a knitting grid, and tell it to send the knitting pattern to the knitting machine via the floppy drive port.

Sarah Spencer Networked knitting machine


I call this set-up OctoKnit in honour of a more famous and widely used tool, OctoPrint for 3D printers, another popular application for Raspberry Pi.

Sarah Spencer Knitting Network Printer

I’ve made the OctoKnit web interface open source. You can find it on GitHub.

This project has been in the works for several years, and there’s been a few modifications to the knitting machine over that time. With the addition of a motor arm and an automatic colour changer, my knitting is getting very close to being hands-free. Here’s a photo of the knitting machine today, although the Raspberry Pi is hiding behind the machine in this shot:

Sarah Spencer Networked knitting machine

I’ve specialised in knitting multicolour work using a double-layered technique called double Jacquard, which requires two beds of needles. Hence the reason the machine has doubled in size from when I first started.

Knitting for Etsy

I made a thing that can make things, so I need to make something with it, right? Here are a few custom orders I’ve completed through my Etsy store:

Sarah Spencer Networked knitting machine


However, none of my previous works quite compares to my latest piece, Stargazing: a knitted tapestry. Knitted in seven panels stitched together by hand, the pattern on the Raspberry Pi is 21 times bigger than the memory available on the vintage knitting machine, so it’s knitted in 21 separate but seamless file transfers. It took over 100 hours of work and weighs 15kg.

Sarah Spencer Networked knitting machine

Stargazing is a celestial map of the night sky, featuring all 88 constellations across both Northern and Southern hemispheres. The line through the center is the Earth’s equator, projected out into space, with the sun, moon and planets of our solar system featured along it. The grey cloud is a representation of our galaxy, the Milky Way.

Heart of Pluto on Twitter

Happy 6pm, Fri 31st Aug 2018 😊 The tapestry is installed and the planets in the sky have now aligned with those in the knitting

When I first picked up a Raspberry Pi and turned it over in my hand, marvelling at the computing power in such a small, affordable unit, I never imagined in my wildest dreams what I’d end up doing with it.

What will you do with your Raspberry Pi?

The post Networked knitting machine: not your average knit one, purl one appeared first on Raspberry Pi.

MagPi 73: make a video game!

Post Syndicated from Rob Zwetsloot original https://www.raspberrypi.org/blog/magpi-73-make-video-game/

Hi folks, Rob from The MagPi here! As far back as I can remember, I always wanted to learn to code to make a video game. I’m technically working on one right now! It’s wildly behind my self-imposed schedule, though. If you too wish to learn how to make games, then check out issue 73 of The MagPi, out today!

The MagPi 73

Make video games in the latest issue of The MagPi!

Let’s play a game

There are many classifications of video games these days, and many tools to help make it easy. We take you through making a purely narrative experience on Twine, up to programming a simple 8-bit game for Pico-8 in this month’s main feature. Don’t forget our ongoing series on how to make games in C/C++ and Pygame as well!

The MagPi 73

Make games today on your Pi!

Boost your home security

If making games aren’t quite your thing, then we also have a feature for our more serious-sided readers on how to secure your home using a Raspberry Pi. We show you how to set up a CCTV camera, an IoT doorbell, and a door security monitor too.

Home security made easy with a Raspberry Pi

Maker Faire Tokyo

We also have a bumper five pages on Maker Faire Tokyo and the Japanese Raspberry Pi community! I went out there earlier this month and managed to drag myself away from the Gundam Base and the Mandarake in Akihabara long enough to see some of the incredible and inventive things Japanese makers had created.

The MagPi 73

See our report from Maker Faire Tokyo!

All of this along with our usual selection of tutorials, projects, and reviews? We spoil you.

The MagPi 73

Amazing projects to inspire!

Get The MagPi 73

You can get The MagPi 72 today from WHSmith, Tesco, Sainsbury’s, and Asda. If you live in the US, head over to your local Barnes & Noble or Micro Center in the next few days for a print copy. You can also get the new issue online from our store, or digitally via our Android or iOS apps. And don’t forget, there’s always the free PDF as well.

Rolling subscription offer!

Want to support the Raspberry Pi Foundation and the magazine? You can now take out a monthly £5 subscription to the magazine, effectively creating a rolling pre-order system that saves you money on each issue.

The MagPi subscription offer — The MagPi 73

You can also take out a twelve-month print subscription and get a Pi Zero W plus case and adapter cables absolutely free! This offer does not currently have an end date.

That’s it for now, see ya real soon!

Edit: I’m sure he’ll run out of Star Trek GIFs eventually – Alex

The post MagPi 73: make a video game! appeared first on Raspberry Pi.

How to Efficiently Extract and Query Tagged Resources Using the AWS Resource Tagging API and S3 Select (SQL)

Post Syndicated from Marcilio Mendonca original https://aws.amazon.com/blogs/architecture/how-to-efficiently-extract-and-query-tagged-resources-using-the-aws-resource-tagging-api-and-s3-select-sql/

AWS customers can use tags to assign metadata to their AWS resources. Each tag is a simple label consisting of a customer-defined key and an optional value that can make it easier to manage, search for, and filter resources. Although there are no inherent types of tags, they enable customers to categorize resources by multiple criteria such as purpose, owner and, environment.

Once a tagging strategy is defined and enforced, customers can use the AWS Tag Editor to view and manage tags on their AWS resources, regardless of service or region. They can use the tag editor to search for resources by resource type, region, or tag, and then manage the tags applied to those resources.

However, customers have asked for guidance on how to build custom automation mechanisms to extract and query tagged resources so that they can extend the built-in functionalities of the Tag Editor. For instance, customers can build automation to generate custom CSV files for tagged resources and perhaps use SQL to query those resources. In addition, automation allows customers to add validation checks to their CI/CD deployment pipelines, for instance, to check whether resources have been properly tagged.

In this blog post, we introduce a simple yet efficient AWS architecture for extracting and querying tagged resources based on AWS cloud-native features such as the Resource Tagging API and S3 Select. We provide sample code for the architecture discussed that can help customers to customize and/or extend the architecture for their own purpose. By relying on AWS cloud-native features, customers can save time and reduce costs while still being able to do customizations.

For customers unfamiliar with the Resource Tagging API and the S3 Select features, below is a very brief introduction.

Resource Tagging API
AWS customers can use the Resource Tagging API to programatically access the same resource group operations that had been accessible only from the AWS Management Console by now using the AWS SDKs or the AWS Command Line Interface (CLI). By doing so, customers can build automation that fits their need, e.g., code that extract, export, and queries tagged resources.

For further details, please read Resource Groups Tagging – Reference

S3 Select
S3 Select enables applications to retrieve only a subset of data from an object by using simple SQL expressions. By using S3 Select to retrieve only the data needed by the application, customers can achieve drastic performance increases – in many cases you can get as much as a 400% improvement.

For further details, please read:

The Overall Solution Architecture

The figure above depict the overall architecture discussed in this post. It is a simple yet efficient architecture for extracting and querying tagged resources based on AWS cloud-native features. The Resource Tagging API is used to extract tagged resources from one or more AWS accounts via the Python AWS SDK, then a custom CSV file is generated and pushed to S3. Once in S3, the tagged resources file can now be efficiently queried via S3 Select also using Python AWS SDK. By leveraging S3 Select, we can now use SQL to query tagged resources and save on S3 data transfer costs since only the filtered results will be returned directly from S3. Pretty neat, eh?

The Extract Process
The extract process was built using Python 3 and relies on the Resource Tagging API to fetch pages of tagged resources and export them to CSV using the csv Python library.

We start importing the required libraries (boto3 is the AWS SDK for Python, argparse helps managing input parameters, and csv supports building valid CSV files):

import boto3
import argparse
import csv

Then, we define the header columns to use when generating the CSV files containing all tagged resources and the writeToCsv function:

field_names = ['ResourceArn', 'TagKey', 'TagValue']

def writeToCsv(writer, args, tag_list):
    for resource in tag_list:
        print("Extracting tags for resource: " +
              resource['ResourceARN'] + "...")
        for tag in resource['Tags']:
            row = dict(
                ResourceArn=resource['ResourceARN'], TagKey=tag['Key'], TagValue=tag['Value'])

We take the CSV output file path as a required parameter so that users can specificy the desired output file name using the argparse library:

def input_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("--output", required=True,
                        help="Output CSV file (eg, /tmp/tagged-resources.csv)")
    return parser.parse_args()

And then, we implement the main extract logic that uses the Resource Tagging API (see boto3.client(‘resourcegroupstaggingapi’) in the code below). Note that we fetch 50 resources at a time and write them to the CSV output file until no more resources are found.

def main():
    args = input_args()
    restag = boto3.client('resourcegroupstaggingapi')
    with open(args.output, 'w') as csvfile:
        writer = csv.DictWriter(csvfile, quoting=csv.QUOTE_ALL,
                                delimiter=',', dialect='excel', fieldnames=field_names)
        response = restag.get_resources(ResourcesPerPage=50)
        writeToCsv(writer, args, response['ResourceTagMappingList'])
        while 'PaginationToken' in response and response['PaginationToken']:
            token = response['PaginationToken']
            response = restag.get_resources(
                ResourcesPerPage=50, PaginationToken=token)
            writeToCsv(writer, args, response['ResourceTagMappingList'])

if __name__ == '__main__':

The extract procedure is pretty simple and illustrates well how to use the Resource Tagging API to customize the output. It will also use the default credentials in your account.

Here is how the extract process can be triggered for the QA account (assuming the python source file is named aws-tagged-resources-extractor.py and that there is a QA_AWS_ACCOUNT AWS profile defined in your ~/.aws/credentials file).

python aws-tagged-resources-extractor.py --output /tmp/qa-tagged-resources.csv

The extract procedure can be applied to other AWS accounts by updating the AWS_PROFILE environment variable accordingly.

The extract procedure can be applied to other AWS accounts by updating the AWS_PROFILE environment variable accordingly.

The ‘Upload to S3’ Process
Once file /tmp/qa-tagged-resources.csv is generated, it can be upload to an S3 bucket using the AWS CLI (or one could extend the extract sample code above to do so):

aws s3 cp /tmp/qa-tagged-resources.csv s3://[REPLACE-WITH-YOUR-S3-BUCKET]

The Query Process
Once the CSV files containing tagged resources for different AWS accounts are uploaded to S3, we can now use S3 Select to perform familiar SQL queries against these files. Another advantage of using S3 Select is that it reduces the amount of data transferred from S3 which is especially relevant in our case when accounts have a very large number of tagged resources.

We again use the boto3 and argparse libraries (Python 3). Required input parameters include the S3 bucket (–bucket) and the S3 key (–key). The SQL query parameter (–query) is optional and will return all results if not provided.

import boto3
import argparse

def input_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("--bucket", required=True, help="SQL query to filter tagged resources output")
    parser.add_argument("--key", required=True, help="SQL query to filter tagged resources output")
    parser.add_argument("--query", default="select * from s3object", help="SQL query to filter tagged resources output")
    return parser.parse_args()

The main query logic is shown below. It uses the boto3.client(‘s3’) to initialize an s3 client that is later used to query the tagged resources CSV file in S3 via the select_object_content() function. This function takes the S3 bucket name, S3 key, and query as parameters. Check the [Boto3] (http://boto3.readthedocs.io/en/latest/reference/services/s3.html) API reference for details on this function and its inputs and outputs.

def main():
    args = input_args()
    s3 = boto3.client('s3')
    response = s3.select_object_content(
        InputSerialization = {'CSV': {"FileHeaderInfo": "Use"}},
        OutputSerialization = {'CSV': {}},

    for event in response['Payload']:
        if 'Records' in event:
            records = event['Records']['Payload'].decode('utf-8')
if __name__ == '__main__':

Here’s a few examples of how to trigger the query procedure against the CSV files stored in S3 (assuming the Python source file for the query procedure is called aws-tagged-resources-querier). We assume that the S3 bucket is located in a single account referenced by profile CENTRAL_AWS_ACCOUNT.

Return the resource ARNs of all route tables containing a tag named ‘aws:cloudformation:stack-name’ in the QA AWS account

python aws-tagged-resources-querier \
     --bucket [REPLACE-WITH-YOUR-S3-BUCKET] \
     --key qa-tagged-resources.csv \
     --query "select ResourceArn from s3object s \
              where s.ResourceArn like 'arn:aws:ec2%route-table%' \
                and s.TagKey='aws:cloudformation:stack-name'"

We invite readers to build more sophisticated SQL queries.

In this blog post, we introduced a simple yet efficient AWS architecture for extracting and querying tagged resources based on AWS cloud-native features such as the Resource Tagging API and S3 Select. We provided sample code that can help customers to customize and/or extend the architecture for their own purpose. By relying on AWS cloud-native features, customers can save time and reduce costs while still being able to do customizations.

The “extract” process discussed above is available in the AWS Serverless Repository under an application called aws-tag-explorer. Check it out!

Happy Resource Tagging!

About the Author

Marcilio Mendonca is a Sr. Consultant in the Global DevOps Team at AWS Professional Services. In the past years, he has been helping AWS customers to design, build and deploy best-in-class cloud-native AWS applications using VMs, containers and serverless architectures. Prior to joining AWS, Marcilio was a Software Development Engineer with Amazon. Marcilio also holds a PhD in Computer Science.



Cangibrina – Admin Dashboard Finder Tool

Post Syndicated from Darknet original https://www.darknet.org.uk/2018/08/cangibrina-admin-dashboard-finder-tool/?utm_source=rss&utm_medium=social&utm_campaign=darknetfeed

Cangibrina – Admin Dashboard Finder Tool

Cangibrina is a Python-based multi platform admin dashboard finder tool which aims to obtain the location of website dashboards by using brute-force, wordlists, Google, Nmap and robots.txt.

It is multi-threaded, supports modifying your user agent, using a TOR proxy, custom dorks, Nmap integration and can use both DuckDuckGo and Google.

Cangibrina Admin Dashboard Finder Requirements

  • Python 2.7
  • mechanize
  • PySocks
  • beautifulsoup4
  • html5lib
  • Nmap
  • TOR

Cangibrina Usage to Find Admin Dashboards

usage: cangibrina.py [-h] -u U [-w W] [-t T] [-v] [–ext EXT] [–user-agent]
[–tor] [–search] [–dork DORK] [–nmap [NMAP]]

Fast and powerful admin finder

optional arguments:
-h, –help show this help message and exit
-u U target site
-w W set wordlist (default: wl_medium)
-t T set threads number (default: 5)
-v enable verbose
–ext EXT filter path by target extension
–user-agent modify user-agent
–sub-domain search for sub domains instead of directories
–tor set TOR proxy
–search use google and duckduckgo to search
–dork DORK set custom dork
–nmap [NMAP] use nmap to scan ports and services

There are other specific tools in this area like WPScan for WordPress and DruPwn for Drupal – and in those cases the dashboard URLs are already known.

Read the rest of Cangibrina – Admin Dashboard Finder Tool now! Only available at Darknet.

Mu, a new Python IDE for beginners

Post Syndicated from Martin O'Hanlon original https://www.raspberrypi.org/blog/mu-python-ide/

Mu is a very simple-to-use Python editor and IDE (integrated development environment) and this week, version 1.0 was released!

Mu Python IDE for beginners Raspberry Pi

New Mu

Mu is designed to be as user-friendly and as helpful as possible for new Python programmers, presenting just the tools that are useful, such as:

  • Syntax highlighting
  • Automatic indentation
  • In-built help
  • Code checking
  • Debugging

Great for new programmers

Mu is intended to be not the only Python IDE you’ll ever need, but the first one — the editor that helps you start your coding journey, but not necessarily the one you finish it with. So when you’re ready, you will have the skills and confidence to move on to using a more advanced Python IDE.

You can use Mu in a number of modes; modes make working with Mu easier by only presenting the options most relevant to what you’re using Mu for:

Mu Python IDE for beginners Raspberry Pi

Available now

Mu version 1.0 is available now for Windows, macOS, Linux, and the Raspberry Pi’s official operating system Raspbian! And to help new Python programmers get started, we have created a guide to Getting Started with Mu for all these operating systems.

Mu Python IDE for beginners Raspberry Pi

Mu is the brainchild of Nicholas Tollervey, who has worked tirelessly to create Mu. I recently met up with him and some of the Mu team at the world’s first Mu-“moot” to celebrate this release:

Nicholas Tollervey on Twitter

World’s first Mu-moot. 🙁

One of the inspirations for Mu was the keynote presentation at EuroPython 2015 given by Raspberry Pi’s Carrie Anne Philbin. She talked about the barriers to children getting started with Python, including the lack of an suitably easy-to-use IDE:

Carrie Anne Philbin – Keynote: Designed for Education: A Python Solution

Carrie Anne Philbin – Keynote: Designed for Education: A Python Solution [EuroPython 2015] [23 July 2015] [Bilbao, Euskadi, Spain] The problem of introducing children to programming and computer science has seen growing attention in the past few years. Initiatives like Raspberry Pi, Code Club, code.org, (and many more) have been created to help solve this problem.

Raspberry Pi has provided support for the project, helping to take Mu from its first implementation as a micro:bit programming tool to a general-purpose and simple-to-use Python editor and IDE!

You can find installation instructions as well as tutorials on Mu’s website.

The post Mu, a new Python IDE for beginners appeared first on Raspberry Pi.

Metta – Information Security Adversarial Simulation Tool

Post Syndicated from Darknet original https://www.darknet.org.uk/2018/06/metta-information-security-adversarial-simulation-tool/?utm_source=rss&utm_medium=social&utm_campaign=darknetfeed

Metta – Information Security Adversarial Simulation Tool

Metta is an information security preparedness tool in Python to help with adversarial simulation, this can help you check various detection and control capabilities within your organisation.

This project uses Redis/Celery, python, and vagrant with virtualbox to do adversarial simulation. This allows you to test (mostly) your host based instrumentation but may also allow you to test any network based detection and controls depending on how you set up your vagrants.

Read the rest of Metta – Information Security Adversarial Simulation Tool now! Only available at Darknet.

Powershell-RAT – Gmail Exfiltration RAT

Post Syndicated from Darknet original https://www.darknet.org.uk/2018/06/powershell-rat-gmail-exfiltration-rat/?utm_source=rss&utm_medium=social&utm_campaign=darknetfeed

Powershell-RAT – Gmail Exfiltration RAT

Powershell-RAT is a Python-based Gmail exfiltration RAT that can be used a Windows backdoor to send screenshots or other data as an e-mail attachment.

This RAT will help you during red team engagements to backdoor any Windows machines. It tracks the user activity using screen capture and sends the information to an attacker as an e-mail attachment.

It claims to not need Administrator access and is not currently detected by Anti-virus software.

Read the rest of Powershell-RAT – Gmail Exfiltration RAT now! Only available at Darknet.

[$] Advanced computing with IPython

Post Syndicated from jake original https://lwn.net/Articles/756192/rss

If you use Python, there’s a good chance you have heard of IPython, which provides an enhanced read-eval-print
loop (REPL) for Python. But there is more to IPython than just a more
convenient REPL. Today’s IPython comes with integrated libraries that turn
it into an assistant for several advanced computing tasks. We will look at
two of those tasks, using multiple languages and distributed computing, in
this article.

Build your own weather station with our new guide!

Post Syndicated from Richard Hayler original https://www.raspberrypi.org/blog/build-your-own-weather-station/

One of the most common enquiries I receive at Pi Towers is “How can I get my hands on a Raspberry Pi Oracle Weather Station?” Now the answer is: “Why not build your own version using our guide?”

Build Your Own weather station kit assembled

Tadaaaa! The BYO weather station fully assembled.

Our Oracle Weather Station

In 2016 we sent out nearly 1000 Raspberry Pi Oracle Weather Station kits to schools from around the world who had applied to be part of our weather station programme. In the original kit was a special HAT that allows the Pi to collect weather data with a set of sensors.

The original Raspberry Pi Oracle Weather Station HAT – Build Your Own Raspberry Pi weather station

The original Raspberry Pi Oracle Weather Station HAT

We designed the HAT to enable students to create their own weather stations and mount them at their schools. As part of the programme, we also provide an ever-growing range of supporting resources. We’ve seen Oracle Weather Stations in great locations with a huge differences in climate, and they’ve even recorded the effects of a solar eclipse.

Our new BYO weather station guide

We only had a single batch of HATs made, and unfortunately we’ve given nearly* all the Weather Station kits away. Not only are the kits really popular, we also receive lots of questions about how to add extra sensors or how to take more precise measurements of a particular weather phenomenon. So today, to satisfy your demand for a hackable weather station, we’re launching our Build your own weather station guide!

Build Your Own Raspberry Pi weather station

Fun with meteorological experiments!

Our guide suggests the use of many of the sensors from the Oracle Weather Station kit, so can build a station that’s as close as possible to the original. As you know, the Raspberry Pi is incredibly versatile, and we’ve made it easy to hack the design in case you want to use different sensors.

Many other tutorials for Pi-powered weather stations don’t explain how the various sensors work or how to store your data. Ours goes into more detail. It shows you how to put together a breadboard prototype, it describes how to write Python code to take readings in different ways, and it guides you through recording these readings in a database.

Build Your Own Raspberry Pi weather station on a breadboard

There’s also a section on how to make your station weatherproof. And in case you want to move past the breadboard stage, we also help you with that. The guide shows you how to solder together all the components, similar to the original Oracle Weather Station HAT.

Who should try this build

We think this is a great project to tackle at home, at a STEM club, Scout group, or CoderDojo, and we’re sure that many of you will be chomping at the bit to get started. Before you do, please note that we’ve designed the build to be as straight-forward as possible, but it’s still fairly advanced both in terms of electronics and programming. You should read through the whole guide before purchasing any components.

Build Your Own Raspberry Pi weather station – components

The sensors and components we’re suggesting balance cost, accuracy, and easy of use. Depending on what you want to use your station for, you may wish to use different components. Similarly, the final soldered design in the guide may not be the most elegant, but we think it is achievable for someone with modest soldering experience and basic equipment.

You can build a functioning weather station without soldering with our guide, but the build will be more durable if you do solder it. If you’ve never tried soldering before, that’s OK: we have a Getting started with soldering resource plus video tutorial that will walk you through how it works step by step.

Prototyping HAT for Raspberry Pi weather station sensors

For those of you who are more experienced makers, there are plenty of different ways to put the final build together. We always like to hear about alternative builds, so please post your designs in the Weather Station forum.

Our plans for the guide

Our next step is publishing supplementary guides for adding extra functionality to your weather station. We’d love to hear which enhancements you would most like to see! Our current ideas under development include adding a webcam, making a tweeting weather station, adding a light/UV meter, and incorporating a lightning sensor. Let us know which of these is your favourite, or suggest your own amazing ideas in the comments!

*We do have a very small number of kits reserved for interesting projects or locations: a particularly cool experiment, a novel idea for how the Oracle Weather Station could be used, or places with specific weather phenomena. If have such a project in mind, please send a brief outline to [email protected], and we’ll consider how we might be able to help you.

The post Build your own weather station with our new guide! appeared first on Raspberry Pi.

Security updates for Friday

Post Syndicated from ris original https://lwn.net/Articles/756260/rss

Security updates have been issued by Debian (kernel, procps, and tiff), Fedora (ca-certificates, chromium, and git), Mageia (kernel, kernel-linus, kernel-tmb, and libvirt), openSUSE (chromium and xen), Oracle (procps, xmlrpc, and xmlrpc3), Red Hat (xmlrpc and xmlrpc3), Scientific Linux (procps, xmlrpc, and xmlrpc3), SUSE (HA kernel modules and kernel), and Ubuntu (libytnef and python-oslo.middleware).

Amazon SageMaker Updates – Tokyo Region, CloudFormation, Chainer, and GreenGrass ML

Post Syndicated from Randall Hunt original https://aws.amazon.com/blogs/aws/sagemaker-tokyo-summit-2018/

Today, at the AWS Summit in Tokyo we announced a number of updates and new features for Amazon SageMaker. Starting today, SageMaker is available in Asia Pacific (Tokyo)! SageMaker also now supports CloudFormation. A new machine learning framework, Chainer, is now available in the SageMaker Python SDK, in addition to MXNet and Tensorflow. Finally, support for running Chainer models on several devices was added to AWS Greengrass Machine Learning.

Amazon SageMaker Chainer Estimator

Chainer is a popular, flexible, and intuitive deep learning framework. Chainer networks work on a “Define-by-Run” scheme, where the network topology is defined dynamically via forward computation. This is in contrast to many other frameworks which work on a “Define-and-Run” scheme where the topology of the network is defined separately from the data. A lot of developers enjoy the Chainer scheme since it allows them to write their networks with native python constructs and tools.

Luckily, using Chainer with SageMaker is just as easy as using a TensorFlow or MXNet estimator. In fact, it might even be a bit easier since it’s likely you can take your existing scripts and use them to train on SageMaker with very few modifications. With TensorFlow or MXNet users have to implement a train function with a particular signature. With Chainer your scripts can be a little bit more portable as you can simply read from a few environment variables like SM_MODEL_DIR, SM_NUM_GPUS, and others. We can wrap our existing script in a if __name__ == '__main__': guard and invoke it locally or on sagemaker.

import argparse
import os

if __name__ =='__main__':

    parser = argparse.ArgumentParser()

    # hyperparameters sent by the client are passed as command-line arguments to the script.
    parser.add_argument('--epochs', type=int, default=10)
    parser.add_argument('--batch-size', type=int, default=64)
    parser.add_argument('--learning-rate', type=float, default=0.05)

    # Data, model, and output directories
    parser.add_argument('--output-data-dir', type=str, default=os.environ['SM_OUTPUT_DATA_DIR'])
    parser.add_argument('--model-dir', type=str, default=os.environ['SM_MODEL_DIR'])
    parser.add_argument('--train', type=str, default=os.environ['SM_CHANNEL_TRAIN'])
    parser.add_argument('--test', type=str, default=os.environ['SM_CHANNEL_TEST'])

    args, _ = parser.parse_known_args()

    # ... load from args.train and args.test, train a model, write model to args.model_dir.

Then, we can run that script locally or use the SageMaker Python SDK to launch it on some GPU instances in SageMaker. The hyperparameters will get passed in to the script as CLI commands and the environment variables above will be autopopulated. When we call fit the input channels we pass will be populated in the SM_CHANNEL_* environment variables.

from sagemaker.chainer.estimator import Chainer
# Create my estimator
chainer_estimator = Chainer(
    hyperparameters={'epochs': 10, 'batch-size': 64}
# Train my estimator
chainer_estimator.fit({'train': train_input, 'test': test_input})

# Deploy my estimator to a SageMaker Endpoint and get a Predictor
predictor = chainer_estimator.deploy(

Now, instead of bringing your own docker container for training and hosting with Chainer, you can just maintain your script. You can see the full sagemaker-chainer-containers on github. One of my favorite features of the new container is built-in chainermn for easy multi-node distribution of your chainer training jobs.

There’s a lot more documentation and information available in both the README and the example notebooks.

AWS GreenGrass ML with Chainer

AWS GreenGrass ML now includes a pre-built Chainer package for all devices powered by Intel Atom, NVIDIA Jetson, TX2, and Raspberry Pi. So, now GreenGrass ML provides pre-built packages for TensorFlow, Apache MXNet, and Chainer! You can train your models on SageMaker then easily deploy it to any GreenGrass-enabled device using GreenGrass ML.


I want to give a quick shout out to all of our wonderful and inspirational friends in the JAWS UG who attended the AWS Summit in Tokyo today. I’ve very much enjoyed seeing your pictures of the summit. Thanks for making Japan an amazing place for AWS developers! I can’t wait to visit again and meet with all of you.


Randomly generated, thermal-printed comics

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/random-comic-strip-generation-vomit-comic-robot/

Python code creates curious, wordless comic strips at random, spewing them from the thermal printer mouth of a laser-cut body reminiscent of Disney Pixar’s WALL-E: meet the Vomit Comic Robot!

The age of the thermal printer!

Thermal printers allow you to instantly print photos, data, and text using a few lines of code, with no need for ink. More and more makers are using this handy, low-maintenance bit of kit for truly creative projects, from Pierre Muth’s tiny PolaPi-Zero camera to the sound-printing Waves project by Eunice Lee, Matthew Zhang, and Bomani McClendon (and our own Secret Santa Babbage).

Vomiting robots

Interaction designer and developer Cadin Batrack, whose background is in game design and interactivity, has built the Vomit Comic Robot, which creates “one-of-a-kind comics on demand by processing hand-drawn images through a custom software algorithm.”

The robot is made up of a Raspberry Pi 3, a USB thermal printer, and a handful of LEDs.

Comic Vomit Robot Cadin Batrack's Raspberry Pi comic-generating thermal printer machine

At the press of a button, Processing code selects one of a set of Cadin’s hand-drawn empty comic grids and then randomly picks images from a library to fill in the gaps.

Vomit Comic Robot Cadin Batrack's Raspberry Pi comic-generating thermal printer machine

Each image is associated with data that allows the code to fit it correctly into the available panels. Cadin says about the concept behing his build:

Although images are selected and placed randomly, the comic panel format suggests relationships between elements. Our minds create a story where there is none in an attempt to explain visuals created by a non-intelligent machine.

The Raspberry Pi saves the final image as a high-resolution PNG file (so that Cadin can sell prints on thick paper via Etsy), and a Python script sends it to be vomited up by the thermal printer.

Comic Vomit Robot Cadin Batrack's Raspberry Pi comic-generating thermal printer machine

For more about the Vomit Comic Robot, check out Cadin’s blog. If you want to recreate it, you can find the info you need in the Imgur album he has put together.

We ❤ cute robots

We have a soft spot for cute robots here at Pi Towers, and of course we make no exception for the Vomit Comic Robot. If, like us, you’re a fan of adorable bots, check out Mira, the tiny interactive robot by Alonso Martinez, and Peeqo, the GIF bot by Abhishek Singh.

Mira Alfonso Martinez Raspberry Pi

The post Randomly generated, thermal-printed comics appeared first on Raspberry Pi.