Tag Archives: testing

Take a Digital Tour of an AWS Data Center to See How AWS Secures Data Centers Around The World

Post Syndicated from Chad Woolf original https://aws.amazon.com/blogs/security/take-a-digital-tour-of-an-aws-data-center-to-see-how-aws-secures-data-centers-around-the-world/

Data center tour banner image

AWS has launched a digital tour of an AWS data center, providing you with a first-ever look at how AWS secures data centers around the world. The videos, pictures, and information in this tour show you how security is intrinsic to the design of our data centers, our global controls, and the AWS culture.

As you will learn when you take this digital tour, the AWS data center security strategy is assembled with scalable security controls and multiple layers of defense that help to protect your information. For example, AWS carefully manages potential flood and seismic activity risks. We use physical barriers, security guards, threat detection technology, and an in-depth screening process to limit access to data centers. We back up our systems, regularly test equipment and processes, and continuously train AWS employees to be ready for the unexpected.

To validate the security of our data centers, external auditors perform testing on more than 2,600 standards and requirements throughout the year. Such independent examination helps ensure that security standards are consistently being met or exceeded. As a result, the most highly regulated organizations in the world trust AWS to protect their data.

Take the tour today to learn more about how we secure our data centers.

– Chad

OWASP ZSC – Obfuscated Code Generator Tool

Post Syndicated from Darknet original https://www.darknet.org.uk/2018/01/owasp-zsc-obfuscated-code-generator-tool/?utm_source=rss&utm_medium=social&utm_campaign=darknetfeed

OWASP ZSC – Obfuscated Code Generator Tool

OWASP ZSC is an open source obfuscated code generator tool in Python which lets you generate customized shellcodes and convert scripts to an obfuscated script.

Shellcodes are small codes in Assembly language which could be used as the payload in software exploitation. Other usages are in malware, bypassing antivirus software, obfuscating code for protection and so on.

This software can be run on Windows/Linux/OSX under Python.

Why use OWASP ZSC Obfuscated Code Generator Tool

Another good reason for obfuscating files or generating shellcode with ZSC is that it can be used for pen-testing assignments.

Read the rest of OWASP ZSC – Obfuscated Code Generator Tool now! Only available at Darknet.

Kernel prepatch 4.15-rc8

Post Syndicated from corbet original https://lwn.net/Articles/744304/rss

The 4.15-rc8 kernel prepatch is out for
testing. Among other things, it includes the “retpoline” mechanism
intended to mitigate variant 2 of the Spectre vulnerability. Testing
of this change will be hard, though, since it requires a version of GCC
that almost nobody has — watch LWN for a full article in the near future.
I’m still hoping that this will be the last
rc, despite all the Meltdown and Spectre hoopla. But we will just have to
see, it obviously requires this upcoming week to not come with any huge
surprises.

Connect Veeam to the B2 Cloud: Episode 1 — Using Synology

Post Syndicated from Roderick Bauer original https://www.backblaze.com/blog/backing-up-veeam-cloud-connect-synology-b2/

Veeam Cloud Connect to Backblaze B2

Veeam is well-known for its easy-to-use software for backing up virtual machines from VMware and Microsoft.

Users of Veeam and Backblaze B2 Cloud Storage have asked for a way to back up a Veeam repository to B2. Backblaze’s B2 is an ideal solution for backing up Veeam’s backup repository due to B2’s combination of low-cost and high availability compared to other cloud solutions such as Microsoft Azure.

This is the first in a series of posts on the topic of backing up Veeam to B2. Future posts will cover other methods.

In this post we provide a step-by-step tutorial on how to configure a Synology NAS as a Veeam backup repository, and in turn use Synology’s CloudSync software to back up that repository to the B2 Cloud.

Our guest contributor, Rhys Hammond, is well qualified to author this tutorial. Rhys is a Senior System Engineer for Data#3 in Australia specializing in Veeam and VMware solutions. He is a VMware vExpert and a member of the Veeam Vanguard program.

Rhy’s tutorial is outlined as follows:

Veeam and Backblaze B2 — Introduction

Introduction

Background on B2 and Veeam, and a discussion of various ways to back up a Veeam backup repository to the cloud.

Phase 1 — Create the Backblaze B2 Bucket

How to create the B2 Bucket that will be the destination for mirroring our Veeam backup repository.

Phase 2 — Install and Configure Synology CloudSync

Get CloudSync ready to perform the backup to B2.

Phase 3 — Configure Veeam Backup Repository

Create a new Veeam backup repository in preparation for upload to B2.

Phase 4 — Create the Veeam Backup Job

Configure the Veeam backup job, with two possible scenarios, primary target and secondary backup target.

Phase 5 — Testing and Tuning

Making sure it all works.

Summary

Some thoughts on the process, other options, and tips.

You can read the full tutorial on Rhy’s website by following the link below. To be sure to receive notice of future posts in this series on Veeam, use the Join button at the top of the page.

Beta Testers Needed: Veeam/Starwind/B2

If you back up Veeam using Starwind VTL, we have a BETA program for you. Help us with the Starwind VTL to Backblaze B2 integration Beta and test whether you can automatically back up Veeam to Backblaze B2 via Starwind VTL. Motivated beta testers can email starwind@backblaze.com for details and how to get started.

The post Connect Veeam to the B2 Cloud: Episode 1 — Using Synology appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Continuous Deployment to Kubernetes using AWS CodePipeline, AWS CodeCommit, AWS CodeBuild, Amazon ECR and AWS Lambda

Post Syndicated from Chris Barclay original https://aws.amazon.com/blogs/devops/continuous-deployment-to-kubernetes-using-aws-codepipeline-aws-codecommit-aws-codebuild-amazon-ecr-and-aws-lambda/

Thank you to my colleague Omar Lari for this blog on how to create a continuous deployment pipeline for Kubernetes!


You can use Kubernetes and AWS together to create a fully managed, continuous deployment pipeline for container based applications. This approach takes advantage of Kubernetes’ open-source system to manage your containerized applications, and the AWS developer tools to manage your source code, builds, and pipelines.

This post describes how to create a continuous deployment architecture for containerized applications. It uses AWS CodeCommit, AWS CodePipeline, AWS CodeBuild, and AWS Lambda to deploy containerized applications into a Kubernetes cluster. In this environment, developers can remain focused on developing code without worrying about how it will be deployed, and development managers can be satisfied that the latest changes are always deployed.

What is Continuous Deployment?

There are many articles, posts and even conferences dedicated to the practice of continuous deployment. For the purposes of this post, I will summarize continuous delivery into the following points:

  • Code is more frequently released into production environments
  • More frequent releases allow for smaller, incremental changes reducing risk and enabling simplified roll backs if needed
  • Deployment is automated and requires minimal user intervention

For a more information, see “Practicing Continuous Integration and Continuous Delivery on AWS”.

How can you use continuous deployment with AWS and Kubernetes?

You can leverage AWS services that support continuous deployment to automatically take your code from a source code repository to production in a Kubernetes cluster with minimal user intervention. To do this, you can create a pipeline that will build and deploy committed code changes as long as they meet the requirements of each stage of the pipeline.

To create the pipeline, you will use the following services:

  • AWS CodePipeline. AWS CodePipeline is a continuous delivery service that models, visualizes, and automates the steps required to release software. You define stages in a pipeline to retrieve code from a source code repository, build that source code into a releasable artifact, test the artifact, and deploy it to production. Only code that successfully passes through all these stages will be deployed. In addition, you can optionally add other requirements to your pipeline, such as manual approvals, to help ensure that only approved changes are deployed to production.
  • AWS CodeCommit. AWS CodeCommit is a secure, scalable, and managed source control service that hosts private Git repositories. You can privately store and manage assets such as your source code in the cloud and configure your pipeline to automatically retrieve and process changes committed to your repository.
  • AWS CodeBuild. AWS CodeBuild is a fully managed build service that compiles source code, runs tests, and produces artifacts that are ready to deploy. You can use AWS CodeBuild to both build your artifacts, and to test those artifacts before they are deployed.
  • AWS Lambda. AWS Lambda is a compute service that lets you run code without provisioning or managing servers. You can invoke a Lambda function in your pipeline to prepare the built and tested artifact for deployment by Kubernetes to the Kubernetes cluster.
  • Kubernetes. Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It provides a platform for running, deploying, and managing containers at scale.

An Example of Continuous Deployment to Kubernetes:

The following example illustrates leveraging AWS developer tools to continuously deploy to a Kubernetes cluster:

  1. Developers commit code to an AWS CodeCommit repository and create pull requests to review proposed changes to the production code. When the pull request is merged into the master branch in the AWS CodeCommit repository, AWS CodePipeline automatically detects the changes to the branch and starts processing the code changes through the pipeline.
  2. AWS CodeBuild packages the code changes as well as any dependencies and builds a Docker image. Optionally, another pipeline stage tests the code and the package, also using AWS CodeBuild.
  3. The Docker image is pushed to Amazon ECR after a successful build and/or test stage.
  4. AWS CodePipeline invokes an AWS Lambda function that includes the Kubernetes Python client as part of the function’s resources. The Lambda function performs a string replacement on the tag used for the Docker image in the Kubernetes deployment file to match the Docker image tag applied in the build, one that matches the image in Amazon ECR.
  5. After the deployment manifest update is completed, AWS Lambda invokes the Kubernetes API to update the image in the Kubernetes application deployment.
  6. Kubernetes performs a rolling update of the pods in the application deployment to match the docker image specified in Amazon ECR.
    The pipeline is now live and responds to changes to the master branch of the CodeCommit repository. This pipeline is also fully extensible, you can add steps for performing testing or adding a step to deploy into a staging environment before the code ships into the production cluster.

An example pipeline in AWS CodePipeline that supports this architecture can be seen below:

Conclusion

We are excited to see how you leverage this pipeline to help ease your developer experience as you develop applications in Kubernetes.

You’ll find an AWS CloudFormation template with everything necessary to spin up your own continuous deployment pipeline at the CodeSuite – Continuous Deployment Reference Architecture for Kubernetes repo on GitHub. The repository details exactly how the pipeline is provisioned and how you can use it to deploy your own applications. If you have any questions, feedback, or suggestions, please let us know!

RuTracker Reveals Innovative Plan For Users to Subvert ISP Blocking

Post Syndicated from Andy original https://torrentfreak.com/rutracker-reveals-innovative-plan-for-users-to-subvert-isp-blocking-180110/

As Russia’s largest torrent site and one that earned itself a mention in TF’s list of most popular torrent sites 2018, RuTracker is continuously under fire.

The site has an extremely dedicated following but Russia’s telecoms watchdog, spurred on by copyright holders brandishing court rulings, does everything in its power to ensure that people can’t access the site easily.

As a result, RuTracker’s main domains are blocked by all ISPs, meaning that people have to resort to VPNs or the many dozens of proxy and mirror sites that have been set up to facilitate access to the popular tracker.

While all of these methods used to work just fine, new legislation that came into force during October means that mirror and proxy sites can be added to block lists without copyright holders having to return to court. And, following legislation introduced in November, local VPN services are forbidden from providing access to blocked sites.

While RuTracker has always insisted that web blockades have little effect on the numbers of people sharing content, direct traffic to their main domains has definitely suffered. To solve this problem and go some way towards mitigating VPN and proxy bans, the site has just come up with a new plan to keep the torrents flowing.

The scheme was quietly announced, not on RuTracker’s main forum, but to a smaller set of users on local site Leprosorium. The idea was that a quieter launch there would allow for controlled testing before a release to the masses. The project is called My.RuTracker and here’s how it works.

Instead of blocked users fruitlessly trying to find public circumvention methods that once seen are immediately blocked, they are invited to register their own domains. These can be single use, for the person who registers them, but it’s envisioned that they’ll be shared out between friends, family, and online groups, to better make use of the resource.

Once domains are registered, users are invited to contact a special user account on the RuTracker site (operated by the site’s operators) which will provide them with precise technical details on how to set up their domain (.ru domains are not allowed) to gain access to RuTracker.

“In response, after a while (usually every other day), a list of NS-addresses will be sent to the registrar’s domain settings. Under this scheme, the user domain will be redirected to the RuTracker site via a dynamic IP address: this will avoid blocking the torrent tracker for a particular IP address,” the scheme envisages.

According to local news resource Tjournal, 62 personal mirrors were launched following the initial appeal, with the operators of RuTracker now planning to publicly announce the project to their community. As more are added, the site will keep track of traffic from each of the personal “mirrors” for balancing the load on the site.

At least in theory, this seems like a pretty innovative scheme. Currently, the authorities rely on the scale and public awareness of a particular proxy or mirror in order to earmark it for blocking. This much more decentralized plan, in which only small numbers of people should know each domain, seems like a much more robust system – at least until the authorities and indeed the law catches up.

And so the cat-and-mouse game continues.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

A hedgehog cam or two

Post Syndicated from Helen Lynn original https://www.raspberrypi.org/blog/a-hedgehog-cam-or-two/

Here we are, hauling ourselves out of the Christmas and New Year holidays and into January proper. It’s dawning on me that I have to go back to work, even though it’s still very cold and gloomy in northern Europe, and even though my duvet is lovely and warm. I found myself envying beings that hibernate, and thinking about beings that hibernate, and searching for things to do with hedgehogs. And, well, the long and the short of it is, today’s blog post is a short meditation on the hedgehog cam.

A hedgehog in a garden, photographed in infrared light by a hedgehog cam

Success! It’s a hedgehog!
Photo by Andrew Wedgbury

Hedgehog watching

Someone called Barker has installed a Raspberry Pi–based hedgehog cam in a location with a distant view of a famous Alp, and as well as providing live views by visible and infrared light for the dedicated and the insomniac, they also make a sped-up version of the previous night’s activity available. With hedgehogs usually being in hibernation during January, you mightn’t see them in any current feed — but don’t worry! You’re guaranteed a few hedgehogs on Barker’s website, because they have also thrown in some lovely GIFs of hoggy (and foxy) divas that their camera captured in the past.

A Hedgehog eating from a bowl on a patio, captured by a hedgehog cam

Nom nom nom!
GIF by Barker’s Site

Build your own hedgehog cam

For pointers on how to replicate this kind of setup, you could do worse than turn to Andrew Wedgbury’s hedgehog cam write-up. Andrew’s Twitter feed reveals that he’s a Cambridge local, and there are hints that he was behind RealVNC’s hoggy mascot for Pi Wars 2017.

RealVNC on Twitter

Another day at the office: testing our #PiWars mascot using a @Raspberry_Pi 3, #VNC Connect and @4tronix_uk Picon Zero. Name suggestions? https://t.co/iYY3xAX9Bk

Our infrared bird box and time-lapse camera resources will also set you well on the way towards your own custom wildlife camera. For a kit that wraps everything up in a weatherproof enclosure made with love, time, and serious amounts of design and testing, take a look at Naturebytes’ wildlife cam kit.

Or, if you’re thinking that a robot mascot is more dependable than real animals for the fluffiness you need in order to start your January with something like productivity and with your soul intact, you might like to put your own spin on our robot buggy.

Happy 2018

While we’re on the subject of getting to grips with the new year, do take a look at yesterday’s blog post, in which we suggest a New Year’s project that’s different from the usual resolutions. However you tackle 2018, we wish you an excellent year of creative computing.

The post A hedgehog cam or two appeared first on Raspberry Pi.

Could you write for Hello World magazine?

Post Syndicated from Dan Fisher original https://www.raspberrypi.org/blog/could-you-write-for-hello-world-magazine/

Thinking about New Year’s resolutions? Ditch the gym and tone up your author muscles instead, by writing an article for Hello World magazine. We’ll help you, you’ll expand your knowledge of a topic you care about, and you’ll be contributing something of real value to the computing education community.

Join our pool of Hello World writers in 2018

The computing and digital making magazine for educators

Hello World is our free computing magazine for educators, published in partnership with Computing At School and kindly supported by BT. We launched at the Bett Show in January 2017, and over the past twelve months, we’ve grown to a readership of 15000 subscribers. You can get your own free copy here.

Our work is sustained by wonderful educational content from around the world in every issue. We’re hugely grateful to our current pool of authors – keep it up, veterans of 2017! – and we want to provide opportunities for new voices in the community to join them. You might be a classroom teacher sharing your scheme of work, a volunteer reflecting on running an after-school club, an industry professional sharing your STEM expertise, or an academic providing insights into new research – we’d love contributions from all kinds of people in all sorts of roles.

Your article doesn’t have to be finished and complete: if you send us an outline, we will work with you to develop it into a full piece.

Like my desk, but tidier

Five reasons to write for Hello World

Here are five reasons why writing for Hello World is a great way to start 2018:

1. You’ll learn something new

Researching an article is one of the best ways to broaden your knowledge about something that interests you.

2. You’ll think more clearly

Notes in hand, you sit at your desk and wonder how to craft all this information into a coherent piece of writing. It’s a situation we’re all familiar with. Writing an article makes you examine and clarify what you really think about a subject.

Share your expertise and make more interesting projects along the way

3. You’ll make cool projects

Testing a project for a Hello World resource is a perfect opportunity to build something amazing that’s hitherto been locked away inside your brain.

4. You’ll be doing something that matters

Sharing your knowledge and experience in Hello World helps others to teach and learn computing. It helps bring the power of digital making to more and more educators and learners.

5. You’ll share with an open and supportive community

The computing education community is full of people who lend their experience to help colleagues. Contributing to Hello World is a great way to take an active part in this supportive community, and you’ll be adding to a body of free, open source learning resources that are available for everyone to use, adapt, and share. It’s also a tremendous platform to broadcast your work: the digital version alone of Hello World has been downloaded over 50000 times.

Yes! What do I do next?

Feeling inspired? Email our editorial team with your idea.

Issue 4 of Hello World is out this month! Subscribe for free today to have it delivered to your inbox or your home.

The post Could you write for Hello World magazine? appeared first on Raspberry Pi.

Instrumenting Web Apps Using AWS X-Ray

Post Syndicated from Bharath Kumar original https://aws.amazon.com/blogs/devops/instrumenting-web-apps-using-aws-x-ray/

This post was written by James Bowman, Software Development Engineer, AWS X-Ray

AWS X-Ray helps developers analyze and debug distributed applications and underlying services in production. You can identify and analyze root-causes of performance issues and errors, understand customer impact, and extract statistical aggregations (such as histograms) for optimization.

In this blog post, I will provide a step-by-step walkthrough for enabling X-Ray tracing in the Go programming language. You can use these steps to add X-Ray tracing to any distributed application.

Revel: A web framework for the Go language

This section will assist you with designing a guestbook application. Skip to “Instrumenting with AWS X-Ray” section below if you already have a Go language application.

Revel is a web framework for the Go language. It facilitates the rapid development of web applications by providing a predefined framework for controllers, views, routes, filters, and more.

To get started with Revel, run revel new github.com/jamesdbowman/guestbook. A project base is then copied to $GOPATH/src/github.com/jamesdbowman/guestbook.

$ tree -L 2
.
├── README.md
├── app
│ ├── controllers
│ ├── init.go
│ ├── routes
│ ├── tmp
│ └── views
├── conf
│ ├── app.conf
│ └── routes
├── messages
│ └── sample.en
├── public
│ ├── css
│ ├── fonts
│ ├── img
│ └── js
└── tests
└── apptest.go

Writing a guestbook application

A basic guestbook application can consist of just two routes: one to sign the guestbook and another to list all entries.
Let’s set up these routes by adding a Book controller, which can be routed to by modifying ./conf/routes.

./app/controllers/book.go:
package controllers

import (
    "math/rand"
    "time"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/endpoints"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/dynamodb"
    "github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
    "github.com/revel/revel"
)

const TABLE_NAME = "guestbook"
const SUCCESS = "Success.\n"
const DAY = 86400

var letters = []rune("ABCDEFGHIJKLMNOPQRSTUVWXYZ")

func init() {
    rand.Seed(time.Now().UnixNano())
}

// randString returns a random string of len n, used for DynamoDB Hash key.
func randString(n int) string {
    b := make([]rune, n)
    for i := range b {
        b[i] = letters[rand.Intn(len(letters))]
    }
    return string(b)
}

// Book controls interactions with the guestbook.
type Book struct {
    *revel.Controller
    ddbClient *dynamodb.DynamoDB
}

// Signature represents a user's signature.
type Signature struct {
    Message string
    Epoch   int64
    ID      string
}

// ddb returns the controller's DynamoDB client, instatiating a new client if necessary.
func (c Book) ddb() *dynamodb.DynamoDB {
    if c.ddbClient == nil {
        sess := session.Must(session.NewSession(&aws.Config{
            Region: aws.String(endpoints.UsWest2RegionID),
        }))
        c.ddbClient = dynamodb.New(sess)
    }
    return c.ddbClient
}

// Sign allows users to sign the book.
// The message is to be passed as application/json typed content, listed under the "message" top level key.
func (c Book) Sign() revel.Result {
    var s Signature

    err := c.Params.BindJSON(&s)
    if err != nil {
        return c.RenderError(err)
    }
    now := time.Now()
    s.Epoch = now.Unix()
    s.ID = randString(20)

    item, err := dynamodbattribute.MarshalMap(s)
    if err != nil {
        return c.RenderError(err)
    }

    putItemInput := &dynamodb.PutItemInput{
        TableName: aws.String(TABLE_NAME),
        Item:      item,
    }
    _, err = c.ddb().PutItem(putItemInput)
    if err != nil {
        return c.RenderError(err)
    }

    return c.RenderText(SUCCESS)
}

// List allows users to list all signatures in the book.
func (c Book) List() revel.Result {
    scanInput := &dynamodb.ScanInput{
        TableName: aws.String(TABLE_NAME),
        Limit:     aws.Int64(100),
    }
    res, err := c.ddb().Scan(scanInput)
    if err != nil {
        return c.RenderError(err)
    }

    messages := make([]string, 0)
    for _, v := range res.Items {
        messages = append(messages, *(v["Message"].S))
    }
    return c.RenderJSON(messages)
}

./conf/routes:
POST /sign Book.Sign
GET /list Book.List

Creating the resources and testing

For the purposes of this blog post, the application will be run and tested locally. We will store and retrieve messages from an Amazon DynamoDB table. Use the following AWS CLI command to create the guestbook table:

aws dynamodb create-table --region us-west-2 --table-name "guestbook" --attribute-definitions AttributeName=ID,AttributeType=S AttributeName=Epoch,AttributeType=N --key-schema AttributeName=ID,KeyType=HASH AttributeName=Epoch,KeyType=RANGE --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5

Now, let’s test our sign and list routes. If everything is working correctly, the following result appears:

$ curl -d '{"message":"Hello from cURL!"}' -H "Content-Type: application/json" http://localhost:9000/book/sign
Success.
$ curl http://localhost:9000/book/list
[
  "Hello from cURL!"
]%

Integrating with AWS X-Ray

Download and run the AWS X-Ray daemon

The AWS SDKs emit trace segments over UDP on port 2000. (This port can be configured.) In order for the trace segments to make it to the X-Ray service, the daemon must listen on this port and batch the segments in calls to the PutTraceSegments API.
For information about downloading and running the X-Ray daemon, see the AWS X-Ray Developer Guide.

Installing the AWS X-Ray SDK for Go

To download the SDK from GitHub, run go get -u github.com/aws/aws-xray-sdk-go/... The SDK will appear in the $GOPATH.

Enabling the incoming request filter

The first step to instrumenting an application with AWS X-Ray is to enable the generation of trace segments on incoming requests. The SDK conveniently provides an implementation of http.Handler which does exactly that. To ensure incoming web requests travel through this handler, we can modify app/init.go, adding a custom function to be run on application start.

import (
    "github.com/aws/aws-xray-sdk-go/xray"
    "github.com/revel/revel"
)

...

func init() {
  ...
    revel.OnAppStart(installXRayHandler)
}

func installXRayHandler() {
    revel.Server.Handler = xray.Handler(xray.NewFixedSegmentNamer("GuestbookApp"), revel.Server.Handler)
}

The application will now emit a segment for each incoming web request. The service graph appears:

You can customize the name of the segment to make it more descriptive by providing an alternate implementation of SegmentNamer to xray.Handler. For example, you can use xray.NewDynamicSegmentNamer(fallback, pattern) in place of the fixed namer. This namer will use the host name from the incoming web request (if it matches pattern) as the segment name. This is often useful when you are trying to separate different instances of the same application.

In addition, HTTP-centric information such as method and URL is collected in the segment’s http subsection:

"http": {
    "request": {
        "url": "/book/list",
        "method": "GET",
        "user_agent": "curl/7.54.0",
        "client_ip": "::1"
    },
    "response": {
        "status": 200
    }
},

Instrumenting outbound calls

To provide detailed performance metrics for distributed applications, the AWS X-Ray SDK needs to measure the time it takes to make outbound requests. Trace context is passed to downstream services using the X-Amzn-Trace-Id header. To draw a detailed and accurate representation of a distributed application, outbound call instrumentation is required.

AWS SDK calls

The AWS X-Ray SDK for Go provides a one-line AWS client wrapper that enables the collection of detailed per-call metrics for any AWS client. We can modify the DynamoDB client instantiation to include this line:

// ddb returns the controller's DynamoDB client, instatiating a new client if necessary.
func (c Book) ddb() *dynamodb.DynamoDB {
    if c.ddbClient == nil {
        sess := session.Must(session.NewSession(&aws.Config{
            Region: aws.String(endpoints.UsWest2RegionID),
        }))
        c.ddbClient = dynamodb.New(sess)
        xray.AWS(c.ddbClient.Client) // add subsegment-generating X-Ray handlers to this client
    }
    return c.ddbClient
}

We also need to ensure that the segment generated by our xray.Handler is passed to these AWS calls so that the X-Ray SDK knows to which segment these generated subsegments belong. In Go, the context.Context object is passed throughout the call path to achieve this goal. (In most other languages, some variant of ThreadLocal is used.) AWS clients provide a *WithContext method variant for each AWS operation, which we need to switch to:

_, err = c.ddb().PutItemWithContext(c.Request.Context(), putItemInput)
    res, err := c.ddb().ScanWithContext(c.Request.Context(), scanInput)

We now see much more detail in the Timeline view of the trace for the sign and list operations:

We can use this detail to help diagnose throttling on our DynamoDB table. In the following screenshot, the purple in the DynamoDB service graph node indicates that our table is underprovisioned. The red in the GuestbookApp node indicates that the application is throwing faults due to this throttling.

HTTP calls

Although the guestbook application does not make any non-AWS outbound HTTP calls in its current state, there is a similar one-liner to wrap HTTP clients that make outbound requests. xray.Client(c *http.Client) wraps an existing http.Client (or nil if you want to use a default HTTP client). For example:

resp, err := ctxhttp.Get(ctx, xray.Client(nil), "https://aws.amazon.com/")

Instrumenting local operations

X-Ray can also assist in measuring the performance of local compute operations. To see this in action, let’s create a custom subsegment inside the randString method:


// randString returns a random string of len n, used for DynamoDB Hash key.
func randString(ctx context.Context, n int) string {
    xray.Capture(ctx, "randString", func(innerCtx context.Context) {
        b := make([]rune, n)
        for i := range b {
            b[i] = letters[rand.Intn(len(letters))]
        }
        s := string(b)
    })
    return s
}

// we'll also need to change the callsite

s.ID = randString(c.Request.Context(), 20)

Summary

By now, you are an expert on how to instrument X-Ray for your Go applications. Instrumenting X-Ray with your applications is an easy way to analyze and debug performance issues and understand customer impact. Please feel free to give any feedback or comments below.

For more information about advanced configuration of the AWS X-Ray SDK for Go, see the AWS X-Ray SDK for Go in the AWS X-Ray Developer Guide and the aws/aws-xray-sdk-go GitHub repository.

For more information about some of the advanced X-Ray features such as histograms, annotations, and filter expressions, see the Analyzing Performance for Amazon Rekognition Apps Written on AWS Lambda Using AWS X-Ray blog post.

Journeying with green sea turtles and the Arribada Initiative

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/sea-turtles/

Today, a guest post: Alasdair Davies, co-founder of Naturebytes, ZSL London’s Conservation Technology Specialist and Shuttleworth Foundation Fellow, shares the work of the Arribada Initiative. The project uses the Raspberry Pi Zero and camera module to follow the journey of green sea turtles. The footage captured from the backs of these magnificent creatures is just incredible – prepare to be blown away!

Pit Stop Camera on Green Sea Turtle 01

Footage from the new Arribada PS-C (pit-stop camera) video tag recently trialled on the island of Principe in unison with the Principe Trust. Engineered by Institute IRNAS (http://irnas.eu/) for the Arribada Initiative (http://blog.arribada.org/).

Access to affordable, open and customisable conservation technologies in the animal tracking world is often limited. I’ve been a conservation technologist for the past ten years, co-founding Naturebytes and working at ZSL London Zoo, and this was a problem that continued to frustrate me. It was inherently expensive to collect valuable data that was necessary to inform policy, to designate marine protected areas, or to identify threats to species.

In March this year, I got a supercharged opportunity to break through these barriers by becoming a Shuttleworth Foundation Fellow, meaning I had the time and resources to concentrate on cracking the problem. The Arribada Initiative was founded, and ten months later, the open source Arribada PS-C green sea turtle tag was born. The video above was captured two weeks ago in the waters of Principe Island, West Africa.

Alasdair Davies on Twitter

On route to Principe island with 10 second gen green sea #turtle tags for testing. This version has a video & accelerometer payload for behavioural studies, plus a nice wireless charging carry case made by @institute_irnas @ShuttleworthFdn

The tag comprises a Raspberry Pi Zero W sporting the Raspberry Pi camera module, a PiRA power management board, two lithium-ion cells, and a rather nice enclosure. It was built in unison with Institute IRNAS, and there’s a nice user-friendly wireless charging case to make it easy for the marine guards to replace the tags after their voyages at sea. When a tag is returned to one of the docking stations in the case, we use resin.io to manage it, download videos, and configure the tag remotely.

Green Sea Turtle Alasdair Davies Raspberry Pi
Green Sea Turtle Alasdair Davies Raspberry Pi

The tags can also be configured to take video clips at timed intervals, meaning we can now observe the presence of marine litter, plastic debris, before/after changes to the ocean environment due to nearby construction, pollution, and other threats.

Discarded fishing nets are lethal to sea turtles, so using this new tag at scale – now finally possible, as the Raspberry Pi Zero helps to drive down costs dramatically whilst retaining excellent video quality – offers real value to scientists in the field. Next year we will be releasing an optimised, affordable GPS version.

green sea turtle Alasdair Davies Raspberry Pi Arribada Initiative

To make this all possible we had to devise a quicker method of attaching the tag to the sea turtles too, so we came up with the “pit-stop” technique (which is what the PS in the name “Arribada PS-C” stands for). Just as a Formula 1 car would visit the pits to get its tyres changed, we literally switch out the tags on the beach when nesting females return, replacing them with freshly charged tags by using a quick-release base plate.

Alasdair Davies on Twitter

About 6 days left now until the first tagged nesting green sea #turtles return using our latest “pit-stop” removeable / replaceable tag method. Counting down the days @arribada_i @institute_irnas

To implement the system we first epoxy the base plate to the turtle, which minimises any possible stress to the turtles as the method is quick. Once the epoxy has dried we attach the tag. When the turtle has completed its nesting cycle (they visit the beach to lay eggs three to four times in a single season, every 10–14 days on average), we simply remove the base plate to complete the field work.

Green Sea Turtle Alasdair Davies Raspberry Pi
Green Sea Turtle Alasdair Davies Raspberry Pi

If you’d like to watch more wonderful videos of the green sea turtles’ adventures, there’s an entire YouTube playlist available here. And to keep up to date with the initiative, be sure to follow Arribada and Alasdair on Twitter.

The post Journeying with green sea turtles and the Arribada Initiative appeared first on Raspberry Pi.

timeShift(GrafanaBuzz, 1w) Issue 27

Post Syndicated from Blogs on Grafana Labs Blog original https://grafana.com/blog/2017/12/22/timeshiftgrafanabuzz-1w-issue-27/

As we wrap up 2017, I wanted to kick off my last timeShift of the year to thank you, the Grafana community, for all your input, feedback, and involvement that’s made Grafana better with every release. While code contributions are extremely important, they’re not the only way to participate in the open source software community. Feature requests, bug reports, writing documentation, testing new features, participating in hackathons and meetups – all contribute to making open source projects better.

Fedora redesigns its modularity initiative

Post Syndicated from corbet original https://lwn.net/Articles/741871/rss

The Fedora Project has announced
a number of changes to its modularity initiative after
failing to meet its initial set of goals.
From an end-user’s perspective, Fedora will ship with two sets of
repositories. One will be the traditional Fedora repositories (fedora,
updates, and updates-testing) and the other will be a new set of
repositories providing alternative and supplementary modules. We haven’t
decided on a final name for these yet, so we will use the placeholder terms
modular, modular-updates, and modular-updates-testing.

Using Trusted Timestamping With Java

Post Syndicated from Bozho original https://techblog.bozho.net/using-trusted-timestamping-java/

Trusted timestamping is the process of having a trusted third party (“Time stamping authority”, TSA) certify the time of a given event in electronic form. The EU regulation eIDAS gives these timestamps legal strength – i.e. nobody can dispute the time or the content of the event if it was timestamped. It is applicable to multiple scenarios, including timestamping audit logs. (Note: timestamping is not sufficient for a good audit trail as it does not prevent a malicious actor from deleting the event altogether)

There are a number of standards for trusted timestamping, the core one being RFC 3161. As most RFCs it is hard to read. Fortunately for Java users, BouncyCastle implements the standard. Unfortunately, as with most security APIs, working with it is hard, even abysmal. I had to implement it, so I’ll share the code needed to timestamp data.

The whole gist can be found here, but I’ll try to explain the main flow. Obviously, there is a lot of code that’s there to simply follow the standard. The BouncyCastle classes are a maze that’s hard to navigate.

The main method is obviously timestamp(hash, tsaURL, username, password):

public TimestampResponseDto timestamp(byte[] hash, String tsaUrl, String tsaUsername, String tsaPassword) throws IOException {
    MessageImprint imprint = new MessageImprint(sha512oid, hash);

    TimeStampReq request = new TimeStampReq(imprint, null, new ASN1Integer(random.nextLong()),
            ASN1Boolean.TRUE, null);

    byte[] body = request.getEncoded();
    try {
        byte[] responseBytes = getTSAResponse(body, tsaUrl, tsaUsername, tsaPassword);

        ASN1StreamParser asn1Sp = new ASN1StreamParser(responseBytes);
        TimeStampResp tspResp = TimeStampResp.getInstance(asn1Sp.readObject());
        TimeStampResponse tsr = new TimeStampResponse(tspResp);

        checkForErrors(tsaUrl, tsr);

        // validate communication level attributes (RFC 3161 PKIStatus)
        tsr.validate(new TimeStampRequest(request));

        TimeStampToken token = tsr.getTimeStampToken();
            
        TimestampResponseDto response = new TimestampResponseDto();
        response.setTime(getSigningTime(token.getSignedAttributes()));
        response.setEncodedToken(Base64.getEncoder().encodeToString(token.getEncoded()));
           
        return response;
    } catch (RestClientException | TSPException | CMSException | OperatorCreationException | GeneralSecurityException e) {
        throw new IOException(e);
    }
}

It prepares the request by creating the message imprint. Note that you are passing the hash itself, but also the hashing algorithm used to make the hash. Why isn’t the API hiding that from you, I don’t know. In my case the hash is obtained in a more complicated way, so it’s useful, but still. Then we get the raw form of the request and send it to the TSA (time stamping authority). It is an HTTP request, sort of simple, but you have to take care of some request and response headers that are not necessarily consistent across TSAs. The username and password are optional, some TSAs offer the service (rate-limited) without authentication.

When you have the raw response back, you parse it to a TimeStampResponse. Again, you have to go through 2 intermediate objects (ASN1StreamParser and TimeStampResp), which may be a proper abstraction, but is not a usable API.

Then you check if the response was successful, and you also have to validate it – the TSA may have returned a bad response. Ideally all of that could’ve been hidden from you. Validation throws an exception, which in this case I just propagate by wrapping in an IOException.

Finally, you get the token and return the response. The most important thing is the content of the token, which in my case was needed as Base64, so I encode it. It could just be the raw bytes as well. If you want to get any additional data from the token (e.g. the signing time), it’s not that simple; you have to parse the low-level attributes (seen in the gist).

Okay, you have the token now, and you can store it in a database. Occasionally you may want to validate whether timestamps have not been tampered with (which is my usecase). The code is here, and I won’t even try to explain it – it’s a ton of boilerplate that is also accounting for variations in the way TSAs respond (I’ve tried a few). The fact that a DummyCertificate class is needed either means I got something very wrong, or confirms my critique for the BouncyCastle APIs. The DummyCertificate may not be needed for some TSAs, but it is for others, and you actually can’t instantiate it that easily. You need a real certificate to construct it (which is not included in the gist; using the init() method in the next gist you can create the dummy with dummyCertificate = new DummyCertificate(certificateHolder.toASN1Structure());). In my code these are all one class, but for presenting them I decided to split it, hence this little duplication.

Okay, now we can timestamp and validate timestamps. That should be enough; but for testing purposes (or limited internal use) you may want to do the timestamping locally instead of asking a TSA. The code can be found here. It uses spring, but you can instead pass the keystore details as arguments to the init method. You need a JKS store with a keypair and a certificate, and I used KeyStore Explorer to create them. If you are running your application in AWS, you may want to encrypt your keystore using KMS (Key Management Service), and then decrypt it on application load, but that’s out of the scope of this article. For the local timestamping validation works as expected, and for timestamping – instead of calling the external service, just call localTSA.timestamp(req);

How did I get to know which classes to instantiate and which parameters to pass – I don’t remember. Looking at tests, examples, answers, sources. It took a while, and so I’m sharing it, to potentially save some trouble of others.

A list of TSAs you can test with: SafeCreative, FreeTSA, time.centum.pl.

I realize this does not seem applicable to many scenarios, but I would recommend timestamping some critical pieces of your application data. And it is generally useful to have it in your “toolbox”, ready to use, rather than trying to read the standard and battling with BouncyCastle classes for days in order to achieve this allegedly simple task.

The post Using Trusted Timestamping With Java appeared first on Bozho's tech blog.

Start off the New Year by earning AWS Certified Solutions Architect – Associate

Post Syndicated from Janna Pellegrino original https://aws.amazon.com/blogs/architecture/start-off-the-new-year-by-earning-aws-certified-solutions-architect-associate/

Do you design applications and systems on AWS? Want to demonstrate your AWS Cloud skills? Ring in 2018 by becoming an AWS Certified Solutions Architect – Associate. It’s a way to validate your expertise with an industry-recognized credential and give your career a boost.

Why get certified, you ask? According to the 2017 Global Knowledge IT Skills and Salary Report, cloud certifications, including AWS Certified Solutions Architect – Associate, generally have salaries well above average. For example, a typical U.S. salary for AWS Certified IT staff is 27.5 percent higher than the normal salary rate. Looking ahead, the report also finds that the IT industry will continue investing heavily in certification as a way to validating employees’ skills and expertise.

Here are our tips for preparing for the AWS Certified Solutions Architect – Associate exam—which we hope you’ll pass with flying colors.

Learn About the Exam

View the AWS Certified Solutions Architect – Associate Exam Guide. It covers concepts within the exam and gives you a blueprint of what you need to study.

The exam tests your technical expertise in designing and deploying scalable, highly-available, and fault-tolerant systems on AWS. It’s for anyone with one or more years of hands-on experience designing distributed applications and systems on the AWS platform.

Continue with Digital and Classroom Training

Next, brush up on key AWS services covered in the exam with our new free digital training offerings at aws.training. Our 100+ bite-sized online courses are each 10 minutes long so you learn AWS fundamentals at your own pace.

Just getting started learning the fundamentals of the AWS Cloud? We recommend you take our AWS Cloud Practitioner Essentials course, part of our free digital training offerings.

For more in-depth technical training, register for our immersive Architecting on AWS course. It’s three days of instructor-led classroom training, books, and labs, built and taught by AWS experts.

Study with Exam Prep Resources

Once you have an idea of what’s on the exam, and you’ve taken training to prepare, it’s time to prepare for the exam itself.

Dig deeper into the exam’s concepts and topics with the AWS Certified Solutions Architect – Associate Exam: Official Study Guide. It provides access to content written by AWS experts, real-world knowledge, key exam essentials, chapter review questions, an interactive online learning environment, and much more.

Next, study AWS whitepapers and FAQs with content related to the exam. You can find links to our suggested whitepapers at FAQs at https://aws.amazon.com/certification/certification-prep/ under the Solutions Architect – Associate tab.

You can also take an Exam Prep Workshop and learn exam strategies from a certified technical instructor.

Once you’re ready, put your knowledge to the (practice) test with sample questions. Register for an online practice exam to test your knowledge in a timed environment.

Schedule Your Exam and Get Certified

Now you’re ready to take the exam! Go to aws.training to schedule an exam at a testing center near you at. Once you’ve passed and are AWS Certified, you’ll enjoy AWS Certification benefits like access to the AWS Certified LinkedIn Community, invitations to AWS Certification Appreciation Receptions, digital AWS Certified badges, access to AWS Certified merchandise, and more.

Learn More

Visit us at aws.amazon.com/training for more information on digital training, classroom training, and AWS Certifications.