Tag Archives: SSO

Releasing Cloudflare Access’ most requested feature

Post Syndicated from Sam Rhea original https://blog.cloudflare.com/releasing-cloudflare-access-most-requested-feature/

Releasing Cloudflare Access’ most requested feature

Cloudflare Access, part of Cloudflare for Teams, replaces legacy corporate VPNs with Cloudflare’s global network. Instead of starting a VPN client to backhaul traffic through an office, users visit the hostname of an internal application and login with your team’s SSO provider. While the applications feel like SaaS apps for end users, your security and IT departments can configure granular controls and audit logging in a single place.

Since Access launched two years ago, customers have been able to integrate multiple SSO providers at the same time. This MultiSSO option makes it seamless for teams to have employees login with Okta or Azure AD while partners and contractors use LinkedIN or GitHub.

The integrations always applied globally. Users would see all SSO options when connecting to any application protected by Cloudflare Access. As more organizations use Cloudflare Access to connect distributed and mixed workforces to resources, listing every provider on every app no longer scales.

For example, your team might have an internal GitLab instance that only employees need to access using your corporate G Suite login. Meanwhile, the marketing department needs to share QA versions of new sites with an external agency who authenticates with LinkedIn. Asking both sets of users to pick an SSO provider on both applications adds a redundant step and can lead to additional questions or IT tickets.

The ability to only show users the relevant identity provider became the most requested feature in Cloudflare Access in the last few months. Starting today, you can use the new Cloudflare for Teams UI to configure identity options on individual applications.

Cloudflare Access

Cloudflare Access secures applications by applying zero-trust enforcement to every request. Rather than trusting anyone on a private network, Access checks for identity any time someone attempts to reach the application. With Cloudflare’s global network, that check takes place in a data center in over 200 cities around the world to avoid compromising performance.

Releasing Cloudflare Access’ most requested feature

Behind the scenes, administrators build rules to decide who should be able to reach the tools protected by Access. In turn, when users need to connect to those tools, they are prompted to authenticate with one of the identity provider options. Cloudflare Access checks their login against the list of allowed users and, if permitted, allows the request to proceed.

The challenge of agreeing on identity

Most zero-trust options, like the VPN appliances they replace, rely on one source of identity. If your team has an application that you need to share with partners or contractors, you need to collectively agree on a single standard.

Some teams opt to solve that challenge by onboarding external users to their own identity provider. When contractors join a project, the IT department receives help desk tickets to create new user accounts in the organization directory. Contractors receive instructions on how to sign-up, they spend time creating passwords and learning the new tool, and then use those credentials to login.

This option gives an organization control of identity, but adds overhead in terms of time and cost. The project owner also needs to pay for new SSO seat licenses, even if those seats are temporary. The IT department must spend time onboarding, helping, and then offboarding those user accounts. And the users themselves need to learn a new system and manage yet another password – this one with permission to your internal resources.

Releasing Cloudflare Access’ most requested feature

Alternatively, other groups decide to “federate” identity. In this flow, an organization will connect their own directory service to their partner’s equivalent service. External users login with their own credentials, but administrators do the work to merge the two services to trust one another.

Releasing Cloudflare Access’ most requested feature

While this method avoids introducing new passwords, both organizations need to agree to dedicate time to integrate their identity providers – assuming that those providers can integrate. Businesses then need to configure this setup with each contractor or partner group. This model also requires that external users be part of a larger organization, making it unavailable to single users or freelancers.

Cloudflare Access avoids forcing the decision on a single source of identity by supporting multiple. When users connect, they are presented with those options. Users choose their specific provider and Access checks that individual’s login against the list of allowed users.

Releasing Cloudflare Access’ most requested feature

Configuring per-app options

Not all of those options apply to every application that an organization secures. To segment those applications, and reduce user confusion, you can now scope specific apps to different providers.

To get started, select the application that you want to segment with a particular provider in the Cloudflare for Teams UI. Click the tab titled “Authentication”.

Releasing Cloudflare Access’ most requested feature

The tab will list all providers integrated with your account. By default, Access will continue to enable all options for end users. You can toggle any provider on or off in this view and save. The next time your users visit this application, they will only see the options enabled.

If you disable all but one option, Access will skip the login page entirely and redirect the user directly to the provider – saving them an unnecessary click.

What’s next?

You can start configuring individual identity providers with specific applications in the new Cloudflare for Teams dashboard. Additional documentation is also available.

The new Teams UI makes this feature possible, but the login page that your end users see still has the legacy design from the older Access dashboard that launched two years ago. Cloudflare for Teams will be releasing a style update to that page in the next month to bring it in line with this new UI.

A single dashboard for Cloudflare for Teams

Post Syndicated from Sam Rhea original https://blog.cloudflare.com/a-single-dashboard-for-cloudflare-for-teams/

A single dashboard for Cloudflare for Teams

Starting today, Cloudflare Access can now be used in the Cloudflare for Teams dashboard. You can manage security policies for your people and devices in the same place that you build zero-trust rules to protect your applications and resources. Everything is now in one place in a single dashboard.

We are excited to launch a new UI that can be used across the entire Teams platform, but we didn’t build this dashboard just for the sake of a new look-and-feel. While migrating the Access dashboard, we focused on solving one of the largest sources of user confusion in the product.

This post breaks down why the original  UI caused some headaches, how we think about objects in Cloudflare for Teams, and how we set out to fix the way we display that to our users.

Cloudflare Access

Cloudflare Access is one-half of Cloudflare for Teams, a security platform that runs on Cloudflare’s network. Teams protects users, devices and data  without compromising experience or performance. We built Cloudflare Access to solve our own headaches with private networks as we grew from a team concentrated in a single office to a globally distributed organization.

A single dashboard for Cloudflare for Teams

Cloudflare Access replaces corporate VPNs with Cloudflare’s network in a zero-trust model. Instead of placing internal tools on a private network, teams deploy them in any environment, including hybrid or multi-cloud models, and secure them consistently with Cloudflare’s network.

When users connect to those tools, they are prompted to login with their team’s identity provider. Cloudflare Access checks their login against the list of allowed users and, if permitted, allows the request to proceed.

Deploying Access does not require exposing new holes in corporate firewalls. Teams connect their resources through a secure outbound connection, Argo Tunnel, which runs in your infrastructure to connect the applications and machines to Cloudflare. That tunnel makes outbound-only calls to the Cloudflare network and organizations can replace complex firewall rules with just one: disable all inbound connections.

Sites vs. Accounts

When you use Cloudflare, you use the platform at two levels: account and site. You have one Cloudflare account, though you can be a member of multiple accounts. That one account captures details like your billing profile and notification settings.

Your account contains sites, the hostnames or zones that you add to Cloudflare. You configure features that apply to a site, like web application firewall (WAF) and caching rules.

When we launched Access nearly two years ago, you could use the product to add an identity check to a site you added to Cloudflare, either at the hostname, subdomain, or path. To do that, users select the site in their Cloudflare dashboard, toggle to the Access tab, and build a rule specific to that site.

A single dashboard for Cloudflare for Teams

To add rules to a different site, a user steps back up a level. They need to select the new site from the dropdown and load the Access tab for that site. However, two components in the UI remained the same and shared configuration:

  • SSO integration
  • Logs

The SSO integration is where Access pulls information about identity. Users integrate their Okta, AzureAD, GSuite accounts, or other identity providers, in this card. We made a decision that the integration should apply across your entire account; you should not need to reconfigure your SSO connection on every site where you want to add an Access rule.

However, we displayed that information in the site-specific page. Cloudflare has account-level concepts, like billing or account users, but we wanted to keep everything related to Access in a single page so we made this compromise. Logs followed a similar pattern.

This decision caused confusion. For example, we add a log table to the bottom of the tab when users view “site{.}com”. However, that table actually presented logs from both “site{.}com” and any other hostname in the account.

As more features were added, this exception grew out of control. At this point, the majority of features you see when you open the Access tab for one of your sites are account-level features stuffed into the site view. The page below is the Access tab for a site in my account, widgetcorp{.}tech. Highlighted in green are the boxes that apply to the site I have selected. Highlighted in red are the boxes that apply to my Access account.

A single dashboard for Cloudflare for Teams

This user experience is unnecessarily complex . Even worse, though, is that confusion in security products can lead to real incidents. Any time that a user asks “am I building something for my account or this site?” We needed to fix both.

Starting with a new design

A few months ago, Cloudflare launched Cloudflare for Teams, which consists of two complementary products: Access and a new solution, Cloudflare Gateway. If Access is a bouncer standing in front of the door, checking identity, Gateway is a bodyguard, keeping your team safe as you navigate the Internet.

Gateway has no concept of sites, at least not sites that you host yourself. Rather than securing your Internet properties, like Cloudflare’s infrastructure products that rely on the reverse proxy, Gateway secures your team from the Internet, and the threats on it. For the first time, you could use a Cloudflare product without a site on Cloudflare.

Gateway introduced other new concepts which have no relation to a domain name in the traditional Cloudflare sense. You can add your office network and your home WiFi to your Gateway account. You can build rules to block any sites on the Internet. You can now use Gateway on mobile devices and soon desktops as well.

To capture that model, we started on a new UI from scratch, and earlier this year we launched a new dashboard for Gateway, dash.teams.cloudflare.com.

A single dashboard for Cloudflare for Teams

Account settings now have a home of their own

The products in Cloudflare for Teams should live in one place; you shouldn’t need to hop back and forth between different dashboards to manage them. Bringing Access into the Teams dashboard puts everything under one roof.

That also gave us an opportunity to solve the confusion in the current Access UI. Since the Teams dashboard is not constrained by the site-specific model, we could break out the dashboard into components that made sense for how people use the Access product.

A single dashboard for Cloudflare for Teams

The new dashboard untangles the tools in Access that apply to your entire account (the methods that you use to secure your resources) from the features that apply to a single site (the rules you build to protect a resource).

One dashboard for your team

Merging Access into the Cloudflare for Teams dashboard, and solving the problems of the original UI, is just the beginning. We’ll be using that foundation to release new features in both Access and Gateway, including more that apply across both products.

You will also be able to continue to extend some of the configuration made in Access to Gateway. For example, an integration with a provider like Okta to build zero-trust policies in Access can eventually be reused for adding group-based policies into Gateway. You’ll see the beginning of that in the new UI, as well, with categories like “My Teams” and “Logs” that apply or will apply to both products. As we continue, we’re going to try to avoid making the same mistake of conflating account, site, and now product objects.

A single dashboard for Cloudflare for Teams

What’s next?

The new Access UI is available to all customers today in the Cloudflare for Teams dashboard. You can get started by visiting this link and signing in with your Cloudflare account.

To use the Access UI, you will first need to enable Cloudflare Access and add a site to Cloudflare in the existing dashboard. Instructions are available here. You can also watch a guided tour of the new site.

No new features have been added, though we’re busy working on them. This release focused entirely on improving how users approach the product based on the feedback we have received over 22 months. We’re still listening to new feedback. Run into an issue or notice an area of improvement? Please tell us.

Open sourcing our Sentry SSO plugin

Post Syndicated from Sam Rhea original https://blog.cloudflare.com/open-sourcing-our-sentry-sso-plugin/

Open sourcing our Sentry SSO plugin

Cloudflare Access, part of Cloudflare for Teams, replaces legacy corporate VPNs with Cloudflare’s global network. Using your existing identity provider, Access enables your end users to login from anywhere — without a clunky agent or traffic backhaul through a centralized appliance or VPN.

Today, we are open sourcing a plugin that continues to improve that experience by making it easier for teams to use Cloudflare Access with one of the software industry’s most popular engineering tools, Sentry.

What is Sentry?

Sentry is an application that helps software teams find and diagnose errors in their products. We use Sentry here at Cloudflare. When you encounter an error when using a Cloudflare product, like our dashboard, we log that event. We then use Sentry to determine what went wrong.

Sentry can categorize and roll up errors, making it easy to identify new problems before investigating them with the tool’s event logging. Engineering managers here can use the dashboards to monitor the health of a new release. Product managers often use those reports as part of prioritizing what to fix next. Engineers on our team can dig into the individual errors as they release a fix.

Sentry is available in two forms: a SaaS model and a self-hosted version. Both modes give engineering teams comprehensive insight into the behavior of their deployed applications and the issues their users encounter.

Connecting users to Sentry

Organizations can deploy the self-hosted version on-premise or in a cloud environment they control. However, they still need to create a secure way to allow their teams to connect to the app.

Historically, most opt for a VPN to solve that challenge. End users outside of the office need to configure a VPN client on their laptop and try to login with credentials that are often different from the ones used for a corporate SSO. Administrators had to make sure their VPN appliance could scale for a few users, but with most in the office, the VPN was a serious inconvenience for a smaller set of users.

Over the last few years, that group of users working outside of the office has grown. The outbreak of COVID-19 is accelerating that growth significantly. Users are working from BYOD laptops, mobile phones, and in unfamiliar networks that all struggle with a VPN. Even worse, a VPN has a load limit because it relies on an actual appliance (whether virtual or physical hardware). Organizations can attempt to stress test their VPN, but will always have a limit that administrators need to continuously monitor.

Cloudflare Access gives administrators the scale of Cloudflare’s global network and provides end users with a SaaS-like experience that just works from any device or network. When teams secure Sentry with Cloudflare Access, end users visit the hostname of the application, login with their identity provider, and are redirected from Cloudflare’s edge to the app if they have permission to reach it.

However, in the case of an app like Sentry, end users need to login one more time to the application itself. That small step adds real friction, which Access can now solve through this open source plugin.

JWT Security with Cloudflare for Teams

When a user logs in to their identity provider when connecting to an application protected by Access, Cloudflare signs a JSON Web Token (JWT).

Open sourcing our Sentry SSO plugin

Cloudflare Access uses that JWT, and its contents, to confirm a user identity before allowing or denying access to sensitive resources. Cloudflare securely creates these through the OAUTH or SAML integration between Cloudflare Access and the configured identity provider. Each JWT consists of three Base64-URL strings: the header, the payload, and the signature.

  • The header defines the cryptographic operation that encrypts the data in the JWT.
  • The payload consists of name-value pairs for at least one and typically multiple claims, encoded in JSON. For example, the payload can contain the identity of a user
  • The signature allows the receiving party to confirm that the payload is authentic.

The token is signed using a public private key pair and saved in the user’s browser. Inside of that token, we store the following details in addition to some general metadata:

  • User identity: typically the email address of the user retrieved from your identity provider.
  • Authentication domain: the domain that signs the token. For Access, we use “example.cloudflareaccess.com” where “example” is a subdomain you can configure.
  • Audience: The domain of the application you are attempting to reach.
  • Expiration: the time at which the token is no longer valid for use.

When a request is made to an application behind Access, Cloudflare looks for the presence of that token. If available, we decrypt it, validate its authenticity, and then read the payload. If the payload contains information about a user who should be able to reach the application, we send their request to an origin.

The Sentry plugin takes that JWT and reuses it, instead of prompting the visitor to login again with separate credentials. The plugin parses the user identity, checks it against the directory of users in Sentry, and maps that token to a Sentry profile and its assigned permissions.

All of this is seamless to the end user and takes just a few milliseconds. The user is instantly redirected to the application, fully authenticated, and only needs to remember their SSO login. Administrators now have one fewer set of credentials to worry about managing and the associated onboarding and offboarding.

Building your own SSO plugin

We believe that the JSON Web Token is a simple and efficient method for sending identity. Applications that use JWTs for authorization only need to support the JWT standard, instead of attempting to integrate with different versions of SAML or other formats like OIDC and OAUTH. A JWT is also information dense and built in a format, JSON, that can be easily parsed by the target application.

Some products, like Redash, already have native support for JWT integration. The Sentry plugin we built joins our Atlassian plugin as both options to extend support to those apps, but also examples that can be used for integration with other products. Other teams, like Auth0, have also published materials to add JWT integration to legacy apps.

What’s next?

Cloudflare Access is available on every Cloudflare account and 5 free seats are included by default. You can follow these instructions to get started.

If you are a small business, you can sign up for the Cloudflare for Teams program right now at the link below.

https://www.cloudflare.com/smallbusiness/

How to automate SAML federation to multiple AWS accounts from Microsoft Azure Active Directory

Post Syndicated from Sepehr Samiei original https://aws.amazon.com/blogs/security/how-to-automate-saml-federation-to-multiple-aws-accounts-from-microsoft-azure-active-directory/

You can use federation to centrally manage access to multiple AWS accounts using credentials from your corporate directory. Federation is the practice of establishing trust between a system acting as an identity provider and other systems, often called service providers, that accept authentication tokens from that identity provider. Amazon Web Services (AWS) supports open federation standards, including Security Assertion Markup Language (SAML) 2.0, to make it easier for the systems and service providers to interact. Here, I’m going to explain how to automate federation between AWS Identity and Access Management (IAM) in multiple AWS accounts and Microsoft Azure Active Directory (Azure AD). I’ll be following the same general patterns that allow SAML federation to AWS from any other identity provider that supports SAML 2.0, but I’m also adding some automation that is specific to Azure AD. I’ll show you how to perform the initial configuration, and then how to automatically keep Azure AD in sync with your AWS IAM roles.

AWS supports any SAML 2.0-compliant identity provider. If you’re interested in configuring federated access using an identity provider other than Azure AD, these links might be useful:

In this post, I’m going to focus on the nuances of using Azure AD as a SAML identity provider for AWS. The approach covered here gives you a solution that makes this option easier and adheres to AWS best practices. The primary objectives of this step-by-step walkthrough, along with the accompanying packaged solution, are:

  • Support any number of AWS accounts and roles, making it easier to scale.
  • Keep configuration of both sides updated automatically.
  • Use AWS short-term credentials so you don’t have to store your credentials with your application. This enhances your security posture because these credentials are dynamically generated, securely delivered, naturally expire after their limited lifetime, and are automatically rotated for you.

Solution overview

I’ll discuss:

  • How to configure Microsoft Azure Active Directory and show the steps needed to prepare it for federation with AWS.
  • How to configure AWS IAM Identity Providers and Roles, and explain the steps you need to carry out in your AWS accounts.
  • How to automatically import your AWS configuration into the Azure AD SSO app for AWS.

The following diagram shows the high-level flow of SAML authentication and how your users will be federated into the AWS Management console:
 

Figure 1: SAML federation between Azure AD and AWS

Figure 1: SAML federation between Azure AD and AWS

Key to the interactions in the diagram

  1. User opens a browser and navigates to Azure AD MyApps access panel (myapps.microsoft.com).
  2. If the user isn’t authenticated, she’ll be redirected to the login endpoint for authentication.
  3. User enters her credentials and the login endpoint will verify them against Azure AD tenant.
  4. Upon successful login, user will be redirected back to the access panel.
  5. User will see the list of available applications, including the AWS Console app, and will select the AWS Console app icon.
  6. The access panel redirects the user to the federated application endpoint, passing the application ID of the AWS SSO app.
  7. The AWS SSO application queries Azure AD and generates a SAML assertion, including all the AWS IAM roles assigned to the user.
  8. SAML assertion is sent back to the user.
  9. User is redirected to AWS federation endpoint, presenting the SAML assertion. The AWS federation endpoint verifies the SAML assertion. The user will choose which of their authorized roles they currently want to operate in. Note: If there’s only one role included, the selection is automatic.
  10. The AWS federation endpoint invokes the AssumeRoleWithSAML API of AWS Security Token Service (STS) and exchanges the SAML token with temporary AWS IAM credentials.
  11. Temporary IAM credentials are used to formulate a specific AWS Console URL that’s passed back to the client browser.
  12. User is redirected to AWS Management Console with permissions of the assumed role.

Automated solution components and flow

At the core of this automated solution, there’s a Docker container that runs inside an AWS ECS Fargate task. The container includes a number of PowerShell scripts that iterate through your IAM Roles, find roles that are associated with the Identity Provider of Azure AD, and update the Azure AD SSO app manifest with the necessary values.

The Fargate task is invoked through an AWS Lambda function that’s scheduled through a CloudWatch Rule to run with the frequency you specify during setup.

All of these components require a number of parameters to run correctly, and you provide these parameters through the setup.ps1 script. The setup.ps1 script is run once and acquires all required parameters from you. It then stores these parameters with encryption inside the SSM Parameter Store. Azure credentials are stored in AWS Secrets Manager. This means you could even go another step further and use Secrets Manager lifecycle management capabilities to automatically rotate your Azure credentials. For encryption of Azure credentials, the template creates a new KMS key, exclusive to this application. If you prefer to use an existing key or a Customer Managed Key (CMK), you can modify the CloudFormation template, or simply pass your own key name to the setup.ps1 script.

The following diagram shows all components of the solution:
 

Figure 2: Solution architecture

Figure 2: Solution architecture

  1. You’ll want any ongoing changes in AWS IAM roles to be replicated into Azure AD. Therefore, you need to have the update task run periodically. A CloudWatch Rule triggers an event and an AWS Lambda Function starts running as a result of this event.
  2. The Lambda Function runs an ECS Fargate Task.
  3. The ECS Task is associated with a Task Role with permission to fetch parameters from Systems Manager (SSM) Parameter Store and Secrets Manager. The task will request parameters from SSM PS, and SSM PS decrypts parameter values using the associated key in AWS Key Management Service (KMS). Azure credentials are securely stored in AWS Secrets Manager.
  4. Fargate Task queries AWS Organizations and gets a list of child accounts. It then constructs cross-account role ARNs. The ECS Task then assumes those cross-account roles and iterates through all IAM roles in each account to find those associated with your IdP for Azure AD.
  5. The ECS Task connects to the Azure AD SSO application and retrieves the existing manifest. Notice that, although you manually retrieved the manifest file during setup, it still needs to be fetched again every time to make sure it’s the latest version. The one you manually downloaded is used to retrieve parameters needed for setup, such as the application identifier or entity ID.
  6. ECS Task stores the existing manifest as a backup in a highly-durable S3 bucket. In case anything goes wrong, the last working state of the application manifest is always available in the S3 bucket. These files are stored with the exact time of their retrieval as their file name. You can find the correct version based on the point in time it was retrieved.
  7. The ECS Task generates a new manifest based on your AWS account/roles as inspected in the preceding steps. It uses the Azure AD credentials retrieved from AWS Secrets Manager and uses them to update the Azure AD SSO app with the new manifest. It also creates any required Azure AD Groups according to the specified custom naming convention. This makes it easier for the Azure AD administrator to map Azure AD users to AWS roles and entitle them to assume those roles.

Prerequisites

To start, download a copy of the sample code package.

You must have AWS Organizations enabled on all of your accounts to take advantage of this solution’s automation. Using AWS Organizations, you can configure one of your accounts as the root account and all other accounts will join your organization as child accounts. The root account will be trusted by all child accounts, so you can manage your child account resources from your root account. This trust is enabled using a role in each of your child accounts. AWS Organizations creates a default role with full permissions on child accounts that are directly created using AWS Organizations. Best practice is to delete this default role and create one with privileges restricted to your requirements. A sample role, named AWSCloudFormationStackSetExecutionRole, is included in cross-account-role-cfn.json
of my code package. You should modify this template based on your requirements.

Setup steps

In following sections, I’ll show the steps to setup federation and deploy the automation package. First, I’ll show the steps to prepare Azure Active Directory for federation. After that, you’ll see how you can configure all of your AWS accounts from a central place, regardless of the number of your accounts. The last step is to deploy the automation package in your master AWS account to automatically handle ongoing changes as you go.

Step 1: Configure Microsoft Azure Active Directory

You need to create two resources on your Azure AD tenant: a User and an Enterprise Application.

First thing you need for accessing Azure AD is an Azure AD user. In following the principle of least privilege, you want a user that can only manipulate the SSO application. Azure AD users with the directory role of User will only have access to resources they “own.” Therefore, you can create a new user specifically for this purpose and assign it as the owner of your SSO app. This user will be used by the automation to access Azure AD and update the SSO app.

Here’s how you can create a user with the directory role of User (default):

  1. Open Azure Portal.
  2. Open Azure Active Directory.
  3. In the left pane, select Users.
  4. In the Manage pane, select All users.
  5. Select New user.
  6. Enter values for the Name and User name fields.
  7. Select the Show Password box and note the auto-generated password for this user. You will need it when you change the password.
  8. Select Create.
  9. Open a browser window and go to https://login.microsoftonline.com.
  10. Log in with the new user. You’ll be prompted to change your password. Note the new password so you don’t forget it.

Next, create an Enterprise Application from the Azure AD application gallery:

  1. Open Azure Portal.
  2. Open Azure Active Directory.
  3. In the Manage pane, select Enterprise applications.
  4. Select New application.
  5. In the gallery text box, type AWS.
  6. You’ll see an option with the name Amazon Web Services (AWS). Select that application. Make sure you don’t choose the other option with the name “AWS Console.” That option uses an alternate integration method that isn’t relevant to this post.
  7.  

    Figure 3: Select "Amazon Web Services (AWS)

    Figure 3: Select “Amazon Web Services (AWS)

  8. Select Add. You can change the name to any name you would prefer.
  9. Open the application using this path: Azure Portal > Azure Active Directory > Enterprise Applications > All Applications > your application name (for example, “Amazon Web Services (AWS)”).
  10. From left pane, select Single Sign-on, and then set Single Sign-on mode to SAML-based Sign-on.
  11. The first instance of the app is pre-integrated with Azure AD and requires no mandatory URL settings. However, if you previously created a similar application, you’ll see this:
  12.  

    Figure 4: Azure AD Application Identifier

    Figure 4: Azure AD Application Identifier

  13. If you see the red “Required” value in the Identifier field, select the Edit button and enter a value for it. This can be any value you prefer (the default is https://signin.aws.amazon.com/saml), but it has to be unique within your Azure AD tenant. If you don’t see the Identifier field, it means it’s already prepopulated and you can proceed with the default value. However, if for any reason you prefer to have a custom Identifier value, you can select the Show advanced URL settings checkbox and enter the preferred value.
  14. In the User Attributes section, select the Edit button.
  15. You need to tell Azure AD what SAML attributes and values are expected and accepted on the AWS side. AWS requires two mandatory attributes in any incoming SAML assertion. The Role attribute defines which roles the federated user is allowed to assume. The RoleSessionName attribute defines the specific, traceable attribute for the user that will appear in AWS CloudTrail logs. Role and RoleSessionName are mandatory attributes. You can also use the optional attribute of SessionDuration to specify how long each session will be valid until the user is requested to get a new token. Add the following attributes to the User Attributes & Claims section in the Azure AD SSO application. You can also remove existing default attributes, if you want, because they’ll be ignored by AWS:

    Name (case-sensitive)ValueNamespace (case-sensitive)Required or optional?
    RoleSessionNameuser.userprincipalname
    (this will show logged in user ID in AWS portal, if you want user name, replace it with user.displayName)
    https://aws.amazon.com/SAML/AttributesRequired
    Roleuser.assignedroleshttps://aws.amazon.com/SAML/AttributesRequired
    SessionDurationAn integer between 900 seconds (15 minutes) and 43200 seconds (12 hours).https://aws.amazon.com/SAML/AttributesOptional

    Note: I assume that you use users that are directly created within your Azure AD tenant. If you’re using an external user such as a Hotmail, Live, or Gmail account for proof-of-concept purposes, RoleSessionName should be set to user.mail instead.

  16. As a good practice, when it approaches its expiration date, you can rotate your SAML certificate. For this purpose, Azure AD allows you to create additional certificates, but only one certificate can be active at a time. In the SAML Signing Certificate section, make sure the status of this certificate is Active, and then select Federation Metadata XML to download the XML document.
  17. Download the Metadata XML file and save it in the setup directory of the package you downloaded in the beginning of this walkthrough. Make sure you save it with file extension of .xml.
  18. Open Azure Portal > Azure Active Directory > App Registrations > your application name (for example, “Amazon Web Services (AWS)”). If you don’t see your application in the list on the App Registrations page, select All apps from the drop-down list on top of that page and search for it.
  19. Select Manifest. All Azure AD applications are described as a JavaScript Object Notification (JSON) document called manifest. For AWS, this manifest defines all AWS to Azure AD role mappings. Later, we’ll be using automation to generate updates to this file.
     
    Figure 5: Azure AD Application Manifest

    Figure 5: Azure AD Application Manifest

  20. Select Download to download the app manifest JSON file. Save it in the setup directory of the package you downloaded in the beginning of this walkthrough. Make sure you save it with file extension of .json.
  21. Now, back on your registered app, select Settings.
  22. In the Settings pane, select Owners.
     
    Figure 6: Application Owner

    Figure 6: Application Owner

  23. Select Add owner and add the user you created previously as owner of this application. Adding the Azure AD user as owner enables the user to manipulate this object. Since this application is the only Azure AD resource owned by our user, it means we’re enforcing the principle of least privilege on Azure AD side.

At this point, we’re done with the initial configuration of Azure AD. All remaining steps will be performed in your AWS accounts.

Step 2: Configure AWS IAM Identity Providers and Roles

In the previous section, I showed how to configure the Azure AD side represented in the Solution architecture in Figure 1. This section explains the AWS side.

As seen in Figure 1, enabling SAML federation in any AWS account requires two types of AWS IAM resources:

You’ll have to create these two resources in all of your AWS accounts participating in SAML federation. There are various options for doing this. You can:

  • Manually create IAM IdP and Roles using AWS Management Console. For one or two accounts, this might be the easiest way. But as the number of your AWS accounts and roles increase, this method becomes more difficult.
  • Use AWS CLI or AWS Tools for PowerShell. You can use these tools to write automation scripts and simplify both creation and maintenance of your roles.
  • Use AWS CloudFormation. CloudFormation templates enable structured definition of all resources and minimize the effort required to create and maintain them.

Here, I’m going to use CloudFormation and show how it can help you create up to thousands of roles in your organization, if you need that many.

Managing multiple AWS accounts from a root account

AWS CloudFormation simplifies provisioning and management on AWS. You can create templates for the service or application architectures you want and have AWS CloudFormation use those templates for quick and reliable provisioning of the services or applications (called “stacks“). You can also easily update or replicate the stacks as needed. Each stack is deployed in a single AWS account and a specific AWS Region. For example, you can write a template that defines your organization roles in AWS IAM and deploy it in your first AWS account and US East (N.Virginia) region.

But if you have hundreds of accounts, it wouldn’t be easy, and if you have time or budget constraints, sometimes not even possible to manually deploy your template in all accounts. Ideally, you’d want to manage all your accounts from a central place. AWS Organizations is the service that gives you this capability.

In my GitHub package there is a CloudFormation template named cross-account-roles-cfn.json. It’s located under the cfn directory. This template includes two cross-account roles. The first one is a role for cross-account access with the minimum required privileges for this solution that trusts your AWS Organizations master account. This role is used to deploy AWS IAM Identity Provider (IdP) for Azure AD and all SAML federation roles, trusting that IdP within all of your AWS child accounts. The second one is used by the automation to inspect your AWS accounts (through describe calls) and keep the Azure AD SSO application updated. I’ve created two roles to ensure that each component executes with the least privilege required. To recap, you’ll have two cross account roles for two different purposes:

  1. A role with full IAM access and Lambda execution permissions. This one is used for creation and maintenance of SAML IdP and associated IAM roles in all accounts.
  2. A role with IAM read-only access. This one is used by the update task to read and detect any changes in your federation IAM roles so it can update Azure AD SSO app with those changes.

You can deploy CloudFormation templates in your child accounts using CloudFormation StackSets. Log in to your root account, go to the CloudFormation console, and select StackSets.

Select Template is ready, select Upload a template file, and then select the cross-account-roles-cfn.json template to deploy it in all of your accounts. AWS IAM is a global service, so it makes no difference which region you choose for this template. You can select any region, such as us-east-1.
 

Figure 7: Upload template to StackSets console

Figure 7: Upload template to StackSets console

This template includes a parameter prompting you to enter root account number. For instructions to find your account number, see this page.

If you create your child accounts through AWS Organizations, you’ll be able to directly deploy StackSets in those child accounts. But, if you add existing accounts to you organization, you have to first manually deploy
cross-account-roles-cfn.json in your existing accounts. This template includes the IAM role and policies needed to enable your root account to execute StackSets on it.

Configure the SAML Identity Provider and Roles

A sample template to create your organization roles as SAML federation IAM roles is included in the saml-roles.json file in the same cfn directory. This template includes the SAML IdP and three sample roles trusting that IdP. The IdP is implemented as an AWS Lambda-backed CloudFormation custom resource. Included roles are samples using AWS IAM Job Functions for Administrator, Observer, and DBA. Modify this template by adding or removing roles as needed in your organization.

If you need different roles in some of your accounts, you’ll have to create separate copies of this template and modify them accordingly. From the CloudFormation StackSets console, you can choose the accounts to which your template should be deployed.

The last modification to make is on the IdentityProvider custom resource. It includes a <Metadata> property. Its value is defined as <MetadataDocument>. You’d have to replace the value with the content of the SAML certificate metadata XML document that you previously saved in the setup directory (see the Configure Microsoft Azure Active Directory section above). You’ll need to escape all of the quotation marks (“) in the XML string with a backslash (\). If you don’t want to do this manually, you can copy the saml-roles.json template file in the setup directory and as you follow the remainder of instructions in this post, my setup script will do that for you.

Step 3: Updating Azure AD from the root AWS account

The third and last template in the cfn directory is setup-env-cfn-template.json. You have to deploy this template only in your root account. This template creates all the components in your root account, as shown in Figure 8. These are resources needed to run the update task and keep Azure AD SSO App updated with your IAM roles. In addition, it also creates a temporary EC2 instance for initial configuration of that update task. The update task uses AWS Fargate, a serverless service that allows you to run Docker containers in AWS. You have to deploy the setup-env-cfn-template.json template in a region where Fargate is available. Check the AWS Region Table to make sure Fargate is available in your target region. Follow these steps to deploy the stack:

  1. Log in to your root account and open the CloudFormation console page.
  2. Select Create Stack, upload the setup-env-cfn-template.json file, and then select Next.
  3. Enter a stack name, such as aws-iam-aad. The stack name must be all lowercase letters. The template uses the stack name to create an S3 bucket, and because S3 does not allow capital letters, if you choose a stack name containing capital letters, the stack creation will fail. The stack name is also used as the appName parameter in all scripts, and all Parameter Store parameter names are prefixed with it.
  4. Enter and select values for the following parameters:
    1. azureADTenantName: You can get the Azure Active Directory Tenant Name from Azure Portal. Go to the Azure Active Directory Overview page and the tenant name should appear at the top of the page. During setup, this is used as the value for the parameter.
    2. ExecFrequency is the time period for the update task to run. For example, if you enter 30, every 30 minutes Azure AD will be updated with any changes in IAM roles of your AWS accounts.
    3. KeyName is a key pair that is used for login and accessing the EC2 instance. You’ll need to have a key pair created before deploying this template. To create a key pair, follow these instructions: Amazon EC2 Key Pairs. Also, for more convenience, if you’re using a MAC or Linux, you can copy your private key in the setup directory. Don’t forget to run chmod 600 <key name> to change the permissions on the key.
    4. NamingConvention is used to map AWS IAM roles to Azure AD roles. The default naming convention is: “AWS {0} – {1}”. The value of {0} is your account number. The value of {1} is the name of your IAM Role.
    5. SSHLocation is used in a Security Group that restricts access to the setup EC2 instance. You only need this instance for initial setup; therefore, the best practice and most secure option is to change this value to your specific IP address. In any case, make sure you only allow access to your internal IP address range.
    6. Subnet is the VPC subnet in which you want the update task to run. This subnet must have egress (outgoing) internet connectivity. The update task needs this to reach Azure AD Graph API endpoints.
       
      Figure 8: Enter parameters for automation stack

      Figure 8: Enter parameters for automation stack

Once you deploy this template in CloudFormation and the associated stack is successfully created, you can get the IP address of the setup EC2 instance from the Output tab in CloudFormation. Now, follow the steps below to complete the setup.

Note: At this point, in addition to all the files already included in the original package, you have two additional, modified files in the setup directory:

  • The SAML Certificate XML file from Azure AD
  • The App Manifest JSON file from Azure AD

Make sure you have following information handy. This info is required in some of the steps:

Now, follow these steps to complete the setup:

  1. If you’re using Mac, Linux, or UNIX, run the initiate_setup.sh script in the setup directory and, when prompted, provide the IP address from the previous procedure. It will copy all the required files to the target setup EC2 instance and automatically take you to the setup.ps1 script. Now, skip to step 3 below.
  2. If you’re using Windows on your local computer, use your favorite tool (such as WinSCP) to copy both setup and docker directories from your local computer to the /home/ec2-user/scripts directory on the target EC2 instance.
  3. Once copied, use your favorite SSH tool to log in to the target setup EC2 instance. For example, you can use PuTTY for this purpose. As soon as you log in, Setup.ps1 will automatically run for you.
  4. Setup.ps1 is interactive. It will prompt for the path to the three files you saved in the setup directory, and also for your Azure AD user credentials. Use the credentials of the user you created in step 1 of the Configure Microsoft Azure Active Directory section. The script will perform following tasks:
    1. Store Azure AD credentials securely in AWS Secrets Manager. The script also extracts necessary values out of the three input files and stores them as additional parameters in AWS Systems Manager (SSM) Parameter Store.

      Important: The credentials of your Azure user will be stored in AWS Secrets Manager. You must make sure that access to Secrets Manager is restricted to users who are also authorized to retrieve these credentials.

    2. Create a Docker image and push it into an AWS Elastic Container Registry (ECR) repository that’s created as part of the CloudFormation template.
    3. The script checks if saml-roles.json is available in setup directory. If it’s available, the script will replace the value of the Metadata property in the IdP custom resource with content of the SAML metadata XML file. It also generates a text file containing a comma-separated list of all your child accounts, extracting account numbers from cross-account-roles-cfn.json. Both of these are copied to the S3 bucket that is created as part of the template. You can use these at any time to deploy, maintain, and manage your SAML roles in child accounts using CloudFormation StackSets.
    4. If saml-roles.json is available, the script will prompt whether you want it to deploy your roles on your behalf. If you select yes (“y“), it will immediately deploy the template in all child accounts. You can also select no (“n“), if you prefer to do this at another time, or if you need different templates and roles in some accounts.
  5. Once the script executes and successfully completes, you should terminate the setup EC2 instance.

You’ve now completed setting up federation on both sides. All AWS IAM roles that trust an IdP with the SAML certificate of your Azure AD (the Metadata XML file) will now automatically be replicated into your Azure AD tenant. This will take place with the frequency you have defined. Therefore, if you have set the ExecFrequency parameter to “30“, after 30 minutes you’ll see the roles replicated in Azure AD.

But to enable your users to use this federation, you have to entitle them to assume roles, which is what I’ll cover in the next section.

Entitling Azure AD users to assume AWS Roles

  1. Open Azure Portal > Azure Active Directory >
    Enterprise applications > All applications > (your application name) > Users and groups.
  2. Select Add user.
  3. In the Users and groups pane, select one of your Azure AD users (or groups), and then select Select.
  4. Select the Select role pane and, on the right hand side, you should now see your AWS roles listed.

You can add and map Azure AD users or groups to AWS IAM roles this way. By adding users to these groups, you’re giving them access to those roles in AWS through pre-established trust. In the case of Groups, any Azure AD users inside that Group will have SSO access to the AWS Console and permitted to assume AWS roles/accounts associated with their Azure AD Group. Azure AD users who are authenticated against login.microsoftonline.com can go to their Access Panel (myapps.microsoft.com) and select the AWS app icon.

Application maintenance

Most of the time, you will not need to do anything else because the Fargate task will execute on each interval and keep the Azure AD manifest aligned with your AWS accounts and roles. However, there are two situations that might require you to take action:

  • If you rotate your Azure AD SAML certificate
  • If you rotate the Azure user credentials used for synchronization

You can use AWS Secrets Manager lifecycle management capabilities to automate the process for the second case. Otherwise, in the event of either of these two situations, you can modify the corresponding values using the AWS Systems Manager Parameter Store and Secrets Manager consoles. Open the Parameter Store console and find parameters having names prefixed with your setup-env-cfn-template.json stack name (you entered this name when you were creating the stack). In case you rotate your Azure AD SAML certificate, you should also update all of your IdP resources in AWS accounts to use the new resource. Here again, StackSets can do the heavy-lifting for you. Use the same saml-roles.json template to update all of your Stack Instances through CloudFormation. You’ll have to replace the Metadata property value with content of the new certificate, and replace quotation mark characters (“) with escaped quotes (\”).

Summary

I’ve demonstrated how to set up and configure SAML federation and SSO using Azure Active Directory to AWS Console following these principles and requirements:

  • Using security best practices to keep both sides of federation (AWS and Azure) secure
  • Saving time and effort by automating the manual effort needed to synchronize two sides of federation
  • Keeping operation cost to a minimum through a serverless solution

If you have feedback about this blog post, submit comments in the Comments section below. If you have questions about this blog post, start a new thread in the forums.

Want more AWS Security how-to content, news, and feature announcements? Follow us on Twitter.

Sepehr Samiei

Sepehr is currently a Senior Microsoft Tech Specialized Solutions Architect at AWS. He started his professional career as a .Net developer, which continued for more than 10 years. Early on, he quickly became a fan of cloud computing and he loves to help customers utilise the power of Microsoft tech on AWS. His wife and daughter are the most precious parts of his life.

EC2 Instance Update – M5 Instances with Local NVMe Storage (M5d)

Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/ec2-instance-update-m5-instances-with-local-nvme-storage-m5d/

Earlier this month we launched the C5 Instances with Local NVMe Storage and I told you that we would be doing the same for additional instance types in the near future!

Today we are introducing M5 instances equipped with local NVMe storage. Available for immediate use in 5 regions, these instances are a great fit for workloads that require a balance of compute and memory resources. Here are the specs:

Instance NamevCPUsRAMLocal StorageEBS-Optimized BandwidthNetwork Bandwidth
m5d.large28 GiB1 x 75 GB NVMe SSDUp to 2.120 GbpsUp to 10 Gbps
m5d.xlarge416 GiB1 x 150 GB NVMe SSDUp to 2.120 GbpsUp to 10 Gbps
m5d.2xlarge832 GiB1 x 300 GB NVMe SSDUp to 2.120 GbpsUp to 10 Gbps
m5d.4xlarge1664 GiB1 x 600 GB NVMe SSD2.210 GbpsUp to 10 Gbps
m5d.12xlarge48192 GiB2 x 900 GB NVMe SSD5.0 Gbps10 Gbps
m5d.24xlarge96384 GiB4 x 900 GB NVMe SSD10.0 Gbps25 Gbps

The M5d instances are powered by Custom Intel® Xeon® Platinum 8175M series processors running at 2.5 GHz, including support for AVX-512.

You can use any AMI that includes drivers for the Elastic Network Adapter (ENA) and NVMe; this includes the latest Amazon Linux, Microsoft Windows (Server 2008 R2, Server 2012, Server 2012 R2 and Server 2016), Ubuntu, RHEL, SUSE, and CentOS AMIs.

Here are a couple of things to keep in mind about the local NVMe storage on the M5d instances:

Naming – You don’t have to specify a block device mapping in your AMI or during the instance launch; the local storage will show up as one or more devices (/dev/nvme*1 on Linux) after the guest operating system has booted.

Encryption – Each local NVMe device is hardware encrypted using the XTS-AES-256 block cipher and a unique key. Each key is destroyed when the instance is stopped or terminated.

Lifetime – Local NVMe devices have the same lifetime as the instance they are attached to, and do not stick around after the instance has been stopped or terminated.

Available Now
M5d instances are available in On-Demand, Reserved Instance, and Spot form in the US East (N. Virginia), US West (Oregon), EU (Ireland), US East (Ohio), and Canada (Central) Regions. Prices vary by Region, and are just a bit higher than for the equivalent M5 instances.

Jeff;

 

Kernel 4.17 released

Post Syndicated from corbet original https://lwn.net/Articles/756373/rss

Linus has released the 4.17 kernel, which
will indeed be called “4.17”.
No, I didn’t call it 5.0, even though all the git object count
numerology was in place for that. It will happen in the not _too_
distant future, and I’m told all the release scripts on kernel.org are
ready for it, but I didn’t feel there was any real reason for it.

Headline features in this release include
improved load estimation in the CPU
scheduler,
raw
BPF tracepoints
,
lazytime support in the XFS filesystem,
full in-kernel TLS protocol support,
histogram triggers for tracing,
mitigations for the latest Spectre variants,
and, of course, the removal of support for eight unloved processor
architectures.

Friday Squid Blogging: Do Cephalopods Contain Alien DNA?

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2018/06/friday_squid_bl_627.html

Maybe not DNA, but biological somethings.

Cause of Cambrian explosion — Terrestrial or Cosmic?“:

Abstract: We review the salient evidence consistent with or predicted by the Hoyle-Wickramasinghe (H-W) thesis of Cometary (Cosmic) Biology. Much of this physical and biological evidence is multifactorial. One particular focus are the recent studies which date the emergence of the complex retroviruses of vertebrate lines at or just before the Cambrian Explosion of ~500 Ma. Such viruses are known to be plausibly associated with major evolutionary genomic processes. We believe this coincidence is not fortuitous but is consistent with a key prediction of H-W theory whereby major extinction-diversification evolutionary boundaries coincide with virus-bearing cometary-bolide bombardment events. A second focus is the remarkable evolution of intelligent complexity (Cephalopods) culminating in the emergence of the Octopus. A third focus concerns the micro-organism fossil evidence contained within meteorites as well as the detection in the upper atmosphere of apparent incoming life-bearing particles from space. In our view the totality of the multifactorial data and critical analyses assembled by Fred Hoyle, Chandra Wickramasinghe and their many colleagues since the 1960s leads to a very plausible conclusion — life may have been seeded here on Earth by life-bearing comets as soon as conditions on Earth allowed it to flourish (about or just before 4.1 Billion years ago); and living organisms such as space-resistant and space-hardy bacteria, viruses, more complex eukaryotic cells, fertilised ova and seeds have been continuously delivered ever since to Earth so being one important driver of further terrestrial evolution which has resulted in considerable genetic diversity and which has led to the emergence of mankind.

Two commentaries.

This is almost certainly not true.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

Read my blog posting guidelines here.

[$] Statistics from the 4.17 kernel development cycle

Post Syndicated from corbet original https://lwn.net/Articles/756031/rss

The 4.17 kernel appears to be on track for a June 3 release, barring an
unlikely last-minute surprise. So the time has come for the usual look at
some development statistics for this cycle. While 4.17 is a normal cycle
for the most part, it does have one characteristic of note: it is the third
kernel release ever to be smaller (in terms of lines of code) than its
predecessor.

Randomly generated, thermal-printed comics

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/random-comic-strip-generation-vomit-comic-robot/

Python code creates curious, wordless comic strips at random, spewing them from the thermal printer mouth of a laser-cut body reminiscent of Disney Pixar’s WALL-E: meet the Vomit Comic Robot!

The age of the thermal printer!

Thermal printers allow you to instantly print photos, data, and text using a few lines of code, with no need for ink. More and more makers are using this handy, low-maintenance bit of kit for truly creative projects, from Pierre Muth’s tiny PolaPi-Zero camera to the sound-printing Waves project by Eunice Lee, Matthew Zhang, and Bomani McClendon (and our own Secret Santa Babbage).

Vomiting robots

Interaction designer and developer Cadin Batrack, whose background is in game design and interactivity, has built the Vomit Comic Robot, which creates “one-of-a-kind comics on demand by processing hand-drawn images through a custom software algorithm.”

The robot is made up of a Raspberry Pi 3, a USB thermal printer, and a handful of LEDs.

Comic Vomit Robot Cadin Batrack's Raspberry Pi comic-generating thermal printer machine

At the press of a button, Processing code selects one of a set of Cadin’s hand-drawn empty comic grids and then randomly picks images from a library to fill in the gaps.

Vomit Comic Robot Cadin Batrack's Raspberry Pi comic-generating thermal printer machine

Each image is associated with data that allows the code to fit it correctly into the available panels. Cadin says about the concept behing his build:

Although images are selected and placed randomly, the comic panel format suggests relationships between elements. Our minds create a story where there is none in an attempt to explain visuals created by a non-intelligent machine.

The Raspberry Pi saves the final image as a high-resolution PNG file (so that Cadin can sell prints on thick paper via Etsy), and a Python script sends it to be vomited up by the thermal printer.

Comic Vomit Robot Cadin Batrack's Raspberry Pi comic-generating thermal printer machine

For more about the Vomit Comic Robot, check out Cadin’s blog. If you want to recreate it, you can find the info you need in the Imgur album he has put together.

We ❤ cute robots

We have a soft spot for cute robots here at Pi Towers, and of course we make no exception for the Vomit Comic Robot. If, like us, you’re a fan of adorable bots, check out Mira, the tiny interactive robot by Alonso Martinez, and Peeqo, the GIF bot by Abhishek Singh.

Mira Alfonso Martinez Raspberry Pi

The post Randomly generated, thermal-printed comics appeared first on Raspberry Pi.

Security and Human Behavior (SHB 2018)

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2018/05/security_and_hu_7.html

I’m at Carnegie Mellon University, at the eleventh Workshop on Security and Human Behavior.

SHB is a small invitational gathering of people studying various aspects of the human side of security, organized each year by Alessandro Acquisti, Ross Anderson, and myself. The 50 or so people in the room include psychologists, economists, computer security researchers, sociologists, political scientists, neuroscientists, designers, lawyers, philosophers, anthropologists, business school professors, and a smattering of others. It’s not just an interdisciplinary event; most of the people here are individually interdisciplinary.

The goal is to maximize discussion and interaction. We do that by putting everyone on panels, and limiting talks to 7-10 minutes. The rest of the time is left to open discussion. Four hour-and-a-half panels per day over two days equals eight panels; six people per panel means that 48 people get to speak. We also have lunches, dinners, and receptions — all designed so people from different disciplines talk to each other.

I invariably find this to be the most intellectually stimulating conference of my year. It influences my thinking in many different, and sometimes surprising, ways.

This year’s program is here. This page lists the participants and includes links to some of their work. As he does every year, Ross Anderson is liveblogging the talks. (Ross also maintains a good webpage of psychology and security resources.)

Here are my posts on the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, and tenth SHB workshops. Follow those links to find summaries, papers, and occasionally audio recordings of the various workshops.

Next year, I’ll be hosting the event at Harvard.

The devil wears Pravda

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/05/the-devil-wears-pravda.html

Classic Bond villain, Elon Musk, has a new plan to create a website dedicated to measuring the credibility and adherence to “core truth” of journalists. He is, without any sense of irony, going to call this “Pravda”. This is not simply wrong but evil.

Musk has a point. Journalists do suck, and many suck consistently. I see this in my own industry, cybersecurity, and I frequently criticize them for their suckage.

But what he’s doing here is not correcting them when they make mistakes (or what Musk sees as mistakes), but questioning their legitimacy. This legitimacy isn’t measured by whether they follow established journalism ethics, but whether their “core truths” agree with Musk’s “core truths”.

An example of the problem is how the press fixates on Tesla car crashes due to its “autopilot” feature. Pretty much every autopilot crash makes national headlines, while the press ignores the other 40,000 car crashes that happen in the United States each year. Musk spies on Tesla drivers (hello, classic Bond villain everyone) so he can see the dip in autopilot usage every time such a news story breaks. He’s got good reason to be concerned about this.

He argues that autopilot is safer than humans driving, and he’s got the statistics and government studies to back this up. Therefore, the press’s fixation on Tesla crashes is illegitimate “fake news”, titillating the audience with distorted truth.

But here’s the thing: that’s still only Musk’s version of the truth. Yes, on a mile-per-mile basis, autopilot is safer, but there’s nuance here. Autopilot is used primarily on freeways, which already have a low mile-per-mile accident rate. People choose autopilot only when conditions are incredibly safe and drivers are unlikely to have an accident anyway. Musk is therefore being intentionally deceptive comparing apples to oranges. Autopilot may still be safer, it’s just that the numbers Musk uses don’t demonstrate this.

And then there is the truth calling it “autopilot” to begin with, because it isn’t. The public is overrating the capabilities of the feature. It’s little different than “lane keeping” and “adaptive cruise control” you can now find in other cars. In many ways, the technology is behind — my Tesla doesn’t beep at me when a pedestrian walks behind my car while backing up, but virtually every new car on the market does.

Yes, the press unduly covers Tesla autopilot crashes, but Musk has only himself to blame by unduly exaggerating his car’s capabilities by calling it “autopilot”.

What’s “core truth” is thus rather difficult to obtain. What the press satisfies itself with instead is smaller truths, what they can document. The facts are in such cases that the accident happened, and they try to get Tesla or Musk to comment on it.

What you can criticize a journalist for is therefore not “core truth” but whether they did journalism correctly. When such stories criticize “autopilot”, but don’t do their diligence in getting Tesla’s side of the story, then that’s a violation of journalistic practice. When I criticize journalists for their poor handling of stories in my industry, I try to focus on which journalistic principles they get wrong. For example, the NYTimes reporters do a lot of stories quoting anonymous government sources in clear violation of journalistic principles.

If “credibility” is the concern, then it’s the classic Bond villain here that’s the problem: Musk himself. His track record on business statements is abysmal. For example, when he announced the Model 3 he claimed production targets that every Wall Street analyst claimed were absurd. He didn’t make those targets, he didn’t come close. Model 3 production is still lagging behind Musk’s twice adjusted targets.

https://www.bloomberg.com/graphics/2018-tesla-tracker/

So who has a credibility gap here, the press, or Musk himself?

Not only is Musk’s credibility problem ironic, so is the name he chose, “Pravada”, the Russian word for truth that was the name of the Soviet Union Communist Party’s official newspaper. This is so absurd this has to be a joke, yet Musk claims to be serious about all this.

Yes, the press has a lot of problems, and if Musk were some journalism professor concerned about journalists meeting the objective standards of their industry (e.g. abusing anonymous sources), then this would be a fine thing. But it’s not. It’s Musk who is upset the press’s version of “core truth” does not agree with his version — a version that he’s proven time and time again differs from “real truth”.

Just in case Musk is serious, I’ve already registered “www.antipravda.com” to start measuring the credibility of statements by billionaire playboy CEOs. Let’s see who blinks first.


I stole the title, with permission, from this tweet:

C is to low level

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/05/c-is-too-low-level.html

I’m in danger of contradicting myself, after previously pointing out that x86 machine code is a high-level language, but this article claiming C is a not a low level language is bunk. C certainly has some problems, but it’s still the closest language to assembly. This is obvious by the fact it’s still the fastest compiled language. What we see is a typical academic out of touch with the real world.

The author makes the (wrong) observation that we’ve been stuck emulating the PDP-11 for the past 40 years. C was written for the PDP-11, and since then CPUs have been designed to make C run faster. The author imagines a different world, such as where CPU designers instead target something like LISP as their preferred language, or Erlang. This misunderstands the state of the market. CPUs do indeed supports lots of different abstractions, and C has evolved to accommodate this.


The author criticizes things like “out-of-order” execution which has lead to the Spectre sidechannel vulnerabilities. Out-of-order execution is necessary to make C run faster. The author claims instead that those resources should be spent on having more slower CPUs, with more threads. This sacrifices single-threaded performance in exchange for a lot more threads executing in parallel. The author cites Sparc Tx CPUs as his ideal processor.

But here’s the thing, the Sparc Tx was a failure. To be fair, it’s mostly a failure because most of the time, people wanted to run old C code instead of new Erlang code. But it was still a failure at running Erlang.

Time after time, engineers keep finding that “out-of-order”, single-threaded performance is still the winner. A good example is ARM processors for both mobile phones and servers. All the theory points to in-order CPUs as being better, but all the products are out-of-order, because this theory is wrong. The custom ARM cores from Apple and Qualcomm used in most high-end phones are so deeply out-of-order they give Intel CPUs competition. The same is true on the server front with the latest Qualcomm Centriq and Cavium ThunderX2 processors, deeply out of order supporting more than 100 instructions in flight.

The Cavium is especially telling. Its ThunderX CPU had 48 simple cores which was replaced with the ThunderX2 having 32 complex, deeply out-of-order cores. The performance increase was massive, even on multithread-friendly workloads. Every competitor to Intel’s dominance in the server space has learned the lesson from Sparc Tx: many wimpy cores is a failure, you need fewer beefy cores. Yes, they don’t need to be as beefy as Intel’s processors, but they need to be close.

Even Intel’s “Xeon Phi” custom chip learned this lesson. This is their GPU-like chip, running 60 cores with 512-bit wide “vector” (sic) instructions, designed for supercomputer applications. Its first version was purely in-order. Its current version is slightly out-of-order. It supports four threads and focuses on basic number crunching, so in-order cores seems to be the right approach, but Intel found in this case that out-of-order processing still provided a benefit. Practice is different than theory.

As an academic, the author of the above article focuses on abstractions. The criticism of C is that it has the wrong abstractions which are hard to optimize, and that if we instead expressed things in the right abstractions, it would be easier to optimize.

This is an intellectually compelling argument, but so far bunk.

The reason is that while the theoretical base language has issues, everyone programs using extensions to the language, like “intrinsics” (C ‘functions’ that map to assembly instructions). Programmers write libraries using these intrinsics, which then the rest of the normal programmers use. In other words, if your criticism is that C is not itself low level enough, it still provides the best access to low level capabilities.

Given that C can access new functionality in CPUs, CPU designers add new paradigms, from SIMD to transaction processing. In other words, while in the 1980s CPUs were designed to optimize C (stacks, scaled pointers), these days CPUs are designed to optimize tasks regardless of language.

The author of that article criticizes the memory/cache hierarchy, claiming it has problems. Yes, it has problems, but only compared to how well it normally works. The author praises the many simple cores/threads idea as hiding memory latency with little caching, but misses the point that caches also dramatically increase memory bandwidth. Intel processors are optimized to read a whopping 256 bits every clock cycle from L1 cache. Main memory bandwidth is orders of magnitude slower.

The author goes onto criticize cache coherency as a problem. C uses it, but other languages like Erlang don’t need it. But that’s largely due to the problems each languages solves. Erlang solves the problem where a large number of threads work on largely independent tasks, needing to send only small messages to each other across threads. The problems C solves is when you need many threads working on a huge, common set of data.

For example, consider the “intrusion prevention system”. Any thread can process any incoming packet that corresponds to any region of memory. There’s no practical way of solving this problem without a huge coherent cache. It doesn’t matter which language or abstractions you use, it’s the fundamental constraint of the problem being solved. RDMA is an important concept that’s moved from supercomputer applications to the data center, such as with memcached. Again, we have the problem of huge quantities (terabytes worth) shared among threads rather than small quantities (kilobytes).

The fundamental issue the author of the the paper is ignoring is decreasing marginal returns. Moore’s Law has gifted us more transistors than we can usefully use. We can’t apply those additional registers to just one thing, because the useful returns we get diminish.

For example, Intel CPUs have two hardware threads per core. That’s because there are good returns by adding a single additional thread. However, the usefulness of adding a third or fourth thread decreases. That’s why many CPUs have only two threads, or sometimes four threads, but no CPU has 16 threads per core.

You can apply the same discussion to any aspect of the CPU, from register count, to SIMD width, to cache size, to out-of-order depth, and so on. Rather than focusing on one of these things and increasing it to the extreme, CPU designers make each a bit larger every process tick that adds more transistors to the chip.

The same applies to cores. It’s why the “more simpler cores” strategy fails, because more cores have their own decreasing marginal returns. Instead of adding cores tied to limited memory bandwidth, it’s better to add more cache. Such cache already increases the size of the cores, so at some point it’s more effective to add a few out-of-order features to each core rather than more cores. And so on.

The question isn’t whether we can change this paradigm and radically redesign CPUs to match some academic’s view of the perfect abstraction. Instead, the goal is to find new uses for those additional transistors. For example, “message passing” is a useful abstraction in languages like Go and Erlang that’s often more useful than sharing memory. It’s implemented with shared memory and atomic instructions, but I can’t help but think it couldn’t better be done with direct hardware support.

Of course, as soon as they do that, it’ll become an intrinsic in C, then added to languages like Go and Erlang.

Summary

Academics live in an ideal world of abstractions, the rest of us live in practical reality. The reality is that vast majority of programmers work with the C family of languages (JavaScript, Go, etc.), whereas academics love the epiphanies they learned using other languages, especially function languages. CPUs are only superficially designed to run C and “PDP-11 compatibility”. Instead, they keep adding features to support other abstractions, abstractions available to C. They are driven by decreasing marginal returns — they would love to add new abstractions to the hardware because it’s a cheap way to make use of additional transitions. Academics are wrong believing that the entire system needs to be redesigned from scratch. Instead, they just need to come up with new abstractions CPU designers can add.

The Benefits of Side Projects

Post Syndicated from Bozho original https://techblog.bozho.net/the-benefits-of-side-projects/

Side projects are the things you do at home, after work, for your own “entertainment”, or to satisfy your desire to learn new stuff, in case your workplace doesn’t give you that opportunity (or at least not enough of it). Side projects are also a way to build stuff that you think is valuable but not necessarily “commercialisable”. Many side projects are open-sourced sooner or later and some of them contribute to the pool of tools at other people’s disposal.

I’ve outlined one recommendation about side projects before – do them with technologies that are new to you, so that you learn important things that will keep you better positioned in the software world.

But there are more benefits than that – serendipitous benefits, for example. And I’d like to tell some personal stories about that. I’ll focus on a few examples from my list of side projects to show how, through a sort-of butterfly effect, they helped shape my career.

The computoser project, no matter how cool algorithmic music composition, didn’t manage to have much of a long term impact. But it did teach me something apart from niche musical theory – how to read a bulk of scientific papers (mostly computer science) and understand them without being formally trained in the particular field. We’ll see how that was useful later.

Then there was the “State alerts” project – a website that scraped content from public institutions in my country (legislation, legislation proposals, decisions by regulators, new tenders, etc.), made them searchable, and “subscribable” – so that you get notified when a keyword of interest is mentioned in newly proposed legislation, for example. (I obviously subscribed for “information technologies” and “electronic”).

And that project turned out to have a significant impact on the following years. First, I chose a new technology to write it with – Scala. Which turned out to be of great use when I started working at TomTom, and on the 3rd day I was transferred to a Scala project, which was way cooler and much more complex than the original one I was hired for. It was a bit ironic, as my colleagues had just read that “I don’t like Scala” a few weeks earlier, but nevertheless, that was one of the most interesting projects I’ve worked on, and it went on for two years. Had I not known Scala, I’d probably be gone from TomTom much earlier (as the other project was restructured a few times), and I would not have learned many of the scalability, architecture and AWS lessons that I did learn there.

But the very same project had an even more important follow-up. Because if its “civic hacking” flavour, I was invited to join an informal group of developers (later officiated as an NGO) who create tools that are useful for society (something like MySociety.org). That group gathered regularly, discussed both tools and policies, and at some point we put up a list of policy priorities that we wanted to lobby policy makers. One of them was open source for the government, the other one was open data. As a result of our interaction with an interim government, we donated the official open data portal of my country, functioning to this day.

As a result of that, a few months later we got a proposal from the deputy prime minister’s office to “elect” one of the group for an advisor to the cabinet. And we decided that could be me. So I went for it and became advisor to the deputy prime minister. The job has nothing to do with anything one could imagine, and it was challenging and fascinating. We managed to pass legislation, including one that requires open source for custom projects, eID and open data. And all of that would not have been possible without my little side project.

As for my latest side project, LogSentinel – it became my current startup company. And not without help from the previous two mentioned above – the computer science paper reading was of great use when I was navigating the crypto papers landscape, and from the government job I not only gained invaluable legal knowledge, but I also “got” a co-founder.

Some other side projects died without much fanfare, and that’s fine. But the ones above shaped my “story” in a way that would not have been possible otherwise.

And I agree that such serendipitous chain of events could have happened without side projects – I could’ve gotten these opportunities by meeting someone at a bar (unlikely, but who knows). But we, as software engineers, are capable of tilting chance towards us by utilizing our skills. Side projects are our “extracurricular activities”, and they often lead to unpredictable, but rather positive chains of events. They would rarely be the only factor, but they are certainly great at unlocking potential.

The post The Benefits of Side Projects appeared first on Bozho's tech blog.

The Practical Effects of GDPR at Backblaze

Post Syndicated from Andy Klein original https://www.backblaze.com/blog/the-practical-effects-of-gdpr-at-backblaze/


GDPR day, May 25, 2018, is nearly here. On that day, will your inbox explode with update notices, opt-in agreements, and offers from lawyers searching for GDPR violators? Perhaps all the companies on earth that are not GDPR ready will just dissolve into dust. More likely, there will be some changes, but business as usual will continue and we’ll all be more aware of data privacy. Let’s go with the last one.

What’s Different With GDPR at Backblaze

The biggest difference you’ll notice is a completely updated Privacy Policy. Last week we sent out a service email announcing the new Privacy Policy. Some people asked what was different. Basically everything. About 95% of the agreement was rewritten. In the agreement, we added in the appropriate provisions required by GDPR, and hopefully did a better job specifying the data we collect from you, why we collect it, and what we are going to do with it.

As a reminder, at Backblaze your data falls into two catagories. The first type of data is the data you store with us — stored data. These are the files and objects you upload and store, and as needed, restore. We do not share this data. We do not process this data, except as requested by you to store and restore the data. We do not analyze this data looking for keywords, tags, images, etc. No one outside of Backblaze has access to this data unless you explicitly shared the data by providing that person access to one or more files.

The second type of data is your account data. Some of your account data is considered personal data. This is the information we collect from you to provide our Personal Backup, Business Backup and B2 Cloud Storage services. Examples include your email address to provide access to your account, or the name of your computer so we can organize your files like they are arranged on your computer to make restoration easier. We have written a number of Help Articles covering the different ways this information is collected and processed. In addition, these help articles outline the various “rights” granted via GDPR. We will continue to add help articles over the coming weeks to assist in making it easy to work with us to understand and exercise your rights.

What’s New With GDPR at Backblaze

The most obvious addition is the Data Processing Addendum (DPA). This covers how we protect the data you store with us, i.e. stored data. As noted above, we don’t do anything with your data, except store it and keep it safe until you need it. Now we have a separate document saying that.

It is important to note the new Data Processing Addendum is now incorporated by reference into our Terms of Service, which everyone agrees to when they sign up for any of our services. Now all of our customers have a shiny new Data Processing Agreement to go along with the updated Privacy Policy. We promise they are not long or complicated, and we encourage you to read them. If you have any questions, stop by our GDPR help section on our website.

Patience, Please

Every company we have dealt with over the last few months is working hard to comply with GDPR. It has been a tough road whether you tried to do it yourself or like Backblaze, hired an EU-based law firm for advice. Over the coming weeks and months as you reach out to discover and assert your rights, please have a little patience. We are all going through a steep learning curve as GDPR gets put into practice. Along the way there are certain to be some growing pains — give us a chance, we all want to get it right.

Regardless, at Backblaze we’ve been diligently protecting our customers’ data for over 11 years and nothing that will happen on May 25th will change that.

The post The Practical Effects of GDPR at Backblaze appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Working with the Scout Association on digital skills for life

Post Syndicated from Philip Colligan original https://www.raspberrypi.org/blog/working-with-scout-association-digital-skills-for-life/

Today we’re launching a new partnership between the Scouts and the Raspberry Pi Foundation that will help tens of thousands of young people learn crucial digital skills for life. In this blog post, I want to explain what we’ve got planned, why it matters, and how you can get involved.

This is personal

First, let me tell you why this partnership matters to me. As a child growing up in North Wales in the 1980s, Scouting changed my life. My time with 2nd Rhyl provided me with countless opportunities to grow and develop new skills. It taught me about teamwork and community in ways that continue to shape my decisions today.

As my own kids (now seven and ten) have joined Scouting, I’ve seen the same opportunities opening up for them, and like so many parents, I’ve come back to the movement as a volunteer to support their local section. So this is deeply personal for me, and the same is true for many of my colleagues at the Raspberry Pi Foundation who in different ways have been part of the Scouting movement.

That shouldn’t come as a surprise. Scouting and Raspberry Pi share many of the same values. We are both community-led movements that aim to help young people develop the skills they need for life. We are both powered by an amazing army of volunteers who give their time to support that mission. We both care about inclusiveness, and pride ourselves on combining fun with learning by doing.

Raspberry Pi

Raspberry Pi started life in 2008 as a response to the problem that too many young people were growing up without the skills to create with technology. Our goal is that everyone should be able to harness the power of computing and digital technologies, for work, to solve problems that matter to them, and to express themselves creatively.

In 2012 we launched our first product, the world’s first $35 computer. Just six years on, we have sold over 20 million Raspberry Pi computers and helped kickstart a global movement for digital skills.

The Raspberry Pi Foundation now runs the world’s largest network of volunteer-led computing clubs (Code Clubs and CoderDojos), and creates free educational resources that are used by millions of young people all over the world to learn how to create with digital technologies. And lots of what we are able to achieve is because of partnerships with fantastic organisations that share our goals. For example, through our partnership with the European Space Agency, thousands of young people have written code that has run on two Raspberry Pi computers that Tim Peake took to the International Space Station as part of his Mission Principia.

Digital makers

Today we’re launching the new Digital Maker Staged Activity Badge to help tens of thousands of young people learn how to create with technology through Scouting. Over the past few months, we’ve been working with the Scouts all over the UK to develop and test the new badge requirements, along with guidance, project ideas, and resources that really make them work for Scouting. We know that we need to get two things right: relevance and accessibility.

Relevance is all about making sure that the activities and resources we provide are a really good fit for Scouting and Scouting’s mission to equip young people with skills for life. From the digital compass to nature cameras and the reinvented wide game, we’ve had a lot of fun thinking about ways we can bring to life the crucial role that digital technologies can play in the outdoors and adventure.

Compass Coding with Raspberry Pi

We are beyond excited to be launching a new partnership with the Raspberry Pi Foundation, which will help tens of thousands of young people learn digital skills for life.

We also know that there are great opportunities for Scouts to use digital technologies to solve social problems in their communities, reflecting the movement’s commitment to social action. Today we’re launching the first set of project ideas and resources, with many more to follow over the coming weeks and months.

Accessibility is about providing every Scout leader with the confidence, support, and kit to enable them to offer the Digital Maker Staged Activity Badge to their young people. A lot of work and care has gone into designing activities that require very little equipment: for example, activities at Stages 1 and 2 can be completed with a laptop without access to the internet. For the activities that do require kit, we will be working with Scout Stores and districts to make low-cost kit available to buy or loan.

We’re producing accessible instructions, worksheets, and videos to help leaders run sessions with confidence, and we’ll also be planning training for leaders. We will work with our network of Code Clubs and CoderDojos to connect them with local sections to organise joint activities, bringing both kit and expertise along with them.




Get involved

Today’s launch is just the start. We’ll be developing our partnership over the next few years, and we can’t wait for you to join us in getting more young people making things with technology.

Take a look at the brand-new Raspberry Pi resources designed especially for Scouts, to get young people making and creating right away.

The post Working with the Scout Association on digital skills for life appeared first on Raspberry Pi.

CloudFrunt – Identify Misconfigured CloudFront Domains

Post Syndicated from Darknet original https://www.darknet.org.uk/2018/05/cloudfrunt-identify-misconfigured-cloudfront-domains/?utm_source=rss&utm_medium=social&utm_campaign=darknetfeed

CloudFrunt – Identify Misconfigured CloudFront Domains

CloudFrunt is a Python-based tool for identifying misconfigured CloudFront domains, it uses DNS and looks for CNAMEs which may be allowed to be associated with CloudFront distributions. This effectively allows for domain hijacking.

How CloudFrunt Works For Misconfigured CloudFront

CloudFront is a Content Delivery Network (CDN) provided by Amazon Web Services (AWS). CloudFront users create “distributions” that serve content from specific sources (an S3 bucket, for example).

Each CloudFront distribution has a unique endpoint for users to point their DNS records to (ex.

Read the rest of CloudFrunt – Identify Misconfigured CloudFront Domains now! Only available at Darknet.

EC2 Instance Update – C5 Instances with Local NVMe Storage (C5d)

Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/ec2-instance-update-c5-instances-with-local-nvme-storage-c5d/

As you can see from my EC2 Instance History post, we add new instance types on a regular and frequent basis. Driven by increasingly powerful processors and designed to address an ever-widening set of use cases, the size and diversity of this list reflects the equally diverse group of EC2 customers!

Near the bottom of that list you will find the new compute-intensive C5 instances. With a 25% to 50% improvement in price-performance over the C4 instances, the C5 instances are designed for applications like batch and log processing, distributed and or real-time analytics, high-performance computing (HPC), ad serving, highly scalable multiplayer gaming, and video encoding. Some of these applications can benefit from access to high-speed, ultra-low latency local storage. For example, video encoding, image manipulation, and other forms of media processing often necessitates large amounts of I/O to temporary storage. While the input and output files are valuable assets and are typically stored as Amazon Simple Storage Service (S3) objects, the intermediate files are expendable. Similarly, batch and log processing runs in a race-to-idle model, flushing volatile data to disk as fast as possible in order to make full use of compute resources.

New C5d Instances with Local Storage
In order to meet this need, we are introducing C5 instances equipped with local NVMe storage. Available for immediate use in 5 regions, these instances are a great fit for the applications that I described above, as well as others that you will undoubtedly dream up! Here are the specs:

Instance NamevCPUsRAMLocal StorageEBS BandwidthNetwork Bandwidth
c5d.large24 GiB1 x 50 GB NVMe SSDUp to 2.25 GbpsUp to 10 Gbps
c5d.xlarge48 GiB1 x 100 GB NVMe SSDUp to 2.25 GbpsUp to 10 Gbps
c5d.2xlarge816 GiB1 x 225 GB NVMe SSDUp to 2.25 GbpsUp to 10 Gbps
c5d.4xlarge1632 GiB1 x 450 GB NVMe SSD2.25 GbpsUp to 10 Gbps
c5d.9xlarge3672 GiB1 x 900 GB NVMe SSD4.5 Gbps10 Gbps
c5d.18xlarge72144 GiB2 x 900 GB NVMe SSD9 Gbps25 Gbps

Other than the addition of local storage, the C5 and C5d share the same specs. Both are powered by 3.0 GHz Intel Xeon Platinum 8000-series processors, optimized for EC2 and with full control over C-states on the two largest sizes, giving you the ability to run two cores at up to 3.5 GHz using Intel Turbo Boost Technology.

You can use any AMI that includes drivers for the Elastic Network Adapter (ENA) and NVMe; this includes the latest Amazon Linux, Microsoft Windows (Server 2008 R2, Server 2012, Server 2012 R2 and Server 2016), Ubuntu, RHEL, SUSE, and CentOS AMIs.

Here are a couple of things to keep in mind about the local NVMe storage:

Naming – You don’t have to specify a block device mapping in your AMI or during the instance launch; the local storage will show up as one or more devices (/dev/nvme*1 on Linux) after the guest operating system has booted.

Encryption – Each local NVMe device is hardware encrypted using the XTS-AES-256 block cipher and a unique key. Each key is destroyed when the instance is stopped or terminated.

Lifetime – Local NVMe devices have the same lifetime as the instance they are attached to, and do not stick around after the instance has been stopped or terminated.

Available Now
C5d instances are available in On-Demand, Reserved Instance, and Spot form in the US East (N. Virginia), US West (Oregon), EU (Ireland), US East (Ohio), and Canada (Central) Regions. Prices vary by Region, and are just a bit higher than for the equivalent C5 instances.

Jeff;

PS – We will be adding local NVMe storage to other EC2 instance types in the months to come, so stay tuned!