Security updates have been issued by Debian (quagga), Mageia (freetype2, kernel-linus, and kernel-tmb), openSUSE (chromium, GraphicsMagick, mupdf, openssl-steam, and xen), Slackware (irssi), SUSE (glibc and quagga), and Ubuntu (quagga).
Security updates have been issued by Debian (jackson-databind, leptonlib, libvorbis, python-crypto, and xen), Fedora (apache-commons-email, ca-certificates, libreoffice, libxml2, mujs, p7zip, python-django, sox, and torbrowser-launcher), openSUSE (libreoffice), SUSE (libreoffice), and Ubuntu (advancecomp, erlang, and freetype).
Post Syndicated from Koen van Blijderveen original https://aws.amazon.com/blogs/security/how-to-patch-linux-workloads-on-aws/
Most malware tries to compromise your systems by using a known vulnerability that the operating system maker has already patched. As best practices to help prevent malware from affecting your systems, you should apply all operating system patches and actively monitor your systems for missing patches.
- Launch an Amazon EC2 instance for use with Systems Manager.
- Configure Systems Manager to patch your Amazon EC2 Linux instances.
In two previous blog posts (Part 1 and Part 2), I showed how to use the AWS Management Console to perform the necessary steps to patch, inspect, and protect Microsoft Windows workloads. You can implement those same processes for your Linux instances running in AWS by changing the instance tags and types shown in the previous blog posts.
Because most Linux system administrators are more familiar with using a command line, I show how to patch Linux workloads by using the AWS CLI in this blog post. The steps to use the Amazon EBS Snapshot Scheduler and Amazon Inspector are identical for both Microsoft Windows and Linux.
What you should know first
To follow along with the solution in this post, you need one or more Amazon EC2 instances. You may use existing instances or create new instances. For this post, I assume this is an Amazon EC2 for Amazon Linux instance installed from Amazon Machine Images (AMIs).
Systems Manager is a collection of capabilities that helps you automate management tasks for AWS-hosted instances on Amazon EC2 and your on-premises servers. In this post, I use Systems Manager for two purposes: to run remote commands and apply operating system patches. To learn about the full capabilities of Systems Manager, see What Is AWS Systems Manager?
As of Amazon Linux 2017.09, the AMI comes preinstalled with the Systems Manager agent. Systems Manager Patch Manager also supports Red Hat and Ubuntu. To install the agent on these Linux distributions or an older version of Amazon Linux, see Installing and Configuring SSM Agent on Linux Instances.
If you are not familiar with how to launch an Amazon EC2 instance, see Launching an Instance. I also assume you launched or will launch your instance in a private subnet. You must make sure that the Amazon EC2 instance can connect to the internet using a network address translation (NAT) instance or NAT gateway to communicate with Systems Manager. The following diagram shows how you should structure your VPC.
Later in this post, you will assign tasks to a maintenance window to patch your instances with Systems Manager. To do this, the IAM user you are using for this post must have the
iam:PassRole permission. This permission allows the IAM user assigning tasks to pass his own IAM permissions to the AWS service. In this example, when you assign a task to a maintenance window, IAM passes your credentials to Systems Manager. You also should authorize your IAM user to use Amazon EC2 and Systems Manager. As mentioned before, you will be using the AWS CLI for most of the steps in this blog post. Our documentation shows you how to get started with the AWS CLI. Make sure you have the AWS CLI installed and configured with an AWS access key and secret access key that belong to an IAM user that have the following AWS managed policies attached to the IAM user you are using for this example:
Step 1: Launch an Amazon EC2 Linux instance
In this section, I show you how to launch an Amazon EC2 instance so that you can use Systems Manager with the instance. This step requires you to do three things:
- Create an IAM role for Systems Manager before launching your Amazon EC2 instance.
- Launch your Amazon EC2 instance with Amazon EBS and the IAM role for Systems Manager.
- Add tags to the instances so that you can add your instances to a Systems Manager maintenance window based on tags.
A. Create an IAM role for Systems Manager
Before launching an Amazon EC2 instance, I recommend that you first create an IAM role for Systems Manager, which you will use to update the Amazon EC2 instance. AWS already provides a preconfigured policy that you can use for the new role and it is called
- Create a JSON file named
trustpolicy-ec2ssm.jsonthat contains the following trust policy. This policy describes which principal (an entity that can take action on an AWS resource) is allowed to assume the role we are going to create. In this example, the principal is the Amazon EC2 service.
- Use the following command to create a role named
EC2SSMthat has the AWS managed policy
AmazonEC2RoleforSSMattached to it. This generates JSON-based output that describes the role and its parameters, if the command is successful.
- Use the following command to attach the AWS managed IAM policy (
AmazonEC2RoleforSSM) to your newly created role.
- Use the following commands to create the IAM instance profile and add the role to the instance profile. The instance profile is needed to attach the role we created earlier to your Amazon EC2 instance.
B. Launch your Amazon EC2 instance
To follow along, you need an Amazon EC2 instance that is running Amazon Linux. You can use any existing instance you may have or create a new instance.
When launching a new Amazon EC2 instance, be sure that:
- The operating system is Amazon Linux.
- You attach the newly created IAM role (
- The Amazon EC2 instance can connect to Systems Manager through a network address translation (NAT) gateway or a NAT instance.
- Use the following command to launch a new Amazon EC2 instance using an Amazon Linux AMI available in the US East (N. Virginia) Region (also known as us-east-1). Replace
YourSubnetIdwith your information. For more information about creating a key pair, see the create-key-pair documentation. Write down the
InstanceIdthat is in the output because you will need it later in this post.
- If you are using an existing Amazon EC2 instance, you can use the following command to attach the instance profile you created earlier to your instance.
C. Add tags
The final step of configuring your Amazon EC2 instances is to add tags. You will use these tags to configure Systems Manager in Step 2 of this post. For this example, I add a tag named
Patch Group and set the value to
Linux Servers. I could have other groups of Amazon EC2 instances that I treat differently by having the same tag name but a different tag value. For example, I might have a collection of other servers with the tag name
Patch Group with a value of
- Use the following command to add the
Patch Grouptag to your Amazon EC2 instance.
Note: You must wait a few minutes until the Amazon EC2 instance is available before you can proceed to the next section. To make sure your Amazon EC2 instance is online and ready, you can use the following AWS CLI command:
At this point, you now have at least one Amazon EC2 instance you can use to configure Systems Manager.
Step 2: Configure Systems Manager
In this section, I show you how to configure and use Systems Manager to apply operating system patches to your Amazon EC2 instances, and how to manage patch compliance.
To start, I provide some background information about Systems Manager. Then, I cover how to:
- Create the Systems Manager IAM role so that Systems Manager is able to perform patch operations.
- Create a Systems Manager patch baseline and associate it with your instance to define which patches Systems Manager should apply.
- Define a maintenance window to make sure Systems Manager patches your instance when you tell it to.
- Monitor patch compliance to verify the patch state of your instances.
You must meet two prerequisites to use Systems Manager to apply operating system patches. First, you must attach the IAM role you created in the previous section,
EC2SSM, to your Amazon EC2 instance. Second, you must install the Systems Manager agent on your Amazon EC2 instance. If you have used a recent Amazon Linux AMI, Amazon has already installed the Systems Manager agent on your Amazon EC2 instance. You can confirm this by logging in to an Amazon EC2 instance and checking the Systems Manager agent log files that are located at
To install the Systems Manager agent on an instance that does not have the agent preinstalled or if you want to use the Systems Manager agent on your on-premises servers, see Installing and Configuring the Systems Manager Agent on Linux Instances. If you forgot to attach the newly created role when launching your Amazon EC2 instance or if you want to attach the role to already running Amazon EC2 instances, see Attach an AWS IAM Role to an Existing Amazon EC2 Instance by Using the AWS CLI or use the AWS Management Console.
A. Create the Systems Manager IAM role
For a maintenance window to be able to run any tasks, you must create a new role for Systems Manager. This role is a different kind of role than the one you created earlier: this role will be used by Systems Manager instead of Amazon EC2. Earlier, you created the role,
EC2SSM, with the policy,
AmazonEC2RoleforSSM, which allowed the Systems Manager agent on your instance to communicate with Systems Manager. In this section, you need a new role with the policy,
AmazonSSMMaintenanceWindowRole, so that the Systems Manager service can execute commands on your instance.
To create the new IAM role for Systems Manager:
- Create a JSON file named
trustpolicy-maintenancewindowrole.jsonthat contains the following trust policy. This policy describes which principal is allowed to assume the role you are going to create. This trust policy allows not only Amazon EC2 to assume this role, but also Systems Manager.
- Use the following command to create a role named
MaintenanceWindowRolethat has the AWS managed policy,
AmazonSSMMaintenanceWindowRole, attached to it. This command generates JSON-based output that describes the role and its parameters, if the command is successful.
- Use the following command to attach the AWS managed IAM policy (
AmazonEC2RoleforSSM) to your newly created role.
B. Create a Systems Manager patch baseline and associate it with your instance
Next, you will create a Systems Manager patch baseline and associate it with your Amazon EC2 instance. A patch baseline defines which patches Systems Manager should apply to your instance. Before you can associate the patch baseline with your instance, though, you must determine if Systems Manager recognizes your Amazon EC2 instance. Use the following command to list all instances managed by Systems Manager. The
--filters option ensures you look only for your newly created Amazon EC2 instance.
If your instance is missing from the list, verify that:
- Your instance is running.
- You attached the Systems Manager IAM role,
- You deployed a NAT gateway in your public subnet to ensure your VPC reflects the diagram shown earlier in this post so that the Systems Manager agent can connect to the Systems Manager internet endpoint.
- The Systems Manager agent logs don’t include any unaddressed errors.
Now that you have checked that Systems Manager can manage your Amazon EC2 instance, it is time to create a patch baseline. With a patch baseline, you define which patches are approved to be installed on all Amazon EC2 instances associated with the patch baseline. The
Patch Group resource tag you defined earlier will determine to which patch group an instance belongs. If you do not specifically define a patch baseline, the default AWS-managed patch baseline is used.
To create a patch baseline:
- Use the following command to create a patch baseline named
AmazonLinuxServers. With approval rules, you can determine the approved patches that will be included in your patch baseline. In this example, you add all
Criticalseverity patches to the patch baseline as soon as they are released, by setting the
Auto approval delayto
0 days. By setting the
Auto approval delayto
2 days, you add to this patch baseline the
Lowseverity patches two days after they are released.
- Use the following command to register the patch baseline you created with your instance. To do so, you use the
Patch Grouptag that you added to your Amazon EC2 instance.
C. Define a maintenance window
Now that you have successfully set up a role, created a patch baseline, and registered your Amazon EC2 instance with your patch baseline, you will define a maintenance window so that you can control when your Amazon EC2 instances will receive patches. By creating multiple maintenance windows and assigning them to different patch groups, you can make sure your Amazon EC2 instances do not all reboot at the same time.
To define a maintenance window:
- Use the following command to define a maintenance window. In this example command, the maintenance window will start every Saturday at 10:00 P.M. UTC. It will have a duration of 4 hours and will not start any new tasks 1 hour before the end of the maintenance window.
For more information about defining a cron-based schedule for maintenance windows, see Cron and Rate Expressions for Maintenance Windows.
- After defining the maintenance window, you must register the Amazon EC2 instance with the maintenance window so that Systems Manager knows which Amazon EC2 instance it should patch in this maintenance window. You can register the instance by using the same
Patch Grouptag you used to associate the Amazon EC2 instance with the AWS-provided patch baseline, as shown in the following command.
- Assign a task to the maintenance window that will install the operating system patches on your Amazon EC2 instance. The following command includes the following options.
nameis the name of your task and is optional. I named mine
task-arnis the name of the task document you want to run.
max-concurrencyallows you to specify how many of your Amazon EC2 instances Systems Manager should patch at the same time.
max-errorsdetermines when Systems Manager should abort the task. For patching, this number should not be too low, because you do not want your entire patch task to stop on all instances if one instance fails. You can set this, for example, to
service-role-arnis the Amazon Resource Name (ARN) of the
AmazonSSMMaintenanceWindowRolerole you created earlier in this blog post.
task-invocation-parametersdefines the parameters that are specific to the
AWS-RunPatchBaselinetask document and tells Systems Manager that you want to install patches with a timeout of 600 seconds (10 minutes).
Now, you must wait for the maintenance window to run at least once according to the schedule you defined earlier. If your maintenance window has expired, you can check the status of any maintenance tasks Systems Manager has performed by using the following command.
D. Monitor patch compliance
You also can see the overall patch compliance of all Amazon EC2 instances using the following command in the AWS CLI.
This command shows you the number of instances that are compliant with each category and the number of instances that are not in JSON format.
You also can see overall patch compliance by choosing Compliance under Insights in the navigation pane of the Systems Manager console. You will see a visual representation of how many Amazon EC2 instances are up to date, how many Amazon EC2 instances are noncompliant, and how many Amazon EC2 instances are compliant in relation to the earlier defined patch baseline.
In this section, you have set everything up for patch management on your instance. Now you know how to patch your Amazon EC2 instance in a controlled manner and how to check if your Amazon EC2 instance is compliant with the patch baseline you have defined. Of course, I recommend that you apply these steps to all Amazon EC2 instances you manage.
In this blog post, I showed how to use Systems Manager to create a patch baseline and maintenance window to keep your Amazon EC2 Linux instances up to date with the latest security patches. Remember that by creating multiple maintenance windows and assigning them to different patch groups, you can make sure your Amazon EC2 instances do not all reboot at the same time.
If you have comments about this post, submit them in the “Comments” section below. If you have questions about or issues implementing any part of this solution, start a new thread on the Amazon EC2 forum or contact AWS Support.
Security updates have been issued by Arch Linux (clamav), Debian (mailman, mpv, and simplesamlphp), Fedora (tomcat-native), openSUSE (docker, docker-runc, containerd,, kernel, mupdf, and python-mistune), Red Hat (kernel), and Ubuntu (mailman and postgresql-9.3, postgresql-9.5, postgresql-9.6).
Security updates have been issued by Debian (django-anymail, libtasn1-6, and postgresql-9.1), Fedora (w3m), Mageia (389-ds-base, gcc, libtasn1, and p7zip), openSUSE (flatpak, ImageMagick, libjpeg-turbo, libsndfile, mariadb, plasma5-workspace, pound, and spice-vdagent), Oracle (kernel), Red Hat (flash-plugin), SUSE (docker, docker-runc, containerd, golang-github-docker-libnetwork and kernel), and Ubuntu (libvirt, miniupnpc, and QEMU).
Security updates have been issued by Debian (xen), Fedora (clamav, community-mysql, dnsmasq, flatpak, libtasn1, mupdf, p7zip, rsync, squid, thunderbird, tomcat, unbound, and zziplib), Mageia (clamav, curl, dovecot, ffmpeg, gcab, kernel, libtiff, libvpx, php-smarty, pure-ftpd, redis, and thunderbird), openSUSE (apache-commons-email), Red Hat (rh-mariadb100-mariadb), SUSE (firefox), and Ubuntu (clamav, squid3, and systemd).
Post Syndicated from Ilya Epshteyn original https://aws.amazon.com/blogs/big-data/dynamically-create-friendly-urls-for-your-amazon-emr-web-interfaces/
Amazon EMR enables data analysts and scientists to deploy a cluster running popular frameworks such as Spark, HBase, Presto, and Flink of any size in minutes. When you launch a cluster, Amazon EMR automatically configures the underlying Amazon EC2 instances with the frameworks and applications that you choose for your cluster. This can include popular web interfaces such as Hue workbench, Zeppelin notebook, and Ganglia monitoring dashboards and tools.
These web interfaces are hosted on the EMR master node and must be accessed using the public DNS name of the master node (master public DNS value). The master public DNS value is dynamically created, not very user friendly and is hard to remember— it looks something like ip-###-###-###-###.us-west-2.compute.internal. Not having a friendly URL to connect to the popular workbench or notebook interfaces may impact the workflow and hinder your gained agility.
Some customers have addressed this challenge through custom bootstrap actions, steps, or external scripts that periodically check for new clusters and register a friendlier name in DNS. These approaches either put additional burden on the data practitioners or require additional resources to execute the scripts. In addition, there is typically some lag time associated with such scripts. They often don’t do a great job cleaning up the DNS records after the cluster has terminated, potentially resulting in a security risk.
The solution in this post provides an automated, serverless approach to registering a friendly master node name for easy access to the web interfaces.
Inspired in part by our colleague’s post on Building a Dynamic DNS for Route 53 using CloudWatch Events and Lambda, this solution leverages Amazon CloudWatch Events, AWS Lambda, and Amazon Route 53 to dynamically register a CNAME with a friendly name in a Route 53 private hosted zone.
Before I dive deeper, I review these key services and how they are part of this solution.
CloudWatch Events delivers a near real-time stream of system events that describe changes in AWS resources. Using simple rules, you can match events and route them to one or more target functions or streams. An event can be generated in one of four ways:
- From an AWS service when resources change state
- From API calls that are delivered via AWS CloudTrail
- From your own code that can generate application-level events
- Issued on cron-style scheduling
In this solution, I cover the first type of event, which is automatically emitted by EMR when the cluster state changes. Based on the state of this event, either create or update the DNS record in Route 53 when the cluster state changes to STARTING, or delete the DNS record when the cluster is no longer needed and the state changes to TERMINATED. For more information about all EMR event details, see Monitor CloudWatch Events.
Route 53 private hosted zones
A private hosted zone is a container that holds information about how to route traffic for a domain and its subdomains within one or more VPCs. Private hosted zones enable you to use custom DNS names for your internal resources without exposing the names or IP addresses to the internet.
Route 53 supports resource record sets with a wide range of record types. In this solution, you use a CNAME record that is used to specify a domain name as an alias for another domain (the ‘canonical’ domain). You use a friendly name of the cluster as the CNAME for the EMR master public DNS value.
You are using private hosted zones because an EMR cluster is typically deployed within a private subnet and is accessed either from within the VPC or from on-premises resources over VPN or AWS Direct Connect. To resolve domain names in private hosted zones from your on-premises network, configure a DNS forwarder, as described in How can I resolve Route 53 private hosted zones from an on-premises network via an Ubuntu instance?.
Lambda is a compute service that lets you run code without provisioning or managing servers. Lambda executes your code only when needed and scales automatically to thousands of requests per second. Lambda takes care of high availability, and server and OS maintenance and patching. You pay only for the consumed compute time. There is no charge when your code is not running.
Lambda provides the ability to invoke your code in response to events, such as when an object is put to an Amazon S3 bucket or as in this case, when a CloudWatch event is emitted. As part of this solution, you deploy a Lambda function as a target that is invoked by CloudWatch Events when the event matches your rule. You also configure the necessary permissions based on the Lambda permissions model, including a Lambda function policy and Lambda execution role.
Putting it all together
Now that you have all of the pieces, you can put together a complete solution. The following diagram illustrates how the solution works:
- Start with a user activity such as launching or terminating an EMR cluster.
- EMR automatically sends events to the CloudWatch Events stream.
- A CloudWatch Events rule matches the specified event, and routes it to a target, which in this case is a Lambda function. In this case, you are using the EMR Cluster State Change
- The Lambda function performs the following key steps:
- Get the clusterId value from the event detail and use it to call EMR. DescribeCluster API to retrieve the following data points:
- MasterPublicDnsName – public DNS name of the master node
- Locate the tag containing the friendly name to use as the CNAME for the cluster. The key name containing the friendly name should be The value should be specified as host.domain.com, where domain is the private hosted zone in which to update the DNS record.
- Update DNS based on the state in the event detail.
- If the state is STARTING, the function calls the Route 53 API to create or update a resource record set in the private hosted zone specified by the domain tag. This is a CNAME record mapped to MasterPublicDnsName.
- Conversely, if the state is TERMINATED, the function calls the Route 53 API to delete the associated resource record set from the private hosted zone.
- Get the clusterId value from the event detail and use it to call EMR. DescribeCluster API to retrieve the following data points:
Deploying the solution
Because all of the components of this solution are serverless, use the AWS Serverless Application Model (AWS SAM) template to deploy the solution. AWS SAM is natively supported by AWS CloudFormation and provides a simplified syntax for expressing serverless resources, resulting in fewer lines of code.
Overview of the SAM template
For this solution, the SAM template has 76 lines of text as compared to 142 lines without SAM resources (and writing the template in YAML would be even slightly smaller). The solution can be deployed using the AWS Management Console, AWS Command Line Interface (AWS CLI), or AWS SAM Local.
CloudFormation transforms help simplify template authoring by condensing a multiple-line resource declaration into a single line in your template. To inform CloudFormation that your template defines a serverless application, add a line under the template format version as follows:
Before SAM, you would use the AWS::Lambda::Function resource type to define your Lambda function. You would then need a resource to define the permissions for the function (AWS::Lambda::Permission), another resource to define a Lambda execution role (AWS::IAM::Role), and finally a CloudWatch Events resource (Events::Rule) that triggers this function.
With SAM, you need to define just a single resource for your function, AWS::Serverless::Function. Using this single resource type, you can define everything that you need, including function properties such as function handler, runtime, and code URI, as well as the required IAM policies and the CloudWatch event.
A few additional things to note in the code example:
- CodeUri – Before you can deploy a SAM template, first upload your Lambda function code zip to S3. You can do this manually or use the aws cloudformation package CLI command to automate the task of uploading local artifacts to a S3 bucket, as shown later.
- Lambda execution role and permissions – You are not specifying a Lambda execution role in the template. Rather, you are providing the required permissions as IAM policy documents. When the template is submitted, CloudFormation expands the AWS::Serverless::Function resource, declaring a Lambda function and an execution role. The created role has two attached policies: a default AWSLambdaBasicExecutionRole and the inline policy specified in the template.
- CloudWatch Events rule – Instead of specifying a CloudWatch Events resource type, you are defining an event source object as a property of the function itself. When the template is submitted, CloudFormation expands this into a CloudWatch Events rule resource and automatically creates the Lambda resource-based permissions to allow the CloudWatch Events rule to trigger the function.
Deploying the solution using the console
1.) Log in to the CloudFormation console and choose Create stack.
2.) For Choose a template, select Specify an Amazon S3 template URL and enter the following URL:
NOTE: If you are trying this solution outside of us-east-1, then you should download the necessary files, upload them to the buckets in your region, edit the script as appropriate and then run it or use the CLI deployment method below.
3.) Choose Next.
4.) On the Specify Details page, keep or modify the stack name and choose Next.
5.) On the Options page, choose Next.
6.) On the Review page, take the following steps:
- Acknowledge the two Transform access capabilities. This allows the CloudFormation transform to create the required IAM resources with custom names.
- Under Transforms, choose Create Change Set.
Wait a few seconds for the change set to be created before proceeding. The change set should look as follows:
7.) Choose Execute to deploy the template.
After the template is deployed, you should see four resources created:
Deploying the solution using the AWS CLI
- Download the Lambda function code emr-dns-setter.py and the SAM template emr-dns-setter-sam.json to your local machine.
- Modify the CodeUri property in emr-dns-setter-sam.json template to the local path of the function code:
- Use the aws cloudformation package CLI command to upload the package.
After the package is successfully uploaded, the output should look as follows:
The CodeUri property in serverless-output.template is now referencing the packaged artifacts in the S3 bucket that you specified:
- Use the aws cloudformation deploy CLI command to deploy the stack:
You should see the following output after the stack has been successfully created:
To test the solution, launch an EMR cluster. The Lambda function looks for the cluster_name tag associated with the EMR cluster. Make sure to specify the friendly name of your cluster as host.domain.com where the domain is the private hosted zone in which to create the CNAME record.
Here is a sample CLI command to launch a cluster within a specific subnet in a VPC with the required tag cluster_name.
After the cluster is launched, log in to the Route 53 console. In the left navigation pane, choose Hosted Zones to view the list of private and public zones currently configured in Route 53. Select the hosted zone that you specified in the ZONE tag when you launched the cluster. Verify that the resource records were created.
You can also monitor the CloudWatch Events metrics that are published to CloudWatch every minute, such as the number of TriggeredRules and Invocations.
Now that you’ve verified that the Lambda function successfully updated the Route 53 resource records in the zone file, terminate the EMR cluster and verify that the records are removed by the same function.
This solution provides a serverless approach to automatically assigning a friendly name for your EMR cluster for easy access to popular notebooks and other web interfaces. CloudWatch Events also supports cross-account event delivery, so if you are running EMR clusters in multiple AWS accounts, all cluster state events across accounts can be consolidated into a single account.
I hope that this solution provides a small glimpse into the power of CloudWatch Events and Lambda and how they can be leveraged with EMR and other AWS big data services. For example, by using the EMR step state change event, you can chain various pieces of your analytics pipeline. You may have a transient cluster perform data ingest and, when the task successfully completes, spin up an ETL cluster for transformation and upload to Amazon Redshift. The possibilities are truly endless.
Security updates have been issued by CentOS (systemd and thunderbird), Debian (squid and squid3), Fedora (firefox), Mageia (java-1.8.0-openjdk and sox), openSUSE (ecryptfs-utils and libXfont), Oracle (systemd and thunderbird), Scientific Linux (thunderbird), and Ubuntu (dovecot and w3m).
Security updates have been issued by Debian (chromium-browser, krb5, and smarty3), Fedora (firefox, GraphicsMagick, and moodle), Mageia (rsync), openSUSE (bind, chromium, freeimage, gd, GraphicsMagick, libtasn1, libvirt, nodejs6, php7, systemd, and webkit2gtk3), Red Hat (chromium-browser, systemd, and thunderbird), Scientific Linux (systemd), and Ubuntu (curl, firefox, and ruby2.3).
Security updates have been issued by Arch Linux (dnsmasq, libmupdf, mupdf, mupdf-gl, mupdf-tools, and zathura-pdf-mupdf), CentOS (kernel), Debian (smarty3, thunderbird, and unbound), Fedora (bind, bind-dyndb-ldap, coreutils, curl, dnsmasq, dnsperf, gcab, java-1.8.0-openjdk, libxml2, mongodb, poco, rubygem-rack-protection, transmission, unbound, and wireshark), Red Hat (collectd, erlang, and openstack-nova), SUSE (bind), and Ubuntu (clamav and webkit2gtk).
Security updates have been issued by Arch Linux (curl, lib32-curl, lib32-libcurl-compat, lib32-libcurl-gnutls, libcurl-compat, libcurl-gnutls, and rsync), Debian (curl), Fedora (clamav and java-1.8.0-openjdk), openSUSE (apache2), Oracle (kernel), and Ubuntu (linux-kvm and thunderbird).
Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/the-floodgates-are-open-increased-network-bandwidth-for-ec2-instances/
I hope that you have configured your AMIs and your current-generation EC2 instances to use the Elastic Network Adapter (ENA) that I told you about back in mid-2016. The ENA gives you high throughput and low latency, while minimizing the load on the host processor. It is designed to work well in the presence of multiple vCPUs, with intelligent packet routing backed up by multiple transmit and receive queues.
Today we are opening up the floodgates and giving you access to more bandwidth in all AWS Regions. Here are the specifics (in each case, the actual bandwidth is dependent on the instance type and size):
EC2 to S3 – Traffic to and from Amazon Simple Storage Service (S3) can now take advantage of up to 25 Gbps of bandwidth. Previously, traffic of this type had access to 5 Gbps of bandwidth. This will be of benefit to applications that access large amounts of data in S3 or that make use of S3 for backup and restore.
EC2 to EC2 – Traffic to and from EC2 instances in the same or different Availability Zones within a region can now take advantage of up to 5 Gbps of bandwidth for single-flow traffic, or 25 Gbps of bandwidth for multi-flow traffic (a flow represents a single, point-to-point network connection) by using private IPv4 or IPv6 addresses, as described here.
EC2 to EC2 (Cluster Placement Group) – Traffic to and from EC2 instances within a cluster placement group can continue to take advantage of up to 10 Gbps of lower-latency bandwidth for single-flow traffic, or 25 Gbps of lower-latency bandwidth for multi-flow traffic.
To take advantage of this additional bandwidth, make sure that you are using the latest, ENA-enabled AMIs on current-generation EC2 instances. ENA-enabled AMIs are available for Amazon Linux, Ubuntu 14.04 & 16.04, RHEL 7.4, SLES 12, and Windows Server (2008 R2, 2012, 2012 R2, and 2016). The FreeBSD AMI in AWS Marketplace is also ENA-enabled, as is VMware Cloud on AWS.
Security updates have been issued by CentOS (389-ds-base, dhcp, kernel, and nautilus), Debian (curl, openssh, and wireshark), Fedora (clamav, firefox, java-9-openjdk, and poco), Gentoo (clamav), openSUSE (curl, libevent, mupdf, mysql-community-server, newsbeuter, php5, redis, and tre), Oracle (389-ds-base, dhcp, kernel, and nautilus), Slackware (mozilla), and Ubuntu (kernel and linux-hwe, linux-azure, linux-gcp, linux-oem).
Security updates have been issued by CentOS (firefox), Debian (firefox-esr, gcab, and poppler), Fedora (clamav and firefox), Mageia (bind, firefox, glibc, graphicsmagick, squid, systemd, and virtualbox), openSUSE (firefox, GraphicsMagick, libexif, and libvpx), Red Hat (389-ds-base, dhcp, kernel, kernel-alt, kernel-rt, and nautilus), Scientific Linux (389-ds-base, dhcp, kernel, and nautilus), Slackware (curl), SUSE (kernel and webkit2gtk3), and Ubuntu (firefox, libtasn1-6, and mysql-5.5).
Post Syndicated from Karthik Thirugnanasambandam original https://aws.amazon.com/blogs/devops/codebuild-support-for-github-enterprise/
Thank you to my colleague Harvey Bendana for this blog on how to do shallow cloning on AWS CodeBuild using GitHub Enterprise as a source.
Today we are announcing support for using GitHub Enterprise as a source type for CodeBuild. You can now initiate build tasks from changes in source code hosted on your own implementation of GitHub Enterprise.
We are also announcing support for shallow cloning of a repo when you use CodeCommit, BitBucket, GitHub, or GitHub Enterprise as a source type. Shallow cloning allows you to truncate history of a repo in order to save space and speed up cloning times.
In this post, I’ll walk you through how to configure GitHub Enterprise as a source type with a defined clone depth for an AWS CodeBuild project. I’ll also show you all the moving parts associated with a successful implementation.
AWS CodeBuild is a fully managed build service. There are no servers to provision and scale, or software to install, configure, and operate. You just specify the location of your source code, choose your build settings, and CodeBuild runs build scripts for compiling, testing, and packaging your code.
GitHub Enterprise is the on-premises version of GitHub.com. It makes collaborative coding possible and enjoyable for large-scale enterprise software development teams.
Many enterprises choose GitHub Enterprise as their preferred source code/version control repository because it can be hosted in their own trusted network, whether that is an on-premises data center or their own Amazon VPCs.
- You’ll need an AWS account.
- You’ll need a GitHub Enterprise implementation with a repo. If you’d like to deploy one inside your own Amazon VPC, check out our Quick Start Guide.
- You’ll need an S3 bucket to store your GitHub Enterprise self-signed SSL certificate.
Download your GitHub Enterprise SSL certificate:
Note: The following steps are required only for self-signed certificates. You can forego installation of a certificate if you are using self-signed certificates and default to HTTP communication with your repo. For this post, I am using a self-signed certificate and the Firefox browser. These steps may vary, depending on your browser of choice.
- Navigate to your GitHub Enterprise environment and sign in with your credentials.
2. Choose the lock icon in the upper-left corner to view and export your GitHub Enterprise certificate.
3. When you export and download your certificate, make sure you select the format type, which includes the entire certificate chain.
4. Open the Amazon S3 console at https://console.aws.amazon.com/s3 and upload your certificate to an S3 bucket in your AWS account.
Now I will show you how to create an AWS CodeBuild project.
Create a sample AWS CodeBuild project:
- Open the AWS CodeBuild console at https://console.aws.amazon.com/codebuild.
2. If there are no existing build projects, a welcome page is displayed. Choose Get started. If you already have build projects, then choose Create project.
3. On the Configure your project page, type a name for your build project. Build project names must be unique across each AWS account.
4. For Source provider, choose GitHub Enterprise.
5. Generate a personal access token in your GitHub Enterprise environment. Under Select scopes, select repo. For information, see Creating a personal access token for the command line on the GitHub Help website.
6. Enter your personal access token in your CodeBuild project and choose Save Token.
7. Enter the repository URL and choose a Git clone depth value that makes sense for you. Allowed values are 1, 5, 25, or Full. For this post, I am using a depth of 1.
8. Select the Webhook check box.
9. Continue with the rest of the configuration for your project, choosing options that best suit your build needs. For this post, I am using an AWS CodeBuild managed image running the Ubuntu OS with the base runtime configuration. Enter your build specifications or build commands. I am using a simple build command of git log . so that it can be easily found in the CloudWatch logs of the CodeBuild project. It will also be used to demonstrate the shallow clone feature.
10. Next, select Install certificate from your S3 to install your GitHub Enterprise self-signed certificate from S3. For Bucket of certificate, I’ve entered the S3 bucket where I uploaded the certificate. For Object key of certificate, I’ve entered the name of the certificate.
11. Lastly, configure artifacts, caching, IAM roles, and VPC configurations. For this post, I chose not to generate any artifacts from this build. From the following screenshot, you’ll see I’ve opted out of cache, requested a new IAM role with the required permissions, and have not defined VPC access. Choose Continue to validate and complete the creation of the CodeBuild project.
Note: If your GitHub Enterprise environment is in an Amazon VPC, configure VPC access for your project. Define the VPC ID, subnet ID, and security group so that your project has access to the EC2 instances hosting your GitHub Enterprise environment.
12. After the project is created, a dialog box displays a CodeBuild payload URL and secret. They are used to create a webhook for the repo in the GitHub Enterprise environment.
Create a webhook in your GitHub Enterprise repo:
1. In your GitHub Enterprise repo, navigate to Settings, choose Hooks & services, and then choose Add webhook.
2. Paste the payload URL and secret into their respective fields. Under Which events would you like to trigger this webhook? choose an option. For this post, I am using Let me select individual events. I then chose Pull request and Push as the two event triggers.
3. Make sure Active is selected and then choose Add webhook.
4. A webhook has now been created in the GitHub Enterprise repo.
Now I will show you how to test this.
Trigger your AWS CodeBuild project by pushing a change to the GitHub Enterprise repo
1. Clone the repo to the local file system. For information, see Clone the Repository Using the Command Line on the GitHub Help website. Now create a feature branch, push a change, and generate a pull request for review, and, ultimately, merge to master. Here is the state of the GitHub Enterprise repo and AWS CodeBuild project before pushing a change:
2. Use Git command line tools to create a new branch in the repo.
3. Update the README.md for the repo with a link to the AWS CodeBuild documentation.
4. After the changes have been saved, push them to the feature branch.
5. There is now notification of a new branch in the GitHub Enterprise environment.
6. Generate a pull request from the feature branch in preparation of review and merge to master.
7. The reviewer(s) will then review and merge the pull request, pushing all changes to the master branch.
8. Here is the updated repo:
9. After the change has been pushed successfully, a new build is initiated.
10. In the following screenshot, you’ll see the initiator is Github-Hookshot/eb0c46 and the source version is 03169095b8f16ac077388471035becb2070aa12c.
11. In the Recent Deliveries section, under the configuration of the GitHub Enterprise repo webhook, the CodeBuild project initiator is defined as User-Agent. The source version is denoted in the Payload output. They match!
12. A successfully completed build should appear under the CodeBuild project.
13. The entire log output of the CodeBuild project can be viewed in CloudWatch logs. In the following screenshot, the source was downloaded successfully from the GitHub Enterprise repo and the build command of git log . was run successfully. Only the most recent commit appears in the git history output. This is because I defined a clone depth of 1.
14. If I query the git history of the repo on the local repository, the output has the full commit history. This is expected because I am doing a full clone locally.
In this blog post, I showed you how to configure GitHub Enterprise as a source type for your AWS CodeBuild project with a clone depth of 1. These new features expand the capabilities of AWS CodeBuild and the suite of AWS Developer Tools for CI/CD and DevOps processes.
I hope you found this post useful. Feel free to leave your feedback or suggestions in the comments.
Security updates have been issued by Debian (smarty3), Fedora (bind, bind-dyndb-ldap, dnsperf, glibc, kernel, libtasn1, libvpx, mariadb, python-bottle, ruby, and sox), Red Hat (rh-eclipse46-jackson-databind), SUSE (kernel), and Ubuntu (kernel, linux, linux-aws, linux-euclid, linux-hwe, linux-azure, linux-gcp, linux-oem, linux-lts-trusty, linux-lts-xenial, linux-aws, and rsync).