Tag Archives: election

Securing Elections

2018-04-20 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2018/04/securing_electi_1.html

Elections serve two purposes. The first, and obvious, purpose is to accurately choose the winner. But the second is equally important: to convince the loser. To the extent that an election system is not transparently and auditably accurate, it fails in that second purpose. Our election systems are failing, and we need to fix them.

Today, we conduct our elections on computers. Our registration lists are in computer databases. We vote on computerized voting machines. And our tabulation and reporting is done on computers. We do this for a lot of good reasons, but a side effect is that elections now have all the insecurities inherent in computers. The only way to reliably protect elections from both malice and accident is to use something that is not hackable or unreliable at scale; the best way to do that is to back up as much of the system as possible with paper.

Recently, there have been two graphic demonstrations of how bad our computerized voting system is. In 2007, the states of California and Ohio conducted audits of their electronic voting machines. Expert review teams found exploitable vulnerabilities in almost every component they examined. The researchers were able to undetectably alter vote tallies, erase audit logs, and load malware on to the systems. Some of their attacks could be implemented by a single individual with no greater access than a normal poll worker; others could be done remotely.

Last year, the Defcon hackers’ conference sponsored a Voting Village. Organizers collected 25 pieces of voting equipment, including voting machines and electronic poll books. By the end of the weekend, conference attendees had found ways to compromise every piece of test equipment: to load malicious software, compromise vote tallies and audit logs, or cause equipment to fail.

It’s important to understand that these were not well-funded nation-state attackers. These were not even academics who had been studying the problem for weeks. These were bored hackers, with no experience with voting machines, playing around between parties one weekend.

It shouldn’t be any surprise that voting equipment, including voting machines, voter registration databases, and vote tabulation systems, are that hackable. They’re computers — often ancient computers running operating systems no longer supported by the manufacturers — and they don’t have any magical security technology that the rest of the industry isn’t privy to. If anything, they’re less secure than the computers we generally use, because their manufacturers hide any flaws behind the proprietary nature of their equipment.

We’re not just worried about altering the vote. Sometimes causing widespread failures, or even just sowing mistrust in the system, is enough. And an election whose results are not trusted or believed is a failed election.

Voting systems have another requirement that makes security even harder to achieve: the requirement for a secret ballot. Because we have to securely separate the election-roll system that determines who can vote from the system that collects and tabulates the votes, we can’t use the security systems available to banking and other high-value applications.

We can securely bank online, but can’t securely vote online. If we could do away with anonymity — if everyone could check that their vote was counted correctly — then it would be easy to secure the vote. But that would lead to other problems. Before the US had the secret ballot, voter coercion and vote-buying were widespread.

We can’t, so we need to accept that our voting systems are insecure. We need an election system that is resilient to the threats. And for many parts of the system, that means paper.

Let’s start with the voter rolls. We know they’ve already been targeted. In 2016, someone changed the party affiliation of hundreds of voters before the Republican primary. That’s just one possibility. A well-executed attack that deletes, for example, one in five voters at random — or changes their addresses — would cause chaos on election day.

Yes, we need to shore up the security of these systems. We need better computer, network, and database security for the various state voter organizations. We also need to better secure the voter registration websites, with better design and better internet security. We need better security for the companies that build and sell all this equipment.

Multiple, unchangeable backups are essential. A record of every addition, deletion, and change needs to be stored on a separate system, on write-only media like a DVD. Copies of that DVD, or — even better — a paper printout of the voter rolls, should be available at every polling place on election day. We need to be ready for anything.

Next, the voting machines themselves. Security researchers agree that the gold standard is a voter-verified paper ballot. The easiest (and cheapest) way to achieve this is through optical-scan voting. Voters mark paper ballots by hand; they are fed into a machine and counted automatically. That paper ballot is saved, and serves as a final true record in a recount in case of problems. Touch-screen machines that print a paper ballot to drop in a ballot box can also work for voters with disabilities, as long as the ballot can be easily read and verified by the voter.

Finally, the tabulation and reporting systems. Here again we need more security in the process, but we must always use those paper ballots as checks on the computers. A manual, post-election, risk-limiting audit varies the number of ballots examined according to the margin of victory. Conducting this audit after every election, before the results are certified, gives us confidence that the election outcome is correct, even if the voting machines and tabulation computers have been tampered with. Additionally, we need better coordination and communications when incidents occur.

It’s vital to agree on these procedures and policies before an election. Before the fact, when anyone can win and no one knows whose votes might be changed, it’s easy to agree on strong security. But after the vote, someone is the presumptive winner — and then everything changes. Half of the country wants the result to stand, and half wants it reversed. At that point, it’s too late to agree on anything.

The politicians running in the election shouldn’t have to argue their challenges in court. Getting elections right is in the interest of all citizens. Many countries have independent election commissions that are charged with conducting elections and ensuring their security. We don’t do that in the US.

Instead, we have representatives from each of our two parties in the room, keeping an eye on each other. That provided acceptable security against 20th-century threats, but is totally inadequate to secure our elections in the 21st century. And the belief that the diversity of voting systems in the US provides a measure of security is a dangerous myth, because few districts can be decisive and there are so few voting-machine vendors.

We can do better. In 2017, the Department of Homeland Security declared elections to be critical infrastructure, allowing the department to focus on securing them. On 23 March, Congress allocated $380m to states to upgrade election security.

These are good starts, but don’t go nearly far enough. The constitution delegates elections to the states but allows Congress to “make or alter such Regulations”. In 1845, Congress set a nationwide election day. Today, we need it to set uniform and strict election standards.

This essay originally appeared in the Guardian.

My letter urging Georgia governor to veto anti-hacking bill

2018-04-16 Robert Graham

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/04/my-letter-urging-georgia-governor-to.html

February 16, 2018

Office of the Governor
206 Washington Street
111 State Capitol
Atlanta, Georgia 30334

Re: SB 315

Dear Governor Deal:

I am writing to urge you to veto SB315, the “Unauthorized Computer Access” bill.

The cybersecurity community, of which Georgia is a leader, is nearly unanimous that SB315 will make cybersecurity worse. You’ve undoubtedly heard from many of us opposing this bill. It does not help in prosecuting foreign hackers who target Georgian computers, such as our elections systems. Instead, it prevents those who notice security flaws from pointing them out, thereby getting them fixed. This law violates the well-known Kirchhoff’s Principle, that instead of secrecy and obscurity, that security is achieved through transparency and openness.

That the bill contains this flaw is no accident. The justification for this bill comes from an incident where a security researcher noticed a Georgia state election system had made voter information public. This remained unfixed, months after the vulnerability was first disclosed, leaving the data exposed. Those in charge decided that it was better to prosecute those responsible for discovering the flaw rather than punish those who failed to secure Georgia voter information, hence this law.

Too many security experts oppose this bill for it to go forward. Signing this bill, one that is weak on cybersecurity by favoring political cover-up over the consensus of the cybersecurity community, will be part of your legacy. I urge you instead to veto this bill, commanding the legislature to write a better one, this time consulting experts, which due to Georgia’s thriving cybersecurity community, we do not lack.

Thank you for your attention.

Sincerely,
Robert Graham
(formerly) Chief Scientist, Internet Security Systems

Building SaaS Services for AWS Customers with PrivateLink

2018-04-04 Morris Singer

Post Syndicated from Morris Singer original https://aws.amazon.com/blogs/architecture/building-saas-services-for-aws-customers-with-privatelink/

With the advent of AWS PrivateLink, you can provide services to AWS customers directly in their Virtual Private Networks by offering cross-account SaaS solutions on private IP addresses rather than over the Internet.

Traffic that flows to the services you provide does so over private AWS networking rather than over the Internet, offering security and performance enhancements, as well as convenience. PrivateLink can tie in with the AWS Marketplace, facilitating billing and providing a straightforward consumption model to your customers.

The use cases are myriad, but, for this blog post, we’ll demonstrate a fictional order-processing resource. The resource accepts JSON data over a RESTful API, simulating an interface. This could easily be an existing application being considered for a PrivateLink-based consumption model. Consumers of this resource send JSON payloads representing new orders and the system responds with order IDs corresponding to newly-created orders in the system. In a real-world scenario, additional APIs, such as authentication, might also represent critical aspects of the system. This example will not demonstrate these additional APIs because they could be consumed over PrivateLink in a similar fashion to the API constructed in the example.

I’ll demonstrate how to expose the resource on a private IP address in a customer’s VPC. I’ll also explain an architecture leveraging PrivateLink and provide detailed instructions for how to set up such a service. Finally, I’ll provide an example of how a customer might consume such a service. I’ll focus not only on how to architect the solution, but also the considerations that drive architectural choices.

Solution Overview

N.B.: Only two subnets and Availability Zones are shown per VPC for simplicity. Resources must cover all Availability Zones per Region, so that the application is available to all consumers in the region. The instructions in this post, which pertain to resources sitting in us-east-1 will detail the deployment of subnets in all six Availability Zones for this region.

This solution exposes an application’s HTTP-based API over PrivateLink in a provider’s AWS account. The application is a stateless web server running on Amazon Elastic Compute Cloud (EC2) instances. The provider places instances within a virtual private network (VPC) consisting of one private subnet per Availability Zone (AZ). Each AZ contains a subnet. Instances populate each subnet inside of Auto Scaling Groups (ASGs), maintaining a desired count per subnet. There is one ASG per subnet to ensure that the service is available in each AZ. An internal Network Load Balancer (NLB) sits in front of the entire fleet of application instances and an endpoint service is connected with the NLB.

In the customer’s AWS account, they create an endpoint that consumes the endpoint service from the provider’s account. The endpoint exposes an Elastic Network Interface (ENI) in each subnet the customer desires. Each ENI is assigned an IP address within the CIDR block associated with the subnet, for any number of subnets in any number of AZs within the region, for each customer.

PrivateLink facilitates cross-account access to services so the customer can use the provider’s service, feeding it data that exist within the customer’s account while using application logic and systems that run in the provider’s account. The routing between accounts is over private networking rather than over the Internet.

Though this example shows a simple, stateless service running on EC2 and sitting behind an NLB, many kinds of AWS services can be exposed through PrivateLink and can serve as pathways into a provider’s application, such as Amazon Kinesis Streams, Amazon EC2 Container Service, Amazon EC2 Systems Manager, and more.

Using PrivateLink to Establish a Service for Consumption

Building a service to be consumed through PrivateLink involves a few steps:

Build a VPC covering all AZs in region with private subnets
Create a NLB, listener, and target group for instances
Create a launch configuration and ASGs to manage the deployment of Amazon
EC2 instances in each subnet
Launch an endpoint service and connect it with the NLB
Tie endpoint-request approval with billing systems or the AWS Marketplace
Provide the endpoint service in multiple regions

Step 1: Build a VPC and private subnets

Start by determining the network you will need to serve the application. Keep in mind, that you will need to serve the application out of each AZ within any region you choose. Customers will expect to consume your service in multiple AZs because AWS recommends they architect their own applications to span across AZs for fault-tolerance purposes.

Additionally, anything less than full coverage across all AZs in a single region will not facilitate straightforward consumption of your service because AWS does not guarantee that a single AZ will carry the same name across accounts. In fact, AWS randomizes AZ names across accounts to ensure even distribution of independent workloads. Telling customers, for example, that you provide a service in us-east-1a may not give them sufficient information to connect with your service.

The solution is to serve your application in all AZs within a region because this guarantees that no matter what AZs a customer chooses for endpoint creation, that customer is guaranteed to find a running instance of your application with which to connect.

You can lay the foundations for doing this by creating a subnet in each AZ within the region of your choice. The subnets can be private because the service, exposed via PrivateLink, will not provide any publicly routable APIs.

This example uses the us-east-1 region. If you use a different region, the number of AZs may vary, which will change the number of subnets required, and thus the size of the IP address range for your VPC may require adjustments.

VPC and Subnet Creation

Start by creating a new VPC:

aws ec2 create-vpc \
--cidr-block "10.3.0.0/25 \
--no-amazon-provided-ipv6-cidr-block

The example above creates a VPC with 128 IP addresses starting at 10.3.0.0. Each subnet will contain 16 IP addresses, using a total of 96 addresses in the space.
Allocating a sufficient block of addresses requires some planning (though you can make adjustments later if needed). I’d suggest an equally-sized address space in each subnet because the provided service should embody the same performance, availability, and functionality regardless of which AZ your customers choose. Each subnet will need a sufficient address space to accommodate the number of instances you run within it. Additionally, you will need enough space to allow for one IP address per subnet to assign to that subnet’s NLB node’s Elastic Network Interface (ENI).

In this simple example, 16 IP addresses per subnet are enough because we will configure ASGs to maintain two instances each and the NLB requires one ENI. Each subnet reserves five IP addresses for internal purposes, for a total of eight IP addresses needed in each subnet to support the service.

Next, create the private subnets for each Availability Zone. The following demonstrates the creation of the first subnet, which sits in the us-east-1a AZ:

aws ec2 create-subnet \
--availability-zone "us-east-1a" \
--cidr-block "10.3.0.0/28" \
--vpc-id "vpc-2cadab54"

Repeat this step for each remaining AZ. If using the us-east-1 region, you will need to create private subnets in all AZs as follows:

For the purpose of this example, the subnets can leverage the default route table, as it contains a single rule for routing requests to private IP addresses in the VPC, as follows:

In a real-world case, additional routing may be required. For example, you may need additional routes to support VPC peering to access dependencies in other VPCs, connectivity to on-premises resources over DirectConnect or VPN, Internet-accessible dependencies via NAT, or other scenarios.

Security Group Creation

Instances will need to be placed in a security group that allows traffic from the NLB nodes that sit in each subnet.

All instances running the service should be in a security group accepting TCP traffic on the traffic port from any other IP address in the VPC. This will allow the NLB to forward traffic to those instances because the NLB nodes sit in the VPC and are assigned IP addresses in the subnets. In this example, the order processing server running on each instance exposes a service on port 3000, so the security group rule covers this port.

Create a security group for instances:

aws ec2 create-security-group \
--group-name "service-sg" \
--description "Security group for service instances" \
--vpc-id "vpc-2cadab54"

Step 2: Create a Network Load Balancer, Listener, and Target Group

The service integrates with PrivateLink using an internal NLB which sits in front of instances that run the service.

Create the internal NLB:

aws elbv2 create-load-balancer \
--name order-processor-nlb \
--subnets "subnet-6d86e242" "subnet-a07e53eb" "subnet-fc871fa1" "subnet-e29ac586" "subnet-8a90fab5" "subnet-448f7e4b" \
--scheme "internal" \
--type "network"

The NLB must have a target group within which instances will be placed. Create the target group:

aws elbv2 create-target-group \
--name "order-processor-tg" \
--protocol "TCP" \
--port "3000" \
--vpc-id "vpc-2cadab54"

Step 3: Create a Launch Configuration and Auto Scaling Groups

Each private subnet in the VPC will require its own ASG in order to ensure that there is always a minimum number of instances in each subnet.

A single ASG spanning all subnets will not guarantee that every subnet contains the appropriate number of instances. For example, while a single ASG could be configured to work across six subnets and maintain twelve instances, there is no guarantee that each of the six subnets will contain two instances. To guarantee the appropriate number of instances on a per-subnet basis, each subnet must be configured with its own ASG.

New instances should be automatically created within each ASG based on a single launch configuration. The launch configuration should be set up to use an existing Amazon Machine Image (AMI).

This post presupposes you have an AMI that can be used to create new instances that serve the application. There are only a few basic assumptions to how this image is configured:

1. The image containes a web server that serves traffic (in this case, on port 3000)
2. The image is configured to automatically launch the web server as a daemon when the instance starts.

For more information on how to create a new AMI, see Creating an Amazon EBS-Backed Linux AMI.

Create a launch configuration for the ASGs, providing the AMI ID, the ID of the security group created in previous steps (above), a key for access, and an instance type:

aws autoscaling create-launch-configuration \
--launch-configuration-name "service-launch-configuration" \
--image-id "ami-08cefe72" \
--key-name "my-key" \
--security-groups "sg-88c693fc" \
--instance-type "t2.micro"

Next, create an ASG in each private subnet. The following demonstrates creation of an ASG in the first subnet:

aws autoscaling create-auto-scaling-group \
--auto-scaling-group-name "order-processor-asg-a" \
--launch-configuration-name "service-launch-configuration" \
--min-size "2" \
--max-size "2" \
--desired-capacity "2" \
--target-group-arns "arn:aws:elasticloadbalancing:us-east-1:196431283258:targetgroup/order-processor-tg/93806d6c9d88a6b6" \
--vpc-zone-identifier "subnet-6d86e242" \

Repeat this process to create an ASG in each remaining subnet, using the same launch configuration and target group.

In this example, only two instances are created in each subnet. In a real-world scenario, additional instances would likely be recommended for both availability and scale. The ASGs use the provided launch configuration as a template for creating new instances.

When creating the ASGs, the ARN of the target group for the NLB is provided. This way, the ASGs automatically register newly-created instances with the target group so that the NLB can begin sending traffic to them.

Step 4: Launch an endpoint service and connect with NLB

Now, expose the service via PrivateLink with an endpoint service, providing the ARN of the NLB:

aws ec2 create-vpc-endpoint-service-configuration \
--acceptance-required \
--network-load-balancer-arns "arn:aws:elasticloadbalancing:us-east-1:196431283258:loadbalancer/net/order-processor-elb/828ab7a07dccb02a"

This endpoint service is configured to require acceptance. This means that new consumers who attempt to add endpoints that consume it will have to wait for the provider to allow access. This provides an opportunity to control access and integrate with billing systems that monetize the provided service.

For more information on this concept, see: Accepting and Rejecting Interface Endpoint Connection Requests

Step 5: Tie endpoint request approval with billing system or the AWS Marketplace

If you’re maintaining your service as a private service, any account that is intended to have access must be whitelisted before it can find the endpoint service and create an endpoint to consume it.

For more information on listing a PrivateLink service in the AWS Marketplace, see How to List Your Product in AWS Marketplace (https://aws.amazon.com/blogs/apn/how-to-list-your-product-in-aws-marketplace/).

Most production-ready services offered through PrivateLink will require acceptance of Endpoint requests before customers can consume them. Typically, some level of automation around processing approvals is helpful. PrivateLink can publish on a Simple Notification Service (SNS) topic when customers request approval.

Setting this up requires two steps:

1. Create a new SNS topic
2. Create an endpoint connection notification that publishes to the SNS topic.

Each is discussed below.

Create an SNS Topic

First, create a new SNS Topic that can receive messages relating to endpoint service access requests:

aws sns create-topic 
--name "service-notification-topic"

This creates a new topic with the name “service-notification-topic”. Endpoint request approval systems can subscribe to this Topic so that acceptance can be automated.

For more information on SNS, see: Amazon Simple Notification Service Documentation.

Create an Endpoint Connection Notification

Next, create a new Endpoint Connection Notification, so that messages will be published to the topic when new Endpoints connect and need to have access requests approved:

aws ec2 create-vpc-endpoint-connection-notification \
--service-id "vpce-svc-02d3cec2f5605e95b" \
--connection-notification-arn "arn:aws:sns:us-east-1:196431283258:service-notification-topic" \
--connection-events "Connect"

A billing system may ultimately tie in with request approval. This can also be done manually, which may be less useful, but is illustrative. As an example, assume that a customer account has already requested an endpoint to consume the service. The customer can be accepted manually, as follows:

aws ec2 accept-vpc-endpoint-connections \
--service-id "vpce-svc-02d3cec2f5605e95b" \
--vpc-endpoint-ids "vpce-075c50b2becc4c030"

At this point, the consumer can begin consuming the service.

Step 6: Take the Service Across Regions

In distributing SaaS via PrivateLink, providers may have to have to think about how to make their services available in different regions because Endpoint Services are only available within the region where they are created. Customers who attempt to consume Endpoint Services will not be able to create Endpoints across regions.

Rather than saddling consumers with the responsibility of making the jump across regions, we recommend providers work to make services available where their customers consume. They are in a better position to adapt their architectures to multiple regions than customers who do not know the internals of how providers have designed their services.

There are several architectural options that can support multi-region adaptation. Selection among them will depend on a number of factors, including read-to-write ratio, latency requirements, budget, amenability to re-architecture, and preference for simplicity.

Generally, the challenge in providing multi-region SaaS is in instantiating stateful components in multiple regions because the data on which such components depend are hard to replicate, synchronize, and access with low latency over large geographical distances.

Of all stateful components, perhaps the most frequently encountered will be databases. Some solutions for overcoming this challenge with respect to databases are as follows:

1. Provide a master in a single region; provide read replicas in every region.
2. Provide a master in every region; assign each tenant to one master only.
3. Create a full multi-master architecture; replicate data efficiently.
4. Rely on a managed service for replicating data cross-regionally (e.g., DynamoDB Global Tables).

For more information on selecting an appropriate architecture for multi-regional stateful components such as databases, see AWS re:Invent 2017: How to Design a Multi-Region Active-Active Architecture (ARC319).

Stateless components can be provisioned in multiple regions more easily. In this example, you will have to re-create all of the VPC resources—including subnets, Routing Tables, Security Groups, and Endpoint Services—as well as all EC2 resources—including instances, NLBs, Listeners, Target Groups, ASGs, and Launch Configurations—in each additional region. Because of the complexity in doing so, in addition to the significant need to keep regional configurations in-sync, you may wish to explore an orchestration tool such as CloudFormation, rather than the command line.

Regardless of what orchestration tooling you choose, you will need to copy your AMI to each region in which you wish to deploy it. Once available, you can build out your service in that region much as you did in the first one.

Copy the AMI using the console:

aws ec2 copy-image \
--source-image-id "ami-08cefe72" \
--source-image-region "us-east-1" \
--name "order-processor" \
--region "us-east-2"

This copies the AMI used to provide the service in us-east-1 into us-east-2. Repeat the process for each region you wish to enable.

Consuming a Service via PrivateLink

To consume a service over PrivateLink, a customer must create a new Endpoint in their VPC within a Security Group that allows traffic on the traffic port.

Start by creating a Security Group to apply to a new Endpoint:

aws ec2 create-security-group \
 --group-name "consumer-sg" \
 --description "Security group for service consumers" \
 --vpc-id "vpc-2cadab54"

Next, create an endpoint in the VPC, placing it in the Security Group:

aws ec2 create-vpc-endpoint \
 --vpc-endpoint-type "Interface" \
 --vpc-id "vpc-2cadab54" \
 --service-name "com.amazonaws.vpce.us-east-1.vpce-svc-02d3cec2f5605e95b" \
 --subnet-ids "subnet-6d86e242" "subnet-a07e53eb" "subnet-fc871fa1" "subnet-e29ac586" "subnet-8a90fab5" "subnet-448f7e4b" \
 --security-group-ids "sg-22a94555"

The response will include an attribute called VpcEndpoint.DnsEntries. The service can be accessed at each of the DNS names in the output under any of the entries there. Before the consumer can access the endpoint service, the provider has to accept the Endpoint.

Access Endpoint Via Custom DNS Names

When creating a new Endpoint, the consumer will receive named endpoint addresses in each AZ where the Endpoint is created, plus a named endpoint that is AZ-agnostic. For example:

AZ	Endpoint Address
n/a	vpce-075c50b2becc4c030-zxxmdhjz.vpce-svc-02d3cec2f5605e95b.us-east-1.vpce.amazonaws.com
us-east-1a	vpce-075c50b2becc4c030-zxxmdhjz-us-east-1a.vpce-svc-02d3cec2f5605e95b.us-east-1.vpce.amazonaws.com
us-east-1b	vpce-075c50b2becc4c030-zxxmdhjz-us-east-1b.vpce-svc-02d3cec2f5605e95b.us-east-1.vpce.amazonaws.com
us-east-1c	vpce-075c50b2becc4c030-zxxmdhjz-us-east-1c.vpce-svc-02d3cec2f5605e95b.us-east-1.vpce.amazonaws.com
us-east-1d	vpce-075c50b2becc4c030-zxxmdhjz-us-east-1d.vpce-svc-02d3cec2f5605e95b.us-east-1.vpce.amazonaws.com
us-east-1e	vpce-075c50b2becc4c030-zxxmdhjz-us-east-1e.vpce-svc-02d3cec2f5605e95b.us-east-1.vpce.amazonaws.com
us-east-1f	vpce-075c50b2becc4c030-zxxmdhjz-us-east-1f.vpce-svc-02d3cec2f5605e95b.us-east-1.vpce.amazonaws.com

The consumer can use Route53 to provide a custom DNS name for the service. This not only allows for using cleaner service names, but also enables the consumer to leverage the traffic management features of Route53, such as fail-over routing.

First, the the consumer must enable DNS Hostnames and DNS Support on the VPC within which the Endpoint was created. The consumer should start by enabling DNS Hostnames:

aws ec2 modify-vpc-attribute \
 --vpc-id "vpc-2cadab54" \
 --enable-dns-hostnames

Next, the consumer must enable DNS Support:

aws ec2 modify-vpc-attribute \
 --vpc-id "vpc-2cadab54" \
 --enable-dns-support

After the VPC is properly configured to work with Route53, the consumer should either select an existing hosted zone or create a new one. Assuming one has not already been created, the consumer should create one as follows:

aws route53 create-hosted-zone \
 --name "endpoints.internal" \
 --caller-reference "$(uuidgen)" \
 --vpc "VPCRegion=us-east-1,VPCId=vpc-2cadab54" \
 --hosted-zone-config "PrivateZone=true"

In the request, the consumer specifies the DNS name, VPC ID, region, and flags the hosted zone as private. Additionally, the consumer must provide a “caller reference” which is a unique ID of the request that can be used to identify it in subsequent actions if the request fails.

Next, the consumer should create a JSON file corresponding to a batch of record change requests. In this file, the consumer can specify the name of the endpoint, as well as a CNAME pointing to the AZ-agnostic DNS name of the Endpoint:

{
"Changes": [
{
"Action": "CREATE"
"ResourceRecordSet": {
"Name": "order-processor.endpoints.internal"
"Type": "CNAME"
"TTL": 300
"ResourceRecords": [
{
"Value": "vpce-075c50b2becc4c030-zxxmdhjz.vpce-svc-02d3cec2f5605e95b.us-east-1.vpce.amazonaws.com"
}
]
}
}
]
}

Next, the consumer should provide the batch file in a request to change resource record sets:

aws route53 change-resource-record-sets \
--hosted-zone-id "/hostedzone/ZRHJB5B686XCI" \
--change-batch "$(cat change-resource-record-sets.json)"

At this point, the Endpoint can be consumed at http://order-processor.endpoints.internal.

Conclusion

AWS PrivateLink is an exciting way to expose SaaS services to customers. This article demonstrated how to expose an existing application on EC2 via PrivateLink in a customer’s VPC, as well as recommended architecture. Finally, it walked through the steps that a customer would have to go through to consume the service.

For more information on setting up AWS PrivateLink, see Interface VPC Endpoints AWS PrivateLink.

About the Author

Morris Singer is a Partner Solutions Architect for the AWS Partner Program.

MagPi 68: an in-depth look at the new Raspberry Pi 3B+

2018-03-29 Rob Zwetsloot

Post Syndicated from Rob Zwetsloot original https://www.raspberrypi.org/blog/magpi-68/

Hi folks, Rob from The MagPi here! You may remember that a couple of weeks ago, the Raspberry Pi 3 Model B+ was released, the updated version of the Raspberry Pi 3 Model B. It’s better, faster, and stronger than the original and it’s also the main topic in The MagPi issue 68, out now!

Everything you need to know about the new Raspberry Pi 3B+

What goes into ‘plussing’ a Raspberry Pi? We talked to Eben Upton and Roger Thornton about the work that went into making the Raspberry Pi 3B+, and we also have all the benchmarks to show you just how much the new Pi 3B+ has been improved.

Super fighting robots

Did you know that the next Pi Wars is soon? The 2018 Raspberry Pi robotics competition is taking place later in April, and we’ve got a full feature on what to expect, as well as top tips on how to make your own kick-punching robot for the next round.

Get The MagPi 68

Issue 68 is available today from WHSmith, Tesco, Sainsbury’s, and Asda. If you live in the US, head over to your local Barnes & Noble or Micro Center in the next few days for a print copy. You can also get the new issue online from our store, or digitally via our Android and iOS apps. And don’t forget, there’s always the free PDF as well.

New subscription offer!

Want to support the Raspberry Pi Foundation and the magazine? We’ve launched a new way to subscribe to the print version of The MagPi: you can now take out a monthly £4 subscription to the magazine, effectively creating a rolling pre-order system that saves you money on each issue.

You can also take out a twelve-month print subscription and get a Pi Zero W, Pi Zero case, and adapter cables absolutely free! This offer does not currently have an end date.

That’s it for now. See you next month!

The post MagPi 68: an in-depth look at the new Raspberry Pi 3B+ appeared first on Raspberry Pi.

EU Content Rules to Improve Access & Reduce Piracy Start April 1

2018-03-28 Andy

Post Syndicated from Andy original https://torrentfreak.com/eu-content-rules-to-improve-access-reduce-piracy-start-april-1-180328/

Any subscriber of a service like Netflix will tell you that where you live can have a big impact on the content made available. Customers in the US enjoy large libraries while less populous countries are treated less well.

For many years and before Netflix largely closed the loophole, customers would bypass these restrictions, using VPNs to trick Netflix into thinking they were elsewhere. Some wouldn’t bother with the complication, choosing to pirate content instead.

But for citizens of the EU, things were even more complex. While the EU mandates free movement of people, the same can’t be said about licensing deals. While a viewer in the Netherlands could begin watching a movie at home, he could travel to France for a weekend break only to find that the content he paid for is not available, or only in French.

Last May, this problem was addressed by the European Parliament with an agreement to introduce new ‘Cross-border portability’ rules that will give citizens the freedom to enjoy their media wherever they are in the EU, without having to resort to piracy or VPNs – if they can find one that still works for any length of time with the service.

Now, almost 11 months on, the rules are about to come into force. From Sunday, content portability in the EU will become a reality.

“Citizens are at the core of all our digital initiatives. As of 1 April, wherever you are traveling to in the EU, you will no longer miss out on your favorite films, TV series, sports broadcasts, games or e-books, that you have digitally subscribed to at home,” European Commission Vice-President Andrus Ansip said in a statement.

“Removing the boundaries that prevented Europeans from traveling with digital media and content subscriptions is yet another success of the Digital Single Market for our citizens, following the effective abolition of roaming charges that consumers all over Europe have enjoyed since June 2017.”

This is how it will work. Consumers in the EU who buy or subscribe to films, sports broadcasts, music, e-books or games in their home Member States will now be able to access this content when they reside temporarily in another EU country.

So, if a person in the UK purchases Netflix to gain access to a TV show to watch in their home country, Netflix will have to add this content to the customer’s library so they can still access it wherever they travel in the EU, regardless of its general availability elsewhere.

“[P]roviders of paid-for online content services (such as online movie, TV or music streaming services) have to provide their subscribers with the same service wherever the subscriber is in the EU,” the Commission explains.

“The service needs to be provided in the same way in other Member States, as in the Member State of residence. So for Netflix for example, you will have access to the same selection (or catalog) anywhere in the EU, if you are temporarily abroad, just as if you were at home.”

The same should hold true for all other digital content. If it’s available at home, it must be made available elsewhere in Europe in order to comply with the regulations. In doing so, providers are allowed some freedom, provided it’s in the customer’s favor. If they want to give customers additional access to full home and overseas catalogs when they’re traveling, for example, that is fine.

There’s also a plus in there for content providers. While a company like Netflix will sometimes acquire rights on a per country basis, when a citizen travels abroad within the EU they will not be required to obtain licenses for those other territories where their subscribers stay temporarily.

There is, however, a question of what “temporarily” means since it’s not tightly defined in the regulations. The term will cover business trips and holidays, for example, but providers will be required to clearly inform their customers of their precise terms and conditions.

Providers will also need to determine a customer’s home country, something that will be established when a customer signs up or renews his contract. This can be achieved in a number of ways, including via payment details, a contract for an Internet or telephone connection, verifying a home address, or using a simple IP address check.

For providers of free online services, which are allowed to choose whether they want to be included in the new rules or not, there are special conditions in place.

“Once they opt-in and allow portability under the Regulation, all rules will apply to them in the same manner as for the paid services. This means that the subscribers will have to log-in to be able to access and use content when temporarily abroad, and service providers will have to verify the Member State of residence of the subscriber,” the Commission explains.

“If providers of free of charge online content services decide to make use of the new portability rules, they are required to inform their subscribers about this decision prior to providing the service. Such information could, for example, be announced on the providers’ websites.”

The good news for consumers is that providers will not be able to charge for offering content portability and if they don’t provide it as required, they’ll be in breach of EU rules. The EU believes that all providers are ready to meet the standard – the public will find out on Sunday.

The new rules can be found here (pdf)

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN reviews, discounts, offers and coupons.

Cambridge Analytica Facebook Data Scandal

2018-03-25 Darknet

Post Syndicated from Darknet original https://www.darknet.org.uk/2018/03/cambridge-analytica-facebook-data-scandal/?utm_source=rss&utm_medium=social&utm_campaign=darknetfeed

One of the biggest stories of the year so far has been the scandal surrounding Cambridge Analytica that came out after a Channel 4 expose that demonstrated the depths they are willing to go to profile voters, manipulate elections and much more.

It’s kicking off in the UK and the US and Mark Zuckerberg has had to come out publically and apologise about the involvement of Facebook.

This goes deep with ties to elections and political activities in Malaysia, Mexico, Brazil, Australia and Kenya.

Read the rest of Cambridge Analytica Facebook Data Scandal now! Only available at Darknet.

Power from wind: Open data on AWS

2018-03-20 Caleb Phillips

Post Syndicated from Caleb Phillips original https://aws.amazon.com/blogs/big-data/power-from-wind-open-data-on-aws/

Data that describe processes in a spatial context are everywhere in our day-to-day lives and they dominate big data problems. Map data, for instance, whether describing networks of roads or remote sensing data from satellites, get us where we need to go. Atmospheric data from simulations and sensors underlie our weather forecasts and climate models. Devices and sensors with GPS can provide a spatial context to nearly all mobile data.

In this post, we introduce the WIND toolkit, a huge (500 TB), open weather model dataset that’s available to the world on Amazon’s cloud services. We walk through how to access this data and some of the open-source software developed to make it easily accessible. Our solution considers a subset of geospatial data that exist on a grid (raster) and explores ways to provide access to large-scale raster data from weather models. The solution uses foundational AWS services and the Hierarchical Data Format (HDF), a well adopted format for scientific data.

The approach developed here can be extended to any data that fit in an HDF5 file, which can describe sparse and dense vectors and matrices of arbitrary dimensions. This format is already popular within the physical sciences for both experimental and simulation data. We discuss solutions to gridded data storage for a massive dataset of public weather model outputs called the Wind Integration National Dataset (WIND) toolkit. We also highlight strategies that are general to other large geospatial data management problems.

Wind Integration National Dataset

As variable renewable power penetration levels increase in power systems worldwide, the importance of renewable integration studies to ensure continued economic and reliable operation of the power grid is also increasing. The WIND toolkit is the largest freely available grid integration dataset to date.

The WIND toolkit was developed by 3TIER by Vaisala. They were under a subcontract to the National Renewable Energy Laboratory (NREL) to support studies on integration of wind energy into the existing US grid. NREL is a part of a network of national laboratories for the US Department of Energy and has a mission to advance the science and engineering of energy efficiency, sustainable transportation, and renewable power technologies.

The toolkit has been used by consultants, research groups, and universities worldwide to support grid integration studies. Less traditional uses also include resource assessments for wind plants (such as those powering Amazon data centers), and studying the effects of weather on California condor migrations in the Baja peninsula.

The diversity of applications highlights the value of accessible, open public data. Yet, there’s a catch: the dataset is huge. The WIND toolkit provides simulated atmospheric (weather) data at a two-km spatial resolution and five-minute temporal resolution at multiple heights for seven years. The entire dataset is half a petabyte (500 TB) in size and is stored in the NREL High Performance Computing data center in Golden, Colorado. Making this dataset publicly available easily and in a cost-effective manner is a major challenge.

As other laboratories and public institutions work to release their data to the world, they may face similar challenges to those that we experienced. Some prior, well-intentioned efforts to release huge datasets as-is have resulted in data resources that are technically available but fundamentally unusable. They may be stored in an unintuitive format or indexed and organized to support only a subset of potential uses. Downloading hundreds of terabytes of data is often impractical. Most users don’t have access to a big data cluster (or super computer) to slice and dice the data as they need after it’s downloaded.

We aim to provide a large amount of data (50 terabytes) to the public in a way that is efficient, scalable, and easy to use. In many cases, researchers can access these huge cloud-located datasets using the same software and algorithms they have developed for smaller datasets stored locally. Only the pieces of data they need for their individual analysis must be downloaded. To make this work in practice, we worked with the HDF Group and have built upon their forthcoming Highly Scalable Data Service.

In the rest of this post, we discuss how the HSDS software was developed to use Amazon EC2 and Amazon S3 resources to provide convenient and scalable access to these huge geospatial datasets. We describe how the HSDS service has been put to work for the WIND Toolkit dataset and demonstrate how to access it using the h5pyd Python library and the REST API. We conclude with information about our ongoing work to release more ‘open’ datasets to the public using AWS services, and ways to improve and extend the HSDS with newer Amazon services like Amazon ECS and AWS Lambda.

Developing a scalable service for big geospatial data

The HDF5 file format and API have been used for many years and is an effective means of storing large scientific datasets. For example, NASA’s Earth Observing System (EOS) satellites collect more than 16 TBs of data per day using HDF5.

With the rise of the cloud, there are new challenges and opportunities to rethink how HDF5 can be enhanced to work effectively as a component in a cloud-native architecture. For the HDF Group, working with NREL has been a great opportunity to put ideas into practice with a production-size dataset.

An HDF5 file consists of a directed graph of group and dataset objects. Datasets can be thought of as a multidimensional array with support for user-defined metadata tags and compression. Typical operations on datasets would be reading or writing data to a regular subregion (a hyperslab) or reading and writing individual elements (a point selection). Also, group and dataset objects may each contain an arbitrary number of the user-defined metadata elements known as attributes.

Many people have used the HDF library in applications developed or ported to run on EC2 instances, but there are a number of constraints that often prove problematic:

The HDF5 library can’t read directly from HDF5 files stored as S3 objects. The entire file (often many GB in size) would need to be copied to local storage before the first byte can be read. Also, the instance must be configured with the appropriately sized EBS volume)
The HDF library only has access to the computational resources of the instance itself (as opposed to a cluster of instances), so many operations are bottlenecked by the library.
Any modifications to the HDF5 file would somehow have to be synchronized with changes that other instances have made to same file before writing back to S3.

Using a pattern common to many offerings from AWS, the solution to these constraints is to develop a service framework around the HDF data model. Using this model, the HDF Group has created the Highly Scalable Data Service (HSDS) that provides all the functionality that traditionally was provided by the HDF5 library. By using the service, you don’t need to manage your own file volumes, but can just read and write whatever data that you need.

Because the service manages the actual data persistence to a durable medium (S3, in this case), you don’t need to worry about disk management. Simply stream the data you need from the service as you need it. Secondly, putting the functionality behind a service allows some tricks to increase performance (described in more detail later). And lastly, HSDS allows any number of clients to access the data at the same time, enabling HDF5 to be used as a coordination mechanism for multiple readers and writers.

In designing the HSDS architecture, we gave much thought to how to achieve scalability of the HSDS service. For accessing HDF5 data, there are two different types of scaling to consider:

Multiple clients making many requests to the service
Single requests that require a significant amount of data processing

To deal with the first scaling challenge, as with most services, we considered how the service responds as the request rate increases. AWS provides some great tools that help in this regard:

Auto Scaling groups
Elastic Load Balancing load balancers
The ability of S3 to handle large aggregate throughput rates

By using a cluster of EC2 instances behind a load balancer, you can handle different client loads in a cost-effective manner.

The second scaling challenge concerns single requests that would take significant processing time with just one compute node. One example of this from the WIND toolkit would be extracting all the values in the seven-year time span for a given geographic point and dataset.

In HDF5, large datasets are typically stored as “chunks”; that is, a regular partition of the array. In HSDS, each chunk is stored as a binary object in S3. The sequential approach to retrieving the time series values would be for the service to read each chunk needed from S3, extract the needed elements, and go on to the next chunk. In this case, that would involve processing 2557 chunks, and would be quite slow.

Fortunately, with HSDS, you can speed this up quite a bit by exploiting the compute and I/O capabilities of the cluster. Upon receiving the request, the receiving node can use other nodes in the cluster to read different portions of the selection. With multiple nodes reading from S3 in parallel, performance improves as the cluster size increases.

The diagram below illustrates how this works in simplified case of four chunks and four nodes.

This architecture has worked in well in practice. In testing with the WIND toolkit and time series extraction, we observed a request latency of ~60 seconds using four nodes vs. ~5 seconds with 40 nodes. Performance roughly scales with the size of the cluster.

A planned enhancement to this is to use AWS Lambda for the worker processing. This enables 1000-way parallel reads at a reasonable cost, as you only pay for the milliseconds of CPU time used with AWS Lambda.

Public access to atmospheric data using HSDS and AWS

An early challenge in releasing the WIND toolkit data was in deciding how to subset the data for different use cases. In general, few researchers need access to the entire 0.5 PB of data and a great deal of efficiency and cost reduction can be gained by making directed constituent datasets.

NREL grid integration researchers initially extracted a 2-TB subset by selecting 120,000 points where the wind resource seemed appropriate for development. They also chose only those data important for wind applications (100-m wind speed, converted to power), the most interesting locations for those performing grid studies. To support the remaining users who needed more data resolution, we down-sampled the data to a 60-minute temporal resolution, keeping all the other variables and spatial resolution intact. This reduced dataset is 50 TB of data describing 30+ atmospheric variables of data for 7 years at a 60-minute temporal resolution.

The WindViz browser-based Gridded Wind Toolkit Visualizer was created as an example implementation of the HSDS REST API in JavaScript. The visualizer is written in the style of ECMAScript 2016 using a modern development toolchain that includes webpack and Babel. The source code is available through our GitHub repository. The demo page is hosted via GitHub pages, and we use a cross-origin AJAX request to fetch data from the HSDS service running on the EC2 infrastructure. The visualizer can be used to explore the gridded wind toolkit data on a map. Achieve full spatial resolution by zooming in to a specific region.

Programmatic access is possible using the h5pyd Python library, a distributed analog to the widely used h5py library. Users interact with the datasets (variables) and slice the data from its (time x longitude x latitude) cube form as they see fit.

Examples and use cases are described in a set of Jupyter notebooks and available on GitHub:

NREL/hsds-examples

To run these notebooks on an EC2 instance in the Oregon Region, run the following commands:

$ sudo yum install git gcc
$ sudo pip install –user jupyter
$ pip install --user git+http://github.com/HDFGroup/h5pyd.git
$ git clone https://github.com/NREL/hsds-examples.git
$ cd hsds-examples
$ jupyter notebook

Now you have a Jupyter notebook server running on your EC2 server.

From your laptop, create an SSH tunnel:

$ ssh –L 8888:localhost:8888 (IP address of the EC2 server)

Now, you can browse to localhost:8888 using the correct token, and interact with the notebooks as if they were local. Within the directory, there are examples for accessing the HSDS API and plotting wind and weather data using matplotlib.

Controlling access and defraying costs

A final concern is rate limiting and access control. Although the HSDS service is scalable and relatively robust, we had a few practical concerns:

How can we protect from malicious or accidental use that may lead to high egress fees (for example, someone who attempts to repeatedly download the entire dataset from S3)?
How can we keep track of who is using the data both to document the value of the data resource and to justify the costs?
If costs become too high, can we charge for some or all API use to help cover the costs?

To approach these problems, we investigated using Amazon API Gateway and its simplified integration with the AWS Marketplace for SaaS monetization as well as third-party API proxies.

In the end, we chose to use API Umbrella due to its close involvement with http://data.gov. While AWS Marketplace is a compelling option for future datasets, the decision was made to keep this dataset entirely open, at least for now. As community use and associated costs grow, we’ll likely revisit Marketplace. Meanwhile, API Umbrella provides controls for rate limiting and API key registration out of the box and was simple to implement as a front-end proxy to HSDS. Those applications that may want to charge for API use can accomplish a similar strategy using Amazon API Gateway and AWS Marketplace.

Ongoing work and other resources

As NREL and other government research labs, municipalities, and organizations try to share data with the public, we expect many of you will face similar challenges to those we have tried to approach with the architecture described in this post. Providing large datasets is one challenge. Doing so in a way that is affordable and convenient for users is an entirely more difficult goal. Using AWS cloud-native services and the existing foundation of the HDF file format has allowed us to tackle that challenge in a meaningful way.

Additional Reading

If you found this post useful, be sure to check out Perform Near Real-time Analytics on Streaming Data with Amazon Kinesis and Amazon Elasticsearch Service, Analyze OpenFDA Data in R with Amazon S3 and Amazon Athena and Querying OpenStreetMap with Amazon Athena.

About the Authors

Dr. Caleb Phillips is a senior scientist with the Data Analysis and Visualization Group within the Computational Sciences Center at the National Renewable Energy Laboratory. Caleb comes from a background in computer science systems, applied statistics, computational modeling, and optimization. His work at NREL spans the breadth of renewable energy technologies and focuses on applying modern data science techniques to data problems at scale.

Dr. Caroline Draxl is a senior scientist at NREL. She supports the research and modeling activities of the US Department of Energy from mesoscale to wind plant scale. Caroline uses mesoscale models to research wind resources in various countries, and participates in on- and offshore boundary layer research and in the coupling of the mesoscale flow features (kilometer scale) to the microscale (tens of meters). She holds a M.S. degree in Meteorology and Geophysics from the University of Innsbruck, Austria, and a PhD in Meteorology from the Technical University of Denmark.

John Readey has been a Senior Architect at The HDF Group since he joined in June 2014. His interests include web services related to HDF, applications that support the use of HDF and data visualization.Before joining The HDF Group, John worked at Amazon.com from 2006–2014 where he developed service-based systems for eCommerce and AWS.

Jordan Perr-Sauer is an RPP intern with the Data Analysis and Visualization Group within the Computational Sciences Center at the National Renewable Energy Laboratory. Jordan hopes to use his professional background in software engineering and his academic training in applied mathematics to solve the challenging problems facing America and the world.

Dotcom Affidavit Calls For Obama to Give Evidence in Megaupload Case

2018-03-20 Andy

Post Syndicated from Andy original https://torrentfreak.com/dotcom-affidavit-calls-for-obama-to-give-evidence-in-megaupload-case-180320/

For more than six years since the raid on Megaupload, founder Kim Dotcom has insisted that the case against him, his co-defendants, and his company, was politically motivated.

The serial entrepreneur states unequivocally that former president Barack Obama’s close ties to Hollywood were the driving force.

Later today, Obama will touch down for a visit to New Zealand. In what appears to be a tightly managed affair, with heavy restrictions placed on the media and publicity, it seems clear that Obama wants to maintain control over his social and business engagements in the country.

But of course, New Zealand is home to Kim Dotcom and as someone who feels wronged by the actions of the former administration, he is determined to use this opportunity to shine more light on Obama’s role in the downfall of his company.

In a statement this morning, Dotcom reiterated his claims that attempts to have him extradited to the United States have no basis in law, chiefly due to the fact that the online dissemination of copyright-protected works by Megaupload’s users is not an extradition offense in New Zealand.

But Dotcom also attacks the politics behind his case, arguing that the Obama administration was under pressure from Hollywood to do something about copyright enforcement or risk losing financial support.

In connection with his case, Dotcom is currently suing the New Zealand government for billions of dollars so while Obama is in town, Dotcom is demanding that the former president gives evidence.

Dotcom’s case is laid out in a highly-detailed sworn affidavit dated March 19, 2018. The Megaupload founder explains that Hollywood has historically been a major benefactor of the Democrats so when seeking re-election for a further term, the Democrats were under pressure from the movie companies to make an example of Megaupload and Dotcom.

Dotcom notes that while he was based in Hong Kong, extradition to the US would be challenging. So, with Dotcom seeking residence in New Zealand, a plot was hatched to allow him into the country, despite the New Zealand government knowing that a criminal prosecution lay in wait for him. Dotcom says that by doing a favor for Hollywood, it could mean that New Zealand became a favored destination for US filmmakers.

“The interests of the United States and New Zealand were therefore perfectly aligned. I provided the perfect opportunity for New Zealand to facilitate the United States’ show of force on copyright enforcement,” Dotcom writes.

Citing documents obtained from Open Secrets, Dotcom shows how the Democrats took an 81% share of more than $46m donated to political parties in the US during the 2008 election cycle. In the 2010 cycle, 76% of more than $24m went to the Democrats and in 2012, they scooped up 78% of more than $56m.

Dotcom then recalls the attempts at passing the Stop Online Piracy Act (SOPA), which would have shifted the enforcement of copyright onto ISPs, assisting Hollywood greatly. Ultimately, Congressional support for the proposed legislation was withdrawn and Dotcom recalls this was followed by a public threat from the MPAA to withdraw campaign contributions on which the Democrats were especially reliant.

“The message to the White House was plain: do not expect funding if you do not advance the MPAA’s legislative agenda. On 20 January 2012, the day after this statement, I was arrested,” Dotcom notes.

Describing Megaupload as a highly profitable and innovative platform that highlighted copyright owners’ failure to keep up with the way in which content is now consumed, Dotcom says it made the perfect target for the Democrats.

Convinced the party was at the root of his prosecution, he utilized his connections in Hong Kong to contact Thomas Hart, a lawyer and lobbyist in Washington, D.C. with strong connections to the Democrats and the White House.

Dotcom said a telephone call between him and Mr Hart revealed that then Vice President Joe Biden was at the center of Dotcom’s prosecution but that Obama was dissatisfied with the way things had been handled.

“Biden did admit to have… you know, kind of started it, you know, along with support from others but it was Biden’s decision…,” Hart allegedly said.

“What he [President Obama] expressed to me was a growing concern about the matter. He indicated an awareness of that it had not gone well, that it was more complicated than he thought, that he will turn his attention to it more prominently after November.”

Dotcom says that Obama was “questioning the whole thing,” a suggestion that he may not have been fully committed to the continuing prosecution.

The affidavit then lists a whole series of meetings in 2011, documented in the White House visitor logs. They include meetings with then United States Attorney Neil McBride, various representatives from Hollywood, MPAA chief Chris Dodd, Mike Ellis of the MPA (who was based in Hong Kong and had met with New Zealand’s then Minister of Justice, Simon Power) and the Obama administration.

In summary, Dotcom suggests there was a highly organized scheme against him, hatched between Hollywood and the Obama administration, that had the provision of funds to win re-election at its heart.

From there, an intertwined agreement was reached at the highest levels of both the US and New Zealand governments where the former would benefit through tax concessions to Hollywood (and a sweetening of relations between the countries) and the latter would benefit financially through investment.

All New Zealand had to do was let Dotcom in for a while and then hand him over to the United States for prosecution. And New Zealand definitely knew that Dotcom was wanted by the US. Emails obtained by Dotcom concerning his residency application show that clearly.

“Kim DOTCOM is not of security concern but is likely to soon become the subject of a joint FBI / NZ Police criminal investigation. We have passed this over to NZ Police,” one of the emails reads. Another, well over a year before the raid, also shows the level of knowledge.

Bad but wealthy, so we have plans for him…

With “political pressure” to grant Dotcom’s application in place, Immigration New Zealand finally gave the Megaupload founder the thumbs-up on November 1, 2010. Dotcom believes that New Zealand was concerned he may have walked away from his application.

“This would have been of grave concern to the Government, which, at that time, was in negotiations with Hollywood lobby,” his affidavit reads.

“The last thing they would have needed at that delicate stage of the negotiations was for me to walk away from New Zealand and return to Hong Kong, where extradition would be more difficult. I believe that this concern is what prompted the ‘political pressure’ that led to my application finally being granted despite the presence of factors that would have caused anyone else’s application to have been rejected.”

Dotcom says that after being granted residency, there were signs things weren’t going to plan for him. The entrepreneur applied to buy his now-famous former mansion for NZ$37m, an application that was initially approved. However, after being passed to Simon Power, the application was denied.

“It would appear that, although my character was apparently good enough for me to be granted residence in November 2010, in July 2011 it was not considered good enough for me to buy property in New Zealand,” Dotcom notes.

“The Honourable Mr Power clearly did not want me purchasing $37 million of real estate, presumably because he knew that the United States was going to seek forfeiture of my assets and he did not want what was then the most expensive property in New Zealand being forfeited to the United States government.”

Of course, Dotcom concludes by highlighting the unlawful spying by New Zealand’s GCSB spy agency and the disproportionate use of force displayed by the police when they raided him in 2010 using dozens of armed officers. This, combined with all of the above, means that questions about his case must now be answered at the highest levels. With Obama in town, there’s no time like the present.

“As the evidence above demonstrates, this improper purpose which was then embraced by the New Zealand authorities, originated in the White House under the Obama administration. It is therefore necessary to examine Mr Obama in this proceeding,” Dotcom concludes.

Press blackouts aside, it appears that Obama has rather a lot of golf lined up for the coming days. Whether he’ll have any time to answer Dotcom’s questions is one thing but whether he’ll even be asked to is perhaps the most important point of all.

The full affidavit and masses of supporting evidence can be found here.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN reviews, discounts, offers and coupons.

Deezer Piles Pressure on Pirates, Deezloader Reborn Throws in the Towel

2018-03-15 Andy

Post Syndicated from Andy original https://torrentfreak.com/deezer-piles-pressure-on-pirates-deezloader-reborn-throws-in-the-towel-180315/

Spotify might grab most of the headlines in the world of music streaming but French firm Deezer is also growing in popularity.

Focused more on non-English speaking regions, the music service still has a massive selection of tens of millions of tracks. More importantly for pirates, it also has a loophole or two that allows users to permanently download songs from the service, a huge ‘selling’ point for the compulsive archiver.

One of the most popular third-party tools for achieving this was Deezloader but last year Deezer put pressure on its operators to cease-and-desist.

“On April 27, 2017 we received takedowns and threatened legal action from Deezer if we don’t shut down by April 29. So we decided to shut down Deezloader permanently,” the team announced.

Rather than kill the scene, the attack on Deezloader only seemed to spur things on. Many other apps underwent development in the months that followed but last December it became evident that Deezer (and probably the record labels supplying its content) were growing increasingly tired of these kinds of applications.

The company sent a wave of DMCA notices to developer platform GitHub, targeting several tools, claiming that they are “in total violation of our rights and of the rights of our music licensors.”

GitHub responded quickly by removing access to repositories referencing Deezloader, DeezerDownload, Deeze, Deezerio, Deezit, Deedown, and their associated forks. Deezer also reportedly modified its API, in order to stop or hinder apps already in existence.

However, pirates are a determined bunch and behind the scenes many sought to breathe new life into their projects, to maintain the flow of free music from Deezer. One of those that gained traction was the obviously-titled ‘Deezloader Reborn’ which enjoyed a new lease of life on both Github and Reddit after taking over from DeezLoader V2.3.1.

But in January 2018, Deezer turned up the pressure again, hitting Github with a wave (1,2) of takedown notices targeting various projects. On January 23, Deezer hit Deezloader Reborn itself with the notice detailed below.

The following project, identified in the paragraph below, makes available a hacked version of our Deezer application by describing methods to bypass Deezer’s security measures to unlawfully download its music catalogue, in total violation of our rights and of the rights of our music licensors (phonographic producers, performing artists, songwriters and composers):

https://github.com/ExtendLord/DeezLoader-Reborn

I therefore ask that you immediately take down the project corresponding to the URL above and all of the related forks by others members who have had access or even contributed to such projects.

Not only did Github comply with Deezer’s request, Reddit did too. According to a thread still listed on the site, Reddit removed a post about Deezloader Reborn following a copyright complaint from Deezer.

Two days later Deezer targeted similar projects on Github but by this time, Deezloader Reborn already had new plans. Speaking with TF, project developer ExtendLord said that he wouldn’t be shutting down but would continue on code repository Gitlab instead. Now, however, those plans have also come to an abrupt end after Gitlab took the page down.

Deezloader Reborn – gone from Gitlab

A copy of the page available on Archive.org shows Deezloader Reborn at version 3.0.5 with the ability to download music ready-tagged and in FLAC quality. Links to newer versions are being shared on Reddit but it appears there is no longer a central trusted source for the application.

There’s no official confirmation yet but it seems likely that Deezer was behind the Gitlab takedown. TorrentFreak has contacted ExtendLord who linked us to this page which states that “DeezLoader Reborn is no longer maintained due to DMCA. [Version] 3.1.0 is the last update, no more updates will be made.”

So, at least for now, it appears that Deezloader Reborn will go the way of various other Deezer-reliant applications. That won’t be the end of the story though, that’s a certainty.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN reviews, discounts, offers and coupons.

Raspbian update: supporting different screen sizes

2018-03-15 Simon Long

Post Syndicated from Simon Long original https://www.raspberrypi.org/blog/raspbian-update-screen-sizes/

You may have noticed that we released a updated Raspbian software image yesterday. While the main reason for the new image was to provide support for the new Raspberry Pi 3 Model B+, the image also includes, alongside the usual set of bug fixes and minor tweaks, one significant chunk of new functionality that is worth pointing out.

Updating Raspbian on your Raspberry Pi

How to update to the latest version of Raspbian on your Raspberry Pi.

Compatibility

As a software developer, one of the most awkward things to deal with is what is known as platform fragmentation: having to write code that works on all the different devices and configurations people use. In my spare time, I write applications for iOS, and this has become increasingly painful over the last few years. When I wrote my first iPhone application, it only had to work on the original iPhone, but nowadays any iOS application has to work across several models of iPhone and iPad (which all have different processors and screens), and also across the various releases of iOS. And that’s before you start to consider making your code run on Android as well…

The good thing about developing for Raspberry Pi is that there is only a relatively small number of different models of Pi hardware. We try our best to make sure that, wherever possible, the Raspberry Pi Desktop software works on every model of Pi ever sold, and we’ve managed to do this for most of the software in the image. The only exceptions are some of the more recent applications like Chromium, which won’t run on the older ARM6 processors in the Pi 1 and the Pi Zero, and some applications that run very slowly due to needing more memory than the older platforms have.

Raspbian with different screen resolutions

But there is one area where we have no control over the hardware, and that is screen resolution. The HDMI port on the Pi supports a wide range of resolutions, and when you include the composite port and display connector as well, people can be using the desktop on a huge number of different screen sizes.

Supporting a range of screen sizes is harder than you might think. One problem is that the Linux desktop environment is made up of a large selection of bits of software from various different developers, and not all of these support resizing. And the bits of software that do support resizing don’t all do it in the same way, so making everything resize at once can be awkward.

This is why one of the first things I did when I first started working on the desktop was to create the Appearance Settings application in order to bring a lot of the settings for things like font and icon sizes into one place. This avoids users having to tweak several configuration files whenever they wanted to change something.

The Appearance Settings application was a good place to start regarding support of different screen sizes. One of the features I originally included was a button to set everything to a default value. This was really a default setting for screens of an average size, and the resulting defaults would not have worked that well on much smaller or much larger screens. Now, there is no longer a single defaults button, but a new Defaults tab with multiple options:

These three options adjust font size, icon size, and various other settings to values which ought to work well on screens with a high or low resolution. (The For medium screens option has the same effect as the previous defaults button.) The results will not be perfect in all circumstances and for all applications — as mentioned above, there are many different components used to create the desktop, and some of them don’t provide any way of resizing what they draw. But using these options should set the most important parts of the desktop and installed applications, such as icons, fonts, and toolbars, to a suitable size.

Pixel doubling

We’ve added one other option for supporting high resolution screens. At the bottom of the System tab in the Raspberry Pi Configuration application, there is now an option for pixel doubling:

We included this option to facilitate the use of the x86 version of Raspbian with ultra-high-resolution screens that have very small pixels, such as Apple’s Retina displays. When running our desktop on one of these, the tininess of the pixels made everything too small for comfortable use.

Enabling pixel doubling simply draws every pixel in the desktop as a 2×2 block of pixels on the screen, making everything exactly twice the size and resulting in a usable desktop on, for example, a MacBook Pro’s Retina display. We’ve included the option on the version of the desktop for the Pi as well, because we know that some people use their Pi with large-screen HDMI TVs.

As pixel doubling magnifies everything on the screen by a factor of two, it’s also a useful option for people with visual impairments.

How to update

As mentioned above, neither of these new functionalities is a perfect solution to dealing with different screen sizes, but we hope they will make life slightly easier for you if you’re trying to run the desktop on a small or large screen. The features are included in the new image we have just released to support the Pi 3B+. If you want to add them to your existing image, the standard upgrade from apt will do so. As shown in the video above, you can just open a terminal window and enter the following to update Raspbian:

sudo apt-get update
sudo apt-get dist-upgrade

As always, your feedback, either in comments here or on the forums, is very welcome.

The post Raspbian update: supporting different screen sizes appeared first on Raspberry Pi.

Raspberry Jam Big Birthday Weekend 2018 roundup

2018-03-13 Ben Nuttall

Post Syndicated from Ben Nuttall original https://www.raspberrypi.org/blog/big-birthday-weekend-2018-roundup/

A couple of weekends ago, we celebrated our sixth birthday by coordinating more than 100 simultaneous Raspberry Jam events around the world. The Big Birthday Weekend was a huge success: our fantastic community organised Jams in 40 countries, covering six continents!

We sent the Jams special birthday kits to help them celebrate in style, and a video message featuring a thank you from Philip and Eben:

Raspberry Jam Big Birthday Weekend 2018

To celebrate the Raspberry Pi’s sixth birthday, we coordinated Raspberry Jams all over the world to take place over the Raspberry Jam Big Birthday Weekend, 3-4 March 2018. A massive thank you to everyone who ran an event and attended.

The Raspberry Jam photo booth

I put together code for a Pi-powered photo booth which overlaid the Big Birthday Weekend logo onto photos and (optionally) tweeted them. We included an arcade button in the Jam kits so they could build one — and it seemed to be quite popular. Some Jams put great effort into housing their photo booth:

Here are some of my favourite photo booth tweets:

RGVSA on Twitter

PiParty photo booth @RGVSA & @ @Nerdvana_io #Rjam

Denis Stretton on Twitter

The @SouthendRPIJams #PiParty photo booth

rpijamtokyo on Twitter

PiParty photo booth

Preston Raspberry Jam on Twitter

Preston Raspberry Jam Photobooth #RJam #PiParty

If you want to try out the photo booth software yourself, find the code on GitHub.

The great Raspberry Jam bake-off

Traditionally, in the UK, people have a cake on their birthday. And we had a few! We saw (and tasted) a great selection of Pi-themed cakes and other baked goods throughout the weekend:

Raspberry Jams everywhere

We always say that every Jam is different, but there’s a common and recognisable theme amongst them. It was great to see so many different venues around the world filling up with like-minded Pi enthusiasts, Raspberry Jam–branded banners, and Raspberry Pi balloons!

Europe

Sergio Martinez on Twitter

Thank you so much to all the attendees of the Ikana Jam in Krakow past Saturday! We shared fun experiences, some of them… also painful 😉 A big thank you to @Raspberry_Pi for these global celebrations! And a big thank you to @hubraum for their hospitality! #PiParty #rjam

NI Raspberry Jam on Twitter

We also had a super successful set of wearables workshops using @adafruit Circuit Playground Express boards and conductive thread at today’s @Raspberry_Pi Jam! Very popular! #PiParty

Suzystar on Twitter

My SenseHAT workshop, going well! @SouthendRPiJams #PiParty

Worksop College Raspberry Jam on Twitter

Learning how to scare the zombies in case of an apocalypse- it worked on our young learners #PiParty @worksopcollege @Raspberry_Pi https://t.co/pntEm57TJl

Africa

Rita on Twitter

Being one of the two places in Kenya where the #PiParty took place, it was an amazing time spending the day with this team and getting to learn and have fun. @TaitaTavetaUni and @Raspberry_Pi thank you for your support. @TTUTechlady @mictecttu ch

GABRIEL ONIFADE on Twitter

@TheMagP1

GABRIEL ONIFADE on Twitter

@GABONIAVERACITY #PiParty Lagos Raspberry Jam 2018 Special International Celebration – 6th Raspberry-Pi Big Birthday! Lagos Nigeria @Raspberry_Pi @ben_nuttall #RJam #RaspberryJam #raspberrypi #physicalcomputing #robotics #edtech #coding #programming #edTechAfrica #veracityhouse https://t.co/V7yLxaYGNx

North America

Heidi Baynes on Twitter

The Riverside Raspberry Jam @Vocademy is underway! #piparty

Brad Derstine on Twitter

The Philly & Pi #PiParty event with @Bresslergroup and @TechGirlzorg was awesome! The Scratch and Pi workshop was amazing! It was overall a great day of fun and tech!!! Thank you everyone who came out!

Houston Raspi on Twitter

Thanks everyone who came out to the @Raspberry_Pi Big Birthday Jam! Special thanks to @PBFerrell @estefanniegg @pcsforme @pandafulmanda @colnels @bquentin3 couldn’t’ve put on this amazing community event without you guys!

Merge Robotics 2706 on Twitter

We are back at @SciTechMuseum for the second day of @OttawaPiJam! Our robot Mergius loves playing catch with the kids! #pijam #piparty #omgrobots

South America

Javier Garzón on Twitter

Así terminamos el #Raspberry Jam Big Birthday Weekend #Bogota 2018 #PiParty de #RaspberryJamBogota 2018 @Raspberry_Pi Nos vemos el 7 de marzo en #ArduinoDayBogota 2018 y #RaspberryJamBogota 2018

Asia

Fablab UP Cebu on Twitter

Happy 6th birthday, @Raspberry_Pi! Greetings all the way from CEBU,PH! #PiParty #IoTCebu Thanks @CebuXGeeks X Ramos for these awesome pics. #Fablab #UPCebu

福野泰介 on Twitter

ラズパイ、6才のお誕生日会スタート in Tokyo PCNブースで、いろいろ展示とhttps://t.co/L6E7KgyNHFとIchigoJamつないだ、こどもIoTハッカソンmini体験やってます at 東京蒲田駅近 https://t.co/yHEuqXHvqe #piparty #pipartytokyo #rjam #opendataday

Ren Camp on Twitter

Happy birthday @Raspberry_Pi! #piparty #iotcebu @coolnumber9 https://t.co/2ESVjfRJ2d

Oceania

Glenunga Raspberry Pi Club on Twitter

PiParty photo booth

Personally, I managed to get to three Jams over the weekend: two run by the same people who put on the first two Jams to ever take place, and also one brand-new one! The Preston Raspberry Jam team, who usually run their event on a Monday evening, wanted to do something extra special for the birthday, so they came up with the idea of putting on a Raspberry Jam Sandwich — on the Friday and Monday around the weekend! This meant I was able to visit them on Friday, then attend the Manchester Raspberry Jam on Saturday, and finally drop by the new Jam at Worksop College on my way home on Sunday.

Ben Nuttall on Twitter

I’m at my first Raspberry Jam #PiParty event of the big birthday weekend! @PrestonRJam has been running for nearly 6 years and is a great place to start the celebrations!

Ben Nuttall on Twitter

Back at @McrRaspJam at @DigInnMMU for #PiParty

Ben Nuttall on Twitter

Great to see mine & @Frans_facts Balloon Pi-Tay popper project in action at @worksopjam #rjam #PiParty https://t.co/GswFm0UuPg

Various members of the Foundation team attended Jams around the UK and US, and James from the Code Club International team visited AmsterJam.

hackerfemo on Twitter

Thanks to everyone who came to our Jam and everyone who helped out. @phoenixtogether thanks for amazing cake & hosting. Ademir you’re so cool. It was awesome to meet Craig Morley from @Raspberry_Pi too. #PiParty

Stuart Fox on Twitter

Great #PiParty today at the @cotswoldjam with bloody delicious cake and lots of raspberry goodness. Great to see @ClareSutcliffe @martinohanlon playing on my new pi powered arcade build:-)

Clare Sutcliffe on Twitter

Happy 6th Birthday @Raspberry_Pi from everyone at the #PiParty at #cotswoldjam in Cheltenham!

Code Club on Twitter

It’s @Raspberry_Pi 6th birthday and we’re celebrating by taking part in @amsterjam__! Happy Birthday Raspberry Pi, we’re so happy to be a part of the family! #PiParty

For more Jammy birthday goodness, check out the PiParty hashtag on Twitter!

The Jam makers!

A lot of preparation went into each Jam, and we really appreciate all the hard work the Jam makers put in to making these events happen, on the Big Birthday Weekend and all year round. Thanks also to all the teams that sent us a group photo:

Lots of the Jams that took place were brand-new events, so we hope to see them continue throughout 2018 and beyond, growing the Raspberry Pi community around the world and giving more people, particularly youths, the opportunity to learn digital making skills.

Philip Colligan on Twitter

So many wonderful people in the @Raspberry_Pi community. Thanks to everyone at #PottonPiAndPints for a great afternoon and for everything you do to help young people learn digital making. #PiParty

Special thanks to ModMyPi for shipping the special Raspberry Jam kits all over the world!

Don’t forget to check out our Jam page to find an event near you! This is also where you can find free resources to help you get a new Jam started, and download free starter projects made especially for Jam activities. These projects are available in English, Français, Français Canadien, Nederlands, Deutsch, Italiano, and 日本語. If you’d like to help us translate more content into these and other languages, please get in touch!

PS Some of the UK Jams were postponed due to heavy snowfall, so you may find there’s a belated sixth-birthday Jam coming up where you live!

S Organ on Twitter

@TheMagP1 Ours was rescheduled until later in the Spring due to the snow but here is Babbage enjoying the snow!

The post Raspberry Jam Big Birthday Weekend 2018 roundup appeared first on Raspberry Pi.

HDD vs SSD: What Does the Future for Storage Hold?

2018-03-06 Roderick Bauer

Post Syndicated from Roderick Bauer original https://www.backblaze.com/blog/ssd-vs-hdd-future-of-storage/

This is part one of a series. Use the Join button above to receive notification of future posts on this and other topics.

Customers frequently ask us whether and when we plan to move our cloud backup and data storage to SSDs (Solid-State Drives). That’s not a surprising question considering the many advantages SSDs have over magnetic platter type drives, also known as HDDs (Hard-Disk Drives).

We’re a large user of HDDs in our data centers (currently 100,000 hard drives holding over 500 petabytes of data). We want to provide the best performance, reliability, and economy for our cloud backup and cloud storage services, so we continually evaluate which drives to use for operations and in our data centers. While we use SSDs for some applications, which we’ll describe below, there are reasons why HDDs will continue to be the primary drives of choice for us and other cloud providers for the foreseeable future.

HDDs vs SSDs

HDD vs SSD

The laptop computer I am writing this on has a single 512GB SSD, which has become a common feature in higher end laptops. The SSD’s advantages for a laptop are easy to understand: they are smaller than an HDD, faster, quieter, last longer, and are not susceptible to vibration and magnetic fields. They also have much lower latency and access times.

Today’s typical online price for a 2.5” 512GB SSD is $140 to $170. The typical online price for a 3.5” 512 GB HDD is $44 to $65. That’s a pretty significant difference in price, but since the SSD helps make the laptop lighter, enables it to be more resistant to the inevitable shocks and jolts it will experience in daily use, and adds of benefits of faster booting, faster waking from sleep, and faster launching of applications and handling of big files, the extra cost for the SSD in this case is worth it.

Some of these SSD advantages, chiefly speed, also will apply to a desktop computer, so desktops are increasingly outfitted with SSDs, particularly to hold the operating system, applications, and data that is accessed frequently. Replacing a boot drive with an SSD has become a popular upgrade option to breathe new life into a computer, especially one that seems to take forever to boot or is used for notoriously slow-loading applications such as Photoshop.

We covered upgrading your computer with an SSD in our blog post SSD 101: How to Upgrade Your Computer With An SSD.

Data centers are an entirely different kettle of fish. The primary concerns for data center storage are reliability, storage density, and cost. While SSDs are strong in the first two areas, it’s the third where they are not yet competitive. At Backblaze we adopt higher density HDDs as they become available — we’re currently using both 10TB and 12TB drives (among other capacities) in our data centers. Higher density drives provide greater storage density per Storage Pod and Vault and reduce our overhead cost through less required maintenance and lower total power requirements. Comparable SSDs in those sizes would cost roughly $1,000 per terabyte, considerably higher than the corresponding HDD. Simply put, SSDs are not yet in the price range to make their use economical for the benefits they provide, which is the reason why we expect to be using HDDs as our primary storage media for the foreseeable future.

What Are HDDs?

HDDs have been around over 60 years since IBM introduced them in 1956. The first disk drive was the size of a car, stored a mere 3.75 megabytes, and cost $300,000 in today’s dollars.

IBM 350 Disk Storage System — 3.75MB in 1956

The 350 Disk Storage System was a major component of the IBM 305 RAMAC (Random Access Method of Accounting and Control) system, which was introduced in September 1956. It consisted of 40 platters and a dual read/write head on a single arm that moved up and down the stack of magnetic disk platters.

The basic mechanism of an HDD remains unchanged since then, though it has undergone continual refinement. An HDD uses magnetism to store data on a rotating platter. A read/write head is affixed to an arm that floats above the spinning platter reading and writing data. The faster the platter spins, the faster an HDD can perform. Typical laptop drives today spin at either 5400 RPM (revolutions per minute) or 7200 RPM, though some server-based platters spin at even higher speeds.

Exploded drawing of a hard drive

The platters inside the drives are coated with a magnetically sensitive film consisting of tiny magnetic grains. Data is recorded when a magnetic write-head flies just above the spinning disk; the write head rapidly flips the magnetization of one magnetic region of grains so that its magnetic pole points up or down, to encode a 1 or a 0 in binary code. If all this sounds like an HDD is vulnerable to shocks and vibration, you’d be right. They also are vulnerable to magnets, which is one way to destroy the data on an HDD if you’re getting rid of it.

The major advantage of an HDD is that it can store lots of data cheaply. One and two terabyte (1,024 and 2,048 gigabytes) hard drives are not unusual for a laptop these days, and 10TB and 12TB drives are now available for desktops and servers. Densities and rotation speeds continue to grow. However, if you compare the cost of common HDDs vs SSDs for sale online, the SSDs are roughly 3-5x the cost per gigabyte. So if you want cheap storage and lots of it, using a standard hard drive is definitely the more economical way to go.

What are the best uses for HDDs?

Disk arrays (NAS, RAID, etc.) where high capacity is needed
Desktops when low cost is priority
Media storage (photos, videos, audio not currently being worked on)
Drives with extreme number of reads and writes

What Are SSDs?

SSDs go back almost as far as HDDs, with the first semiconductor storage device compatible with a hard drive interface introduced in 1978, the StorageTek 4305.

Storage Technology 4305 SSD

The StorageTek was an SSD aimed at the IBM mainframe compatible market. The STC 4305 was seven times faster than IBM’s popular 2305 HDD system (and also about half the price). It consisted of a cabinet full of charge-coupled devices and cost $400,000 for 45MB capacity with throughput speeds up to 1.5 MB/sec.

SSDs are based on a type of non-volatile memory called NAND (named for the Boolean operator “NOT AND,” and one of two main types of flash memory). Flash memory stores data in individual memory cells, which are made of floating-gate transistors. Though they are semiconductor-based memory, they retain their information when no power is applied to them — a feature that’s obviously a necessity for permanent data storage.

Samsung SSD 850 Pro

Compared to an HDD, SSDs have higher data-transfer rates, higher areal storage density, better reliability, and much lower latency and access times. For most users, it’s the speed of an SSD that primarily attracts them. When discussing the speed of drives, what we are referring to is the speed at which they can read and write data.

For HDDs, the speed at which the platters spin strongly determines the read/write times. When data on an HDD is accessed, the read/write head must physically move to the location where the data was encoded on a magnetic section on the platter. If the file being read was written sequentially to the disk, it will be read quickly. As more data is written to the disk, however, it’s likely that the file will be written across multiple sections, resulting in fragmentation of the data. Fragmented data takes longer to read with an HDD as the read head has to move to different areas of the platter(s) to completely read all the data requested.

Because SSDs have no moving parts, they can operate at speeds far above those of a typical HDD. Fragmentation is not an issue for SSDs. Files can be written anywhere with little impact on read/write times, resulting in read times far faster than any HDD, regardless of fragmentation.

Samsung SSD 850 Pro (back)

Due to the way data is written and read to the drive, however, SSD cells can wear out over time. SSD cells push electrons through a gate to set its state. This process wears on the cell and over time reduces its performance until the SSD wears out. This effect takes a long time and SSDs have mechanisms to minimize this effect, such as the TRIM command. Flash memory writes an entire block of storage no matter how few pages within the block are updated. This requires reading and caching the existing data, erasing the block and rewriting the block. If an empty block is available, a write operation is much faster. The TRIM command, which must be supported in both the OS and the SSD, enables the OS to inform the drive which blocks are no longer needed. It allows the drive to erase the blocks ahead of time in order to make empty blocks available for subsequent writes.

The effect of repeated reading and erasing on an SSD is cumulative and an SSD can slow down and even display errors with age. It’s more likely, however, that the system using the SSD will be discarded for obsolescence before the SSD begins to display read/write errors. Hard drives eventually wear out from constant use as well, since they use physical recording methods, so most users won’t base their selection of an HDD or SSD drive based on expected longevity.

SSD circuit board

Overall, SSDs are considered far more durable than HDDs due to a lack of mechanical parts. The moving mechanisms within an HDD are susceptible to not only wear and tear over time, but to damage due to movement or forceful contact. If one were to drop a laptop with an HDD, there is a high likelihood that all those moving parts will collide, resulting in potential data loss and even destructive physical damage that could kill the HDD outright. SSDs have no moving parts so, while they hold the risk of a potentially shorter life span due to high use, they can survive the rigors we impose upon our portable devices and laptops.

What are the best uses for SSDs?

Notebooks, laptops, where performance, lightweight, areal storage density, resistance to shock and general ruggedness are desirable
Boot drives holding operating system and applications, which will speed up booting and application launching
Working files (media that is being edited: photos, video, audio, etc.)
Swap drives where SSD will speed up disk paging
Cache drives
Database servers
Revitalizing an older computer. If you’ve got a computer that seems slow to start up and slow to load applications and files, updating the boot drive with an SSD could make it seem, if not new, at least as if it just came back refreshed from spending some time on the beach.

Stay Tuned for Part 2 of HDD vs SSD

That’s it for part 1. In our second part we’ll take a deeper look at the differences between HDDs and SSDs, how both HDD and SSD technologies are evolving, and how Backblaze takes advantage of SSDs in our operations and data centers.

Here’s a tip on finding all the posts tagged with SSD on our blog. Just follow https://www.backblaze.com/blog/tag/ssd/.

Don’t miss future posts on HDDs, SSDs, and other topics, including hard drive stats, cloud storage, and tips and tricks for backing up to the cloud. Use the Join button above to receive notification of future posts on our blog.

The post HDD vs SSD: What Does the Future for Storage Hold? appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Setting up bug bounties for success

2018-03-03 Michal Zalewski

Post Syndicated from Michal Zalewski original https://lcamtuf.blogspot.com/2018/03/setting-up-bug-bounties-for-success.html

Bug bounties end up in the news with some regularity, usually for the wrong reasons. I’ve been itching to write
about that for a while – but instead of dwelling on the mistakes of the bygone days, I figured it may be better to
talk about some of the ways to get vulnerability rewards right.

What do you get out of bug bounties?

There’s plenty of differing views, but I like to think of such programs
simply as a bid on researchers’ time. In the most basic sense, you get three benefits:

Improved ability to detect bugs in production before they become major incidents.
A comparatively unbiased feedback loop to help you prioritize and measure other security work.
A robust talent pipeline for when you need to hire.

What bug bounties don’t offer?

You don’t get anything resembling a comprehensive security program or a systematic assessment of your platforms.
Researchers end up looking for bugs that offer favorable effort-to-payoff ratios for their skills and given the
very imperfect information they have about your enterprise. In other words, you may end up with a hundred
people looking for XSS and just one person looking for RCE.

Your reward structure can steer them toward the targets and bugs you care about, but it’s difficult to fully
eliminate this inherent skew. There’s only so far you can jack up your top-tier rewards, and only so far you can
go lowering the bottom-tier ones.

Don’t you have to outcompete the black market to get all the “good” bugs?

There is a free market price discovery component to it all: if you’re not getting the engagement you
were hoping for, you should probably consider paying more.

That said, there are going to be researchers who’d rather hurt you than work for you, no matter how much you pay;
you don’t have to win them over, and you don’t have to outspend every authoritarian government or
every crime syndicate. A bug bounty is effective simply if it attracts enough eyeballs to make bugs statistically
harder to find, and reduces the useful lifespan of any zero-days in black market trade. Plus, most
researchers don’t want their work to be used to crack down on dissidents in Egypt or Vietnam.

Another factor is that you’re paying for different things: a black market buyer probably wants a reliable exploit
capable of delivering payloads, and then demands silence for months or years to come; a vendor-run
bug bounty program is usually perfectly happy with a reproducible crash and doesn’t mind a researcher blogging
about their work.

In fact, while money is important, you will probably find out that it’s not enough to retain your top talent;
many folks want bug bounties to be more than a business transaction, and find a lot of value in having a close
relationship with your security team, comparing notes, and growing together. Fostering that partnership can
be more important than adding another $10,000 to your top reward.

How do I prevent it all from going horribly wrong?

Bug bounties are an unfamiliar beast to most lawyers and PR folks, so it’s a natural to be wary and try to plan
for every eventuality with pages and pages of impenetrable rules and fine-print legalese.

This is generally unnecessary: there is a strong self-selection bias, and almost every participant in a
vulnerability reward program will be coming to you in good faith. The more friendly, forthcoming, and
approachable you seem, and the more you treat them like peers, the more likely it is for your relationship to stay
positive. On the flip side, there is no faster way to make enemies than to make a security researcher feel that they
are now talking to a lawyer or to the PR dept.

Most people have strong opinions on disclosure policies; instead of imposing your own views, strive to patch reported bugs
reasonably quickly, and almost every reporter will play along. Demand researchers to cancel conference appearances,
take down blog posts, or sign NDAs, and you will sooner or later end up in the news.

But what if that’s not enough?

As with any business endeavor, mistakes will happen; total risk avoidance is seldom the answer. Learn to sincerely
apologize for mishaps; it’s not a sign of weakness to say “sorry, we messed up”. And you will almost certainly not end
up in the courtroom for doing so.

It’s good to foster a healthy and productive relationship with the community, so that they come to your defense when
something goes wrong. Encouraging people to disclose bugs and talk about their experiences is one way of accomplishing that.

What about extortion?

You should structure your program to naturally discourage bad behavior and make it stand out like a sore thumb.
Require bona fide reports with complete technical details before any reward decision is made by a panel of named peers;
and make it clear that you never demand non-disclosure as a condition of getting a reward.

To avoid researchers accidentally putting themselves in awkward situations, have clear rules around data exfiltration
and lateral movement: assure them that you will always pay based on the worst-case impact of their findings; in exchange,
ask them to stop as soon as they get a shell and never access any data that isn’t their own.

So… are there any downsides?

Yep. Other than souring up your relationship with the community if you implement your program wrong, the other consideration
is that bug bounties tend to generate a lot of noise from well-meaning but less-skilled researchers.

When this happens, do not get frustrated and do not penalize such participants; instead, help them grow. Consider
publishing educational articles, giving advice on how to investigate and structure reports, or
offering free workshops every now and then.

The other downside is cost; although bug bounties tend to offer far more bang for your buck than your average penetration
test, they are more random. The annual expenses tend to be fairly predictable, but there is always
some possibility of having to pay multiple top-tier rewards in rapid succession. This is the kind of uncertainty that
many mid-level budget planners react badly to.

Finally, you need to be able to fix the bugs you receive. It would be nuts to prefer to not know about the
vulnerabilities in the first place – but once you invite the research, the clock starts ticking and you need to
ship fixes reasonably fast.

So… should I try it?

There are folks who enthusiastically advocate for bug bounties in every conceivable situation, and people who dislike them
with fierce passion; both sentiments are usually strongly correlated with the line of business they are in.

In reality, bug bounties are not a cure-all, and there are some ways to make them ineffectual or even dangerous.
But they are not as risky or expensive as most people suspect, and when done right, they can actually be fun for your
team, too. You won’t know for sure until you try.

AWS Federated Authentication with Active Directory Federation Services (AD FS)

2018-03-03 Kevin Higgins

Post Syndicated from Kevin Higgins original https://aws.amazon.com/blogs/security/aws-federated-authentication-with-active-directory-federation-services-ad-fs/

Today we’d like to walk you through AWS Identity and Access Management (IAM), federated sign-in through Active Directory (AD) and Active Directory Federation Services (ADFS). With IAM, you can centrally manage users, security credentials such as access keys, and permissions that control which resources users can access. Customers have the option of creating users and group objects within IAM or they can utilize a third-party federation service to assign external directory users access to AWS resources. To streamline the administration of user access in AWS, organizations can utilize a federated solution with an external directory, allowing them to minimize administrative overhead. Benefits of this approach include leveraging existing passwords and password policies, roles and groups. This guide provides a walk-through on how to automate the federation setup across multiple accounts/roles with an Active Directory backing identity store. This will establish the minimum baseline for the authentication architecture, including the initial IdP deployment and elements for federation.

ADFS Federated Authentication Process

The following describes the process a user will follow to authenticate to AWS using Active Directory and ADFS as the identity provider and identity brokers:

Corporate user accesses the corporate Active Directory Federation Services portal sign-in page and provides Active Directory authentication credentials.
AD FS authenticates the user against Active Directory.
Active Directory returns the user’s information, including AD group membership information.
AD FS dynamically builds ARNs by using Active Directory group memberships for the IAM roles and user attributes for the AWS account IDs, and sends a signed assertion to the users browser with a redirect to post the assertion to AWS STS.
Temporary credentials are returned using STS AssumeRoleWithSAML.
The user is authenticated and provided access to the AWS management console.

Configuration Steps

Configuration requires setup in the Identity Provider store (e.g. Active Directory), the identity broker (e.g. Active Directory Federation Services), and AWS. It is possible to configure AWS to federate authentication using a variety of third-party SAML 2.0 compliant identity providers, more information can be found here.

AWS Configuration

The configuration steps outlined in this document can be completed to enable federated access to multiple AWS accounts, facilitating a single sign on process across a multi-account AWS environment. Access can also be provided to multiple roles in each AWS account. The roles available to a user are based on their group memberships in the identity provider (IdP). In a multi-role and/or multi-account scenario, role assumption requires the user to select the account and role they wish to assume during the authentication process.

Identity Provider

A SAML 2.0 identity provider is an IAM resource that describes an identity provider (IdP) service that supports the SAML 2.0 (Security Assertion Markup Language 2.0) standard. AWS SAML identity provider configurations can be used to establish trust between AWS and SAML-compatible identity providers, such as Shibboleth or Microsoft Active Directory Federation Services. These enable users in an organization to access AWS resources using existing credentials from the identity provider.

A SAML identify provider can be configured using the AWS console by completing the following steps.

1. Access the “Identity Providers” section of the AWS IAM console at the following URL: https://console.aws.amazon.com/iam/home?region=us-east-1#/providers. Click on the “Create Provider” button.

2. Select SAML for the provider type. Select a provider name of your choosing (this will become the logical name used in the identity provider ARN). Lastly, download the FederationMetadata.xml file from your ADFS server to your client system file (https://yourADFSserverFQDN/FederationMetadata/2007-06/FederationMetadata.xml). Click “Choose File” and upload it to AWS.

3. Click “Next Step” and then verify the information you have entered. Click “Create” to complete the AWS identity provider configuration process.

IAM Role Naming Convention for User Access
Once the AWS identity provider configuration is complete, it is necessary to create the roles in AWS that federated users can assume via SAML 2.0. An IAM role is an AWS identity with permission policies that determine what the identity can and cannot do in AWS. In a federated authentication scenario, users (as defined in the IdP) assume an AWS role during the sign-in process. A role should be defined for each access delineation that you wish to define. For example, create a role for each line of business (LOB), or each function within a LOB. Each role will then be assigned a set of policies that define what privileges the users who will be assuming that role will have.

The following steps detail how to create a single role. These steps should be completed multiple times to enable assumption of different roles within AWS, as required.

1. Access the “Roles” section of the AWS IAM console at the following URL: https://console.aws.amazon.com/iam/home?region=us-east-1#/roles. Click on the “Create Role” button.

2. Select “SAML” as the trusted entity type. Click Next Step.

3. Select your previously created identity provider. Click Next: Permissions.

4. The next step requires selection of policies that represent the desired permissions the user should obtain in AWS, once they have authenticated and successfully assumed the role. This can be either a custom policy or preferably an AWS managed policy. AWS recommends leveraging existing AWS access policies for job functions for common levels of access. For example, the “Billing” AWS Managed policy should be utilized to provide financial analyst access to AWS billing and cost information.

5. Provide a name for your role. All roles should be created with the prefix ADFS-<rolename> to simplify the identification of roles in AWS that are accessed through the federated authentication process. Next click, “Create Role”.

Active Directory Configuration

Determining how you will create and delineate your AD groups and IAM roles in AWS is crucial to how you secure access to your account and manage resources. SAML assertions to the AWS environment and the respective IAM role access will be managed through regular expression (regex) matching between your on-premises AD group name to an AWS IAM role.

One approach for creating the AD groups that uniquely identify the AWS IAM role mapping is by selecting a common group naming convention. For example, your AD groups would start with an identifier, for example AWS-, as this will distinguish your AWS groups from others within the organization. Next, include the 12-digit AWS account number. Finally, add the matching role name within the AWS account. Here is an example:

You should do this for each role and corresponding AWS account you wish to support with federated access. Users in Active Directory can subsequently be added to the groups, providing the ability to assume access to the corresponding roles in AWS. If a user is associated with multiple Active Directory groups and AWS accounts, they will see a list of roles by AWS account and will have the option to choose which role to assume. A user will not be able to assume more than one role at a time, but has the ability to switch between them as needed.

Note: Microsoft imposes a limit on the number of groups a user can be a member of (approximately 1,015 groups) due to the size limit for the access token that is created for each security principal. This limitation, however, is not affected by how the groups may or may not be nested.

Active Directory Federation Services Configuration

ADFS federation occurs with the participation of two parties; the identity or claims provider (in this case the owner of the identity repository – Active Directory) and the relying party, which is another application that wishes to outsource authentication to the identity provider; in this case Amazon Secure Token Service (STS). The relying party is a federation partner that is represented by a claims provider trust in the federation service.

Relying Party

You can configure a new relying party in Active Directory Federation Services by doing the following.

1. From the ADFS Management Console, right-click ADFS and select Add Relying Party Trust.

2. In the Add Relying Party Trust Wizard, click Start.

3. Check Import data about the relying party published online or on a local network, enter
https://signin.aws.amazon.com/static/saml-metadata.xml, and then click Next. The metadata XML file is a standard SAML metadata document that describes AWS as a relying party.

Note: SAML federations use metadata documents to maintain information about the public keys and certificates that each party utilizes. At run time, each member of the federation can then use this information to validate that the cryptographic elements of the distributed transactions come from the expected actors and haven’t been tampered with. Since these metadata documents do not contain any sensitive cryptographic material, AWS publishes federation metadata at https://signin.aws.amazon.com/static/saml-metadata.xml

4. Set the display name for the relying party and then click Next.

5. We will not choose to enable/configure the MFA settings at this time.

6. Select “Permit all users to access this relying party” and click Next.

7. Review your settings and then click Next.

8. Choose Close on the Finish page to complete the Add Relying Party Trust Wizard. AWS is now configured as a relying party.

Custom Claim Rules

Microsoft Active Directory Federation Services (AD FS) uses Claims Rule Language to issue and transform claims between claims providers and relying parties. A claim is information about a user from a trusted source. The trusted source is asserting that the information is true, and that source has authenticated the user in some manner. The claims provider is the source of the claim. This can be information pulled from an attribute store such as Active Directory (AD). The relying party is the destination for the claims, in this case AWS.

AD FS provides administrators with the option to define custom rules that they can use to determine the behavior of identity claims with the claim rule language. The Active Directory Federation Services (AD FS) claim rule language acts as the administrative building block to help manage the behavior of incoming and outgoing claims. There are four claim rules that need to be created to effectively enable Active Directory users to assume roles in AWS based on group membership in Active Directory.

Right-click on the relying party (in this case Amazon Web Services) and then click Edit Claim Rules

Here are the steps used to create the claim rules for NameId, RoleSessionName, Get AD Groups and Roles.

1. NameId

a) In the Edit Claim Rules for <relying party> dialog box, click Add Rule.
b) Select Transform an Incoming Claim and then click Next.
c) Use the following settings:

i) Claim rule name: NameId
ii) Incoming claim type: Windows Account Name
iii) Outgoing claim type: Name ID
iv) Outgoing name ID format: Persistent Identifier
v) Pass through all claim values: checked

d) Click OK.

Rule language:
c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname"] => issue(Type = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier", Issuer = c.Issuer, OriginalIssuer = c.OriginalIssuer, Value = c.Value, ValueType = c.ValueType, Properties["http://schemas.xmlsoap.org/ws/2005/05/identity/claimproperties/format"] = "urn:oasis:names:tc:SAML:2.0:nameid-format:persistent");

2. RoleSessionName

a) Click Add Rule
b) In the Claim rule template list, select Send LDAP Attributes as Claims.
c) Use the following settings:

i) Claim rule name: RoleSessionName
ii) Attribute store: Active Directory
iii) LDAP Attribute: E-Mail-Addresses
iv) Outgoing Claim Type: https://aws.amazon.com/SAML/Attributes/RoleSessionName

d) Click OK

Rule language:
c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname", Issuer == "AD AUTHORITY"] => issue(store = "Active Directory", types = ("https://aws.amazon.com/SAML/Attributes/RoleSessionName"), query = ";mail;{0}", param = c.Value);

3. Get AD Groups

a) Click Add Rule.
b) In the Claim rule template list, select Send Claims Using a Custom Rule and then click Next.
c) For Claim Rule Name, select Get AD Groups, and then in Custom rule, enter the following:

Rule language:
c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname", Issuer == "AD AUTHORITY"] => add(store = "Active Directory", types = ("http://temp/variable"), query = ";tokenGroups;{0}", param = c.Value);

This custom rule uses a script in the claim rule language that retrieves all the groups the authenticated user is a member of and places them into a temporary claim named http://temp/variable. Think of this as a variable you can access later.

Note: Ensure there’s no trailing whitespace to avoid unexpected results.

4. Role Attributes

a) Unlike the two previous claims, here we used custom rules to send role attributes. This is done by retrieving all the authenticated user’s AD groups and then matching the groups that start with to IAM roles of a similar name. I used the names of these groups to create Amazon Resource Names (ARNs) of IAM roles in my AWS account (i.e., those that start with AWS-). Sending role attributes requires two custom rules. The first rule retrieves all the authenticated user’s AD group memberships and the second rule performs the transformation to the roles claim.

i) Click Add Rule.
ii) In the Claim rule template list, select Send Claims Using a Custom Rule and then click Next.
iii) For Claim Rule Name, enter Roles, and then in Custom rule, enter the following:

Rule language:
c:[Type == "http://temp/variable", Value =~ "(?i)^AWS-([\d]{12})"] => issue(Type = "https://aws.amazon.com/SAML/Attributes/Role", Value = RegExReplace(c.Value, "AWS-([\d]{12})-", "arn:aws:iam::$1:saml-provider/idp1,arn:aws:iam::$1:role/"));

This custom rule uses regular expressions to transform each of the group memberships of the form AWS-<Account Number>-<Role Name> into in the IAM role ARN, IAM federation provider ARN form AWS expects.

Note: In the example rule language above idp1 represents the logical name given to the SAML identity provider in the AWS identity provider setup. Please change this based on the logical name you chose in the IAM console for your identity provider.

Adjusting Session Duration

By default, the temporary credentials that are issued by AWS IAM for SAML federation are valid for an hour. Depending on your organizations security stance, you may wish to adjust. You can allow your federated users to work in the AWS Management Console for up to 12 hours. This can be accomplished by adding another claim rule in your ADFS configuration. To add the rule, do the following:

1. Access ADFS Management Tool on your ADFS Server.
2. Choose Relying Party Trusts, then select your AWS Relying Party configuration.
3. Choose Edit Claim Rules.
4. Choose Add Rule to configure a new rule, and then choose Send claims using a custom rule. Finally, choose Next.
5. Name your Rule “Session Duration” and add the following rule syntax.
6. Adjust the value of 28800 seconds (8 hours) as appropriate.

Rule language:
=> issue(Type = "https://aws.amazon.com/SAML/Attributes/SessionDuration", Value = "28800");

Note: AD FS 2012 R2 and AD FS 2016 tokens have a sixty-minute validity period by default. This value is configurable on a per-relying party trust basis. In addition to adding the “Session Duration” claim rule, you will also need to update the security token created by AD FS. To update this value, run the following command:

Set-ADFSRelyingPartyTrust -TargetName “[Display Name]” -TokenLifetime 480

The Parameter “-TokenLifetime” determines the Lifetime in Minutes. In this example, we set the Lifetime to 480 minutes, eight hours.

These are the main settings related to session lifetimes and user authentication. Once updated, any new console session your federated users initiate will be valid for the duration specified in the SessionDuration claim.

API/CLI Access
Access to the AWS API and command-line tools using federated access can be accomplished using techniques in the following blog article:

https://blogs.aws.amazon.com/security/post/Tx1LDN0UBGJJ26Q/How-to-Implement-Federated-API-and-CLI-Access-Using-SAML-2-0-and-AD-FS

This will enable your users to access your AWS environment using their domain credentials through the AWS CLI or one of the AWS SDKs.

Conclusion
In this post, I’ve shown you how to provide identity federation, and thus SSO, to the AWS Management Console for multiple accounts using SAML assertions. With this approach, the AWS Security Token service (STS) will provide temporary credentials (via SAML) for the user to ‘assume’ a role (that they have access to use, as denoted by AD Group membership) that has specific permissions associated; as opposed to providing long-term access credentials to the AWS resources. By adopting this model, you will have a secure and robust IAM approach for accessing AWS resources that align with AWS security best practices.

Dotcom: Obama Admitted “Mistakes Were Made” in Megaupload Case

2018-03-01 Andy

Post Syndicated from Andy original https://torrentfreak.com/dotcom-obama-admitted-mistakes-were-made-in-megaupload-case-180301/

When Megaupload was forcefully shut down in 2012, it initially appeared like ‘just’ another wave of copyright enforcement action by US authorities.

When additional details began to filter through, the reality of what had happened was nothing short of extraordinary.

Not only were large numbers of Megaupload servers and millions of dollars seized, but Kim Dotcom’s home in New Zealand was subjected to a military-style raid comprised of helicopters and dozens of heavily armed special tactics police. The whole thing was monitored live by the FBI.

Few people who watched the events of that now-infamous January day unfold came to the conclusion this was a routine copyright-infringement case. According to Kim Dotcom, whose life had just been turned upside down, something of this scale must’ve filtered down from the very top of the US government. It was hard to disagree.

At the time, Dotcom told TorrentFreak that then-Vice President Joe Biden directed attorney Neil MacBride to target the cloud storage site and ever since the Megaupload founder has leveled increasingly serious allegations at officials of the former government of Barack Obama.

For example, Dotcom says that since the US would have difficulty gaining access to him in his former home of Hong Kong, the government of New Zealand was persuaded to welcome him in, knowing they would eventually turn him over to the United States. More recently he’s been turning up the pressure again, such as a tweet on February 20th which cast more light on that process.

“Joe Biden had a White House meeting with an ‘extradition expert’ who worked for Hong Kong police and a handful of Hollywood executives to discuss my case. A week prior to this meeting Neil MacBride hand-delivered his action plan to Biden’s chief of staff, also at the White House,” Dotcom wrote.

But this claim is just the tip of an extremely large iceberg that’s involved illegal spying on Dotcom in New Zealand and a dizzying array of legal battles that are set to go on for years to come. But perhaps of most interest now is that rather than wilting away under the pressure, Dotcom appears to be just warming up.

A few hours ago Dotcom commented on an article published in The Hill which revealed that Barack Obama will visit New Zealand in March, possibly to celebrate the opening of Air New Zealand’s new route to the U.S.

Rather than expressing disappointment, the Megaupload founder seemed pleased that the former president would be touching down next month.

“Great. I’ll have a Court subpoena waiting for him in New Zealand,” Dotcom wrote.

But that was just a mere hors d’oeuvre, with the main course was yet to come. But come it did.

“A wealthy Asian Megaupload shareholder hired a friend of the Obamas to enquire about our case. This person was recommended by a member of the Chinese politburo ‘if you want to get to Obama directly’. We did,” Dotcom revealed.

Dotcom says he’ll release a transcript detailing what Obama told his friend on March 21 when Obama arrives in town but in the meantime, he offered another little taster.

“Mistakes were made. It hasn’t gone well,” Obama reportedly told the person reporting back to Megaupload. “It’s a problem. I’ll see to it after the election.”

Of course, Obama’s position after the election was much different to what had gone before, but that didn’t stop Dotcom’s associates infiltrating the process aimed at keeping the Democrats in power.

“Our friendly Obama contact smuggled an @EFF lawyer into a re-election fundraiser hosted by former Vice President Joe Biden,” he revealed.

“When Biden was asked about the Megaupload case he bragged that it was his case and that he ‘took care of it’,” which is what Dotcom has been claiming all along.

On March 21, when Obama lands in New Zealand, Dotcom says he’ll be waiting.

“I’m looking forward to @BarackObama providing some insight into the political dimension of the Megaupload case when he arrives in the New Zealand jurisdiction,” he teased.

Better get the popcorn ready….

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

Italian Election: Last Week Tonight with John Oliver (HBO)

2018-02-26 LastWeekTonight

Post Syndicated from LastWeekTonight original https://www.youtube.com/watch?v=LdhQzXHYLZ4

Most Users of Exclusive Torrent Site Also Pay For Services Like Netflix or Prime

2018-02-25 Andy

Post Syndicated from Andy original https://torrentfreak.com/most-users-of-exclusive-torrent-site-also-pay-for-services-like-netflix-or-prime-180225/

Despite a notable move to unlicensed streaming portals, millions of people still use public torrent sites every day to obtain the latest movies and TV shows. The process is easy, relatively quick, and free.

While these open-to-all platforms are undoubtedly popular, others prefer to use so-called ‘private trackers’, torrent sites with a private members’ club feel. Barriers to entry are much higher and many now require either an invitation from someone who is already a member or the passing of what amounts to an entrance exam.

Once accepted as a member, however, the rewards can be great. While public sites are a bit of a free-for-all, private trackers tend to take control of the content on offer, weeding out poor quality releases and ensuring only the best reach the user. Seeders are also plentiful, meaning that downloads complete in the fastest times.

On the flipside, some of the most exclusive trackers are almost impossible to join. A prime example is HDBits, a site that at last count wouldn’t accept more than 21,000 users yet keeps actual memberships down to around the 18,000 mark. Invites are extremely rare and those already inside tend to guard their accounts with their lives.

Second chances are rare on a site indexing more than 234,000 high-quality releases seeded by more than 950,000 peers and one of the broadest selection of Blu-ray offerings around. That’s what makes the results of a survey currently being carried out on the site even more remarkable.

In a poll launched by site staff, HDBits members – who by definition are already part of one of the most exclusive pirate haunts around – were asked whether they also pay for legal streaming services such as Netflix, Hulu or Amazon Prime.

At the time of writing more than 5,300 members have responded, with a surprising 57% (3,036) stating that they do indeed subscribe to at least one legal streaming service. When questioned on usage, more than a quarter of respondents said they actually use the legal service MORE than they use HDBits, which for a site of that caliber is quite a revelation.

HDBits poll – 57% of pirates pay for legal services

Keeping in mind that the site is creeping towards a quarter of a million torrents and is almost impossible to get into, it’s perhaps no surprise that unscrupulous people with access to an invitation on the site are selling them (against the site’s wishes) for up to $350 each online.

Let that sink in. For access to a pirate service, people are being asked to pay the equivalent of three years’ worth of Netflix subscriptions. Yet of those that are already members, more than a quarter use their Netflix, Hulu or Amazon Prime account more than they do HDBits. That’s a huge feather in the cap for the legal platforms that have nowhere near the selection that HDBits does.

One commenter in the HDBits survey thread gave his opinion on why Netflix might be winning the war.

“A thread several years ago like this was why I bought Netflix stock. Stunned not just that people here would actually pay for streaming 1 year old content in poor quality, but that almost everyone seemed to be doing it. If Netflix can win over [HDBits] then it is clearly a solution that will win over everyone,” he wrote.

Of course, perhaps the most important thing here is that even the most hardcore pirates have no problem purchasing official content, when the environment is right.

Unlike other surveys that can scare people away from admitting they’re breaking the law, most people on HDBits have nothing to hide from their peers. They know they’re pirates and aren’t afraid to admit it, yet almost 60% of them are happy to pay for legal content on top.

Entertainment companies often like to put pirates in one box and legitimate customers in another. Once again it’s now being made clear that such neatly defined barriers aren’t easy to come by.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

Election Security

2018-02-23 Bruce Schneier

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2018/02/election_securi_2.html

I joined a letter supporting the Secure Elections Act (S. 2261):

The Secure Elections Act strikes a careful balance between state and federal action to secure American voting systems. The measure authorizes appropriation of grants to the states to take important and time-sensitive actions, including:

Replacing insecure paperless voting systems with new equipment that will process a paper ballot;
Implementing post-election audits of paper ballots or records to verify electronic tallies;

Conducting “cyber hygiene” scans and “risk and vulnerability” assessments and supporting state efforts to remediate identified vulnerabilities.

The legislation would also create needed transparency and accountability in elections systems by establishing clear protocols for state and federal officials to communicate regarding security breaches and emerging threats.

The Challenges of Opening a Data Center — Part 1

2018-02-23 Roderick Bauer

Post Syndicated from Roderick Bauer original https://www.backblaze.com/blog/choosing-data-center/

This is part one of a series. The second part will be posted later this week. Use the Join button above to receive notification of future posts in this series.

Though most of us have never set foot inside of a data center, as citizens of a data-driven world we nonetheless depend on the services that data centers provide almost as much as we depend on a reliable water supply, the electrical grid, and the highway system. Every time we send a tweet, post to Facebook, check our bank balance or credit score, watch a YouTube video, or back up a computer to the cloud we are interacting with a data center.

In this series, The Challenges of Opening a Data Center, we’ll talk in general terms about the factors that an organization needs to consider when opening a data center and the challenges that must be met in the process. Many of the factors to consider will be similar for opening a private data center or seeking space in a public data center, but we’ll assume for the sake of this discussion that our needs are more modest than requiring a data center dedicated solely to our own use (i.e. we’re not Google, Facebook, or China Telecom).

Data center technology and management are changing rapidly, with new approaches to design and operation appearing every year. This means we won’t be able to cover everything happening in the world of data centers in our series, however, we hope our brief overview proves useful.

What is a Data Center?

A data center is the structure that houses a large group of networked computer servers typically used by businesses, governments, and organizations for the remote storage, processing, or distribution of large amounts of data.

While many organizations will have computing services in the same location as their offices that support their day-to-day operations, a data center is a structure dedicated to 24/7 large-scale data processing and handling.

Depending on how you define the term, there are anywhere from a half million data centers in the world to many millions. While it’s possible to say that an organization’s on-site servers and data storage can be called a data center, in this discussion we are using the term data center to refer to facilities that are expressly dedicated to housing computer systems and associated components, such as telecommunications and storage systems. The facility might be a private center, which is owned or leased by one tenant only, or a shared data center that offers what are called “colocation services,” and rents space, services, and equipment to multiple tenants in the center.

A large, modern data center operates around the clock, placing a priority on providing secure and uninterrrupted service, and generally includes redundant or backup power systems or supplies, redundant data communication connections, environmental controls, fire suppression systems, and numerous security devices. Such a center is an industrial-scale operation often using as much electricity as a small town.

Types of Data Centers

There are a number of ways to classify data centers according to how they will be used, whether they are owned or used by one or multiple organizations, whether and how they fit into a topology of other data centers; which technologies and management approaches they use for computing, storage, cooling, power, and operations; and increasingly visible these days: how green they are.

Data centers can be loosely classified into three types according to who owns them and who uses them.

Exclusive Data Centers are facilities wholly built, maintained, operated and managed by the business for the optimal operation of its IT equipment. Some of these centers are well-known companies such as Facebook, Google, or Microsoft, while others are less public-facing big telecoms, insurance companies, or other service providers.

Managed Hosting Providers are data centers managed by a third party on behalf of a business. The business does not own data center or space within it. Rather, the business rents IT equipment and infrastructure it needs instead of investing in the outright purchase of what it needs.

Colocation Data Centers are usually large facilities built to accommodate multiple businesses within the center. The business rents its own space within the data center and subsequently fills the space with its IT equipment, or possibly uses equipment provided by the data center operator.

Backblaze, for example, doesn’t own its own data centers but colocates in data centers owned by others. As Backblaze’s storage needs grow, Backblaze increases the space it uses within a given data center and/or expands to other data centers in the same or different geographic areas.

Availability is Key

When designing or selecting a data center, an organization needs to decide what level of availability is required for its services. The type of business or service it provides likely will dictate this. Any organization that provides real-time and/or critical data services will need the highest level of availability and redundancy, as well as the ability to rapidly failover (transfer operation to another center) when and if required. Some organizations require multiple data centers not just to handle the computer or storage capacity they use, but to provide alternate locations for operation if something should happen temporarily or permanently to one or more of their centers.

Organizations operating data centers that can’t afford any downtime at all will typically operate data centers that have a mirrored site that can take over if something happens to the first site, or they operate a second site in parallel to the first one. These data center topologies are called Active/Passive, and Active/Active, respectively. Should disaster or an outage occur, disaster mode would dictate immediately moving all of the primary data center’s processing to the second data center.

While some data center topologies are spread throughout a single country or continent, others extend around the world. Practically, data transmission speeds put a cap on centers that can be operated in parallel with the appearance of simultaneous operation. Linking two data centers located apart from each other — say no more than 60 miles to limit data latency issues — together with dark fiber (leased fiber optic cable) could enable both data centers to be operated as if they were in the same location, reducing staffing requirements yet providing immediate failover to the secondary data center if needed.

This redundancy of facilities and ensured availability is of paramount importance to those needing uninterrupted data center services.

LEED Certification

Leadership in Energy and Environmental Design (LEED) is a rating system devised by the United States Green Building Council (USGBC) for the design, construction, and operation of green buildings. Facilities can achieve ratings of certified, silver, gold, or platinum based on criteria within six categories: sustainable sites, water efficiency, energy and atmosphere, materials and resources, indoor environmental quality, and innovation and design.

Green certification has become increasingly important in data center design and operation as data centers require great amounts of electricity and often cooling water to operate. Green technologies can reduce costs for data center operation, as well as make the arrival of data centers more amenable to environmentally-conscious communities.

The ACT, Inc. data center in Iowa City, Iowa was the first data center in the U.S. to receive LEED-Platinum certification, the highest level available.

ACT Data Center exterior

ACT Data Center interior

Factors to Consider When Selecting a Data Center

There are numerous factors to consider when deciding to build or to occupy space in a data center. Aspects such as proximity to available power grids, telecommunications infrastructure, networking services, transportation lines, and emergency services can affect costs, risk, security and other factors that need to be taken into consideration.

The size of the data center will be dictated by the business requirements of the owner or tenant. A data center can occupy one room of a building, one or more floors, or an entire building. Most of the equipment is often in the form of servers mounted in 19 inch rack cabinets, which are usually placed in single rows forming corridors (so-called aisles) between them. This allows staff access to the front and rear of each cabinet. Servers differ greatly in size from 1U servers (i.e. one “U” or “RU” rack unit measuring 44.50 millimeters or 1.75 inches), to Backblaze’s Storage Pod design that fits a 4U chassis, to large freestanding storage silos that occupy many square feet of floor space.

Location

Location will be one of the biggest factors to consider when selecting a data center and encompasses many other factors that should be taken into account, such as geological risks, neighboring uses, and even local flight paths. Access to suitable available power at a suitable price point is often the most critical factor and the longest lead time item, followed by broadband service availability.

With more and more data centers available providing varied levels of service and cost, the choices increase each year. Data center brokers can be employed to find a data center, just as one might use a broker for home or other commercial real estate.

Websites listing available colocation space, such as upstack.io, or entire data centers for sale or lease, are widely used. A common practice is for a customer to publish its data center requirements, and the vendors compete to provide the most attractive bid in a reverse auction.

Business and Customer Proximity

The center’s closeness to a business or organization may or may not be a factor in the site selection. The organization might wish to be close enough to manage the center or supervise the on-site staff from a nearby business location. The location of customers might be a factor, especially if data transmission speeds and latency are important, or the business or customers have regulatory, political, tax, or other considerations that dictate areas suitable or not suitable for the storage and processing of data.

Climate

Local climate is a major factor in data center design because the climatic conditions dictate what cooling technologies should be deployed. In turn this impacts uptime and the costs associated with cooling, which can total as much as 50% or more of a center’s power costs. The topology and the cost of managing a data center in a warm, humid climate will vary greatly from managing one in a cool, dry climate. Nevertheless, data centers are located in both extremely cold regions and extremely hot ones, with innovative approaches used in both extremes to maintain desired temperatures within the center.

Geographic Stability and Extreme Weather Events

A major obvious factor in locating a data center is the stability of the actual site as regards weather, seismic activity, and the likelihood of weather events such as hurricanes, as well as fire or flooding.

Backblaze’s Sacramento data center describes its location as one of the most stable geographic locations in California, outside fault zones and floodplains.

Sometimes the location of the center comes first and the facility is hardened to withstand anticipated threats, such as Equinix’s NAP of the Americas data center in Miami, one of the largest single-building data centers on the planet (six stories and 750,000 square feet), which is built 32 feet above sea level and designed to withstand category 5 hurricane winds.

Equinix “NAP of the Americas” Data Center in Miami

Most data centers don’t have the extreme protection or history of the Bahnhof data center, which is located inside the ultra-secure former nuclear bunker Pionen, in Stockholm, Sweden. It is buried 100 feet below ground inside the White Mountains and secured behind 15.7 in. thick metal doors. It prides itself on its self-described “Bond villain” ambiance.

Bahnhof Data Center under White Mountain in Stockholm

Usually, the data center owner or tenant will want to take into account the balance between cost and risk in the selection of a location. The Ideal quadrant below is obviously favored when making this compromise.

Cost = Construction/lease, power, bandwidth, cooling, labor, taxes
Risk = Environmental (seismic, weather, water, fire), political, economic

Risk mitigation also plays a strong role in pricing. The extent to which providers must implement special building techniques and operating technologies to protect the facility will affect price. When selecting a data center, organizations must make note of the data center’s certification level on the basis of regulatory requirements in the industry. These certifications can ensure that an organization is meeting necessary compliance requirements.

Power

Electrical power usually represents the largest cost in a data center. The cost a service provider pays for power will be affected by the source of the power, the regulatory environment, the facility size and the rate concessions, if any, offered by the utility. At higher level tiers, battery, generator, and redundant power grids are a required part of the picture.

Fault tolerance and power redundancy are absolutely necessary to maintain uninterrupted data center operation. Parallel redundancy is a safeguard to ensure that an uninterruptible power supply (UPS) system is in place to provide electrical power if necessary. The UPS system can be based on batteries, saved kinetic energy, or some type of generator using diesel or another fuel. The center will operate on the UPS system with another UPS system acting as a backup power generator. If a power outage occurs, the additional UPS system power generator is available.

Many data centers require the use of independent power grids, with service provided by different utility companies or services, to prevent against loss of electrical service no matter what the cause. Some data centers have intentionally located themselves near national borders so that they can obtain redundant power from not just separate grids, but from separate geopolitical sources.

Higher redundancy levels required by a company will of invariably lead to higher prices. If one requires high availability backed by a service-level agreement (SLA), one can expect to pay more than another company with less demanding redundancy requirements.

Stay Tuned for Part 2 of The Challenges of Opening a Data Center

That’s it for part 1 of this post. In subsequent posts, we’ll take a look at some other factors to consider when moving into a data center such as network bandwidth, cooling, and security. We’ll take a look at what is involved in moving into a new data center (including stories from Backblaze’s experiences). We’ll also investigate what it takes to keep a data center running, and some of the new technologies and trends affecting data center design and use. You can discover all posts on our blog tagged with “Data Center” by following the link https://www.backblaze.com/blog/tag/data-center/.

The second part of this series on The Challenges of Opening a Data Center will be posted later this week. Use the Join button above to receive notification of future posts in this series.

The post The Challenges of Opening a Data Center — Part 1 appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

MagPi 67: back to the future with retro computing on your Pi

2018-02-22 Rob Zwetsloot

Post Syndicated from Rob Zwetsloot original https://www.raspberrypi.org/blog/magpi-67/

Hey folks, Rob from The MagPi here! While we do love modern computers here at The MagPi, we also have a soft spot for the classic machines of yesteryear, which is why we have a huge feature on emulating and upcycling retro computers in The MagPi issue 67, out right now.

The MagPi 67 Retro Gaming Privacy Security

Retro computing and security in the latest issue of The MagPi

Retro computing

Noted retro computing enthusiast K.G. Orphanides takes you through using the Raspberry Pi to emulate these classic machines, listing the best emulators out there and some of the homebrew software people have created for them. There’s even a guide on how to put a Pi in a Speccy!

Retro fun for all

While I’m a bit too young to have had a Commodore 64 or a Spectrum, there are plenty of folks who read the mag with nostalgia for that age of computing. And it’s also important for us young’uns to know the history of our hobby. So get ready to dive in!

Security and more

We also have an in-depth article about improving your security and privacy online and on your Raspberry Pi, and about using your Pi to increase your network security. It’s an important topic, and one that I’m pretty passionate about, so hopefully you’ll find the piece useful!

The new issue also includes our usual selection of inspiring projects, informative guides, and definitive reviews, as well as a free DVD with the latest version of the Raspberry Pi Desktop for Windows and Apple PCs!

Get The MagPi 67

Issue 67 is available today from WHSmith, Tesco, Sainsbury’s, and Asda. If you live in the US, head over to your local Barnes & Noble or Micro Center in the next few days for a print copy. You can also get the new issue online from our store, or digitally via our Android and iOS apps. And don’t forget, there’s always the free PDF as well.

New subscription offer!

You can also take out a twelve-month print subscription and get a Pi Zero W, Pi Zero case, and adapter cables absolutely free! This offer does not currently have an end date.

We hope you enjoy this issue! See you next time…

The post MagPi 67: back to the future with retro computing on your Pi appeared first on Raspberry Pi.

Solution Overview

Using PrivateLink to Establish a Service for Consumption

Step 1: Build a VPC and private subnets

Step 2: Create a Network Load Balancer, Listener, and Target Group

Step 3: Create a Launch Configuration and Auto Scaling Groups

Step 4: Launch an endpoint service and connect with NLB

Step 5: Tie endpoint request approval with billing system or the AWS Marketplace

Step 6: Take the Service Across Regions

Consuming a Service via PrivateLink

Access Endpoint Via Custom DNS Names

Conclusion

Super fighting robots

More to read

Get The MagPi 68

New subscription offer!

Wind Integration National Dataset

Developing a scalable service for big geospatial data

Public access to atmospheric data using HSDS and AWS

Ongoing work and other resources

Additional Reading

About the Authors

Compatibility

Raspbian with different screen resolutions

Pixel doubling

How to update

The Raspberry Jam photo booth

The great Raspberry Jam bake-off

Raspberry Jams everywhere

Europe

Africa

North America

South America

Asia

Oceania

The Jam makers!

HDDs vs SSDs

What Are HDDs?

What Are SSDs?

Stay Tuned for Part 2 of HDD vs SSD

What do you get out of bug bounties?

What bug bounties don’t offer?

Don’t you have to outcompete the black market to get all the “good” bugs?

How do I prevent it all from going horribly wrong?

But what if that’s not enough?

What about extortion?

So… are there any downsides?

So… should I try it?

What is a Data Center?

Types of Data Centers

Availability is Key

LEED Certification

Factors to Consider When Selecting a Data Center

Location

Business and Customer Proximity

Climate

Geographic Stability and Extreme Weather Events

Power

Stay Tuned for Part 2 of The Challenges of Opening a Data Center

Retro computing

Security and more

Get The MagPi 67

New subscription offer!

The collective thoughts of the interwebz