Tag Archives: things

What’s the Diff: Programs, Processes, and Threads

Post Syndicated from Roderick Bauer original https://www.backblaze.com/blog/whats-the-diff-programs-processes-and-threads/

let's talk about Threads

How often have you heard the term threading in relation to a computer program, but you weren’t exactly sure what it meant? How about processes? You likely understand that a thread is somehow closely related to a program and a process, but if you’re not a computer science major, maybe that’s as far as your understanding goes.

Knowing what these terms mean is absolutely essential if you are a programmer, but an understanding of them also can be useful to the average computer user. Being able to look at and understand the Activity Monitor on the Macintosh, the Task Manager on Windows, or Top on Linux can help you troubleshoot which programs are causing problems on your computer, or whether you might need to install more memory to make your system run better.

Let’s take a few minutes to delve into the world of computer programs and sort out what these terms mean. We’ll simplify and generalize some of the ideas, but the general concepts we cover should help clarify the difference between the terms.

Programs

First of all, you probably are aware that a program is the code that is stored on your computer that is intended to fulfill a certain task. There are many types of programs, including programs that help your computer function and are part of the operating system, and other programs that fulfill a particular job. These task-specific programs are also known as “applications,” and can include programs such as word processing, web browsing, or emailing a message to another computer.

Program

Programs are typically stored on disk or in non-volatile memory in a form that can be executed by your computer. Prior to that, they are created using a programming language such as C, Lisp, Pascal, or many others using instructions that involve logic, data and device manipulation, recurrence, and user interaction. The end result is a text file of code that is compiled into binary form (1’s and 0’s) in order to run on the computer. Another type of program is called “interpreted,” and instead of being compiled in advance in order to run, is interpreted into executable code at the time it is run. Some common, typically interpreted programming languages, are Python, PHP, JavaScript, and Ruby.

The end result is the same, however, in that when a program is run, it is loaded into memory in binary form. The computer’s CPU (Central Processing Unit) understands only binary instructions, so that’s the form the program needs to be in when it runs.

Perhaps you’ve heard the programmer’s joke, “There are only 10 types of people in the world, those who understand binary, and those who don’t.”

Binary is the native language of computers because an electrical circuit at its basic level has two states, on or off, represented by a one or a zero. In the common numbering system we use every day, base 10, each digit position can be anything from 0 to 9. In base 2 (or binary), each position is either a 0 or a 1. (In a future blog post we might cover quantum computing, which goes beyond the concept of just 1’s and 0’s in computing.)

Decimal—Base 10 Binary—Base 2
0 0000
1 0001
2 0010
3 0011
4 0100
5 0101
6 0110
7 0111
8 1000
9 1001

How Processes Work

The program has been loaded into the computer’s memory in binary form. Now what?

An executing program needs more than just the binary code that tells the computer what to do. The program needs memory and various operating system resources that it needs in order to run. A “process” is what we call a program that has been loaded into memory along with all the resources it needs to operate. The “operating system” is the brains behind allocating all these resources, and comes in different flavors such as macOS, iOS, Microsoft Windows, Linux, and Android. The OS handles the task of managing the resources needed to turn your program into a running process.

Some essential resources every process needs are registers, a program counter, and a stack. The “registers” are data holding places that are part of the computer processor (CPU). A register may hold an instruction, a storage address, or other kind of data needed by the process. The “program counter,” also called the “instruction pointer,” keeps track of where a computer is in its program sequence. The “stack” is a data structure that stores information about the active subroutines of a computer program and is used as scratch space for the process. It is distinguished from dynamically allocated memory for the process that is known as “the heap.”

diagram of how processes work

There can be multiple instances of a single program, and each instance of that running program is a process. Each process has a separate memory address space, which means that a process runs independently and is isolated from other processes. It cannot directly access shared data in other processes. Switching from one process to another requires some time (relatively) for saving and loading registers, memory maps, and other resources.

This independence of processes is valuable because the operating system tries its best to isolate processes so that a problem with one process doesn’t corrupt or cause havoc with another process. You’ve undoubtedly run into the situation in which one application on your computer freezes or has a problem and you’ve been able to quit that program without affecting others.

How Threads Work

So, are you still with us? We finally made it to threads!

A thread is the unit of execution within a process. A process can have anywhere from just one thread to many threads.

Process vs. Thread

diagram of threads in a process over time

When a process starts, it is assigned memory and resources. Each thread in the process shares that memory and resources. In single-threaded processes, the process contains one thread. The process and the thread are one and the same, and there is only one thing happening.

In multithreaded processes, the process contains more than one thread, and the process is accomplishing a number of things at the same time (technically, it’s almost at the same time—read more on that in the “What about Parallelism and Concurrency?” section below).

diagram of single and multi-treaded process

We talked about the two types of memory available to a process or a thread, the stack and the heap. It is important to distinguish between these two types of process memory because each thread will have its own stack, but all the threads in a process will share the heap.

Threads are sometimes called lightweight processes because they have their own stack but can access shared data. Because threads share the same address space as the process and other threads within the process, the operational cost of communication between the threads is low, which is an advantage. The disadvantage is that a problem with one thread in a process will certainly affect other threads and the viability of the process itself.

Threads vs. Processes

So to review:

  1. The program starts out as a text file of programming code,
  2. The program is compiled or interpreted into binary form,
  3. The program is loaded into memory,
  4. The program becomes one or more running processes.
  5. Processes are typically independent of each other,
  6. While threads exist as the subset of a process.
  7. Threads can communicate with each other more easily than processes can,
  8. But threads are more vulnerable to problems caused by other threads in the same process.

Processes vs. Threads — Advantages and Disadvantages

Process Thread
Processes are heavyweight operations Threads are lighter weight operations
Each process has its own memory space Threads use the memory of the process they belong to
Inter-process communication is slow as processes have different memory addresses Inter-thread communication can be faster than inter-process communication because threads of the same process share memory with the process they belong to
Context switching between processes is more expensive Context switching between threads of the same process is less expensive
Processes don’t share memory with other processes Threads share memory with other threads of the same process

What about Concurrency and Parallelism?

A question you might ask is whether processes or threads can run at the same time. The answer is: it depends. On a system with multiple processors or CPU cores (as is common with modern processors), multiple processes or threads can be executed in parallel. On a single processor, though, it is not possible to have processes or threads truly executing at the same time. In this case, the CPU is shared among running processes or threads using a process scheduling algorithm that divides the CPU’s time and yields the illusion of parallel execution. The time given to each task is called a “time slice.” The switching back and forth between tasks happens so fast it is usually not perceptible. The terms parallelism (true operation at the same time) and concurrency (simulated operation at the same time), distinguish between the two type of real or approximate simultaneous operation.

diagram of concurrency and parallelism

Why Choose Process over Thread, or Thread over Process?

So, how would a programmer choose between a process and a thread when creating a program in which she wants to execute multiple tasks at the same time? We’ve covered some of the differences above, but let’s look at a real world example with a program that many of us use, Google Chrome.

When Google was designing the Chrome browser, they needed to decide how to handle the many different tasks that needed computer, communications, and network resources at the same time. Each browser window or tab communicates with multiple servers on the internet to retrieve text, programs, graphics, audio, video, and other resources, and renders that data for display and interaction with the user. In addition, the browser can open many windows, each with many tasks.

Google had to decide how to handle that separation of tasks. They chose to run each browser window in Chrome as a separate process rather than a thread or many threads, as is common with other browsers. Doing that brought Google a number of benefits. Running each window as a process protects the overall application from bugs and glitches in the rendering engine and restricts access from each rendering engine process to others and to the rest of the system. Isolating JavaScript programs in a process prevents them from running away with too much CPU time and memory, and making the entire browser non-responsive.

Google made the calculated trade-off with a multi-processing design as starting a new process for each browser window has a higher fixed cost in memory and resources than using threads. They were betting that their approach would end up with less memory bloat overall.

Using processes instead of threads provides better memory usage when memory gets low. An inactive window is treated as a lower priority by the operating system and becomes eligible to be swapped to disk when memory is needed for other processes, helping to keep the user-visible windows more responsive. If the windows were threaded, it would be more difficult to separate the used and unused memory as cleanly, wasting both memory and performance.

You can read more about Google’s design decisions on Google’s Chromium Blog or on the Chrome Introduction Comic.

The screen capture below shows the Google Chrome processes running on a MacBook Air with many tabs open. Some Chrome processes are using a fair amount of CPU time and resources, and some are using very little. You can see that each process also has many threads running as well.

activity monitor of Google Chrome

The Activity Monitor or Task Manager on your system can be a valuable ally in helping fine-tune your computer or troubleshooting problems. If your computer is running slowly, or a program or browser window isn’t responding for a while, you can check its status using the system monitor. Sometimes you’ll see a process marked as “Not Responding.” Try quitting that process and see if your system runs better. If an application is a memory hog, you might consider choosing a different application that will accomplish the same task.

Windows Task Manager view

Made it This Far?

We hope this Tron-like dive into the fascinating world of computer programs, processes, and threads has helped clear up some questions you might have had.

The next time your computer is running slowly or an application is acting up, you know your assignment. Fire up the system monitor and take a look under the hood to see what’s going on. You’re in charge now.

We love to hear from you

Are you still confused? Have questions? If so, please let us know in the comments. And feel free to suggest topics for future blog posts.

The post What’s the Diff: Programs, Processes, and Threads appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Community Profile: David Pride

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/community-profile-david-pride/

This column is from The MagPi issue 55. You can download a PDF of the full issue for free, or subscribe to receive the print edition in your mailbox or the digital edition on your tablet. All proceeds from the print and digital editions help the Raspberry Pi Foundation achieve its charitable goals.

David Pride’s experiences in computer education came slightly later in life. He admits to not being a grade-A student: he left school with few qualifications, unable to pursue further education at university. There was, however, a teacher who instilled in him a passion for computers and coding which would stick with him indefinitely.

David Pride The MagPi Raspberry Pi Community Profile

David joined us at the St James’s Palace community celebration, mingling with the likes of the Duke of York, plus organisers of Jams and clubs, such as Grace and Femi

Welcome to the Community

Twenty years later, back in 2012, David heard of the Raspberry Pi – a soon-to-be-released “new little marvel” that he instantly fell for, head first. Despite a lack of knowledge in Linux and Python, he experimented and had fun. He found a Raspberry Jam and, with it, Pi enthusiasts like Mike Horne and Peter Onion. The projects on display at the Jam were enough to push David further into the Raspberry Pi rabbit hole and, after working his way through several Python books, he began to take steps into the world of formal higher education.

David Pride The MagPi Raspberry Pi Community Profile

David’s determination to access and complete further education in computing has earned him a three-year PhD studentship. Not bad for a “lousy student”

Back to School

With a Mooc qualification from Rice University under his belt, he continued to improve upon his self-taught knowledge, and was fortunate enough to be accepted to study for a master’s degree in Computer Science at the University of Hertfordshire. With a distinction for his final dissertation, David completed the course with an overall distinction for his MSc, and was recently awarded a fully funded PhD studentship with The Open University’s Knowledge Media Institute.

David Pride The MagPi Raspberry Pi Community Profile

Self-playing xylophones, Wiimote air drums, Lego sorters, Pi Wars robots, and more. David is continually hacking toys, giving them new Pi-powered life

Maker of things

The portfolio of projects that helped him to achieve his many educational successes has provided regular retweet material for the Raspberry Pi Twitter account, and we’ve highlighted his fun, imaginative work on this blog before. His builds have travelled to a range of Jams and made their way to the Raspberry Pi and Code Club stands at the Bett Show, as well as to our birthday celebrations.

David Pride The MagPi Raspberry Pi Community Profile

“Pi & Chips – with a little extra source”

His website, the pun-tastic Pi and Chips, is home to the majority of his work; David also links to YouTube videos and walk-throughs of his projects, and relates his experiences at various events. If you’ve followed any of the action across the Raspberry Pi social media channels – or indeed read any previous issues of The MagPi magazine – you’ll no doubt have seen a couple of David’s projects.

David Pride The MagPi Raspberry Pi Community Profile 4-Bot

Many readers will have come across the wonderful 4-Bot before, and it has even made an appearance alongside David in a recent Bloomberg interview. Considering the trillions of possible game positions, David made a compromise and, if you’re lucky, you may just be able to beat it

The 4-Bot, a robotic second player for the family game Connect Four, allows people to go head to head with a Pi-powered robotic arm. Using a Python imaging library, the 4-Bot splits the game grid into 42 squares, and recognises them as being red, yellow, or empty by reading the RGB value of the space. Using the minimax algorithm, 4-Bot is able to play each move within 25 seconds. Believe us when we say that it’s not as easy to beat as you’d hope. Then there’s his more recent air drum kit, which uses an old toy found at a car boot sale together with a Wiimote to make a functional air drum that showcases David’s toy-hacking abilities… and his complete lack of rhythm. He does fare much better on his homemade laser harp, though!

The post Community Profile: David Pride appeared first on Raspberry Pi.

Hacking a Gene Sequencer by Encoding Malware in a DNA Strand

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2017/08/hacking_a_gene_.html

One of the common ways to hack a computer is to mess with its input data. That is, if you can feed the computer data that it interprets — or misinterprets — in a particular way, you can trick the computer into doing things that it wasn’t intended to do. This is basically what a buffer overflow attack is: the data input overflows a buffer and ends up being executed by the computer process.

Well, some researchers did this with a computer that processes DNA, and they encoded their malware in the DNA strands themselves:

To make the malware, the team translated a simple computer command into a short stretch of 176 DNA letters, denoted as A, G, C, and T. After ordering copies of the DNA from a vendor for $89, they fed the strands to a sequencing machine, which read off the gene letters, storing them as binary digits, 0s and 1s.

Erlich says the attack took advantage of a spill-over effect, when data that exceeds a storage buffer can be interpreted as a computer command. In this case, the command contacted a server controlled by Kohno’s team, from which they took control of a computer in their lab they were using to analyze the DNA file.

News articles. Research paper.

Launch – AWS Glue Now Generally Available

Post Syndicated from Randall Hunt original https://aws.amazon.com/blogs/aws/launch-aws-glue-now-generally-available/

Today we’re excited to announce the general availability of AWS Glue. Glue is a fully managed, serverless, and cloud-optimized extract, transform and load (ETL) service. Glue is different from other ETL services and platforms in a few very important ways.

First, Glue is “serverless” – you don’t need to provision or manage any resources and you only pay for resources when Glue is actively running. Second, Glue provides crawlers that can automatically detect and infer schemas from many data sources, data types, and across various types of partitions. It stores these generated schemas in a centralized Data Catalog for editing, versioning, querying, and analysis. Third, Glue can automatically generate ETL scripts (in Python!) to translate your data from your source formats to your target formats. Finally, Glue allows you to create development endpoints that allow your developers to use their favorite toolchains to construct their ETL scripts. Ok, let’s dive deep with an example.

In my job as a Developer Evangelist I spend a lot of time traveling and I thought it would be cool to play with some flight data. The Bureau of Transportations Statistics is kind enough to share all of this data for anyone to use here. We can easily download this data and put it in an Amazon Simple Storage Service (S3) bucket. This data will be the basis of our work today.

Crawlers

First, we need to create a Crawler for our flights data from S3. We’ll select Crawlers in the Glue console and follow the on screen prompts from there. I’ll specify s3://crawler-public-us-east-1/flight/2016/csv/ as my first datasource (we can add more later if needed). Next, we’ll create a database called flights and give our tables a prefix of flights as well.

The Crawler will go over our dataset, detect partitions through various folders – in this case months of the year, detect the schema, and build a table. We could add additonal data sources and jobs into our crawler or create separate crawlers that push data into the same database but for now let’s look at the autogenerated schema.

I’m going to make a quick schema change to year, moving it from BIGINT to INT. Then I can compare the two versions of the schema if needed.

Now that we know how to correctly parse this data let’s go ahead and do some transforms.

ETL Jobs

Now we’ll navigate to the Jobs subconsole and click Add Job. Will follow the prompts from there giving our job a name, selecting a datasource, and an S3 location for temporary files. Next we add our target by specifying “Create tables in your data target” and we’ll specify an S3 location in Parquet format as our target.

After clicking next, we’re at screen showing our various mappings proposed by Glue. Now we can make manual column adjustments as needed – in this case we’re just going to use the X button to remove a few columns that we don’t need.

This brings us to my favorite part. This is what I absolutely love about Glue.

Glue generated a PySpark script to transform our data based on the information we’ve given it so far. On the left hand side we can see a diagram documenting the flow of the ETL job. On the top right we see a series of buttons that we can use to add annotated data sources and targets, transforms, spigots, and other features. This is the interface I get if I click on transform.

If we add any of these transforms or additional data sources, Glue will update the diagram on the left giving us a useful visualization of the flow of our data. We can also just write our own code into the console and have it run. We can add triggers to this job that fire on completion of another job, a schedule, or on demand. That way if we add more flight data we can reload this same data back into S3 in the format we need.

I could spend all day writing about the power and versatility of the jobs console but Glue still has more features I want to cover. So, while I might love the script editing console, I know many people prefer their own development environments, tools, and IDEs. Let’s figure out how we can use those with Glue.

Development Endpoints and Notebooks

A Development Endpoint is an environment used to develop and test our Glue scripts. If we navigate to “Dev endpoints” in the Glue console we can click “Add endpoint” in the top right to get started. Next we’ll select a VPC, a security group that references itself and then we wait for it to provision.


Once it’s provisioned we can create an Apache Zeppelin notebook server by going to actions and clicking create notebook server. We give our instance an IAM role and make sure it has permissions to talk to our data sources. Then we can either SSH into the server or connect to the notebook to interactively develop our script.

Pricing and Documentation

You can see detailed pricing information here. Glue crawlers, ETL jobs, and development endpoints are all billed in Data Processing Unit Hours (DPU) (billed by minute). Each DPU-Hour costs $0.44 in us-east-1. A single DPU provides 4vCPU and 16GB of memory.

We’ve only covered about half of the features that Glue has so I want to encourage everyone who made it this far into the post to go read the documentation and service FAQs. Glue also has a rich and powerful API that allows you to do anything console can do and more.

We’re also releasing two new projects today. The aws-glue-libs provide a set of utilities for connecting, and talking with Glue. The aws-glue-samples repo contains a set of example jobs.

I hope you find that using Glue reduces the time it takes to start doing things with your data. Look for another post from me on AWS Glue soon because I can’t stop playing with this new service.
Randall

Curb Your Enthusiasm on Those HBO Leaks

Post Syndicated from Ernesto original https://torrentfreak.com/curb-your-enthusiasm-on-those-hbo-leaks-170814/

Late July, news broke that a hacker, or hackers, had compromised the network of the American cable and television network HBO.

Those responsible contacted reporters, informing them about the prominent breach, and leaked files surfaced on the dedicated website Winter-leak.com.

The website wasn’t around for long, but last week the hackers reached out to the press again with a curated batch of new leaks shared through Mega.nz. Among other things, it contained more Game of Thrones spoilers, marketing plans, and other confidential HBO files.

Fast forward another week and there’s yet another freshly curated batch of leaks. This time it includes episodes of the highly anticipated return of ‘Curb Your Enthusiasm,’ which officially airs in October, as well as episodes from “Barry,” “Insecure” and “The Deuce,” AP reports.

These shows are part of the treasure trove of 1.5 terabytes that was taken from HBO. These and several other titles were already teased last week in a screenshot the hackers released to the press.

There’s no reason to doubt that the leaks are real, but thus far they haven’t been widely distributed. It appears that the various journalists who received the latest batch of Mega.nz links are not very eager to post them in public.

TorrentFreak scoured popular torrent sites and streaming portals for public copies of the new Curb Your Enthusiasm episodes and came up empty-handed. And we’re certainly not the only ones having trouble spotting the leaks in public.

“I searched around a lot a few hours ago and couldn’t find anything,” one Curb Your Enthusiasm watcher commented on Reddit. “Why can’t these hackers be courteous and place links?” another added.

This is quite different from the leaked episode of Game of Thrones that came out before its official release two weeks ago. That leak was not related to the HBO hack, but before the news broke in the mainstream press, thousands of copies were already available on pirate sites.

HBO, meanwhile, appears to have had enough of the continued enthusiasm the hacker is managing to generate in the press.

“We are not in communication with the hacker and we’re not going to comment every time a new piece of information is released,” a company spokesperson said.

“It has been widely reported that there was a cyber incident at HBO. The hacker may continue to drop bits and pieces of stolen information in an attempt to generate media attention. That’s a game we’re not going to participate in.”

As for the Curb Your Enthusiasm fans who were hoping for an early preview of the new season. They may have to, well… you know. For now at least.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

New Premier League Blocking Disrupts Pirate IPTV Providers

Post Syndicated from Andy original https://torrentfreak.com/new-premier-league-blocking-disrupts-pirate-iptv-providers-170814/

Top tier football in the UK is handled by the English Premier League (EPL) and broadcasting partners Sky and BT Sport. All face considerable problems with Internet piracy, through free web or Kodi-based streaming and premium IPTV feeds.

To mitigate the threat, earlier this year the Premier League obtained a unique High Court injunction which required ISPs such as Sky, BT, and Virgin to block ‘pirate’ football streams in real-time.

Although the success of the program was initially up for debate, the EPL reported it was able to block 5,000 server IP addresses that were streaming its content. When that temporary injunction ran out, the EPL went back to court for a new one, valid for the season that began this past weekend. There are signs the EPL may have upped its game.

As soon as the matches began on Saturday, issues were reported at several of the more prominent IPTV providers. Within minutes of the match streams going live, subscribers to affected services were met with black screens, causing anger and frustration. While some clearly knew that action was on the cards, relatively few had an effective plan in place.

One provider, which targets subscribers in the UK, scrambled to obtain new domain names, thinking that the existing domains had been placed on some kind of Premier League blacklist. While that may have indeed been the case, making a service more obscure in that sense was never going to outwit the systems deployed by the anti-piracy outfits involved.

Indeed, the provider in question was subjected to much chaos over both Saturday and Sunday, since it’s clear that large numbers of subscribers had absolutely no idea what was going on. Even if they understood that the EPL was blocking, the change of domain flat-footed the rest. The subsequent customer service chaos was not a pretty sight but would’ve been a pleasure for the EPL to behold.

An interesting side effect of this EPL action is that even if IPTV subscribers don’t care about football, many were affected this past weekend anyway.

TF is aware of at least three services (there are probably many more) that couldn’t service their UK customers with any other channels whatsoever while the Premier League games were being aired. This suggests that the IP addresses hit by the EPL and blocked by local ISPs belonged to the same servers carrying the rest of the content offered by the IPTV providers.

When the High Court handed down its original injunction it accepted that some non-Premier League content could be blocked at the same time but since that “consists almost exclusively of [infringing] commercial broadcast content such as other sports, films, and television programs,” there was little concern over collateral damage.

So the big question now is what can IPTV providers and/or subscribers do to tackle the threat?

The first interesting thing to note is not all of the big providers were affected this past weekend, so for many customers the matches passed by as normal. It isn’t clear whether EPL simply didn’t have all of the providers on the list or whether steps were taken to mitigate the threat, but that was certainly the case in a handful of cases.

Information passed to TF shows that at least a small number of providers were not only waiting for the EPL action but actually had a backup plan in place. This appears to have resulted in a minimum of disruption for their customers, something that will prove of interest to the many frustrated subscribers looking for a new service this morning.

While the past few days have been somewhat chaotic, other issues have been muddying the waters somewhat.

TF has learned that at least two, maybe three suppliers, were subjected to DDoS attacks around the time the matches were due to air. It seems unlikely that the EPL has been given permission to carry out such an attack but since the High Court injunction is secret in every way that describes its anti-piracy methods, that will remain a suspicion. In the meantime, rival IPTV services remain possible suspects.

Also, a major IPTV stream ‘wholesaler’ is reported to have had technical issues on Saturday, which affected its ability to serve lower-tier providers. Whether that was also linked to the Premier League action is unknown and TF couldn’t find any source willing to talk about the provider in any detail.

So, sports fans who rely on IPTV for their fix are wondering how things will pan out later this week. If this last weekend is anything to go by, disruption is guaranteed, but it will be less of a surprise given the problems of the last few days. While some don’t foresee huge problems, several providers are already advising customers that VPNs will be necessary.

An IPTV provider suggesting the use of VPNs

While a VPN will indeed solve the problem in most cases, for many subscribers that will amount to an additional expense, not to mention more time spent learning about VPNs, what they can do, and how they can be setup on the hardware they’re using for IPTV.

For users on Android devices running IPTV apps or Kodi-type setups, VPNs are both easy to install and use. However, Mag Box STB users cannot run a VPN directly on the device, meaning that they’ll need either a home router that can run a VPN or a smaller ‘travel’ type router with OpenVPN capabilities to use as a go-between.

Either way, costs are beginning to creep up, if IPTV providers can’t deal with the EPL’s blocking efforts. That makes the new cheaper football packages offered by various providers that little bit more attractive. But that was probably the plan all along.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

New – AWS SAM Local (Beta) – Build and Test Serverless Applications Locally

Post Syndicated from Randall Hunt original https://aws.amazon.com/blogs/aws/new-aws-sam-local-beta-build-and-test-serverless-applications-locally/

Today we’re releasing a beta of a new tool, SAM Local, that makes it easy to build and test your serverless applications locally. In this post we’ll use SAM local to build, debug, and deploy a quick application that allows us to vote on tabs or spaces by curling an endpoint. AWS introduced Serverless Application Model (SAM) last year to make it easier for developers to deploy serverless applications. If you’re not already familiar with SAM my colleague Orr wrote a great post on how to use SAM that you can read in about 5 minutes. At it’s core, SAM is a powerful open source specification built on AWS CloudFormation that makes it easy to keep your serverless infrastructure as code – and they have the cutest mascot.

SAM Local takes all the good parts of SAM and brings them to your local machine.

There are a couple of ways to install SAM Local but the easiest is through NPM. A quick npm install -g aws-sam-local should get us going but if you want the latest version you can always install straight from the source: go get github.com/awslabs/aws-sam-local (this will create a binary named aws-sam-local, not sam).

I like to vote on things so let’s write a quick SAM application to vote on Spaces versus Tabs. We’ll use a very simple, but powerful, architecture of API Gateway fronting a Lambda function and we’ll store our results in DynamoDB. In the end a user should be able to curl our API curl https://SOMEURL/ -d '{"vote": "spaces"}' and get back the number of votes.

Let’s start by writing a simple SAM template.yaml:

AWSTemplateFormatVersion : '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  VotesTable:
    Type: "AWS::Serverless::SimpleTable"
  VoteSpacesTabs:
    Type: "AWS::Serverless::Function"
    Properties:
      Runtime: python3.6
      Handler: lambda_function.lambda_handler
      Policies: AmazonDynamoDBFullAccess
      Environment:
        Variables:
          TABLE_NAME: !Ref VotesTable
      Events:
        Vote:
          Type: Api
          Properties:
            Path: /
            Method: post

So we create a [dynamo_i] table that we expose to our Lambda function through an environment variable called TABLE_NAME.

To test that this template is valid I’ll go ahead and call sam validate to make sure I haven’t fat-fingered anything. It returns Valid! so let’s go ahead and get to work on our Lambda function.

import os
import os
import json
import boto3
votes_table = boto3.resource('dynamodb').Table(os.getenv('TABLE_NAME'))

def lambda_handler(event, context):
    print(event)
    if event['httpMethod'] == 'GET':
        resp = votes_table.scan()
        return {'body': json.dumps({item['id']: int(item['votes']) for item in resp['Items']})}
    elif event['httpMethod'] == 'POST':
        try:
            body = json.loads(event['body'])
        except:
            return {'statusCode': 400, 'body': 'malformed json input'}
        if 'vote' not in body:
            return {'statusCode': 400, 'body': 'missing vote in request body'}
        if body['vote'] not in ['spaces', 'tabs']:
            return {'statusCode': 400, 'body': 'vote value must be "spaces" or "tabs"'}

        resp = votes_table.update_item(
            Key={'id': body['vote']},
            UpdateExpression='ADD votes :incr',
            ExpressionAttributeValues={':incr': 1},
            ReturnValues='ALL_NEW'
        )
        return {'body': "{} now has {} votes".format(body['vote'], resp['Attributes']['votes'])}

So let’s test this locally. I’ll need to create a real DynamoDB database to talk to and I’ll need to provide the name of that database through the enviornment variable TABLE_NAME. I could do that with an env.json file or I can just pass it on the command line. First, I can call:
$ echo '{"httpMethod": "POST", "body": "{\"vote\": \"spaces\"}"}' |\
TABLE_NAME="vote-spaces-tabs" sam local invoke "VoteSpacesTabs"

to test the Lambda – it returns the number of votes for spaces so theoritically everything is working. Typing all of that out is a pain so I could generate a sample event with sam local generate-event api and pass that in to the local invocation. Far easier than all of that is just running our API locally. Let’s do that: sam local start-api. Now I can curl my local endpoints to test everything out.
I’ll run the command: $ curl -d '{"vote": "tabs"}' http://127.0.0.1:3000/ and it returns: “tabs now has 12 votes”. Now, of course I did not write this function perfectly on my first try. I edited and saved several times. One of the benefits of hot-reloading is that as I change the function I don’t have to do any additional work to test the new function. This makes iterative development vastly easier.

Let’s say we don’t want to deal with accessing a real DynamoDB database over the network though. What are our options? Well we can download DynamoDB Local and launch it with java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb. Then we can have our Lambda function use the AWS_SAM_LOCAL environment variable to make some decisions about how to behave. Let’s modify our function a bit:

import os
import json
import boto3
if os.getenv("AWS_SAM_LOCAL"):
    votes_table = boto3.resource(
        'dynamodb',
        endpoint_url="http://docker.for.mac.localhost:8000/"
    ).Table("spaces-tabs-votes")
else:
    votes_table = boto3.resource('dynamodb').Table(os.getenv('TABLE_NAME'))

Now we’re using a local endpoint to connect to our local database which makes working without wifi a little easier.

SAM local even supports interactive debugging! In Java and Node.js I can just pass the -d flag and a port to immediately enable the debugger. For Python I could use a library like import epdb; epdb.serve() and connect that way. Then we can call sam local invoke -d 8080 "VoteSpacesTabs" and our function will pause execution waiting for you to step through with the debugger.

Alright, I think we’ve got everything working so let’s deploy this!

First I’ll call the sam package command which is just an alias for aws cloudformation package and then I’ll use the result of that command to sam deploy.

$ sam package --template-file template.yaml --s3-bucket MYAWESOMEBUCKET --output-template-file package.yaml
Uploading to 144e47a4a08f8338faae894afe7563c3  90570 / 90570.0  (100.00%)
Successfully packaged artifacts and wrote output template to file package.yaml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file package.yaml --stack-name 
$ sam deploy --template-file package.yaml --stack-name VoteForSpaces --capabilities CAPABILITY_IAM
Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - VoteForSpaces

Which brings us to our API:
.

I’m going to hop over into the production stage and add some rate limiting in case you guys start voting a lot – but otherwise we’ve taken our local work and deployed it to the cloud without much effort at all. I always enjoy it when things work on the first deploy!

You can vote now and watch the results live! http://spaces-or-tabs.s3-website-us-east-1.amazonaws.com/

We hope that SAM Local makes it easier for you to test, debug, and deploy your serverless apps. We have a CONTRIBUTING.md guide and we welcome pull requests. Please tweet at us to let us know what cool things you build. You can see our What’s New post here and the documentation is live here.

Randall

Turning an Amazon Echo into an Eavesdropping Device

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2017/08/turning_an_amaz.html

For once, the real story isn’t as bad as it seems. A researcher has figured out how to install malware onto an Echo that causes it to stream audio back to a remote controller, but:

The technique requires gaining physical access to the target Echo, and it works only on devices sold before 2017. But there’s no software fix for older units, Barnes warns, and the attack can be performed without leaving any sign of hardware intrusion.

The way to implement this attack is by intercepting the Echo before it arrives at the target location. But if you can do that, there are a lot of other things you can do. So while this is a vulnerability that needs to be fixed — and seems to have inadvertently been fixed — it’s not a cause for alarm.

Backblaze Cloud Backup 5.0: The Rapid Access Release

Post Syndicated from Yev original https://www.backblaze.com/blog/cloud-backup-5-0-rapid-access/

Announcing Backblaze Cloud Backup 5.0: the Rapid Access Release. We’ve been at the backup game for a long time now, and we continue to focus on providing the best unlimited backup service on the planet. A lot of the features in this release have come from listening to our customers about how they want to use their data. “Rapid Access” quickly became the theme because, well, we’re all acquiring more and more data and want to access it in a myriad of ways.

This release brings a lot of new functionality to Backblaze Computer Backup: faster backups, accelerated file browsing, image preview, individual file download (without creating a “restore”), and file sharing. To top it all off, we’ve refreshed the user interface on our client app. We hope you like it!

Speeding Things Up

New code + new hardware + elbow grease = things are going to move much faster.

Faster Backups

We’ve doubled the number of threads available for backup on both Mac and PC . This gives our service the ability to intelligently detect the right settings for you (based on your computer, capacity, and bandwidth). As always, you can manually set the number of threads — keep in mind that if you have a slow internet connection, adding threads might have the opposite effect and slow you down. On its default settings, our client app will now automatically evaluate what’s best given your environment. We’ve internally tested our service backing up at over 100 Mbps, which means if you have a fast-enough internet connection, you could back up 50 GB in just one hour.

Faster Browsing

We’ve introduced a number of enhancements that increase file browsing speed by 3x. Hidden files are no longer displayed by default, but you can still show them with one click on the restore page. This gives the restore interface a cleaner look, and helps you navigate backup history if you need to roll back time.

Faster Restore Preparation

We take pride in providing a variety of ways for consumers to get their data back. When something has happened to your computer, getting your files back quickly is critical. Both web download restores and Restore by Mail will now be much faster. In some cases up to 10x faster!

Preview — Access — Share

Our system has received a number of enhancements — all intended to give you more access to your data.

Image Preview

If you have a lot of photos, this one’s for you. When you go to the restore page you’ll now be able to click on each individual file that we have backed up, and if it’s an image you’ll see a preview of that file. We hope this helps people figure out which pictures they want to download (this especially helps people with a lot of photos named something along the lines of: 2017-04-20-9783-41241.jpg). Now you can just click on the picture to preview it.

Access

Once you’ve clicked on a file (30MB and smaller), you’ll be able to individually download that file directly in your browser. You’ll no longer need to wait for a single-file restore to be built and zipped up; you’ll be able to download it quickly and easily. This was a highly requested feature and we’re stoked to get it implemented.

Share

We’re leveraging Backblaze B2 Cloud Storage and giving folks the ability to publicly share their files. In order to use this feature, you’ll need to enable Backblaze B2 on your account (if you haven’t already, there’s a simple wizard that will pop up the first time you try to share a file). Files can be shared anywhere in the world via URL. All B2 accounts have 10GB/month of storage and 1GB/day of downloads (equivalent to sharing an iPhone photo 1,000 times per month) for free. You can increase those limits in your B2 Settings. Keep in mind that any file you share will be accessible to anybody with the link. Learn more about File Sharing.

For now, we’ve limited the Preview/Access/Share functionality to files 30MB and smaller, but larger files will be supported in the coming weeks!

Other Goodies

In addition to adding 2FV via ToTP, we’ve also been hard at work on the client. In version 5.0 we’ve touched up the user interface to make it a bit more lively, and we’ve also made the client IPv6 compatible.

Backblaze 5.0 Available: August 10, 2017

We will slowly be auto-updating all users in the coming weeks. To update now:

This version is now the default download on www.backblaze.com.

We hope you enjoy Backblaze Cloud Backup v5.0!

The post Backblaze Cloud Backup 5.0: The Rapid Access Release appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

[$] An alternative device-tree source language

Post Syndicated from corbet original https://lwn.net/Articles/730217/rss

Device trees have become, in a relatively short time, the preferred way to
inform the kernel of the available hardware on systems where that hardware
is not discoverable — most ARM systems, among others. In short, a
device tree is a textual description of a system’s hardware that is
compiled to a simple binary format and passed to the kernel by the
bootloader. The source format for device trees has been established for a
long time — longer than Linux has been using it. Perhaps it’s time for a
change, but a proposal for a new
device-tree source format has generated a fair amount of controversy in the
small corner of the community that concerns itself with such things.

Growing up alongside tech

Post Syndicated from Eevee original https://eev.ee/blog/2017/08/09/growing-up-alongside-tech/

IndustrialRobot asks… or, uh, asked last month:

industrialrobot: How has your views on tech changed as you’ve got older?

This is so open-ended that it’s actually stumped me for a solid month. I’ve had a surprisingly hard time figuring out where to even start.


It’s not that my views of tech have changed too much — it’s that they’ve changed very gradually. Teasing out and explaining any one particular change is tricky when it happened invisibly over the course of 10+ years.

I think a better framework for this is to consider how my relationship to tech has changed. It’s gone through three pretty distinct phases, each of which has strongly colored how I feel and talk about technology.

Act I

In which I start from nothing.

Nothing is an interesting starting point. You only really get to start there once.

Learning something on my own as a kid was something of a magical experience, in a way that I don’t think I could replicate as an adult. I liked computers; I liked toying with computers; so I did that.

I don’t know how universal this is, but when I was a kid, I couldn’t even conceive of how incredible things were made. Buildings? Cars? Paintings? Operating systems? Where does any of that come from? Obviously someone made them, but it’s not the sort of philosophical point I lingered on when I was 10, so in the back of my head they basically just appeared fully-formed from the æther.

That meant that when I started trying out programming, I had no aspirations. I couldn’t imagine how far I would go, because all the examples of how far I would go were completely disconnected from any idea of human achievement. I started out with BASIC on a toy computer; how could I possibly envision a connection between that and something like a mainstream video game? Every new thing felt like a new form of magic, so I couldn’t conceive that I was even in the same ballpark as whatever process produced real software. (Even seeing the source code for GORILLAS.BAS, it didn’t quite click. I didn’t think to try reading any of it until years after I’d first encountered the game.)

This isn’t to say I didn’t have goals. I invented goals constantly, as I’ve always done; as soon as I learned about a new thing, I’d imagine some ways to use it, then try to build them. I produced a lot of little weird goofy toys, some of which entertained my tiny friend group for a couple days, some of which never saw the light of day. But none of it felt like steps along the way to some mountain peak of mastery, because I didn’t realize the mountain peak was even a place that could be gone to. It was pure, unadulterated (!) playing.

I contrast this to my art career, which started only a couple years ago. I was already in my late 20s, so I’d already spend decades seeing a very broad spectrum of art: everything from quick sketches up to painted masterpieces. And I’d seen the people who create that art, sometimes seen them create it in real-time. I’m even in a relationship with one of them! And of course I’d already had the experience of advancing through tech stuff and discovering first-hand that even the most amazing software is still just code someone wrote.

So from the very beginning, from the moment I touched pencil to paper, I knew the possibilities. I knew that the goddamn Sistine Chapel was something I could learn to do, if I were willing to put enough time in — and I knew that I’m not, so I’d have to settle somewhere a ways before that. I knew that I’d have to put an awful lot of work in before I’d be producing anything very impressive.

I did it anyway (though perhaps waited longer than necessary to start), but those aren’t things I can un-know, and so I can never truly explore art from a place of pure ignorance. On the other hand, I’ve probably learned to draw much more quickly and efficiently than if I’d done it as a kid, precisely because I know those things. Now I can decide I want to do something far beyond my current abilities, then go figure out how to do it. When I was just playing, that kind of ambition was impossible.


So, I played.

How did this affect my views on tech? Well, I didn’t… have any. Learning by playing tends to teach you things in an outward sprawl without many abrupt jumps to new areas, so you don’t tend to run up against conflicting information. The whole point of opinions is that they’re your own resolution to a conflict; without conflict, I can’t meaningfully say I had any opinions. I just accepted whatever I encountered at face value, because I didn’t even know enough to suspect there could be alternatives yet.

Act II

That started to seriously change around, I suppose, the end of high school and beginning of college. I was becoming aware of this whole “open source” concept. I took classes that used languages I wouldn’t otherwise have given a second thought. (One of them was Python!) I started to contribute to other people’s projects. Eventually I even got a job, where I had to work with other people. It probably also helped that I’d had to maintain my own old code a few times.

Now I was faced with conflicting subjective ideas, and I had to form opinions about them! And so I did. With gusto. Over time, I developed an idea of what was Right based on experience I’d accrued. And then I set out to always do things Right.

That’s served me decently well with some individual problems, but it also led me to inflict a lot of unnecessary pain on myself. Several endeavors languished for no other reason than my dissatisfaction with the architecture, long before the basic functionality was done. I started a number of “pure” projects around this time, generic tools like imaging libraries that I had no direct need for. I built them for the sake of them, I guess because I felt like I was improving some niche… but of course I never finished any. It was always in areas I didn’t know that well in the first place, which is a fine way to learn if you have a specific concrete goal in mind — but it turns out that building a generic library for editing images means you have to know everything about images. Perhaps that ambition went a little haywire.

I’ve said before that this sort of (self-inflicted!) work was unfulfilling, in part because the best outcome would be that a few distant programmers’ lives are slightly easier. I do still think that, but I think there’s a deeper point here too.

In forgetting how to play, I’d stopped putting any of myself in most of the work I was doing. Yes, building an imaging library is kind of a slog that someone has to do, but… I assume the people who work on software like PIL and ImageMagick are actually interested in it. The few domains I tried to enter and revolutionize weren’t passions of mine; I just happened to walk through the neighborhood one day and decided I could obviously do it better.

Not coincidentally, this was the same era of my life that led me to write stuff like that PHP post, which you may notice I am conspicuously not even linking to. I don’t think I would write anything like it nowadays. I could see myself approaching the same subject, but purely from the point of view of language design, with more contrasts and tradeoffs and less going for volume. I certainly wouldn’t lead off with inflammatory puffery like “PHP is a community of amateurs”.

Act III

I think I’ve mellowed out a good bit in the last few years.

It turns out that being Right is much less important than being Not Wrong — i.e., rather than trying to make something perfect that can be adapted to any future case, just avoid as many pitfalls as possible. Code that does something useful has much more practical value than unfinished code with some pristine architecture.

Nowhere is this more apparent than in game development, where all code is doomed to be crap and the best you can hope for is to stem the tide. But there’s also a fixed goal that’s completely unrelated to how the code looks: does the game work, and is it fun to play? Yes? Ship the damn thing and forget about it.

Games are also nice because it’s very easy to pour my own feelings into them and evoke feelings in the people who play them. They’re mine, something with my fingerprints on them — even the games I’ve built with glip have plenty of my own hallmarks, little touches I added on a whim or attention to specific details that I care about.

Maybe a better example is the Doom map parser I started writing. It sounds like a “pure” problem again, except that I actually know an awful lot about the subject already! I also cleverly (accidentally) released some useful results of the work I’ve done thusfar — like statistics about Doom II maps and a few screenshots of flipped stock maps — even though I don’t think the parser itself is far enough along to release yet. The tool has served a purpose, one with my fingerprints on it, even without being released publicly. That keeps it fresh in my mind as something interesting I’d like to keep working on, eventually. (When I run into an architecture question, I step back for a while, or I do other work in the hopes that the solution will reveal itself.)

I also made two simple Pokémon ROM hacks this year, despite knowing nothing about Game Boy internals or assembly when I started. I just decided I wanted to do an open-ended thing beyond my reach, and I went to do it, not worrying about cleanliness and willing to accept a bumpy ride to get there. I played, but in a more experienced way, invoking the stuff I know (and the people I’ve met!) to help me get a running start in completely unfamiliar territory.


This feels like a really fine distinction that I’m not sure I’m doing justice. I don’t know if I could’ve appreciated it three or four years ago. But I missed making toys, and I’m glad I’m doing it again.

In short, I forgot how to have fun with programming for a little while, and I’ve finally started to figure it out again. And that’s far more important than whether you use PHP or not.

Getting Your Data into the Cloud is Just the Beginning

Post Syndicated from Andy Klein original https://www.backblaze.com/blog/cost-data-of-transfer-cloud-storage/

Total Cloud Storage Cost

Organizations should consider not just the cost of getting their data into the cloud, but also long-term costs for storage and retrieval when deciding which cloud storage solution meets their needs.

As cloud storage has become ubiquitous, organizations large and small are joining in. For larger organizations the lure of reducing capital expenses and their associated operational costs is enticing. For smaller organizations, cloud storage often replaces an unmanageable closet full of external hard drives, thumb drives, SD cards, and other devices. With terabytes or even petabytes of data, the common challenge facing organizations, large and small, is how to get their data up to the cloud.

Transferring Data to the Cloud

The obvious solution for getting your data to the cloud is to upload your data from your internal network through the internet to the cloud storage vendor you’ve selected. Cloud storage vendors don’t charge you for uploading your data to their cloud, but you, of course, have to pay your network provider and that’s where things start to get interesting. Here are a few things to consider.

  • The initial upload: Unless you are just starting out, you will have a large amount of data you want to upload to the cloud. This could be data you wish to archive or have had archived previously, for example data stored on LTO tapes or kept stored on external hard drives.
  • Pipe size: This is the amount of upload bandwidth of your network connection. This is measured in Mbps (megabits per second). Remember, your data is stored in MB (megabytes), so an upload connection of 80 Mbps will transfer no more than 10 MB of data per second and most likely a lot less.
  • Cost and caps: In some places, organizations pay a flat monthly rate for a specified level of service (speed) for internet access. In other locations, internet access is metered, or pay as you go. In either case, there can be internet service caps that limit or completely stop data transfer once you reach your contracted threshold.

One or more of these challenges has the potential to make the initial upload of your data expensive and potentially impossible. You could wait until cloud storage companies start buying up internet providers and make data upload cheap (or free with Amazon Prime!), but there is another option.

Data Transfer Devices

Given the potential challenges of using your network for the initial upload of your data to the cloud, a handful of cloud storage companies have introduced data transfer or data ingest services. Backblaze has the B2 Fireball, Amazon has Snowball (and other similar devices), and Google recently introduced their Transfer Appliance.

KLRU-TV Austin PBS uploaded their Austin City Limits musical anthology series to Backblaze using a B2 Fireball.

These services work as follows:

  • The provider sends you a portable (or somewhat portable) storage device.
  • You connect the device to your network and load some amount of data on the device over your internal network connection.
  • You return the device, loaded with your data, to the provider, who uploads your data to your cloud storage account from inside their own data center.

Data Transfer Devices Save Time

Assuming your Internet connection is a flat rate service that has no caps or limits and your organizational operations can withstand the traffic, you still may want to opt to use a data transfer service to move your data to the cloud. Why? Time. For example, if your initial data upload is 100 TB here’s how long it would take using different network upload connection speeds:

Network Speed Upload Time
10 Mbps 3 years
100 Mbps 124 days
500 Mbps 25 days
1 Gbps 12 days

This assumes you are using most of your upload connection to upload your data, which is probably not realistic if you want to stay in business. You could potentially rent a better connection or upgrade your connection permanently, both of which add to the cost of running your business.

Speaking of cost, there is of course a charge for the data transfer service that can be summarized as follows:

  • Backblaze B2 Fireball — Up to 40 TB of data per trip for $550.00 for 30 days in use at your site.
  • Amazon Snowball — up to 50 TB of data per trip for $200.00 for 10 days use at your site, plus $15/day each day in use at your site thereafter.
  • Google Transfer Appliance — up to 100 TB of data per trip for $300.00 for 10 days use at your site, plus $10/day each day in use at your site thereafter.

These prices do not include shipping, which can range from $100 to $900 depending on shipping method, location, etc.

Both Amazon and Google have transfer devices that are larger and cost more. For comparison purposes below we’ll use the three device versions listed above.

The Real Cost of Uploading Your Data

If we stopped our review at the previous paragraph and we were prepared to load up our transfer device in 10 days or less, the clear winner would be Google. But, this leaves out two very important components of any cloud storage project; the cost of storing your data and the cost of downloading your data.

Let’s look at two examples:

Example 1 — Archive 100 TB of data:

  • Use the data transfer service move 100 TB of data to the cloud storage service.
  • Accomplish the transfer within 10 days.
  • Store that 100 TB of data for 1 year.
Service Transfer Cost Cloud Storage Total
Backblaze B2 $1,650 (3 trips) $6,000 $7,650
Google Cloud $300 (1 trip) $24,000 $24,300
Amazon S3 $400 (2 trips) $25,200 $25,600

Results:

  • Using the B2 Fireball to store data in Backblaze B2 saves you $16,650 over a one-year period versus the Google solution.
  • The payback period for using a Backblaze B2 FireBall versus a Google Transfer Appliance is less than 1 month.

Example 2 — Store and use 100 TB of data:

  • Use the data transfer service to move 100 TB of data to the cloud storage service.
  • Accomplish the transfer within 10 days.
  • Store that 100 TB of data for 1 year.
  • Add 5 TB a month (on average) to the total stored.
  • Delete 2 TB a month (on average) from the total stored.
  • Download 10 TB a month (on average) from the total stored.
Service Transfer Cost Cloud Storage Total
Backblaze B2 $1,650 (3 trips) $9,570 $11,220
Google Cloud $300 (1 trip) $39,684 $39,984
Amazon S3 $400 (2 trips) $36,114 $36,514

Results:

  • Using the B2 Fireball to store data in Backblaze B2 saves you $28,764 over a one-year period versus the Google solution.
  • The payback period for using a Backblaze B2 FireBall versus a Google Transfer Appliance is less than 1 month.

Notes:

  • All prices listed are based on list prices from the vendor websites as of the date of this blog post.
  • We are accomplishing the transfer of your data to the device within the 10 day “free” period specified by Amazon and Google.
  • We are comparing cloud storage services that have similar performance. For example, once the data is uploaded, it is readily available for download. The data is also available for access via a Web GUI, CLI, API, and/or various applications integrated with the cloud storage service. Multiple versions of files can be kept as desired. Files can be deleted any time.

To be fair, it requires Backblaze three trips to move 100 TB while it only takes 1 trip for the Google Transfer Appliance. This adds some cost to prepare, monitor, and ship three B2 Fireballs versus one Transfer Appliance. Even with that added cost, the Backblaze B2 solution will still be significantly less expensive over the one year period and beyond.

Have a Data Transfer Device Owner

Before you run out and order a transfer device, make sure the transfer process is someone’s job once the device arrives at your organization. Filling a transfer device should only take a few days, but if it is forgotten, you’ll find you’ve had the device for 2 or 3 weeks. While that’s not much of a problem with a B2 Fireball, it could start to get expensive otherwise.

Just the Beginning

As with most “new” technologies and services, you can expect other companies to jump in and provide various data ingest services. The cost will get cheaper or even free as cloud storage companies race to capture and lock up the data you have kept locally all these years. When you are evaluating cloud storage solutions, it’s best to look past the data ingest loss-leader price, and spend a few minutes to calculate the long-term cost of storing and using your data.

The post Getting Your Data into the Cloud is Just the Beginning appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

RIAA’s Piracy Claims are Misleading and Inaccurate, ISP Says

Post Syndicated from Ernesto original https://torrentfreak.com/riaas-piracy-claims-are-misleading-and-inaccurate-isp-says-170807/

For more than a decade, copyright holders have been sending ISPs takedown notices to alert them that their subscribers are sharing copyrighted material.

Under US law, providers have to terminate the accounts of repeat infringers “in appropriate circumstances” and increasingly they are being held to this standard.

Earlier this year several major record labels, represented by the RIAA, filed a lawsuit in a Texas District Court, accusing ISP Grande Communications of failing to take action against its pirating subscribers.

The ISP is not happy with the claims and was quick to submit a motion to dismiss the lawsuit. One of the arguments is that the RIAA’s evidence is insufficient.

In its original motion, Grande doesn’t deny receiving millions of takedown notices from piracy tracking company Rightscorp. However, it believes that these notices are flawed as Rightscorp is incapable of monitoring actual copyright infringements.

The RIAA disagreed and pointed out that their evidence is sufficient. They stressed that Rightcorp is able to monitor actual downloads, as opposed to simply checking if a subscriber is offering certain infringing content.

In a response from Grande, late last week, the ISP argues that this isn’t good enough to build a case. While Rightcorp may be able to track the actual infringing downloads to which the RIAA labels hold the copyrights, there is no such evidence provided in the present case, the ISP notes.

“Importantly, Plaintiffs do not allege that Rightscorp has ever recorded an instance of a Grande subscriber actually distributing even one of Plaintiffs’ copyrighted works. Plaintiffs certainly have not alleged any concrete facts regarding such an act,” Grande’s legal team writes (pdf).

According to the ISP, the RIAA’s evidence merely shows that Rightscorp sent notices of alleged infringements on behalf of other copyright holders, who are not involved in the lawsuit.

“Instead, Plaintiffs generally allege that Rightscorp has sent notices regarding ‘various copyrighted works,’ encompassing all of the notices sent by Rightscorp on behalf of entities other than Plaintiffs.”

While the RIAA argues that this circumstantial evidence is sufficient, the ISP believes that there are grounds to have the entire case dismissed.

The record labels can’t hold Grande liable for secondary copyright infringement, without providing concrete evidence that their works were actively distributed by Grande subscribers, the company claims.

“Plaintiffs cannot allege direct infringement without alleging concrete facts which show that a Grande subscriber actually infringed one of Plaintiffs’ copyrights,” Grande’s lawyers note.

“For this reason, it is incredibly misleading for Plaintiffs to repeatedly refer to Grande having received ‘millions’ of notices of alleged infringement, as if those notices all pertained to Plaintiffs’ asserted copyrights.”

The “misleading” copyright infringement evidence argument is only one part of the ISPs defense. The company also notes that it has no control over what its subscribers do, nor do they control the BitTorrent clients that were allegedly used to download content.

If the court ruled otherwise, Grande and other ISPs would essentially be forced to become an “unpaid enforcement agent of the recording industry,” the company’s lawyers note.

The RIAA, however, sees things quite differently.

The music industry group believes that Grande failed to take proper action in response to repeat infringers and should pay damages to compensate the labels. This claim is very similar to the one BMG brought against Cox, where the latter was eventually ordered to pay $25 million.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

[$] Escape from QuickBooks (with data in hand)

Post Syndicated from corbet original https://lwn.net/Articles/729087/rss

When a small business contemplates getting away from a proprietary
accounting tool like QuickBooks in favor of free software like GnuCash, the
first order of business is usually finding a way to liberate that
business’s accounting data for input into a new system. Strangely enough,
Intuit, the creator of QuickBooks, never quite got around to making that
easy to do. But it turns out
that, with a bit of effort, this move can be made. Getting there involves
wandering through an undocumented wilderness; this article is at attempt to
make things easier for the next people to come along.

Lawyer Says He Was Deceived Into BitTorrent Copyright Trolling Scheme

Post Syndicated from Andy original https://torrentfreak.com/lawyer-says-he-was-deceived-into-bittorrent-copyright-trolling-scheme-170807/

For more than a decade, companies around the world have been trying to turn piracy into profit. For many this has meant the development of “copyright trolling” schemes, in which alleged pirates are monitored online and then pressured into cash settlements.

The shadowy nature of this global business means that its true scale will never be known but due to the controversial activities of some of the larger players, it’s occasionally possible to take a peek inside their operations. One such opportunity has just raised its head.

According to a lawsuit filed in California, James Davis is an attorney licensed in Oregon and California. Until two years ago, he was largely focused on immigration law. However, during March 2015, Davis says he was approached by an old classmate with an opportunity to get involved in a new line of business.

That classmate was Oregon lawyer Carl Crowell, who over the past several years has been deeply involved in copyright-trolling cases, including a deluge of Dallas Buyers Club and London Has Fallen litigation. He envisioned a place for Davis in the business.

Davis seemed to find the proposals attractive and became seriously involved in the operation, filing 58 cases on behalf of the companies involved. In common with similar cases, the lawsuits were brought in the name of the entities behind each copyrighted work, such as Dallas Buyers Club, LLC and LHF Productions, Inc.

In time, however, things started to go wrong. Davis claims that he discovered that Crowell, in connection with and on behalf of the other named defendants, “misrepresented the true nature of the Copyright Litigation Campaign, including the ownership of the works at issue and the role of the various third-parties involved in the litigation.”

Davis says that Crowell and the other defendants (which include the infamous Germany-based troll outfit Guardaley) made false representations to secure his participation, while holding back other information that might have made him think twice about becoming involved.

“Crowell and other Defendants withheld numerous material facts that were known to Crowell and the knowledge of which would have cast doubt on the value and ethical propriety of the Copyright Litigation Campaign for Mr. Davis,” the lawsuit reads.

Davis goes on to allege serious misconduct, including that representations regarding ownership of various entities were false and used to deceive him into participating in the scheme.

As time went on, Davis said he had increasing doubts about the operation. Then, in August 2016 as a result of a case underway in California, he began asking questions which resulted in him uncovering additional facts. These undermined both the representations of the people he was working for and his own belief in the “value and ethical propriety of the Copyright Litigation Campaign,” the lawsuit claims.

Davis said this spurred him on to “aggressively seek further information” from Crowell and other people involved in the scheme, including details of its structure and underlying support. He says all he received were “limited responses, excuses, and delays.”

The case was later dismissed by mutual agreement of the parties involved but of course, Davis’ concerns about the underlying case didn’t come to the forefront until the filing of his suit against Crowell and the others.

Davis says that following a meeting in Santa Monica with several of the main players behind the litigation campaign, he decided its legal and factual basis were unsound. He later told Crowell and Guardaley that he was withdrawing from their project.

As the result of the misrepresentations made to him, Davis is now suing the defendants on a number of counts, detailed below.

“Defendants’ business practices are unfair, unlawful, and fraudulent. Davis has suffered monetary damage as a direct result of the unfair, unlawful, and fraudulent business practices set forth herein,” the lawsuit reads.

Requesting a trial by jury, Davis is seeking actual damages, statutory damages, punitive or treble damages “in the amount of no less than $300,000.”

While a payment of that not insignificant amount would clearly satisfy Davis, the prospect of a trial in which the Guardaley operation is laid bare would be preferable when the interests of its thousands of previous targets are considered.

Only time will tell how things will pan out but like the vast majority of troll cases, this one too seems destined to be settled in private, to ensure the settlement machine keeps going.

Note: The case was originally filed in June, only to be voluntarily dismissed. It has now been refiled in state court.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

ESET Tries to Scare People Away From Using Torrents

Post Syndicated from Andy original https://torrentfreak.com/eset-tries-to-scare-people-away-from-using-torrents-170805/

Any company in the security game can be expected to play up threats among its customer base in order to get sales.

Sellers of CCTV equipment, for example, would have us believe that criminals don’t want to be photographed and will often go elsewhere in the face of that. Car alarm companies warn us that since X thousand cars are stolen every minute, an expensive Immobilizer is an anti-theft must.

Of course, they’re absolutely right to point these things out. People want to know about these offline risks since they affect our quality of life. The same can be said of those that occur in the online world too.

We ARE all at risk of horrible malware that will trash our computers and steal our banking information so we should all be running adequate protection. That being said, how many times do our anti-virus programs actually trap a piece of nasty-ware in a year? Once? Twice? Ten times? Almost never?

The truth is we all need to be informed but it should be done in a measured way. That’s why an article just published by security firm ESET on the subject of torrents strikes a couple of bad chords, particularly with people who like torrents. It’s titled “Why you should view torrents as a threat” and predictably proceeds to outline why.

“Despite their popularity among users, torrents are very risky ‘business’,” it begins.

“Apart from the obvious legal trouble you could face for violating the copyright of musicians, filmmakers or software developers, there are security issues linked to downloading them that could put you or your computer in the crosshairs of the black hats.”

Aside from the use of the phrase “very risky” (‘some risk’ is a better description), there’s probably very little to complain about in this opening shot. However, things soon go downhill.

“Merely downloading the newest version of BitTorrent clients – software necessary for any user who wants to download or seed files from this ‘ecosystem’ – could infect your machine and irreversibly damage your files,” ESET writes.

Following that scary statement, some readers will have already vowed never to use a torrent again and moved on without reading any more, but the details are really important.

To support its claim, ESET points to two incidents in 2016 (which to its great credit the company actually discovered) which involved the Transmission torrent client. Both involved deliberate third-party infection and in the latter hackers attacked Transmission’s servers and embedded malware in its OSX client before distribution to the public.

No doubt these were both miserable incidents (to which the Transmission team quickly responded) but to characterize this as a torrent client problem seems somewhat unfair.

People intent on spreading viruses and malware do not discriminate and will happily infect ANY piece of computer software they can. Sadly, many non-technical people reading the ESET post won’t read beyond the claim that installing torrent clients can “infect your machine and irreversibly damage your files.”

That’s a huge disservice to the hundreds of millions of torrent client installations that have taken place over a decade and a half and were absolutely trouble free. On a similar basis, we could argue that installing Windows is the main initial problem for people getting viruses from the Internet. It’s true but it’s also not the full picture.

Finally, the piece goes on to detail other incidents over the years where torrents have been found to contain malware. The several cases highlighted by ESET are both real and pretty unpleasant for victims but the important thing to note here is torrent users are no different to any other online user, no matter how they use the Internet.

People who download files from the Internet, from ALL untrusted sources, are putting themselves at risk of getting a virus or other malware. Whether that content is obtained from a website or a P2P network, the risks are ever-present and only a foolish person would do so without decent security software (such as ESET’s) protecting them.

The take home point here is to be aware of security risks and put them into perspective. It’s hard to put a percentage on these things but of the hundreds of millions of torrent and torrent client downloads that have taken place since their inception 15 years ago, the overwhelming majority have been absolutely fine.

Security situations do arise and we need to be aware of them, but presenting things in a way that spreads unnecessary concern in a particular sector isn’t necessary to sell products.

The AV-TEST Institute registers around 390,000 new malicious programs every day that don’t involve torrents, plenty for any anti-virus firm to deal with.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

Court Won’t Drop Case Against Alleged KickassTorrents Owner

Post Syndicated from Ernesto original https://torrentfreak.com/court-wont-drop-case-against-alleged-kickasstorrents-owner-170804/

kickasstorrents_500x500Last summer, Polish law enforcement officers arrested Artem Vaulin, the alleged founder of KickassTorrents.

Polish authorities acted on a criminal complaint from the US Government, which accused Vaulin of criminal copyright infringement and money laundering.

While Vaulin is still awaiting the final decision in his extradition process in Poland, his US counsel tried to have the entire case thrown out with a motion to dismiss submitted to the Illinois District Court late last year.

One of the fundamental flaws of the case, according to the defense, is that torrent files themselves are not copyrighted content. In addition, they argued that any secondary copyright infringement claims would fail as these are non-existent under criminal law.

After a series of hearings and a long wait afterwards, US District Judge John Z. Lee has now issued his verdict (pdf).

In a 28-page memorandum and order, the motion to dismiss was denied on various grounds.

The court doesn’t contest that torrent files themselves are not protected content under copyright law. However, this argument ignores the fact that the files are used to download copyrighted material, the order reads.

“This argument, however, misunderstands the indictment. The indictment is not concerned with the mere downloading or distribution of torrent files,” Judge Lee writes.

“Granted, the indictment describes these files and charges Vaulin with operating a website dedicated to hosting and distributing them. But the protected content alleged to have been infringed in the indictment is a number of movies and other copyright protected media that users of Vaulin’s network purportedly downloaded and distributed..,” he adds.

In addition, the defense’s argument that secondary copyright infringement claims are non-existent under criminal law doesn’t hold either, according to the Judge’s decision.

Vaulin’s defense noted that the Government’s theory could expose other search engines, such as Google, to criminal liability. While this is theoretically possible, the court sees distinct differences and doesn’t aim to rule on all search engines in general.

“For present purposes, though, the Court need not decide whether and when a search engine operator might engage in conduct sufficient to constitute aiding and abetting criminal copyright infringement. The issue here is whether 18 U.S.C. § 2 applies to 17 U.S.C. § 506. The Court is persuaded that it does,” Judge Lee writes.

Based on these and other conclusions, the motion to dismiss was denied. This means that the case will move forward. The next step will be to see how the Polish court rules on the extradition request.

Vaulin’s lead counsel Ira Rothken is disappointed with the outcome. He stresses that while courts commonly construe indictments in a light most favorable to the government, it went too far in this case.

“Currently a person merely ‘making available’ a file on a network in California wouldn’t even be committing a civil copyright infringement under the ruling in Napster but under today’s ruling that same person doing it in Illinois could be criminally prosecuted by the United States,” Rothken informs TorrentFreak.

“If federal judges disagree on the state of the federal copyright law then people shouldn’t be criminally prosecuted absent clarification by Congress,” he adds.

The defense team is still considering the best options for appeal, and whether they want to go down that road. However, Rothken hopes that the Seventh Circuit Court of Appeals will address the issue in the future.

“We hope one day that the Seventh Circuit Court of Appeals will undo this ruling and the chilling effect it will have on internet search engines, user generated content sites, and millions of netizens globally,” Rothken notes.

For now, however, Vaulin’s legal team will likely shift its focus to preventing his extradition to the United States.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.