Tag Archives: groups

Healthy Aussie Pirates Set To Face Cash ‘Fines’, Poor & Sick Should Be OK

Post Syndicated from Andy original https://torrentfreak.com/healthy-aussie-pirates-set-to-face-cash-fines-poor-sick-should-be-ok-170821/

One of the oldest methods of trying to get people to stop downloading and sharing pirated material is by hitting them with ‘fines’.

The RIAA began the practice in September 2003, tracking people sharing music on early peer-to-peer networks, finding out their identities via ISPs, and sending them cease-and-desist orders with a request to pay hundreds to thousands of dollars.

Many thousands of people were fined and the campaign raised awareness, but it did nothing to stop millions of file-sharers who continue to this day.

That’s something that Village Roadshow co-chief Graham Burke now wants to do something about. He says his company will effectively mimic the RIAA’s campaign of 14 years ago and begin suing Internet pirates Down Under. He told AFR that his company is already setting things up, ready to begin suing later in the year.

Few details have been made available at this stage but it’s almost certain that Village Roadshow’s targets will be BitTorrent users. It’s possible that users of other peer-to-peer networks could be affected but due to their inefficiency and relative obscurity, it’s very unlikely.

That leaves users of The Pirate Bay and any other torrent site vulnerable to the company, which will jump into torrent swarms masquerading as regular users, track IP addresses, and trace them back to Internet service providers. What happens next will depend on the responses of those ISPs.

If the ISPs refuse to cooperate, they will have to be taken to court to force them to hand over the personal details of their subscribers to Village Roadshow. It’s extremely unlikely they’ll hand them over voluntarily, so it could be some time before any ISP customer hears anything from the film distributor.

The bottom line is that Village Roadshow will want money to go away and Burke is already being open over the kind of sums his company will ask for.

“We will be looking for damages commensurate with what they’ve done. We’ll be saying ‘You’ve downloaded our Mad Max: Fury Road, our Red Dog, and we want $40 for the four movies plus $200 in costs’,” he says.

While no one will relish any kind of ‘bill’ dropping through a mail box, in the scheme of things a AUS$240 settlement demand isn’t huge, especially when compared to the sums demanded by companies such as Voltage Pictures, who tried and failed to start piracy litigation in Australia two years ago.

However, there’s even better news for some, who have already been given a heads-up that they won’t have to pay anything.

“We will identify people who are stealing our product, we will ask them do they have ill health or dire circumstances, and if they do and undertake to stop, we’ll drop the case,” Burke says.

While being upfront about such a policy has its pros and cons, Burke is also reducing his range of targets, particularly if likes to be seen as a man of his word, whenever those words were delivered. In March 2016, when he restated his intention to begin suing pirates, he also excluded some other groups from legal action.

“We don’t want to sue 16-year-olds or mums and dads,” Burke said. “It takes 18 months to go through the courts and all that does is make lawyers rich and clog the court system. It’s not effective.”

It will remain to be seen what criteria Village Roadshow ultimately employs but it’s likely the company will be asked to explain its intentions to the court, when it embarks on the process to discover alleged pirates’ identities. When it’s decided who is eligible, Burke says the gloves will come off, with pirates being “pursued vigorously” and “sued for damages.”

While Village Roadshow’s list of films is considerable, any with a specifically Australian slant seem the most likely to feature in any legal action. Burke tends to push the narrative that he’s looking after local industry so something like Mad Max: Fury Road would be perfect. It would also provide easy pickings for any anti-piracy company seeking to harvest Aussie IP addresses since it’s still very popular.

Finally, it’s worth noting that Australians who use pirate streaming services will be completely immune to the company’s planned lawsuit campaign. However, Burke appears to be tackling that threat using a couple of popular tactics currently being deployed elsewhere by the movie industry.

“Google are not doing enough and could do a lot more,” he told The Australian (subscription)

Burke said that he was “shocked” at how easy it was to find streaming content using Google’s search so decided to carry out some research of his own at home. He said he found Christopher Nolan’s Dunkirk with no difficulty but that came with a sting in the tail.

According to the movie boss, his computer was immediately infected with malware and began asking for his credit card details. He doesn’t say whether he put them in.

As clearly the world’s most unlucky would-be movie pirate, Burke deserves much sympathy. It’s also completely coincidental that Hollywood is now pushing a “danger” narrative to keep people away from pirate sites.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

On ISO standardization of blockchains

Post Syndicated from Robert Graham original http://blog.erratasec.com/2017/08/on-iso-standardization-of-blockchains.html

So ISO, the primary international standards organization, is seeking to standardize blockchain technologies. On the surface, this seems a reasonable idea, creating a common standard that everyone can interoperate with.

But it can be silly idea in practice. I mean, it should not be assumed that this is a good thing to do.

The value of official standards

You don’t need the official imprimatur of a government committee for something to be a “standard”. The Internet itself is a prime example of that.

In the 1980s, the ISO and the IETF (Internet Engineering Task Force) pursued competing standards for creating a world-wide “internet”. The IETF was an informal group of technologist that had essentially no official standing.

The ISO version of the Internet failed. Their process was to bring multiple stakeholders from business, government, and universities together in committees to debate competing interests. The result was something so horrible that it could never work in practice.

The IETF succeeded. It consisted of engineers just building things. Rather than officially “standardized”, these things were “described”, so that others knew enough to build their own version that interoperated. Once lots of different people built interoperating versions of something, then it became a “standard”.

In other words, the way the Internet came to be, standardization followed interoperability — it didn’t create interoperability.

In the end, the ISO gave up on their standards and adopted the IETF standards. The ISO brought no value to the development of Internet standards. Whether they ratified the Internet’s “TCP/IP” standard, ignored it, or condemned it, the Internet would exist today anyway, and a competing ISO-blessed internetwork would not.

The same question exists for blockchain technologies. Groups are off busy innovating quickly, creating their own standards. If the ISO blesses one, or creates its own, it’s unlikely to have any impact on interoperability.

Blockchain vs. chaining blocks

The excitement over blockchains is largely driven by people who don’t know the details, who don’t understand the difference between a blockchain like Bitcoin and the problem they are trying to solve.

Consider a record keeping system, especially public records. Storing them in a blockchain seems like a natural idea.

But in fact, it’s a terrible idea. A Bitcoin-style blockchain has a lot of features you don’t want, like “proof-of-work” signing. It is also missing necessary features, like bulk storage with redundancy (backups). Sure, Bitcoin has redundancy, but by brute force, storing the blockchain in thousands of places around the Internet. This is far from what a public records system would need, which would store a lot more data with far fewer backup copies (fewer than 10).

The only real overlap between Bitcoin and a public records system is a “signing chain”. But this is something that already existed before Bitcoin. It’s what Bitcoin blockchain was built on top of — it’s not the blockchain itself.

It’s like people discovering “cryptography” for the first time when they looked at Bitcoin, ignoring the thousand year history of crypto, and now every time they see a need for “crypto” they think “Bitcoin blockchain”.

Consensus and forking

The entire point of Bitcoin, the reason it was created, was as the antithesis to centralized standardization like ISO. Standardizing blockchains misses the entire point of their existence. The Bitcoin manifesto is that standardization comes from acclamation not proclamation, and that many different standards are preferable to a single one.

This is not just a theoretical idea but one built into Bitcoin’s blockchain technology. “Consensus” is achieved by the proof-of-work mechanism, so that those who do the most work are the ones that drive the consensus. When irreconcilable differences arise, the blockchain “forks”, with each side continuing on with their now non-interoperable blockchains. Such forks are not a sin, but part of the natural evolution.

We saw this with the recent fork of Bitcoin. There are now so many transactions that they exceed the size of blocks. One group chose a change to make transactions smaller. Another group chose a change to make block sizes larger.

It is this problem, of consensus, that is the innovation that Bitcoin created with blockchains, not the chain signing of public transaction records.

Ethereum

What “blockchain standardization” is going to mean in practice is not the blockchain itself, but trying to standardize the Ethereum version. What makes Ethereum different is the “smart contracts” programming language, which has financial institutions excited.

This is a bad idea because from a cybersecurity perspective, Ethereum’s programming language is flawed. Different bugs in “smart contracts” have led to multiple $100-million hacks, such as the infamous “DAO collapse”.

While it has interesting possibilities, we should be scared of standardizing Ethereum’s language before it works.

Conclusion

People who matter are too busy innovating, creating their own blockchain standards. There is little that the ISO can do to improve this. Their official imprimatur is not needed to foster innovation and interoperability — if they are consequential at anything, it’ll just be interfering.

Announcing Dedicated IP Pools

Post Syndicated from Brent Meyer original https://aws.amazon.com/blogs/ses/announcing-dedicated-ip-pools/

The Amazon SES team is pleased to announce that you can now create groups of dedicated IP addresses, called dedicated IP pools, for your email sending activities.

Prior to the availability of this feature, if you leased several dedicated IP addresses to use with Amazon SES, there was no way to specify which dedicated IP address to use for a specific email. Dedicated IP pools solve this problem by allowing you to send emails from specific IP addresses.

This post includes information and procedures related to dedicated IP pools.

What are dedicated IP pools?

In order to understand dedicated IP pools, you should first be familiar with the concept of dedicated IP addresses. Customers who send large volumes of email will typically lease one or more dedicated IP addresses to use when sending mail from Amazon SES. To learn more, see our blog post about dedicated IP addresses.

If you lease several dedicated IP addresses for use with Amazon SES, you can organize these addresses into groups, called pools. You can then associate each pool with a configuration set. When you send an email that specifies a configuration set, that email will be sent from the IP addresses in the associated pool.

When should I use dedicated IP pools?

Dedicated IP pools are especially useful for customers who send several different types of email using Amazon SES. For example, if you use Amazon SES to send both marketing emails and transactional emails, you can create a pool for marketing emails and another for transactional emails.

By using dedicated IP pools, you can isolate the sender reputations for each of these types of communications. Using dedicated IP pools gives you complete control over the sender reputations of the dedicated IP addresses you lease from Amazon SES.

How do I create and use dedicated IP pools?

There are two basic steps for creating and using dedicated IP pools. First, create a dedicated IP pool in the Amazon SES console and associate it with a configuration set. Next, when you send email, be sure to specify the configuration set associated with the IP pool you want to use.

For step-by-step procedures, see Creating Dedicated IP Pools in the Amazon SES Developer Guide.

Will my email sending process change?

If you do not use dedicated IP addresses with Amazon SES, then your email sending process will not change.

If you use dedicated IP pools, your email sending process may change slightly. In most cases, you will need to specify a configuration set in the emails you send. To learn more about using configuration sets, see Specifying a Configuration Set When You Send Email in the Amazon SES Developer Guide.

Any dedicated IP addresses that you lease that are not part of a dedicated IP pool will automatically be added to a default pool. If you send email without specifying a configuration set that is associated with a pool, then that email will be sent from one of the addresses in the default pool.

Dedicated IP pools are now available in the following AWS Regions: us-west-2 (Oregon), us-east-1 (Virginia), and eu-west-1 (Ireland).

We hope you enjoy this feature. If you have any questions or comments, please leave a comment on this post, or let us know in the Amazon SES Forum.

New – VPC Endpoints for DynamoDB

Post Syndicated from Randall Hunt original https://aws.amazon.com/blogs/aws/new-vpc-endpoints-for-dynamodb/

Starting today Amazon Virtual Private Cloud (VPC) Endpoints for Amazon DynamoDB are available in all public AWS regions. You can provision an endpoint right away using the AWS Management Console or the AWS Command Line Interface (CLI). There are no additional costs for a VPC Endpoint for DynamoDB.

Many AWS customers run their applications within a Amazon Virtual Private Cloud (VPC) for security or isolation reasons. Previously, if you wanted your EC2 instances in your VPC to be able to access DynamoDB, you had two options. You could use an Internet Gateway (with a NAT Gateway or assigning your instances public IPs) or you could route all of your traffic to your local infrastructure via VPN or AWS Direct Connect and then back to DynamoDB. Both of these solutions had security and throughput implications and it could be difficult to configure NACLs or security groups to restrict access to just DynamoDB. Here is a picture of the old infrastructure.

Creating an Endpoint

Let’s create a VPC Endpoint for DynamoDB. We can make sure our region supports the endpoint with the DescribeVpcEndpointServices API call.


aws ec2 describe-vpc-endpoint-services --region us-east-1
{
    "ServiceNames": [
        "com.amazonaws.us-east-1.dynamodb",
        "com.amazonaws.us-east-1.s3"
    ]
}

Great, so I know my region supports these endpoints and I know what my regional endpoint is. I can grab one of my VPCs and provision an endpoint with a quick call to the CLI or through the console. Let me show you how to use the console.

First I’ll navigate to the VPC console and select “Endpoints” in the sidebar. From there I’ll click “Create Endpoint” which brings me to this handy console.

You’ll notice the AWS Identity and Access Management (IAM) policy section for the endpoint. This supports all of the fine grained access control that DynamoDB supports in regular IAM policies and you can restrict access based on IAM policy conditions.

For now I’ll give full access to my instances within this VPC and click “Next Step”.

This brings me to a list of route tables in my VPC and asks me which of these route tables I want to assign my endpoint to. I’ll select one of them and click “Create Endpoint”.

Keep in mind the note of warning in the console: if you have source restrictions to DynamoDB based on public IP addresses the source IP of your instances accessing DynamoDB will now be their private IP addresses.

After adding the VPC Endpoint for DynamoDB to our VPC our infrastructure looks like this.

That’s it folks! It’s that easy. It’s provided at no cost. Go ahead and start using it today. If you need more details you can read the docs here.

Nazis, are bad

Post Syndicated from Eevee original https://eev.ee/blog/2017/08/13/nazis-are-bad/

Anonymous asks:

Could you talk about something related to the management/moderation and growth of online communities? IOW your thoughts on online community management, if any.

I think you’ve tweeted about this stuff in the past so I suspect you have thoughts on this, but if not, again, feel free to just blog about … anything 🙂

Oh, I think I have some stuff to say about community management, in light of recent events. None of it hasn’t already been said elsewhere, but I have to get this out.

Hopefully the content warning is implicit in the title.


I am frustrated.

I’ve gone on before about a particularly bothersome phenomenon that hurts a lot of small online communities: often, people are willing to tolerate the misery of others in a community, but then get up in arms when someone pushes back. Someone makes a lot of off-hand, off-color comments about women? Uses a lot of dog-whistle terms? Eh, they’re not bothering anyone, or at least not bothering me. Someone else gets tired of it and tells them to knock it off? Whoa there! Now we have the appearance of conflict, which is unacceptable, and people will turn on the person who’s pissed off — even though they’ve been at the butt end of an invisible conflict for who knows how long. The appearance of peace is paramount, even if it means a large chunk of the population is quietly miserable.

Okay, so now, imagine that on a vastly larger scale, and also those annoying people who know how to skirt the rules are Nazis.


The label “Nazi” gets thrown around a lot lately, probably far too easily. But when I see a group of people doing the Hitler salute, waving large Nazi flags, wearing Nazi armbands styled after the SS, well… if the shoe fits, right? I suppose they might have flown across the country to join a torch-bearing mob ironically, but if so, the joke is going way over my head. (Was the murder ironic, too?) Maybe they’re not Nazis in the sense that the original party doesn’t exist any more, but for ease of writing, let’s refer to “someone who espouses Nazi ideology and deliberately bears a number of Nazi symbols” as, well, “a Nazi”.

This isn’t a new thing, either; I’ve stumbled upon any number of Twitter accounts that are decorated in Nazi regalia. I suppose the trouble arises when perfectly innocent members of the alt-right get unfairly labelled as Nazis.

But hang on; this march was called “Unite the Right” and was intended to bring together various far right sub-groups. So what does their choice of aesthetic say about those sub-groups? I haven’t heard, say, alt-right coiner Richard Spencer denounce the use of Nazi symbology — extra notable since he was fucking there and apparently didn’t care to discourage it.


And so begins the rule-skirting. “Nazi” is definitely overused, but even using it to describe white supremacists who make not-so-subtle nods to Hitler is likely to earn you some sarcastic derailment. A Nazi? Oh, so is everyone you don’t like and who wants to establish a white ethno state a Nazi?

Calling someone a Nazi — or even a white supremacist — is an attack, you see. Merely expressing the desire that people of color not exist is perfectly peaceful, but identifying the sentiment for what it is causes visible discord, which is unacceptable.

These clowns even know this sort of thing and strategize around it. Or, try, at least. Maybe it wasn’t that successful this weekend — though flicking through Charlottesville headlines now, they seem to be relatively tame in how they refer to the ralliers.

I’m reminded of a group of furries — the alt-furries — who have been espousing white supremacy and wearing red armbands with a white circle containing a black… pawprint. Ah, yes, that’s completely different.


So, what to do about this?

Ignore them” is a popular option, often espoused to bullied children by parents who have never been bullied, shortly before they resume complaining about passive-aggressive office politics. The trouble with ignoring them is that, just like in smaller communitiest, they have a tendency to fester. They take over large chunks of influential Internet surface area like 4chan and Reddit; they help get an inept buffoon elected; and then they start to have torch-bearing rallies and run people over with cars.

4chan illustrates a kind of corollary here. Anyone who’s steeped in Internet Culture™ is surely familiar with 4chan; I was never a regular visitor, but it had enough influence that I was still aware of it and some of its culture. It was always thick with irony, which grew into a sort of ironic detachment — perhaps one of the major sources of the recurring online trope that having feelings is bad — which proceeded into ironic racism.

And now the ironic racism is indistinguishable from actual racism, as tends to be the case. Do they “actually” “mean it”, or are they just trying to get a rise out of people? What the hell is unironic racism if not trying to get a rise out of people? What difference is there to onlookers, especially as they move to become increasingly involved with politics?

It’s just a joke” and “it was just a thoughtless comment” are exceptionally common defenses made by people desperate to preserve the illusion of harmony, but the strain of overt white supremacy currently running rampant through the US was built on those excuses.


The other favored option is to debate them, to defeat their ideas with better ideas.

Well, hang on. What are their ideas, again? I hear they were chanting stuff like “go back to Africa” and “fuck you, faggots”. Given that this was an overtly political rally (and again, the Nazi fucking regalia), I don’t think it’s a far cry to describe their ideas as “let’s get rid of black people and queer folks”.

This is an underlying proposition: that white supremacy is inherently violent. After all, if the alt-right seized total political power, what would they do with it? If I asked the same question of Democrats or Republicans, I’d imagine answers like “universal health care” or “screw over poor people”. But people whose primary goal is to have a country full of only white folks? What are they going to do, politely ask everyone else to leave? They’re invoking the memory of people who committed genocide and also tried to take over the fucking world. They are outright saying, these are the people we look up to, this is who we think had a great idea.

How, precisely, does one defeat these ideas with rational debate?

Because the underlying core philosophy beneath all this is: “it would be good for me if everything were about me”. And that’s true! (Well, it probably wouldn’t work out how they imagine in practice, but it’s true enough.) Consider that slavery is probably fantastic if you’re the one with the slaves; the issue is that it’s reprehensible, not that the very notion contains some kind of 101-level logical fallacy. That’s probably why we had a fucking war over it instead of hashing it out over brunch.

…except we did hash it out over brunch once, and the result was that slavery was still allowed but slaves only counted as 60% of a person for the sake of counting how much political power states got. So that’s how rational debate worked out. I’m sure the slaves were thrilled with that progress.


That really only leaves pushing back, which raises the question of how to push back.

And, I don’t know. Pushing back is much harder in spaces you don’t control, spaces you’re already struggling to justify your own presence in. For most people, that’s most spaces. It’s made all the harder by that tendency to preserve illusory peace; even the tamest request that someone knock off some odious behavior can be met by pushback, even by third parties.

At the same time, I’m aware that white supremacists prey on disillusioned young white dudes who feel like they don’t fit in, who were promised the world and inherited kind of a mess. Does criticism drive them further away? The alt-right also opposes “political correctness”, i.e. “not being a fucking asshole”.

God knows we all suck at this kind of behavior correction, even within our own in-groups. Fandoms have become almost ridiculously vicious as platforms like Twitter and Tumblr amplify individual anger to deafening levels. It probably doesn’t help that we’re all just exhausted, that every new fuck-up feels like it bears the same weight as the last hundred combined.

This is the part where I admit I don’t know anything about people and don’t have any easy answers. Surprise!


The other alternative is, well, punching Nazis.

That meme kind of haunts me. It raises really fucking complicated questions about when violence is acceptable, in a culture that’s completely incapable of answering them.

America’s relationship to violence is so bizarre and two-faced as to be almost incomprehensible. We worship it. We have the biggest military in the world by an almost comical margin. It’s fairly mainstream to own deadly weapons for the express stated purpose of armed revolution against the government, should that become necessary, where “necessary” is left ominously undefined. Our movies are about explosions and beating up bad guys; our video games are about explosions and shooting bad guys. We fantasize about solving foreign policy problems by nuking someone — hell, our talking heads are currently in polite discussion about whether we should nuke North Korea and annihilate up to twenty-five million people, as punishment for daring to have the bomb that only we’re allowed to have.

But… violence is bad.

That’s about as far as the other side of the coin gets. It’s bad. We condemn it in the strongest possible terms. Also, guess who we bombed today?

I observe that the one time Nazis were a serious threat, America was happy to let them try to take over the world until their allies finally showed up on our back porch.

Maybe I don’t understand what “violence” means. In a quest to find out why people are talking about “leftist violence” lately, I found a National Review article from May that twice suggests blocking traffic is a form of violence. Anarchists have smashed some windows and set a couple fires at protests this year — and, hey, please knock that crap off? — which is called violence against, I guess, Starbucks. Black Lives Matter could be throwing a birthday party and Twitter would still be abuzz with people calling them thugs.

Meanwhile, there’s a trend of murderers with increasingly overt links to the alt-right, and everyone is still handling them with kid gloves. First it was murders by people repeating their talking points; now it’s the culmination of a torches-and-pitchforks mob. (Ah, sorry, not pitchforks; assault rifles.) And we still get this incredibly bizarre both-sides-ism, a White House that refers to the people who didn’t murder anyone as “just as violent if not more so“.


Should you punch Nazis? I don’t know. All I know is that I’m extremely dissatisfied with discourse that’s extremely alarmed by hypothetical punches — far more mundane than what you’d see after a sporting event — but treats a push for ethnic cleansing as a mere difference of opinion.

The equivalent to a punch in an online space is probably banning, which is almost laughable in comparison. It doesn’t cause physical harm, but it is a use of concrete force. Doesn’t pose quite the same moral quandary, though.

Somewhere in the middle is the currently popular pastime of doxxing (doxxxxxxing) people spotted at the rally in an attempt to get them fired or whatever. Frankly, that skeeves me out, though apparently not enough that I’m directly chastizing anyone for it.


We aren’t really equipped, as a society, to deal with memetic threats. We aren’t even equipped to determine what they are. We had a fucking world war over this, and now people are outright saying “hey I’m like those people we went and killed a lot in that world war” and we give them interviews and compliment their fashion sense.

A looming question is always, what if they then do it to you? What if people try to get you fired, to punch you for your beliefs?

I think about that a lot, and then I remember that it’s perfectly legal to fire someone for being gay in half the country. (Courts are currently wrangling whether Title VII forbids this, but with the current administration, I’m not optimistic.) I know people who’ve been fired for coming out as trans. I doubt I’d have to look very far to find someone who’s been punched for either reason.

And these aren’t even beliefs; they’re just properties of a person. You can stop being a white supremacist, one of those people yelling “fuck you, faggots”.

So I have to recuse myself from this asinine question, because I can’t fairly judge the risk of retaliation when it already happens to people I care about.

Meanwhile, if a white supremacist does get punched, I absolutely still want my tax dollars to pay for their universal healthcare.


The same wrinkle comes up with free speech, which is paramount.

The ACLU reminds us that the First Amendment “protects vile, hateful, and ignorant speech”. I think they’ve forgotten that that’s a side effect, not the goal. No one sat down and suggested that protecting vile speech was some kind of noble cause, yet that’s how we seem to be treating it.

The point was to avoid a situation where the government is arbitrarily deciding what qualifies as vile, hateful, and ignorant, and was using that power to eliminate ideas distasteful to politicians. You know, like, hypothetically, if they interrogated and jailed a bunch of people for supporting the wrong economic system. Or convicted someone under the Espionage Act for opposing the draft. (Hey, that’s where the “shouting fire in a crowded theater” line comes from.)

But these are ideas that are already in the government. Bannon, a man who was chair of a news organization he himself called “the platform for the alt-right”, has the President’s ear! How much more mainstream can you get?

So again I’m having a little trouble balancing “we need to defend the free speech of white supremacists or risk losing it for everyone” against “we fairly recently were ferreting out communists and the lingering public perception is that communists are scary, not that the government is”.


This isn’t to say that freedom of speech is bad, only that the way we talk about it has become fanatical to the point of absurdity. We love it so much that we turn around and try to apply it to corporations, to platforms, to communities, to interpersonal relationships.

Look at 4chan. It’s completely public and anonymous; you only get banned for putting the functioning of the site itself in jeopardy. Nothing is stopping a larger group of people from joining its politics board and tilting sentiment the other way — except that the current population is so odious that no one wants to be around them. Everyone else has evaporated away, as tends to happen.

Free speech is great for a government, to prevent quashing politics that threaten the status quo (except it’s a joke and they’ll do it anyway). People can’t very readily just bail when the government doesn’t like them, anyway. It’s also nice to keep in mind to some degree for ubiquitous platforms. But the smaller you go, the easier it is for people to evaporate away, and the faster pure free speech will turn the place to crap. You’ll be left only with people who care about nothing.


At the very least, it seems clear that the goal of white supremacists is some form of destabilization, of disruption to the fabric of a community for purely selfish purposes. And those are the kinds of people you want to get rid of as quickly as possible.

Usually this is hard, because they act just nicely enough to create some plausible deniability. But damn, if someone is outright telling you they love Hitler, maybe skip the principled hand-wringing and eject them.

Controlling Millions of Potential Internet Pirates Won’t Be Easy

Post Syndicated from Andy original https://torrentfreak.com/controlling-millions-of-potential-internet-pirates-wont-be-easy-170813/

For several decades the basic shape of the piracy market hasn’t changed much. At the top of the chain there has always been a relatively small number of suppliers. At the bottom, the sprawling masses keen to consume whatever content these suppliers make available, while sharing it with everyone else.

This model held in the days of tapes and CDs and transferred nicely to the P2P file-sharing era. For nearly two decades people have been waiting for those with the latest content to dump it onto file-sharing networks. After grabbing it for themselves, people share that content with others.

For many years, the majority of the latest music, movies, and TV shows appeared online having been obtained by, and then leaked from, ‘The Scene’. However, with the rise of BitTorrent and an increase in computer skills demonstrated by the public, so-called ‘P2P release groups’ began flexing their muscles, in some cases slicing the top of the piracy pyramid.

With lower barriers to entry, P2P releasers can be almost anyone who happens to stumble across some new content. That being said, people still need the skill to package up that content and make it visible online, on torrent sites for example, without getting caught.

For most people that’s prohibitively complex, so it’s no surprise that Average Joe, perhaps comforted by the air of legitimacy, has taken to uploading music and movies to sites like YouTube instead. These days that’s nothing out of the ordinary and perhaps a little boring by piracy standards, but people still have the capacity to surprise.

This week a man from the United States, without a care in the world, obtained a login for a STARZ press portal, accessed the final three episodes of ‘Power’, and then streamed them on Facebook using nothing but a phone and an Internet connection.

From the beginning, the whole thing was ridiculous, comical even. The man in question, whose name and personal details TF obtained in a matter of minutes, revealed how he got the logins and even recorded his own face during one of the uploaded videos.

He really, really couldn’t have cared any less but he definitely should have. After news broke of the leaks, STARZ went public confirming the breach and promising to do something about it.

“The final three episodes of Power’s fourth season were leaked online due to a breach of the press screening room,” Starz said in a statement. “Starz has begun forensic investigations and will take legal action against the responsible parties.”

At this point, we should consider the magnitude of what this guy did. While we all laugh at his useless camera skills, the fact remains that he unlawfully distributed copyright works online, in advance of their commercial release. In the United States, that is a criminal offense, one that can result in a prison sentence of several years.

It would be really sad if the guy in question was made an example of since his videos suggest he hadn’t considered the consequences. After all, this wasn’t some hi-tech piracy group, just a regular guy with a login and a phone, and intent always counts for something. Nevertheless, the situation this week nicely highlights how new technology affects piracy.

In the past, the process of putting an unreleased movie or TV show online could only be tackled by people with expertise in several areas. These days a similar effect is possible with almost no skill and no effort. Joe Public, pre-release TV/movie/sports pirate, using nothing but a phone, a Facebook account, and an urge?

That’s the reality today and we won’t have to wait too long for a large scale demonstration of what can happen when millions of people with access to these ubiquitous tools have an urge to share.

In a little over two weeks’ time, boxing legend Floyd Mayweather Jr fights UFC lightweight champion, Conor McGregor. It’s set to be the richest combat sports event in history, not to mention one of the most expensive for PPV buyers. That means it’s going to be pirated to hell and back, in every way possible. It’s going to be massive.

Of course, there will be high-quality paid IPTV productions available, more grainy ‘Kodi’ streams, hundreds of web portals, and even some streaming torrents, for those that way inclined. But there will also be Average Joes in their hundreds, who will point their phones at Showtime’s PPV with the intent of live streaming the biggest show on earth to their friends, family, and the Internet. For free.

Quite how this will be combatted remains to be seen but it’s fair to say that this is a problem that’s only going to get bigger. In ten years time – in five years time – many millions of people will have the ability to become pirate releasers on a whim, despite knowing nothing about the occupation.

Like ‘Power’ guy, the majority won’t be very good at it. Equally, some will turn it into an art form. But whatever happens, tackling millions of potential pirates definitely won’t be easy for copyright holders. Twenty years in, it seems the battle for control has only just begun.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

Piracy Narrative Isn’t About Ethics Anymore, It’s About “Danger”

Post Syndicated from Andy original https://torrentfreak.com/piracy-narrative-isnt-about-ethics-anymore-its-about-danger-170812/

Over the years there have been almost endless attempts to stop people from accessing copyright-infringing content online. Campaigns have come and gone and almost two decades later the battle is still ongoing.

Early on, when panic enveloped the music industry, the campaigns centered around people getting sued. Grabbing music online for free could be costly, the industry warned, while parading the heads of a few victims on pikes for the world to see.

Periodically, however, the aim has been to appeal to the public’s better nature. The idea is that people essentially want to do the ‘right thing’, so once they understand that largely hard-working Americans are losing their livelihoods, people will stop downloading from The Pirate Bay. For some, this probably had the desired effect but millions of people are still getting their fixes for free, so the job isn’t finished yet.

In more recent years, notably since the MPAA and RIAA had their eyes blacked in the wake of SOPA, the tone has shifted. In addition to educating the public, torrent and streaming sites are increasingly being painted as enemies of the public they claim to serve.

Several studies, largely carried out on behalf of the Digital Citizens Alliance (DCA), have claimed that pirate sites are hotbeds of malware, baiting consumers in with tasty pirate booty only to offload trojans, viruses, and God-knows-what. These reports have been ostensibly published as independent public interest documents but this week an advisor to the DCA suggested a deeper interest for the industry.

Hemanshu Nigam is a former federal prosecutor, ex-Chief Security Officer for News Corp and Fox Interactive Media, and former VP Worldwide Internet Enforcement at the MPAA. In an interview with Deadline this week, he spoke about alleged links between pirate sites and malware distributors. He also indicated that warning people about the dangers of pirate sites has become Hollywood’s latest anti-piracy strategy.

“The industry narrative has changed. When I was at the MPAA, we would tell people that stealing content is wrong and young people would say, yeah, whatever, you guys make a lot of money, too bad,” he told the publication.

“It has gone from an ethical discussion to a dangerous one. Now, your parents’ bank account can be raided, your teenage daughter can be spied on in her bedroom and extorted with the footage, or your computer can be locked up along with everything in it and held for ransom.”

Nigam’s stance isn’t really a surprise since he’s currently working for the Digital Citizens Alliance as an advisor. In turn, the Alliance is at least partly financed by the MPAA. There’s no suggestion whatsoever that Nigam is involved in any propaganda effort, but recent signs suggest that the DCA’s work in malware awareness is more about directing people away from pirate sites than protecting them from the alleged dangers within.

That being said and despite the bias, it’s still worth giving experts like Nigam an opportunity to speak. Largely thanks to industry efforts with brands, pirate sites are increasingly being forced to display lower-tier ads, which can be problematic. On top, some sites’ policies mean they don’t deserve any visitors at all.

In the Deadline piece, however, Nigam alleges that hackers have previously reached out to pirate websites offering $200 to $5000 per day “depending on the size of the pirate website” to have the site infect users with malware. If true, that’s a serious situation and people who would ordinarily use ‘pirate’ sites would definitely appreciate the details.

For example, to which sites did hackers make this offer and, crucially, which sites turned down the offer and which ones accepted?

It’s important to remember that pirates are just another type of consumer and they would boycott sites in a heartbeat if they discovered they’d been paid to infect them with malware. But, as usual, the claims are extremely light in detail. Instead, there’s simply a blanket warning to stay away from all unauthorized sites, which isn’t particularly helpful.

In some cases, of course, operational security will prevent some details coming to light but without these, people who don’t get infected on a ‘pirate’ site (the vast majority) simply won’t believe the allegations. As the author of the Deadline piece pointed out, it’s a bit like Reefer Madness all over again.

The point here is that without hard independent evidence to back up these claims, with reports listing sites alongside the malware they’ve supposed to have spread and when, few people will respond to perceived scaremongering. Free content trumps a few distant worries almost every time, whether that involves malware or the threat of a lawsuit.

It’ll be up to the DCA and their MPAA paymasters to consider whether the approach is working but thus far, not even having government heavyweights on board has helped.

Earlier this year the DCA launched a video campaign, enrolling 15 attorney generals to publish their own anti-piracy PSAs on YouTube. Thus far, interest has been minimal, to say the least.

At the time of writing the 15 PSAs have 3,986 views in total, with 2,441 of those contributed by a single video contributed by Wisconsin Attorney General Brad Schimel. Despite the relative success, even that got slammed with 2 upvotes and 127 downvotes.

A few of the other videos have a couple of hundred views each but more than half have less than 70. Perhaps most worryingly for the DCA, apart from the Schimel PSA, none have any upvotes at all, only down. It’s unclear who the viewers were but it seems reasonable to conclude they weren’t entertained.

The bottom line is nobody likes malware or having their banking details stolen but yet again, people who claim to have the public interest at heart aren’t actually making a difference on the ground. It could be argued that groups advocating online safety should be publishing guides on how to stay protected on the Internet period, not merely advising people to stay away from certain sites.

But of course, that wouldn’t achieve the goals of the MPAA Digital Citizens Alliance.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

Usenet Pirate Pays €4,800 ‘Fine’ After Being Exposed by Provider

Post Syndicated from Ernesto original https://torrentfreak.com/usenet-pirate-pays-e4800-fine-after-being-exposed-by-provider-170811/

Dutch anti-piracy outfit BREIN has been very active over the past several years, targeting uploaders on various sharing sites and services.

They cast their net wide and have gone after torrent users, Facebook groups, YouTube pirates and Usenet uploaders as well.

To pinpoint the latter group, BREIN contacts Usenet providers asking them to reveal the identity of a suspected user. This is also what happened in a case involving a former customer of Eweka.

The person in question, known under the alias ‘Badfan69,’ was accused of uploading 9,538 infringing works to Usenet, mostly older titles. After Eweka handed over his home address, BREIN reached out to him and negotiated a settlement.

The 44-year-old man has now agreed to pay a settlement of €4,800. If he continues to upload infringing content he will face an additional penalty of €2,000 per day, to a maximum of €50,000.

The case is an important victory for BREIN, not just because of the money.

When the anti-piracy group reached out to Usenet provider Eweka, the company initially refused to hand over any personal details. The Usenet provider argued that it’s a neutral intermediary that would rather not perform the role of piracy police. Instead, it wanted the court to decide whether the request was legitimate.

This resulted in a legal dispute where, earlier this year, a local court sided with BREIN. The Court stressed that in these type of copyright infringement cases, the Usenet provider is required to hand over the requested details.

Under Dutch law, ISPs can be obliged to hand over the personal details of their customers if the infringing activity is plausible and the damaged party has a legitimate interest. Importantly, the legal case clarified that this generally doesn’t require an intervention from the court.

“Providers must decide on a motivated request for the handover of a user’s address, based on their own consideration. A refusal to provide the information must be motivated, otherwise, it will be illegal and the provider will be charged for the costs,” BREIN notes.

While these Usenet cases are relatively rare, BREIN and other parties in the Netherlands, such as Dutch Filmworks, are also planning to go after large groups of torrent users. With the Usenet decision in hand, BREIN may want to argue that regular ISPs must also expose pirating users, without an intervention of the court.

This is not going to happen easily though. Several ISPs, most prominently Ziggo, announced that they would not voluntarily cooperate and are likely to fight out these requests in court to get a solid ‘torrent’ precedent.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

Automating Blue/Green Deployments of Infrastructure and Application Code using AMIs, AWS Developer Tools, & Amazon EC2 Systems Manager

Post Syndicated from Ramesh Adabala original https://aws.amazon.com/blogs/devops/bluegreen-infrastructure-application-deployment-blog/

Previous DevOps blog posts have covered the following use cases for infrastructure and application deployment automation:

An AMI provides the information required to launch an instance, which is a virtual server in the cloud. You can use one AMI to launch as many instances as you need. It is security best practice to customize and harden your base AMI with required operating system updates and, if you are using AWS native services for continuous security monitoring and operations, you are strongly encouraged to bake into the base AMI agents such as those for Amazon EC2 Systems Manager (SSM), Amazon Inspector, CodeDeploy, and CloudWatch Logs. A customized and hardened AMI is often referred to as a “golden AMI.” The use of golden AMIs to create EC2 instances in your AWS environment allows for fast and stable application deployment and scaling, secure application stack upgrades, and versioning.

In this post, using the DevOps automation capabilities of Systems Manager, AWS developer tools (CodePipeLine, CodeDeploy, CodeCommit, CodeBuild), I will show you how to use AWS CodePipeline to orchestrate the end-to-end blue/green deployments of a golden AMI and application code. Systems Manager Automation is a powerful security feature for enterprises that want to mature their DevSecOps practices.

Here are the high-level phases and primary services covered in this use case:

 

You can access the source code for the sample used in this post here: https://github.com/awslabs/automating-governance-sample/tree/master/Bluegreen-AMI-Application-Deployment-blog.

This sample will create a pipeline in AWS CodePipeline with the building blocks to support the blue/green deployments of infrastructure and application. The sample includes a custom Lambda step in the pipeline to execute Systems Manager Automation to build a golden AMI and update the Auto Scaling group with the golden AMI ID for every rollout of new application code. This guarantees that every new application deployment is on a fully patched and customized AMI in a continuous integration and deployment model. This enables the automation of hardened AMI deployment with every new version of application deployment.

 

 

We will build and run this sample in three parts.

Part 1: Setting up the AWS developer tools and deploying a base web application

Part 1 of the AWS CloudFormation template creates the initial Java-based web application environment in a VPC. It also creates all the required components of Systems Manager Automation, CodeCommit, CodeBuild, and CodeDeploy to support the blue/green deployments of the infrastructure and application resulting from ongoing code releases.

Part 1 of the AWS CloudFormation stack creates these resources:

After Part 1 of the AWS CloudFormation stack creation is complete, go to the Outputs tab and click the Elastic Load Balancing link. You will see the following home page for the base web application:

Make sure you have all the outputs from the Part 1 stack handy. You need to supply them as parameters in Part 3 of the stack.

Part 2: Setting up your CodeCommit repository

In this part, you will commit and push your sample application code into the CodeCommit repository created in Part 1. To access the initial git commands to clone the empty repository to your local machine, click Connect to go to the AWS CodeCommit console. Make sure you have the IAM permissions required to access AWS CodeCommit from command line interface (CLI).

After you’ve cloned the repository locally, download the sample application files from the part2 folder of the Git repository and place the files directly into your local repository. Do not include the aws-codedeploy-sample-tomcat folder. Go to the local directory and type the following commands to commit and push the files to the CodeCommit repository:

git add .
git commit -a -m "add all files from the AWS Java Tomcat CodeDeploy application"
git push

After all the files are pushed successfully, the repository should look like this:

 

Part 3: Setting up CodePipeline to enable blue/green deployments     

Part 3 of the AWS CloudFormation template creates the pipeline in AWS CodePipeline and all the required components.

a) Source: The pipeline is triggered by any change to the CodeCommit repository.

b) BuildGoldenAMI: This Lambda step executes the Systems Manager Automation document to build the golden AMI. After the golden AMI is successfully created, a new launch configuration with the new AMI details will be updated into the Auto Scaling group of the application deployment group. You can watch the progress of the automation in the EC2 console from the Systems Manager –> Automations menu.

c) Build: This step uses the application build spec file to build the application build artifact. Here are the CodeBuild execution steps and their status:

d) Deploy: This step clones the Auto Scaling group, launches the new instances with the new AMI, deploys the application changes, reroutes the traffic from the elastic load balancer to the new instances and terminates the old Auto Scaling group. You can see the execution steps and their status in the CodeDeploy console.

After the CodePipeline execution is complete, you can access the application by clicking the Elastic Load Balancing link. You can find it in the output of Part 1 of the AWS CloudFormation template. Any consecutive commits to the application code in the CodeCommit repository trigger the pipelines and deploy the infrastructure and code with an updated AMI and code.

 

If you have feedback about this post, add it to the Comments section below. If you have questions about implementing the example used in this post, open a thread on the Developer Tools forum.


About the author

 

Ramesh Adabala is a Solutions Architect in Southeast Enterprise Solution Architecture team at Amazon Web Services.

‘US Should Include Fair Use and Safe Harbors in NAFTA Negotiations’

Post Syndicated from Ernesto original https://torrentfreak.com/us-should-include-fair-use-and-safe-harbors-in-nafta-negotiations-170806/

The North American Free Trade Agreement (NAFTA) between the United States, Canada, and Mexico was negotiated more than 25 years ago.

Over the past quarter century trade has changed drastically, especially online, so the United States is now planning to modernize the international deal.

Various copyright industry groups recognized this as an opportunity to demand tougher copyright enforcement. The MPAA and RIAA previously presented their demands, proposing various new limitations, including restrictions to the existing safe harbor protections against copyright infringement claims.

While no concrete plans have been made public yet, the U.S Trade Representative (USTR) recently gave an overview of its NAFTA renegotiation objectives. The language leaves plenty of wiggle room, but it’s clear that strong copyright enforcement takes a central role.

“Provide strong protection and enforcement for new and emerging technologies and new methods of transmitting and distributing products embodying intellectual property, including in a manner that facilitates legitimate digital trade,” one of the key points reads.

It is no surprise that copyright enforcement plays a central role in a possible extension of NAFTA. However, according to the Re:Create Coalition, which includes members such as the the Consumer Technology Association, the American Library Association and EFF, future proposals should be more balanced.

This means that if copyright enforcement is included, the US Government should also make sure that fair use, safe harbor protections and other copyright limitations and exceptions are added as well.

“The United States government should promote balance in copyright law to unlock the fullest potential of innovation and creativity globally, and to help U.S. innovators, creators, and small businesses reach foreign audiences.” Re:Create Executive Director Josh Lamel tells TorrentFreak.

“If a re-negotiated NAFTA includes a chapter on copyright, which seems likely, it must have mandatory language on copyright limitations and exceptions, including fair use and protections from intermediary liability.”

The USTR stressed that the NAFTA agreement should cover copyright protections similar to those found in US law. If that is the case, the coalition urges the US Government to ‘export’ fair use and other copyright limitations as well, to keep the balance.

Strong enforcement without balance could lead to all sorts of abuse, according to the Re:Create coalition. Just recently, a Colombian student faced a hefty prison sentence for sharing a research paper on Scribd, something which would be less likely with a proper fair use defense.

“Trade agreements should reflect the realities of the world we live in today. If strong intellectual property protections and enforcement measures are included in a trade agreement, so should exceptions and limitations to copyright law,” Lamel says.

“You can’t have one without the other. Furthermore, the copyright system cannot function effectively without fair use, and neither can the U.S. economy. 16 percent of the U.S. economy depends on fair use, and 18 million U.S. workers across the country are employed in fair use industries.”

In addition to fair use, Re:Create argues that DMCA-style safe harbor provisions are essential for Internet services to operate freely on the Internet. The RIAA wants to restrict safe harbor protection to limit copyright infringement and abuse, but the coalition believes that these proposals go too far.

If the RIAA had its way, many large Internet service providers wouldn’t be able to operate freely. This would result in a loss of American jobs, and innovation would be stifled, Re:Create notes.

“If you looked up excessive overreach in the dictionary, there would be a picture of the RIAA and MPAA submissions. Limiting safe harbors would be corporate cronyism at its worst,” Lamel tells TorrentFreak.

“The safe harbors are at the cornerstone of the Internet economy and consumer Internet experience. It would be an economic disaster. Recent economic analysis found that weakened safe harbors would result in the loss of 4.25 million American jobs and cost nearly half a trillion dollars over the next decade,” he adds.

While it’s still early days, it will be interesting to see what concrete proposals will come out of the negotiations and if fair use and other copyright protections are indeed going to be included. Re-Create promises to keep a close eye on the developments, and they’re certainly not alone.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

Turbocharge your Apache Hive queries on Amazon EMR using LLAP

Post Syndicated from Jigar Mistry original https://aws.amazon.com/blogs/big-data/turbocharge-your-apache-hive-queries-on-amazon-emr-using-llap/

Apache Hive is one of the most popular tools for analyzing large datasets stored in a Hadoop cluster using SQL. Data analysts and scientists use Hive to query, summarize, explore, and analyze big data.

With the introduction of Hive LLAP (Low Latency Analytical Processing), the notion of Hive being just a batch processing tool has changed. LLAP uses long-lived daemons with intelligent in-memory caching to circumvent batch-oriented latency and provide sub-second query response times.

This post provides an overview of Hive LLAP, including its architecture and common use cases for boosting query performance. You will learn how to install and configure Hive LLAP on an Amazon EMR cluster and run queries on LLAP daemons.

What is Hive LLAP?

Hive LLAP was introduced in Apache Hive 2.0, which provides very fast processing of queries. It uses persistent daemons that are deployed on a Hadoop YARN cluster using Apache Slider. These daemons are long-running and provide functionality such as I/O with DataNode, in-memory caching, query processing, and fine-grained access control. And since the daemons are always running in the cluster, it saves substantial overhead of launching new YARN containers for every new Hive session, thereby avoiding long startup times.

When Hive is configured in hybrid execution mode, small and short queries execute directly on LLAP daemons. Heavy lifting (like large shuffles in the reduce stage) is performed in YARN containers that belong to the application. Resources (CPU, memory, etc.) are obtained in a traditional fashion using YARN. After the resources are obtained, the execution engine can decide which resources are to be allocated to LLAP, or it can launch Apache Tez processors in separate YARN containers. You can also configure Hive to run all the processing workloads on LLAP daemons for querying small datasets at lightning fast speeds.

LLAP daemons are launched under YARN management to ensure that the nodes don’t get overloaded with the compute resources of these daemons. You can use scheduling queues to make sure that there is enough compute capacity for other YARN applications to run.

Why use Hive LLAP?

With many options available in the market (Presto, Spark SQL, etc.) for doing interactive SQL  over data that is stored in Amazon S3 and HDFS, there are several reasons why using Hive and LLAP might be a good choice:

  • For those who are heavily invested in the Hive ecosystem and have external BI tools that connect to Hive over JDBC/ODBC connections, LLAP plugs in to their existing architecture without a steep learning curve.
  • It’s compatible with existing Hive SQL and other Hive tools, like HiveServer2, and JDBC drivers for Hive.
  • It has native support for security features with authentication and authorization (SQL standards-based authorization) using HiveServer2.
  • LLAP daemons are aware about of the columns and records that are being processed which enables you to enforce fine-grained access control.
  • It can use Hive’s vectorization capabilities to speed up queries, and Hive has better support for Parquet file format when vectorization is enabled.
  • It can take advantage of a number of Hive optimizations like merging multiple small files for query results, automatically determining the number of reducers for joins and groupbys, etc.
  • It’s optional and modular so it can be turned on or off depending on the compute and resource requirements of the cluster. This lets you to run other YARN applications concurrently without reserving a cluster specifically for LLAP.

How do you install Hive LLAP in Amazon EMR?

To install and configure LLAP on an EMR cluster, use the following bootstrap action (BA):

s3://aws-bigdata-blog/artifacts/Turbocharge_Apache_Hive_on_EMR/configure-Hive-LLAP.sh

This BA downloads and installs Apache Slider on the cluster and configures LLAP so that it works with EMR Hive. For LLAP to work, the EMR cluster must have Hive, Tez, and Apache Zookeeper installed.

You can pass the following arguments to the BA.

Argument Definition Default value
--instances Number of instances of LLAP daemon Number of core/task nodes of the cluster
--cache Cache size per instance 20% of physical memory of the node
--executors Number of executors per instance Number of CPU cores of the node
--iothreads Number of IO threads per instance Number of CPU cores of the node
--size Container size per instance 50% of physical memory of the node
--xmx Working memory size 50% of container size
--log-level Log levels for the LLAP instance INFO

LLAP example

This section describes how you can try the faster Hive queries with LLAP using the TPC-DS testbench for Hive on Amazon EMR.

Use the following AWS command line interface (AWS CLI) command to launch a 1+3 nodes m4.xlarge EMR 5.6.0 cluster with the bootstrap action to install LLAP:

aws emr create-cluster --release-label emr-5.6.0 \
--applications Name=Hadoop Name=Hive Name=Hue Name=ZooKeeper Name=Tez \
--bootstrap-actions '[{"Path":"s3://aws-bigdata-blog/artifacts/Turbocharge_Apache_Hive_on_EMR/configure-Hive-LLAP.sh","Name":"Custom action"}]' \ 
--ec2-attributes '{"KeyName":"<YOUR-KEY-PAIR>","InstanceProfile":"EMR_EC2_DefaultRole","SubnetId":"subnet-xxxxxxxx","EmrManagedSlaveSecurityGroup":"sg-xxxxxxxx","EmrManagedMasterSecurityGroup":"sg-xxxxxxxx"}' 
--service-role EMR_DefaultRole \
--enable-debugging \
--log-uri 's3n://<YOUR-BUCKET/' --name 'test-hive-llap' \
--instance-groups '[{"InstanceCount":1,"EbsConfiguration":{"EbsBlockDeviceConfigs":[{"VolumeSpecification":{"SizeInGB":32,"VolumeType":"gp2"},"VolumesPerInstance":1}],"EbsOptimized":true},"InstanceGroupType":"MASTER","InstanceType":"m4.xlarge","Name":"Master - 1"},{"InstanceCount":3,"EbsConfiguration":{"EbsBlockDeviceConfigs":[{"VolumeSpecification":{"SizeInGB":32,"VolumeType":"gp2"},"VolumesPerInstance":1}],"EbsOptimized":true},"InstanceGroupType":"CORE","InstanceType":"m4.xlarge","Name":"Core - 2"}]' 
--region us-east-1

After the cluster is launched, log in to the master node using SSH, and do the following:

  1. Open the hive-tpcds folder:
    cd /home/hadoop/hive-tpcds/
  2. Start Hive CLI using the testbench configuration, create the required tables, and run the sample query:

    hive –i testbench.settings
    hive> source create_tables.sql;
    hive> source query55.sql;

    This sample query runs on a 40 GB dataset that is stored on Amazon S3. The dataset is generated using the data generation tool in the TPC-DS testbench for Hive.It results in output like the following:
  3. This screenshot shows that the query finished in about 47 seconds for LLAP mode. Now, to compare this to the execution time without LLAP, you can run the same workload using only Tez containers:
    hive> set hive.llap.execution.mode=none;
    hive> source query55.sql;


    This query finished in about 80 seconds.

The difference in query execution time is almost 1.7 times when using just YARN containers in contrast to running the query on LLAP daemons. And with every rerun of the query, you notice that the execution time substantially decreases by the virtue of in-memory caching by LLAP daemons.

Conclusion

In this post, I introduced Hive LLAP as a way to boost Hive query performance. I discussed its architecture and described several use cases for the component. I showed how you can install and configure Hive LLAP on an Amazon EMR cluster and how you can run queries on LLAP daemons.

If you have questions about using Hive LLAP on Amazon EMR or would like to share your use cases, please leave a comment below.


Additional Reading

Learn how to to automatically partition Hive external tables with AWS.


About the Author

Jigar Mistry is a Hadoop Systems Engineer with Amazon Web Services. He works with customers to provide them architectural guidance and technical support for processing large datasets in the cloud using open-source applications. In his spare time, he enjoys going for camping and exploring different restaurants in the Seattle area.

 

 

 

 

Newly Updated: Example AWS IAM Policies for You to Use and Customize

Post Syndicated from Deren Smith original https://aws.amazon.com/blogs/security/newly-updated-example-policies-for-you-to-use-and-customize/

To help you grant access to specific resources and conditions, the Example Policies page in the AWS Identity and Access Management (IAM) documentation now includes more than thirty policies for you to use or customize to meet your permissions requirements. The AWS Support team developed these policies from their experiences working with AWS customers over the years. The example policies cover common permissions use cases you might encounter across services such as Amazon DynamoDB, Amazon EC2, AWS Elastic Beanstalk, Amazon RDS, Amazon S3, and IAM.

In this blog post, I introduce the updated Example Policies page and explain how to use and customize these policies for your needs.

The new Example Policies page

The Example Policies page in the IAM User Guide now provides an overview of the example policies and includes a link to view each policy on a separate page. Note that each of these policies has been reviewed and approved by AWS Support. If you would like to submit a policy that you have found to be particularly useful, post it on the IAM forum.

To give you an idea of the policies we have included on this page, the following are a few of the EC2 policies on the page:

To see the full list of available policies, see the Example Polices page.

In the following section, I demonstrate how to use a policy from the Example Policies page and customize it for your needs.

How to customize an example policy for your needs

Suppose you want to allow an IAM user, Bob, to start and stop EC2 instances with a specific resource tag. After looking through the Example Policies page, you see the policy, Allows Starting or Stopping EC2 Instances a User Has Tagged, Programmatically and in the Console.

To apply this policy to your specific use case:

  1. Navigate to the Policies section of the IAM console.
  2. Choose Create policy.
    Screenshot of choosing "Create policy"
  3. Choose the Select button next to Create Your Own Policy. You will see an empty policy document with boxes for Policy Name, Description, and Policy Document, as shown in the following screenshot.
  4. Type a name for the policy, copy the policy from the Example Policies page, and paste the policy in the Policy Document box. In this example, I use “start-stop-instances-for-owner-tag” as the policy name and “Allows users to start or stop instances if the instance tag Owner has the value of their user name” as the description.
  5. Update the placeholder text in the policy (see the full policy that follows this step). For example, replace <REGION> with a region from AWS Regions and Endpoints and <ACCOUNTNUMBER> with your 12-digit account number. The IAM policy variable, ${aws:username}, is a dynamic property in the policy that automatically applies to the user to which it is attached. For example, when the policy is attached to Bob, the policy replaces ${aws:username} with Bob. If you do not want to use the key value pair of Owner and ${aws:username}, you can edit the policy to include your desired key value pair. For example, if you want to use the key value pair, CostCenter:1234, you can modify “ec2:ResourceTag/Owner”: “${aws:username}” to “ec2:ResourceTag/CostCenter”: “1234”.
    {
        "Version": "2012-10-17",
        "Statement": [
           {
          "Effect": "Allow",
          "Action": [
              "ec2:StartInstances",
              "ec2:StopInstances"
          ],
                 "Resource": "arn:aws:ec2:<REGION>:<ACCOUNTNUMBER>:instance/*",
                 "Condition": {
              "StringEquals": {
                  "ec2:ResourceTag/Owner": "${aws:username}"
              }
          }
            },
            {
                 "Effect": "Allow",
                 "Action": "ec2:DescribeInstances",
                 "Resource": "*"
            }
        ]
    }

  6. After you have edited the policy, choose Create policy.

You have created a policy that allows an IAM user to stop and start EC2 instances in your account, as long as these instances have the correct resource tag and the policy is attached to your IAM users. You also can attach this policy to an IAM group and apply the policy to users by adding them to that group.

Summary

We updated the Example Policies page in the IAM User Guide so that you have a central location where you can find examples of the most commonly requested and used IAM policies. In addition to these example policies, we recommend that you review the list of AWS managed policies, including the AWS managed policies for job functions. You can choose these predefined policies from the IAM console and associate them with your IAM users, groups, and roles.

We will add more IAM policies to the Example Policies page over time. If you have a useful policy you would like to share with others, post it on the IAM forum. If you have comments about this post, submit them in the “Comments” section below.

– Deren

Introducing the GameDay Essentials Show on AWS Twitch Channel

Post Syndicated from Tara Walker original https://aws.amazon.com/blogs/aws/game-day-essentials-show-on-twitch/

Imagine if you will, you have obtained a new position at Unicorn.Rentals, a company that specializes in LARM, Legendary Animal Rental Market. Given the chance, what child wouldn’t happily exchange anything for the temporary use of a unicorn? What parent could refuse the opportunity to make their children happy? Let’s estimate the year to be 2017 and Unicorn.Rentals continues to dominate in the animal rental market.

You are about to enter another dimension, a dimension as vast as space and as timeless as infinity. It is the middle ground between light and shadow, between science and superstition, and lies at the beginning of man’s cloud knowledge. This is a journey into a wondrous land of imagination, a land of both shadow and substance. You are crossing over into the GameDay Essentials Zone.

Well, maybe not another dimension but almost as cool. Maybe, kinda? Either way, I am very excited to introduce the newest show on the AWS Twitch Channel named GameDay Essentials. The GameDay Essentials show is a  “new hire training program” for the aforementioned Unicorn.Rentals company scenario. You will step into the shoes of a new employee being ramped up and trained on cloud computing in order to work successfully for a company using Amazon Web Services.

 

With the GameDay Essentials show, you will get hands-on computing experience to help with the growth of the Unicorn.Rentals startup. The first episode, Recon, premiered on July 25th and provided information on logging services with CloudTrail and Cloudwatch, as well as, how to assess the configuration and identify existing inventory resources in an AWS Account. You can check out the recording of Episode 1–Recon here. The rest of season one for this six-part series airs on Tuesdays at 11:30 AM PT, the next three episodes discussing the following topics:

  • Episode 2 – Scaling: Learn how to scale your application infrastructure by diving into the how to of implementing scaling techniques and auto scaling groups. Airing on August 1 
  • Episode 3 – Changes: Winston Churchill is quoted saying “To improve is to change; to be perfect is to change often”. This GameDay episode is all about managing change as a key component to success. You will learn how to use native AWS security and deployment tools to track and manage change and discuss how to handle changes in team dynamics. Airing on August 8th
  • Episode 4 – Decoupling: Most people in the technology industry understand that you should avoid creating tightly coupled systems. Therefore, you will discover how loosely coupled systems operate and gain knowledge on how to diagnose any failures that may occur with these systems. Airing on August 15th 

Summary

Our latest show, GameDay Essentials is designed to help you “get into the game” and learn more about cloud computing and the AWS Platform. GameDay Essentials joins our other live coding shows already featured each week on the AWS Twitch Channel: Live Coding with AWS and AWS Maker Studio.

Tune in each week to the AWS Twitch channel to visit another dimension: a dimension of sound, a dimension of sight, a dimension of cloud. This is the dimension of imagination. It is an area, which we call the GameDay Essentials Zone. Get it, like the Twilight Zone, still no? Oh well, check out the GameDay Essentials show on Twitch on the AWS Channel, it is a great resource for interactive learning about cloud computing with AWS, so enjoy the ride.

Tara

[$] A milestone for control groups

Post Syndicated from corbet original https://lwn.net/Articles/729215/rss

Changes to core-kernel subsystems take time but, even so, one can only
imagine that Tejun Heo never expected the process of fixing the
control-group interface to take more than five years. Disagreements over
the design of the new control-group interface have delayed its adoption;
even though most of the code has been in the kernel for some time, not all
controllers work with it. It would now appear, however, that agreement has
been reached on an important final piece, which is currently on track to be
merged for the 4.14 development cycle.

China Says It Will “Severely Strike” Websites Involved in Piracy

Post Syndicated from Andy original https://torrentfreak.com/china-says-it-will-severely-strike-websites-involved-in-piracy-170729/

When it comes to the protection of intellectual property, China is often viewed as one of the world’s leading scofflaws. Everything is copied in the country, from designer watches to cars. Not even major landmarks can escape the replica treatment.

In more recent times, however, there have been signs that China might be at least warming to the idea that IP protection should be given more priority.

For example, every few months authorities announce a new crackdown on Internet piracy, such as the “Jian Wang 2016” program which shuttered 290 piracy websites in the final six months of last year.

Maintaining the same naming convention, this week China’s National Copyright Administration revealed the new “Jian Wang 2017” anti-piracy program. During a meeting in Beijing attended by other state bodies, copyright groups, rights organizations, and representatives from the news media, the administration detailed its latest plans.

The anti-piracy program will focus on protecting the copyrights of the film, television, and news industries in China. Infringing websites, e-commerce and cloud storage services, social networks, plus mobile Internet applications will all be put under the spotlight, with authorities investigating and prosecuting major cases.

The program, which will run for the next four months, has a mission to improve compliance in three key areas.

The first aims to assist the film and TV industries by cracking down on ‘pirate’ websites, the unlawful use of file-sharing software, plus “forum communities and other channels that supply infringing film and television works.”

Also on the cards is a blitz against users of the hugely popular social media and instant messaging app, WeChat.

Released in 2011, WeChat now has more than 930 million users, some of which use the platform to republish news articles without permission from creators. Chinese authorities want to reduce this activity, noting that too many articles are stripped from their sources and reproduced on personal blogs and similar platforms.

The second area for attention is the booming market for pirate apps. Chinese authorities say that cracked app stores and the software they provide are contributing to a huge rise in the unlawful spread of films, TV shows, music, news and other literature. Set-top boxes that utilize such apps will also be targeted in the crackdown.

Finally, there will be a “strengthening of copyright supervision” on large-scale e-commerce platforms that supply audio and video products, eBooks, and other publications. Cloud storage platforms will also be subjected to additional scrutiny, as these are often used to share copyright works without permission.

What kind of effect the program will have on overall copyrighted content availability will remain to be seen, but if previous patterns are maintained, the National Copyright Administration should reveal the results of its blitz in December.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.