Tag Archives: anonymous

Tech wishes for 2018

Post Syndicated from Eevee original https://eev.ee/blog/2018/02/18/tech-wishes-for-2018/

Anonymous asks, via money:

What would you like to see happen in tech in 2018?

(answer can be technical, social, political, combination, whatever)

Hmm.

Less of this

I’m not really qualified to speak in depth about either of these things, but let me put my foot in my mouth anyway:

The Blockchain™

Bitcoin was a neat idea. No, really! Decentralization is cool. Overhauling our terrible financial infrastructure is cool. Hash functions are cool.

Unfortunately, it seems to have devolved into mostly a get-rich-quick scheme for nerds, and by nearly any measure it’s turning into a spectacular catastrophe. Its “success” is measured in how much a bitcoin is worth in US dollars, which is pretty close to an admission from its own investors that its only value is in converting back to “real” money — all while that same “success” is making it less useful as a distinct currency.

Blah, blah, everyone already knows this.

What concerns me slightly more is the gold rush hype cycle, which is putting cryptocurrency and “blockchain” in the news and lending it all legitimacy. People have raked in millions of dollars on ICOs of novel coins I’ve never heard mentioned again. (Note: again, that value is measured in dollars.) Most likely, none of the investors will see any return whatsoever on that money. They can’t, really, unless a coin actually takes off as a currency, and that seems at odds with speculative investing since everyone either wants to hoard or ditch their coins. When the coins have no value themselves, the money can only come from other investors, and eventually the hype winds down and you run out of other investors.

I fear this will hurt a lot of people before it’s over, so I’d like for it to be over as soon as possible.


That said, the hype itself has gotten way out of hand too. First it was the obsession with “blockchain” like it’s a revolutionary technology, but hey, Git is a fucking blockchain. The novel part is the way it handles distributed consensus (which in Git is basically left for you to figure out), and that’s uniquely important to currency because you want to be pretty sure that money doesn’t get duplicated or lost when moved around.

But now we have startups trying to use blockchains for website backends and file storage and who knows what else? Why? What advantage does this have? When you say “blockchain”, I hear “single Git repository” — so when you say “email on the blockchain”, I have an aneurysm.

Bitcoin seems to have sparked imagination in large part because it’s decentralized, but I’d argue it’s actually a pretty bad example of a decentralized network, since people keep forking it. The ability to fork is a feature, sure, but the trouble here is that the Bitcoin family has no notion of federation — there is one canonical Bitcoin ledger and it has no notion of communication with any other. That’s what you want for currency, not necessarily other applications. (Bitcoin also incentivizes frivolous forking by giving the creator an initial pile of coins to keep and sell.)

And federation is much more interesting than decentralization! Federation gives us email and the web. Federation means I can set up my own instance with my own rules and still be able to meaningfully communicate with the rest of the network. Federation has some amount of tolerance for changes to the protocol, so such changes are more flexible and rely more heavily on consensus.

Federation is fantastic, and it feels like a massive tragedy that this rekindled interest in decentralization is mostly focused on peer-to-peer networks, which do little to address our current problems with centralized platforms.

And hey, you know what else is federated? Banks.

AI

Again, the tech is cool and all, but the marketing hype is getting way out of hand.

Maybe what I really want from 2018 is less marketing?

For one, I’ve seen a huge uptick in uncritically referring to any software that creates or classifies creative work as “AI”. Can we… can we not. It’s not AI. Yes, yes, nerds, I don’t care about the hair-splitting about the nature of intelligence — you know that when we hear “AI” we think of a human-like self-aware intelligence. But we’re applying it to stuff like a weird dog generator. Or to whatever neural network a website threw into production this week.

And this is dangerously misleading — we already had massive tech companies scapegoating The Algorithm™ for the poor behavior of their software, and now we’re talking about those algorithms as though they were self-aware, untouchable, untameable, unknowable entities of pure chaos whose decisions we are arbitrarily bound to. Ancient, powerful gods who exist just outside human comprehension or law.

It’s weird to see this stuff appear in consumer products so quickly, too. It feels quick, anyway. The latest iPhone can unlock via facial recognition, right? I’m sure a lot of effort was put into ensuring that the same person’s face would always be recognized… but how confident are we that other faces won’t be recognized? I admit I don’t follow all this super closely, so I may be imagining a non-problem, but I do know that humans are remarkably bad at checking for negative cases.

Hell, take the recurring problem of major platforms like Twitter and YouTube classifying anything mentioning “bisexual” as pornographic — because the word is also used as a porn genre, and someone threw a list of porn terms into a filter without thinking too hard about it. That’s just a word list, a fairly simple thing that any human can review; but suddenly we’re confident in opaque networks of inferred details?

I don’t know. “Traditional” classification and generation are much more comforting, since they’re a set of fairly abstract rules that can be examined and followed. Machine learning, as I understand it, is less about rules and much more about pattern-matching; it’s built out of the fingerprints of the stuff it’s trained on. Surely that’s just begging for tons of edge cases. They’re practically made of edge cases.


I’m reminded of a point I saw made a few days ago on Twitter, something I’d never thought about but should have. TurnItIn is a service for universities that checks whether students’ papers match any others, in order to detect cheating. But this is a paid service, one that fundamentally hinges on its corpus: a large collection of existing student papers. So students pay money to attend school, where they’re required to let their work be given to a third-party company, which then profits off of it? What kind of a goofy business model is this?

And my thoughts turn to machine learning, which is fundamentally different from an algorithm you can simply copy from a paper, because it’s all about the training data. And to get good results, you need a lot of training data. Where is that all coming from? How many for-profit companies are setting a neural network loose on the web — on millions of people’s work — and then turning around and selling the result as a product?

This is really a question of how intellectual property works in the internet era, and it continues our proud decades-long tradition of just kinda doing whatever we want without thinking about it too much. Nothing if not consistent.

More of this

A bit tougher, since computers are pretty alright now and everything continues to chug along. Maybe we should just quit while we’re ahead. There’s some real pie-in-the-sky stuff that would be nice, but it certainly won’t happen within a year, and may never happen except in some horrific Algorithmic™ form designed by people that don’t know anything about the problem space and only works 60% of the time but is treated as though it were bulletproof.

Federation

The giants are getting more giant. Maybe too giant? Granted, it could be much worse than Google and Amazon — it could be Apple!

Amazon has its own delivery service and brick-and-mortar stores now, as well as providing the plumbing for vast amounts of the web. They’re not doing anything particularly outrageous, but they kind of loom.

Ad company Google just put ad blocking in its majority-share browser — albeit for the ambiguously-noble goal of only blocking obnoxious ads so that people will be less inclined to install a blanket ad blocker.

Twitter is kind of a nightmare but no one wants to leave. I keep trying to use Mastodon as well, but I always forget about it after a day, whoops.

Facebook sounds like a total nightmare but no one wants to leave that either, because normies don’t use anything else, which is itself direly concerning.

IRC is rapidly bleeding mindshare to Slack and Discord, both of which are far better at the things IRC sadly never tried to do and absolutely terrible at the exact things IRC excels at.

The problem is the same as ever: there’s no incentive to interoperate. There’s no fundamental technical reason why Twitter and Tumblr and MySpace and Facebook can’t intermingle their posts; they just don’t, because why would they bother? It’s extra work that makes it easier for people to not use your ecosystem.

I don’t know what can be done about that, except that hope for a really big player to decide to play nice out of the kindness of their heart. The really big federated success stories — say, the web — mostly won out because they came along first. At this point, how does a federated social network take over? I don’t know.

Social progress

I… don’t really have a solid grasp on what’s happening in tech socially at the moment. I’ve drifted a bit away from the industry part, which is where that all tends to come up. I have the vague sense that things are improving, but that might just be because the Rust community is the one I hear the most about, and it puts a lot of effort into being inclusive and welcoming.

So… more projects should be like Rust? Do whatever Rust is doing? And not so much what Linus is doing.

Open source funding

I haven’t heard this brought up much lately, but it would still be nice to see. The Bay Area runs on open source and is raking in zillions of dollars on its back; pump some of that cash back into the ecosystem, somehow.

I’ve seen a couple open source projects on Patreon, which is fantastic, but feels like a very small solution given how much money is flowing through the commercial tech industry.

Ad blocking

Nice. Fuck ads.

One might wonder where the money to host a website comes from, then? I don’t know. Maybe we should loop this in with the above thing and find a more informal way to pay people for the stuff they make when we find it useful, without the financial and cognitive overhead of A Transaction or Giving Someone My Damn Credit Card Number. You know, something like Bitco— ah, fuck.

Year of the Linux Desktop

I don’t know. What are we working on at the moment? Wayland? Do Wayland, I guess. Oh, and hi-DPI, which I hear sucks. And please fix my sound drivers so PulseAudio stops blaming them when it fucks up.

Man Handed Conditional Prison Sentence for Spreading Popcorn Time Information

Post Syndicated from Andy original https://torrentfreak.com/man-handed-conditional-prison-sentence-spreading-popcorn-time-information-180208/

In August 2015, police in Denmark announced they had arrested a man in his thirties said to be the operator of a Popcorn Time-focused website. Popcorntime.dk was subsequently shut down and its domain placed under the control of the state prosecutor.

“The Danish State Prosecutor for Serious Economic and International Crime is presently conducting a criminal investigation that involves this domain name,” a seizure notice on the site reads.

“As part of the investigation the state prosecutor has requested a Danish District Court to transfer the rights of the domain name to the state prosecutor. The District Court has complied with the request.”

In a circumstance like this, it’s common to conclude that the site was offering copyright-infringing content or software. That wasn’t the case though, not even close.

PopcornTime.dk was an information resource, offering news on Popcorn Time-related developments, guides, plus tips on how to use the software while staying anonymous.

PopcornTime.dk as it appeared in 2015

Importantly, PopcornTime.dk hosted no software, preferring to link to other sites where the application could be downloaded instead. That didn’t prevent an aggressive prosecution though and now, two-and-half years later, the verdict’s in and it’s bound to raise more than a few eyebrows.

On Wednesday, a court in Odense, Denmark, handed the now 39-year-old man behind PopcornTime.dk a six-month conditional prison sentence for spreading information about the controversial movie streaming service.

Senior prosecutor Dorte Køhler Frandsen from SØIK (State Attorney for Special Economic and International Crime), who was behind the criminal proceedings, described the successful prosecution as a first-of-its-kind moment for the entire region.

“Never before has a person been convicted of helping to spread streaming services. The judgment is therefore an important step in combating illegal streaming on the Internet and will reverberate throughout Europe,” Frandsen said.

According to a statement from the prosecutor, the 39-year-old earned 506,003 Danish Krone ($83,363) in advertising revenue from his website in 2015. In addition to forfeiting this amount and having his domain confiscated, the man will also be required to complete 120 hours of community service.

“The verdict is a clear signal to those who spread illegal pirate services. The film industry and others lose billions in revenue each year because criminals illegally offer films for free. It’s a loss for everyone. Also the consumer,” Frandsen added.

The convicted man now has two weeks to decide whether he will take his appeal to the Østre Landsret, one of Denmark’s two High Courts.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

Locating Secret Military Bases via Fitness Data

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2018/01/locating_secret.html

In November, the company Strava released an anonymous data-visualization map showing all the fitness activity by everyone using the app.

Over this weekend, someone realized that it could be used to locate secret military bases: just look for repeated fitness activity in the middle of nowhere.

News article.

Researchers Use a Blockchain to Boost Anonymous Torrent Sharing

Post Syndicated from Ernesto original https://torrentfreak.com/researchers-use-a-blockchain-to-boost-anonymous-torrent-sharing-180129/

The Tribler client has been around for over a decade. We first covered it in 2006 and since then it’s developed into a truly decentralized BitTorrent client.

Even if all torrent sites were shut down today, Tribler users would still be able to find and add new content.

The project is not run by regular software developers but by a team of quality researchers at Delft University of Technology. There are currently more than 45 masters students, various thesis students, five dedicated scientific developers, and several professors involved.

Simply put, Triber aims to make the torrent ecosystem truly decentralized and anonymous. A social network of peers that can survive even if all torrent sites ceased to exist.

“Search and download torrents with less worries or censorship,” Triber’s tagline reads.

Like many other BitTorrent clients, Tribler has a search box at the top of the application. However, the search results that appear when users type in a keyword don’t come from a central index. Instead, they come directly from other peers.

Thriber’s search results

With the latest release, Tribler 7.0, the project adds another element to the mix, it’s very own blockchain. This blockchain keeps track of how much people are sharing and rewards them accordingly.

“Tribler is a torrent client for social people, who help each other. You can now earn tokens by helping others. It is specifically designed to prevent freeriding and detect hit-and-run peers.” Tribler leader Dr. Johan Pouwelse tells TF.

“You help other Tribler users by seeding and by enhancing their privacy. In return, you get faster downloads, as your tokens show you contribute to the community.”

Pouwelse, who aims to transform BitTorrent into an ethical Darknet, just presented the latest release at Stanford University. In addition, the Internet Engineering Task Force is also considering the blockchain implementation as an official Internet standard.

This recognition from academics and technology experts is welcome, of course, but Triber’s true power comes from the users. The client has gathered a decent userbase of the years but there sure is plenty room for improvement on this front.

The anonymity aspect is perhaps one of the biggest selling points and Pouwelse believes that this will greatly benefit from the blockchain implementation.

Triber provides users with pseudo anonymity by routing the transfers through other users. However, this means that the amount of bandwith used by the application inceases as well. Thus far, this hasn’t worked very well, which resulted in slow anonymous downloads.

“With the integrated blockchain release today we think we can start fixing the problem of both underseeded swarms and fast proxies,” Dr. Pouwelse says.

“Our solution is basically very simple, only social people get decent performance on Tribler. This means in a few years we will end up with only users that act nice. Others leave.”

Tribler’s trust stats

Tribler provides users with quite a bit of flexibility on the anonymity site. The feature can be turned off completely, or people can choose a protection layer ranging from one to four hops.

What’s also important to note is that users don’t operate as exit nodes by default. The IP-addresses of the exit nodes are public ouitside the network and can be monitored, so that would only increase liability.

So who are the exit-nodes in this process then? According to Pouwelse’s rather colorful description, these appear to be volunteers that run their code through a VPN a or a VPS server.

“The past years we have created an army of bots we call ‘Self-replicating Autonomous Entities’. These are Terminator-style self-replicating pieces of code which have their own Bitcoin wallet to go out there and buy servers to run more copies of themselves,” he explains.

“They utilize very primitive genetic evolution to improve survival, buy a VPN for protection, earn credits using our experimental credit mining preview release, and sell our bandwidth tokens on our integrated decentral market for cold hard Bitcoin cash to renew the cycle of life for the next month billing cycle of their VPS provider.”

Some might question why there’s such a massive research project dedicated to building an anonymous BitTorrent network. What are the benefits to society?

The answer is clear, according to Pouwelse. The ethical darknet they envision will be a unique micro-economy where sharing is rewarded, without having to expose one’s identity.

“We are building the Internet of Trust. The Internet can do amazing things, it even created honesty among drugs dealers,” he says, referring to the infamous Silk Road.

“Reliability rating of drugs lords gets you life imprisonment. That’s not something we want. We are creating our own trustworthy micro-economy for bandwidth tokens and real Bitcoins,” he adds.

People who are interested in taking Tribler for a spin can download the latest version from the official website.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

The problematic Wannacry North Korea attribution

Post Syndicated from Robert Graham original http://blog.erratasec.com/2018/01/the-problematic-wannacry-north-korea.html

Last month, the US government officially “attributed” the Wannacry ransomware worm to North Korea. This attribution has three flaws, which are a good lesson for attribution in general.

It was an accident

The most important fact about Wannacry is that it was an accident. We’ve had 30 years of experience with Internet worms teaching us that worms are always accidents. While launching worms may be intentional, their effects cannot be predicted. While they appear to have targets, like Slammer against South Korea, or Witty against the Pentagon, further analysis shows this was just a random effect that was impossible to predict ahead of time. Only in hindsight are these effects explainable.
We should hold those causing accidents accountable, too, but it’s a different accountability. The U.S. has caused more civilian deaths in its War on Terror than the terrorists caused triggering that war. But we hold these to be morally different: the terrorists targeted the innocent, whereas the U.S. takes great pains to avoid civilian casualties. 
Since we are talking about blaming those responsible for accidents, we also must include the NSA in that mix. The NSA created, then allowed the release of, weaponized exploits. That’s like accidentally dropping a load of unexploded bombs near a village. When those bombs are then used, those having lost the weapons are held guilty along with those using them. Yes, while we should blame the hacker who added ETERNAL BLUE to their ransomware, we should also blame the NSA for losing control of ETERNAL BLUE.

A country and its assets are different

Was it North Korea, or hackers affilliated with North Korea? These aren’t the same.

It’s hard for North Korea to have hackers of its own. It doesn’t have citizens who grow up with computers to pick from. Moreover, an internal hacking corps would create tainted citizens exposed to dangerous outside ideas. Update: Some people have pointed out that Kim Il-sung University in the capital does have some contact with the outside world, with academics granted limited Internet access, so I guess some tainting is allowed. Still, what we know of North Korea hacking efforts largley comes from hackers they employ outside North Korea. It was the Lazurus Group, outside North Korea, that did Wannacry.
Instead, North Korea develops external hacking “assets”, supporting several external hacking groups in China, Japan, and South Korea. This is similar to how intelligence agencies develop human “assets” in foreign countries. While these assets do things for their handlers, they also have normal day jobs, and do many things that are wholly independent and even sometimes against their handler’s interests.
For example, this Muckrock FOIA dump shows how “CIA assets” independently worked for Castro and assassinated a Panamanian president. That they also worked for the CIA does not make the CIA responsible for the Panamanian assassination.
That CIA/intelligence assets work this way is well-known and uncontroversial. The fact that countries use hacker assets like this is the controversial part. These hackers do act independently, yet we refuse to consider this when we want to “attribute” attacks.

Attribution is political

We have far better attribution for the nPetya attacks. It was less accidental (they clearly desired to disrupt Ukraine), and the hackers were much closer to the Russian government (Russian citizens). Yet, the Trump administration isn’t fighting Russia, they are fighting North Korea, so they don’t officially attribute nPetya to Russia, but do attribute Wannacry to North Korea.
Trump is in conflict with North Korea. He is looking for ways to escalate the conflict. Attributing Wannacry helps achieve his political objectives.
That it was blatantly politics is demonstrated by the way it was released to the press. It wasn’t released in the normal way, where the administration can stand behind it, and get challenged on the particulars. Instead, it was pre-released through the normal system of “anonymous government officials” to the NYTimes, and then backed up with op-ed in the Wall Street Journal. The government leaks information like this when it’s weak, not when its strong.

The proper way is to release the evidence upon which the decision was made, so that the public can challenge it. Among the questions the public would ask is whether it they believe it was North Korea’s intention to cause precisely this effect, such as disabling the British NHS. Or, whether it was merely hackers “affiliated” with North Korea, or hackers carrying out North Korea’s orders. We cannot challenge the government this way because the government intentionally holds itself above such accountability.

Conclusion

We believe hacking groups tied to North Korea are responsible for Wannacry. Yet, even if that’s true, we still have three attribution problems. We still don’t know if that was intentional, in pursuit of some political goal, or an accident. We still don’t know if it was at the direction of North Korea, or whether their hacker assets acted independently. We still don’t know if the government has answers to these questions, or whether it’s exploiting this doubt to achieve political support for actions against North Korea.

Pirate Bay Founder’s Domain Service “Mocks” NY Times Legal Threats

Post Syndicated from Ernesto original https://torrentfreak.com/pirate-bay-founders-domain-service-mocks-ny-times-legal-threats-180125/

Back in the day, The Pirate Bay was famous for its amusing responses to legal threats. Instead of complying with takedown notices, it sent witty responses to embarrass the senders.

Today the notorious torrent site gives copyright holders the silent treatment, but the good-old Pirate Bay spirit still lives on elsewhere.

Earlier today the anonymous domain registration service Njalla, which happens to be a venture of TPB co-founder Peter Sunde, posted a series of noteworthy responses it sent to The New York Times’ (NYT) legal department.

The newspaper warned the registration service about one of its customers, paywallnews.com, which offers the news service’s content without permission. Since this is a violation of The Times’ copyrights, according to the paper, Njalla should take action or face legal consequences.

NYT: Accordingly, we hereby demand that you immediately provide us with contact information — including email addresses — for both the actual owner of the paywallnew.com website, and for the hosting provider on which the paywallnew.com website is located.

If we have not heard from you within three (3) business days of receipt of this letter, we will have no choice but to pursue all available legal remedies.

Njalla is no stranger to threats of this kind but were somewhat offended by the harsh language, it seems. The company, therefore, decided to inform the NYT that there are more friendly ways to reach out.

Njalla: Thanks for that lovely e-mail. It’s always good to communicate with people that in their first e-mail use words as “we demand”, “pursue all available legal remedies” and so forth. I’d like to start out with some free (as in no cost) advice: please update your boiler threat letters to actually try what most people try first: being nice. It’s not expensive (actually the opposite) and actually it works much better than your method (source: a few tens of thousands years of human development that would not have been as efficient with threats as it would have been with cooperation).

In addition, Njalla also included a request of its own. They kindly asked (no demand) the newspaper’s legal department for proof that they are who they say they are. You can never be too cautious, after all.

Njalla: Now, back to the questions you sent us. We’re not sure who you are, so in order to move further we’d like to see a copy of your ID card, as well as a notarised power of attorney showing that you are actually representing the people you’re claiming to do.

This had the desired effect, for Njalla at least. The NYT replied with an apology for the tough language that was used, noting that they usually deal with companies that employ people who are used to reading legal documents.

The newspaper did, however, submit a notarized letter signed by the company’s Executive Vice President, General Counsel and Secretary, and once again asked for details on the Njalla customer.

NYT: Once again, as I mention above, the referenced website is stealing large amounts of New York Times content. If you click on this link: http://www.paywallnews.com/sites/nytimes

As this abuse — aside from being an egregious infringement of The Times’s copyright — breaches your own Terms of Service, I hope you will be able to see your way to helping me to put a stop to this practice by providing me with the name and contact information for the owner of paywallnews.com and for the ISP on which it is hosted.

This is when things started to get really interesting. Founded by someone with an extensive background in “sharing,” Njalla clearly has a different definition of stealing than the NYT’s legal department.

The reply, which is worth reading in full along with the rest of the communication, makes this quite clear.

Njalla: Stealing content seem quite harsh of this website though, didn’t know that they did that! Is there anyway you can get the stolen items back though? You should either go to the police and request them to help you get the stolen items back. Or maybe talk to your insurance company, they might help to compensate you for the loss. But a helpful idea; if they’ve stolen something and then put copies of that on a website that you can freely access, I would suggest just copying it, so that both of you have the same things. That’s a great thing with the digital world, everyone can have copies of things. I am surprised they stole something when they could just have copied it. I’m guessing it’s some older individuals that don’t know the possibilities of modern day technology to make copies.

It’s obvious that the domain registration service makes a clear distinction between copying and stealing.

Piracy vs. Theft

In addition, Njalla contests that the site is problematic at all, noting that this might be a “cultural difference.”

Njalla spotted something even more worrying though. The NYT claims that the site in question violates its terms of service. Specifically, they reference the section that prohibits sites from spreading content that is illegal according to local law.

Is the NYT perhaps spreading illegal content itself, Njalla questions?

Njalla: Deborah, I was quite shocked and appalled that you referred to this part of our ToS. It made me actually not visit the website in question even though you’ve linked it now a few times. You’re admitting to spreading illegal content at your newspaper, for profit, is that correct?

We’re quite big proponents of freedom of speech, let me assure you of that, but we also have limits. If you spread illegal content, and our customers stole that illegal content and are now handing out free copies of that, that’s a huge issue for us. Since it would be illegal for us to get those copies if they’re illegal, I’m asking you what type of content it is?

As an attachment to the reply, Njalla also sent back a “notarized” letter of their own, by simply copying the NYT letter and sticking their own logo on it, to show how easily these can be fabricated.

TorrentFreak reached out to Sunde who informed us that they never heard from The New York Times after the last reply. As a domain registrant, Njalla is not obliged to comply with takedown requests, he explains.

“If they need help from us on copyright issues, they’re totally missing what we’re doing, and that they should look somewhere else anyhow. But I think most domain services gets tons of these threat emails, and a lot of them think they’re responsible because they don’t have access to legal help and just shut customers down.

“That’s what a lot of our customers say at least, since they migrated from a shitty service which doesn’t know their own business,” Sunde adds.

The NYT is not completely without options though. If they take the case to court in Sweden and win an injunction against paywallnews.com, Njalla will comply. The same is true if a customer really violates the terms of service.

Meanwhile, paywallnews.com remains online.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

MPAA Wins $19.8 Million From Pirate Site Pubfilm

Post Syndicated from Ernesto original https://torrentfreak.com/mpaa-wins-19-8-million-from-pirate-site-pubfilm-180119/

In recent years the MPAA has pursued legal action against several pirate sites and the streaming service Pubfilm is one of their latest targets.

Hollywood’s industry group initially kept the lawsuit secret. This was done to prevent Pubfilm’s operator from moving to a new domain preemptively. While this strategy worked, Pubfilm didn’t throw in the towel.

Soon after the pubfilm.com domain name was suspended, the site moved to pubfilm.ac. And that wasn’t all. Pubfilm also started to actively advertise its new domain through Google Adsense to regain its lost traffic.

Today, close to a year has passed and Pubfilm is still around. The site moved from domain to domain and currently resides at Pubfilm.is and a few other domains that are advertised on the site.

All this time the company failed to responded in court, so the case saw little movement. This week, however, the MPAA made its demands clear and soon after the court issued a default against the site and its unknown operators.

“Defendants are Internet pirates who own, operate, and promote a ring of interconnected websites under the name “PubFilm” and variants thereof whose purpose is to profit from the infringement of copyrighted works,” the group wrote in its request.

Because of this continued infringement, the MPAA demanded the maximum amount of statutory copyright infringement damages. With 132 titles listed in the complaint, this totals nearly $20 million.

“Given the egregious circumstances of this case, Plaintiffs should be awarded the full amount of statutory damages of $150,000 for each of the 132 Works identified in the Complaint, for a total of $19,800,000,” the MPAA writes in its memorandum.

In addition, the Hollywood studios requested a permanent injunction that will require domain registries to put associated domain names on hold and sign them over to the MPAA.

Both requests were granted by the court on Thursday.

Pubfilm domain hopping

Previously, several domain names were aready seized through a preliminary injunction that resulted in Pubfilm moving from domain to domain in recent months. While these seizures can be effective, not all domain registries will comply with a US court order.

One of Pubfilm’s main domain names at the moment uses the Icelandic .is cTLD. In the past, Iceland’s domain registry ISNIC told TorrentFreak that it would only take action when an Icelandic Court tells it to.

This means that the MPAA’s win might be one without teeth.

Getting millions of dollars from an anonymous site operator, presumably outside the United States, is not easy. And since the site still has several hard-to-shutdown domains, taking it offline isn’t that straightforward either.

The streaming site operators didn’t appear to be impressed by the legal battle either. For the time being, they seem more concerned with fighting fake versions, judging from their most recent Facebook update.

Pubfilm’s latest Facebook post

A copy of the MPAA’s Memorandum in support of the default judgment and permanent injunction is available here (pdf). A copy of court’s order can be found here (pdf).

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

Media Giant Can Keep Seized Ad Revenue From Pirate Sites

Post Syndicated from Ernesto original https://torrentfreak.com/media-giant-can-keep-seized-ad-revenue-from-pirate-sites-180109/

For several decades the MPAA and RIAA have been the prime anti-piracy groups in the United States.

While that may be true, there’s another player making a massive impact, while getting barely any press.

ABS-CBN, the largest media and entertainment company in the Philippines, has filed a series of lawsuits against pirate sites in the US, with the popular streaming portal Fmovies as the biggest target.

The company has already won several cases with damages ranging from a few hundred thousand to millions of dollars. However, the associated injunctions in these cases are perhaps even more significant.

We previously covered how ABS-CBN managed to get court orders to seize domain names, without the defendants getting actively involved. This is also the case in a recent lawsuit where a Florida federal court signed a broad injunction targeting more than two dozen sites that offered the company’s content.

The websites, including abscbn-teleserye.com, dramascools.com, tvnijuan.org, pinoydailyshows.com and weeklywarning.org, may not be known to a broad audience but their domain names have all been suspended, linking to a takedown message instead.

What’s most interesting, however, is that the advertising revenues of these sites were previously frozen. This was done to ensure that ABS-CBN would at least get some money if the defendants failed to respond, a strategy that seems to have paid off.

After the targeted site owners failed to respond, ABS-CBN requested a default judgment with damages for trademark and copyright infringement.

U.S. District Court Judge Cecilia Altonaga has now signed the order, awarding the media company over a million dollars in statutory trademark infringement damages. In addition, several of the sites must also pay copyright infringement damages.

Damages

The default judgment also orders associated registrars and registries to hand over the domain names to ABS-CBN. Thus far several domains have been seized already, but some foreign companies have not complied, most likely because they fall outside the US jurisdiction.

The most interesting part of the order, however, is that Judge Altonaga grants ABS-CBN the previously seized advertising revenues.

“All funds currently restrained by the advertising services, networks, and/or platforms […], pursuant to the temporary restraining order and preliminary injunction in this action are to be immediately (within five business days) transferred to Plaintiffs in partial satisfaction of the monetary judgment entered herein against each Defendant,” the Judge writes.

List of sites and their ad-networks

The sites in question used advertising services from a variety of well-known networks, including Google Adsense, MGID, Popads, AdsKeeper, and Bidvertiser. None of these companies responded in court after the initial seizure order, suggesting that they did not object.

This is the first time, to our knowledge, that a copyright holder has been granted advertising revenue from pirate sites in this manner. While it’s not known how much revenue the sites were making, there is bound to be some.

This could be a common legal tactic going forward because, generally speaking, it is very hard to get money from defaulting defendants who are relatively anonymous, or living in a foreign jurisdiction. By going after the advertisers, copyright holders have a good chance of securing some money, at least.

A copy of the default judgment is available here (pdf) and all affected websites are listed below.

– abscbn-teleserye.com
– astigvideos.com
– cinepinoy.lol
– cinepinoy.ag
– pinoyflix.ag
– pinoyflix.lol
– cinezen.me
– dramascools.com
– dramasget.com
– frugalpinoytv.org
– lambingan.cn
– pinoylambingan.ph
– lambingan.io
– lambingans.net
– latestpinoymovies.com
– pinasnews.net
– pinastvreplay.com
– pinoybay.ch
– pinoychannel.me
– pinoydailyshows.com
– pinoyplayback.net
– pinoytvshows.net
– pinoytv-shows.net
– rondownload.net
– sarapmanood.com
– tambayanshow.net
– thelambingan.com
– tvnijuan.org
– tvtambayan.org
– vianowpe.com
– weeklywarning.org
– weeklywarning.com

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

Physics cheats

Post Syndicated from Eevee original https://eev.ee/blog/2018/01/06/physics-cheats/

Anonymous asks:

something about how we tweak physics to “work” better in games?

Ho ho! Work. Get it? Like in physics…?

Hitboxes

Hitbox” is perhaps not the most accurate term, since the shape used for colliding with the environment and the shape used for detecting damage might be totally different. They’re usually the same in simple platformers, though, and that’s what most of my games have been.

The hitbox is the biggest physics fudge by far, and it exists because of a single massive approximation that (most) games make: you’re controlling a single entity in the abstract, not a physical body in great detail.

That is: when you walk with your real-world meat shell, you perform a complex dance of putting one foot in front of the other, a motion you spent years perfecting. When you walk in a video game, you press a single “walk” button. Your avatar may play an animation that moves its legs back and forth, but since you’re not actually controlling the legs independently (and since simulating them is way harder), the game just treats you like a simple shape. Fairly often, this is a box, or something very box-like.

An Eevee sprite standing on faux ground; the size of the underlying image and the hitbox are outlined

Since the player has no direct control over the exact placement of their limbs, it would be slightly frustrating to have them collide with the world. This is especially true in cases like the above, where the tail and left ear protrude significantly out from the main body. If that Eevee wanted to stand against a real-world wall, she would simply tilt her ear or tail out of the way, so there’s no reason for the ear to block her from standing against a game wall. To compensate for this, the ear and tail are left out of the collision box entirely and will simply jut into a wall if necessary — a goofy affordance that’s so common it doesn’t even register as unusual. As a bonus (assuming this same box is used for combat), she won’t take damage from projectiles that merely graze past an ear.

(One extra consideration for sprite games in particular: the hitbox ought to be horizontally symmetric around the sprite’s pivot — i.e. the point where the entity is truly considered to be standing — so that the hitbox doesn’t abruptly move when the entity turns around!)

Corners

Treating the player (and indeed most objects) as a box has one annoying side effect: boxes have corners. Corners can catch on other corners, even by a single pixel. Real-world bodies tend to be a bit rounder and squishier and this can tolerate grazing a corner; even real-world boxes will simply rotate a bit.

Ah, but in our faux physics world, we generally don’t want conscious actors (such as the player) to rotate, even with a realistic physics simulator! Real-world bodies are made of parts that will generally try to keep you upright, after all; you don’t tilt back and forth much.

One way to handle corners is to simply remove them from conscious actors. A hitbox doesn’t have to be a literal box, after all. A popular alternative — especially in Unity where it’s a standard asset — is the pill-shaped capsule, which has semicircles/hemispheres on the top and bottom and a cylindrical body in 3D. No corners, no problem.

Of course, that introduces a new problem: now the player can’t balance precariously on edges without their rounded bottom sliding them off. Alas.

If you’re stuck with corners, then, you may want to use a corner bump, a term I just made up. If the player would collide with a corner, but the collision is only by a few pixels, just nudge them to the side a bit and carry on.

An Eevee sprite trying to move sideways into a shallow ledge; the game bumps her upwards slightly, so she steps onto it instead

When the corner is horizontal, this creates stairs! This is, more or less kinda, how steps work in Doom: when the player tries to cross from one sector into another, if the height difference is 24 units or less, the game simply bumps them upwards to the height of the new floor and lets them continue on.

Implementing this in a game without Doom’s notion of sectors is a little trickier. In fact, I still haven’t done it. Collision detection based on rejection gets it for free, kinda, but it’s not very deterministic and it breaks other things. But that’s a whole other post.

Gravity

Gravity is pretty easy. Everything accelerates downwards all the time. What’s interesting are the exceptions.

Jumping

Jumping is a giant hack.

Think about how actual jumping works: you tense your legs, which generally involves bending your knees first, and then spring upwards. In a platformer, you can just leap whenever you feel like it, which is nonsense. Also you go like twenty feet into the air?

Worse, most platformers allow variable-height jumping, where your jump is lower if you let go of the jump button while you’re in the air. Normally, one would expect to have to decide how much force to put into the jump beforehand.

But of course this is about convenience of controls: when jumping is your primary action, you want to be able to do it immediately, without any windup for how high you want to jump.

(And then there’s double jumping? Come on.)

Air control is a similar phenomenon: usually you’d jump in a particular direction by controlling how you push off the ground with your feet, but in a video game, you don’t have feet! You only have the box. The compromise is to let you control your horizontal movement to a limit degree in midair, even though that doesn’t make any sense. (It’s way more fun, though, and overall gives you more movement options, which are good to have in an interactive medium.)

Air control also exposes an obvious place that game physics collide with the realistic model of serious physics engines. I’ve mentioned this before, but: if you use Real Physics™ and air control yourself into a wall, you might find that you’ll simply stick to the wall until you let go of the movement buttons. Why? Remember, player movement acts as though an external force were pushing you around (and from the perspective of a Real™ physics engine, this is exactly how you’d implement it) — so air-controlling into a wall is equivalent to pushing a book against a wall with your hand, and the friction with the wall holds you in place. Oops.

Ground sticking

Another place game physics conflict with physics engines is with running to the top of a slope. On a real hill, of course, you land on top of the slope and are probably glad of it; slopes are hard to climb!

An Eevee moves to the top of a slope, and rather than step onto the flat top, she goes flying off into the air

In a video game, you go flying. Because you’re a box. With momentum. So you hit the peak and keep going in the same direction. Which is diagonally upwards.

Projectiles

To make them more predictable, projectiles generally aren’t subject to gravity, at least as far as I’ve seen. The real world does not have such an exemption. The real world imposes gravity even on sniper rifles, which in a video game are often implemented as an instant trace unaffected by anything in the world because the bullet never actually exists in the world.

Resistance

Ah. Welcome to hell.

Water

Water is an interesting case, and offhand I don’t know the gritty details of how games implement it. In the real world, water applies a resistant drag force to movement — and that force is proportional to the square of velocity, which I’d completely forgotten until right now. I am almost positive that no game handles that correctly. But then, in real-world water, you can push against the water itself for movement, and games don’t simulate that either. What’s the rough equivalent?

The Sonic Physics Guide suggests that Sonic handles it by basically halving everything: acceleration, max speed, friction, etc. When Sonic enters water, his speed is cut; when Sonic exits water, his speed is increased.

That last bit feels validating — I could swear Metroid Prime did the same thing, and built my own solution around it, but couldn’t remember for sure. It makes no sense, of course, for a jump to become faster just because you happened to break the surface of the water, but it feels fantastic.

The thing I did was similar, except that I didn’t want to add a multiplier in a dozen places when you happen to be underwater (and remember which ones need it to be squared, etc.). So instead, I calculate everything completely as normal, so velocity is exactly the same as it would be on dry land — but the distance you would move gets halved. The effect seems to be pretty similar to most platformers with water, at least as far as I can tell. It hasn’t shown up in a published game and I only added this fairly recently, so I might be overlooking some reason this is a bad idea.

(One reason that comes to mind is that velocity is now a little white lie while underwater, so anything relying on velocity for interesting effects might be thrown off. Or maybe that’s correct, because velocity thresholds should be halved underwater too? Hm!)

Notably, air is also a fluid, so it should behave the same way (just with different constants). I definitely don’t think any games apply air drag that’s proportional to the square of velocity.

Friction

Friction is, in my experience, a little handwaved. Probably because real-world friction is so darn complicated.

Consider that in the real world, we want very high friction on the surfaces we walk on — shoes and tires are explicitly designed to increase it, even. We move by bracing a back foot against the ground and using that to push ourselves forward, so we want the ground to resist our push as much as possible.

In a game world, we are a box. We move by being pushed by some invisible outside force, so if the friction between ourselves and the ground is too high, we won’t be able to move at all! That’s complete nonsense physically, but it turns out to be handy in some cases — for example, highish friction can simulate walking through deep mud, which should be difficult due to fluid drag and low friction.

But the best-known example of the fakeness of game friction is video game ice. Walking on real-world ice is difficult because the low friction means low grip; your feet are likely to slip out from under you, and you’ll simply fall down and have trouble moving at all. In a video game, you can’t fall down, so you have the opposite experience: you spend most of your time sliding around uncontrollably. Yet ice is so common in video games (and perhaps so uncommon in places I’ve lived) that I, at least, had never really thought about this disparity until an hour or so ago.

Game friction vs real-world friction

Real-world friction is a force. It’s the normal force (which is the force exerted by the object on the surface) times some constant that depends on how the two materials interact.

Force is mass times acceleration, and platformers often ignore mass, so friction ought to be an acceleration — applied against the object’s movement, but never enough to push it backwards.

I haven’t made any games where variable friction plays a significant role, but my gut instinct is that low friction should mean the player accelerates more slowly but has a higher max speed, and high friction should mean the opposite. I see from my own source code that I didn’t even do what I just said, so let’s defer to some better-made and well-documented games: Sonic and Doom.

In Sonic, friction is a fixed value subtracted from the player’s velocity (regardless of direction) each tic. Sonic has a fixed framerate, so the units are really pixels per tic squared (i.e. acceleration), multiplied by an implicit 1 tic per tic. So far, so good.

But Sonic’s friction only applies if the player isn’t pressing or . Hang on, that isn’t friction at all; that’s just deceleration! That’s equivalent to jogging to a stop. If friction were lower, Sonic would take longer to stop, but otherwise this is only tangentially related to friction.

(In fairness, this approach would decently emulate friction for non-conscious sliding objects, which are never going to be pressing movement buttons. Also, we don’t have the Sonic source code, and the name “friction” is a fan invention; the Sonic Physics Guide already uses “deceleration” to describe the player’s acceleration when turning around.)

Okay, let’s try Doom. In Doom, the default friction is 90.625%.

Hang on, what?

Yes, in Doom, friction is a multiplier applied every tic. Doom runs at 35 tics per second, so this is a multiplier of 0.032 per second. Yikes!

This isn’t anything remotely like real friction, but it’s much easier to implement. With friction as acceleration, the game has to know both the direction of movement (so it can apply friction in the opposite direction) and the magnitude (so it doesn’t overshoot and launch the object in the other direction). That means taking a semi-costly square root and also writing extra code to cap the amount of friction. With a multiplier, neither is necessary; just multiply the whole velocity vector and you’re done.

There are some downsides. One is that objects will never actually stop, since multiplying by 3% repeatedly will never produce a result of zero — though eventually the speed will become small enough to either slip below a “minimum speed” threshold or simply no longer fit in a float representation. Another is that the units are fairly meaningless: with Doom’s default friction of 90.625%, about how long does it take for the player to stop? I have no idea, partly because “stop” is ambiguous here! If friction were an acceleration, I could divide it into the player’s max speed to get a time.

All that aside, what are the actual effects of changing Doom’s friction? What an excellent question that’s surprisingly tricky to answer. (Note that friction can’t be changed in original Doom, only in the Boom port and its derivatives.) Here’s what I’ve pieced together.

Doom’s “friction” is really two values. “Friction” itself is a multiplier applied to moving objects on every tic, but there’s also a move factor which defaults to \(\frac{1}{32} = 0.03125\) and is derived from friction for custom values.

Every tic, the player’s velocity is multiplied by friction, and then increased by their speed times the move factor.

$$
v(n) = v(n – 1) \times friction + speed \times move factor
$$

Eventually, the reduction from friction will balance out the speed boost. That happens when \(v(n) = v(n – 1)\), so we can rearrange it to find the player’s effective max speed:

$$
v = v \times friction + speed \times move factor \\
v – v \times friction = speed \times move factor \\
v = speed \times \frac{move factor}{1 – friction}
$$

For vanilla Doom’s move factor of 0.03125 and friction of 0.90625, that becomes:

$$
v = speed \times \frac{\frac{1}{32}}{1 – \frac{29}{32}} = speed \times \frac{\frac{1}{32}}{\frac{3}{32}} = \frac{1}{3} \times speed
$$

Curiously, “speed” is three times the maximum speed an actor can actually move. Doomguy’s run speed is 50, so in practice he moves a third of that, or 16⅔ units per tic. (Of course, this isn’t counting SR40, a bug that lets Doomguy run ~40% faster than intended diagonally.)

So now, what if you change friction? Even more curiously, the move factor is calculated completely differently depending on whether friction is higher or lower than the default Doom amount:

$$
move factor = \begin{cases}
\frac{133 – 128 \times friction}{544} &≈ 0.244 – 0.235 \times friction & \text{ if } friction \ge \frac{29}{32} \\
\frac{81920 \times friction – 70145}{1048576} &≈ 0.078 \times friction – 0.067 & \text{ otherwise }
\end{cases}
$$

That’s pretty weird? Complicating things further is that low friction (which means muddy terrain, remember) has an extra multiplier on its move factor, depending on how fast you’re already going — the idea is apparently that you have a hard time getting going, but it gets easier as you find your footing. The extra multiplier maxes out at 8, which makes the two halves of that function meet at the vanilla Doom value.

A graph of the relationship between friction and move factor

That very top point corresponds to the move factor from the original game. So no matter what you do to friction, the move factor becomes lower. At 0.85 and change, you can no longer move at all; below that, you move backwards.

From the formula above, it’s easy to see what changes to friction and move factor will do to Doomguy’s stable velocity. Move factor is in the numerator, so increasing it will increase stable velocity — but it can’t increase, so stable velocity can only ever decrease. Friction is in the denominator, but it’s subtracted from 1, so increasing friction will make the denominator a smaller value less than 1, i.e. increase stable velocity. Combined, we get this relationship between friction and stable velocity.

A graph showing stable velocity shooting up dramatically as friction increases

As friction approaches 1, stable velocity grows without bound. This makes sense, given the definition of \(v(n)\) — if friction is 1, the velocity from the previous tic isn’t reduced at all, so we just keep accelerating freely.

All of this is why I’m wary of using multipliers.

Anyway, this leaves me with one last question about the effects of Doom’s friction: how long does it take to reach stable velocity? Barring precision errors, we’ll never truly reach stable velocity, but let’s say within 5%. First we need a closed formula for the velocity after some number of tics. This is a simple recurrence relation, and you can write a few terms out yourself if you want to be sure this is right.

$$
v(n) = v_0 \times friction^n + speed \times move factor \times \frac{friction^n – 1}{friction – 1}
$$

Our initial velocity is zero, so the first term disappears. Set this equal to the stable formula and solve for n:

$$
speed \times move factor \times \frac{friction^n – 1}{friction – 1} = (1 – 5\%) \times speed \times \frac{move factor}{1 – friction} \\
friction^n – 1 = -(1 – 5\%) \\
n = \frac{\ln 5\%}{\ln friction}
$$

Speed” and move factor disappear entirely, which makes sense, and this is purely a function of friction (and how close we want to get). For vanilla Doom, that comes out to 30.4, which is a little less than a second. For other values of friction:

A graph of time to stability which leaps upwards dramatically towards the right

As friction increases (which in Doom terms means the surface is more slippery), it takes longer and longer to reach stable speed, which is in turn greater and greater. For lesser friction (i.e. mud), stable speed is lower, but reached fairly quickly. (Of course, the extra “getting going” multiplier while in mud adds some extra time here, but including that in the graph is a bit more complicated.)

I think this matches with my instincts above. How fascinating!

What’s that? This is way too much math and you hate it? Then don’t use multipliers in game physics.

Uh

That was a hell of a diversion!

I guess the goofiest stuff in basic game physics is really just about mapping player controls to in-game actions like jumping and deceleration; the rest consists of hacks to compensate for representing everything as a box.

Blockchain Startup White Rabbit Calls on Pirate Sites to Do Business, Legally

Post Syndicated from Andy original https://torrentfreak.com/blockchain-startup-white-rabbit-calls-on-pirate-sites-to-do-business-legally-180102/

For as long as piracy has been mainstream, people have tried to find ways to monetize the system. While many have had good intentions, only models focusing on the negative (copyright trolling, for example) have enjoyed any level of success.

Blockchain startup White Rabbit is hoping to buck that trend but it’s not going to be easy. Then again, nothing worthwhile is, so what do they have to offer?

White Rabbit begins with the assumption that while they love their pirate sites, a many as 60% of pirates would happily reward creators if it was made easy enough. The startup deals with this by inviting pirates to carry on using the kinds of unauthorized sites and services they’re using already, but with a twist.

By installing the White Rabbit browser plug-in, the company will be able to see what content the user is accessing. It will then attempt to match that download to deals it’s made with the companies behind those movies or TV shows. They’ll then get paid a set amount.

“White Rabbit is a content ecosystem accessed through a plugin that recognizes the film and series you stream. The streaming sites are P2P or open server, meaning users can choose where they want to stream,” White Rabbit CEO Alan R. Milligan informs TF.

“We already have a library of films that have won and been nominated for Oscars, Cannes, Berlin and Venice film festival best film prizes – but will continue adding more films and series as we near launch.”

It’s envisioned that this mechanism will prove popular with reluctant pirates since instead of paying Netflix, Amazon, and dozens of other services, users can pay for content through one channel. And, since White Rabbit uses blockchain technology, rights holders can be ensured complete financial transparency, with user payments going straight to them without delay, cutting out the middleman.

“Users are anonymous but can offer filmmakers, artists or other content right holders (investors, distributors, sales agents) our tokens (WRT) as good faith that they are willing to pay for the content. Should the rights holders accept, we enter into a contract with the rights holder that allows them to receive revenue – and accept P2P streaming. We find, and research shows, that most people that are forced to piracy [do so] because they are just not able to access content,” Milligan adds.

White Rabbit’s CEO, who is a filmmaker himself, also sees opportunities to bring fans and filmmakers closer together. Once users have paid for content, they continue to get access via something called the Rabbit Hole, an interface which provides extras that are normally found on a DVD, such as deleted scenes etc.

The team behind White Rabbit describe themselves as “responsible rebels” hoping to spark a revolution. While that’s clearly the goal, by any measure there is a mountain to climb, not least on the content front.

When TorrentFreak first started speaking with the startup in October last year, we were told they were “closing in on 500 films” with contracts, although they wouldn’t elaborate on who might be on board. Nevertheless, that is quite a lot of movies, especially given the mainstream studios’ hatred of pirate sites and anything they might be involved in.

However, subsequent discussion suggests that those with more niche tastes might be White Rabbit’s initial target audience.

“I believe timing is of big relevance and right now a lot of producers are scared of where they´re going to go now that Netflix is enforcing its 50/50 policy. There are also so many amazing films out there that get no or little digital distribution at all,” Milligan says.

“As a Norwegian film producer there is little chance of the film being streamed in my home country – even if we won awards in Cannes and Venice. My latest film Valley of Shadows got US digital distribution, but in Norway – nada.

“My colleagues around the world are suffering the same way, not to mention all the fans who cant watch local films and series. So the indie part of the industry – which is most of us (and still representing 20-30% of cinema sales) – are very ready for change.”

But while indie producers could benefit nicely from White Rabbit, Milligan highlights problems that the big studios have, and suggests that they might like to see the startup succeed too.

“The studios will likely want to see our business model work – but they also have a problem with Netflix which has become a studio. So they´re competitors now, but Netflix has a 100M subscriber advantage. Will they all break out and create each their streaming site for their content only? That would be terrible for fans,” he notes.

That would indeed be a huge problem and it’s an issue we’ve raised here on TF on several occasions. However, if White Rabbit is to succeed, it needs to overcome significant hurdles. We raised just a handful of these with its CEO. First up, Partner Streaming Sites (PSS).

PSS sites appear to be pirate sites that will partner with White Rabbit, so the latter can tap into the formers’ userbases. When White Rabbit users stream ‘pirate’ content from a PSS, that content will be monetized, with the creator getting paid quickly and transparently. At that point, it seems, the content will become non-infringing.

But while that sounds intriguing in theory, plenty of questions remain. White Rabbit says it will share “up to $1M” from its token sale “with the most innovative, brand conscious, film and series loving streaming sites either already out there, planned or about to launch.”

The start-up says the best projects could get $100,000 each but, since its goal is to convert pirates, that necessarily means doing business with pirate sites.

So we asked; how will it be possible to do business with people that are regularly described as criminals? How will it then become possible to secure deals with filmmakers that will undoubtedly come under huge pressure from industry players not to participate in the White Rabbit scheme?

“What we are trying to do is to change digital distribution to everyone´s benefit. We have no interest in financing illegal content, we are interested in spurring innovation in streaming, access for fans and due payment for the rights holders,” Milligan explains.

“That´s what PSS can help us achieve using the WRT (White Rabbit Token) – that helps us find out who wants to be part of this model. No revenue exchanges hands until rights holders accept the token. What is important for rights holders is that we generate more revenue for them than current business models, and we haven´t even included the Rabbit Hole revenue yet.”

So what happens if a White Rabbit user tries to stream something that isn’t part of the program? According to Milligan, PSS sites must remove the content and let White Rabbit users know they must get the content legally elsewhere.

Clearly, the vast majority of pirate site users aren’t White Rabbit users now, nor will they be so in the future, so the removal of content is massively counter-productive for pirate sites. Indeed, it’s this reluctance to take down infringing content that causes them most of their problems.

So, hypothetically, what happens when the operators of streaming site X (that previously partnered with White Rabbit) get arrested and their site shut down for distributing Hollywood content that isn’t part of the program?

“PSS´s would never distribute illegal content, we are offering an opportunity to monetize. We are allowing a platform to those that see monetized P2P as beneficial to their income stream,” Milligan says.

“Hollywood is tricky though, I admit. The proof is in the pudding, so if we have to prove the value through indie and arthouse films first that´s OK. That is still 30% of the multi-billion dollar film market, so we are OK to start with that.”

The final issue is the price and where revenue goes. White Rabbit envisions a user paying $2 for film and $1 for a TV show, although producers are free to set their own price. That means 11 TV shows or five movies per month, given the Netflix model/budget of roughly $11.00 for the same period.

Revenue generated would then be split, with 75% going to the rightsholders, 15% to White Rabbit, and 10% to PSS sites. There’s also a provision for non-PSS sites to be a part of the program, but they would only get 5%, with the remaining 5% going to White Rabbit.

With an incredibly ambitious project like this, it’s easy to find reasons why it might not succeed or even fail to get off the ground. But the team behind the operation have lots of experience in relevant fields and from what we’ve seen are putting considerable effort into getting things moving, as their white paper (pdf) explains.

Currently, White Rabbit is seeking conversation with prospective Partner Streaming Sites, who will provide the content on which White Rabbit will survive. It will certainly be interesting to see which sites put themselves forward for consideration.

This is one of those projects that raises a dizzying volume of questions, with each living up to their billing as part of the Rabbit Hole. The big question is whether the Rabbit Hole will eventually lead to Wonderland or will render everyone who ventures inside feeling surreal and disorientated.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

Popular Danish Torrent Tracker Shuts Down After Hack

Post Syndicated from Ernesto original https://torrentfreak.com/popular-danish-torrent-tracker-shuts-down-after-hack-180102/

Torrent sites come in all shapes and sizes, but generally speaking there’s a clear divide netween private and public sites.

The latter includes the likes of The Pirate Bay and are open to anyone, while private trackers require an account to gain access.

Because many of these close communities also enforce ratio requirements and other rules, they can log quite a bit of data. This generally isn’t the type of information users would like to see out on the streets, but such leaks are no rarity.

In recent days the Danish torrent tracker Hounddawgs.org also ran into some issues. Out of the blue, the site’s 40,000 users received a message signed by ‘Anonymous’ stating that it had been hacked.

Hacked?

The hacker also noted that everyone had been promoted to “staff” but soon after the site went dark. It eventually returned with a message from the operator, accusing another private torrent site of ‘messing around.’

“We’re sorry, but due to server maintenance, we’ll be offline for a little while. Some kiddies from another Danish torrent site don’t like to share users so they found a way to mess a little with the site,” the notice read.

“No harm has been done, and we will be back up as soon as we have found the error and corrected it.”

The message seemed reassuring, but at the same time, a partially redacted file with usernames, emails, and IP-addresses started to circulate.

As a result, the rumor mill went into full swing, and people reported that other accounts where they used the same information, were being compromised. The Hounddawgs operators maintained, however, that allegations of a full database breach were false.

The site’s staff posted a new message refuting the hacking claims. At the same time, they also announced that the site would remain offline indefinitely.

Hounddawgs’ operators say they started the site as a counter-movement to the “tyranny” of other Danish trackers. However, these other trackers allegedly didn’t like the newcomer and fought back, up to a point where Hounddawgs decided to throw in the towel.

Hounddawgs’message (translated)

Private tracker feats are by no means new. They’re as old as private trackers. And while there are plenty opinions, since most of it takes place behind closed doors, the truth is often hard to find.

After the site’s operators said their goodbyes, pointing users to the new infinity-t.org tracker, the alleged hacker responded once more. This time posting over 20 gigabytes of data, said to be the full database and the site’s code.

“But how is that possible? The superheroes of the world, the people behind Hounddawgs, clearly stated on their frontpage that no database was leaked, so how could I possibly have it?” the hacker posted.

“They are lying! Like they have done for years, they don’t care one bit for their users,” the message adds, noting that the server was minimally secured.

The leaked files do indeed include site code and a database, which several people claim to be legitimate. The operators of Hounddawgs also changed their earlier tune. In a message posted on the site yesterday. They now apologize for not dealing with the security issues.

“It has NEVER been our intention to hurt any of you, and we were very happy with all the good users we had. We chose to close the site as a precaution, but unfortunately too late,” they write.

The site was running on the Gazelle script which logs quite a bit of data by default, including users’ IP-addresses. With this info out in the open, many users fear that anti-piracy groups may use the logs to identify individual pirates.

While it’s unlikely that copyright holders will pursue casual sharers based on leaked files, it’s never a pleasant thought to have one’s IP-addresses and other information leaked.

Although the local anti-piracy group, RettighedsAlliancen, might not spring into action right away, it won’t mind seeing the second largest tracker in Denmark go offline.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

Filmmakers Want The Right to Break DRM and Rip Blu-Rays

Post Syndicated from Ernesto original https://torrentfreak.com/filmmakers-want-the-right-to-break-drm-and-rip-blu-rays-171228/

The major movie studios are doing everything in their power to stop the public from copying films.

While nearly every movie and TV-show leaks on the Internet, these companies still see DRM as a vital tool to prevent piracy from spiraling out of control.

Technically speaking it’s not hard to rip a DVD or Blu-Ray disc nowadays, and the same is true for ripping content from Netflix or YouTube. However, people who do this are breaking the law.

The DMCA’s anti-circumvention provisions specifically forbid it. There are some exemptions, for educational use for example, and to allow for other types of fair use, but the line between legal and illegal is not always clear.

Interestingly, filmmakers are not happy with the current law either. They often want to use small pieces of other videos in their films, but under the current exemptions, this is only permitted for documentaries.

The International Documentary Association, Kartemquin Films, Independent Filmmaker Project, University of Film and Video Association and several other organizations hope this will change.

In a comment to the Copyright Office, which is currently considering updates to the exemptions, they argue that all filmmakers should be allowed by break DRM and rip Blu-Rays.

According to the filmmakers, the documentary genre is vaguely defined. This leads to a lot of confusion whether or not the exemptions apply. They, therefore, suggest to apply it to all filmmakers, instead of criminalizing those who don’t identify themselves as documentarians.

“Since 2010, exemptions applicable to documentary filmmaking have been in effect. This exemption has helped many filmmakers, and there has been neither evidence nor any allegation that this exemption has harmed rightsholders in any way.

“There is no reason this would change if the ‘documentary’ limitation were removed. All filmmakers regularly need access to footage on DVDs and without an exemption to DVDs, many non-infringing uses simply cannot be made,” the groups add.

The submission includes letters from several filmmakers who explain why an exemption would be crucial to them.

Filmmakers Steve Boettcher and Mike Trinklein explain that they refrained from making a film how they wanted it to be, fearing legal trouble. Their film included a lot of drama elements and was not a typical documentary.

“Given the significant amount of drama in the film [we are working on], we decided early on that our storytelling toolbox could not include fair use of materials from DVD or Blu-ray, because the exemption did not cover accessing that material for use in a drama,” they write

“Already, we were hindered in our ability to tell these stories. So, there is already a chilling effect in that a drama-heavy documentary might be seen as a drama outright, and thus under a different set of rules.”

Another filmmaker, who wants to remain anonymous, plans on making a hybrid documentary/narrative feature about a famous film duo. Without ripping the clips he needs, this movie is never going to be made.

“I am unsure of whether my project would fall under the exemption because it is a combination of documentary and narrative, and my fear of a lawsuit once my project is publicly viewed and distributed stops me from ripping from these sources.”

These are just two of many examples where filmmakers show that they need to break DRM and rip content to make the work they want.

The MPAA and others have previously argued that these changes are not required. Instead, they pointed out that people could point their cameras or phones at the screen to record something, or use screen capture software.

However, these are not viable alternatives according to the filmmakers, as the quality is inferior. They, therefore, call on the Copyright Office to expand the exemption to cover all films and filmmakers.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

CrimeStoppers Campaign Targets Pirate Set-Top Boxes & Their Users

Post Syndicated from Andy original https://torrentfreak.com/crimestoppers-campaign-targets-pirate-set-top-boxes-their-users-171209/

While many people might believe CrimeStoppers to be an official extension of the police in the UK, the truth is a little more subtle.

CrimeStoppers is a charity that operates a service through which members of the public can report crime anonymously, either using a dedicated phone line or via a website. Callers are not required to give their name, meaning that for those concerned about reprisals or becoming involved in a case for other sensitive reasons, it’s the perfect buffer between them and the authorities.

The people at CrimeStoppers deal with all kinds of crime but perhaps a little surprisingly, they’ve just got involved in the set-top box controversy in the UK.

“Advances in technology have allowed us to enjoy on-screen entertainment in more ways than ever before, with ever increasing amounts of exciting and original content,” the CrimeStoppers campaign begins.

“However, some people are avoiding paying for this content by using modified streaming hardware devices, like a set-top box or stick, in conjunction with software such as illegal apps or add-ons, or illegal mobile apps which allow them to watch new movie releases, TV that hasn’t yet aired, and subscription sports channels for free.”

The campaign has been launched in partnership with the Intellectual Property Office and unnamed “industry partners”. Who these companies are isn’t revealed but given the standard messages being portrayed by the likes of ACE, Premier League and Federation Against Copyright Theft lately, it wouldn’t be a surprise if some or all of them were involved.

Those messages are revealed in a series of four video ads, each taking a different approach towards discouraging the public from using devices loaded with pirate software.

The first video clearly targets the consumer, dispelling the myth that watching pirate video isn’t against the law. It is, that’s not in any doubt, but from the constant tone of the video, one could be forgiven that it’s an extremely serious crime rather than something which is likely to be a civil matter, if anything at all.

It also warns people who are configuring and selling pirate devices that they are breaking the law. Again, this is absolutely true but this activity is clearly several magnitudes more serious than simply viewing. The video blurs the boundaries for what appears to be dramatic effect, however.

Selling and watching is illegal

The second video is all about demonizing the people and groups who may offer set-top boxes to the public.

Instead of portraying the hundreds of “cottage industry” suppliers behind many set-top box sales in the UK, the CrimeStoppers video paints a picture of dark organized crime being the main driver. By buying from these people, the charity warns, criminals are being welcomed in.

“It is illegal. You could also be helping to fund organized crime and bringing it into your community,” the video warns.

Are you funding organized crime?

The third video takes another approach, warning that set-top boxes have few if any parental controls. This could lead to children being exposed to inappropriate content, the charity warns.

“What are your children watching. Does it worry you?” the video asks.

Of course, the same can be said about the Internet, period. Web browsers don’t filter what content children have access to unless parents take pro-active steps to configure special services or software for the purpose.

There’s always the option to supervise children, of course, but Netflix is probably a safer option for those with a preference to stand off. It’s also considerably more expensive, a fact that won’t have escaped users of these devices.

Got kids? Take care….

Finally, video four picks up a theme that’s becoming increasingly common in anti-piracy campaigns – malware and identity theft.

“Why risk having your identity stolen or your bank account or home network hacked. If you access entertainment or sports using dodgy streaming devices or apps, or illegal addons for Kodi, you are increasing the risks,” the ad warns.

Danger….Danger….

Perhaps of most interest is that this entire campaign, which almost certainly has Big Media behind the scenes in advisory and financial capacities, barely mentions the entertainment industries at all.

Indeed, the success of the whole campaign hinges on people worrying about the supposed ill effects of illicit streaming on them personally and then feeling persuaded to inform on suppliers and others involved in the chain.

“Know of someone supplying or promoting these dodgy devices or software? It is illegal. Call us now and help stop crime in your community,” the videos warn.

That CrimeStoppers has taken on this campaign at all is a bit of a head-scratcher, given the bigger crime picture. Struggling with severe budget cuts, police in the UK are already de-prioritizing a number of crimes, leading to something called “screening out”, a process through which victims are given a crime number but no investigation is carried out.

This means that in 2016, 45% of all reported crimes in Greater Manchester weren’t investigated and a staggering 57% of all recorded domestic burglaries weren’t followed up by the police. But it gets worse.

“More than 62pc of criminal damage and arson offenses were not investigated, along with one in three reported shoplifting incidents,” MEN reports.

Given this backdrop, how will police suddenly find the resources to follow up lots of leads from the public and then subsequently prosecute people who sell pirate boxes? Even if they do, will that be at the expense of yet more “screening out” of other public-focused offenses?

No one is saying that selling pirate devices isn’t a crime or at least worthy of being followed up, but is this niche likely to be important to the public when they’re being told that nothing will be done when their homes are emptied by intruders? “NO” says a comment on one of the CrimeStoppers videos on YouTube.

“This crime affects multi-million dollar corporations, I’d rather see tax payers money invested on videos raising awareness of crimes committed against the people rather than the 0.001%,” it concludes.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

GDPR – A Practical Guide For Developers

Post Syndicated from Bozho original https://techblog.bozho.net/gdpr-practical-guide-developers/

You’ve probably heard about GDPR. The new European data protection regulation that applies practically to everyone. Especially if you are working in a big company, it’s most likely that there’s already a process for gettign your systems in compliance with the regulation.

The regulation is basically a law that must be followed in all European countries (but also applies to non-EU companies that have users in the EU). In this particular case, it applies to companies that are not registered in Europe, but are having European customers. So that’s most companies. I will not go into yet another “12 facts about GDPR” or “7 myths about GDPR” posts/whitepapers, as they are often aimed at managers or legal people. Instead, I’ll focus on what GDPR means for developers.

Why am I qualified to do that? A few reasons – I was advisor to the deputy prime minister of a EU country, and because of that I’ve been both exposed and myself wrote some legislation. I’m familiar with the “legalese” and how the regulatory framework operates in general. I’m also a privacy advocate and I’ve been writing about GDPR-related stuff in the past, i.e. “before it was cool” (protecting sensitive data, the right to be forgotten). And finally, I’m currently working on a project that (among other things) aims to help with covering some GDPR aspects.

I’ll try to be a bit more comprehensive this time and cover as many aspects of the regulation that concern developers as I can. And while developers will mostly be concerned about how the systems they are working on have to change, it’s not unlikely that a less informed manager storms in in late spring, realizing GDPR is going to be in force tomorrow, asking “what should we do to get our system/website compliant”.

The rights of the user/client (referred to as “data subject” in the regulation) that I think are relevant for developers are: the right to erasure (the right to be forgotten/deleted from the system), right to restriction of processing (you still keep the data, but mark it as “restricted” and don’t touch it without further consent by the user), the right to data portability (the ability to export one’s data), the right to rectification (the ability to get personal data fixed), the right to be informed (getting human-readable information, rather than long terms and conditions), the right of access (the user should be able to see all the data you have about them), the right to data portability (the user should be able to get a machine-readable dump of their data).

Additionally, the relevant basic principles are: data minimization (one should not collect more data than necessary), integrity and confidentiality (all security measures to protect data that you can think of + measures to guarantee that the data has not been inappropriately modified).

Even further, the regulation requires certain processes to be in place within an organization (of more than 250 employees or if a significant amount of data is processed), and those include keeping a record of all types of processing activities carried out, including transfers to processors (3rd parties), which includes cloud service providers. None of the other requirements of the regulation have an exception depending on the organization size, so “I’m small, GDPR does not concern me” is a myth.

It is important to know what “personal data” is. Basically, it’s every piece of data that can be used to uniquely identify a person or data that is about an already identified person. It’s data that the user has explicitly provided, but also data that you have collected about them from either 3rd parties or based on their activities on the site (what they’ve been looking at, what they’ve purchased, etc.)

Having said that, I’ll list a number of features that will have to be implemented and some hints on how to do that, followed by some do’s and don’t’s.

  • “Forget me” – you should have a method that takes a userId and deletes all personal data about that user (in case they have been collected on the basis of consent, and not due to contract enforcement or legal obligation). It is actually useful for integration tests to have that feature (to cleanup after the test), but it may be hard to implement depending on the data model. In a regular data model, deleting a record may be easy, but some foreign keys may be violated. That means you have two options – either make sure you allow nullable foreign keys (for example an order usually has a reference to the user that made it, but when the user requests his data be deleted, you can set the userId to null), or make sure you delete all related data (e.g. via cascades). This may not be desirable, e.g. if the order is used to track available quantities or for accounting purposes. It’s a bit trickier for event-sourcing data models, or in extreme cases, ones that include some sort of blcokchain/hash chain/tamper-evident data structure. With event sourcing you should be able to remove a past event and re-generate intermediate snapshots. For blockchain-like structures – be careful what you put in there and avoid putting personal data of users. There is an option to use a chameleon hash function, but that’s suboptimal. Overall, you must constantly think of how you can delete the personal data. And “our data model doesn’t allow it” isn’t an excuse.
  • Notify 3rd parties for erasure – deleting things from your system may be one thing, but you are also obligated to inform all third parties that you have pushed that data to. So if you have sent personal data to, say, Salesforce, Hubspot, twitter, or any cloud service provider, you should call an API of theirs that allows for the deletion of personal data. If you are such a provider, obviously, your “forget me” endpoint should be exposed. Calling the 3rd party APIs to remove data is not the full story, though. You also have to make sure the information does not appear in search results. Now, that’s tricky, as Google doesn’t have an API for removal, only a manual process. Fortunately, it’s only about public profile pages that are crawlable by Google (and other search engines, okay…), but you still have to take measures. Ideally, you should make the personal data page return a 404 HTTP status, so that it can be removed.
  • Restrict processing – in your admin panel where there’s a list of users, there should be a button “restrict processing”. The user settings page should also have that button. When clicked (after reading the appropriate information), it should mark the profile as restricted. That means it should no longer be visible to the backoffice staff, or publicly. You can implement that with a simple “restricted” flag in the users table and a few if-clasues here and there.
  • Export data – there should be another button – “export data”. When clicked, the user should receive all the data that you hold about them. What exactly is that data – depends on the particular usecase. Usually it’s at least the data that you delete with the “forget me” functionality, but may include additional data (e.g. the orders the user has made may not be delete, but should be included in the dump). The structure of the dump is not strictly defined, but my recommendation would be to reuse schema.org definitions as much as possible, for either JSON or XML. If the data is simple enough, a CSV/XLS export would also be fine. Sometimes data export can take a long time, so the button can trigger a background process, which would then notify the user via email when his data is ready (twitter, for example, does that already – you can request all your tweets and you get them after a while).
  • Allow users to edit their profile – this seems an obvious rule, but it isn’t always followed. Users must be able to fix all data about them, including data that you have collected from other sources (e.g. using a “login with facebook” you may have fetched their name and address). Rule of thumb – all the fields in your “users” table should be editable via the UI. Technically, rectification can be done via a manual support process, but that’s normally more expensive for a business than just having the form to do it. There is one other scenario, however, when you’ve obtained the data from other sources (i.e. the user hasn’t provided their details to you directly). In that case there should still be a page where they can identify somehow (via email and/or sms confirmation) and get access to the data about them.
  • Consent checkboxes – this is in my opinion the biggest change that the regulation brings. “I accept the terms and conditions” would no longer be sufficient to claim that the user has given their consent for processing their data. So, for each particular processing activity there should be a separate checkbox on the registration (or user profile) screen. You should keep these consent checkboxes in separate columns in the database, and let the users withdraw their consent (by unchecking these checkboxes from their profile page – see the previous point). Ideally, these checkboxes should come directly from the register of processing activities (if you keep one). Note that the checkboxes should not be preselected, as this does not count as “consent”.
  • Re-request consent – if the consent users have given was not clear (e.g. if they simply agreed to terms & conditions), you’d have to re-obtain that consent. So prepare a functionality for mass-emailing your users to ask them to go to their profile page and check all the checkboxes for the personal data processing activities that you have.
  • “See all my data” – this is very similar to the “Export” button, except data should be displayed in the regular UI of the application rather than an XML/JSON format. For example, Google Maps shows you your location history – all the places that you’ve been to. It is a good implementation of the right to access. (Though Google is very far from perfect when privacy is concerned)
  • Age checks – you should ask for the user’s age, and if the user is a child (below 16), you should ask for parent permission. There’s no clear way how to do that, but my suggestion is to introduce a flow, where the child should specify the email of a parent, who can then confirm. Obviosuly, children will just cheat with their birthdate, or provide a fake parent email, but you will most likely have done your job according to the regulation (this is one of the “wishful thinking” aspects of the regulation).

Now some “do’s”, which are mostly about the technical measures needed to protect personal data. They may be more “ops” than “dev”, but often the application also has to be extended to support them. I’ve listed most of what I could think of in a previous post.

  • Encrypt the data in transit. That means that communication between your application layer and your database (or your message queue, or whatever component you have) should be over TLS. The certificates could be self-signed (and possibly pinned), or you could have an internal CA. Different databases have different configurations, just google “X encrypted connections. Some databases need gossiping among the nodes – that should also be configured to use encryption
  • Encrypt the data at rest – this again depends on the database (some offer table-level encryption), but can also be done on machine-level. E.g. using LUKS. The private key can be stored in your infrastructure, or in some cloud service like AWS KMS.
  • Encrypt your backups – kind of obvious
  • Implement pseudonymisation – the most obvious use-case is when you want to use production data for the test/staging servers. You should change the personal data to some “pseudonym”, so that the people cannot be identified. When you push data for machine learning purposes (to third parties or not), you can also do that. Technically, that could mean that your User object can have a “pseudonymize” method which applies hash+salt/bcrypt/PBKDF2 for some of the data that can be used to identify a person
  • Protect data integrity – this is a very broad thing, and could simply mean “have authentication mechanisms for modifying data”. But you can do something more, even as simple as a checksum, or a more complicated solution (like the one I’m working on). It depends on the stakes, on the way data is accessed, on the particular system, etc. The checksum can be in the form of a hash of all the data in a given database record, which should be updated each time the record is updated through the application. It isn’t a strong guarantee, but it is at least something.
  • Have your GDPR register of processing activities in something other than Excel – Article 30 says that you should keep a record of all the types of activities that you use personal data for. That sounds like bureaucracy, but it may be useful – you will be able to link certain aspects of your application with that register (e.g. the consent checkboxes, or your audit trail records). It wouldn’t take much time to implement a simple register, but the business requirements for that should come from whoever is responsible for the GDPR compliance. But you can advise them that having it in Excel won’t make it easy for you as a developer (imagine having to fetch the excel file internally, so that you can parse it and implement a feature). Such a register could be a microservice/small application deployed separately in your infrastructure.
  • Log access to personal data – every read operation on a personal data record should be logged, so that you know who accessed what and for what purpose
  • Register all API consumers – you shouldn’t allow anonymous API access to personal data. I’d say you should request the organization name and contact person for each API user upon registration, and add those to the data processing register. Note: some have treated article 30 as a requirement to keep an audit log. I don’t think it is saying that – instead it requires 250+ companies to keep a register of the types of processing activities (i.e. what you use the data for). There are other articles in the regulation that imply that keeping an audit log is a best practice (for protecting the integrity of the data as well as to make sure it hasn’t been processed without a valid reason)

Finally, some “don’t’s”.

  • Don’t use data for purposes that the user hasn’t agreed with – that’s supposed to be the spirit of the regulation. If you want to expose a new API to a new type of clients, or you want to use the data for some machine learning, or you decide to add ads to your site based on users’ behaviour, or sell your database to a 3rd party – think twice. I would imagine your register of processing activities could have a button to send notification emails to users to ask them for permission when a new processing activity is added (or if you use a 3rd party register, it should probably give you an API). So upon adding a new processing activity (and adding that to your register), mass email all users from whom you’d like consent.
  • Don’t log personal data – getting rid of the personal data from log files (especially if they are shipped to a 3rd party service) can be tedious or even impossible. So log just identifiers if needed. And make sure old logs files are cleaned up, just in case
  • Don’t put fields on the registration/profile form that you don’t need – it’s always tempting to just throw as many fields as the usability person/designer agrees on, but unless you absolutely need the data for delivering your service, you shouldn’t collect it. Names you should probably always collect, but unless you are delivering something, a home address or phone is unnecessary.
  • Don’t assume 3rd parties are compliant – you are responsible if there’s a data breach in one of the 3rd parties (e.g. “processors”) to which you send personal data. So before you send data via an API to another service, make sure they have at least a basic level of data protection. If they don’t, raise a flag with management.
  • Don’t assume having ISO XXX makes you compliant – information security standards and even personal data standards are a good start and they will probably 70% of what the regulation requires, but they are not sufficient – most of the things listed above are not covered in any of those standards

Overall, the purpose of the regulation is to make you take conscious decisions when processing personal data. It imposes best practices in a legal way. If you follow the above advice and design your data model, storage, data flow , API calls with data protection in mind, then you shouldn’t worry about the huge fines that the regulation prescribes – they are for extreme cases, like Equifax for example. Regulators (data protection authorities) will most likely have some checklists into which you’d have to somehow fit, but if you follow best practices, that shouldn’t be an issue.

I think all of the above features can be implemented in a few weeks by a small team. Be suspicious when a big vendor offers you a generic plug-and-play “GDPR compliance” solution. GDPR is not just about the technical aspects listed above – it does have organizational/process implications. But also be suspicious if a consultant claims GDPR is complicated. It’s not – it relies on a few basic principles that are in fact best practices anyway. Just don’t ignore them.

The post GDPR – A Practical Guide For Developers appeared first on Bozho's tech blog.

Object models

Post Syndicated from Eevee original https://eev.ee/blog/2017/11/28/object-models/

Anonymous asks, with dollars:

More about programming languages!

Well then!

I’ve written before about what I think objects are: state and behavior, which in practice mostly means method calls.

I suspect that the popular impression of what objects are, and also how they should work, comes from whatever C++ and Java happen to do. From that point of view, the whole post above is probably nonsense. If the baseline notion of “object” is a rigid definition woven tightly into the design of two massively popular languages, then it doesn’t even make sense to talk about what “object” should mean — it does mean the features of those languages, and cannot possibly mean anything else.

I think that’s a shame! It piles a lot of baggage onto a fairly simple idea. Polymorphism, for example, has nothing to do with objects — it’s an escape hatch for static type systems. Inheritance isn’t the only way to reuse code between objects, but it’s the easiest and fastest one, so it’s what we get. Frankly, it’s much closer to a speed tradeoff than a fundamental part of the concept.

We could do with more experimentation around how objects work, but that’s impossible in the languages most commonly thought of as object-oriented.

Here, then, is a (very) brief run through the inner workings of objects in four very dynamic languages. I don’t think I really appreciated objects until I’d spent some time with Python, and I hope this can help someone else whet their own appetite.

Python 3

Of the four languages I’m going to touch on, Python will look the most familiar to the Java and C++ crowd. For starters, it actually has a class construct.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
class Vector:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __neg__(self):
        return Vector(-self.x, -self.y)

    def __div__(self, denom):
        return Vector(self.x / denom, self.y / denom)

    @property
    def magnitude(self):
        return (self.x ** 2 + self.y ** 2) ** 0.5

    def normalized(self):
        return self / self.magnitude

The __init__ method is an initializer, which is like a constructor but named differently (because the object already exists in a usable form by the time the initializer is called). Operator overloading is done by implementing methods with other special __dunder__ names. Properties can be created with @property, where the @ is syntax for applying a wrapper function to a function as it’s defined. You can do inheritance, even multiply:

1
2
3
4
class Foo(A, B, C):
    def bar(self, x, y, z):
        # do some stuff
        super().bar(x, y, z)

Cool, a very traditional object model.

Except… for some details.

Some details

For one, Python objects don’t have a fixed layout. Code both inside and outside the class can add or remove whatever attributes they want from whatever object they want. The underlying storage is just a dict, Python’s mapping type. (Or, rather, something like one. Also, it’s possible to change, which will probably be the case for everything I say here.)

If you create some attributes at the class level, you’ll start to get a peek behind the curtains:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
class Foo:
    values = []

    def add_value(self, value):
        self.values.append(value)

a = Foo()
b = Foo()
a.add_value('a')
print(a.values)  # ['a']
b.add_value('b')
print(b.values)  # ['a', 'b']

The [] assigned to values isn’t a default assigned to each object. In fact, the individual objects don’t know about it at all! You can use vars(a) to get at the underlying storage dict, and you won’t see a values entry in there anywhere.

Instead, values lives on the class, which is a value (and thus an object) in its own right. When Python is asked for self.values, it checks to see if self has a values attribute; in this case, it doesn’t, so Python keeps going and asks the class for one.

Python’s object model is secretly prototypical — a class acts as a prototype, as a shared set of fallback values, for its objects.

In fact, this is also how method calls work! They aren’t syntactically special at all, which you can see by separating the attribute lookup from the call.

1
2
3
print("abc".startswith("a"))  # True
meth = "abc".startswith
print(meth("a"))  # True

Reading obj.method looks for a method attribute; if there isn’t one on obj, Python checks the class. Here, it finds one: it’s a function from the class body.

Ah, but wait! In the code I just showed, meth seems to “know” the object it came from, so it can’t just be a plain function. If you inspect the resulting value, it claims to be a “bound method” or “built-in method” rather than a function, too. Something funny is going on here, and that funny something is the descriptor protocol.

Descriptors

Python allows attributes to implement their own custom behavior when read from or written to. Such an attribute is called a descriptor. I’ve written about them before, but here’s a quick overview.

If Python looks up an attribute, finds it in a class, and the value it gets has a __get__ method… then instead of using that value, Python will use the return value of its __get__ method.

The @property decorator works this way. The magnitude property in my original example was shorthand for doing this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
class MagnitudeDescriptor:
    def __get__(self, instance, owner):
        if instance is None:
            return self
        return (instance.x ** 2 + instance.y ** 2) ** 0.5

class Vector:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    magnitude = MagnitudeDescriptor()

When you ask for somevec.magnitude, Python checks somevec but doesn’t find magnitude, so it consults the class instead. The class does have a magnitude, and it’s a value with a __get__ method, so Python calls that method and somevec.magnitude evaluates to its return value. (The instance is None check is because __get__ is called even if you get the descriptor directly from the class via Vector.magnitude. A descriptor intended to work on instances can’t do anything useful in that case, so the convention is to return the descriptor itself.)

You can also intercept attempts to write to or delete an attribute, and do absolutely whatever you want instead. But note that, similar to operating overloading in Python, the descriptor must be on a class; you can’t just slap one on an arbitrary object and have it work.

This brings me right around to how “bound methods” actually work. Functions are descriptors! The function type implements __get__, and when a function is retrieved from a class via an instance, that __get__ bundles the function and the instance together into a tiny bound method object. It’s essentially:

1
2
3
4
5
class FunctionType:
    def __get__(self, instance, owner):
        if instance is None:
            return self
        return functools.partial(self, instance)

The self passed as the first argument to methods is not special or magical in any way. It’s built out of a few simple pieces that are also readily accessible to Python code.

Note also that because obj.method() is just an attribute lookup and a call, Python doesn’t actually care whether method is a method on the class or just some callable thing on the object. You won’t get the auto-self behavior if it’s on the object, but otherwise there’s no difference.

More attribute access, and the interesting part

Descriptors are one of several ways to customize attribute access. Classes can implement __getattr__ to intervene when an attribute isn’t found on an object; __setattr__ and __delattr__ to intervene when any attribute is set or deleted; and __getattribute__ to implement unconditional attribute access. (That last one is a fantastic way to create accidental recursion, since any attribute access you do within __getattribute__ will of course call __getattribute__ again.)

Here’s what I really love about Python. It might seem like a magical special case that descriptors only work on classes, but it really isn’t. You could implement exactly the same behavior yourself, in pure Python, using only the things I’ve just told you about. Classes are themselves objects, remember, and they are instances of type, so the reason descriptors only work on classes is that type effectively does this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
class type:
    def __getattribute__(self, name):
        value = super().__getattribute__(name)
        # like all op overloads, __get__ must be on the type, not the instance
        ty = type(value)
        if hasattr(ty, '__get__'):
            # it's a descriptor!  this is a class access so there is no instance
            return ty.__get__(value, None, self)
        else:
            return value

You can even trivially prove to yourself that this is what’s going on by skipping over types behavior:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
class Descriptor:
    def __get__(self, instance, owner):
        print('called!')

class Foo:
    bar = Descriptor()

Foo.bar  # called!
type.__getattribute__(Foo, 'bar')  # called!
object.__getattribute__(Foo, 'bar')  # ...

And that’s not all! The mysterious super function, used to exhaustively traverse superclass method calls even in the face of diamond inheritance, can also be expressed in pure Python using these primitives. You could write your own superclass calling convention and use it exactly the same way as super.

This is one of the things I really like about Python. Very little of it is truly magical; virtually everything about the object model exists in the types rather than the language, which means virtually everything can be customized in pure Python.

Class creation and metaclasses

A very brief word on all of this stuff, since I could talk forever about Python and I have three other languages to get to.

The class block itself is fairly interesting. It looks like this:

1
2
class Name(*bases, **kwargs):
    # code

I’ve said several times that classes are objects, and in fact the class block is one big pile of syntactic sugar for calling type(...) with some arguments to create a new type object.

The Python documentation has a remarkably detailed description of this process, but the gist is:

  • Python determines the type of the new class — the metaclass — by looking for a metaclass keyword argument. If there isn’t one, Python uses the “lowest” type among the provided base classes. (If you’re not doing anything special, that’ll just be type, since every class inherits from object and object is an instance of type.)

  • Python executes the class body. It gets its own local scope, and any assignments or method definitions go into that scope.

  • Python now calls type(name, bases, attrs, **kwargs). The name is whatever was right after class; the bases are position arguments; and attrs is the class body’s local scope. (This is how methods and other class attributes end up on the class.) The brand new type is then assigned to Name.

Of course, you can mess with most of this. You can implement __prepare__ on a metaclass, for example, to use a custom mapping as storage for the local scope — including any reads, which allows for some interesting shenanigans. The only part you can’t really implement in pure Python is the scoping bit, which has a couple extra rules that make sense for classes. (In particular, functions defined within a class block don’t close over the class body; that would be nonsense.)

Object creation

Finally, there’s what actually happens when you create an object — including a class, which remember is just an invocation of type(...).

Calling Foo(...) is implemented as, well, a call. Any type can implement calls with the __call__ special method, and you’ll find that type itself does so. It looks something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# oh, a fun wrinkle that's hard to express in pure python: type is a class, so
# it's an instance of itself
class type:
    def __call__(self, *args, **kwargs):
        # remember, here 'self' is a CLASS, an instance of type.
        # __new__ is a true constructor: object.__new__ allocates storage
        # for a new blank object
        instance = self.__new__(self, *args, **kwargs)
        # you can return whatever you want from __new__ (!), and __init__
        # is only called on it if it's of the right type
        if isinstance(instance, self):
            instance.__init__(*args, **kwargs)
        return instance

Again, you can trivially confirm this by asking any type for its __call__ method. Assuming that type doesn’t implement __call__ itself, you’ll get back a bound version of types implementation.

1
2
>>> list.__call__
<method-wrapper '__call__' of type object at 0x7fafb831a400>

You can thus implement __call__ in your own metaclass to completely change how subclasses are created — including skipping the creation altogether, if you like.

And… there’s a bunch of stuff I haven’t even touched on.

The Python philosophy

Python offers something that, on the surface, looks like a “traditional” class/object model. Under the hood, it acts more like a prototypical system, where failed attribute lookups simply defer to a superclass or metaclass.

The language also goes to almost superhuman lengths to expose all of its moving parts. Even the prototypical behavior is an implementation of __getattribute__ somewhere, which you are free to completely replace in your own types. Proxying and delegation are easy.

Also very nice is that these features “bundle” well, by which I mean a library author can do all manner of convoluted hijinks, and a consumer of that library doesn’t have to see any of it or understand how it works. You only need to inherit from a particular class (which has a metaclass), or use some descriptor as a decorator, or even learn any new syntax.

This meshes well with Python culture, which is pretty big on the principle of least surprise. These super-advanced features tend to be tightly confined to single simple features (like “makes a weak attribute“) or cordoned with DSLs (e.g., defining a form/struct/database table with a class body). In particular, I’ve never seen a metaclass in the wild implement its own __call__.

I have mixed feelings about that. It’s probably a good thing overall that the Python world shows such restraint, but I wonder if there are some very interesting possibilities we’re missing out on. I implemented a metaclass __call__ myself, just once, in an entity/component system that strove to minimize fuss when communicating between components. It never saw the light of day, but I enjoyed seeing some new things Python could do with the same relatively simple syntax. I wouldn’t mind seeing, say, an object model based on composition (with no inheritance) built atop Python’s primitives.

Lua

Lua doesn’t have an object model. Instead, it gives you a handful of very small primitives for building your own object model. This is pretty typical of Lua — it’s a very powerful language, but has been carefully constructed to be very small at the same time. I’ve never encountered anything else quite like it, and “but it starts indexing at 1!” really doesn’t do it justice.

The best way to demonstrate how objects work in Lua is to build some from scratch. We need two key features. The first is metatables, which bear a passing resemblance to Python’s metaclasses.

Tables and metatables

The table is Lua’s mapping type and its primary data structure. Keys can be any value other than nil. Lists are implemented as tables whose keys are consecutive integers starting from 1. Nothing terribly surprising. The dot operator is sugar for indexing with a string key.

1
2
3
4
5
local t = { a = 1, b = 2 }
print(t['a'])  -- 1
print(t.b)  -- 2
t.c = 3
print(t['c'])  -- 3

A metatable is a table that can be associated with another value (usually another table) to change its behavior. For example, operator overloading is implemented by assigning a function to a special key in a metatable.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
local t = { a = 1, b = 2 }
--print(t + 0)  -- error: attempt to perform arithmetic on a table value

local mt = {
    __add = function(left, right)
        return 12
    end,
}
setmetatable(t, mt)
print(t + 0)  -- 12

Now, the interesting part: one of the special keys is __index, which is consulted when the base table is indexed by a key it doesn’t contain. Here’s a table that claims every key maps to itself.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
local t = {}
local mt = {
    __index = function(table, key)
        return key
    end,
}
setmetatable(t, mt)
print(t.foo)  -- foo
print(t.bar)  -- bar
print(t[3])  -- 3

__index doesn’t have to be a function, either. It can be yet another table, in which case that table is simply indexed with the key. If the key still doesn’t exist and that table has a metatable with an __index, the process repeats.

With this, it’s easy to have several unrelated tables that act as a single table. Call the base table an object, fill the __index table with functions and call it a class, and you have half of an object system. You can even get prototypical inheritance by chaining __indexes together.

At this point things are a little confusing, since we have at least three tables going on, so here’s a diagram. Keep in mind that Lua doesn’t actually have anything called an “object”, “class”, or “method” — those are just convenient nicknames for a particular structure we might build with Lua’s primitives.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
                    ╔═══════════╗        ...
                    ║ metatable ║         ║
                    ╟───────────╢   ┌─────╨───────────────────────┐
                    ║ __index   ╫───┤ lookup table ("superclass") │
                    ╚═══╦═══════╝   ├─────────────────────────────┤
  ╔═══════════╗         ║           │ some other method           ┼─── function() ... end
  ║ metatable ║         ║           └─────────────────────────────┘
  ╟───────────╢   ┌─────╨──────────────────┐
  ║ __index   ╫───┤ lookup table ("class") │
  ╚═══╦═══════╝   ├────────────────────────┤
      ║           │ some method            ┼─── function() ... end
      ║           └────────────────────────┘
┌─────╨─────────────────┐
│ base table ("object") │
└───────────────────────┘

Note that a metatable is not the same as a class; it defines behavior, not methods. Conversely, if you try to use a class directly as a metatable, it will probably not do much. (This is pretty different from e.g. Python, where operator overloads are just methods with funny names. One nice thing about the Lua approach is that you can keep interface-like functionality separate from methods, and avoid clogging up arbitrary objects’ namespaces. You could even use a dummy table as a key and completely avoid name collisions.)

Anyway, code!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
local class = {
    foo = function(a)
        print("foo got", a)
    end,
}
local mt = { __index = class }
-- setmetatable returns its first argument, so this is nice shorthand
local obj1 = setmetatable({}, mt)
local obj2 = setmetatable({}, mt)
obj1.foo(7)  -- foo got 7
obj2.foo(9)  -- foo got 9

Wait, wait, hang on. Didn’t I call these methods? How do they get at the object? Maybe Lua has a magical this variable?

Methods, sort of

Not quite, but this is where the other key feature comes in: method-call syntax. It’s the lightest touch of sugar, just enough to have method invocation.

1
2
3
4
5
6
7
8
9
-- note the colon!
a:b(c, d, ...)

-- exactly equivalent to this
-- (except that `a` is only evaluated once)
a.b(a, c, d, ...)

-- which of course is really this
a["b"](a, c, d, ...)

Now we can write methods that actually do something.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
local class = {
    bar = function(self)
        print("our score is", self.score)
    end,
}
local mt = { __index = class }
local obj1 = setmetatable({ score = 13 }, mt)
local obj2 = setmetatable({ score = 25 }, mt)
obj1:bar()  -- our score is 13
obj2:bar()  -- our score is 25

And that’s all you need. Much like Python, methods and data live in the same namespace, and Lua doesn’t care whether obj:method() finds a function on obj or gets one from the metatable’s __index. Unlike Python, the function will be passed self either way, because self comes from the use of : rather than from the lookup behavior.

(Aside: strictly speaking, any Lua value can have a metatable — and if you try to index a non-table, Lua will always consult the metatable’s __index. Strings all have the string library as a metatable, so you can call methods on them: try ("%s %s"):format(1, 2). I don’t think Lua lets user code set the metatable for non-tables, so this isn’t that interesting, but if you’re writing Lua bindings from C then you can wrap your pointers in metatables to give them methods implemented in C.)

Bringing it all together

Of course, writing all this stuff every time is a little tedious and error-prone, so instead you might want to wrap it all up inside a little function. No problem.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
local function make_object(body)
    -- create a metatable
    local mt = { __index = body }
    -- create a base table to serve as the object itself
    local obj = setmetatable({}, mt)
    -- and, done
    return obj
end

-- you can leave off parens if you're only passing in 
local Dog = {
    -- this acts as a "default" value; if obj.barks is missing, __index will
    -- kick in and find this value on the class.  but if obj.barks is assigned
    -- to, it'll go in the object and shadow the value here.
    barks = 0,

    bark = function(self)
        self.barks = self.barks + 1
        print("woof!")
    end,
}

local mydog = make_object(Dog)
mydog:bark()  -- woof!
mydog:bark()  -- woof!
mydog:bark()  -- woof!
print(mydog.barks)  -- 3
print(Dog.barks)  -- 0

It works, but it’s fairly barebones. The nice thing is that you can extend it pretty much however you want. I won’t reproduce an entire serious object system here — lord knows there are enough of them floating around — but the implementation I have for my LÖVE games lets me do this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
local Animal = Object:extend{
    cries = 0,
}

-- called automatically by Object
function Animal:init()
    print("whoops i couldn't think of anything interesting to put here")
end

-- this is just nice syntax for adding a first argument called 'self', then
-- assigning this function to Animal.cry
function Animal:cry()
    self.cries = self.cries + 1
end

local Cat = Animal:extend{}

function Cat:cry()
    print("meow!")
    Cat.__super.cry(self)
end

local cat = Cat()
cat:cry()  -- meow!
cat:cry()  -- meow!
print(cat.cries)  -- 2

When I say you can extend it however you want, I mean that. I could’ve implemented Python (2)-style super(Cat, self):cry() syntax; I just never got around to it. I could even make it work with multiple inheritance if I really wanted to — or I could go the complete opposite direction and only implement composition. I could implement descriptors, customizing the behavior of individual table keys. I could add pretty decent syntax for composition/proxying. I am trying very hard to end this section now.

The Lua philosophy

Lua’s philosophy is to… not have a philosophy? It gives you the bare minimum to make objects work, and you can do absolutely whatever you want from there. Lua does have something resembling prototypical inheritance, but it’s not so much a first-class feature as an emergent property of some very simple tools. And since you can make __index be a function, you could avoid the prototypical behavior and do something different entirely.

The very severe downside, of course, is that you have to find or build your own object system — which can get pretty confusing very quickly, what with the multiple small moving parts. Third-party code may also have its own object system with subtly different behavior. (Though, in my experience, third-party code tries very hard to avoid needing an object system at all.)

It’s hard to say what the Lua “culture” is like, since Lua is an embedded language that’s often a little different in each environment. I imagine it has a thousand millicultures, instead. I can say that the tedium of building my own object model has led me into something very “traditional”, with prototypical inheritance and whatnot. It’s partly what I’m used to, but it’s also just really dang easy to get working.

Likewise, while I love properties in Python and use them all the dang time, I’ve yet to use a single one in Lua. They wouldn’t be particularly hard to add to my object model, but having to add them myself (or shop around for an object model with them and also port all my code to use it) adds a huge amount of friction. I’ve thought about designing an interesting ECS with custom object behavior, too, but… is it really worth the effort? For all the power and flexibility Lua offers, the cost is that by the time I have something working at all, I’m too exhausted to actually use any of it.

JavaScript

JavaScript is notable for being preposterously heavily used, yet not having a class block.

Well. Okay. Yes. It has one now. It didn’t for a very long time, and even the one it has now is sugar.

Here’s a vector class again:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
class Vector {
    constructor(x, y) {
        this.x = x;
        this.y = y;
    }

    get magnitude() {
        return Math.sqrt(this.x * this.x + this.y * this.y);
    }

    dot(other) {
        return this.x * other.x + this.y * other.y;
    }
}

In “classic” JavaScript, this would be written as:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
function Vector(x, y) {
    this.x = x;
    this.y = y;
}

Object.defineProperty(Vector.prototype, 'magnitude', {
    configurable: true,
    enumerable: true,
    get: function() {
        return Math.sqrt(this.x * this.x + this.y * this.y);
    },
});


Vector.prototype.dot = function(other) {
    return this.x * other.x + this.y * other.y;
};

Hm, yes. I can see why they added class.

The JavaScript model

In JavaScript, a new type is defined in terms of a function, which is its constructor.

Right away we get into trouble here. There is a very big difference between these two invocations, which I actually completely forgot about just now after spending four hours writing about Python and Lua:

1
2
let vec = Vector(3, 4);
let vec = new Vector(3, 4);

The first calls the function Vector. It assigns some properties to this, which here is going to be window, so now you have a global x and y. It then returns nothing, so vec is undefined.

The second calls Vector with this set to a new empty object, then evaluates to that object. The result is what you’d actually expect.

(You can detect this situation with the strange new.target expression, but I have never once remembered to do so.)

From here, we have true, honest-to-god, first-class prototypical inheritance. The word “prototype” is even right there. When you write this:

1
vec.dot(vec2)

JavaScript will look for dot on vec and (presumably) not find it. It then consults vecs prototype, an object you can see for yourself by using Object.getPrototypeOf(). Since vec is a Vector, its prototype is Vector.prototype.

I stress that Vector.prototype is not the prototype for Vector. It’s the prototype for instances of Vector.

(I say “instance”, but the true type of vec here is still just object. If you want to find Vector, it’s automatically assigned to the constructor property of its own prototype, so it’s available as vec.constructor.)

Of course, Vector.prototype can itself have a prototype, in which case the process would continue if dot were not found. A common (and, arguably, very bad) way to simulate single inheritance is to set Class.prototype to an instance of a superclass to get the prototype right, then tack on the methods for Class. Nowadays we can do Object.create(Superclass.prototype).

Now that I’ve been through Python and Lua, though, this isn’t particularly surprising. I kinda spoiled it.

I suppose one difference in JavaScript is that you can tack arbitrary attributes directly onto Vector all you like, and they will remain invisible to instances since they aren’t in the prototype chain. This is kind of backwards from Lua, where you can squirrel stuff away in the metatable.

Another difference is that every single object in JavaScript has a bunch of properties already tacked on — the ones in Object.prototype. Every object (and by “object” I mean any mapping) has a prototype, and that prototype defaults to Object.prototype, and it has a bunch of ancient junk like isPrototypeOf.

(Nit: it’s possible to explicitly create an object with no prototype via Object.create(null).)

Like Lua, and unlike Python, JavaScript doesn’t distinguish between keys found on an object and keys found via a prototype. Properties can be defined on prototypes with Object.defineProperty(), but that works just as well directly on an object, too. JavaScript doesn’t have a lot of operator overloading, but some things like Symbol.iterator also work on both objects and prototypes.

About this

You may, at this point, be wondering what this is. Unlike Lua and Python (and the last language below), this is a special built-in value — a context value, invisibly passed for every function call.

It’s determined by where the function came from. If the function was the result of an attribute lookup, then this is set to the object containing that attribute. Otherwise, this is set to the global object, window. (You can also set this to whatever you want via the call method on functions.)

This decision is made lexically, i.e. from the literal source code as written. There are no Python-style bound methods. In other words:

1
2
3
4
5
// this = obj
obj.method()
// this = window
let meth = obj.method
meth()

Also, because this is reassigned on every function call, it cannot be meaningfully closed over, which makes using closures within methods incredibly annoying. The old approach was to assign this to some other regular name like self (which got syntax highlighting since it’s also a built-in name in browsers); then we got Function.bind, which produced a callable thing with a fixed context value, which was kind of nice; and now finally we have arrow functions, which explicitly close over the current this when they’re defined and don’t change it when called. Phew.

Class syntax

I already showed class syntax, and it’s really just one big macro for doing all the prototype stuff The Right Way. It even prevents you from calling the type without new. The underlying model is exactly the same, and you can inspect all the parts.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
class Vector { ... }

console.log(Vector.prototype);  // { dot: ..., magnitude: ..., ... }
let vec = new Vector(3, 4);
console.log(Object.getPrototypeOf(vec));  // same as Vector.prototype

// i don't know why you would subclass vector but let's roll with it
class Vectest extends Vector { ... }

console.log(Vectest.prototype);  // { ... }
console.log(Object.getPrototypeOf(Vectest.prototype))  // same as Vector.prototype

Alas, class syntax has a couple shortcomings. You can’t use the class block to assign arbitrary data to either the type object or the prototype — apparently it was deemed too confusing that mutations would be shared among instances. Which… is… how prototypes work. How Python works. How JavaScript itself, one of the most popular languages of all time, has worked for twenty-two years. Argh.

You can still do whatever assignment you want outside of the class block, of course. It’s just a little ugly, and not something I’d think to look for with a sugary class.

A more subtle result of this behavior is that a class block isn’t quite the same syntax as an object literal. The check for data isn’t a runtime thing; class Foo { x: 3 } fails to parse. So JavaScript now has two largely but not entirely identical styles of key/value block.

Attribute access

Here’s where things start to come apart at the seams, just a little bit.

JavaScript doesn’t really have an attribute protocol. Instead, it has two… extension points, I suppose.

One is Object.defineProperty, seen above. For common cases, there’s also the get syntax inside a property literal, which does the same thing. But unlike Python’s @property, these aren’t wrappers around some simple primitives; they are the primitives. JavaScript is the only language of these four to have “property that runs code on access” as a completely separate first-class concept.

If you want to intercept arbitrary attribute access (and some kinds of operators), there’s a completely different primitive: the Proxy type. It doesn’t let you intercept attribute access or operators; instead, it produces a wrapper object that supports interception and defers to the wrapped object by default.

It’s cool to see composition used in this way, but also, extremely weird. If you want to make your own type that overloads in or calling, you have to return a Proxy that wraps your own type, rather than actually returning your own type. And (unlike the other three languages in this post) you can’t return a different type from a constructor, so you have to throw that away and produce objects only from a factory. And instanceof would be broken, but you can at least fix that with Symbol.hasInstance — which is really operator overloading, implement yet another completely different way.

I know the design here is a result of legacy and speed — if any object could intercept all attribute access, then all attribute access would be slowed down everywhere. Fair enough. It still leaves the surface area of the language a bit… bumpy?

The JavaScript philosophy

It’s a little hard to tell. The original idea of prototypes was interesting, but it was hidden behind some very awkward syntax. Since then, we’ve gotten a bunch of extra features awkwardly bolted on to reflect the wildly varied things the built-in types and DOM API were already doing. We have class syntax, but it’s been explicitly designed to avoid exposing the prototype parts of the model.

I admit I don’t do a lot of heavy JavaScript, so I might just be overlooking it, but I’ve seen virtually no code that makes use of any of the recent advances in object capabilities. Forget about custom iterators or overloading call; I can’t remember seeing any JavaScript in the wild that even uses properties yet. I don’t know if everyone’s waiting for sufficient browser support, nobody knows about them, or nobody cares.

The model has advanced recently, but I suspect JavaScript is still shackled to its legacy of “something about prototypes, I don’t really get it, just copy the other code that’s there” as an object model. Alas! Prototypes are so good. Hopefully class syntax will make it a bit more accessible, as it has in Python.

Perl 5

Perl 5 also doesn’t have an object system and expects you to build your own. But where Lua gives you two simple, powerful tools for building one, Perl 5 feels more like a puzzle with half the pieces missing. Clearly they were going for something, but they only gave you half of it.

In brief, a Perl object is a reference that has been blessed with a package.

I need to explain a few things. Honestly, one of the biggest problems with the original Perl object setup was how many strange corners and unique jargon you had to understand just to get off the ground.

(If you want to try running any of this code, you should stick a use v5.26; as the first line. Perl is very big on backwards compatibility, so you need to opt into breaking changes, and even the mundane say builtin is behind a feature gate.)

References

A reference in Perl is sort of like a pointer, but its main use is very different. See, Perl has the strange property that its data structures try very hard to spill their contents all over the place. Despite having dedicated syntax for arrays — @foo is an array variable, distinct from the single scalar variable $foo — it’s actually impossible to nest arrays.

1
2
3
my @foo = (1, 2, 3, 4);
my @bar = (@foo, @foo);
# @bar is now a flat list of eight items: 1, 2, 3, 4, 1, 2, 3, 4

The idea, I guess, is that an array is not one thing. It’s not a container, which happens to hold multiple things; it is multiple things. Anywhere that expects a single value, such as an array element, cannot contain an array, because an array fundamentally is not a single value.

And so we have “references”, which are a form of indirection, but also have the nice property that they’re single values. They add containment around arrays, and in general they make working with most of Perl’s primitive types much more sensible. A reference to a variable can be taken with the \ operator, or you can use [ ... ] and { ... } to directly create references to anonymous arrays or hashes.

1
2
3
my @foo = (1, 2, 3, 4);
my @bar = (\@foo, \@foo);
# @bar is now a nested list of two items: [1, 2, 3, 4], [1, 2, 3, 4]

(Incidentally, this is the sole reason I initially abandoned Perl for Python. Non-trivial software kinda requires nesting a lot of data structures, so you end up with references everywhere, and the syntax for going back and forth between a reference and its contents is tedious and ugly.)

A Perl object must be a reference. Perl doesn’t care what kind of reference — it’s usually a hash reference, since hashes are a convenient place to store arbitrary properties, but it could just as well be a reference to an array, a scalar, or even a sub (i.e. function) or filehandle.

I’m getting a little ahead of myself. First, the other half: blessing and packages.

Packages and blessing

Perl packages are just namespaces. A package looks like this:

1
2
3
4
5
6
7
package Foo::Bar;

sub quux {
    say "hi from quux!";
}

# now Foo::Bar::quux() can be called from anywhere

Nothing shocking, right? It’s just a named container. A lot of the details are kind of weird, like how a package exists in some liminal quasi-value space, but the basic idea is a Bag Of Stuff.

The final piece is “blessing,” which is Perl’s funny name for binding a package to a reference. A very basic class might look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
package Vector;

# the name 'new' is convention, not special
sub new {
    # perl argument passing is weird, don't ask
    my ($class, $x, $y) = @_;

    # create the object itself -- here, unusually, an array reference makes sense
    my $self = [ $x, $y ];

    # associate the package with that reference
    # note that $class here is just the regular string, 'Vector'
    bless $self, $class;

    return $self;
}

sub x {
    my ($self) = @_;
    return $self->[0];
}

sub y {
    my ($self) = @_;
    return $self->[1];
}

sub magnitude {
    my ($self) = @_;
    return sqrt($self->x ** 2 + $self->y ** 2);
}

# switch back to the "default" package
package main;

# -> is method call syntax, which passes the invocant as the first argument;
# for a package, that's just the package name
my $vec = Vector->new(3, 4);
say $vec->magnitude;  # 5

A few things of note here. First, $self->[0] has nothing to do with objects; it’s normal syntax for getting the value of a index 0 out of an array reference called $self. (Most classes are based on hashrefs and would use $self->{value} instead.) A blessed reference is still a reference and can be treated like one.

In general, -> is Perl’s dereferencey operator, but its exact behavior depends on what follows. If it’s followed by brackets, then it’ll apply the brackets to the thing in the reference: ->{} to index a hash reference, ->[] to index an array reference, and ->() to call a function reference.

But if -> is followed by an identifier, then it’s a method call. For packages, that means calling a function in the package and passing the package name as the first argument. For objects — blessed references — that means calling a function in the associated package and passing the object as the first argument.

This is a little weird! A blessed reference is a superposition of two things: its normal reference behavior, and some completely orthogonal object behavior. Also, object behavior has no notion of methods vs data; it only knows about methods. Perl lets you omit parentheses in a lot of places, including when calling a method with no arguments, so $vec->magnitude is really $vec->magnitude().

Perl’s blessing bears some similarities to Lua’s metatables, but ultimately Perl is much closer to Ruby’s “message passing” approach than the above three languages’ approaches of “get me something and maybe it’ll be callable”. (But this is no surprise — Ruby is a spiritual successor to Perl 5.)

All of this leads to one little wrinkle: how do you actually expose data? Above, I had to write x and y methods. Am I supposed to do that for every single attribute on my type?

Yes! But don’t worry, there are third-party modules to help with this incredibly fundamental task. Take Class::Accessor::Fast, so named because it’s faster than Class::Accessor:

1
2
3
package Foo;
use base qw(Class::Accessor::Fast);
__PACKAGE__->mk_accessors(qw(fred wilma barney));

(__PACKAGE__ is the lexical name of the current package; qw(...) is a list literal that splits its contents on whitespace.)

This assumes you’re using a hashref with keys of the same names as the attributes. $obj->fred will return the fred key from your hashref, and $obj->fred(4) will change it to 4.

You also, somewhat bizarrely, have to inherit from Class::Accessor::Fast. Speaking of which,

Inheritance

Inheritance is done by populating the package-global @ISA array with some number of (string) names of parent packages. Most code instead opts to write use base ...;, which does the same thing. Or, more commonly, use parent ...;, which… also… does the same thing.

Every package implicitly inherits from UNIVERSAL, which can be freely modified by Perl code.

A method can call its superclass method with the SUPER:: pseudo-package:

1
2
3
4
sub foo {
    my ($self) = @_;
    $self->SUPER::foo;
}

However, this does a depth-first search, which means it almost certainly does the wrong thing when faced with multiple inheritance. For a while the accepted solution involved a third-party module, but Perl eventually grew an alternative you have to opt into: C3, which may be more familiar to you as the order Python uses.

1
2
3
4
5
6
use mro 'c3';

sub foo {
    my ($self) = @_;
    $self->next::method;
}

Offhand, I’m not actually sure how next::method works, seeing as it was originally implemented in pure Perl code. I suspect it involves peeking at the caller’s stack frame. If so, then this is a very different style of customizability from e.g. Python — the MRO was never intended to be pluggable, and the use of a special pseudo-package means it isn’t really, but someone was determined enough to make it happen anyway.

Operator overloading and whatnot

Operator overloading looks a little weird, though really it’s pretty standard Perl.

1
2
3
4
5
6
7
8
package MyClass;

use overload '+' => \&_add;

sub _add {
    my ($self, $other, $swap) = @_;
    ...
}

use overload here is a pragma, where “pragma” means “regular-ass module that does some wizardry when imported”.

\&_add is how you get a reference to the _add sub so you can pass it to the overload module. If you just said &_add or _add, that would call it.

And that’s it; you just pass a map of operators to functions to this built-in module. No worry about name clashes or pollution, which is pretty nice. You don’t even have to give references to functions that live in the package, if you don’t want them to clog your namespace; you could put them in another package, or even inline them anonymously.

One especially interesting thing is that Perl lets you overload every operator. Perl has a lot of operators. It considers some math builtins like sqrt and trig functions to be operators, or at least operator-y enough that you can overload them. You can also overload the “file text” operators, such as -e $path to test whether a file exists. You can overload conversions, including implicit conversion to a regex. And most fascinating to me, you can overload dereferencing — that is, the thing Perl does when you say $hashref->{key} to get at the underlying hash. So a single object could pretend to be references of multiple different types, including a subref to implement callability. Neat.

Somewhat related: you can overload basic operators (indexing, etc.) on basic types (not references!) with the tie function, which is designed completely differently and looks for methods with fixed names. Go figure.

You can intercept calls to nonexistent methods by implementing a function called AUTOLOAD, within which the $AUTOLOAD global will contain the name of the method being called. Originally this feature was, I think, intended for loading binary components or large libraries on-the-fly only when needed, hence the name. Offhand I’m not sure I ever saw it used the way __getattr__ is used in Python.

Is there a way to intercept all method calls? I don’t think so, but it is Perl, so I must be forgetting something.

Actually no one does this any more

Like a decade ago, a council of elder sages sat down and put together a whole whizbang system that covers all of it: Moose.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
package Vector;
use Moose;

has x => (is => 'rw', isa => 'Int');
has y => (is => 'rw', isa => 'Int');

sub magnitude {
    my ($self) = @_;
    return sqrt($self->x ** 2 + $self->y ** 2);
}

Moose has its own way to do pretty much everything, and it’s all built on the same primitives. Moose also adds metaclasses, somehow, despite that the underlying model doesn’t actually support them? I’m not entirely sure how they managed that, but I do remember doing some class introspection with Moose and it was much nicer than the built-in way.

(If you’re wondering, the built-in way begins with looking at the hash called %Vector::. No, that’s not a typo.)

I really cannot stress enough just how much stuff Moose does, but I don’t want to delve into it here since Moose itself is not actually the language model.

The Perl philosophy

I hope you can see what I meant with what I first said about Perl, now. It has multiple inheritance with an MRO, but uses the wrong one by default. It has extensive operator overloading, which looks nothing like how inheritance works, and also some of it uses a totally different mechanism with special method names instead. It only understands methods, not data, leaving you to figure out accessors by hand.

There’s 70% of an object system here with a clear general design it was gunning for, but none of the pieces really look anything like each other. It’s weird, in a distinctly Perl way.

The result is certainly flexible, at least! It’s especially cool that you can use whatever kind of reference you want for storage, though even as I say that, I acknowledge it’s no different from simply subclassing list or something in Python. It feels different in Perl, but maybe only because it looks so different.

I haven’t written much Perl in a long time, so I don’t know what the community is like any more. Moose was already ubiquitous when I left, which you’d think would let me say “the community mostly focuses on the stuff Moose can do” — but even a decade ago, Moose could already do far more than I had ever seen done by hand in Perl. It’s always made a big deal out of roles (read: interfaces), for instance, despite that I’d never seen anyone care about them in Perl before Moose came along. Maybe their presence in Moose has made them more popular? Who knows.

Also, I wrote Perl seriously, but in the intervening years I’ve only encountered people who only ever used Perl for one-offs. Maybe it’ll come as a surprise to a lot of readers that Perl has an object model at all.

End

Well, that was fun! I hope any of that made sense.

Special mention goes to Rust, which doesn’t have an object model you can fiddle with at runtime, but does do things a little differently.

It’s been really interesting thinking about how tiny differences make a huge impact on what people do in practice. Take the choice of storage in Perl versus Python. Perl’s massively common URI class uses a string as the storage, nothing else; I haven’t seen anything like that in Python aside from markupsafe, which is specifically designed as a string type. I would guess this is partly because Perl makes you choose — using a hashref is an obvious default, but you have to make that choice one way or the other. In Python (especially 3), inheriting from object and getting dict-based storage is the obvious thing to do; the ability to use another type isn’t quite so obvious, and doing it “right” involves a tiny bit of extra work.

Or, consider that Lua could have descriptors, but the extra bit of work (especially design work) has been enough of an impediment that I’ve never implemented them. I don’t think the object implementations I’ve looked at have included them, either. Super weird!

In that light, it’s only natural that objects would be so strongly associated with the features Java and C++ attach to them. I think that makes it all the more important to play around! Look at what Moose has done. No, really, you should bear in mind my description of how Perl does stuff and flip through the Moose documentation. It’s amazing what they’ve built.

UK Government Publishes Advice on ‘Illicit Streaming Devices’

Post Syndicated from Andy original https://torrentfreak.com/uk-government-publishes-advice-on-illicit-streaming-devices-171120/

With torrents and other methods of obtaining content simmering away in the background, unauthorized streaming is the now the method of choice for millions of pirates around the globe.

Previously accessible only via a desktop browser, streaming is now available on a wide range of devices, from tablets and phones through to dedicated set-top box. These, collectively, are now being branded Illicit Streaming Devices (ISD) by the entertainment industries.

It’s terminology the UK government’s Intellectual Property Office has adopted this morning. In a new public advisory, the IPO notes that illicit streaming is the watching of content without the copyright owner’s permission using a variety of devices.

“Illicit streaming devices are physical boxes that are connected to your TV or USB sticks that plug into the TV such as adapted Amazon Fire sticks and so called ‘Kodi’ boxes or Android TV boxes,” the IPO reports.

“These devices are legal when used to watch legitimate, free to air, content. They become illegal once they are adapted to stream illicit content, for example TV programmes, films and subscription sports channels without paying the appropriate subscriptions.”

The IPO notes that streaming devices usually need to be loaded with special software add-ons in order to view copyright-infringing content. However, there are now dedicated apps available to view movies and TV shows which can be loaded straight on to smartphones and tablets.

But how can people know if the device they have is an ISD or not? According to the IPO it’s all down to common sense. If people usually charge for the content you’re getting for free, it’s illegal.

“If you are watching television programmes, films or sporting events where you would normally be paying to view them and you have not paid, you are likely to be using an illicit streaming device (ISD) or app. This could include a film recently released in the cinema, a sporting event that is being broadcast by BT Sport or a television programme, like Game of Thrones, that is only available on Sky,” the IPO says.

In an effort to familiarize the public with some of the terminology used by ISD sellers on eBay, Amazon or Gumtree, for example, the IPO then wanders into a bit of a minefield that really needs much greater clarification.

First up, the government states that ISDs are often described online as being “Fully loaded”, which is a colloquial term for a device with addons already installed. Although they won’t all be infringing, it’s very often the case that the majority are intended to be, so no problems here.

However, the IPO then says that people should keep an eye out for the term ‘jail broken’, which many readers will understand to be the process some hardware devices, such as Apple products, are put through in order for third-party software to be run on them. On occasion, some ISD sellers do put this term on Android devices, for example, but it’s incorrect, in a tiny minority, and of course misleading.

The IPO also warns people against devices marketed as “Plug and Play” but again this is a dual-use term and shouldn’t put consumers off a purchase without a proper investigation. A search on eBay this morning for that exact term didn’t yield any ISDs at all, only games consoles that can be plugged in and played with a minimum of fuss.

“Subscription Gift”, on the other hand, almost certainly references an illicit IPTV or satellite card-sharing subscription and is rarely used for anything else. 100% illegal, no doubt.

The government continues by giving reasons why people should avoid ISDs, not least since their use deprives the content industries of valuable revenue.

“[The creative industries] provide employment for more than 1.9 million people and contributes £84.1 billion to our economy. Using illicit streaming devices is illegal,” the IPO writes.

“If you are not paying for this content you are depriving industry of the revenue it needs to fund the next generation of TV programmes, films and sporting events we all enjoy. Instead it provides funds for the organized criminals who sell or adapt these illicit devices.”

Then, in keeping with the danger-based narrative employed by the entertainment industries’ recently, the government also warns that ISDs can have a negative effect on child welfare, not to mention on physical safety in the home.

“These devices often lack parental controls. Using them could expose children or young people to explicit or age inappropriate content,” the IPO warns.

“Another important reason for consumers to avoid purchasing these streaming devices is from an electrical safety point of view. Where devices and their power cables have been tested, some have failed EU safety standards and have the potential to present a real danger to the public, causing a fire in your home or premises.”

While there can be no doubt whatsoever that failing EU electrical standards in any way is unacceptable for any device, the recent headlines stating that “Kodi Boxes Can Kill Their Owners” are sensational at best and don’t present the full picture.

As reported this weekend, simply not having a recognized branding on such devices means that they fail electrical standards, with non-genuine phone chargers presenting a greater risk around the UK.

Finally, the government offers some advice for people who either want to get off the ISD gravy train or ensure that others don’t benefit from it.

“These devices can be used legally by removing the software. If you are unsure get advice to help you use the device legally. If you wish to watch content that’s only available via subscription, such as sports, you should approach the relevant provider to find out about legal ways to watch,” the IPO advises.

Get it Right from a Genuine Site helps you get the music, TV, films, games, books, newspapers, magazines and sport that you love from genuine services.”

And, if the public thinks that people selling such devices deserve a visit from the authorities, people are asked to report them to the Crimestoppers charity via an anonymous hotline.

The government’s guidance is exactly what one might expect, given that the advisory is likely to have been strongly assisted by companies including the Federation Against Copyright Theft, Premier League, and Sky, who have taken the lead in this area during the past year or so.

The big question is, however, whether many people using these devices really believe that obtaining subscription TV, movies, and sports for next to free is 100% legal. If there are people out there they must be in the minority but at least the government itself is now putting them on the right path.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN discounts, offers and coupons

Sky: People Can’t Pirate Live Soccer in the UK Anymore

Post Syndicated from Andy original https://torrentfreak.com/sky-people-cant-pirate-live-soccer-in-the-uk-anymore-171108/

The commotion over the set-top box streaming phenomenon is showing no signs of dying down and if day one at the Cable and Satellite Broadcasting Association of Asia (CASBAA) Conference 2017 was anything to go by, things are only heating up.

Held at Studio City in Macau, the conference has a strong anti-piracy element and was opened by Joe Welch, CASBAA Board Chairman and SVP Public Affairs Asia, 21st Century Fox. He began Tuesday by noting the important recent launch of a brand new anti-piracy initiative.

“CASBAA recently launched the Coalition Against Piracy, funded by 18 of the region’s content players and distribution partners,” he said.

TF reported on the formation of the coalition mid-October. It includes heavyweights such as Disney, Fox, HBO, NBCUniversal and BBC Worldwide, and will have a strong focus on the illicit set-top box market.

Illegal streaming devices (or ISDs, as the industry calls them), were directly addressed in a segment yesterday afternoon titled Face To Face. Led by Dr. Ros Lynch, Director of Copyright & IP Enforcement at the UK Intellectual Property Office, the session detailed the “onslaught of online piracy” and the rise of ISDs that is apparently “shaking the market”.

Given the apparent gravity of those statements, the following will probably come as a surprise. According to Lynch, the UK IPO sought the opinion of UK-based rightsholders about the pirate box phenomenon a while back after being informed of their popularity in the East. The response was that pirate boxes weren’t an issue. It didn’t take long, however, for things to blow up.

“The UKIPO provides intelligence and evidence to industry and the Police Intellectual Property Crime Unit (PIPCU) in London who then take enforcement actions,” Lynch explained.

“We first heard about the issues with ISDs from [broadcaster] TVB in Hong Kong and we then consulted the UK rights holders who responded that it wasn’t a problem. Two years later the issue just exploded.”

The evidence of that in the UK isn’t difficult to find. In addition to millions of devices with both free Kodi addon and subscription-based systems deployed, the app market has bloomed too, offering free or near to free content to all.

This caught the eye of the Premier League who this year obtained two pioneering injunctions (1,2) to tackle live streams of football games. Streams are blocked by local ISPs in real-time, making illicit online viewing a more painful experience than it ever has been. No doubt progress has been made on this front, with thousands of streams blocked, but according to broadcaster Sky, the results are unprecedented.

“Site-blocking has moved the goalposts significantly,” said Matthew Hibbert, head of litigation at Sky UK.

“In the UK you cannot watch pirated live Premier League content anymore,” he said.

While progress has been good, the statement is overly enthusiastic. TF sources have been monitoring the availability of pirate streams on around dozen illicit sites and services every Saturday (when it is actually illegal to broadcast matches in the UK) and service has been steady on around half of them and intermittent at worst on the rest.

There are hundreds of other platforms available so while many are definitely affected by Premier League blocking, it’s safe to assume that live football piracy hasn’t been wiped out. Nevertheless, it would be wrong to suggest that no progress has been made, in this and other related areas.

Kevin Plumb, Director of Legal Services at The Premier League, said that pubs showing football from illegal streams had also massively dwindled in numbers.

“In the past 18 months the illegal broadcasting of live Premier League matches in pubs in the UK has been decimated,” he said.

This result is almost certainly down to prosecutions taken in tandem with the Federation Against Copyright Theft (FACT), that have seen several landlords landed with large fines. Indeed, both sides of the market have been tackled, with both licensed premises and IPTV device sellers being targeted.

“The most successful thing we’ve done to combat piracy has been to undertake criminal prosecutions against ISD piracy,” said FACT chief Kieron Sharp yesterday. “Everyone is pleading guilty to these offenses.”

Most if not all of FACT-led prosecutions target device and subscription sellers under fraud legislation but that could change in the future, Lynch of the Intellectual Property Office said.

“While the UK works to update its legislation, we can’t wait for the new legislation to take enforcement actions and we rely heavily on ‘conspiracy to defraud’ charges, and have successfully prosecuted a number of ISD retailers,” she said.

Finally, information provided yesterday by network company CISCO shine light on what it costs to run a subscription-based pirate IPTV operation.

Director of Intelligence & Security Operations Avigail Gutman said a pirate IPTV server offering 1,000 channels to around 1,000 subscribers can cost as little as 2,000 euros per month to run but can generate 12,000 euros in revenue during the same period.

“In April of 2017, ten major paid TV and content providers had relinquished 3.09 million euros per month to 285 ISD-based streaming pirate syndicates,” she said.

There’s little doubt that IPTV piracy, both paid and free, is here to stay. The big question is how it will be tackled short and long-term and whether any changes in legislation will have any unintended knock-on effects.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

MPAA Warns Australia Not to ‘Mess’ With Fair Use and Geo-Blocking

Post Syndicated from Ernesto original https://torrentfreak.com/mpaa-warns-australia-not-mess-with-fair-use-and-geo-blocking-171107/

Last year, the Australian Government’s Productivity Commission published a Draft Report on Intellectual Property Arrangements, recommending various amendments to local copyright law.

The Commission suggested allowing the use of VPNs and similar technologies to enable consumers to bypass restrictive geo-blocking. It also tabled proposals to introduce fair use exceptions and to expand safe harbors for online services.

Two months ago the Government responded to these proposals. It promised to expand the safe harbor protections and announced a consultation on fair use, describing the current fair dealing exceptions as restrictive. The Government also noted that circumvention of geo-blocks may be warranted, in some cases.

While the copyright reform plans have been welcomed with wide support from the public and companies such as Google and Wikipedia, there’s also plenty of opposition. From Hollywood, for example, which fears that the changes will set back Australia’s progress to combat piracy.

A few days ago, the MPAA submitted its 2018 list of foreign trade barriers to the U.S. Government. The document in question highlights key copyright challenges in the most crucial markets, Australia included. According to the movie industry group, the tabled proposals are problematic.

“If the Commission’s recommendations were adopted, they could result in legislative changes that undermine the current balance of protection in Australia. These changes could create significant market uncertainty and effectively weaken Australia’s infrastructure for intellectual property protection,” the MPAA writes.

“Of concern is a proposal to introduce a vague and undefined ‘fair use’ exception unmoored from decades of precedent in the United States. Another proposal would expand Australia’s safe harbor regime in piecemeal fashion,” the group adds.

The fair use opposition is noteworthy since the Australian proposal is largely modeled after US law. The MPAA’s comment suggests, however, that this can’t be easily applied to another country, as that would lack the legal finetuning that’s been established in dozens of court cases.

That the MPAA isn’t happy with the expansion of safe harbor protections for online service providers is no surprise. In recent years, copyright holders have often complained that these protections hinder progress on the anti-piracy front, as companies such as Google and Facebook have no incentive to proactively police copyright infringement.

Moving on, the movie industry group highlights that circumvention of geo-blocking for copyrighted content and other protection measures are also controversial topics for Hollywood.

“Still another would allow circumvention of geo-blocking and other technological protection measures. Australia has one of the most vibrant creative economies in the world and its current legal regime has helped the country become the site of major production investments.

“Local policymakers should take care to ensure that Australia’s vibrant market is not inadvertently impaired and that any proposed relaxation of copyright and related rights protection does not violate Australia’s international obligations,” the MPAA adds.

Finally, while it was not included in the commission’s recommendations, the MPAA stresses once again that Australia’s anti-camcording laws are not up to par.

Although several camming pirates have been caught in recent years, the punishments don’t meet Hollywood’s standards. For example, in 2012 a man connected to a notorious release group was convicted for illicitly recording 14 audio captures, for which he received an AUS$2,000 fine.

“Australia should adopt anticamcording legislation. While illegal copying is a violation of the Copyright Act, more meaningful deterrent penalties are required,” the MPAA writes. “Such low penalties fail to reflect the devastating impact that this crime has on the film industry.”

The last suggestion has been in the MPAA’s recommendations for several years already, but the group is persistent.

In closing, the MPAA asks the US Government to keep these and other issues in focus during future trade negotiations and policy discussions with Australia and other countries, while thanking it for the critical assistance Hollywood has received over the years.

MPAA’s full submission, which includes many of the recommendations that were made in previous years, is available here (pdf).

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

Top 10 Torrent Site TorrentDownloads Blocked By Chrome and Firefox

Post Syndicated from Andy original https://torrentfreak.com/top-10-torrent-site-torrentdownloads-blocked-by-chrome-and-firefox-171107/

While the popularity of torrent sites isn’t as strong as it used to be, dozens of millions of people use them on a daily basis.

Content availability is rich and the majority of the main movie, TV show, game and software releases appear on them within minutes, offering speedy and convenient downloads. Nevertheless, things don’t always go as smoothly as people might like.

Over the past couple of days that became evident to visitors of TorrentDownloads, one of the Internet’s most popular torrent sites.

TorrentDownloads – usually a reliable and tidy platform

Instead of viewing the rather comprehensive torrent index that made the Top 10 Most Popular Torrent Site lists in 2016 and 2017, visitors receive a warning.

“Attackers on torrentdownloads.me may trick you into doing something dangerous like installing software or revealing your personal information (for example, passwords, phone numbers or credit cards),” Chrome users are warned.

“Google Safe Browsing recently detected phishing on torrentdownloads.me. Phishing sites pretend to be other websites to trick you.”

Chrome warning

People using Firefox also receive a similar warning.

“This web page at torrentdownloads.me has been reported as a deceptive site and has been blocked based on your security preferences,” the browser warns.

“Deceptive sites are designed to trick you into doing something dangerous, like installing software, or revealing your personal information, like passwords, phone numbers or credit cards.”

A deeper check on Google’s malware advisory service echoes the same information, noting that the site contains “harmful content” that may “trick visitors into sharing personal info or downloading software.” Checks carried out with MalwareBytes reveal that service blocking the domain too.

TorrentFreak spoke with the operator of TorrentDownloads who told us that the warnings had been triggered by a rogue advertiser which was immediately removed from the site.

“We have already requested a review with Google Webmaster after we removed an old affiliates advertiser and changed the links on the site,” he explained.

“In Google Webmaster they state that the request will be processed within 72 Hours, so I think it will be reviewed today when 72 hours are completed.”

This statement suggests that the site itself wasn’t the direct culprit, but ads hosted elsewhere. That being said, these kinds of warnings look very scary to visitors and sites have to take responsibility, so completely expelling the bad player from the platform was the correct choice. Nevertheless, people shouldn’t be too surprised at the appearance of suspect ads.

Many top torrent sites have suffered from similar warnings, including The Pirate Bay and KickassTorrents, which are often a product of anti-piracy efforts from the entertainment industries.

In the past, torrent and streaming sites could display ads from top-tier providers with few problems. However, in recent years, the so-called “follow the money” anti-piracy tactic has forced the majority away from pirate sites, meaning they now have to do business with ad networks that may not always be as tidy as one might hope.

While these warnings are the very last thing the sites in question want (they’re hardly good for increasing visitor numbers), they’re a gift to entertainment industry groups.

At the same time as the industries are forcing decent ads away, these alerts provide a great opportunity to warn users about the potential problems left behind as a result. A loose analogy might be deliberately cutting off beer supply to an unlicensed bar then warning people not to go there because the homebrew sucks. It some cases it can be true, but it’s a problem only being exacerbated by industry tactics.

It’s worth noting that no warnings are received by visitors to TorrentDownloads using Android devices, meaning that desktop users were probably the only people at risk. In any event, it’s expected that the warnings will disappear during the next day, so the immediate problems will be over. As far as TF is informed, the offending ads were removed days ago.

That appears to be backed up by checks carried out on a number of other malware scanning services. Norton, Opera, SiteAdvisor, Spamhaus, Yandex and ESET all declare the site to be clean.

Technical Chrome and Firefox users who are familiar with these types of warnings can take steps (Chrome, FF) to bypass the blocks, if they really must.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.

Aussie ‘Pirate’ Blocking Efforts Switch to Premium IPTV

Post Syndicated from Andy original https://torrentfreak.com/aussie-pirate-blocking-efforts-switch-to-premium-iptv-171106/

Website blocking has become one of the leading anti-piracy mechanisms in recent years and is particularly prevalent across Europe, where thousands of sites are now off-limits by regular means.

More recently the practice spread to Australia, where movie and music industry bodies have filed several applications at the Federal Court. This has rendered dozens of major torrent and streaming inaccessible in the region, after local ISPs complied with orders compelling them to prevent subscriber access.

While such blocking is now commonplace, Village Roadshow and a coalition of movie studios have now switched tack, targeting an operation offering subscription-based IPTV services.

The action targets HDSubs+, a fairly well-known service that provides hundreds of otherwise premium live channels, movies, and sports for a relatively small monthly fee, at least versus the real deal.

A small selection of channels in the HDSubs+ package

ComputerWorld reports that the application for the injunction was filed last month. In common with earlier requests, it targets Australia’s largest ISPs including Telstra, Optus, TPG, and Vocus, plus the subsidiaries.

Access to HDSubs.com appears to be limited, possibly by the platform’s operators, so that visitors from desktop machines are redirected back to Google. However, access to the platform is available by other means and that reveals a fairly pricey IPTV offering.

As seen in the image below, the top package (HD Subs+), which includes all the TV anyone could need plus movies and TV shows on demand, weighs in at US$239.99 per year, around double the price of similar packages available elsewhere.

Broad selection of channels but quite pricey

If the court chooses to grant the injunction, ISPs will not only have to block the service’s main domain (HDSubs.com) but also a range of others which provide the infrastructure for the platform.

Unlike torrent and streaming sites which tend to be in one place (if we discount proxies and mirrors), IPTV services like HD Subs often rely on a number of domains to provide a sales platform, EPG (electronic program guide), software (such as an Android app), updates, and sundry other services.

As per CW, in the HD Subs case they are: ois001wfr.update-apk.com, ois005yfs.update-apk.com, ois003slp.update-apk.com, update002zmt.hiddeniptv.com, apk.hiddeniptv.com, crossepg003uix.hiddeniptv.com, crossepg002gwj.hiddeniptv.com, mpbs001utb.hiddeniptv.com, soft001rqv.update-apk.com and hdsubs.com.

This switch in tactics by Village Roadshow and the other studios involved is subtle but significant. While torrent and streaming sites provide a largely free but fragmented experience, premium IPTV services are direct commercial competitors, often providing a more comprehensive range of channels and services than the broadcasters themselves.

While quality may not always be comparable with their licensed counterparts, presentation is often first class, giving the impression of an official product which is comfortably accessed via a living room TV. This is clearly a concern to commercial broadcasters.

As reported last week, global IPTV traffic is both huge and growing, so expect more of these requests Down Under.

Previous efforts to block IPTV services include those in the UK, where the Premier League takes targeted action against providers who provide live soccer. These measures only target live streams when matches are underway and as far as we’re aware, there are no broader measures in place against any provider.

This could mean that the action in Australia, to permanently block a provider in its entirety, is the first of its kind anywhere.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and ANONYMOUS VPN services.