Tag Archives: P2P

Idea: A Generic P2P Network Client

Post Syndicated from Bozho original https://techblog.bozho.net/idea-a-generic-p2p-network-client/

Every now and then one has a half-baked idea about some project that they aren’t likely to be able to do because of lack of time. I’ve written about such random app ideas before, but they were mostly about small apps.

Here I’d like to share an idea for something a bit bigger (and therefor harder to spare time for) – a generic P2P network client. P2P networks are popular in various domains, most notably file sharing and cryptocurrencies. However, in theory they can be applied to many more problems, social networks, search engines, ride sharing, distributed AI, etc. All of these examples have been implemented in p2p context, and they even work okay, but they lack popularity.

The popularity is actually the biggest issue with these applications – in order to get a service to be popular, in many cases you need a network effect – a p2p file sharing with 100 users doesn’t benefit from being p2p. A social network with 100 users is, well, useless. And it is hard to get traction with these p2p services because they require an additional step – installing software. You can’t just open a webpage and register, you have to install some custom software that will be used to join the p2p network.

P2P networks are distributed, i.e. there is no central node that has control over what happens. That control is held over the binary that gets installed – and it’s usually open source. And you need that binary in order to establish an overlay network. These networks reuse the internet’s transport layer, but do not rely on the world wide web standards, and most importantly, don’t rely heavily on DNS (except, they actually do when run for the first time in order to find a few known seed nodes). So once you are connected to the network, you don’t need to make HTTP or DNS queries, everything stays in the specifics of the particular protocol (e.g. bittorrent).

But the fact that not only you have to trust and install some piece of software, you have to be part of the network and exchange data regularly with peers. So it really doesn’t scale if you want to be part of dozens of p2p networks – each of them might be hungry for resources, you have to keep dozens of applications running all the time (and launching on startup).

So here’s the idea – why don’t we have a generic p2p client. A software that establishes the link to other peers and is agnostic on what data is going to be transferred. From what I’ve seen, the p2p layer is pretty similar in different products – you try to find peers in your immediate network, if none are found, you connect to a known seed node (first by DNS which uses DNS round-robin and then by a list of hardcoded IPs), and when you connect the seed node gives you a list of peers to connect to. Each of those peers has knowledge of other peers, so you can quickly be connected to a significant number of peer nodes (I’m obviously simplifying the flow, but that’s roughly how it works).

Once you have an established list of peers, you start doing the application-specific stuff – sharing files, downloading a cryptocurrency ledger, sharing search indexes, sharing a social network profile database, etc. But the p2p network part can be, I think, generalized.

So we can have that generic client and make it pluggable – every application developer can write their own application ontop of it. The client will not only be a single point for the user, but can also manage resources automatically – inbound and outbound traffic, CPU/GPU usage (e.g. in case of cryptocurrency mining). And all of these applications (i.e. plugins) can either be installed by downloading them from the vendor’s website (which is somewhat similar to the original problem), or can be downloaded from a marketplace available within the client itself. That would obviously mean a centralized marketplace, unless the marketplace itself is a p2p application that’s built-in the client.

So, effectively, you’d be able to plug-in your next file sharing solution, you next cryptocurrency, encrypted messaging, or your next distributed social network. And your users won’t have the barrier of installing yet another desktop app. That alone does not solve the network effect, as you still need enough users to add your plugin to their client (and for many to have the client to begin with), but it certainly makes it easier for them. Imagine if we didn’t have Android and Apple app stores and we had to find relevant apps by other means.

This generic client can possibly be even a browser plugin, so that it’s always on when you are online. It doesn’t have to be, but it might ease adoption. And while it will be complicated to write it as a plugin, it’s certainly possible – there are already p2p solutions working as browser plugins.

Actually, many people think of blockchain smart contracts as a way to do exactly that – have distributed applications. But they have an unnecessary limitation – they work on data that’s shared on a blockchain. And in some cases you don’t need that. You don’t need consensus in the cryptocurrency sense. For example in file sharing, all you need to do is compute the hash of the file (and of its chunks) and start sending them to interested peers. No need to store the file on the blockchain. Same with instant messaging – you don’t need to store the message on a shared immutable database, you only need to send it to the recipients. So smart contracts are not as generic solution as what I’m proposing.

Whether a generic client can accommodate an unlimited amount of different protocols and use cases, how would the communication protocol look like, what programming languages it should support and what security implications that has for the client (e.g. what’s the sandbox that the client provides), what UI markup will be used, are all important operational details, but are besides the point of this post.

You may wonder whether there isn’t anything similar done already. Well, I couldn’t find one. But there’s a lot done that can support such a project: Telehash (a mesh protocol), JXTA (a p2p protocol) and its Chaupal implementation, libp2p and Chimera (p2p libraries), Kademlia (a distributed hash table).

Whether such a project is feasible – certainly. Whether its adoption is inevitable – not so certainly, as it would require immediate usefulness in order to be installed in the first place. So, as every “platform” it will face a chicken-and-egg problem – will people install it if there are no useful plugins, and will people write plugins if there are no user installations. That is solvable in a number of ways (e.g. paying developers initially to write plugins, bundling some standard applications (e.g. file sharing and instant messaging). It could be a business opportunity (monetized through the marketplace or subscriptions) as well as a community project.

I’m just sharing the idea, hoping that someone with more time and more knowledge of distributed networks might pick it up and make the best of it. If not, well, it’s always nice to think about what can the future of the internet look like. Centralization is inevitable, so I don’t see p2p getting rid of centralized services anytime soon (or ever), but some things arguably work better and safer when truly decentralized.

The post Idea: A Generic P2P Network Client appeared first on Bozho's tech blog.

Resources on Distributed Hash Tables

Post Syndicated from Bozho original https://techblog.bozho.net/resources-on-distributed-hash-tables/

Distributed p2p technologies have always been fascinating to me. Bittorrent is cool not because you can download pirated content for free, but because it’s an amazing piece of technology.

At some point I read and researched a lot about how DHTs (distributed hash tables) work. DHTs are not part of the original bittorrent protocol, but after trackers were increasingly under threat to be closed for copyright infringment, “trackerless” features were added to the protocol. A DHT is distributed among all peers and holds information about which peer holds what data. Once you are connected to a peer, you can query it for their knowledge on who has what.

During my research (which was with no particular purpose) I took a note on many resources that I thought useful for understanding how DHTs work and possibly implementing something ontop of them in the future. In fact, a DHT is a “shared database”, “just like” a blockchain. You can’t trust it as much, but proving digital events does not require a blockchain anyway. My point here is – there is a lot more cool stuff to distributed / p2p systems than blockchain. And maybe way more practical stuff.

It’s important to note that the DHT used in BitTorrent is Kademlia. You’ll see a lot about it below.

Anyway, the point of this post is to share the resources that I collected. For my own reference and for everyone who wants to start somewhere on the topic of DHTs.

I hope the list is interesting and useful. It’s not trivial to think of other uses of DHTs, but simply knowing about them and how they work is a good thing.

The post Resources on Distributed Hash Tables appeared first on Bozho's tech blog.

Majority of Canadians Consume Online Content Legally, Survey Finds

Post Syndicated from Andy original https://torrentfreak.com/majority-of-canadians-consume-online-content-legally-survey-finds-180531/

Back in January, a coalition of companies and organizations with ties to the entertainment industries called on local telecoms regulator CRTC to implement a national website blocking regime.

Under the banner of Fairplay Canada, members including Bell, Cineplex, Directors Guild of Canada, Maple Leaf Sports and Entertainment, Movie Theatre Association of Canada, and Rogers Media, spoke of an industry under threat from marauding pirates. But just how serious is this threat?

The results of a new survey commissioned by Innovation Science and Economic Development Canada (ISED) in collaboration with the Department of Canadian Heritage (PCH) aims to shine light on the problem by revealing the online content consumption habits of citizens in the Great White North.

While there are interesting findings for those on both sides of the site-blocking debate, the situation seems somewhat removed from the Armageddon scenario predicted by the entertainment industries.

Carried out among 3,301 Canadians aged 12 years and over, the Kantar TNS study aims to cover copyright infringement in six key content areas – music, movies, TV shows, video games, computer software, and eBooks. Attitudes and behaviors are also touched upon while measuring the effectiveness of Canada’s copyright measures.

General Digital Content Consumption

In its introduction, the report notes that 28 million Canadians used the Internet in the three-month study period to November 27, 2017. Of those, 22 million (80%) consumed digital content. Around 20 million (73%) streamed or accessed content, 16 million (59%) downloaded content, while 8 million (28%) shared content.

Music, TV shows and movies all battled for first place in the consumption ranks, with 48%, 48%, and 46% respectively.

Copyright Infringement

According to the study, the majority of Canadians do things completely by the book. An impressive 74% of media-consuming respondents said that they’d only accessed material from legal sources in the preceding three months.

The remaining 26% admitted to accessing at least one illegal file in the same period. Of those, just 5% said that all of their consumption was from illegal sources, with movies (36%), software (36%), TV shows (34%) and video games (33%) the most likely content to be consumed illegally.

Interestingly, the study found that few demographic factors – such as gender, region, rural and urban, income, employment status and language – play a role in illegal content consumption.

“We found that only age and income varied significantly between consumers who infringed by downloading or streaming/accessing content online illegally and consumers who did not consume infringing content online,” the report reads.

“More specifically, the profile of consumers who downloaded or streamed/accessed infringing content skewed slightly younger and towards individuals with household incomes of $100K+.”

Licensed services much more popular than pirate haunts

It will come as no surprise that Netflix was the most popular service with consumers, with 64% having used it in the past three months. Sites like YouTube and Facebook were a big hit too, visited by 36% and 28% of content consumers respectively.

Overall, 74% of online content consumers use licensed services for content while 42% use social networks. Under a third (31%) use a combination of peer-to-peer (BitTorrent), cyberlocker platforms, or linking sites. Stream-ripping services are used by 9% of content consumers.

“Consumers who reported downloading or streaming/accessing infringing content only are less likely to use licensed services and more likely to use peer-to-peer/cyberlocker/linking sites than other consumers of online content,” the report notes.

Attitudes towards legal consumption & infringing content

In common with similar surveys over the years, the Kantar research looked at the reasons why people consume content from various sources, both legal and otherwise.

Convenience (48%), speed (36%) and quality (34%) were the most-cited reasons for using legal sources. An interesting 33% of respondents said they use legal sites to avoid using illegal sources.

On the illicit front, 54% of those who obtained unauthorized content in the previous three months said they did so due to it being free, with 40% citing convenience and 34% mentioning speed.

Almost six out of ten (58%) said lower costs would encourage them to switch to official sources, with 47% saying they’d move if legal availability was improved.

Canada’s ‘Notice-and-Notice’ warning system

People in Canada who share content on peer-to-peer systems like BitTorrent without permission run the risk of receiving an infringement notice warning them to stop. These are sent by copyright holders via users’ ISPs and the hope is that the shock of receiving a warning will turn consumers back to the straight and narrow.

The study reveals that 10% of online content consumers over the age of 12 have received one of these notices but what kind of effect have they had?

“Respondents reported that receiving such a notice resulted in the following: increased awareness of copyright infringement (38%), taking steps to ensure password protected home networks (27%), a household discussion about copyright infringement (27%), and discontinuing illegal downloading or streaming (24%),” the report notes.

While these are all positives for the entertainment industries, Kantar reports that almost a quarter (24%) of people who receive a notice simply ignore them.


Once upon a time, people obtaining music via P2P networks was cited as the music industry’s greatest threat but, with the advent of sites like YouTube, so-called stream-ripping is the latest bogeyman.

According to the study, 11% of Internet users say they’ve used a stream-ripping service. They are most likely to be male (62%) and predominantly 18 to 34 (52%) years of age.

“Among Canadians who have used a service to stream-rip music or entertainment, nearly half (48%) have used stream-ripping sites, one-third have used downloader apps (38%), one-in-seven (14%) have used a stream-ripping plug-in, and one-in-ten (10%) have used stream-ripping software,” the report adds.

Set-Top Boxes and VPNs

Few general piracy studies would be complete in 2018 without touching on set-top devices and Virtual Private Networks and this report doesn’t disappoint.

More than one in five (21%) respondents aged 12+ reported using a VPN, with the main purpose of securing communications and Internet browsing (57%).

A relatively modest 36% said they use a VPN to access free content while 32% said the aim was to access geo-blocked content unavailable in Canada. Just over a quarter (27%) said that accessing content from overseas at a reasonable price was the main motivator.

One in ten (10%) of respondents reported using a set-top box, with 78% stating they use them to access paid-for content. Interestingly, only a small number say they use the devices to infringe.

“A minority use set-top boxes to access other content that is not legal or they are unsure if it is legal (16%), or to access live sports that are not legal or they are unsure if it is legal (11%),” the report notes.

“Individuals who consumed a mix of legal and illegal content online are more likely to use VPN services (42%) or TV set-top boxes (21%) than consumers who only downloaded or streamed/accessed legal content.”

Kantar says that the findings of the report will be used to help policymakers evaluate how Canada’s Copyright Act is coping with a changing market and technological developments.

“This research will provide the necessary information required to further develop copyright policy in Canada, as well as to provide a foundation to assess the effectiveness of the measures to address copyright infringement, should future analysis be undertaken,” it concludes.

The full report can be found here (pdf)

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN reviews, discounts, offers and coupons.

Danish Traffic to Pirate Sites Increases 67% in Just a Year

Post Syndicated from Andy original https://torrentfreak.com/danish-traffic-to-pirate-sites-increases-67-in-just-a-year-180501/

For close to 20 years, rightsholders have tried to stem the tide of mainstream Internet piracy. Yet despite increasingly powerful enforcement tools, infringement continues on a grand scale.

While the problem is global, rightsholder groups often zoom in on their home turf, to see how the fight is progressing locally. Covering Denmark, the Rights Alliance Data Report 2017 paints a fairly pessimistic picture.

Published this week, the industry study – which uses SimilarWeb and MarkMonitor data – finds that Danes visited 2,000 leading pirate sites 596 million times in 2017. That represents a 67% increase over the 356 million visits to unlicensed platforms made by citizens during 2016.

The report notes that, at least in part, this explosive growth can be attributed to mobile-compatible sites and services, which make it easier than ever to consume illicit content on the move, as well as at home.

In a sea of unauthorized streaming sites, Rights Alliance highlights one platform above all the others as a particularly bad influence in 2017 – 123movies (also known as GoMovies and GoStream, among others).

“The popularity of this service rose sharply in 2017 from 40 million visits in 2016 to 175 million visits in 2017 – an increase of 337 percent, of which most of the traffic originates from mobile devices,” the report notes.

123movies recently announced its closure but before that the platform was subjected to web-blocking in several jurisdictions.

Rights Alliance says that Denmark has one of the most effective blocking systems in the world but that still doesn’t stop huge numbers of people from consuming pirate content from sites that aren’t yet blocked.

“Traffic to infringing sites is overwhelming, and therefore blocking a few sites merely takes the top of the illegal activities,” Rights Alliance chief Maria Fredenslund informs TorrentFreak.

“Blocking is effective by stopping 75% of traffic to blocked sites but certainly, an upscaled effort is necessary.”

Rights Alliance also views the promotion of legal services as crucial to its anti-piracy strategy so when people visit a blocked site, they’re also directed towards legitimate platforms.

“That is why we are working at the moment with Denmark’s Ministry of Culture and ISPs on a campaign ‘Share With Care 2′ which promotes legal services e.g. by offering a search function for legal services which will be placed in combination with the signs that are put on blocked websites,” the anti-piracy group notes.

But even with such measures in place, the thirst for unlicensed content is great. In 2017 alone, 500 of the most popular films and TV shows were downloaded from P2P networks like BitTorrent more than 15 million times from Danish IP addresses, that’s up from 11.9 million in 2016.

Given the dramatic rise in visits to pirate sites overall, the suggestion is that plenty of consumers are still getting through. Rights Alliance says that the number of people being restricted is also hampered by people who don’t use their ISP’s DNS service, which is the method used to block sites in Denmark.

Additionally, interest in VPNs and similar anonymization and bypass-capable technologies is on the increase. Between 3.5% and 5% of Danish Internet users currently use a VPN, a number that’s expected to go up. Furthermore, Rights Alliance reports greater interest in “closed” pirate communities.

“The data is based on closed [BitTorrent] networks. We also address the challenges with private communities on Facebook and other [social media] platforms,” Fredenslund explains.

“Due to the closed doors of these platforms it is not possible for us to say anything precisely about the amount of infringing activities there. However, we receive an increasing number of notices from our members who discover that their products are distributed illegally and also we do an increased monitoring of these platforms.”

But while more established technologies such as torrents and regular web-streaming continue in considerable volumes, newer IPTV-style services accessible via apps and dedicated platforms are also gaining traction.

“The volume of visitors to these services’ websites has been sharply rising in 2017 – an increase of 84 percent from January to December,” Rights Alliance notes.

“Even though the number of visitors does not say anything about actual consumption, as users usually only visit pages one time to download the program, the number gives an indication that the interest in IPTV is increasing.”

To combat this growth market, Rights Alliance says it wants to establish web-blockades against sites hosting the software applications.

Also on the up are visits to platforms offering live sports illegally. In 2017, Danish IP addresses made 2.96 million visits to these services, corresponding to almost 250,000 visits per month and representing an annual increase of 28%.

Rights Alliance informs TF that in future a ‘live’ blocking mechanism similar to the one used by the Premier League in the UK could be deployed in Denmark.

“We already have a dynamic blocking system, and we see an increasing demand for illegal TV products, so this could be a natural next step,” Fredenslund explains.

Another small but perhaps significant detail is how users are accessing pirate sites. According to the report, large volumes of people are now visiting platforms directly, with more than 50% doing so in preference to referrals from search engines such as Google.

In terms of deterrence, the Rights Alliance report sticks to the tried-and-tested approaches seen so often in the anti-piracy arena.

Firstly, the group notes that it’s increasingly encountering people who are paying for legal services such as Netflix and Spotify so believe that allows them to grab something extra from a pirate site. However, in common with similar organizations globally, the group counters that pirate sites can serve malware or have other nefarious business interests behind the scenes, so people should stay away.

Whether significant volumes will heed this advice will remain to be seen but if a 67% increase last year is any predictor of the future, piracy is here to stay – and then some. Rights Alliance says it is ready for the challenge but will need some assistance to achieve its goals.

“As it is evident from the traffic data, criminal activities are not something that we, private companies (right holders in cooperation with ISPs), can handle alone,” Fredenslund says.

“Therefore, we are very pleased that DK Government recently announced that the IP taskforce which was set down as a trial period has now been made permanent. In that regard it is important and necessary that the police will also obtain the authority to handle blocking of massively infringing websites. Police do not have the authority to carry out blocking as it is today.”

The full report is available here (Danish, pdf)

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN reviews, discounts, offers and coupons.

WannaCry Ransomware Foiled By Domain Killswitch

Post Syndicated from Darknet original http://feedproxy.google.com/~r/darknethackers/~3/ASy1wNCVg7I/

Whilst I was away on a tropical island enjoying myself the Infosec Internet was on fire with news of the global WannaCry ransomware threat which showed up in the UK NHS and was spreading across 74 different countries. The Ransomware seems to be the first that is P2P using an SMB exploit from the NSA […]

The post WannaCry Ransomware Foiled…

Read the full post at darknet.org.uk

Yes, we can validate the Wikileaks emails

Post Syndicated from Robert Graham original http://blog.erratasec.com/2016/10/yes-we-can-validate-wikileaks-emails.html

Recently, WikiLeaks has released emails from Democrats. Many have repeatedly claimed that some of these emails are fake or have been modified, that there’s no way to validate each and every one of them as being true. Actually, there is, using a mechanism called DKIM.

DKIM is a system designed to stop spam. It works by verifying the sender of the email. Moreover, as a side effect, it verifies that the email has not been altered.
Hillary’s team uses “hillaryclinton.com”, which as DKIM enabled. Thus, we can verify whether some of these emails are true.

Recently, in response to a leaked email suggesting Donna Brazile gave Hillary’s team early access to debate questions, she defended herself by suggesting the email had been “doctored” or “falsified”. That’s not true. We can use DKIM to verify it.
You can see the email in question at the WikiLeaks site: https://wikileaks.org/podesta-emails/emailid/5205. The title suggests they have early access to debate questions, and includes one specifically on the death penalty, with the text:

since 1973, 156 people have been on death row and later set free. Since 1976, 1,414 people have been executed in the U.S

Indeed, during the debate the next day, they asked the question:

Secretary Clinton, since 1976, we have executed 1,414 people in this country.  Since 1973, 156 who were convicted have been exonerated from the death row.

It’s not a smoking gun, but at the same time, it both claims they got questions in advance while having a question in advance. Trump gets hung on similar chains of evidence, so it’s not something we can easily ignore.
Anyway, this post isn’t about the controversy, but the fact that we can validate the email. When an email server sends a message, it’ll include an invisible “header”. They aren’t especially hidden, most email programs allow you to view them, it’s just that they are boring, so hidden by default. The DKIM header in this email looks like:
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=hillaryclinton.com; s=google;
How do you verify this is true. There are a zillion ways with various “DKIM verifiers”. I use the popular Thunderbird email reader (from the Mozilla Firefox team). They have an addon designed specifically to verify DKIM. Normally, email readers don’t care, because it’s the email server‘s job to verify DKIM, not the client. So we need a client addon to enable verification.
Downloading the raw email from WikiLeaks and opening in Thunderbird, with the addon, I get the following verification that the email is valid. Specifically, it validates that the HillaryClinton.com sent precisely this content, with this subject, on that date.
Let’s see what happens when somebody tries to doctor the email. In the following, I added “MAKE AMERICA GREAT AGAIN” to the top of the email.
As you can see, we’ve proven that DKIM will indeed detect if anybody has “doctored” or “falsified” this email.
I was just listening to ABC News about this story. It repeated Democrat talking points that the WikiLeaks emails weren’t validated. That’s a lie. This email in particular has been validated. I just did it, and shown you how you can validate it, too.

Btw, if you can forge an email that validates correctly as I’ve shown, I’ll give you 1-bitcoin. It’s the easiest way of solving arguments whether this really validates the email — if somebody tells you this blogpost is invalid, then tell them they can earn about $600 (current value of BTC) proving it. Otherwise, no.

Update: I’m a bit late writing this blog post. Apparently, others have validated these, too.

Update: In the future, when HilaryClinton.com changes their DKIM key, it will no longer be able to verify. Thus, I’m recording the domain key here:

google._domainkey.hillaryclinton.com: type TXT, class IN
v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCJdAYdE2z61YpUMFqFTFJqlFomm7C4Kk97nzJmR4YZuJ8SUy9CF35UVPQzh3EMLhP+yOqEl29Ax2hA/h7vayr/f/a19x2jrFCwxVry+nACH1FVmIwV3b5FCNEkNeAIqjbY8K9PeTmpqNhWDbvXeKgFbIDwhWq0HP2PbySkOe4tTQIDAQAB

Поредните вируси

Post Syndicated from Григор original http://www.gatchev.info/blog/?p=1940

Напоследък буквално не съм видял е-майл, който да не прелива от вируси. И преди е имало кампании, но тази направо смайва с мащабите си.

Почти месец вече нямам време да се огледам и всеки ден отлагам писането за тази опасност, с надеждата пороят да секне и да ми спести усилието. Той обаче продължава. Слава Богу, нощта срещу Великден е, така че мога да заделя половин час. :-) Който е шаран в тези неща сигурно се е хванал вече, но все пак по-добре късно, отколкото никога.

Е-майлите се изпращат от масивен ботнет – мисля, че е поне 200 000 компютъра. Разпространението му е по целия свят – засякъл съм машини в Северна и Южна Америка, Европа, Русия, Индия, Китай… Изглежда като да е под контрола на руска кибербанда. Не съм успял да пипна лично заразена машина, нито съм имал времето да заразя и тествам някоя, но май системата му е модулна P2P. Линуксите май не са в опасност, за Маковете не съм имал възможност да проверя.

Получените е-майли са адресирани както до реални акаунти по списък, така и до предполагаеми често срещани акаунти в домейни – [email protected] и прочее. Всички съдържат прикрепен ZIP файл и текст, който да подлъже получателя да отвори файла. Разнообразието на текстовете е огромно – „неплатени фактури“, „снимки“, „сканирани изображения“, „съобщения от шефа“, „застрахователни полици“, „сметки за плащане“, дори немалко „прикрепен файл“, „прикрепена картинка“ и прочее. „Изпращачът“ също е най-различен – някакво име, или е-майл адрес в същия или друг домейн, или дори адресът на получателя…

Съдържанието на ZIP-а e .js файл. Най-често името му е от десетина цифри, долно тире и още десетина цифри. Размерът варира, обикновено около 10-20 КБ некомпресиран. Голямата част от съдържанието му е някакъв текст, сложен да заблуди скенерите на антивирусите. Пак със същата цел променливите в малкото (около 1 К) реален код са с дълги (често над 50 знака), генерирани на случаен принцип имена, а присвояваните на тях стойности са обфускирани по наглед некадърно прост, но често успяващ да заблуди антивируса начин. Написан е от руски киберкримки.

Единствената задача на JavaScript кода е свалянето (от предварително пробит легален сървър) и изпълнението на определен файл. (Тъй като JS файлът се изпълнява през браузер, сваленият файл се изпълнява с правата на браузера, обикновено това ще рече с тези на текущия юзер.) Нямах време да го чопля подробно – видимото на пръв поглед е, че пробва срещу Windows набор експлойти, повечето от тях май вече запушени от ъпдейтове. Логично е да се очаква, че успешно заразените машини стават част от ботнета.

Какво може да се очаква от вируса? За каквото бъде изкомандван да си свали и стартира модул. Очевадното до момента е, че компютърът ще започне да сее спам от описания по-горе. Възможно е и всичко друго, от шифроване на информацията ви и искане на откуп, през използване на компютъра ви за DDOS атаки, та до събиране на уличаваща ви в каквото и да е информация и изпращането ѝ на някой, който ще може да ви изнудва с нея после. (Нямате такава? А оня филм, дето го изпиратствахте? Знаете ли какви наказания предвижда законът за пиратство?…)


– ако получите е-майл с прикрепено нещо, го стартирайте само ако познавате изпращача и той ви се е обадил да ви предупреди, че ви изпраща нещото. Ако то изглежда дори най-малко подозрително, му звъннете да попитате пращал ли го е.

– настройте си Windows и разархивиращите програми да ви показват разширенията на файловете. (И „благодарете“ на Майкрософт, че в последните версии на Windows те по подразбиране са скрити.) Ако нещото съдържа какъвто и да е изпълним код (например е документ на Microsoft Office), то е вирус до лично гласово или визуално потвърждение от познат ви изпращач за противното.

– ако нещото пристига от непознат и се опитва да ви убеждава в нещо или да ви кара да „отворите“ нещо, то е вирус, без право на обжалване. (Ако наистина е писмо от български данъчни, че им дължите пари, според мен е особено опасен вирус. Имам опит с такъв – ако мислите, че има на кого да обясниш къде точно са сбъркали, не живеете на този свят.)

– ако нещото се опитва да ви бие по емоциите (и особено по страха или желанието за секс), то е вирус, независимо как изглежда. (Или наистина писмо от шефа или любовницата, но тогава ще научите за него и по гласов и т.н. път. Въпросът дали тези пратки не са сред най-опасните вируси също го оставяме настрана.)

Успех и най-вече бистра глава! Данните ви не са Христос – умрат ли, няма да възкръснат дори на Великден.

“Turing Test” for OTT Video Streaming: Can a viewer distinguish between Streaming and Broadcast Video in 2016?

Post Syndicated from yahoo original https://yahooeng.tumblr.com/post/140395457866

P.P.S. Narayan, VP of EngineeringIntroductionAbout sixty-five years ago, Alan Turing, considered to be the father of Computer Science and modern computing machines, put forth a deep and philosophical question: can machines think? Asked differently, “Can a machine exhibit (intelligent) behavior indistinguishable from a human [1]?” In order to confirm the conclusion, he devised a test, coined the imitation game.  While there have been various interpretations (and extensive debate) of the test, our objective is to look at its application to video streaming by understanding the premise and appreciating the concept.imageFigure 1: Imitation GameIn my interpretation, Alan Turing proposed that there were 3 participants: A, B and C. The role of participant C is that of the interrogator, and C communicates with A and B only via written text. C cannot see or talk to the other participants and only knows them as labels [2]. By asking questions to A and B, C tries to determine which of A and B, is human and the other is machine.Turing says:I believe that in about fifty years’ time it will be possible to program computers, with a storage capacity of about 109, to make them play the imitation game so well that an average interrogator will not have more than 70 percent chance of making the right identification after five minutes of questioning. … I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted.Over the years, the Turing test has become an important concept in artificial intelligence and the evolution of computing in modern times. In fact, some modifications to the Turing test have been proposed and have been adopted widely. CAPTCHA is a form of reverse Turing test, where we have participant C replaced by a machine, and the task of C, as a gatekeeper, is to distinguish the humans (in A or B or …) from machines [1].TV on the internetTelevision and video broadcasting has been around for decades. Starting with the first television broadcasts in the 1930s to modern broadcasts in 3D and 4K UHD, the evolution has been phenomenal and eye-popping. Over the years, TV broadcast has moved from terrestrial over-the-air radio waves to satellite or fiber delivery to the home. Analog signals have modernized to digital; black-and-white to color. Televisions have evolved from mechanical to electronic to digital, and traditional CRT displays have changed to ultra thin LED displays.In the mid-1990s, with the explosion of the consumer Internet, a new wave of video consumption started on devices other than the traditional television sets. Video streaming was soon available on desktops and then laptops, and with the WiFi and mobile revolution, we began streaming video on our phones and tablets.Nearly a century of evolution of standards, technology, infrastructure, and innovation helped the US TV industry grow to more than a $100B [6]. We are in a new generation where internet streaming is a revolution and growing rapidly. “Over-the-Top” video streaming, or OTT as it is commonly referred to, is delivered through the open unmanaged internet, with the “last-mile” companies (e.g., Comcast) acting only as the internet service provider [3]. Netflix, an OTT video streaming service, already accounts for more than 35% of peak US internet traffic. In fact, all of OTT video streaming accounts for more than 50% of both internet and mobile traffic in the US [4].This new form of video consumption is pushing the infrastructure and technology built for the Internet beyond its original design parameters and capabilities. While content and user interfaces have advanced quite drastically with OTT, the overall quality of experience is still years behind.The “OTT Turing Test”Several months ago, I attended the Streaming Media West conference. And, during one of the panels, the question was asked about the “success metrics” for OTT streaming. Rather than invent what we will call as “success”, I believe we need to use existing TV quality as a baseline benchmark.Say, we modify the original Turing test, to replace A and B with simple LED TV-like monitors. The video inputs for A and B will have the similar videos (e.g. say any NFL game) from either over-the-air HD signals or from an OTT stream. The participant C, or the interrogator, has to interact with the two monitors to determine which of A or B is displaying video that is OTT streamed. We coin this as the “OTT Turing test”.imageFigure 2: OTT Turing TestWhat is the holy grail? Quoting Turing again, an average interrogator will not have more than 70 percent chance of making the right identification after five minutes of viewing.Note that we modified the original Turing test in just a couple of ways. For one, C can only determine the outcome based on the visual perceptions of the video being played on the two monitors. And second,  we have restricted the interaction of the interrogator, with the monitors, to be very limited. The only instruction that C can deliver is to start or stop the video via a “remote” controller. While C can consume the video experience on the TV, they cannot use it like a DVR/VCR or a cable box. A few things that the interrogator cannot do, are “pause” and “play”. C cannot “change channels”. C cannot choose from a “channel guide”. And so on.And, to make it even more challenging, C could be either a human or a machine. Therefore, any variance in human perception will be eliminated via more systematic and objective evaluation of the inputs.  If we did this test today, would OTT pass the Turing test? My answer is no. The reason is because there are a number of challenges that have yet to be completely solved. Let’s examine some of these challenges, why they exist, and how far away the technology is for us to achieve this goal.Startup TimeWhen we want to start watching television today, we switch “on” the TV set and/or our connected device (e.g. cable box). Typically, these startup in at most 2-3 seconds, and you can see the video on your screen in that time. At times, the audio even starts up sooner. In traditional television, the latency for startup can be broken down into two parts: the time taken for the first few frames of the video to reach the device which is determined by the speed of light, and then to the additional time taken for decoding the signal and constructing the frames to be rendered on the display. This holds true for the over-the-air broadcasts as well as for cable or satellite transmissions.imageFigure 3: Propagation Delay in televisionFor internet OTT streaming to be indistinguishable, we need to have startup times as seen on traditional television today. Current streaming protocols have been designed and evolved to take into account the “best-effort” nature of the internet. The packetization of video in protocols like HLS and DASH enable the playback to start faster with the ability to deliver key frames in a few packets. Also modern protocols like P2P and UDP based network coding help in increasing the reliability of packet delivery on a best-effort network. In addition to the delivery of the video packets, other overheads like player (e.g., flash and javascript) downloads impact the speed of video startup.Video QualityThese days, most displays support HD and resolution of 720p or 1080i/p – table stakes in video broadcasting. In fact, most television HDTV broadcasts are shot at 1080i/60 fields-per-sec. Modern television sets have inbuilt sophisticated upscaling algorithms and technologies that can take lower resolution incoming signals and upscale them to 1080p or even 4K. However, these algorithms vary from device to device, manufacturer to manufacturer, and upscaling can have artifacts introduced which can appear on certain types of content (e.g., high motion sports like NFL or video games).imageFigure 4: Common Display resolutions [5]So having OTT streaming start at lower resolutions and lower frame-rates will not produce an indistinguishable viewing experience as compared to traditional television broadcasts. It is imperative that OTT streaming has the whole video pipeline producing content at high enough resolutions with purity in signal. Starting with cameras, the whole video pipeline including, video signal acquisition, video mixing, encoding, transcoding and all the packetization must do 1080p or higher resolutions natively.The SpinnerAnyone who has watched video on the internet knows (or should know) what the “Spinner” is. It is the non-technical term given to the manifestation of not being able to deliver content reliably, over a network which is not dedicated.imageFigure 5: Typical “Spinner” during Internet Streaming (Image Courtesy [8])Buffering was a concept introduced in internet streaming that allowed the video player to collect enough video frames (and keep collecting additional frames regularly) so as to maintain a smooth playback on a non-reliable best-effort Internet. On traditional television broadcasts, there is no concept of buffering (unless watching some video-on-demand content from your cable provider) with no visual manifestations (e.g., spinner) shown to the viewers.Buffering and re-buffering (after the video has started) have become a pervasive pain point for viewers of OTT streaming. And there are many different causes of rebuffering. The reasons vary from end viewer device capabilities (and software), to network congestion inside the viewer’s home/premise, to ISP or local area bandwidth issues, to the core getting clogged and so on.  Rebuffering ratio is a typical industry measure to determine the perceived user quality. It is a ratio, measured for a time window, as the total time spent rebuffering across all users to the total time spent viewing the stream. Rebuffering ratios of 2-4% are considered acceptable across the industry today. For example, if you are watching a video online for 2 minutes, it is normal for you (and all users) to see the spinner for 2-3 seconds. This typically gets worse during live streams, so if you watch an NFL game for more than 2 hours, you are more than likely to experience many interruptions in your viewing totaling up to 2-3 minutes!! Imagine that experience as compared to what you get on TV or cable today.At Yahoo, our goal for re-buffering is ZERO. Plus, we’ve focused on some new metrics. For example, the percentage of rebuffering impacted views is more important and relevant. Having 100% of your viewers impacted for 1 minute is far worse than 1% of your viewers impacted for 100 minutes. Basically, this would mean that 99% of your viewers had a FLAWLESS experience. That number needs to grow to five 9s like traditional television.Visual ArtifactsVideo encoding is a complex process from signal acquisition to mixing to adding audio tracks and then shipping the packets to the device. The receiving device capabilities can vary quite drastically. And, this is handled when encoding video digitally for transmission, by picking a few different resolutions. This allows for rendering the video well on various form factors (e.g., a 4K display vs a VGA device), and also allows for optimizing the number of bits transmitted depending on the viewing device. The optimization comes from the fact that lower resolutions would require lower number of bits to encode.Usual OTT streams are encoded in a few different resolutions or “bitrates”. The availability of various bitrates has become a mechanism for handling poor network conditions. Video players on devices “adapt” and deliver a continuous experience (i.e. avoid rebuffering). The tradeoff has become straightforward – reduce resolution rather than stop the video.imageFigure 6: Blockiness of Video (Image courtesy [7])What does it result in? OTT streaming regularly causes video to be “blocky” where the rendering of lower bitrates at lower resolutions. While the artifacts are clearly perceptible for viewers of OTT, it is also easy for a machine to find the introduction of noise into the original video signal as a result of the encoding bitrates. Traditional television transmissions rarely, if ever, have any blockiness or pixelations for viewers. This effect is so prolific in OTT stream for optimized content, that an interrogator may use this knowledge to deliberately look for blockiness in scenes that are visually complex or contain fast motion, as a way of identifying which display is OTT.Ad TransitionAdvertising on television is native and seamless. What do I mean by native? A viewer sees very little or no difference between content and advertising. In other words, when there is an imperceptible break of under 500ms between the content and ads and from the viewer perspective, it is considered “seamless”.  Over time, the television ad ecosystem has evolved and local stations have the ability to insert local video ads into global or national content.imageFigure 7: Ad insertion technologyIn OTT streaming, we need to move closer to TV-like experience for ads. Today OTT video ad insertion for mid-roll (during a content or broadcast) is typically done from the viewing device. Advertisers are excited by the many advantages of this technology. It allows the ads to be more personalized and targeted than local ads. Plus, the medium also allows advertisers get more detailed engagement metrics as compared to traditional television. However, this challenges the seamless experience and typically viewers see “spinners” and blank screens during ad insertion.  SynchronizationConsider watching a live (American) football, cricket or soccer game on television. Maybe with a bunch of friends or strangers in a bar, with multiple big screen televisions. We quite frequently do this, to enjoy a social viewing experience. With the current television technology, typically all the television sets in the bar are within a few milliseconds of each other – not much lag or lack of synchronization.However, with OTT, multiple viewers of the same “live” event cannot be guaranteed a synchronized viewing experience. Even if they all started streaming at the same exact moment! This common problem is due to the inherent nature of various streaming protocols and buffer management. In OTT streaming, especially for live events, it is likely that a viewer may watch or experience a goal at a significantly different moment in time as compared to his neighbors in the same apartment complex, which creates a bad user experience.SummaryIn summary, as an industry, we have a lot of challenges to deliver a TV-like experience for OTT streaming. Some of them are easy, while the others are quite difficult to overcome.It is likely that very soon, startup time, video quality and ad transitions will improve significantly and be indistinguishable from the current television experience. The tougher technical challenges will be to get to ZERO rebuffering and to enable synchronized watching experience for OTT. These are fundamental challenges in the today’s technology, that will require significant innovation and even some revolutionary new concepts such as P2P or new protocols to up the game for OTT.However, I am very confident that we will overcome these challenges, and win the OTT Turing Test – much before the original Turing Test is solved!!References[1] https://en.wikipedia.org/wiki/Turing_test[2] http://plato.stanford.edu/entries/turing-test/#Tur195ImiGam [3] https://en.wikipedia.org/wiki/Internet_television [4] http://www.statista.com/chart/1620/top-10-traffic-hogs/ [5] https://en.wikipedia.org/wiki/Display_resolution#/media/File:Vector_Video_Standards8.svg [6] http://www.statista.com/topics/977/television/ [7] http://www.bestshareware.net/howto/img1/how-to-remove-pixellation-from-video-1.jpg [8] http://blogs.tigarus.com/patcoola/wp-content/uploads/sites/patcoola/2014/buffering-2014.png