Post Syndicated from nellyo original https://nellyo.wordpress.com/2018/03/03/illegal_/
На 1 март 2018 Европейската комисия публикува Препоръка относно мерките за ефективно справяне с незаконното онлайн съдържание, която представя идеите на Комисията за това как да се ускори премахването на незаконно съдържание. Отделно от това, подобни идеи са развити в предложенията за ревизия на медийното и авторското право, както и в дискусиите за борба с дезинформацията и фалшивите новини.
Аз също говорих за това на конференцията за фалшивите новини, организирана от АЕЖ през ноември 2017: ЕК препоръчва на частни търговски дружества да се даде възможност да заличават съдържание, качено от граждани. Сега Европейската комисия продължава идеите в тази посока.
В правна система, основана на върховенство на правото, съдът е този, който трябва да се произнася при намеса в свободата на изразяване, поне досега това беше неоспорвано положение. ЕК насърчава тенденцията е да се овластят доставчици да правят такава преценка – точно както идеята за трите удара преди време.
Реакцията на European Digital Rights (EDRi):
Европейските политици работят за най-големия интернет филтър, който някога сме виждали. Това може да звучи драматично, но наистина не е преувеличено. Ако предложението бъде прието, уеб сайтове като Soundcloud, eBay, Facebook и Flickr ще бъдат принудени да филтрират всичко, което искате да качите. Алгоритъм ще определя кое от съдържанието, което качвате, ще се вижда от останалия свят и кое – няма.
Този интернет филтър е предвиден в предложенията за нова европейска нормативна уредба. Интернет филтрите не могат и не трябва да се използват за регулиране на авторското право. Те не работят. Но има много по-голям проблем: след като бъде инсталиран, интернет филтърът може и ще бъде използван за безброй други цели. Обзалагаме се, че политиците радостно очакват интернет филтъра, за да го използват в биткаите си с фалшиви новини, тероризъм или нежелани политически мнения.
EDRi подчертава, че има много причини да сте срещу тези предложения – ето три:
- Това е атака срещу вашата свобода на изразяване.
- Филтри като тези правят много грешки.
- Платформите ще бъдат насърчени да избягват риска – за сметка на вашата свобода.
Figure 3: CaffeOnSpark as a Spark Deep Learning package
Figure 5: System Architecture
It is hard to believe that 10 years have already passed since Hadoop was started at Yahoo. We initially applied it to web search, but since then, Hadoop has become central to everything we do at the company. Today, Hadoop is the de facto platform for processing and storing big data for thousands of companies around the world, including most of the Fortune 500. It has also given birth to a thriving industry around it, comprised of a number of companies who have built their businesses on the platform and continue to invest and innovate to expand its capabilities.
Another commonly adjusted setting is chroma-subsampling, which alters the amount of color information stored in a JPEG file. With a setting of 4:4:4, the two chroma (color) channels in a JPG have as much information as the luminance channel. In an image with a setting of 4:2:0, each chroma channel has only a quarter as much information as in an a 4:4:4 image.
q=96, chroma=4:4:4 (125KB)
q=70, chroma=4:4:4 (67KB)
q=96, chroma=4:2:0 (62KB)
a=70, chroma=4:2:0 (62KB)Table 1: JPEG stored at different quality and chroma levels. The top image is saved at high quality and chroma level – notice the color and detail in the folds of the red flag. The bottom image has the lowest quality – notice artifacts along the right edges of the red flag.Perceptual JPEG CompressionIdeally, we’d have an algorithm which automatically tuned all JPEG parameters to make a file smaller, but which would limit perceptible changes to the image. Technology exists that attempts to do this and can decrease image file size by over 30%. This compression ratio is highly dependent on image content and dimensions.
Fig 2. Compressed (l) and uncompressed ® images. Compressed image is 36% smaller.We were pleased with perceptually compressed images in cursory examinations, compressed images were smaller and nearly indistinguishable from their sources. But we wanted to really quantify how well it worked before rolling it out. The standard computational tools for evaluating compression, such as SSIM, are fairly simplistic and don’t do a great job at modeling how a user sees things. To really evaluate this technology had to use a better measure of perceptibility: human minds. 
To test whether our image compression would impact user perception of image quality, we put together a “taste test”. The taste test was constructed as a game with multiple rounds where users looked at both compressed and uncompressed images. Users accumulated points the longer they played, and got more points for doing well at the game. We maintained a leaderboard to encourage participation and used only internal testers. The game’s test images came from a diverse collection of 250 images contributed by Flickr staff. The images came from a variety of cameras and included a number of subjects from photographers with varying skill levels.
In each round, our test code randomly selected a test image, and presented two variants of this image side by side. 50% of the time we presented the user two identical images; the rest of the time we presented one compressed image and one uncompressed image. We asked the tester if the two images looked the same or different. We expected a user choosing randomly OR a user unable to distinguish the two cases would answer correctly about half the time. We randomly swapped the location of the compressed images to compensate for user bias to the left or the right. If testers chose correctly, they were presented with a second question: "Which image did you prefer, and why?”
Fig 4. Screenshot of taste test.Our test displayed images simultaneously to prevent testers noticing a longer load time for the larger, non-compressed image. The images were presented with either 320, 640, or 1600 pixels on their longest side. The 320 & 640px images were shown for 12 seconds before being dimmed out. The intent behind this detail was to represent how real users interacted with our images. The 1600px images stayed on screen for 20 seconds, as we expected larger images to be viewed for longer periods of time by real users.Taste Test Outcome and DeploymentWe ran our taste test for two weeks and analyzed our results. Although we let users play as long as they liked, we skipped the first result per user as a “warm-up” and considered only the subsequent ten results, which limited the potential for users training themselves to spot compression artifacts. We disregarded users that had fewer than eleven results.
Table 2. Taste test results. Testers selected “identical” at nearly the same rate, whether the input was identical or not.When our testers were presented with two identical images, they thought the images were identical only 68.8% of the time(!), and when presented with a compressed image next to a non-compressed image, our testers thought the images were identical slightly less often: 67.6% of the time. This difference was small enough for us, and our statisticians told us it was statistically insignificant. Our image pairs were so similar that multiple testers thought all images were identical and reported that the test system was buggy. We inspected the images most often labeled different, and found no significant artifacts in the compressed versions.So even in this side-by-side test, perceptual image compression was just barely noticeable when images were presented side-by-side. As the Flickr website wouldn’t ever show compressed and uncompressed images at the same time, and the use of compression had large benefits in storage footprint and site performance, we elected to go forward.At the beginning of 2014 we silently rolled out perceptual-based compression on our image thumbnails (we don’t alter the “original” images uploaded by our users). The slight changes to image appearance went unnoticed by users, but user interactions with Flickr became much faster, especially for users with slow connections, while our storage footprint became much smaller. This was a best-case scenario for us.Evaluating perceptual compression was a considerable task, but it gave the confidence we needed to apply this compression in production to our users. This marked the first time Flickr had adjusted image settings in years, and, it was fun.
Fig 5. Taste test high score listEpilogueAfter eighteen months of perceptual compression at Flickr, we adjusted our settings slightly to shrink images an additional 15%. For our users on mobile devices, 15% fewer bytes per image makes for a much more responsive experience. We had run a taste test on this newer setting and users were were able to spot our compression slightly more often than with our original settings. When presented a pair of identical images, our testers declared these images identical 65.2% of the time, when presented with different images, our testers declared the images identical 62% of the time. It wasn’t as imperceptible as our original approach, but, we decided it was close enough to roll out.Boy were we wrong! A few very vocal users spotted the compression and didn’t like it at all. The Flickr Help Forum had a very lively thread which 
The ChallengeOur computational footprint made it easy to create per-user Magic View categories for over 12 billion images on Flickr; however, we also needed to combine this with updating the categories with the the tens of millions of tags generated from photos as they are uploaded in real-time. Ideally, the system has to allow us to efficiently but separately manage the bulk and real-time data that only computes the final state when requested. We turned to 
Lambda ArchitectureThis equation is shown graphically in the figure above. The real-time and bulk compute subsystem write to independent databases, which could be totally different systems. When dealing with a high volume of realtime data, the operational advantage here can be significant – there’s no need to have the expense of combining it with bulk data every time an event comes in.Concerns around this approach center on the complicated nature of the Combiner function – there is the developer and systems cost from the need to maintain two separate databases, the differing latencies of querying both sides and then the mechanics of merging the result. Our ApproachWe addressed the complications of the Combiner by instead using a single database to store the real-time and bulk data. A Combiner is still required to compute a final result:result = Combiner(Query(data))
How was this achieved? We implement our simplified Lambda architecture in HBase by giving each row two sets of columns – real-time and bulk – which are managed independently by the real-time subsystem (Storm and Java) and the bulk compute subsystem (Pig Latin and Oozie). It’s worth noting that 