Security updates have been issued by Arch Linux (lame, salt, and xorg-server), Debian (ffmpeg, imagemagick, libxfont, wordpress, and xen), Fedora (ImageMagick, rubygem-rmagick, and tor), Oracle (kernel), SUSE (kernel, SLES 12 Docker image, SLES 12-SP1 Docker image, and SLES 12-SP2 Docker image), and Ubuntu (curl, glance, horizon, kernel, keystone, libxfont, libxfont1, libxfont2, libxml2, linux, linux-aws, linux-gke, linux-kvm, linux-raspi2, linux-snapdragon, linux, linux-raspi2, linux-gcp, linux-hwe, linux-lts-xenial, nova, openvswitch, swift, and thunderbird).
Security updates have been issued by Arch Linux (ffmpeg2.8, nvidia, and openvpn), Fedora (git, mercurial, moodle, php-horde-Horde-Image, poppler, and pure-ftpd), openSUSE (fmpeg and vlc), Oracle (firefox, kernel, and nss), Red Hat (firefox and nss), Slackware (mozilla), and SUSE (firefox).
Security updates have been issued by Arch Linux (tomcat7), Debian (kernel and perl), Fedora (libwmf and mpg123), Mageia (bluez, ffmpeg, gstreamer0.10-plugins-good, gstreamer1.0-plugins-good, libwmf, tomcat, and tor), openSUSE (emacs, fossil, freexl, php5, and xen), Red Hat (augeas, rh-mysql56-mysql, samba, and samba4), Scientific Linux (augeas, samba, and samba4), Slackware (samba), SUSE (emacs and kernel), and Ubuntu (qemu).
Security updates have been issued by Arch Linux (ffmpeg, lib32-libgcrypt, libgcrypt, linux-zen, and newsbeuter), Debian (emacs25, freexl, and tomcat8), Fedora (cyrus-imapd, FlightGear, freexl, gdm, kernel, LibRaw, ruby, and xen), Gentoo (binutils, chkrootkit, curl, gdk-pixbuf, gimps, git, kpathsea, mod_gnutls, perl, squirrelmail, subversion, supervisor, and webkit-gtk), Mageia (389-ds-base, kernel, kernel-linus, kernel-tmb, and mpg123), openSUSE (ffmpeg, ffmpeg2, qemu, and xen), Slackware (kernel), SUSE (xen), and Ubuntu (gdk-pixbuf).
Post Syndicated from Janina Ander original https://www.raspberrypi.org/blog/digitising-reels-pi-film-capture/
Joe Herman’s Pi Film Capture project combines old projectors and a stepper motor with a Raspberry Pi and a Raspberry Pi Camera Module, to transform his grandfather’s 8- and 16-mm home movies into glorious digital films.
We chatted to him about his Pi Film Capture build at Maker Faire New York 2016:
Uploaded by Raspberry Pi on 2017-08-25.
What inspired Pi Film Capture?
Joe’s grandfather, Leo Willmott, loved recording home movies of his family of eight children and their grandchildren. He passed away when Joe was five, but in 2013 Joe found a way to connect with his legacy: while moving house, a family member uncovered a box of more than a hundred of Leo’s film reels. These covered decades of family history, and some dated back as far as 1939.
This provided an unexpected opportunity for Leo’s family to restore some of their shared history. Joe immediately made plans to digitise the material, knowing that the members of his extensive family tree would provide an eager audience.
Building Pi Film Capture
After a failed attempt with a DSLR camera, Joe realised he couldn’t simply re-film the movies — instead, he would have to capture each frame individually. He combined a Raspberry Pi with an old Super 8 projector, and set about rigging up something to do just that.
He went through numerous stages of prototyping, and his final hardware setup works very well. A NEMA 17 stepper motor moves the film reel forward in the projector. A magnetic reed switch triggers the Camera Module each time the reel moves on to the next frame. Joe hacked the Camera Module so that it has a different focal distance, and he also added a magnifying lens. Moreover, he realised it would be useful to have a diffuser to ‘smooth’ some of the faults in the aged film reel material. To do this, he mounted “a bit of translucent white plastic from an old ceiling fixture” parallel with the film.
In addition to capturing every single frame (sometimes with multiple exposure settings), Joe found that he needed intensive post-processing to restore some of the films. He settled on sending the images from the Pi to a more powerful Linux machine. To enable processing of the raw data, he had to write Python scripts implementing several open-source software packages. For example, to deal with the varying quality of the film reels more easily, Joe implemented a GUI (written with the help of PyQt), which he uses to change the capture parameters. This was a demanding job, as he was relatively new to using these tools.
If a frame is particularly damaged, Joe can capture multiple instances of the image at different settings. These are then merged to achieve a good-quality image using OpenCV functionality. Joe uses FFmpeg to stitch the captured images back together into a film. Some of his grandfather’s reels were badly degraded, but luckily Joe found scripts written by other people to perform advanced digital restoration of film with AviSynth. He provides code he has written for the project on his GitHub account.
What does Pi Film Capture deliver?
Joe provides videos related to Pi Film Capture on two sites: on his YouTube channel, you’ll find videos in which he has documented the build process of his digitising project. Final results of the project live on Joe’s Vimeo channel, where so far he has uploaded 55 digitised home videos.
Shot on 8mm by Leo Willmott, captured and restored by Joe Herman (Not a Wozniak film, but placed in that folder b/c it may be of interest to Hermans)
We’re beyond pleased that our tech is part of this amazing project, helping to reconnect the entire Herman/Willmott clan with their past. And it was great to be able to catch up with Joe, and talk about his build at Maker Faire last year!
Maker Faire New York 2017
We’ll be at Maker Faire New York again on the 23-24 September, and we can’t wait to see the amazing makes the Raspberry Pi community will be presenting there!
Are you going to be at MFNY to show off your awesome Pi-powered project? Tweet us, so we can meet up, check it out and share your achievements!
Post Syndicated from Andy Klein original https://www.backblaze.com/blog/media-archive-solutions/
Does this sound familiar?
You spent almost all day Saturday archiving your latest video project onto two 8 TB external hard drives. You need to archive four months’ worth of work from a recently finished video project to external hard drives to make room on your local storage system for the next project. You diligently label each of the newly minted archive drives with the project name and stack the drives neatly in your closet. There must be 50 drives in there. The final step is to add the file list from each drive to the catalog you keep on a shared spreadsheet so your employees and contractors can find content from previous projects. In reality, this type of searching is rarely done as rummaging through the closet for the correct archive drive is time consuming and on more than one occasion the drive has failed.
Are you thinking that maybe it is time to upgrade your media archive solution?
Media Archiving Solutions
There is no shortage of media archiving solutions and you’ve looked at everything from tape drive systems to SAN and NAS systems. Some are expensive, some are complex, and some are both. Here are a few things you’ve decided that you would like your media archive solution to do:
- Fit into your company’s workflow
- Make the archive more accessible and useable
- Protect your archive off site
Archiving Your Media Content with Archiware P5 Archive
One proven solution is Archiware P5 Archive, which is part of the P5 software suite that provides data management at every step of your data’s life cycle. The P5 suite works with all kinds of data, but it has become well known for how well it archives and backs up media files, e.g. video and photos.
P5 Archive lets you easily archive data from your primary storage system to less expensive storage such as external disk, tape, and the cloud. Once the data is archived, you can use P5 to quickly locate your data in the archive by searching with keywords or previews. For example:
- Search assets by keyword — Besides the default search parameters, you can add custom metadata to individualize your data storage. You can include categories such as time of day, lens, or film location, and thus locate and re-use your data more quickly and effectively.
- Search assets with previews — P5 Archive has a direct integration with FFmpeg and ImageMagick and can create low resolution previews and proxies for all common video and audio formats.
With these capabilities, P5 can function as a bare bones asset management solution managing video and image archives. When you are ready to move forward with a more robust media asset management (MAM) system, P5 has integrations to leading providers including axle Video, Cat DV, and Cantemo Portal.
P5 Archive also includes the ability to customize the end-user experience so that users can access data (archived or live) based on their user profile.
Your Media Archive can be an Asset
P5 Archive lets you move or migrate data to disk, tape and the cloud. With Archiware P5 version 5.5, you can backup and archive your media files to Backblaze B2 Cloud Storage. With B2, your archived files are readily available for retrieval via P5. When this is combined with the P5 preview and keyword search capabilities, you can locate and retrieve archived video clips, images, and files in minutes versus hours or days when using external disk or tape.
Getting Rid of Your Closet Full of Hard Drives
Even if you move forward with P5, you still have your current closet full of archived data. To help with that, Backblaze provides the B2 Fireball data transfer service, which allows you to transfer up to 40 TB of data per trip from your location to your B2 Cloud Storage account. In this case, you’ll have to transfer the data from each external drive in your closet to a server or NAS device. Once there, the collected data is transferred to the Fireball and the Fireball is returned to Backblaze, where the data is extracted and placed in your B2 account.
As noted, each Fireball holds up to 40 terabytes of data — that’s ten 4 TB external drives — so it will take three round trips to transfer the 100 TB of archived data in your closet. Of course you can decide to transfer some or all of the data residing in your closet. How much you transfer depends on how much you want to protect it offsite and how valuable ready access to it on B2 from anywhere is to you.
If your media archive is a pile of hard drives or an aging tape library, the combination of Archiware P5 and Backblaze B2 provides a practical, affordable way to move your media archive library to the cloud. This will let you improve access to your archived data, reduce the management of your local storage system, protect your valuable assets offsite, and best of all you’ll be able to use your closet to store old computer monitors and pristine user manuals like everyone else.
На FOSDEM 2016 видео потоците в локалната мрежа бяха носени през UDP, което при загуби по мрежата водеше до разни неприятни прекъсвания и обърквания на ffmpeg-а.
След разговори по темата за мрежа без загуби, пакети, пренасяни от еднорози и изграждане на infiniband мрежа в ULB, бях стигнал до идеята да търся или нещо с forward error correction, или някакъв reliable multicast. За FEC се оказа, че има някаква реализация от едно време за ffmpeg за PRO-MPEG, която не е била приета по някакви причини, за reliable multicast открих два протокола – PGM и NORM.
За PGM се оказа, че има хубава реализация, която 1) я има в Debian, 2) има прилични примери и 3) може да има средно ужасна документация, но source е сравнително четим и става за дебъгване. Измъкнах си старото ttee, разчистих кода от разни ненужни неща и си направих едно тривиално proxy, което да разнася пакети между UDP и PGM (и stdin/stdout за дебъгване). Може да се намери на https://github.com/krokodilerian/pgmproxy, като в момента е в proof-of-concept състояние и единственото, което мога да кажа е, че успявам да прекарам през него един FLAC през мрежата и да го слушам 🙂 Следват тестове в мрежа със загуби (щото в моя локален wifi са доста малко) и доизчистване, че да го ползваме на FOSDEM.
Security updates have been issued by Debian (ffmpeg, fontforge, and openjdk-7), Fedora (cvs, java-1.8.0-openjdk-aarch32, krb5, and mercurial), Mageia (chromium and libgxps), Red Hat (rh-nginx110-nginx), SUSE (java-1_7_1-ibm), and Ubuntu (ghostscript, kernel, linux, linux-aws, linux-gke, linux-raspi2, linux-snapdragon, linux, linux-raspi2, linux-hwe, linux-lts-xenial, and python-crypto).
Security updates have been issued by Debian (botan1.10, cvs, firefox-esr, iortcw, libgd2, libgxps, supervisor, and zabbix), Fedora (curl, firefox, git, jackson-databind, libgxps, libsoup, openjpeg2, potrace, python-dbusmock, spatialite-tools, and sqlite), Mageia (cacti, ffmpeg, git, heimdal, jackson-databind, kernel-linus, kernel-tmb, krb5, php-phpmailer, ruby-rubyzip, and supervisor), openSUSE (firefox, librsvg, libsoup, ncurses, and tcmu-runner), Oracle (firefox), Red Hat (java-1.8.0-ibm), Slackware (git, libsoup, mercurial, and subversion), and SUSE (kernel).
Post Syndicated from Christie Gifrin original https://aws.amazon.com/blogs/compute/messaging-fanout-pattern-for-serverless-architectures-using-amazon-sns/
Sam Dengler, Amazon Web Services Solutions Architect
Serverless architectures allow solution builders to focus on solving challenges particular to their business, without assuming the overhead of managing infrastructure in AWS. AWS Lambda is a service that lets you run code without provisioning or managing servers.
When using Lambda in a serverless architecture, the goal should be to design tightly focused functions that do one thing and do it well. When these functions are composed to accomplish larger goals in microservice architectures, the complexity shifts from the internal components to the external communication between components. It’s all too easy to accidentally back into an architecture that is rigid to change because components are too knowledgeable of each other via the communication paths between them.
Solution builders can address this architectural challenge by using messaging patterns, resulting in loosely coupled communication between highly cohesive components to manage complexity in serverless architectures. As introduced in the recent Building Scalable Applications and Microservices: Adding Messaging to Your Toolbox post, a common approach when one component wishes to deliver the same message to multiple receivers is to use the fanout publish/subscribe messaging pattern.
The fanout pattern for message communication can be implemented in code. However, depending on your requirements, alternative solutions exist to offload this undifferentiated responsibility from the application. Amazon SNS is a fully managed pub/sub messaging service that lets you fan out messages to large numbers of recipients.
In this post, I review a serverless architecture from PlayOn! Sports as a case study for migration of fanout functionality from application code to SNS.
PlayOn! Sports serverless video processing platform
PlayOn! Sports is one of the nation’s leading high school sports media companies. They operate a comprehensive technology platform, enabling high-quality, low-cost productions of live sports events for the NFHS High School Sports Network.
At the 2014 AWS re:Invent conference, Lambda was announced. The PlayOn! Sports technology team recognized the parallels between serverless demos featuring image processing using ImageMagick to video processing using ffmpeg.
At the time, PlayOn! Sports was broadcasting live video with adaptive bit rates, requiring a transcoding of the video stream to multiple quality levels for consumption on desktop, mobile, and connected devices. This is not unusual for an internet media company. However, with over 50,000 live broadcasts produced in 2014, the traditional media and entertainment technological approaches and pricing models would not work.
After some consultation with the Lambda team to validate support for custom binary execution, PlayOn! Sports moved forward with the development of a new serverless video processing platform according to the architecture diagram below.
In the architecture, a laptop in the field captures the video from a camera source and divides it into small fragments, according to the HLS protocol. The fragments are published to Amazon S3, through an Amazon CloudFront distribution for accelerated upload. When the file has been written to S3, it triggers a Lambda function to initiate the video segment processing.
Video transcoding fanout implementation in Lambda
Given Lambda’s integration growth across AWS, it’s easy to forget that it did not include managed integration with SNS when it was announced in November 2014.
PlayOn! Sports was actively experimenting with approaches to video processing to address quality control, audience growth, and cost constraints. Lambda was a great tool for rapid innovation. A goal of the architecture design was the ability to add and remove video processing alternatives to the workflow, using the fanout pattern to identify optimal solutions. Below is example code from the initial implementation:
import json import logging import boto3 logger = logging.getLogger('boto3') logger.setLevel(logging.INFO) client = boto3.client('lambda') fanout_functions = ['media_info', 'transcode_audio'] def lambda_handler(event, context): logger.info(json.dumps(event)) logger.info('fanout_functions: %s', fanout_functions) for fanout_function in fanout_functions: logger.info('invoke: %s', fanout_function) response = client.invoke( FunctionName=fanout_function, InvocationType='Event', Payload=json.dumps(event) ) logger.info('response: %s', response) return 'done'
Each Lambda function is invoked asynchronously, injecting the same S3 event that triggered the original Lambda function. For example, the media_info Lambda function could be scaffolded similar to the following code snippet:
import json import logging import boto3 logger = logging.getLogger('boto3') logger.setLevel(logging.INFO) def lambda_handler(event, context): logger.info(json.dumps(event)) # MediaInfo Processing # see: https://aws.amazon.com/blogs/compute/extracting-video-metadata-using-lambda-and-mediainfo/ return 'done'
Refactoring fanout implementation using SNS
The PlayOn! Sports development team was familiar with SNS, but had not used it previously to support system-to-system messaging patterns. After the announcement of SNS triggering of Lambda functions, the PlayOn! Sports team planned to migrate to the new feature to offload the overhead of managing the fanout Lambda function.
When invoking a Lambda function, SNS wraps the original event with SNSEvent. The Lambda function can be refactored by adding a function to parse the S3 event from SNSEvent, as seen in the following code:
import json import logging import boto3 logger = logging.getLogger('boto3') logger.setLevel(logging.INFO) def lambda_handler(event, context): s3_event = parse_event(event) logger.info(json.dumps(s3_event)) # MediaInfo Processing # see: https://aws.amazon.com/blogs/compute/extracting-video-metadata-using-lambda-and-mediainfo/ return 'done' def parse_event(event): record = event['Records'] if 'EventSource' in record and record['EventSource'] == 'aws:sns': return json.loads(record['Sns']['Message']) return event
This Lambda function modification can be authored, tested, and deployed before enabling the SNS integration to verify that the existing Lambda fanout execution path continues to operate as before. The Lambda function invocation can now be transferred from the fanout Lambda function to SNS without disruption to S3 processing.
As the diagram below shows, the resulting architecture is similar to the original. The exception is that objects written to S3 now trigger a message to be published to an SNS topic. This sends the S3 event to multiple Lambda functions to be processed independently.
Sample architecture deployment using AWS CloudFormation
AWS CloudFormation gives developers and system administrators an easy way to create and manage a collection of related AWS resources. CloudFormation provisions and updates resources in an orderly and predictable fashion. To launch the CloudFormation stack for the sample fanout architecture, choose the following button:
Follow these steps to complete the architecture deployment:
- If necessary, sign into the console for your account when prompted.
- On the Select Template page, choose Next.
- Under Parameters, for S3BucketName, enter a globally unique name.
- For SnsTopicName, enter a region-unique name.
- Choose Next, Next.
- Select the checkbox for I acknowledge that AWS CloudFormation might create IAM resources, and choose Create.
After the stack has completed creation, you can test both paths of execution by uploading files to the S3 bucket that you created. Uploading a file to the “/uploads/lambda/” directory in S3 triggers the Lambda fanout function. Uploading a file to the “/uploads/sns/” directory in S3 triggers the SNS fanout execution path. You can verify execution by monitoring the Lambda function outputs in CloudWatch Logs.
In this post, I reviewed the fanout messaging pattern and options for its inclusion in a serverless architectures using Lambda application code and SNS. Using the PlayOn! Sports serverless video processing pipeline use case, I demonstrated how easy it is to refactor an existing application to use the SNS fanout approach.
I also provided a sample architecture in CloudFormation that you can run in your own account. Try it out and expand the sample architecture by adding other Lambda functions to the SNS topic, to demonstrate the flexibility of the fanout messaging pattern!
You can get started with SNS using the AWS Management Console, or the SDK of your choice. For more information about how to use SNS fanout messaging, see the following resources:
- Amazon SNS Developer Guide:
- AWS Compute Blog: Fanout S3 Event Notifications to Multiple Endpoints
If you have questions or suggestions, please comment below.
Security updates have been issued by Arch Linux (bind, qt5-webengine, and systemd), Debian (puppet and sudo), Fedora (drupal7, globus-ftp-client, globus-gass-cache-program, globus-gass-copy, globus-gram-job-manager, globus-gridftp-server, globus-gssapi-gsi, globus-io, globus-net-manager, globus-xio, globus-xio-gsi-driver, globus-xio-pipe-driver, globus-xio-udt-driver, libgcrypt, and myproxy), openSUSE (ffmpeg), Slackware (kernel), SUSE (unrar), and Ubuntu (libgcrypt11, libgcrypt20).
Security updates have been issued by Arch Linux (irssi, lib32-libtasn1, and wireshark-cli), Debian (libmwaw, otrs2, and tor), Fedora (ansible, freeradius, gnutls, mingw-poppler, mosquitto, oniguruma, perltidy, picocom, systemd, and wget), Mageia (ansible, dropbear, gajim, libsndfile, libxslt, lxc, zoneminder, and zziplib), openSUSE (ffmpeg, libnettle, mysql-connector-cpp, mysql-workbench, and wireshark), and Ubuntu (irssi).
Security updates have been issued by Arch Linux (lib32-nss), Debian (bind9, exiv2, fop, imagemagick, libical, libonig, libsndfile, mosquitto, openjdk-7, rzip, strongswan, and tnef), Fedora (git, kernel, lynis, moodle, mupdf, samba, systemd, and webkitgtk4), Mageia (perl-Image-Info and vlc), openSUSE (ffmpeg2, git, java-1_7_0-openjdk, libplist, libsndfile, and samba), Oracle (kernel and samba3x), Red Hat (nss), Scientific Linux (nss), and Ubuntu (imagemagick, juju-core, libtiff, strongswan, and webkit2gtk).
Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/youtube-live-streaming-docker/
Looking to share your day, event, or the observations of your nature box live on the internet via a Raspberry Pi? Then look no further, for Alex Ellis has all you need to get started with YouTube live-streaming from your Pi.
If you spend any time on social media, be it Facebook, Instagram, YouTube, or Twitter, chances are you’ve been notified of someone ‘going live’.
Live-streaming video on social platforms has become almost ubiquitous, whether it’s content by brands, celebrities, or your cousin or nan – everyone is doing it.
Carrie Anne and Alex offer up a quick tour of the Pi Towers lobby while trying to figure out how Facebook Live video works.
YouTube live-streaming with Alex Ellis and Docker
In his tutorial, Alex demonstrates an easy, straightforward approach to live-streaming via a Raspberry Pi with the help of a Docker image of FFmpeg he has built. He says that with the image, instead of “having to go through lots of manual steps, we can type in a handful of commands and get started immediately.”
Why is the Docker image so helpful?
As Alex explains on his blog, if you want to manually configure your Raspberry Pi Zero for YouTube live-streaming, you need to dedicate more than a few hours of your day.
Normally this would have involved typing in many manual CLI commands and waiting up to 9 hours for some video encoding software (
ffmpeg) to compile itself.
Get anything wrong (like Alex did the first time) and you have to face another nine hours of compilation time before you’re ready to start streaming – not ideal if your project is time-sensitive.
See you in 8-12 hours? Building ffmpeg on a my @Raspberry_Pi #pizero with @docker
Using the Docker image
In his tutorial, Alex uses a Raspberry Pi Zero and advises that the project will work with either Raspbian Jessie Lite or PIXEL. Once you’ve installed Docker, you can pull the FFmpeg image he has created directly to your Pi from the Docker Hub. (We advise that while doing so, you should feel grateful to Alex for making the image available and saving you so much time.)
It goes without saying that you’ll need a YouTube account in order to live-stream to YouTube; go to the YouTube live streaming dashboard to obtain a streaming key.
Get live streaming to @YouTube with this new weekend project and guide using your @Raspberry_Pi and @docker. https://t.co/soqZ9D9jbS
For a comprehensive breakdown of how to stream to YouTube via a Raspberry Pi, head to Alex’s blog. You’ll also find plenty of other Raspberry Pi projects there to try out.
Why live-stream from a Raspberry Pi?
We see more and more of our community members build Raspberry Pi projects that involve video capture. The minute dimensions of the Raspberry Pi Zero and Zero W make them ideal for fitting into robots, nature boxes, dash cams, and more. What better way to get people excited about your video than to share it with them live?
If you have used a Raspberry Pi to capture or stream footage, make sure to link to your project in the comments below. And if you give Alex’s Docker image a go, do let us know how you get on.