Tag Archives: videography

3-2-1 Backup Best Practices Using the Cloud

Post Syndicated from Roderick Bauer original https://www.backblaze.com/blog/3-2-1-backup-best-practices-using-the-cloud/

Archive 3-2-1

Whether you’re a hobbyist or a professional photographer or videographer, employing a 3-2-1 backup strategy for your valuable photos and videos is critical. A good backup strategy can protect you from accidental or incidental data loss and make sure your working or archived files are available when you need them.

Most photographers and videographers are aware of the necessity to back up their data, but like a lot of things that are good for us, like eating kale and exercising regularly, putting good habits into practice can be challenging. Maybe you’re currently using the cloud as part of your backup or archive strategy, or perhaps you’re still juggling hard disk drives between your workstations, a storage closet, and an offsite location.

If you’re not yet using the cloud, or are still getting familiar with the cloud for data backup and archiving, I’d like to go over some ways in which the cloud can make managing your data easier and provide you with a number of benefits you might not currently enjoy.

Let’s first do a quick review of 3-2-1 backup strategy.

The 3-2-1 Backup Strategy

A 3-2-1 strategy means having at least three total copies of your data, two of which are local but on different media, and at least one copy that is offsite.

A Typical 3-2-1 Scenario

Let’s use landscape.cr2 as an example file for this scenario. Landscape.cr2 lives on your primary computer. That’s one copy of the data file. You also have an external hard drive or Network-Attached Storage (NAS) that you use for backing up your computer. Your backup program runs on a regular schedule, or whenever a file is added to your system, and backs up landscape.cr2 to your external drive(s). That’s a second copy on a different device or medium. In addition to that external hard drive, you also have an online backup solution that makes another copy of your data. The backup program continuously scans your computer and uploads your data to a data center (aka the cloud). Landscape.cr2 is included in this upload, and that becomes the third copy of your data.

Why Two Onsite Copies and One Offsite Copy?

Whichever kind of computer you are using, an onsite backup is a simple way of having quick access to your data should anything happen to your computer. If your laptop or desktop’s hard drive crashes, and you have been regularly backing up to an external hard drive or NAS, you can quickly get the majority of your data back (or use the external drive on another computer while yours gets fixed or replaced). If you use an automatic backup program, the exposure for data loss is fairly minimal.

Synology NAS and cloud backup symbol

Having an onsite backup is a great start, but having an offsite backup is a key component to completing a backup strategy. Onsite backups are easy to set up, but unfortunately having a backup near the device that it’s backing up (for example, having a desktop PC or Mac and an external hard drive on the same desk), means that both of those copies of your data are susceptible to loss in case of fire, theft, water damage, or other unforeseen occurrences.

Backblaze data center

Most often, if the two devices you have as your local copies are close together, they’ll both be affected if the unfortunate should happen. A continuously updated copy of your data that’s not in the same physical location as the other two is paramount in protecting your files. Even the United States Government recommends this approach. In a 2012 paper for US-CERT (United States Computer Emergency Readiness Team), Carnegie Mellon recommended the 3-2-1 method in their publication titled: Data Backup Options.

The Cloud as Part of 3-2-1

a storage vault in the middle of a cloud

The cloud can make fulfilling the 3-2-1 strategy much easier. And, with recent advances in technology and cost competition, the cloud brings other advantages:

Broadband speed and coverage — Broadband bandwidth has increased and is more widely available while the reach of cellular data service has made many remote locations accessible. It’s possible to upload data to the cloud from home, office, and even when traveling to remote locations. For example, the summit of Mt. Everest now has mobile network service.

Competitive costCompetition in cloud storage has made for competitive pricing and a range of services. The cloud is more affordable than ever.

Advantages of Adding the Cloud to 3-2-1

If you’re already using 3-2-1, then you’ve made a great start in keeping your data safe. If you’re not yet using the cloud as part of your backup strategy, then you might consider the following advantages of adding it to your data security plans.

Convenience
The two offsite copies of your data required by 3-2-1 can be anywhere that’s geographically separated from your primary location. That can be convenient for some, such as for a photographer friend who takes a backup hard disk to leave at his mother’s house during their regular Sunday dinner. It’s not so easy for others, who have to transport or ship disks to other locations to fulfill the diverse location requirement. The cloud handles this without any extra effort.

Durability
Cloud data centers are designed to protect data against outages, service interruptions, hardware failures, and natural disasters. Backblaze claims 99.999999999% (11 9s) annual durability for its customers’ data.

Sharing & Collaboration
Having data in the cloud can make sharing much easier. Users can control who has access and to what data. Backblaze Backup and B2 Cloud Storage support sharing links that can be sent to anyone who needs permanent or temporary access to stored data. This is ideal if you’re working with second shooters on a project or relaying final deliverables to a client.

Data Ingest/Seeding
As digital resolutions increase, media files grow larger and larger. Forty-five megapixel images and 8K digital videos can quickly fill up any storage media and put demands on the time and bandwidth required to transfer data. Some cloud services provide seeding services that enable physical transfer of data directly to the cloud. An example is the Backblaze B2 Fireball, which is a 70 TB hard disk array with 1 GB connectivity that enables the customer to load and ship data securely to Backblaze’s data centers.

Challenges of the Cloud

For some, there are real challenges using the cloud for backing up or archiving data, especially when they have a lot of data, as many photographers and videographers do. As services expand and new technologies are adopted, transfer speeds will continue to increase and should help overcome that hurdle.

Data center racks

In the meantime, here are some tips for meeting these challenges:

  • Schedule your data uploads for off hours when the network load is light and the transfers won’t impede other data traffic.
  • Leverage multi-threaded uploads to improve transfer speed.
  • Take advantage of data ingest options to seed data to the cloud. It’s definitely faster and can even be more economical compared to other data transfer options.
  • Be patient. Once you get your initial files uploaded or seeded to the cloud, it becomes much easier to upload incremental updates. In the near-future we will see 5G mobile networks and higher broadband speeds that will make data transfers even faster.

Are you Using the Cloud to Best Advantage?

Backups are great for your active projects, but how do you handle your archives? We recently wrote about the difference between backing up and archiving, and knowing the difference will improve your data management strategy.

Many photographers and videographers are using a backup or even a sync solution for their data when archiving is the approach that better suits their needs. Briefly, a data backup is for recovery from hardware failure or recent data corruption or loss, and an archive is for space management and long term retention. If you’re using a data backup or sync service to store data that you wish to keep permanently or long-term, you’re trying to fit a round peg into a square hole.

What’s the Best Use for Backup?

  • Working files currently being edited, or in a live project.
  • Documents, correspondence, application settings, and other transient system information.

What’s the Best Use for Archive?

  • Finished projects for which you wish to retain all or just the primary data files used.
  • Photos and videos that you might use again at some time in the future.
  • Media that has value to your business for possible future sales.

Making the Most of the Cloud

If you’re following a 3-2-1 backup strategy that includes the cloud, you’ll be ahead of 90% of your peers. The cloud is becoming more useful and more economical every day. When you add the security of the cloud, collaboration with clients and peers, and proven durability to that list, the cloud is an unbeatable choice for upping your game in data backup and archiving.

You can read more posts in this series written in conjunction with Lensrentals.com on photography and videography.

•  •  •

Note: This post originally appeared on Lensrentals.com on September 18, 2018.

The post 3-2-1 Backup Best Practices Using the Cloud appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Five Best Practices to Securely Preserve Your Video, Photo, and Other Data

Post Syndicated from Roderick Bauer original https://www.backblaze.com/blog/five-best-practices-to-securely-preserve-your-video-photo-and-other-data/

computer and camera overlooking a lake

Whether you’re working with video, photo, audio, or other data, preserving the security of your data has to be at the top of your priority list. Data security might sound like a challenging proposition, but by following just a handful of guidelines it becomes a straightforward and easily accomplished task.

We’d like to share what we consider best practices for maintaining the safety of your data. For both seasoned pros and those just getting started with digital media, these best practices are important to implement and revisit regularly. We believe that by following these practices — independently of which specific data storage software, service, or device you use — you will ensure that all your media and other data are kept secure to the greatest extent possible.

The Five Best Practices to Keep Your Digital Media Safe

1 — Keep Multiple Copies of Your Media Files

Everyone by now is likely familiar with the 3-2-1 strategy for maintaining multiple copies of your data (video, photos, digital asset management catalogs, etc.). Following a 3-2-1 strategy simply means that you should always have at least three copies of your active data, two of which are local, and at least one that is in another location.

a tech standing looking at a pod full of hard drives in a data center
Choose a reliable storage provider

Mind you, this is for active data, that is, files and other data that you are currently working on and want to have backed up in case of accident, theft, or hardware failure. Once you’re finished working with your data, you should consider archiving your data, which we’ve also written about on our blog.

2 — Use Trustworthy Vendors

There are times when you can legitimately cut corners to save money, and there are times when you shouldn’t. When it comes to your digital media and services, you want to go with the best. That means using topnotch memory sticks, HDD and SSD drives, software, and cloud services.

For hardware devices and software, it’s always helpful to read reviews or talk with others using the devices to find out how well they work. For hard drive reliability, our Drive Stats blog posts can be informative and are a unique source of information in the data storage industry.

For cloud storage, you want a vendor with a strong track record of reliability and cost stability. You don’t want to use a cloud service or other SaaS vendor that has a history of making it difficult or expensive to access or download your data from their service. A topnotch service vendor will be transparent in their business practices, inform you when there are any outages in their service or maintenance windows, and try as hard as possible to make things right if problems occur.

3 — Always Use Encryption (The Strongest Available)

Encrypting your data provides a number of benefits. It protects your data no matter where it is stored, and also when it is being moved — potentially the most vulnerable exposure your data will have.

Encrypted data can’t be altered or corrupted without the changes being detected, which provides another advantage. Encryption also enables you to meet requirements for privacy and security compliance and to keep up with changing rules and regulations.

Encryption comes in different flavors. You should always select the strongest encryption available, and make sure that any passwords or multi-factor authentication you use are strong and unique for each application.

4 — Automate Whenever Possible

Don’t rely on your memory or personal discipline alone to remember to regularly back up your data. While we always start with the best of intentions, we are busy and we often let things slide (much like resolving to exercise regularly). It’s better to have a regular schedule that you commit to, and best if the backups happen automatically. Many backup and archive apps let you specify when backups, incremental backups, or snapshots occur. You usually can set how many copies of your data to keep, and whether backups are triggered by the date and time or when data changes.

Automating your backups and archives means that you won’t forget to back up and results in a greater likelihood that your data will not only be recoverable after an accident or hardware failure, but up to date. You’ll be glad for the reduced stress and worry in your life, as well.

5 — Be Mindful of Security in Your Workflow

Nobody wants to worry about security all the time, but if it’s ignored, sooner or later that inattention will catch up with you. The best way to both increase the security of your data and reduce stress in your life is to have a plan and implement it.

At its simplest, the concept of security mindfulness means that you should be conscious of how you handle your data during all stages of your workflow. Being mindful shouldn’t require you to overthink, stress or worry, but just to be aware of the possible outcomes of your decisions about how you’re handling your data.

If you follow the first four practices in this list, then this fifth concept should flow naturally from them. You’ve taken the right steps to a long term plan for maintaining your data securely.

Data Security Can Be Both Simple and Effective

The best security practices are the ones that are easy to follow consistently. If you pay attention to the five best practices we’ve outlined here, then you’re well on your way to secure data and peace of mind.

•  •  •

Note:  This post originally appeared on Lensrentals.com on September 18, 2018.

The post Five Best Practices to Securely Preserve Your Video, Photo, and Other Data appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

How Much Photo & Video Data Do You Have Stored?

Post Syndicated from Jim Goldstein original https://www.backblaze.com/blog/how-much-photo-video-data-do-you-have-stored/

How Much Photo and Video Data Do You Have?

Backblaze’s Director of Marketing Operations, Jim, is not just a marketing wizard, he’s worked as a professional photographer and run marketing for a gear rental business. He knows a lot of photographers. We thought that our readers would be interested in the results of an informal poll he recently conducted among his media friends about the amount of media data they store.You’re invited to contribute to the poll, as well!

— Editor

I asked my circle of professional and amateur photographer friends how much digital media data they have stored. It was a quick survey, and not in any way scientific, but it did show the range of data use by photographers and videographers.

Jim's media data storage poll

I received 64 responses. The answers ranged from less than 5 TB (17 users) to 2 petabytes (1 user). The most popular response was 10-19 TB (18 users). Here are the results.

Digital media storage poll results

Jim's digital media storage poll results

How Much Digital Media Do You Have Stored?

I wondered if the results would be similar if I expanded our survey to a wider audience.

The poll below replicates what I asked of my circle of professional and non-professional photographer and videographer friends. The poll results will be updated in real-time. I ask that you respond only once.

Backblaze is interested in the results as it will help us write blog articles that will be useful to our readership, and also offer cloud services suitable for the needs of our users. Please feel free to ask questions in the comments about cloud backup and storage, and about our products Backblaze Backup and Backblaze B2 Cloud Storage.

I’m anxious to see the results.

Our Poll — Please Vote!

How much photo/video data do you have in total (TB)?

Thanks for participating in the poll. If you’d like to provide more details about the data you store and how you do it, we’d love to hear from you in the comments.

The post How Much Photo & Video Data Do You Have Stored? appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Stories of Camera and Data Catastrophes

Post Syndicated from Roderick Bauer original https://www.backblaze.com/blog/stories-of-camera-and-data-catastrophes/

Salt water damaged camera

This is the third post in a series of post exchanges with our friends at Lensrentals.com, a popular online site for renting photography, videography, and lighting equipment. Seeing as how Halloween is just a few days away, we thought it appropriate to offer some scary tales of camera and data catastrophes. Enjoy.

Note: You can read all of Lensrentals’ posts on our blog. Find all of our posts on the Lensrentals blog.

— Editor

Stories of Camera and Data Catastrophes
by Zach Sutton, Editor-in-chief, Lensrentals.com

As one of the largest photo and video gear rental companies in the world, Lensrentals.com ships out thousands of pieces of gear each day. It would be impossible to expect that all of our gear would return to us in the same condition it was in when we rented it out. More often than not, the damage is the result of things being dropped, but now and then some pretty interesting things happen to the gear we rent out.

We have an incredible customer base, and when this kind of damage happens, they’re more than happy to pay the necessary repair fees. Stuff happens, mistakes are made, and we have a full-service repair center to keep the costs low. And while we have insurance policies for accidental damage such as drops, dings, and other accidents, it doesn’t cover neglect, which accounts for the stories we’re going to share with you below. Let’s take a look at some of our more exciting camera and data catastrophe stories.

Camera Data Catastrophes

Data catastrophes happen more often than anything else, but aren’t exactly the most exciting stories we’ve gotten over the years. The stories are usually similar. Someone rents a memory card or SSD from us, uses the card/SSD, then sends it back without pulling the footage off of it. When we receive gear back into our warehouse, we inspect and format all the media. If you realize your mistake and call or email us before that happens, we can usually put a hold on the media and ship it back to you to pull the data off of it. If we’ve already formatted the media, we will perform a recovery on the data using software such as TestDisk and PhotoRec, and let you know if we had any success. We then give you the option whether or not you want to rent the product again to have it shipped to you so you can pull the files.

The Salty Sony A7sII

A common issue we run into — and have addressed a number of times on our blog — is the dubious term “weather resistant.” This term is often used by equipment marketers and doesn’t give you the protection that people might assume by its name.

One example of that was last year, when we received a nonfunctioning Sony a7sII back from the California coast, and had to disassemble it to determine what was wrong. Upon opening the camera, it was quite apparent that it had been submerged in salt water. Water isn’t good for electronics, but the real killer is impurities, such as salt. Salt builds up on electronics, is a conductor of electricity, and will fry electronics in no time when power is applied. So, once we saw the salt corrosion, we knew that the camera was irreparable. Still, we disassembled it for no other reason than to provide evidence to others on what salt water can do to your electronics. You can read more about this and see the full break down in our post, About Getting Your Camera Wet… Teardown of a Salty Sony A7sII.

Sony A7sII disassembled into partsSony A7sII salt water damage

The Color Run Cleanup

Color runs are 5K running events that happen all over the world. If you haven’t seen one, participants and spectators toss colorful powders throughout the run, so that by the time the runners reach the finish line, they’re covered head to toe in colorful powder. This event sounds like a lot of fun, and one would naturally want to take photos of the spectacle, but any camera gear used for the event will definitely require a deep cleaning.

Color run damage to camera lens

Color run damage to camera

We’ve asked our clients multiple times not to take our cameras to color runs, but each year we get another system back that is covered in pink, green, and blue dust. The dust used for these events is incredibly fine, making it easy to get into every nook and cranny within the camera body and lenses. This requires the gear to be completely disassembled, cleaned, and reassembled. We have two photos in this post of the results of a color run, but you can view more on the post we did about Color runs back in 2013, How to Ruin Your (or Our) Gear in 5 Minutes (Without Water).

The Eclipse That Killed Cameras

About a year ago, we had the incredible phenomenon here in the United States of a total solar eclipse. It was the first total solar eclipse to occur in the continental United States since 1979, hence a pretty exciting moment for all of us, but we braced ourselves for the damage it would do to cameras.

Eclipse camera lens damage

For weeks leading up to the event, we sent out fliers with our rentals that encouraged people to not only wear eye protection, but to protect their camera lenses with high-density ND filters. Despite that, in the days following the eclipse, we had gear coming back to us with aperture blades melted and holes burnt into sensors.

Eclipse camera damage

Eclipse camera shutter damage

As one would expect, it’s not a good idea to point your camera directly at the sun, especially for long periods of time. Most of the damage done from the eclipse was caused by people who had set up their camera and lens on a tripod pointing at the sun while waiting for the eclipse. This prolonged exposure causes a lot of heat to build up and will eventually start burning through apertures, shutters, sensors and anything else in its way. Not only do we recommend ND filters for the front of your lens, but also black cards to stop light from entering the camera until it’s go time for the total eclipse. You can read about the whole experience in our blog post on the topic, Rental Camera Gear Destroyed by the Solar Eclipse of 2017.

Damage from Burning Man

While we have countless stories of gear being destroyed, we figured it’d be best to just leave you with this one. Burning Man is an annual event that takes place in the deserts of Nevada. Touted as an art installation and experience, tens of thousands of people spend a few days living in the remote desert with fellow Burners to create and participate in a wide range of activities. And where there is a desert, there always are sand, dust, and dust storms.

Burning Man camera damage

Burning Man dust damage

One might think that sand is the biggest nuisance for camera gear at Burning Man, but it’s actually the fine dust that the wind picks up. One of the more interesting phenomena that happens during Burning Man are the dust storms. The dust storms occur with little warning, kicking up the fine dust buried within the sand that can quickly cause damage to your electronics, your skin, and your lungs. Because it is so fine, it is easily able to enter your cameras and lenses.

Burning Man damage to Nikon camera

While Burning Man doesn’t always totally destroy gear, it does result in a lot of cleaning and disassembling of gear after the event. This takes time and patience and costs the customer money. While there are stories of people who bring camera gear to Burning Man wrapped in nothing more than plastic and gaffer tape, we don’t recommend that for good gear. It’s best to just leave your camera at home, or buy an old camera for cheap to document the week. To see more of what can happen to gear at Burning Man, you can read our blog post on the topic, Please, Don’t Take Our Photography and Video Gear to Burning Man.

Those are just a few stories of some of the data and camera catastrophes that we’ve experienced over the years. We hope this serves as a warning to those who might be considering putting their gear through some of the experiences above and hopefully sway them against it. If you have some of your own stories on data or gear catastrophes, feel free to share them below in the comments.

— Zach Sutton, Editor-in-chief, Lensrentals.com

The post Stories of Camera and Data Catastrophes appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Securely Managing Your Digital Media (SD, CF, SSD, and Beyond)

Post Syndicated from Roderick Bauer original https://www.backblaze.com/blog/securely-managing-your-digital-media-sd-cf-ssd-and-beyond/

3 rows of 3 memory cards

This is the second in our post exchange series with our friends Zach Sutton and Ryan Hill at Lensrentals.com, who have an online site for renting photography, videography, and lighting equipment. You can read our post from last month on their blog, 3-2-1 Backup Best Practices using Cloud Archiving, and all posts on our blog in this series at Lensrentals post series.

— Editor

Managing digital media securely is crucial for all photographers and videographers. At Lensrentals.com, we take media security very seriously, with dozens of rented memory cards, hard drives, and other data devices returned to our facility every day. All of our media is inspected with each and every rental customer. Most of the cards returned to us in rental shipments are not properly reformatted and erased, so it’s part of our usual service to clear all the data from returned media to keep each client’s identity and digital property secure.

We’ve gotten pretty good at the routine of managing data and formatting storage devices for our clients while making sure our media has a long life and remains free from corruption. Before we get too involved in our process of securing digital media, we should first talk fundamentals.

The Difference Between Erasing and Reformatting Digital Media

When you insert a card in the camera, you’re likely given two options, either erase the card or format the card. There is an important distinction between the two. Erasing images from a card does just that — erases them. That’s it. It designates the area the prior data occupied on the card as available to write over and confirms to you that the data has been removed.

The term erase is a bit misleading here. The underlying data, the 1’s and 0’s that are recorded on the media, are still there. What really happens is that the drive’s address table is changed to show that the space the previous file occupied is available for new data.

This is the reason that simply erasing a file does not securely remove it. Data recovery software can be used to recover that old data as long as it hasn’t been overwritten with new data.

Formatting goes further. When you format a drive or memory card, all of the files are erased (even files you’re designated as “protected”) and also usually adds a file system. This is a more effective method for removing all the data on the drive since all the space previously divided up for specific files has a brand new structure unencumbered by whatever size files were previously stored. Be beware, however, that it’s possible to retrieve older data even after a format. Whether that can happen depends on the formatting method and whether new data has overwritten what was previously stored.

To make sure that the older data cannot be recovered, a secure erase goes further. Rather than simply designating the data that can be overwritten with new data, a secure erase writes a random selection of 1s and 0s to the disk to make sure the old data is no longer available. This takes longer and is more taxing on the card because data is being overwritten rather than simply removed.

Always Format a Card for the Camera You’re Going to Be Using

If you’ve ever tried to use the same memory card on cameras of different makes without formatting it, you may have seen problems with how the data files are displayed. Each camera system handles its file structure a little differently.

For this reason it’s advisable to format the card for the specific camera you’re using. If this is not done, there is a risk of corrupting data on the card.

Our Process For Securing Data

Our inspection process for recording media varies a little depending on what kind of card we’re inspecting. For standardized media like SD cards or compact flash cards, we simply use a card reader to format the card to exFAT. This is done in Disk Utility on the Apple Macbooks that we issue to each of our Video Technicians. We use exFAT specifically because it’s recognizable by just about every device. Since these cards are used in a wide variety of different cameras, recorders, and accessories, and we have no way of knowing at the point of inspection what device they’ll be used with, we have to choose a format that will allow any camera to recognize the card. While our customer may still have to format a card in a camera for file structure purposes, the card will at least always come formatted in a way that the camera can recognize.

Sony SxS media
For proprietary media — things like REDMAGs, SxS, and other cards that we know will only be used in a particular camera — we use cameras to do the formatting. While the exFAT system would technically work, a camera-specific erase and format process saves the customer a step and allows us to more regularly double-check the media ports on our cameras. In fact, we actually format these cards twice at inspection. First, the Technician erases the card to clear out any customer footage that may have been left on it. Next, they record a new clip to the card, around 30 seconds, just to make sure everything is working as it’s supposed to. Finally, they format the card again, erasing the test footage before sending it to the shelf where it awaits use by another customer.

REDMAG Red Mini-Mag You’ll notice that at no point in this process do we do a full secure erase. This is both to save time and to prevent unnecessary wear and tear on the cards. About 75% of the media we get back from orders still has footage on it, so we don’t get the impression that many of our customers are overly concerned with keeping their footage private once they’re done shooting. However, if you are one of those 25% that may have a personal or professional interest in keeping your footage secure after shooting, we’d recommend that you securely erase the media before returning rented memory cards and drives. Or, if you’d rather we handle it, just send an email or note with your return order requesting that we perform a secure erase rather than simply formatting the cards, and we’ll be happy to oblige.

Managing your digital media securely can be easy if done right. Data management and backing up files, on the other hand, can be more involved and require more planning. If you have any questions on that topic, be sure to check out our recent blog post on proper data backup.

— Zach Sutton and Ryan Hill, lensrentals.com

The post Securely Managing Your Digital Media (SD, CF, SSD, and Beyond) appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Protecting Your Data From Camera to Archive

Post Syndicated from Roderick Bauer original https://www.backblaze.com/blog/protecting-your-data-from-camera-to-archive/

Camera data getting backed up to Backblaze B2 cloud

Lensrentals.com is a highly respected company that rents photography and videography equipment. We’re a fan of their blog and asked Zach Sutton and Ryan Hill of Lensrentals to contribute something for our audience. We also contributed a post to their blog that was posted today: 3-2-1 Backup Best Practices using Cloud Archiving.

Enjoy!

— Editor

At Lensrentals.com we get a number of support calls, but unfortunately one of them is among the most common: data catastrophes.

The first of the frequent calls is from someone who thought they transferred over their footage or photos before returning their rental and discovered later that they were missing some images or footage. If we haven’t already gone through an inspection of those cards, it’s usually not a problem to send the cards back to them so they can collect their data. But if our techs have inspected the memory cards, then there isn’t much we can do. Our team at Lensrentals.com perform a full and secure reformatting of the cards to keep each customer’s data safe from the next renter. Once that footage is gone, it is unrecoverable and gone forever. This is never a fun conversation to have.

The second scenario is when a customer calls to tell us that they did manage to transfer all the footage over, but one or more of the clips or images were corrupted in the transferring process. Typically, people don’t discover this until after they’ve sent back the memory cards, and after we’ve already formatted the original media. This is another tough phone call to have. On occasion, data corruption happens in camera, but more often than not, the file gets corrupted during the transfer from the media to the computer or hard drive.

These kinds of problems aren’t entirely avoidable and are inherent risks users take when working with digital media. However, as with all risks, you can take proper steps to assure that your data is safe. If a problem arises, there are techniques you can use to work around it.

We’ve summarized our best suggestions for protecting your data from camera to archive in the following sections. We hope you find them useful.

How to Protect Your Digital Assets

Before Your Shoot

The first and most obvious step to take to assure your data is safe is to make sure you use reliable media. For us, we recommend using cards from brands you trust, such as Sandisk, Lexar or ProGrade Digital (a company that took the reins from Lexar). For hard drives, SanDisk, Samsung, Western Digital, and Intel are all considered incredibly reliable. These brands may be more expensive than bargain brands but have been proven time and time again to be more reliable. The few extra dollars spent on reliable media will potentially save you thousands in the long run and will assure that your data is safe and free of corruption.

One of the most important things you should do before any shoot is format your memory card in the camera. Formatting in camera is a great way to minimize file corruption as it keeps the card’s file structure conforming to that camera manufacturer’s specifications, and it should be done every time before every shoot. Equally important, if the camera gives you an option to do a complete or secure format, take that option over the other low-level formatting options available. In the same vein, it’s essential to also take the time to research and see if your camera needs to unmount or “eject” the media before removing it physically. While this option applies more for video camera recording systems, like those found on the RED camera platform and the Odyssey 7Q, it’s always worth checking into to avoid any corruption of the data. More often than not, preventable data corruption happens when the users turn off the camera system before the media has been unmounted.

Finally, if you’re shooting for the entire day, you’ll want to make sure you have enough media on hand for the entire day, so that you do not need to back up and reformat cards throughout the shoot. While it’s possible to take footage off of the card, reformat it, and use it again for the same day, that is not something you’d want to be doing during the hectic environment of a shoot day — it’s best to have extra media on hand. We’ve all made a mistake and deleted a file we didn’t mean to, so it’s best to avoid that mistake by not having to delete or manage files while shooting. Play it safe, and only reformat when you have the time and clear head to do so.

During Your Shoot

On many modern camera systems, you have the option of dual-recording using two different card slots. If your camera offers this option, we cannot recommend it enough. Doubling the media you’re recording onto can overcome a failure in one of the memory cards. While the added cost may be a hard sell, it’s negligible when compared to all the money spent on lights, cameras, actors and lousy pizza for the day. Additionally, develop a system that works for you and keeps everything as organized as possible. Spent media shouldn’t be in the same location as unused media, and your file structure should be consistent throughout the entire shoot. A proper file structure not only saves time but assures that none of the footage goes missing after the shoot, lost in some random folder.

Camera memory cards

Among one of the most critical jobs while on set is the work of a DIT (Digital Imaging Technician) for video, and a DT (Digital Technician) for photography. Essentially, the responsibilities of these positions are to keep the data archived and organized on a set, as well as metadata logging and other technical tasks involved in keeping a shoot organized. While it may not be cost effective to have a DIT/DT on every shoot, if the budget allows for it, I highly recommend you hire one to take on the responsibilities. Having someone on set who is solely responsible for safely backing up and organizing footage helps keep the rest of the crew focused on their obligations to assure nothing goes wrong. When they’re not transferring and archiving data, DIT/DT’s also log metadata, color correct footage and help with the other preliminary editing processes. Even if the budget doesn’t allow for this position to be filled, work to find someone who can solely handle these processes while on set. You don’t want your camera operator to be in charge of also backing up and organizing footage if you can help it.

Ingest Software

If there is one piece of information we’d like for videographers and photographers to take away from this article, it is this: file-moving or ‘offloading’ software is worth the investment and should be used every time you shoot anything. For those who are unfamiliar with offload software, it’s any application that is designed to make it easier for you to back up footage from one location to another, and one shoot to another. In short, to avoid accidents or data corruption, it’s always best to have your media on a MINIMUM of two different devices. The easiest way to do this is to simply dump media onto two separate hard drives, and keep those drives separately stored. Ideally (if the budget allows), you’ll also keep all of your data on the original media for the day as well, making sure you have multiple copies stored in various locations. Many other options are available and recommended if possible, such as RAID arrays or even copying the data over to a cloud service such as Backblaze B2. What offloading software does is just this process, and helps build a platform of automation while verifying all the data as it’s transferred.

There are a few different recommendations I give for offloading software, all at different price points and with unique features. At the highest end of video production, you’ll often see DITs using a piece of software called Silverstack, which offers color grading functionalities, LTO tape support, and basic editing tools for creating daily edits. At a $600 annual price, it is the most expensive in this field and is probably overkill for most users. As for my recommendation, I recommend a tool call Shotput Pro. At $129, Shotput Pro offers all the tools you’d need to build a great archiving process while sacrificing some of the color editing tools. Shotput Pro can simultaneously copy and transfer files to multiple locations, build PDF reports, and verify all transfers. If you’re looking for something even cheaper, there are additional options such as Offload and Hedge. They’re both available for $99 each and give all the tools you’d need within their simple interfaces.

When it comes to photo, the two most obvious choices are Adobe Lightroom and Capture One Pro. While both tools are known more for their editing tools, they also have a lot of archiving functions built into their ingest systems, allowing you to unload cards to multiple locations and make copies on the fly.

workstation with video camera and RAID NAS

When it comes to video, the most crucial feature all of the apps should have is an option called “checksum verification.” This subject can get complicated, but all you really need to know is that larger files are more likely to be corrupted when transferring and copying, so what checksum verification does is verify the file to assure that it’s identical to the original version down to the individual byte. It is by far the most reliable and effective way to ensure that entire volumes of data are copied without corruption or loss of data. Whichever application you choose, make sure checksum verification is an available feature, and part of your workflow every time you’re copying video files. While available on select photo ingesting software, corruption happens less on smaller files and is generally less of an issue. Still, if possible, use it.

Post-Production

Once you’ve completed your shoot and all of your data is safely transfered over to external drives, it’s time to look at how you can store your information long term. Different people approach archiving in different ways because none of us will have an identical workflow. There is no correct way to handle how to archive your photos and videos, but there are a few rules that you’ll want to implement.

The first rule is the most obvious. You’ll want to make sure your media is stored on multiple drives. That way, if one of your drives dies on you, you still have a backup version of the work ready to go. The second rule of thumb is that you’ll want to store these backups in different locations. This can be extremely important if there is a fire in your office, or you’re a victim of a robbery. The most obvious way to do this is to back up or archive into a cloud service such as Backblaze B2. In my production experience I’ve seen multiple production houses implement a system where they store their backup hard drives in a safety deposit box at their bank. The final rule of thumb is especially important when you’re working with significant amounts of data, and that is to keep a working drive separate from an archive drive. The reason for this is an obvious one: all hard drives have a life expectancy, and you can prolong that by minimizing drive use. Having a working drive separate from your archive drives means that your archive drives will have fewer hours on them, thereby extending their practical life.

Ryan Hill’s Workflow

To help visualize what we discussed above, I’ll lay out my personal workflow for you. Please keep in mind that I’m mainly a one-man band, so my workflow is based on me handling everything. I’m also working with a large variety of mediums, so nothing I’m doing is going to be video and camera specific as all of my video projects, photo projects, and graphic projects are organized in the same way. I won’t bore you with details on my file structure, except to say that everything in my root folder is organized by job number, followed by sub-folders with the data classified into categories. I will keep track of which jobs are which, and have a Google Spreadsheet that organizes the job numbers with descriptions and client information. All of this information is secured within my Google account but also allows me to access it from anywhere if needed.

With archiving, my system is pretty simple. I’ve got a 4-drive RAID array in my office that gets updated every time I’m working on a new project. The array is set to RAID 1+0, which means I could lose two of the four hard drives, and still be able to recover the data. Usually, I’ll put 1TB drives in each bay, fill them as I work on projects, and replace them when they’re full. Once they’re full, I label them with the corresponding job numbers and store them in a plastic case on my bookshelf. By no means am I suggesting that my system is a perfect system, but for me, it’s incredibly adaptable to the various projects I work on. In case I was to get robbed, or if my house caught fire, I still have all of my work also archived onto a cloud system, giving me a second level of security.

Finally, to finish up my backup solution, I also keep a two-bay Thunderbolt hard drive dock on my desk as my working drive system. Solid state drives (SSD) and the Thunderbolt connection give me the speed and reliability that I’d need from a drive that I’ll be working from, and rendering outputs off of. For now, there is a single 960gb SSD in the first bay, with the option to extend to the second bay if I need additional storage. I start work by transferring the job file from my archive to the working drive, do whatever I need to do to the files, then replace the old job folder on my archive with the updated one at the end of the day. This way, if I were to have a drive failure, the worst I will lose is a day’s worth of work. For video projects or anything that takes a lot of data, I usually keep copies of all my source files on both my working and archive drive, and just replace the Adobe Premiere project file as I go. Again, this is just my system that works for me, and I recommend you develop one that works for your workflow while keeping your data safe.

The Takeaway

The critical point you should take away is that these sorts of strategies are things you should be thinking about at every step of your production. How does your camera or codec choice affect your media needs? How are you going to ensure safe data backup in the field? How are you going to work with all of this footage in post-production in a way that’s both secure and efficient? Answering all of these questions ahead of time will keep your media safe and your clients happy.

— Zach Sutton and Ryan Hill, lensrentals.com

The post Protecting Your Data From Camera to Archive appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.