Tag Archives: LTO

Breaking the Cycle of Archive Migrations With B2 Cloud Storage

Post Syndicated from Janet Lafleur original https://www.backblaze.com/blog/cloud-data-archiving/

Assorted tapes

Back in the 1980s, my family and I took a trip to visit our friends, the Bremers. We all used to live next door, but the Bremers had moved away a decade prior. As our parents were reminiscing on old times, one of the Bremer teens pulled out a 8mm movie projector and we watched home movies his dad had shot of us playing together in the backyard: on the swings, the see-saw, and running about. What I wouldn’t give to see that footage today! It would be the only video of my sisters and me as kids.

Perhaps Mr. Bremer digitized his home movie collection before he passed away. But it’s more likely his children inherited the box of reels, and it’s now buried and decaying in a closet (or gone entirely). And, if they had the tape, would they have a projector or anything to play it? What a pity. Those precious moments captured once upon a time on film are probably lost forever.

Obsolescence isn’t just a concern for home video enthusiasts. Professional content creators likely have content stored on obsolete technology, whether it’s videotape, LTO digital tape, or external drives. And unlike the simplicity of Mr. Brehmer’s film reels and projectors, there are many more factors that can make digital content inaccessible.

Common Causes of Data Obsolescence

Media Failure

The most obvious issue is storage media degradation. If film is carefully stored in a cold, dry environment, it can last an extremely long time. Yet for both videotape and digital tape, there are a myriad of pitfalls: magnetic particles can lose their charge; the tape substrate can deteriorate; and heavily used tapes can stretch. Tapes over 15 years old are at greatest risk, even if stored in the ideal conditions of low-heat and low-humidity.

Hard disk drives have shortfalls too: mechanical failure, overheating, and power spikes. External drives in particular, are at risk of shock damage from being dropped. Even a drive standing on its side, then tipping over, can generate enough shock to damage the drive internals. At our Backblaze data centers, we replace disk drives after four years, and earlier for drive models that show higher-than-usual failure rates. We have ~100,000 drives in our data centers, and document which ones are more likely to fail in our quarterly drive stats posts.

Obsolete Technology

Even if the storage media remains intact and the data uncorrupted, the data format can become obsolete, often more quickly than you’d expect. For example, manufacturers of the commonly used LTO digital tape are now shipping LTO-8 and only guarantee two generations of backward compatibility. That means if you upgrade your tape system for higher-capacity 12TB LTO-8 tapes, you won’t be able to read the LTO-6 tapes that were introduced just six years ago.

Also, if the file data itself was encoded in a proprietary format, you’ll likely need proprietary software installed on a computer running a potentially outdated operating system version to be able to read its data. This is a bigger topic than we’ll cover today, because there can be layers of encoding involved: backup formats, graphics formats, codecs, etc. But suffice to say that you might find yourself having to hunt down a Mac that’s still running macOS X Leopard to migrate some content.

Museum of Obsolete Media

Not sure how much your content is at risk? The Museum of Obsolete Media rates all imaginable media types on both media stability and obsolescence, from Endangered to In Use.

Spoiler alert:  VHS tapes are rated Endangered for media stability and rated Vulnerable for obsolescence.

Migrate…Then Migrate Again

The only way to combat this sort of media decay and obsolescence and maintain access to your content is to migrate it to newer media and/or a newer technology. This unglamorous task sounds simple — read the data off the old media and copy it to new media — but the devil is in the details. Here is a checklist for trying to maintain your physical media:

The Eight Steps of Data Migration

  1. Determine which content is obsolete or at risk. Choose a media and format for the new archive, and calculate whether you can afford to migrate everything. If not, decide what you can afford to lose forever.
  2. Gather all the tapes or drives to be migrated. Are you sure you have the complete set? Your content spreadsheet might not be up to date. You might need to interview team members to gather any unwritten tribal knowledge about the backup sets.
  3. Identify a migration workstation or server that can run the application that wrote the archived media files. Attach the tape drive or disk device and test it. Can it still properly read, write, and then restore test files?
  4. Using a checklist system, feed tapes into the drive or attach the external drive in order. You might need to track down obscure adapters for older technologies like a SATA to EIDE adapter for parallel port disk drives, or a SCSI card and cables.
  5. Initiate the copy of all files to local storage. Hope you have enough space.
  6. Carefully monitor the entire process and make sure that all files are copied completely, and only then can you check the tape or disk off of your migration list. Then repeat with the next tape or disk.
  7. When you’re done extracting all the old files (or earlier if you’re pinched for disk space), reverse the process. Attach any needed devices and write the files to the new media. Cross your fingers that you bought enough tapes or disk drives (but not too many).
  8. Repeat again in 4-7 years before the new media ages or technologies change.

If all of that sounds too painful, you can pay a transfer service to migrate your whole archive for you, but that’s not cheap, and remember you’ll have to pay to do it again sooner than you think. Alternatively, you can migrate content on-demand and cross your fingers that it’s still readable and that you can retrieve it fast enough. The longer you wait, the greater the risk of media failure. You might only get one shot at reading an old tape or film. Few find that an acceptable risk.

Why Data Archiving to the Cloud Is a Better Solution

Migrate Once with Backblaze B2 Cloud Storage

You can break this migration cycle by migrating once to Backblaze B2 Cloud Storage. We’ll take over from there, moving your data to newer storage technologies as needed over time. Backblaze’s erasure coding technology that protects your data from loss happens to make upgrading technologies easier for us. Not that you need to worry about it; it’s included in our service.

No New Media or Hardware

Moving to B2 Cloud Storage for your archive means you won’t have any hardware or media to purchase, manage, or house. No tapes or disks to buy, no clearing off shelf space as your archive grows. You won’t have to feed tapes into an autoloader every time you want to write or retrieve content from the archive. And moving to B2 Cloud Storage gives you the benefit of only paying for what you’re actually using. Pay-as-you-go means your storage costs move from a capital expense to an operating expense.

B2 is Less Expensive than LTO

Did you know that Backblaze B2 is the first cloud storage that’s more affordable than LTO storage solutions? If you want to see the math, check out our LTO vs B2 calculator. Enter the size of your existing archive and how much you expect to add each year and it will show you cost differences after 1-10 years. To understand its cost and operational assumptions, read our recent blog post, LTO Versus Cloud Storage Costs — the Math Revealed. It details the many factors for storage costs that many media professionals don’t always consider.

Data That’s Always Accessible

The only thing worse than having a tape or disk you can’t read is having one that you can read go missing in action. Your content database or spreadsheet is only as accurate as what’s on the shelf. You may believe that an external drive is still in your archive closet when it went home over the weekend with a staff member and never came back. With B2 Cloud Storage, your archived content is stored in a central location that’s not only always accessible, it’s accessible from anywhere through a web browser.

B2 is Proven Technology

With Backblaze, you get a partner with over a decade of cloud storage experience. The erasure coding we use to encode data gives B2 customers a 99.999999999% durability (11 nines) rating for their data stored in our cloud. As NASA says, there’s higher probability of an asteroid destroying the planet than you losing a file with B2.

Make Your Final Migration Painless and Smart

Of course, you’ll still have to migrate once, but we can help make that final migration as painless and smart as possible. B2 Cloud Storage has several options for moving dataAPIs, Web UI, CLIplus our Fireball rapid ingest service for large data sets. We’ve also partnered with vendors and system integrators who have deep experience in managing media archives.

Streamlined LTO Migration

If your current archive is on LTO tapes, we have a newly announced partnership with StorageDNA that can speed migration of LTFS archives. The Storage DNA Smart Migration bundle combines the latest version of their DNAfabric storage with Backblaze B2 cloud storage, plus an autoloading LTO library so you won’t waste time manually loading tapes. To learn more about how it works, register for our upcoming webinar, From LTO to the Cloud: Your Last Data Migration with Backblaze and StorageDNA, on Friday, December 14.

Organize Content with a MAM

Archive migrations are a great time to evaluate your asset management strategy. If you haven’t rolled out a media asset manager (MAM) yet, or you’re dissatisfied with your current one, know that more and more MAMs are integrated with cloud storage and can simplify collaboration across remote teams. With a cloud-integrated MAM solution, your content can be easily searched, filtered, sorted and previewed all from a web browser, from anywhere. To see B2 in action with a cloud MAM solution, watch our recent webinar, Three Steps to Making Your Cloud Media Archive Active with iconik and Backblaze B2.

Automated Backup and Archive

Finally, B2 isn’t just an archive solution, it’s great for backup, too. Most of our customers who archive content to B2 also back up active production data to the same B2 account. We have a growing list of backup, sync and other tools integrated with B2 to make the data movement to the cloud seamless and to make retrieval intuitive and straightforward.

Pro Tip: syncing newly ingested footage or assets to B2 will spare you a big headache when someone accidentally deletes a critical file.

If you have content that’s on media or in a format that’s aging fast, now’s the time to plan for its migration. By migrating it to B2 Cloud Storage, you can not only make it your last migration, it’s priced so that you can afford to migrate ALL your content. You never know what you’ll need, or when you’ll need it. And some content, like Mr. Bremer’s home movies, simply can’t be re-created.

The post Breaking the Cycle of Archive Migrations With B2 Cloud Storage appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Moving Tape Content to Backblaze Fireball with Canister

Post Syndicated from Skip Levens original https://www.backblaze.com/blog/moving-tape-content-to-cloud-storage/

Canister for Fireball: LTO tape to Backblaze B2 migration made 'drag and drop' easy
If you shoot video on the run and wrangle video from multiple sources, you know that reliably offloading files from your camera carts, storage cards, or pluggable SSDs can be a logistical challenge. All of your source files need to be copied over, verified, and backed up before you can begin the rest of your post-production work.  It’s arguably the most critical step in your post-production workflow.

Knowing how critical this step is, videographers and data wranglers alike have long relied on an app for Mac and Windows called Hedge to take charge of their file copy and verification needs.

Hedge source and target progress

Hedge for Mac and Windows — drag and drop source file copy and verify tool

With an intuitive drag and drop interface, Hedge makes it simple to select your cards, disks, or other sources, identify your destination drives, then copy and verify using a custom “Fast Lane” engine to speed transfers dramatically. You can log when copies were completed, and even back up to multiple destinations in the same action, including your local SAN, NAS, or Backblaze Fireball, then on to your Backblaze B2 cloud storage.

But How Do You “Data-Wrangle” Tape Content to the Cloud?

But what if you have content, backup sets, or massive media archives on LTO tape?

You may find yourself in one of these scenarios:

  • You may have “inherited” an older LTO tape system that is having a hard time keeping up with your daily workflow, and you aren’t ready to sign up for more capital expense and support contracts.
  • You may have valuable content “stuck” on tape that you can’t easily access and want it on cloud for content monetization workflows that would overwhelm your tape system.
  • Your existing tape based workflow is working fine for now, but you want to get all of that content into the cloud quickly to get ready for future growth and new customers with a solution similar to Hedge.

While many people decide to move tape workflows to cloud for simple economic reasons, having all of that content securely stored in the cloud means that the individual files and entire folders can be instantly pulled into workflows and directly shared from Backblaze B2 with no need for copying, moving, restoring, or waiting.

For more information about how Backblaze B2 can replace LTO solutions, including an LTO calculator:  Backblaze LTO Replacement Calculator

Whichever scenario fits your need, getting tape content into the cloud involves moving a lot of content at once, and in a perfect world it would be as easy to drag and drop that content from tape to Backblaze B2!

Meet Canister for Fireball

To meet this exact need the team that developed Hedge have created an “LTO tape content to Fireball” solution called Canister for Fireball.

Fireball is Backblaze’s solution to help you quickly get massive amounts of data into Backblaze B2 Cloud Storage. When you sign up for the service, Backblaze sends you a 70TB Fireball that is yours to use for 30 days. Simply attach it to your local network and copy content over to the device at the speed of your local network. You’re free to fill up and send in your Fireball device as many times as needed. When Backblaze receives your Fireball with your files, all of the content is ingested directly into Backblaze’s data centers and appears in your Backblaze B2 online storage.

Backblaze B2 Fireball Rapid Ingest Service

Canister for Fireball makes it incredibly easy to move your content and archives from your tape device to your Backblaze B2 Fireball. With an intuitive interface similar to Hedge, Canister copies over and verifies files read from your tapes.

Using Canister with B2

flow chart for moving data from tape to the cloudInsert LTO tapes in your tape system and Canister for Backblaze will move them to your Backblaze B2 Fireball for rapid ingest into your B2 Cloud Storage

Cannister to Fireball user interfaceSelect from any tape devices with LTO media…

Cannister data progression screenshot…and watch the files on the tape copy and verify to your Backblaze B2 Fireball

Here’s how the solution works:

Steps to Migrate Your LTO Content to the Cloud with Canister for Fireball

  1. Order a Fireball system: As part of the signup step you will choose a B2 bucket that you’d like your Fireball content moved to.
  2. Connect your Fireball system to your network, making sure that the workstation that connects to your tape device can also mount the storage volume presented by your Backblaze Fireball.
  3. Install Canister for Fireball on your Mac workstation.
  4. Connect your tape device. Any tape system that can read your tapes and mount them as an LTFS volume will work. Canister will automatically mount tapes inside the app for you.
  5. Launch Canister for Fireball. You can now select the tape device volume as your source, the Fireball as your target, and copy the files over to your Fireball.
  6. Repeat as needed until you have copied and verified all of your tapes securely to your Fireball. You can fill and send in your Fireball as many times as needed during your 30 day period. (And you can always extend your loaner period.)
LTFS or Linear Tape File System is an industry adopted way to make the contents of an entire tape cartridge available as if it were a single volume of files. Typically, the tape stores a list of the files and their location on that tape in the beginning, or header of the tape. When a tape is read into your tape device, that directory section is read in and the tape system then presents it to you as a volume of files and folders. Say you want to select an individual file from that LTFS volume to copy to your desktop. When you move that to your desktop, the tape spools out to wherever that file is stored, reads the entire stream of tape containing that file, then finally copies it to your desktop. It can be a very slow process indeed and why many people choose to store content in cloud storage like Backblaze B2 so that they get instant access to every file.

Now — Put Your LTO Tape Ingest Plan Into Action

If you have content on tape that needs to get into your Backblaze B2 storage, Canister for Fireball and a Backblaze B2 Fireball are the perfect solution.

Canister for Fireball can be licensed for 30 days of use for $99 and includes priority support. The full version is $199. If you decide to upgrade from the 30 Day license you’ll pay only the difference to the full version.

Get more information about Canister for Fireball

And of course, make sure that you’ve ordered your Fireball:

Order a Backblaze B2 Fireball

Now with your content and archives no longer “trapped” on tape, you can browse them in your asset manager, share links directly from Backblaze B2, and have your content ready to pull into new content creation workflows by your team located anywhere in the world.

The post Moving Tape Content to Backblaze Fireball with Canister appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

LTO versus Cloud Storage: Choosing the Model That Fits Your Business

Post Syndicated from Andy Klein original https://www.backblaze.com/blog/lto-vs-cloud-storage-vs-hybrid/

Choose Your Solution: Cloud Storage, LTO, Hybrid Cloud Storage/LTO

Years ago, when I did systems administration for a small company, we used RAID 1 for in-house data redundancy and an LTO tape setup for offsite data backup. Yes, the LTO cataloging and versioning were a pain, so was managing the tapes, and sometimes a tape would be unreadable, but the setup worked. And given there were few affordable alternatives out there at the time you lived and died with your tapes.

Over the last few years, cloud storage has emerged as a viable alternative to using LTO for offsite backups. Improvements in network speed coupled with lower costs are a couple of the factors that have changed the calculus of cloud storage. To see if enough has changed to make cloud storage a viable competitor to LTO, we’ll start by comparing the current and ongoing cost of LTO versus cloud storage and then dig into assumptions underlying the cost model. We’ll finish up by reviewing the pluses and minuses of three potential outcomes: switching to cloud storage, staying with LTO, or using a hybrid LTO/cloud storage solution.

Comparing the Cost of LTO Versus Cloud Storage

Cost calculators for comparing LTO to Cloud Storage have a tendency to be very simple or very complex. The simple ones generally compare hardware and tape costs to cloud storage costs and neglect things like personnel costs, maintenance costs, and so on. In the complex models you might see references to the cost of capital, interest on leasing equipment, depreciation, and the tax implications of buying equipment versus paying for a monthly subscription service.

The Backblaze LTO vs Cloud calculator is somewhere in between. The underlying model takes into account many factors, which we’ll get into in a moment, but if you are a Fortune 500 company with a warehouse full of tape robots, this model is not for you.

Calculator: LTO vs B2

To use the Backblaze calculator you enter:

  1. the amount of Existing Data you have on LTO tape
  2. the amount of data you expect to add in a given year
  3. the amount of incremental data you backup each day

Then you can use the slider to compare your total cost from 1 to 10 years. You can run the model as many times as you like under different scenarios.

Assumptions Behind the Model

To see the assumptions that were made in creating the model, start on the LTO Replacement page and scroll down past the LTO vs. B2 calculator. Click on the following text which will display the “Cost and Operational Assumptions” page.

+ See details on Cost and Operational Assumptions

Let’s take a few minutes to review some of the most relevant points and how they affect the cost numbers reported:

  • LTO Backup Model: We used the Grandfather-Father-Son (GFS) model. There are several others, but this was the most prevalent. If you use the “Tower of Hanoi” model for example, it uses fewer tapes and would lower the cost of the total LTO cost by some amount.
  • Data Compression: We assumed a 2-1 compression ratio for the data stored on the LTO tapes. If your data is principally video or photos, you will most likely not use compression. As such, film studios and post-production houses will need to double the cost of the total LTO solution to compensate for the increased number of tapes, the increased number of LTO tape units, and increased personnel costs.
  • Data Retention: We used a 30 day retention period as this is common in the GFS model. If you keep your incremental tapes/data for 2 weeks, then you would lower the number of tapes needed for incremental backups, but you would also lower the amount of incremental data you keep in the cloud storage system.
  • Tape Units: There are a wide variety of LTO tape systems. You can increase or decrease the total LTO cost based on the systems you are using. For example, you are considering the purchase of an LTO tape system which reads/writes up to 5 tapes simultaneously. That system is more expensive and has higher maintenance costs, but it also would mean you would have to purchase fewer tape units.
  • LTO-8 Tape Units: We used LTO-8 tape units as they are the currently available LTO system most likely to be around in 10 years.
  • Tape Migration: We made no provision for migration from an unsupported LTO version to a supported LTO version. During the next 10 years, many users with older LTO systems will find it likely they will have to migrate to newer systems as LTO only supports 2 generations back and is currently offering a new generation every 2 years.
  • Pickup Cost: The cost of having your tapes picked up so they are offsite. This cost can vary widely based on geography and service level. Our assumption of the cost is $60 per week or $3,120/year. You can adjust the LTO total cost according to your particular circumstances.
  • Network Cost: Using cloud storage requires that you have a reasonable amount of network bandwidth available. The number we used is incremental to your existing monthly cost for bandwidth. Network costs vary widely, so depending on your circumstance you can increase or decrease to the total cost of the cloud storage solution.
  • Personnel Cost: This is the total cost of what you are paying someone to manage and operate your LTO system. This raises or lowers the cost of both the LTO and cloud storage solutions at the same rate, so adjusting this number doesn’t affect the comparison, just the total values for each.
  • Time Savings Versus LTO: With a cloud storage solution, there are no tapes or tape machines to deal with. This saves a significant amount of time for the person managing the backup process. Increasing this value will increase the cost of the cloud storage solution relative to the LTO solution.

As hinted at earlier, we don’t consider the cost of capital, depreciation, etc. in our calculations. The general model is that a company purchases a number of LTO systems and the cost is spread over a 10 year period. After 10 years a replacement unit is purchased. Other items such as tapes and equipment maintenance are purchased and expensed as needed.

Choosing a Data Backup Model

We noted earlier the three potential outcomes when evaluating LTO versus cloud storage for data backup: switching to cloud storage, staying with LTO, or using a hybrid LTO/cloud storage solution. Here’s a look at each.

Switching to Cloud Storage

After using the calculator you find cloud storage is less expensive for your business or organization versus LTO. You don’t have a large amount of existing data, 100 terabytes for example, and you’d rather get out of the tape business entirely.

Your first challenge is to move your existing data to the cloud — quickly. One solution is the Backblaze B2 Fireball data transfer service. You can move up to 70 TB of data each trip from your location to Backblaze in days. This saves your bandwidth and saves time as well.

As the existing data is being transferred to Backblaze, you’ll want to select a product or service to move your daily generated information to the cloud on a regular basis. Backblaze has a number of integration partners that perform data backup services to Backblaze B2

Staying with LTO

After using the calculator you find cloud storage is less expensive, but you are one of those unlucky companies that can’t get reasonably priced bandwidth in their area. Or perhaps, the new LTO-8 equipment you ordered arrived minutes before you read this blog post. Regardless, you are destined to use LTO for at least a while longer. Tried and true, LTO does work and has the added benefit of making the person who manages the LTO setup nearly indispensable. Still, when you are ready, you can look at moving to the hybrid model described next.

Hybrid LTO/Cloud Storage model

In practice, many organizations that use LTO for backup and archive often store some data in the cloud as well, even if haphazardly. For our purposes, Hybrid LTO/Cloud Storage is defined as one of the following:

  1. Date Hybrid: All backups and archives from prior to the cut over date remain stored in LTO; everything after the cut over date date forward is stored in cloud storage.
  2. Classic Hybrid: All of the incremental backups are stored in cloud storage and all full backups and archives are stored on LTO.
  3. Type Hybrid: All data of a given type, say employee data, is stored on LTO, while all customer data is stored in cloud storage. We see this hybrid use case occur as a function of convenience and occasionally compliance, although some regulatory requirements such as GDPR may not be accommodated by LTO solutions.

You can imagine there being other splits, but in essence, there may be situations where keeping the legacy system going in some capacity for some period of time is the prudent business option.

If you have a large tape library, it can be almost paralyzing to think about moving to the cloud, even if it is less expensive. Being open to the hybrid LTO/cloud model is a way to break the task down into manageable steps. For example, solutions like Starwind VTL and Archiware P5 allow you to start backing up to the cloud with minimal changes to your existing tape-based backup schemes.

Many companies that start down the hybrid road typically begin with moving their daily incremental files to the cloud. This immediately reduces the amount of “tape work” you have to do each day and it has the added benefit of making the files readily available should they need to be restored. Once a company is satisfied that their cloud based backups for their daily incremental files are under control, they can consider whether or not they need to move the rest of their data to the cloud.

Will Cloud Storage Replace LTO?

At some point, the LTO tapes you have will need to be migrated to something else as the equipment to read your old tapes will become outdated, then unsupported, and finally unavailable. Users with LTO 4 and, to some degree, LTO 5 are already feeling this pain. To migrate all of that data from your existing LTO system to LTO version “X,” cloud storage, or something else, will be a monumental task. It is probably a good idea to start planning for that now.

In summary, many people will find that they can now choose cloud storage over LTO as an affordable way to store their data going forward. But, having a hybrid environment of both LTO and cloud storage is not only possible, it is a practical way to reduce your overall backup cost while maximizing your existing LTO investment. The hybrid model creates an improved operational environment and provides a pathway forward should you decide to move exclusively to storing your data in the cloud at some point in the future.

The post LTO versus Cloud Storage: Choosing the Model That Fits Your Business appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.