Tag Archives: Cloud Storage

Explore the Backblaze S3 Compatible API With Our New Postman Collection

Post Syndicated from Pat Patterson original https://www.backblaze.com/blog/explore-the-backblaze-s3-compatible-api-with-our-new-postman-collection/

Postman is a platform for building and using APIs. API providers such as Backblaze can use Postman to build API documentation and provide a live environment for developers to experiment with those APIs. Today, you can interact with Backblaze B2 Cloud Storage via our new Postman Collection for the Backblaze S3 Compatible API.

Using the Backblaze S3 Compatible API

The Backblaze S3 Compatible API implements the most commonly used S3 operations, allowing applications to integrate with Backblaze B2 in exactly the same way they do with Amazon S3. Many of our Alliance Partners have used the S3 Compatible API in integrating their products and services with Backblaze B2. Often, integration is as simple as allowing the user to specify a custom endpoint, for example, https://s3.us-west-001.backblazeb2.com, alongside their API credentials in the S3 settings, and verifying that the application works as expected with Backblaze B2.

The Backblaze B2 Native API, introduced alongside Backblaze B2 back in 2015, provides a low-level interface to B2 Cloud Storage. We generally recommend that developers use the S3 Compatible API when writing new applications and integrations, as it is supported by a wider range of SDKs and libraries, and many developers already have experience with Amazon S3. You can use the Backblaze B2 web console or the B2 Native API to access functionality, such as application key management and lifecycle rules, that is not covered by the S3 Compatible API.
 
Our post on the B2 Native and S3 Compatible APIs provides a more detailed comparison.

Most applications and scripts use one of the AWS SDKs or the S3 commands in the AWS CLI to access Backblaze B2. All of the SDKs, and the CLI, allow you to override the default Amazon S3 endpoint in favor of Backblaze B2. Sometimes, though, you might want to interact directly with Backblaze B2 via the S3 Compatible API, perhaps in debugging an issue, or just to better understand how the service works.

Exploring the Backblaze S3 Compatible API in Postman

Our new Backblaze S3 Compatible API Documentation page is the definitive reference for developers wishing to access Backblaze B2 directly via the S3 Compatible API.

In addition to reading the documentation, you can click the Run in Postman button on the top right of the page, log in to the Postman website or desktop app (creating a Postman account is free), and interact with the API.

Integrate With Backblaze B2

Whether you are backing up, archiving data, or serving content via the web, Backblaze B2 is an easy to use and, at a quarter of the cost of Amazon S3, cost-effective cloud object storage solution. If you’re not already using Backblaze B2, sign up now and try it out—your first 10GB of storage is free!

The post Explore the Backblaze S3 Compatible API With Our New Postman Collection appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

What Is Private Cloud Storage?

Post Syndicated from Molly Clancy original https://www.backblaze.com/blog/what-is-private-cloud-storage/

No matter what business you’re in, you’re in the data business. Your files are the backbone of your entire operation and losing access to them would be a disaster. As such, it’s perfectly understandable if you’re a little protective of these all-important 1s and 0s. As you begin your search for a data storage solution, understanding some cloud basics can help you make the best decision, and one option that may cross your desk is the private cloud.

What Is the Private Cloud?

A private cloud is essentially storage dedicated solely to your organization, but accessible from anywhere. While you can outsource the physical infrastructure of this private cloud to a data center, you do have the option to keep it on-premises. It is a costlier solution than typical public cloud storage, but the trade-off is a higher level of security and control over your data.

Refresher: What Is the Public Cloud?

By contrast, a public cloud storage service would be open to many different organizations or tenants. In a public cloud, tenants share the same physical hardware resources as other tenants, and their individual instances are virtually separated so that one tenant can’t access another’s data. Public clouds typically don’t involve on-premises hardware as everything is managed by the public cloud provider in a data center.

Public and Private Clouds: By the Numbers

In 2020, spending on cloud services grew significantly: Public cloud spending reached $14 billion, and private cloud spending reached $5 billion, according to an IDC study. IDC also found that “on-premises private clouds” make up nearly two-thirds (i.e., 64%) of private cloud spending.

According to the Flexera 2021 State of the Cloud Report, 97% of respondents used a public cloud and 80% of respondents used a private cloud. As you can see, there is plenty of overlap. Using a public cloud versus a private cloud is not an either/or decision—78% of respondents use both public and private clouds in a hybrid cloud solution.

Advantages and Disadvantages of Private Clouds

Both the private cloud and the public cloud have their advantages. Benefits of the public cloud include low cost and a higher adoption rate than the private cloud. Therefore, a public cloud can be cheaper to run and it can be easier to find employees who know how to use the public cloud. The high adoption rate also means it is easier to find third-party consultants and experts to provide help when you need it. On the other hand, a private cloud offers advantages such as faster access to data when using on-premises infrastructure and more control over security.

Private Cloud: Use Cases

To better understand how private cloud services can be beneficial, let’s take a closer look at use cases. There are a few reasons organizations are investing time and resources into private cloud deployments:

  • Regulatory compliance. Some industries face high demand for security. For example, financial services and health care organizations frequently handle highly sensitive data. As a result, these organizations may find that the greater security safeguards available through a private cloud may be a good fit. Further, Gartner points out that the U.S. government is starting to increase its adoption of private cloud services. If your organization operates in a highly regulated industry with significant IT security or privacy protection requirements, using a private cloud may make sense. Finally, government contractors may be expected to use the private cloud to give their government clients extra peace of mind regarding security.
  • Meeting high-performance requirements. In some situations, a private cloud may provide access to faster IT services. For instance, an on-premises private cloud may serve data to local users much faster than a private cloud located thousands of miles away. A large organization that needs to work with large volumes of data like video files and big data analytics may find private cloud services a good fit. If a company sees a competitive opportunity in speed (e.g., high-frequency trading in financial services), then a private cloud could be an attractive option.
  • Specific hardware requirements. In a public cloud environment, the end user generally has limited input regarding specific hardware. By contrast, a company willing to pay for a private cloud service will have more control over specific hardware investments.

Public vs. Private: Why Not Both?

Using both private and public clouds simultaneously is called a hybrid cloud solution, and it may be the best way to balance your security needs against your budget constraints to optimize your storage infrastructure. You can use an on-premises private cloud for sensitive files or larger files that will be accessed on a daily basis, while maintaining public cloud storage space for archives or files that need to be accessed remotely by outside vendors or clients. The reduced cost of public cloud storage gives you greater flexibility in tweaking your private vs. public cloud needs to fit your budget.

Managing a Private Cloud Effectively

Using private cloud data storage services has the potential to offer greater security. If your cloud portfolio includes the private cloud, use the following tips to get the most from those services.

  • Manage the scalability limitations of the private cloud. In contrast to a public cloud service, you may find it more difficult or time-consuming to scale up and scale down private cloud services. This challenge can be mitigated in a few ways. For instance, you could pursue a hybrid cloud approach using the private cloud for smaller amounts of highly sensitive data and put the bulk of your corporate data in a public cloud.
  • Enhance your cost management process. A private cloud service comes with higher up-front costs and maintenance costs. And, in a public cloud, your costs can vary based on the amount of storage you’re using at any given time. With a private cloud, once you’ve made that capital expenditure, whether on-premises or off, the capacity is already paid for. However, it requires very accurate assessments of future needs to predict any further spending on capacity. If you run out of space, you’re facing another capital investment.
  • Managing maintenance risk. Compared to a public cloud deployment, private cloud storage can be relatively complex to manage. If upkeep slips, you may start to suffer increasing security risks and lost performance.
  • Challenge customization requests. The ability to customize a private cloud is a double-edged sword. The flexibility is attractive for departments with unusual requirements. On the other hand, each private cloud customization request may involve additional fees and security complexities. Therefore, it may be wise to challenge requests for cloud customization and see if business users can achieve their goals with minimal customization.

Are You Using a Private Cloud?

We’re all in the data business, but nobody knows your business like you. Ultimately, the decision to balance data security against financial concerns will come down to your individual needs and budgetary realities. But for those files that require an extra layer of security, whether it’s for regulatory or operational reasons, a private cloud solution gives you control over your data that is worth the added expense.

Are you using a public cloud, a private cloud, or both? Let us know in the comments.

The post What Is Private Cloud Storage? appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Free Image Hosting With Cloudflare Transform Rules and Backblaze B2

Post Syndicated from Pat Patterson original https://www.backblaze.com/blog/free-image-hosting-with-cloudflare-transform-rules-and-backblaze-b2/

Before I dive into using Cloudflare Transform Rules to implement image hosting on Backblaze B2 Cloud Storage, I’d like to take a moment to introduce myself. I’m Pat Patterson, recently hired by Backblaze as chief developer evangelist. I’ve been working with technology and technical communities for close to two decades, at companies such as Sun Microsystems and Salesforce. I’ll be creating and delivering technical content for you, our Backblaze B2 community, and advocating on your behalf within Backblaze. Feel free to follow my journey and reach out to me via Twitter or LinkedIn.

Cloudflare Transform Rules

Now, on with the show! Cloudflare Transform Rules give you access to HTTP traffic at the CDN edge server, allowing you to manipulate the URI path, query string, and HTTP headers of incoming requests and outgoing responses. Where Cloudflare Workers allows you to write JavaScript code that executes in the same environment, Transform Rules give you much of the same power without the semi-colons and curly braces.

Let’s look at a specific use case: implementing image hosting on top of a cloud object store. Backblaze power user James Ross wrote an excellent blog post back in August 2019, long before the introduction of Transform Rules, explaining how to do this with Cloudflare Workers and Backblaze B2. We’ll see how much of James’ solution we can recreate with Transform Rules, without writing any code. We’ll also discover how the combination of Cloudflare and Backblaze allows you to create your own, personal 10GB image hosting site for free.

Implementing Image Hosting on a Cloud Object Store

James’ requirements were simple:

  • Serve image files from a custom domain, such as files.example.com, rather than the cloud storage provider’s domain.
  • Remove the bucket name, and any other extraneous information, from the URL.
  • Remove extraneous headers, such as the object ID, from the HTTP response.
  • Improve caching (both browser and edge cache) for images.
  • Add basic CORS headers to allow embedding of images on external sites.

I’ll work through each of these requirements in this blog post, and wrap up by explaining why Backblaze B2 might be a better long term provider for this and many other cloud object storage use cases than other cloud object stores.

It’s worth noting that nothing here is Backblaze B2-specific—the user’s browser is requesting objects from a B2 Cloud Storage public bucket via their URLs, just as it would with any other cloud object store. The techniques are exactly the same on Amazon S3, for example.

Prerequisites

You’ll need accounts with both Cloudflare and Backblaze. You can get started for free with both:

You’ll also need your own DNS domain, which I’ll call example.com in this article, on which you can create subdomains such as files.example.com. If you’ve read this far, you likely already have at least one. Otherwise, you can register a new domain at Cloudflare for a few dollars a year, or your local equivalent.

Create a Bucket for Your Images

If you already have a B2 Cloud Storage bucket you want to use for your image store, you can skip this section. Note: It doesn’t matter whether you created the bucket and its objects via the B2 Native API, the Backblaze S3 Compatible API, or any other mechanism—your objects are accessible to Cloudflare via their friendly URLs.

Log in to Backblaze, and click Buckets on the left under B2 Cloud Storage, then Create a Bucket. You will need to give your bucket a unique name, and make it public. Leave the other settings with their default values.

Note that the bucket name must be globally unique within Backblaze B2, so you can’t just call it something like “myfiles.” You’ll hide the bucket name from public view, so you can call it literally anything, as long as there isn’t already a Backblaze B2 bucket with that name.

Finally, click Upload/Download and upload a test file to your new bucket.

Click the file to see its details, including its various URLs.

In the next step, you’ll rewrite requests that use your custom subdomain, for example, https://files.example.com/smiley.png, to the friendly URL of the form, https://f004.backblazeb2.com/file/metadaddy-public/smiley.png.

Make a note of the hostname in the friendly URL. As you can see in the previous paragraph, mine is f004.backblazeb2.com.

Create a DNS Subdomain for Your Image Host

You will need to activate your domain (example.com, rather than files.example.com) in your Cloudflare account, if you have not already done so.

Now, in the Cloudflare dashboard, create your subdomain by adding a DNS CNAME record pointing to the bucket hostname you made a note of earlier.

I created files.superpat.com, which points to my bucket’s hostname, f004.backblazeb2.com.

If you test this right now by going to your test file’s URL in your custom subdomain, for example, https://files.example.com/file/my-unique-bucket-name/smiley.png, after a few seconds you will see a 522 “connection timed out” error from Cloudflare:

This is because, by default, Cloudflare accesses the upstream server via plain HTTP, rather than HTTPS. Backblaze only supports secure HTTPS connections, so the HTTP request fails. To remedy this, in the SSL/TLS section of the Cloudflare dashboard, change the encryption mode from “Flexible” to “Full (strict),” so that Cloudflare connects to Backblaze via HTTPS, and requires a CA-issued certificate.

Now you should be able to access your test file in your custom subdomain via a URL of the form https://files.example.com/file/my-unique-bucket-name/smiley.png. The next task is to create the first Transform Rule to remove /file/my-unique-bucket-name from the URL.

Rewrite the URL Path on Incoming Requests

There are three varieties of Cloudflare Transform Rules:

  • URL Rewrite Rules: Rewrite the URL path and query string of an HTTP request.
  • HTTP Request Header Modification Rules: Set the value of an HTTP request header or remove a request header.
  • HTTP Response Header Modification Rules: Set the value of an HTTP response header or remove a response header.

Click Rules on the left of the Cloudflare dashboard, then Transform Rules. You’ll see that the Cloudflare free plan includes 10 Transform Rules—plenty for our purposes. Click Create Transform Rule, then Rewrite URL.

It’s useful to pause for a moment and think about what we need to ask Cloudflare to do. Users will be requesting URLs of the form https://files.example.com/smiley.png, and we want the request to Backblaze B2 to be like https://f004.backblazeb2.com/file/metadaddy-public/smiley.png. We’ve already taken care of the domain part of the URL, so it becomes clear that all we need to do is prefix the outgoing URL with /file/<bucket name>.

Give your rule a descriptive name such as “Add file and bucket name.”

There is an opportunity to set a condition that incoming requests must match to fire the trigger. In James’ article, he tested that the path did not already begin with the /file/<bucket name> prefix, so that you can refer to a file with either the short or long URL.

At first glance, the Cloudflare dashboard doesn’t offer “does not start with” as an operator.

However, clicking Edit expression reveals a more powerful way of specifying the condition:

The Cloudflare Rules language allows us to express our condition precisely:

Moving on, Cloudflare offers static and dynamic options for rewriting the path. A static rewrite would apply the same value to the URL path of every request. This use case requires a dynamic rewrite, where, for each request, Cloudflare evaluates the value as an expression which yields the path.

Your expression would prepend the existing path with /file/<bucket name>, like this:

Save the Transform Rule, and try to access your test file again, this time without the /file/<bucket name> prefix in the URL path, for example: https://files.example.com/smiley.png.

You should see your test file, as expected:

Great! Now, let’s take a look at those HTTP headers in the response.

Remove HTTP Headers From the Response

You could use Chrome Developer Tools to view the response headers, but I prefer the curl command line tool. I used the --head argument to show the HTTP headers without the response body, since my terminal would not be happy with binary image data!

Note: I’ve removed some extraneous headers from this and subsequent HTTP responses for clarity and length.

% curl --head https://files.superpat.com/smiley.png
HTTP/2 200
date: Thu, 20 Jan 2022 01:26:10 GMT
content-type: image/png
content-length: 23889
x-bz-file-name: smiley.png
x-bz-file-id: 4_zf1f51fb913357c4f74ed0c1b_f1163cc3f37a60613_d20220119_m204457_c004_v0402000_t0044
x-bz-content-sha1: 3cea1118fbaab607a7afd930480670970b278586
x-bz-upload-timestamp: 1642625097000
x-bz-info-src_last_modified_millis: 1642192830529
cache-control: max-age=14400
cf-cache-status: MISS
last-modified: Thu, 20 Jan 2022 01:26:10 GMT

Our goal is to remove all the x-bz headers. Create a Modify Response Header rule and set its name to something like “Remove Backbaze B2 Headers.” We want this rule to apply to all traffic, so the match expression is simple:

Unfortunately there isn’t a way to tell Cloudflare to remove all the headers that are prefixed x-bz, so we just have to list them all:

Save the rule, and request your test file again. You should see fewer headers:

% curl --head https://files.superpat.com/smiley.png
HTTP/2 200
date: Thu, 20 Jan 2022 01:57:01 GMT
content-type: image/png
content-length: 23889
x-bz-info-src_last_modified_millis: 1642192830529
cache-control: max-age=14400
cf-cache-status: HIT
age: 1851
last-modified: Thu, 20 Jan 2022 01:26:10 GMT

Note: As you can see, for some reason Cloudflare does not remove the x-bz-info-src_last_modified_millis header. I’ve reported this to Cloudflare as a bug.

Optimize Cache Efficiency via the ETag and Cache-Control HTTP Headers

We can follow James’ lead in making caching more efficient by leveraging the ETag header. As explained in the MDN Web Docs for ETag:

The ETag (or entity tag) HTTP response header is an identifier for a specific version of a resource. It lets caches be more efficient and save bandwidth, as a web server does not need to resend a full response if the content was not changed.

Essentially, a cache can just request the HTTP headers for a resource and only proceed to fetch the resource body if the ETag has changed.

James constructed the ETag by using one of x-bz-content-sha1, x-bz-info-src_last_modified_millis, or x-bz-file-id, in that order. If none of those headers are set, then neither is ETag. It’s not possible to express this level of complexity in a Transform Rule, but we can apply a little lateral thinking to the problem. We can easily concatenate the three headers to create a result that will change when any one or more of them changes:

concat(http.response.headers["x-bz-content-sha1"][0],
http.response.headers["x-bz-info-src_last_modified_millis"][0],
http.response.headers["x-bz-file-id"][0])

Note that it’s possible for there to be multiple values of a given HTTP header, so http.response.headers["<header-name>"] is an array. http.response.headers["<header-name>"][0] yields the first, and in most cases only, element of the array.

Edit the Transform Rule you just created, update its name to something like “Remove Backblaze B2 Headers, set ETag,” and add a header with a dynamic value:

Don’t worry about the ordering; Cloudflare will reorder the operations so that “set” occurs before “remove.” Also, if none of those headers are present in the response, resulting in an empty value for the ETag header, Cloudflare will not set that header at all. Exactly the behavior we need!

Another test shows the result. Note that HTTP headers are not case-sensitive, so etag has just the same meaning as ETag:

% curl --head https://files.superpat.com/smiley.png
HTTP/2 200
date: Thu, 20 Jan 2022 02:01:19 GMT
content-type: image/png
content-length: 23889
x-bz-info-src_last_modified_millis: 1642192830529
cache-control: max-age=14400
cf-cache-status: HIT
age: 2198
last-modified: Thu, 20 Jan 2022 01:24:41 GMT
etag: 3cea1118fbaab607a7afd930480670970b27858616421928305294_zf1f51fb913357c4f74ed0c1b_f1163cc3f37a60613_d20220119_m204457_c004_v0402000_t0044

The other cache-related header is Cache-Control, which tells the browser how to cache the resource. As you can see in the above responses, Cloudflare sets Cache-Control to a max-age of 14400 seconds, or four hours.

James’ code, on the other hand, sets Cache-Control according to whether or not the request to B2 Cloud Storage is successful. For an HTTP status code of 200, Cache-Control is set to public, max-age=31536000, instructing the browser to cache the response for 31,536,000 seconds; in other words, a year. For any other HTTP status, Cache-Control is set to public, max-age=300, so the browser only caches the response for five minutes. In both cases, the public directive indicates that the response can be cached in a shared cache, even if the request contained an Authorization header field.

Note: We’re effectively assuming that once created, files on the image host are immutable. This is often true for this use case, but you should think carefully about cache policy when you build your own solutions.

At present, Cloudflare Transform Rules do not give access to the HTTP status code, but, again, we can satisfy the requirement with a little thought and investigation. As mentioned above, for successful operations, Cloudflare sets Cache-Control to max-age=14400, or four hours. For failed operations, for example, requesting a non-existent object, Cloudflare passes back the Cache-Control header from Backblaze B2 of max-age=0, no-cache, no-store. With this information, it’s straightforward to construct a Transform Rule to increase max-age from 14400 to 31536000 for the successful case:

Again, we need to use [0] to select the first matching HTTP header. Notice that this rule uses a static value for the header—it’s the same for every matching response.

We’ll leave the header as it’s set by B2 Cloud Storage for failure cases, though it would be just as easy to override it.

Another test shows the results of our efforts:

% curl --head https://files.superpat.com/smiley.png
HTTP/2 200
date: Thu, 20 Jan 2022 02:31:38 GMT
content-type: image/png
content-length: 23889
x-bz-info-src_last_modified_millis: 1642192830529
cache-control: public, max-age=31536000
cf-cache-status: HIT
age: 4017
last-modified: Thu, 20 Jan 2022 01:24:41 GMT
etag: 3cea1118fbaab607a7afd930480670970b27858616421928305294_zf1f51fb913357c4f74ed0c1b_f1163cc3f37a60613_d20220119_m204457_c004_v0402000_t0044

Checking the failure case—notice that there is no ETag header, since B2 Cloud Storage did not return any x-bz headers:

% curl --head https://files.superpat.com/badname.png
HTTP/2 404
date: Thu, 20 Jan 2022 02:32:35 GMT
content-type: application/json;charset=utf-8
content-length: 94
cache-control: max-age=0, no-cache, no-store
cf-cache-status: BYPASS

Success! Browsers and caches will aggressively cache responses, reducing the burden on Cloudflare and Backblaze B2.

Set a CORS Header for Image Files

We’re almost done! Our final requirement is to set a cross-origin resource sharing (CORS) header for images so that they can be manipulated in web pages from any domain on the web.

The Transform Rule must match a range of file extensions, and set the Access-Control-Allow-Origin HTTP response header to allow any webpage to access resources:

Upload a text file and run a final couple of tests to see the results. First, the image:

% curl --head https://files.superpat.com/smiley.png
HTTP/2 200
date: Thu, 20 Jan 2022 02:50:52 GMT
content-type: image/png
content-length: 23889
x-bz-info-src_last_modified_millis: 1642192830529
cache-control: public, max-age=31536000
cf-cache-status: HIT
age: 4459
last-modified: Thu, 20 Jan 2022 01:36:33 GMT
etag: 3cea1118fbaab607a7afd930480670970b27858616421928305294_zf1f51fb913357c4f74ed0c1b_f1163cc3f37a60613_d20220119_m204457_c004_v0402000_t0044
access-control-allow-origin: *

The Access-Control-Allow-Origin header is present, as expected.

Finally, the text file, without an Access-Control-Allow-Origin header. You can use the --include argument rather than --head to see the file content as well as the headers:

% curl --include https://files.superpat.com/hello.txt
HTTP/2 200
date: Thu, 20 Jan 2022 02:48:51 GMT
content-type: text/plain
content-length: 14
accept-ranges: bytes
x-bz-info-src_last_modified_millis: 1642646740075
cf-cache-status: DYNAMIC
etag: 60fde9c2310b0d4cad4dab8d126b04387efba28916426467400754_zf1f51fb913357c4f74ed0c1b_f1092902424a40504_d20220120_m024635_c004_v0402003_t0000

Hello, World!

Troubleshooting

The most frequent issue I encountered while getting all this working was mixing up request and response when referencing HTTP headers. If things are not working as expected, double check that you don’t have http.response.headers["<header-name>"] where you need http.request.headers["<header-name>"] or vice versa.

Can I Really Do This Free of Charge?

Backblaze B2 pricing is very simple:

Storage
  • The first 10GB of storage is free of charge.
  • Above 10GB, we charge $0.005/GB/month, around a quarter of the cost of other leading cloud object stores (cough, S3, cough).
  • Storage cost is calculated hourly, with no minimum retention requirement, and billed monthly.
Downloaded Data
  • The first 1GB of data downloaded each day is free.
  • Above 1GB, we charge $0.01/GB, but…
  • Downloads through our CDN and compute partners, of which Cloudflare is one, are free.
Transactions
  • Each download operation counts as one class B transaction.
  • The first 2,500 class B transactions each day are free.
  • Beyond 2,500 class B transactions, they are charged at a rate of $0.004 per 10,000.
No Surprise Bills
  • If you already signed up for Backblaze B2, you might have noticed that you didn’t have to provide a credit card number. Your 10GB of free storage never expires, and there is no chance of you unexpectedly incurring any charges.

By serving your images via Cloudflare’s global CDN and optimizing your cache configuration as described above, you will incur no download costs from B2 Cloud Storage, and likely stay well within the 2,500 free download operations per day. Similarly, Cloudflare’s free plan does not require a credit card for activation, and there are no data or transaction limits.

Sign up for Backblaze B2 today, deploy your own personal image host, explore our off-the-shelf integrations, and consider what you can create with an affordable, S3-compatible cloud object storage platform.

The post Free Image Hosting With Cloudflare Transform Rules and Backblaze B2 appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Cloud Performance and When It Matters

Post Syndicated from Amrit Singh original https://www.backblaze.com/blog/cloud-performance-and-when-it-matters/

If you run an application that’s bandwidth intensive like media streaming, game hosting, or an e-commerce platform, performance is probably top of mind. You need to be able to deliver content to your users fast and without errors in order to keep them happy. But, what specific performance metrics matter for your use case?

As it turns out, you might think you need a Porsche when what you really need and want to transport your data with is a trusty, reliable (Still speedy!) Volvo.

In this post, we’re taking a closer look at performance metrics and when they matter as well as some strategies that can impact performance, including range requests, prefetching, and others. When you’re assessing a cloud solution for application development, taking these factors into consideration can help you make the best decision for your business.

Performance Metrics: Time to First Byte

Time to first byte (TTFB) is the time between a page request and when the page receives the first byte of information from the server. In other words, TTFB is measured by how long it takes between the start of the request and the start of the response, including DNS lookup and establishing the connection using a TCP handshake and SSL handshake if you’ve made the request over HTTPS.

TTFB identifies pages that load slowly due to server-side calculations that could instead benefit from client-side scripting. It’s often used to assess search rankings by displaying websites that respond to a request faster and appear more usable before other websites.

TTFB is a useful metric, but it doesn’t tell the whole story every time and shouldn’t be the only metric used to make decisions when it comes to choosing a cloud storage solution. For example, when David Liu, Founder and CEO of Musify, a music streaming app, approached his search for a new cloud storage provider, he had a specific TTFB benchmark in mind. He thought he absolutely needed to meet this benchmark in order for his new storage solution to work for his use case, however, upon further testing, he found that his initial benchmark was more aggressive than he actually needed. The performance he got by utilizing Cloudflare in front of his origin store in Backblaze B2 Cloud Storage more than met his needs and served his users well.

Optimizing Cloud Storage Performance

TTFB is the dominant method of measuring performance, but TTFB can be impacted by any number of factors—your location, your connection, the data being sent, etc. As such, there are ways to improve TTFB, including using a content delivery network (CDN) on top of origin storage, range requests, and prefetching.

Performance and Content Delivery Networks

A CDN helps speed content delivery by storing content at the edge, meaning faster load times and reduced latency. For high-bandwidth use cases, a CDN can optimize media delivery.

Companies like Kanopy, a media streaming service; Big Cartel, an e-commerce platform; and CloudSpot, a professional photo gallery platform, use a CDN between their origin storage in Backblaze B2 and their end users to great effect. Kanopy offers a library of 25,000+ titles to 45 million patrons worldwide. Latency and poor performance is not an option. “Video needs to have a quick startup time,” Kanopy’s Lead Video Software Engineer, Pierre-Antoine Tible said. “With Backblaze over [our CDN] Cloudflare, we didn’t have any issues.”

For Big Cartel, hosting one million customer sites likewise demands high-speed performance. Big Cartel’s Technical Director, Lee Jensen, noted, “We had no problems with the content served from Backblaze B2. The time to serve files in our 99th percentile, including fully rendering content, was under one second, and that’s our worst case scenario.” The time to serve files in their 75th percentile was under just 200 to 300 milliseconds, and that’s when content needs to be pulled from origin storage in Backblaze B2 when it’s not already cached in their CDN Fastly’s edge servers.

“We had no problems with the content served from Backblaze B2. The time to serve files in our 99th percentile, including fully rendering content, was under one second, and that’s our worst case scenario.”
—Lee Jensen, Technical Director, Big Cartel

Range Requests and Performance

HTTP range requests allow sending only a portion of an HTTP message from a server to a client. Partial requests are useful for large media or downloading files with pause and resume functions, and they’re common for developers who like to concatenate files and store them as big files. For example, if a user wants to skip to a clip of a full video or a specific frame in a video, using range requests means the application doesn’t have to serve the whole file.

Because the Backblaze B2 vault architecture separates files into shards, you get the same performance whether you call the whole file or just part of the file in a range request. Rather than wasting time learning how to optimize performance on a new platform or adjusting your code to comply with frustrating limitations, developers moving over to Backblaze B2 can utilize existing code they’re already invested in.

Prefetching and Performance

Prefetching is a way to “queue up” data before it’s actually required. This improves latency if that data is subsequently requested. When you’re using a CDN in front of your origin storage, this means the user queues up data/files/content in the CDN before someone asks for it.

Video streaming service, Kanopy, uses prefetching with popular videos they expect will see high demand in certain regions. This would violate some cloud storage providers’ terms of service because they egress out more than they store. Because Kanopy gets free egress between their origin store in Backblaze B2 and their CDN Cloudflare, the initial download cost for prefetching is $0. (Backblaze also has partnerships with other CDN providers like Fastly and bunny.net to offer zero egress.) The partnership means Kanopy doesn’t have to worry about running up egress charges, and they’re empowered to use prefetching to optimize their infrastructure.

Other Metrics to Consider When Assessing Cloud Performance

In addition to TTFB, there are a number of other metrics to consider when it comes to assessing cloud performance, including availability, the provider’s service level agreements (SLAs), and durability.

Availability measures the percentage of time the data is available to be accessed. All data occasionally becomes unavailable due to regular operating procedures like system maintenance. But, obviously data availability is very important when you’re serving content around the globe 24/7. Backblaze B2, for example, commits to a 99.9% uptime with no cold delays. Commitments like uptime are usually outlined in a cloud provider’s SLA—an agreement that lists the performance metrics the cloud provider agrees to provide.

Durability measures how healthy your data is. Object storage providers express data durability as an annual percentage in nines, as in two nines before the decimal point and as many nines as warranted after the decimal point. For example, 11 nines of durability is expressed as 99.999999999%. What this means is that the storage vendor is promising that your data will remain intact while it is under their care without losing any more than 0.000000001% of your data in a year (in the case of 11 nines annual durability).

Ready to Get Started?

Understanding the different performance metrics that might impact your data can help when you’re evaluating cloud storage providers. Ready to get started with Backblaze B2? We offer the first 10GB free.

The post Cloud Performance and When It Matters appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

How to Build the Right Tech Stack for Your MSP

Post Syndicated from Kari Rivas original https://www.backblaze.com/blog/how-to-build-the-right-tech-stack-for-your-msp/

As a managed service provider (MSP) or IT consultant, your bottom line depends on having the right tools at the right prices to maintain your margins while still providing the resources and functionality your clients need. And you’ve likely seen the resources and functionality your clients need changing over the past few years towards an increased focus on cybersecurity and disaster recovery.

More and more companies are hiring remotely, which means increased security risks, ransomware attacks on small and medium-sized businesses (SMBs) have increased, and severe natural disasters are threatening on-premises office technology. Having the right tech stack for your MSP demonstrates to current and potential clients that they can trust you to safeguard their valuable data and systems against the threats of today and tomorrow.

Level up your value proposition with insights on building a competitive “right-sized” tech stack at our upcoming webinar, “The Essential MSP Tech Stack,” on Tuesday, February 15th at 1 p.m. CST/2 p.m. EST.

➔ Sign Up for the Webinar

Read on to get a preview of what will be covered during the webinar.

The Top Considerations for an Essential MSP Tech Stack

SMBs outsource their IT to MSPs and consultants because they don’t have the time, knowledge, or resources to shop around for the right tech solution for themselves. They may not even know what criteria they should be using to evaluate solutions, and this can lead to them shopping around among MSPs based on price alone.

Sourcing solutions with a lower cost to you means you can price your services more competitively and better attract customers. But pricing is just one of the considerations you should make when purchasing software. Have you also thought about scalability, and whether your tech stack can grow with you as your client base grows? Or what kinds of support options your software provider has available?

Pricing is important, yes, but there are several other factors by which you should judge your tech stack options, including features, automation options, and integrations, which will be covered in more detail during the webinar.

Right-sizing Your MSP Tech Stack

To develop your MSP offering, you’ll also want to think about what MSP services are most in demand in your area and what solutions you can offer the most efficiently and cost-effectively. It’s not “essential” to offer everything. The right tech stack is the one that brings you the most clients at the greatest profitability.

You may even want to do some research on the other MSPs in your geographic area. Is there something you can offer that they do not? Play to your strengths—what technical areas do you know the best?

As you start to develop your offering, consider the following areas of managed IT services and how they might help you attract clients:

Backup and Cloud Storage for MSPs

When it comes to managed backup and cloud storage, Backblaze and our partner, MSP360, have you covered. Backblaze provides easy and affordable server and workstation backup, and our integration with MSP360 provides a seamless experience to back up standalone and multiple servers to Backblaze B2 Cloud Storage.

MSPs and IT organizations with multiple servers can manage all of their machines from one, centralized, web-based admin console. Backblaze B2 backups are “set it and forget it” after the initial setup. Data is kept in hot storage and available immediately when needed. And B2 Cloud Storage is extremely affordable at $5/TB per month without any additional fees or tiered pricing structure.

Our integration with MSP360 includes advanced backup protection features like flexible scheduling, compression, encryption, and ransomware protection. We’ve even made it super easy to get started on your own. Just use our online onboarding tool to create both Backblaze B2 and MSP360 accounts at the same time.

Bundling MSP Services to Streamline the Purchase Decision

Consider bundling your services to make it easier for clients to buy from you and understand how you’ll help protect their business. For instance, the joint solution from MSP360 and Backblaze can be bundled as part of a disaster recovery, backup, and storage package. You could also create tiers of services, like a “bronze” level disaster recovery, backup, and storage package; a “silver” level package that includes all of the above plus monitoring, tech management, and installation services; and a “gold” level package that functions essentially like fully outsourced IT.

Non-IT Tools for the MSP Tech Stack

Finally, as you build your MSP, don’t forget that your tech stack may need to include non-IT tools as well. You’ll need a way to oversee business accounting and your books, a way to manage your client relationships, leads, and sales, plus software to manage employees, payroll, and other aspects of general business management.

Ready to Upgrade Your Tech Stack?

Having the right tech stack isn’t a matter of checking all the boxes on a list of software. It’s a strategic decision about what your potential clients will most value, what you’re best equipped to offer, and how you can make a profit. Instead of trying to meet every possible need, ensure that you have the “right-sized” tech stack to service the types of clients you represent without paying extra for bloated software that may go unused. You can often have a healthier business by specializing in just a few areas and attracting the right types of clients, rather than trying to cater to everyone.

Want to learn more? Join our webinar on Tuesday, February 15th at 1 p.m. CST/2 p.m. EST to learn more about how to build the tech stack for your MSP.

The post How to Build the Right Tech Stack for Your MSP appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Backblaze Drive Stats for 2021

Post Syndicated from original https://www.backblaze.com/blog/backblaze-drive-stats-for-2021/

In 2021, Backblaze added 40,460 hard drives and as of December 31, 2021, we had 206,928 drives under management. Of that number, there were 3,760 boot drives and 203,168 data drives. This report will focus on our data drives. We will review the hard drive failure rates for 2021, compare those rates to previous years, and present the lifetime failure statistics for all the hard drive models active in our data center as of the end of 2021. Along the way, we share our observations and insights on the data presented and, as always, we look forward to you doing the same in the comments section at the end of the post.

2021 Hard Drive Failure Rates

At the end of 2021, Backblaze was monitoring 203,168 hard drives used to store data. For our evaluation, we removed 409 drives from consideration which were used for either testing purposes or drive models for which we did not have at least 60 drives. This leaves us with 202,759 hard drives to analyze for this report.

Observations and Notes

The Old Guy Rules: For 2021, the 6TB Seagate (model: ST6000DX000) had the lowest failure rate of any drive model, clocking in with an annualized failure rate (AFR) of 0.11%. This is even more impressive when you consider that this 6TB drive model is the oldest in the fleet with an average age of 80.4 months. The number of drives, 886, and 2021 drive days, 323,390, are on the lower side, but after nearly seven years in operation, these drives are thumbing their nose at the tail end of the bathtub curve.

The Kids Are Alright: Two drive models are new for 2021 and both are performing well. The 16TB WDC drive cohort (model: WUH721816ALE6L0) has an average age of 5.06 months and an AFR of 0.14%. While the 16TB Toshiba drive cohort (model: MG08ACA16TE) has an average age of 3.57 months and an AFR of 0.91%. In both cases, the number of drive days is on the lower side, but these two drive models are off to a good start.

AFR, What Does That Mean?

AFR stands for annualized failure rate. This is different from an annual failure rate in which the number of drives is the same for each model (cohort) throughout the annual period. In our environment, drives are added and leave throughout the year. For example, a new drive installed in Q4 might contribute just 43 days, while a drive that failed in July might contribute 186 days, while drives in continuous operation for the year could contribute 365 days each. We count the number of drive days each drive contributes throughout the period and annualize the total using this formula:

AFR = (drive failures / (drive days / 365)) * 100

The Patient Is Stable: Last quarter, we reported on the state of our 14TB Seagate drives (model: ST14000NM0138) provisioned in Dell storage servers. They were failing at a higher than expected rate and everyone—Backblaze, Seagate, and Dell—wanted to know why. The failed drives were examined by fault analysis specialists and in late Q3 it was decided as a first step to upgrade the firmware for that cohort of drives still in service. The results were that the quarterly failure rate dropped from 6.29% in Q3 to 4.66% in Q4, stabilizing the rapid rise in failures we’d seen in Q2 and Q3. The 19 drives that failed in Q4 were shipped off for further analysis. We’ll continue to follow this process over the coming quarters.

The AFR for 2021 for all drive models was 1.01%, which was slightly higher than the 0.93% we reported for 2020. The next section will compare the data from the last three years.

Comparing Drive Stats for 2019, 2020, and 2021

The chart below compares the AFR for each of the last three years. The data for each year is inclusive of that year only and for the active drive models present at the end of each year.

Digging a little deeper, we can aggregate the different drive models by manufacturer to see how failure rates per manufacturer have fared over the last three years.

Note that for the WDC data, a blank value means we did not have any countable WDC drives in our data center in that quarter.

Trends for 2021

The AFR Stayed Low in 2021: In 2021, the AFR for all drives was 1.01%. This was slightly higher than 2020 at 0.93%, but a good sign that the drop in 2020 from 1.83% in 2019 was not an anomaly. What’s behind the 1.01% for 2021? Large drives, as seen below:

The AFR for larger drives, defined here as 12TB, 14TB, and 16TB drives, are all below the 2021 AFR of 1.01% for all drives. The larger drives make up 69% of the total drive population, but more importantly, they total 66% of the drive days total, while only producing 57% of the drive failures.

The larger drives are also the newer drives, which tend to fail less versus older drives. In fact, the oldest large drive has an average age 33 months, while the youngest “small” (4TB, 6TB, 8TB, and 10TB) drive has an average age of 44.9 months.

In summary, the lower AFR for the larger drives is a major influence in keeping the overall AFR for 2021 low.

Drive Model Diversity Continues: In 2021, we added two new drive models to our farm with no models retired. We now have a total of 24 different drive models in operation. That’s up from a low point of 14 in 2019 and 22 in 2020. The chart below for “Backblaze Quarterly Hard Drive Population Percentage by Manufacturer” examines the changing complexion of our drive farm as we look at the number of models from each manufacturer we used over the past six years.

When we first started, we often mixed and matched drive models, mostly out of financial necessity—we bought what we could afford. As we grew, we bought and deployed drives in larger lots and drive homogeneity settled in. Over the past few years, we have gotten more comfortable with mixing and matching again, enabled by our Backblaze Vault architecture. A Vault is composed of sixty tomes, with each tome being 20 drives. We make each tome the same drive model, but each of the tomes within a vault can have different drive models, and even different drive sizes. This allows us to be less reliant on any particular drive model, so the more drive models the better.

Drive Vendor Diversity Continues, Too: When looking at the chart above for “Backblaze Hard Drive Population by Model Count per Manufacturer Over Time,” you might guess that we have increased the percentage of Seagate drives over the last couple of years. Let’s see if that’s true.

It appears the opposite is true, we have lowered the percentage of Seagate drives in our data centers, even though we have added additional Seagate models.

Why is it important to diversify across multiple manufacturers? Flexibility, just like increasing the number of models. Having relationships with all the primary hard drive vendors gives us the opportunity to get the resources we need in a timely fashion. The fact that we can utilize any one of several different models from these vendors adds to that flexibility.

Lifetime Hard Drive Stats

The chart below shows the lifetime annualized failure rates of all the drive models in production as of December 31, 2021.

Observations and Caveats

The lifetime AFR for all the drives listed above is 1.4% and continues to go down year over year. At the end of 2020, the AFR was 1.54% and at the end of 2019, the AFR stood at 1.62%.

When looking at the chart above, several of the drives have a fairly wide confidence interval (>0.5). In these cases, we do not really have enough information about the drive’s performance to be reasonably confident (>95%) in the AFR listed. This is typically the case with lower drive counts or newer drives.

Looking for SSD Numbers?

We’ll be covering our annual failure rates for our SSD drives in a separate post in the next few weeks. We realized that combining the analysis of our data drives and our boot drives in one post was confusing. Stay tuned.

The Hard Drive Stats Data

The complete data set used to create the information used in this review is available on our Hard Drive Test Data page. You can download and use this data for free for your own purpose. All we ask are three things: 1) you cite Backblaze as the source if you use the data, 2) you accept that you are solely responsible for how you use the data, and 3) you do not sell this data to anyone; it is free.

If you just want the summarized data used to create the tables and charts in this blog post, you can download the ZIP file containing the CSV files for each chart.

Good luck and let us know if you find anything interesting.

The post Backblaze Drive Stats for 2021 appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Hybrid Cloud and Modern Workflows for Media Teams

Post Syndicated from Amanda Fesunoff original https://www.backblaze.com/blog/hybrid-cloud-and-modern-workflows-for-media-teams/

By any metric, the demands on media workflows are growing at an unprecedented rate. A Coughlin Associates Report of media and entertainment professionals predicts that overall cloud storage capacity for media and entertainment is expected to grow over 13.8 times between 2020 and 2026 (101.1EB to 140EB). It also predicts that, by the next decade, total video captured for a high-end digital production could be hundreds of petabytes, approaching one exabyte.

Businesses in the media and entertainment industry—from creative teams to production houses to agencies—must manage larger and larger stores of data and streamline production workflows that interact with those stores of data. Optimizing data-heavy workflows provides you with time and cost savings you can reinvest to prioritize the creative work that drives your business.

In today’s post, we’ll examine the trends shaping the media storage landscape, walk through each step of the media workflow, and provide strategies and tactics for reducing friction at each step along the way. Read on to learn how to modernize your media workflow to meet today’s data-heavy demands.

➔ Download Our Media Workflows E-book

Media Technology Trends and Impacts on Media Workflows

Technology is driving changes in media workflows. The media landscape of today looks very different than it did even a few short years ago. If you’re responsible for managing data and workflows for a creative team, understanding the broad trends in the media landscape can help you prepare to optimize your workflows and future-proof your data infrastructure. Here are a few key trends we see driving change across the media storage landscape.

Trend 1: Increased Demand for VR and Higher Resolution 4K and 8K Video Is Driving Workflow Modernization

While VR has been somewhat slow to build steam, demand for VR experiences has grown as the technology evolved. The industry as a whole is growing at a fast pace, with the global VR market size projected to increase from less than $5 billion in 2021 to more than $12 billion by 2024. Today, demands for stereoscopic VR, and VR in general, have increased storage requirements as data sets grow exponentially. Similarly, higher resolution demands more from media workflows, including more storage space, greater standards for compression, and higher performance hardware. All of these files also need to be constantly available and secure. As such, media workflows increasingly value scalable storage, as having to wait for additional storage may cause delays in project momentum/delivery.

Trend 2: Archiving and Content Preservation Needs Are Driving Storage Growth

While the need to digitally convert data from traditional film and tape has slowed, the enormous demand for digital storage for archived content continues to grow. According to the Coughlin Report, more than 174 exabytes of new digital storage will be used for archiving and content conversion and preservation by 2024.

Just as your storage needs for active projects continues to grow as file sizes continue to expand, expect to invest in storage for archival purposes as production continues apace. Furthermore, if you have content conversion or preservation needs, plan for storage needs to house digital copies. The plus side of this surge in archival and preservation demand is that the storage market will continue to be competitive, giving you plenty of choices at competitive rates.

Trend 3: Cloud Adoption Is Playing an Important Role in Enabling Collaboration Across Teams and Geographies

A study by Mesa of nearly 700 decision-makers and managers from media and entertainment companies found that they expect that 50% of their workforce will continue to work remotely. Accessing resources remotely used to be a challenge mired by latency issues, restrictions on file size, and subpar collaboration tools, but cloud adoption has eased these issues and will continue to do so as companies increasingly embrace long-term remote collaboration.

As you think about future-proofing your architecture, one factor to consider is cost, but also designing an architecture that enables your existing workflows to function remotely. A cloud storage provider with predictable pricing can address cost considerations and make cloud adoption even more of a no-brainer. And media workflows can adopt cloud-native solutions or integrate existing on-premises infrastructure with the cloud without additional hardware purchasing and maintenance. The result is that time and money that would have been spent on hardware can be reinvested into adopting new technology, meeting customers’ needs, and differentiating from competitors.

Steps in the Modern Media Workflow

With an understanding of these overarching trends, media and entertainment professionals can evaluate and analyze their workflow to meet future demands. To illustrate that, we’ll walk through an example cloud storage setup within a media workflow, including:

  1. Ingest to Local Storage.
  2. Video Editing Software.
  3. Media Asset Managers.
  4. Archive.
  5. Backup.
  6. Transcoding Software.
  7. Content Delivery.
  8. Cloud Storage.

Ingest to Local Storage

Creatives doing work in progress need high performance, local access storage such as NAS, SANs, etc. These are often backed up to cloud storage to have an off-site version of the current projects. Some examples include Synology and QNAP NAS devices as well as the OWC Jellyfish system. With Synology, you can use their Cloud Sync application to sync your files directly to your cloud bucket. Synology also offers many built-in integrations to various cloud providers. For QNAP, you can use QNAP Hybrid Backup Sync to archive or back up your content to your cloud account. OWC Jellyfish is optimized for video production workflows, and the Jellyfish lineup is embraced by video production teams for on-prem storage.

Video Editing Software

Video editing software is used to edit, modify, generate, or manipulate a video or movie file. Backblaze has a number of tools we support depending on your workflow. Adobe Premiere Pro and Avid Media Composer are two examples of film and video editing software. They are used to create videos, television shows, films, and commercials.

Media Asset Managers

A media asset manager, or MAM, is software used to add metadata, manage content, store media in a hybrid cloud, and share media. Examples of MAMs include iconik, eMAM, EditShare, and Archiware. You can store your media files directly to the cloud from these and other media asset managers, enabling monetization and quicker content delivery of older content.

Archive

An archive often consists of completed projects and infrequently-used assets that are stored away to keep primary production storage capacities under control. Examples of archive tools include LTO tape, external hard drives, servers, and cloud providers.

Backup

A backup is often used with new projects where raw media files are ingested into their systems and then backed up in case of accidental deletion so that they can be easily restored. Examples include LTO tape, external hard drives, servers, and cloud providers.

Transcoding Software

Transcoding software converts encoded digital files into an alternative digital format so that it can be viewed on the widest possible range of devices.

Content Delivery

Content delivery networks (CDNs) enable easy distribution of your content to customers. Examples include Fastly and Cloudflare. CDNs store content on edge servers closer to end users, speeding performance and reducing latency.

Cloud Storage

Cloud storage is integrated with all of the above tools, making it easy to store high resolution, native files for backup, active archives, primary storage, and origin stores. The media workflow tools have easy access to the stored content in the cloud via their user interface. Storing content in the cloud allows teams to easily collaborate, share, reuse, and distribute content. Cloud storage is also emerging as the storage of choice for workflows that use cloud-based MAMs.

illustration of a NAS device and cloud storage

The Benefits of Using a Hybrid Cloud Model for Media Workflows

Because media teams need both fast access and scalable storage, many adopt a hybrid cloud storage strategy. A hybrid cloud strategy combines a private cloud with a public cloud. For most media teams, the private cloud is typically hosted on on-premises infrastructure, but can be hosted by a third party. The key difference between a private and public cloud is that the infrastructure, hardware, and software for a private cloud are maintained on a private network used exclusively by your business or organization.

In a hybrid cloud workflow, media teams have fast, on-premises storage for active projects combined with the scalability of a public cloud to accommodate the large amounts of data media teams generate. Looking specifically at the cloud storage functions above, it is important to keep your local storage lean and mean so that it is fast and operating at peak performance for your creative team. This achieves two things. First, you don’t have to invest more in local storage which can be expensive and time consuming to maintain. And second, you can offload older projects to the cloud while maintaining easy accessibility.

According to a survey of IT decision makers who adopted a hybrid cloud approach: 26% of them said faster innovation was the most important benefit their business gained. 25% said it allowed them to have faster responses to their customers. 22% said it provided their business with better collaboration. Benefits of a hybrid cloud approach for media teams include:

  1. Affordability: Cloud storage can be lower cost versus expanding your own physical infrastructure.
  2. Accessibility: A hybrid cloud provides increased collaboration for a remote workforce.
  3. Scalability: Cloud scalability provides ease and control with scaling up or down.
  4. Innovation: Media teams have an increased ability to quickly test and launch new products or projects, when not bogged down by physical infrastructure.
  5. Data Protection & Security: Media teams benefit from reduced downtime and can bounce back quicker from events, failures, or disasters.
  6. Flexibility: Hybrid solutions allow media teams to maintain control of sensitive or frequently used data on-premises while providing the flexibility to scale in the cloud.

Want to learn more about hybrid clouds? Download our free e-book, “Optimizing Media Workflows in the Cloud,” today.

The post Hybrid Cloud and Modern Workflows for Media Teams appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Recent QNAP and Synology Security Alerts: How to Protect Your NAS

Post Syndicated from Jeremy Milk original https://www.backblaze.com/blog/recent-qnap-and-synology-security-alerts-how-to-protect-your-nas/

Editor’s Note: This post has been updated since it was originally published on January 18, 2022 to reflect the most recent security alerts from NAS manufacturers.

If you use a NAS device for your business or home network, chances are one of the likely reasons you invested in NAS in the first place was to achieve a greater level of data redundancy and protection. You’re deliberate about the care and protection of your data. Unfortunately, ransomware operators have been ramping up attacks on NAS devices over the past year, especially in the past few weeks, which could mean you’re facing more risk.

Integrated Backblaze partners QNAP and Synology have smartly issued alerts and offered new guidance to help users better protect their data from these attacks. QNAP’s recent alerts urged users to take immediate action to ensure the security of their devices.

Since many of you use Backblaze B2 Cloud Storage to back up or build from your NAS devices, this post outlines the recent alerts, the nature of the attacks, and the steps you can take to protect your data.

Recent Alerts

How Cybercriminals Attack Your NAS

Attackers will typically exploit known vulnerabilities and may use brute force attacks where they try passwords until they gain access so they can plant ransomware software on NAS devices. In August of 2021, Palo Alto Networks, a security research firm, identified a variant of eCh0raix ransomware that targets Synology and QNAP NAS devices.

QNAP’s January 7 alert didn’t specify the ransomware strain involved in the attacks they’re seeing, but if they’re not using eCh0raix, they’re likely using something similar.

QNAP’s January 26 alert identified a new type of ransomware named DeadBolt. DeadBolt has been widely targeting all NAS exposed to the internet without any protection and encrypting users’ data for Bitcoin ransom.

What You Can Do

Even conscientious NAS users may occasionally miss the latest security updates and patches, leaving devices vulnerable. And cybercriminals exploit these vulnerabilities.

Here are a few immediate steps you can take to protect your data:

  1. Sign up for security alerts from your device manufacturer, and apply the latest security patches as soon as possible. For the latest security updates, check the QNAP Security Advisories page and Synology Product Security Advisory page regularly.
  2. Use best practices when it comes to passwords to make brute-force attacks more of a challenge for attackers, including changing passwords regularly and using complex passwords.
  3. Prevent network attacks by limiting device connections to a hard-coded list of IP addresses.

QNAP-specific Prevention

QNAP issued specific instructions on both January 7 and January 26 urging all users to disconnect their devices from the internet immediately. They recommend the following steps:

First, check whether your NAS is exposed to the internet. Open the Security Counselor on your QNAP NAS. Your NAS is exposed to the internet and at high risk if it shows the system administration service can be directly accessible from an external IP address via the following protocols: HTTP on the dashboard.

Source: QNAP.

Note: QNAP recommended users check here to know which ports are exposed to the internet.

If your NAS is exposed to the internet, QNAP recommends the following steps:

  1. Disable the port forwarding function of the router.
  2. Go to the management interface of your router, check the virtual server, NAT, or port forwarding settings, and disable the port forwarding setting of the NAS management service port (port 8080 and 433 by default).

    Source: QNAP.
  3. Disable the UPnP function of the QNAP NAS.
  4. Go to myQNAPcloud on the QTS menu, click Auto Router Configuration, and deselect Enable UPnP Port forwarding.

    Source: QNAP.

Synology-recommended Prevention

Synology provides users with a number of resources to help them increase the security of their NAS devices. To keep your Synology data secured, check out their knowledge base article on how to add extra security to your NAS or their blog post outlining “10 Security Tips to Keep Your Data Safe.”

Following security best practices, they recommend using complex passwords, setting expiration dates for passwords, and being very cautious with public ports. They also recommend enabling Security Advisor. Security Advisor is a built-in DiskStation Manager (DSM) app that scans your Synology NAS, checks your DSM settings, and gives you advice on how to address security weaknesses.

Protect Your Data With NAS Backups

Keeping your device up to date on security patches and updates and closely monitoring alerts from your device manufacturer will go a long way toward protecting your data. For the latest security updates, check the QNAP Security Advisories page and Synology Product Security Advisory page regularly. However, if you are operating without a backup, you’re at risk of data loss. Data recovery is much easier with a backup copy of your data saved in cloud storage.

Your vigilance plus a strong backup system could make all the difference in the event of a ransomware attack. Learn more by downloading our Complete Guide to Ransomware.

The post Recent QNAP and Synology Security Alerts: How to Protect Your NAS appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Let’s Be Honest—Retention Minimums Are Delete Penalties

Post Syndicated from Jeremy Milk original https://www.backblaze.com/blog/lets-be-honest-retention-minimums-are-delete-penalties/

People often think of “retention” as a good thing when it comes to cloud and object storage—after all, the point of storing data is to retain it. But retention’s only a good thing when you actually want to retain data—that nuance is sometimes hidden from people, and yes, I say hidden intentionally.

A number of cloud storage providers from big to small are doing their best to hide the darker side of retention—retention minimums. They loudly promote attractive storage tier rates while making little mention of their data retention minimums that allow them to charge those rates for as many as 90 or 180 days after bytes uploaded have been deleted.

We don’t believe in charging you for data you deleted. Today, we’re explaining more about what that means for you, and highlighting some real-world stories of discovering these hidden fees.

Our Stance on Retention Minimums aka Delete Penalties

First, let’s call retention minimums what they really are: delete penalties. We stand against delete penalties. We don’t charge them. We see them as the enemy of every use case in which data is intentionally replaced or deprecated in hours, days, or weeks instead of months. Delete penalties go against agility and flexibility. We also think it’s despicable when a vendor shouts about how they don’t charge fees for things like data egress, while quietly padding their topline with hidden retention penalties.

At Backblaze, our pricing has nothing to hide. When you delete data, you stop paying for it within the hour. End of story.

Retention Minimums: The Fine Print or the Finer Print

Obviously, cloud providers aren’t going to advertise that they charge you for deleted data, but some are more transparent than others. AWS with its S3 Glacier services, for example, at least acknowledges these products are meant primarily for longer term storage. They disclose minimum retention details in the footnotes on their pricing page—the information is less prominent, but to their credit, it’s disclosed on the page. It may seem unusual for us to praise AWS, but by comparison, they’re actually a lesser evil in this regard.

Others? Let’s just say you really need a magnifying glass to dig through the fine print. Their minimum retention requirements are buried deep in their terms of service or FAQs. Unless you have an eagle eye and/or click through many pages of their website, you’re left to find out just how much you’re paying for deleted data when you get your bill. What’s more, the disappointment and disillusionment from budget surprises like that can turn people off from the many gains they can derive from leveraging cloud storage.

Delete Penalties in the Wild: Testimonials

Here’s what we’ve heard from folks who experienced delete penalties for themselves…

“Initially, I was worried about egress, so I went with [name redacted]. But I was misled. My egress was nominal. Meanwhile, I found that one-third or more of my bill was for backup I had deleted. That’s not how I want to do business.”
—MSP Leader

“I looked at an up-and-coming provider called [name redacted] because their whole thing is they’ve got great prices. I soured on them when I realized that they don’t really tell you that they bill you for a minimum of 90 days of object duration. There’s little I need to store for 90 days for my application. All of my cursory research seemed okay, and the pricing calculator on the pricing page made no mention of any of this. I’m not a fan of using a vendor that buries something that important.”
—Brian, Software Developer

“We got burned by [name redacted] with regard to their deletion and how we do our backups. I deleted data off their system, and they’re billing me for data they’re not storing? And what’s more, they’re irritated by the fact that their hard drives had to delete data? I don’t understand that level of…I’m not even going to say the word, but it’s just stupid.”
—Joe Valentine, Software Engineer II, Webjogger

Delete the Delete Penalties

To be sure, compared to the high costs of on-premises infrastructure, cloud storage delete penalties may go unnoticed or be characterized as a cost of saving money. But that’s exactly what companies who levy these penalties want you to think. Don’t let them misrepresent their true costs or mislead you. It’s not right. It’s not aligned with their messaging. It’s not what you deserve. And it’s not going to support your business growth especially when fees add up fast for many terabytes and petabytes.

It’s time to delete the delete penalties. Full stop.

If you’ve been hit with unexpected penalties after deleting data, share your experience below with the broader community or reach out to us to learn more about how you can eliminate them.

The post Let’s Be Honest—Retention Minimums Are Delete Penalties appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Announcing Kubernetes Backup and Ransomware Protection via Kasten and Backblaze

Post Syndicated from Natasha Rabinov original https://www.backblaze.com/blog/announcing-kubernetes-backup-and-ransomware-protection-via-kasten-and-backblaze/

According to the Cloud Native Computing Foundation’s annual survey, Kubernetes use in production has increased 300% since 2016 to 83% of respondents. There’s no doubt that a fundamental shift has taken place over the past few years—applications are being deployed in container environments and those deployments are being managed by Kubernetes.

But customers that are deploying Kubernetes environments need a new tool to protect all of that data. Purpose-built for Kubernetes, Kasten by Veeam is that tool.

Now, through a new partnership, joint Kasten and Backblaze customers will be able to name Backblaze B2 Cloud Storage as a storage destination where they can store and protect copies of their applications affordably.

This partnership enables developers to:

  • Back up and restore production stateful Kubernetes applications.
  • Safeguard application data from ransomware encryption with Object Lock for immutability.
  • Support regulatory compliance and corporate disaster recovery mandates.

“Kubernetes containers are the standard for many organizations building, deploying, and scaling applications with portability and efficiency. Backblaze and Kasten together offer a compelling solution to support these organizations’ business continuity needs with set-and-forget-it ease and cost effectiveness.”
—Nilay Patel, VP of Sales and Partnerships, Backblaze

The joint solution is fully scalable at enterprise grade. What’s more, organizations only pay for storage used, with no data retention penalties for deleting past backups.

About Kasten

Owned by Veeam, Kasten is an award-winning market leader in Kubernetes backup. Their product was built natively for containers, and the software-only solution runs directly on your cluster within its own namespace.

Together, Kasten and Backblaze provide a simple, seamless integration for shared customers, focusing on ease of use.

“Backup and protection are paramount in a world in which data is everything and cyberattacks continue to rise. The Backblaze-Kasten partnership offers the application protection and disaster recovery support companies seek, with flexibility and freedom to choose their preferred storage partner.”
—Gaurav Rishi, VP of Product, Kasten by Veeam

How to Get Started With Backblaze B2 + Kasten

After creating your Backblaze B2 account, check out our Quickstart Guide for Kasten K10 and Backblaze B2 to create a Kasten namespace in your cluster and configure Backblaze B2 as a storage destination.

Customers can test the Kasten solution for free with clusters made up of 10 nodes or less. And Backblaze covers the first 10GB stored.

Data Protection That Scales With You

Kubernetes was built to provide scalability, giving businesses the flexibility to manage and optimize resources. Through this partnership, customers now have storage that matches that flexibility in Backblaze B2. With Backblaze, customers are able to scale their application backups as their applications scale.

Interested in learning more? Join us for a webinar on February 2, 2022 at 10 a.m. PST to discover how to add seamless ransomware protection to your Kubernetes environments—stay tuned for more details and a link to register. Or, if you are ready to future-proof your application, click here to get started today.

The post Announcing Kubernetes Backup and Ransomware Protection via Kasten and Backblaze appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Five Ways to Use Object Lock Immutability

Post Syndicated from Molly Clancy original https://www.backblaze.com/blog/five-ways-to-use-object-lock-immutability/

If you want to understand how Object Lock immutability works, think of a valuable necklace sitting behind a window. If someone really wants that necklace, they will find a way to break that window. Reinforce the glass, add a silent alarm, none of it will change the fact that thieves can find a way.

With Object Lock immutability, there is no window. Instead, picture a fully realistic holographic representation of the necklace. You can still see your necklace, you can still enjoy its sparkle, but nothing anyone does to that hologram can alter the actual physical necklace.

Object Lock immutability works in a similar fashion, at least metaphorically speaking. (Object Lock doesn’t create a facsimile, per se, but it does protect objects from being manipulated, copied, encrypted, changed, or deleted for as long as the lock is set.) And it protects something far more valuable than some trinket: It protects your data.

In this post, learn about five different ways you can use Object Lock as well as some tips for using Object Lock effectively.

What Is Object Lock Immutability?

In functional programming, immutability is a characteristic of an object whose state cannot be changed after it has been created. Conversely, mutable objects are variable. But what does all that mean when it comes to Object Lock? By creating a model in which an individual object (i.e., a unit of data that contains all of the bytes that constitute what you would typically think of as a “file”) remains static and unchangeable, Object Lock immutability prevents important files from being deleted, corrupted, or otherwise damaged. Your files can, however, be freely accessed, giving you free rein to view important documents. This Write Once, Read Many (WORM) model is the cornerstone of Object Lock immutability.

Those of us above a certain age may recall the days when data was regularly transferred via floppy disc. Back in those dark days of dial-up, there was a simple plastic switch on each floppy disc that marked the disc as read-only or read-and-write. If the switch was flipped, the data on the disc could be read or transferred at will, but it was a one-way street. You were unable to alter the original data stored on the disc.

Object Lock Immutability serves the same function as that plastic switch, only it operates entirely within the code of your storage software. You can view, share, and open files at will. But the contents of that file cannot be changed.

Object Lock Immutability Use Cases

With the right approach, Object Lock immutability can be used to solve a few problems, including:

  • Aiding recovery from ransomware attacks.
  • Migrating from an LTO tape system.
  • Supporting records retention requirements.
  • Protecting your company during lawsuits.
  • Enhancing version control during software development.

Aid Recovery From Ransomware Attacks

Ransomware attacks are a major challenge for many businesses. In our research on the true cost of ransomware, we found that the cost of these incidents can exceed $1 million! That’s the bad news.

The good news is advanced planning can make it easier to recover from attacks. Specifically, preserving data backups with Object Lock is a helpful way to speed up your recovery from ransomware attacks. For example, you might decide to make daily backups of your most critical files and retain those backups for three months. In this case, you would have plenty of options for how to recover from an attack.

To achieve consistent security protection, we recommend integrating Object Lock into your IT security policy. For further guidance on how to do this, see our post, “How to Add Object Lock to Your IT Security Policy.”

Migrating From an LTO Tape System

Until recently, the most common way to protect data from being overwritten, corrupted, or deleted was by creating an “air gap” by way of LTO tapes. Under this system, sensitive files would be backed up to a tape and then physically removed from the drive. This created an effective shield of literal, physical air around the data by removing it entirely from your network, but it came at a cost.

Tape systems require an up-front investment and require ongoing maintenance, management, and eventual replacement. With Object Lock immutability as part of your cloud backup solution, the entire process of creating an air gap happens virtually, saving time and money while accomplishing the same goal. You can calculate the savings between cloud storage and tape yourself.

Support Records Retention Requirements

Your company probably has a variety of records retention requirements to fulfill. For example, the finance department likely has to retain records in case you are audited by tax authorities. In addition, your contracts with customers may expect you to retain records for a specific project for a set period. Once you determine which records need to be retained, Object Lock can preserve the records so they cannot be modified or deleted for the required duration. Object Lock means accidental deletion of records is much less likely.

Protect the Company’s Interests During Lawsuits

Lawsuits and disputes are a fact of life in today’s environment, but there are steps you can take to reduce the impact and expense associated with them.

By applying Object Lock, your company will be better able to navigate the challenges of a lawsuit. You can focus on the substance of the dispute rather than spending endless hours answering questions about your data integrity.

Enhance Version Control During Software Development

New versions of files are created on a nearly constant basis during software development projects. Some software projects release new versions every day or every week. With many different software versions on your plate, there is a risk your team might get disorganized. Now, imagine if a new release of your software ends up having a serious security or performance flaw. In that case, rolling back to the previous version may save a tremendous amount of time and energy.

By using Object Lock on previous versions of your software, you can have confidence in your ability to access previous versions. For companies that produce custom software for clients, enhancing version control through Object Lock may be helpful for other reasons. In the event of a problem, a client might ask for access to earlier versions of the software. Preserving earlier versions of your software development with Object Lock makes it easier to respond to such requests.

Tips for Using Object Lock Immutability Effectively

As with any technology, achieving optimal results from Object Lock requires a thoughtful, guided approach. From a technical standpoint, there is no limit to how much data you can protect with Object Lock. However, excessive use of Object Lock may consume a significant amount of your data storage resources, negating any time and cost savings you’ve achieved.

Altering the amount of time an object is placed in Object Lock is just one way to ensure you’re getting the most out of this technology. Others include:

  • Reviewing Older Object Lock Files: You might find that you rarely need to access Object Lock-protected data that is older than six months. Obviously, this amount of time will vary greatly depending on your needs, but it’s important to make sure you’re not spending resources protecting files that don’t need the extra protection. Depending on what you find, you may want to adjust guidance to employees accordingly.
  • Ensuring Consistency: To achieve more consistent usage of Object Lock immutability, start by clarifying your expectations in a company policy. This could be as simple as a checklist document explaining when and how to use Object Lock or an appendix to your IT security policy. In addition, ask managers to periodically review (e.g., every six months) how Object Lock is used in their departments and provide feedback to employees as needed.

As a concept, Object Lock immutability is fairly easy to understand and even easier to use in protecting your vital data from incursion, corruption or deletion. Beyond simply protecting valuable data from cyber threats, it can create a clear timeline in case of litigation or simplify complicated development projects. By understanding how this tool works and how best to use it, you can secure your data, increase your efficiency, and improve the operation of your cloud storage.

The post Five Ways to Use Object Lock Immutability appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

What Is the Public Cloud?

Post Syndicated from Molly Clancy original https://www.backblaze.com/blog/what-is-the-public-cloud/

The wonderful thing about technology is that it’s always changing. This, of course, is also the frustrating thing about technology. We think we understand it, or at least have a basic grasp of it, and then the rules get completely rewritten. What seemed simple and straightforward is suddenly a morass of technical terms and jargon. Such is the case with cloud concepts.

If you’re looking for a data storage solution, it’s sometimes helpful to revisit the fundamentals to ensure you find the solution that works best for you. Today, we’re sharing a few helpful guidelines to cut through any confusion around the public cloud, including how it’s different from a private cloud, when it makes sense to use one or both, and how to go about choosing a cloud provider.

What Is the Public Cloud?

Essentially, public cloud storage operates like a self-storage unit, only one that holds your personal files instead of old lamps that you’re sure you’re going to donate or part with in a garage sale someday. The difference being (apart, obviously, from cloud storage being virtual) that cloud storage lets you pay as you go, so you only pay for the amount of storage you need.

In researching public cloud storage, you may come across other data storage terms such as “private cloud” and “on-premises.” We’ll briefly define these terms to help you navigate the options available to you.

  • Private cloud. As the name implies, a private cloud is similar to a public cloud, but specifically designed for a single user or business. Think of a private cloud like one of those restaurant booths that’s permanently reserved for a VIP—no other customer can use that space. As a result, private cloud services can be more expensive than public clouds. Private clouds are typically, but not always, on-premises. Some companies choose to outsource their private cloud infrastructure to a data center, but the key defining factor is that the servers and network are dedicated to one company or “tenant” rather than shared with other tenants as in a public cloud.
  • On-premises data storage. Before cloud services were created, many organizations stored their data in corporate data centers built on location or “on-premises.” Since these facilities are owned and controlled by the company that built them, they offer a very high level of security. On-premises data storage is also sometimes referred to as a private cloud, but private clouds don’t necessarily have to be on-premises anymore—they can be anywhere as long as they’re dedicated to one tenant.

There are several advantages to using a public cloud service for data storage. You generally only pay for the storage you use. Unlike an on-premises data center, there are usually no significant up-front costs to purchase hardware and plan for ongoing administration. In addition, a public cloud data storage solution does not take up physical space in your office.

Private Cloud vs. Public Cloud: When Does It Make Sense to Use Them?

To help you decide between public vs. private cloud data storage, let’s recap the critical differences first. A public cloud service is usually open to any customer, while a private cloud is dedicated to select users. In addition, a private cloud service is usually operated on-premises.

There are three significant differences between private cloud and public cloud.

  • Tenancy. A private cloud is dedicated to one tenant whereas a public cloud is open to many tenants. In a public cloud, tenants share the same physical hardware resources as other tenants, and their individual instances are virtually separated so that one tenant can’t access another’s data.
  • Approach to security. A private cloud is dedicated exclusively to your needs. You own and manage it. As such, you have more control over the physical security of your servers as well as your private networks. In a public cloud, you trust the provider to handle some of the security for you.
  • Affordability. A public cloud service is defined to achieve mass economies of scale in order to offer affordable prices. A private cloud requires up-front investment in hardware and networking equipment and ongoing maintenance and upgrade costs.

Ultimately, it comes down to the solution that best fits your company’s needs for privacy and security balanced against your budgetary constraints. The benefit of a private cloud is more important in some industries than others. For example, in instances where there might be national security concerns—like national defense or government contractors—or a greater need for privacy like healthcare or financial services, a firm may opt to use a private cloud service located on-site for certain sensitive data.

Sometimes, however, your needs will extend beyond simply using either a private or a public cloud service. When that happens, you need a hybrid cloud.

Hybrid Cloud: When Does It Make Sense to Use Both?

As we noted, depending on your data storage needs and industry, you may decide to use a mix of public cloud and private cloud providers, otherwise known as a hybrid cloud. For example, in data-heavy industries like media and entertainment, a hybrid cloud approach offers a powerful solution. Using a hybrid solution means staff have fast access to large files for day-to-day work on a private, on-premises cloud, combined with scalable, affordable storage in a public cloud for archives and remote collaboration. Similarly, in industries that handle sensitive data, a private cloud could be used to have more control over security, while less sensitive data could be stored in a more affordable public cloud.

Using a combination of public cloud and private cloud data storage gives you the best of both worlds. Private, on-premises storage provides greater ease of access to files and enhanced security, but with the financial trade-off of fixed costs for maintenance and IT assistance. Public storage is more affordable and makes sharing files with outside vendors, subcontractors, and clients simpler.

How to Compare Public Cloud Providers

If you’ve decided a public cloud or a hybrid cloud approach is right for you, your next step is choosing a cloud provider as your trusted partner. As you evaluate different cloud providers, it is helpful to use some standard criteria to measure each provider. Use the following points of comparison to inform your selection process.

  • Ease of use. As with most technology, the more customization you need, the less user-friendly your cloud storage is generally going to be. The end goal of cloud storage is to make your life easier, and if you’re having to stack new skill sets just to make it work, then you’ve officially moved beyond “easy.”
  • Customer support. The insufferable thing about tech headaches is that they have no respect for your schedule. As such, it’s important to check the support package offered by the provider. How easy is it to reach an actual person, not a bot or autoresponder? Will they be able to respond to issues that may arise from your particular setup? The wrong answer could be costly.
  • Data egress. Check the costs and speed of data egress if and when you need to pull all of your data out from the cloud provider. You don’t want to feel locked into a cloud provider because they charge high data transfer fees.
  • Pricing tiers. There is no one-size-fits-all solution when it comes to cloud storage, so make sure you’re not throwing money out the window. Or, into the cloud. Double check that you won’t get locked into a price structure that charges for storage you don’t need. Keep an eye out for things like minimum retention requirements where you’re essentially charged for deleted data for a period of time after you delete it.
  • Security. Lost or compromised data goes far beyond a mere inconvenience. In the worst-case scenario, it can represent a serious security breach and possible legal issue. Some cloud providers offer more security features than others. For example, check if the provider offers Object Lock, a feature that prevents data from being modified, copied, encrypted, changed, or deleted during a set period of time. If a bad actor somehow got access to your account, they wouldn’t be able to tamper with your data. (Even you can’t modify it before the lock period ends!)

Assess Public Cloud Storage for Your Organization

Cloud computing has had a transformative impact on the way we do business. Between the cost savings and the flexibility it gives us in handling our data, it can play a tremendous role in your company’s growth. Now that you’ve learned the basics of public cloud data storage and how it can potentially impact your business, you can make a more informed decision about how you set up your infrastructure and manage your company’s important data.

Are you using a private cloud, a public cloud, or some combination of the two? Let us know in the comments. We’d love to hear how it’s going.

The post What Is the Public Cloud? appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Getting the Most Out of Your NAS

Post Syndicated from Molly Clancy original https://www.backblaze.com/blog/getting-the-most-out-of-your-nas/

Who has the original copy of that report on their machine? Which hard drive has the footage from that shoot a few years ago? Are those photos from our vacation on the laptop, the external hard drive, or the sync service? If you’ve ever asked yourself a question like these, you have felt the pain of digital scatter. In today’s world, with as many devices as we use, it’s almost unavoidable.

Almost. When you start to feel the pain of digital scatter, either at work or at home, take it as a sign that it’s time to look into upgrading your data storage systems. One of the best ways to do that is investing in a NAS device.

As you start exploring more sophisticated data storage options than juggling external drives or managing sync services, understanding what you can do with a NAS system can help your decision making. This post explains what NAS is and all the different ways you can use NAS to supercharge your business or home office setup.

What Is NAS?

Network attached storage, or NAS, is a computer connected to a network that provides file-based data storage services to other devices on the network. It’s primarily used to expand storage capacity and enable file sharing across an organization or across devices in a home.

The primary strength of NAS is how simple it is to set up and deploy. NAS volumes appear to the user as network mounted volumes. The files to be served are typically contained on one or more hard drives in the system, often arranged in a RAID scheme. Generally, the more drive bays available within the NAS, the larger and more flexible storage options you have. The device itself is a network node—much like computers and other TCP/IP devices, all of which maintain their own IP address—and the NAS file service uses the Ethernet network to send and receive files.

NAS devices offer an easy way for multiple users in diverse locations to access data, which is valuable when users are collaborating on projects or sharing information. NAS provides good access controls and security to support collaboration, while also enabling someone who is not an IT professional to administer and manage access to the data via an onboard web server. It also offers good fundamental data resiliency through the use of redundant data structures—often RAID—making multiple drives appear like a single, large volume that can tolerate failure of a few of its individual drives.

How Does RAID Work?
A redundant array of independent disks, or RAID, combines multiple hard drives into one or more storage volumes. RAID distributes data and parity (drive recovery information) across the drives in different ways, and each layout provides different degrees of data protection.

Getting the Most Out of Your NAS: NAS Use Cases

The first two NAS use cases are fairly straightforward. They are exactly what NAS was built for:

  1. File storage and file sharing: NAS is ideal for centralizing data storage for your home or business and making files available to multiple users. The primary benefits of a NAS system are the added storage capacity and file sharing compared to relying on workstations and hard drives.
  2. Local backups and data protection: NAS can serve as a storage repository for local backups of machines on your network. Most NAS systems have built-in software where you can configure automatic backups, including what you back up and when. Furthermore, the RAID configuration in a NAS system ensures that the data you store can survive the failure of one or more of its hard drives. Hard drives fail. NAS helps to make that statement of fact less scary.

But that’s not all NAS can do. With large storage capacity and a whole host of add-ons, NAS offers a lot of versatility. Here are a few additional use cases that you can take advantage of.

Host Business Applications on NAS

Small to medium-sized businesses find NAS useful for running shared business applications like customer relationship management software, human resources management software, messaging, and even office suites. Compared to expensive, server-based versions of these applications, companies can install and run open-source versions quickly and easily on NAS. Some NAS devices may have these features built in or available on a proprietary app store.

Create a Private Cloud With NAS

Most NAS devices give you the ability to access your data over the public internet in addition to accessing it through your private network, essentially functioning as a cloud service. If a NAS device manufacturer doesn’t already have a cloud application built in, there are a number of open-source cloud applications like Nextcloud or ownCloud.

Use NAS to Run Virtual Machines

Virtualization software providers, like VMware, support running their products on NAS. With proper configuration, including potentially adding RAM to your device, you can easily spin up virtual machines using NAS.

Develop and Test Applications on NAS

Many NAS devices offer developer packages, including apps for different programming languages and tools like Docker and Git Server. With these add-ons, you can turn your NAS into your own private lab for developing and testing applications before moving them to a server for production.

Use NAS as a File Server

Although a NAS device is usually not designed to be a general-purpose server (it’s underpowered compared to a file server and comes with less robust access management capabilities), NAS vendors and third parties are increasingly offering other software to provide server-like functionality. For home use or for a small team, higher-end NAS devices can function as a file server.

Manage Security Cameras on NAS

Apps like Synology’s Surveillance Station allow you to set up a security camera system yourself using IP cameras rather than paying for a more expensive enterprise or home system. With a large storage capacity, NAS is perfect for storing large amounts of video footage.

Stream Media With NAS

NAS is a great place to store large media files, and apps like Plex allow you to stream directly to a device or smart TV in your home or business. You can consolidate video or audio files from your devices into one place and stream them anywhere.

Ready to Get Started With NAS?

Understanding the many use cases can help you see where NAS might fit into your business or make file sharing at home easier. There are many ways to make your NAS device work for you and ensure you get the most out of it in the process.

Do you have more questions about shopping for or buying a NAS? Check out our Complete NAS Guide. It provides comprehensive information on NAS and what it can do for your business, how to evaluate and purchase a NAS system, and how to deploy your NAS.

The post Getting the Most Out of Your NAS appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

How Long Do Disk Drives Last?

Post Syndicated from original https://www.backblaze.com/blog/how-long-do-disk-drives-last/

Editor’s Note: This post has been updated since it was originally published in 2013 to provide the latest information and statistics.

How long do disk drives last? We asked that question several years ago, and at the time the answer was: We didn’t know yet. Nevertheless, we did present the data we had up to that point and we made a few of predictions. Since that time, we’ve gone to school on hard disk drive (HDD) and solid-state drive (SSD) failure rates. Let’s see what we’ve learned.

The initial drive life study was done with 25,000 disk drives and about four years of data. Today’s study includes data from over 200,000 disk drives, many of which have survived six years and longer. This gives us more data to review and lets us extend our projections. For example, in our original report we reported that 78% of the drives we purchased were living longer than four years. Today, about 90% of the drives we own have lasted four years and 65% are living longer than six years. So how long do drives last? Keep reading.

How Drives Are Used at Backblaze

Backblaze currently uses over 200,000 hard drives to store our customers’ data. Drives range in size from 4TB to 18TB in size. When added together, we have over two exabytes of hard drive space under management. Most of these drives are mounted in a storage server which accommodates 60 drives, plus a boot drive. There are also a handful of storage servers which use only 45 hard drives. The storage servers consist of Storage Pods (our own homegrown storage servers) and storage servers from external manufacturers. Twenty storage servers are grouped into a Backblaze Vault, which utilizes our own Reed-Solomon erasure coding algorithm to replicate and store customer data across the 20 servers in a Backblaze Vault.

Types of Hard Drives in the Analysis

The hard drives we use to store customer data are standard 3.5 inch drives you can buy online or in stores. The redundancy provided by the Backblave Vault software ensures the data is safe, while allowing us to use off-the-shelf drives from the three primary disk drive manufacturers: Seagate, Western Digital, and Toshiba. The following chart breaks down our current drive count by manufacturer. Note that HGST is now part of Western Digital, but the drives themselves report as HGST drives so they are listed separately in the chart.

Each of the storage servers also uses a boot drive. Besides the obvious function of booting the server, we also use these drives to store log files recording system access and activities which are used for analytics and compliance purposes. A boot drive can be either an HDD or an SSD. If you’re interested, we’ve compared the reliability of HDDs versus SSDs as it relates to these boot drives.

Number of Hard Drives

As stated earlier, we currently have over 200,000 disk drives we manage and use for customer data storage. We use several different disk drive sizes as the table below shows, with over 60% of those drives being 12TB or 14TB in size.

Drive Failure Rates

Before diving into the data on failure rates, it’s worth spending a little time clarifying what exactly a failure rate means. The term failure rate alone is not very useful as it is missing the notion of time. For example, if you bought a hard drive, what is the failure rate of a hard drive that failed one week after you purchased it? What about one year after you purchased it? Five years? They can’t all be the same failure rate. What’s missing is time. When we produce our quarterly and annual Drive Stats reports, we calculate and publish the annualized failure rate (AFR). By using the AFR, all failure rates are translated to be annual so that regardless of the timeframe (e.g., one month, one year, three years) we can compare different cohorts of drives. Along with the reports, we include links to the drive data we use to calculate the stated failures rates.

The Bathtub Curve

Reliability engineers use something called the bathtub curve to describe expected failure rates. The idea is that defects come from three factors: (1) factory defects, resulting in “infant mortality,” (2) random failures, and (3) parts that wear out, resulting in failures after much use. The chart below (from Wikimedia Commons) shows how these three factors can be expected to produce a bathtub-shaped failure rate curve.

When our initial drive life study was done, the Backblaze experience matched the bathtub curve theory. When we recently revisited the bathtub curve, we found the bathtub to be leaking, as the left side of the Backblaze bathtub curve (decreasing failure rate) was much lower and more consistent with the constant failure rate. This can be seen in the chart below which covers the most recent six years worth of disk drive failure data.

The failure rate (the red line) is below 2% for the first three and a half years and then increases rapidly through year six. When we plot a trendline of the data (the blue dotted line, a second order polynomial) a parabolic curve emerges, but it is significantly lower on the left hand side, looking less like a bathtub and more like a shallow ladle or perhaps a hockey stick.

Calculating Life Expectancy

What’s the life expectancy of a hard disk drive? To answer that question, we first need to decide what we mean by “life expectancy.”

When measuring the life expectancy of people, the usual measure is the average number of years remaining at a given age. For example, the World Health Organization estimates that the life expectancy of all newborns in the world is currently 73 years. This means if we wait until all of those new people have lived out their lives in 120 or 130 years, the average of their lifespans will be 73.0.

For disk drives, it may be that all of them will wear out before they are 10 years old. Or it may be that some of them last 20 or 30 years. If some of them live a long, long time, it makes it hard to compute the average. Also, a few outliers can throw off the average and make it less useful.

The number that should be able to compute is the median lifespan of a new drive. That is the age at which half of the drives fail. Let’s see how close we can get to predicting the median lifespan of a new drive given all the data we’ve collected over the years.

Disk Drive Survival Rates

To this day it is surprisingly hard to get an answer to the question “How long will a hard drive last?” As noted, we regularly publish our Drive Stats reports, which lists the AFRs for the drive models we use. While these reports answer the question at what rate disk drives will fail, they don’t tell us how long they will last. Interestly, the same data we collect and use to predict drive failure can be used to figure out the life expectancy of the hard drive models we use. It is all a matter of how you look at the data.

When we apply life expectancy forecasting techniques to the drive data we have collected, we get the following chart:

The life expectancy decreases at a fairly stable rate of 2% to 2.5% a year for the first four years, then the decrease begins to accelerate. Looking back at the AFR by quarter chart above, this makes sense as the failure rate increases beginning in year four. After six years we end up with a life expectancy of 65%. Stated another way, if we bought a hard drive six years ago, there is a 65% chance it is still alive today.

How Long WILL the Hard Drives Last?

What happens to drives when they’re older than six years? We do have drives that are older than six years, so why did we stop there? We didn’t have enough data to be confident beyond six years as the number of drives drops off at that point and becomes composed almost entirely of one or two drive models versus a diverse selection. Instead, we used the data we had through six years and extrapolated from the life expectancy line to estimate the point at which half the drives will have died.

How long do drives last? It would appear a reasonable estimate of the median life expectancy is six years and nine months. That aligns with the minimal amount of data we have collected to date, but as noted, we don’t have quite enough data to be certain. Still, we know it is longer than six years for all the different drive models we use. We will continue to build up data over the coming months and years and see if anything changes.

In the meantime, how long should you assume a hard drive you are going to buy will last? The correct answer is to always have at least one backup and preferably two, keep them separate, and check them often一the 3-2-1 backup strategy. Every hard drive you buy will fail at some point—it could be in one day or 10 years—be prepared.

The post How Long Do Disk Drives Last? appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

NAS and the Hybrid Cloud

Post Syndicated from Molly Clancy original https://www.backblaze.com/blog/nas-and-the-hybrid-cloud/

illustration of a NAS device and cloud storage

Upgrading to network attached storage (NAS) can be a game changer for your business. When you invest in NAS, you get easier collaboration, faster restores, 24/7 file availability, and added redundancy. But you can get an even bigger return on your investment by pairing it with cloud storage. When you combine NAS with a trusted cloud storage provider in a hybrid cloud strategy, you gain access to features that complement the security of your data and your ability to share files both locally and remotely.

In this post, we’ll look at how you can achieve a hybrid cloud strategy with NAS and cloud storage.

What Is Hybrid Cloud?

A hybrid cloud strategy uses a private cloud and public cloud in combination. To expand on that a bit, we can say that the hybrid cloud refers to a cloud environment made up of a mixture of typically on-premises, private cloud resources combined with third-party public cloud resources that use some kind of orchestration between them. A private cloud doesn’t necessarily need to live on-premises—some companies rent space in a data center to host the infrastructure for their private cloud—the important defining factor is that a private cloud is dedicated to only one “tenant” or organization.

illustration of hybrid cloud - private cloud sending data to public cloud
A hybrid cloud strategy combines a private cloud with a public cloud.

In this case, your NAS device serves as the on-premises private cloud, as it’s dedicated to only you or your organization, and then you connect it to the public cloud.

What Are the Benefits of Hybrid Cloud?

A hybrid cloud model offers a number of benefits, including:

  • Off-site backup protection.
  • Added security features.
  • Remote sync capabilities.
  • Flexibility and cost savings.

Hybrid Cloud Benefit 1: Off-site Backup Protection

To start with, cloud storage provides off-site backup protection. This aligns your NAS setup with the industry standard for data protection: a 3-2-1 backup strategy—which ensures that you have three copies of your data on two different media (read: devices) with one stored off-site. When using NAS and the cloud in a hybrid strategy, you have three copies of your data—the source data and two backups. One of those backups lives on your NAS and one is stored off-site in the cloud. In the event of data loss, you can restore your systems directly from the cloud even if all the systems in your office are knocked out or destroyed.

Hybrid Cloud Benefit 2: Added Security Features

Data sent to the cloud is encrypted in-flight via SSL, and you can also encrypt your backups so that they are only openable with your team’s encryption key. The cloud can also give you advanced storage options for your backup files, like Object Lock. Object Lock allows you to store data using a Write Once, Read Many (WORM) model. Once you set Object Lock and the retention timeframe, your data stored with Object Lock is unchangeable for a defined period of time. You can also set custom data lifecycle rules at the bucket level to help match your ideal backup workflow.

Hybrid Cloud Benefit 3: Remote Sync Capabilities

Cloud storage provides valuable access to your data and documents from your NAS through sync capabilities. In case anyone on your team needs to access a file when they are away from the office, or as is more common now, when your entire team is working from home, they’ll be able to access the files that have been synced to the cloud through your NAS’s secure sync program. You can even sync across multiple locations using the cloud as a two-way sync to quickly replicate data across locations. For employees collaborating remotely, this helps to ensure they’re not waiting on the internet to deliver critical files: They’re already on-site.

What’s the Difference Between Cloud Sync, Cloud Backup, and Cloud Storage?

Sync services allow multiple users across multiple devices to access the same file. Backup stores a copy of those files somewhere remote from your work environment, usually in an off-site server—like cloud storage. It’s important to know that a “sync” is not a backup, but they can work well together when properly coordinated.

Hybrid Cloud Benefit 4: Flexibility and Cost Savings

Additionally, two of the biggest advantages of the hybrid cloud are flexibility and cost savings. Provisioning an additional device to store backups and physically separating it from your production data is time consuming and costly. The cloud eliminates the need to provision and maintain additional hardware while keeping your data protected with a 3-2-1 strategy, and it can be scaled up or down flexibly as needed.

With NAS on-site for fast, local access combined with the cloud for off-site backups and storage of less frequently used files, you get the best of both worlds.

How to Set Up a Hybrid Cloud With NAS

Some cloud providers are already integrated with NAS systems. (Backblaze B2 Cloud Storage is integrated with NAS systems from Synology and QNAP, for example.) Check if your preferred NAS system is already integrated with a cloud storage provider to ensure setting up cloud backup, storage, and sync is as easy as possible.

Your NAS should come with a built-in backup manager, like Hyper Backup from Synology or Hybrid Backup Sync from QNAP. Once you download and install the appropriate backup manager app, you can configure it to send backups to your preferred cloud provider. You can also fine-tune the behavior of the backup jobs, including what gets backed up and how often.

Now, you can send backups to the cloud as a third, off-site backup and use your cloud instance to access files anywhere in the world with an internet connection.

Wondering If NAS Is Right for You?

Our Complete NAS Guide provides comprehensive information on NAS and what it can do for your business, how to evaluate and purchase a NAS system, and how to deploy your NAS. Download the guide today for more on all things NAS.

The post NAS and the Hybrid Cloud appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Will Containers Replace Virtual Machines?

Post Syndicated from Molly Clancy original https://www.backblaze.com/blog/will-containers-replace-virtual-machines/

Virtualization fundamentally changed the way the data center operates, and it wasn’t all that long ago. VMware released the ESX Server 1.5, its first hypervisor and arguably launched mainstream adoption of VMs in the enterprise, in 2002. To this elder millennial, it feels like just yesterday(ish).

The technology revolutionized infrastructure utilization—you could run multiple operating systems on the same hardware, maximizing your hardware resources. Then, containers—VMs’ younger, sleeker Gen Z cousin—came on the scene in a big way with Docker’s release in 2013, arguably the product that spurred mainstream adoption of containers. Containers allow multiple, isolated applications to share the same operating system kernel.

While they work differently, the two technologies share enough similarities that it begs the question—will containers replace virtual machines?

Today, we’re weighing in on the debate.

VMs vs. Containers: A Recap

VMs are software-defined computers. They run atop hypervisor software that allows multiple operating systems to share the same underlying physical hardware and operate in isolation from each other. The hypervisor virtualizes the underlying hardware, and each VM runs its own operating system (OS).

Before VMs, a single server could only support a single OS. If you wanted to run business applications with different OS requirements on the same server, or you wanted to run those applications without them interfering with each other, well, you couldn’t. Each business application you wanted to run—say, email or your CRM—had to have its own (usually over-provisioned, underutilized) set of hardware. After VMs, a single server could host multiple operating systems and run multiple applications. They drastically reduced the amount companies had to spend on hardware and hardware management.

Containers are also software-defined computing instances, but they run atop a shared, host OS. They package software into portable, executable units with all of the necessary dependencies to run anywhere—on a laptop, on a server, in the cloud, etc. The table below outlines the differences between VMs and containers.

VMs Containers
Heavyweight. Lightweight.
Limited performance. Native performance.
Each VM runs in its own OS. All containers share the host OS.
Hardware-level virtualization. OS virtualization.
Startup time in minutes. Startup time in milliseconds.
Allocates required memory. Requires less memory space.
Fully isolated and hence more secure. Process-level isolation, possibly less secure.

Without having to lug around all that OS code, containers are lighter weight (megabytes vs. VMs’ gigabytes) and start up faster than VMs. You can run two to three times as many applications on a single server with containers than you can with VMs.

Based on that alone, one could make the case that containers are the future and VMs are old news.

So, Will Containers Replace VMs?

The case seemed too easy, and we weren’t satisfied with basing a prediction on that alone. It’s just not that simple. To present a structured debate, we’ll look at the arguments for and against. In one corner: the case for VMs. In the other corner: the case for containers.

The Case for VMs

Even though containers are smaller and faster, it still might be a bridge too far to predict they’ll kill VMs forever. Here’s why:

  1. Familiarity and Maturity: While the tech industry is often fast to adopt the next, new thing, VMs are well-established in many organizations. Most businesses have entrenched processes, procedures, and teams in place to manage VM deployments, including backups and monitoring. “The way we’ve always done it” isn’t always the best reason to keep doing something, but there’s the “if it ain’t broke, don’t fix it” argument to contend with as well. If a monolithic application works well in a VM environment, there has to be a good reason to go through the effort of refactoring it to work in a container environment and sometimes, the effort isn’t worth the reward.
  2. Need for Full OS Capabilities: Using a VM with all OS services working together as one machine is sometimes preferable to a more orchestrated “Docker compose” or Kubernetes cluster with unnecessary complexity, especially for some complex applications.
  3. Heterogenous Environments: Need to run more than one OS? That’s what VMs are made for. Containers share the same OS kernel, and thus require a homogenous environment. There’s an argument to be made that enterprise data centers are moving toward more homogenous environments to standardize for more operational efficiency, but it hasn’t happened yet, broadly speaking. Many organizations still need to run more than one OS and will for the foreseeable future.
  4. Security: Container security is a flag that VM loyalists like to fly. The argument is that containers are more vulnerable because they share host OS resources whereas VMs include the OS within their virtual walls. If a container is attacked, it could infect other containers that share the host OS. If a VM is attacked, there’s an additional barrier between the VM, the host server, and other VMs on it.
  5. It’s Not an Either/or, But a Both/and: You can run containers on VMs, thus allowing companies with established VM procedures to adopt containers. They can take advantage of containers’ speed and agility without abandoning VMs wholesale.

The Case for Containers

But, containers have a lot of advantages in their favor. What makes them contenders to replace VMs? Here are a few counterpoints:

  1. Speed and Portability: Containers start up in just seconds compared to minutes for VMs, but the speed advantages containers bring go beyond start-up time. Because they solve the problem of environment consistency, developers spend far less time configuring environments and dealing with bugs. Container engines like Docker also have libraries of container images developers can use to speed up their work. And projects like the Open Container Initiative have been established to standards for containers, streamlining deployment.
  2. Scalability: With container orchestrators like Kubernetes or Cycle.io, containers scale automatically based on traffic demands. They can replicate container images when needed and spin them down when not. Containers’ small size and startup speed means they can scale faster than VMs.
  3. Modularity: Containers revolutionized application development such that applications could be developed as individual microservices loosely coupled together. This development approach enabled DevOps and Ci/CD practices, substantially speeding up application delivery. As one of our senior software engineers put it, the ability to lego-stack an entire environment is really powerful. Containers’ small size and portability mean applications can be built in small component parts and those parts can be stacked together to build a complete application.
  4. Ease of Updates: Updating and redeploying one small part of your application built as a microservice in a container is much easier to manage than a full-featured OS with update and downtime needs.
  5. OS Abstraction: Because containers abstract the operating system, they solve some virtualization problems more easily than VMs. Virtualization can be tricky in MacOS and M1 operating systems, and containers get virtual environments running easily on those operating systems.

The Last Word

Will containers replace VMs in development environments? They already have. But will containers replace VMs wholesale? We’d argue no.

Developers aren’t going back to building monolithic applications any time soon. Containers enable them to leave environment inconsistencies in the past and focus more on building code, creating better software, and creating it faster. With more pressure than ever to do just that, containers are firmly cemented as the dominant virtualization technology for developers.

But, it’s important to differentiate between containers as an application delivery mechanism versus an infrastructure mechanism. VMs are slower, less portable, and heavy, but they are still the go-to technology when it comes to provisioning infrastructure in an enterprise and running legacy, monolithic applications. For example, while containers have become the technology of choice in app development, it’s less likely that IT teams in other industries, say education, will be as interested in spending the time to refactor their custom-made student information system into a microservices architecture. Nor could they feasibly substitute containers for the thousands of VMs they assign to students.

Final answer? Both containers and VMs have a place in the modern data center.

Do you disagree? Have at it in the comments.

The post Will Containers Replace Virtual Machines? appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Why the World Needs Lower Egress Fees

Post Syndicated from original https://www.backblaze.com/blog/why-the-world-needs-lower-egress-fees/

Imagine a world where you can move data freely wherever you want, a world where you can use best-of-breed cloud providers to get the ideal service, optimize cost for your overall infrastructure, and build redundancy into your cloud platform—all without fear of lock-in.

In short, a world with lower egress fees.

Are lower egress fees a cure-all for the world’s problems? No, but it sure would make life easier for millions of companies. We believe in a multi-cloud world powered by low or zero egress fees. Today, we’re digging into what that means.

The Need for Multi-cloud

As companies seek to avoid vendor lock-in, get the best from different clouds, and provide redundancy for mission critical data, multi-cloud deployments are becoming more commonplace. According to Frost & Sullivan’s 2020 Global Cloud User Survey, respondents indicated growing from 43% adoption of multi-cloud in 2020 to 84% planned adoption of multi-cloud in 2022.

Multi-cloud deployments represent the future of cloud computing, and unlocking that potential means low or no egress fees.

The Case Against High Egress Fees

Cloud companies charge egress fees when customers want to move their data out of the provider’s platform. Existing diversified cloud providers such as Amazon charge egregiously high egress fees that make it expensive for customers to use multi-cloud infrastructures and therefore lock customers in to their services. These walled gardens hamper innovation and long-term growth.

The bottom line: high egress fees are a direct impediment to a multi-cloud future.

The Benefits of a Multi-cloud Future

The demand for multi-cloud infrastructure is rising and the case against high egress fees is gaining traction because a robust, multi-cloud system offers clear benefits:

  • Best-of-breed providers: Specialized cloud providers are focusing on key parts of the tech stack and offering the best services. Multi-cloud deployments enable customers to pick the best parts for their environment.
  • Disaster recovery: One of the biggest advantages of operating a multi-cloud environment is to achieve redundancy and plan for disaster recovery in a cloud-native deployment. If one public cloud suffers an outage, customers have mechanisms in place to direct their applications to failover to a second public cloud.
  • Cost optimization: Challenger companies (Shameless plug: that’s us!) can offer better pricing through innovation and a focus on specialized use cases than the traditional diversified clouds—Amazon, Google, and Microsoft.
  • Freedom from vendor lock-in: Many companies initially adopted one of the existing diversified cloud providers like Amazon or Google because they were the only game in town, but later felt restricted by their closed systems. Companies like Amazon seek to lock customers in with proprietary services. Adopting a multi-cloud infrastructure with interoperable providers gives these customers more negotiating power and control over their cloud deployments.
  • Unconflicted partners: Some customers find themselves in direct competition with companies like Amazon. Shifting their cloud usage to independent, unconflicted partners in a multi-cloud deployment means they don’t have to worry about how their data might be used against them, and they’re not spending money to support a direct competitor.

One Small Step in the Right Direction

After a recent AWS announcement, we’re one small, incremental step closer to that future where egress fees don’t hold you back. Last week, AWS announced an expansion of their free tier for data transfer to the internet from 1GB per month to 100GB per month. Fees above the free tier will remain the same, starting at $0.09/GB for 100GB to 9.999TB of data transfer out per month.

To understand this change in action, here’s a quick example. Previously, a customer egressing 100GB per month was paying $9 per month in egress fees. Now, they’ll pay nothing, and customers egressing above 100GB per month will get a $9 per month discount. (It’s worth noting that egress begins to be discounted beyond 9.999TB per month, but you’d be paying $900/month before that discount would take effect.)

Groundbreaking, it is not. But it’s a move that indicates AWS is bowing to industry pressure that calls their egress fees exactly what they are—exorbitant and egregious.

While it’s not going to be a payday for most customers, we still believe this is a good thing. Even a small reduction in fees could enable a customer to consider a multi-cloud solution when they hadn’t before. For example, let’s say a customer stores 1TB of data in Amazon S3, and they egress out 100GB per month. Egress used to cost them $9, now it’s free. That $9 per month savings could be used to store a backup copy of their 1TB of data (And then some!) in Backblaze B2 Cloud Storage, allowing them to achieve double the redundancy without changing their overall cloud spend. They get the peace of mind of knowing they’re better protected against data loss for the same cost.

Lower Egress = A Better, Multi-cloud World

At Backblaze, we’ve always believed it’s good and right to enable customers to readily use their data. Our egress fees stand at just $0.01/GB, and we’re proud to be a founding member of Cloudflare’s Bandwidth Alliance. Backblaze also waives egress fees to Fastly, Vultr, and other forward-thinking cloud companies, and we cover the cost of data migrations above 10TB through our Cloud to Cloud Migration service.

Lower egress fees support a flexible, open ecosystem on which customers are free to build multi-cloud deployments that level up their businesses. As a leading independent cloud storage platform, it’s part of our commitment to simplify the process of storing and using data. We’re just happy to see companies like Amazon starting to follow suit.

The post Why the World Needs Lower Egress Fees appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

What Are Microservices?

Post Syndicated from Molly Clancy original https://www.backblaze.com/blog/what-are-microservices/

Back in 2008, Netflix was facing scaling challenges: service outages, database corruption, a three-day lapse in DVD shipments. (Remember when Netflix still shipped DVDs?) Netflix solved these problems by refactoring their monolithic application to avoid the single points of failure that caused these issues. They implemented a microservices architecture before the term “microservices” even existed, making them pioneers in the field.

Today, almost all of the most popular applications—Uber, Amazon, Etsy—run in a microservices environment. If you’ve ever wondered what that means, you’re not alone. In this post, we’re digging into this popular method for developing web applications: the benefits, drawbacks, and considerations for choosing a cloud provider to support your microservices approach.

First, Some History: Monolithic Software Development

How would you develop a large, complex software system before the age of microservices? For developers who learned their craft in the dot com boom, it meant a large and complex development process, with tightly interlocking subsystems, a waterfall development model, and an extensive QA phase. Most of the code was built from scratch. There was a lot of focus on developing extremely reliable code, since it was very difficult and expensive to update products in the field. This is how software was developed for many years. The approach works, but it has some major issues. It’s slower, and it tends to produce complex software monoliths that are tough to maintain and upgrade.

What Are Microservices?

Current software engineering practices encourage a DevOps model using small, reusable units of functionality, or microservices. Microservices run essentially as a standalone process, carrying whatever information or intelligence they need to do their job. Their interfaces are limited and standardized to encourage reuse. This restricted access to internal logic very effectively implements the information-hiding practices that prevent other code from “peeking inside” your modules and making internal tweaks that destabilize your code. The result is (hopefully) an assortment of tools (databases, web servers, etc.) that can be snapped together like Legos™ to greatly reduce the development effort.

Microservices: Pros and Cons

This newer approach has many advantages:

  • It encourages and supports code reuse. Developers usually have access to an extensive library of microservices that can easily be plugged into their application.
  • It enforces logical module isolation to simplify the architecture and improve reliability. This makes initial design, implementation, product updates, enhancements, and bug fixes much easier and less error-prone.
  • It enables much more nimble development and delivery techniques, like DevOps and Agile. It’s much easier to add new functionality when you can just tweak a small piece of code (not worrying about potential invisible linkages to other modules), and then instantly push out an update to your users. You can improve your time to market by getting a “good enough” solution out there, and improve it to “ultimate” through ongoing updates.
  • It’s inherently scalable. By implementing microservices with containers, you can use an orchestration tool like Kubernetes or Cycle.io to handle scaling, failover, and load balancing. Each microservice component is independently scalable—if one part of your application experiences high load, it can respond to that demand without impacting other components.

However, like anything else, there are drawbacks. For one, moving from a monolithic architecture to a microservices architecture requires not just a change to the way your software is built but also the way your software team functions. And while a microservices architecture is more nimble, it also introduces complexity where none may have existed before in a monolithic deployment.

Microservices Use Cases

Microservices, and especially the containers and orchestration used to structure them, enable a number of different use cases that could benefit your organization, including:

  • Making a legacy application cloud-ready. If you want to modernize a legacy application and move it to the cloud, taking a microservices approach to your architecture is helpful. Refactoring a monolithic application and moving it to the cloud allows you to achieve the cost savings associated with the cloud’s pay-as-you-go model.
  • Cloud-native development. Similarly, if you want to take a cloud-first approach, it can help to start with a microservices architecture as it will serve you well later as your application scales.
  • Moving to DevOps. Microservices as an architectural model lend themselves to, and in many cases require, a change to a DevOps or Agile operational model. If you’re interested in moving from Waterfall to Agile development, microservices go hand in hand.
  • Running big data applications. Applications that ingest and process large amounts of data benefit from being broken down into microservices where each step in the data processing pipeline is handled independently.

When NOT to Use Microservices

The microservices model can introduce unnecessary complexity into an otherwise simple solution. If you have a simple or monolithic implementation and it’s meeting your needs just fine, then there is no need to throw it away and microservice-ize it. But if (when) it becomes difficult to update or maintain it, or when you’re planning new development, consider a more modular architecture for the new implementation.

Microservices + Cloud Storage: Considerations for Choosing a Cloud Provider

One impact of moving to a containerized, microservices model is the need for persistent storage. The state of a container can be lost at any time due to situations like hardware/software crashes or spinning down excess containers when load drops. The application running in the container should store its state in external (usually cloud) storage, and read the current state when it starts up.

Thus, administrators should carefully consider different providers before selecting one to trust with their data. Consider the following factors in an evaluation of any cloud provider:

  • Integrations/partner network: One of the risks of moving to the cloud is vendor lock-in. Avoid getting stuck in one cloud ecosystem by researching the providers’ partner network and integrations. Does the provider already work with software you have in place? Will it be easy to change vendors should you need to? Consider the provider’s egress fees both in general and between partners, especially if you have a high-bandwidth use case.
  • Interoperability and API compatibility: Similarly, make sure the cloud provider you’re considering favors an open ecosystem and offers APIs that are compatible with your architecture.
  • Security: What protections does the provider have against ransomware and other data corruption? Does the provider include features like Object Lock to make data immutable? Protection like this is recommended considering the rising threat of ransomware attacks.
  • Infrastructure as Code capability: Does the provider enable you to use infrastructure as code (IaC) to provision storage automatically? Using IaC to provision storage enables you to scale your storage without manually managing the process.
  • Pricing transparency: With varying data retention requirements, transparent pricing tiers will help you budget more easily. Understand how the provider prices their service including fees for things like egress, retention minimums, and other fine print. Look for backup providers that offer pricing compatible with your organization’s needs.

Are You Using Microservices?

Are you using microservices to build your applications? According to a TechRepublic survey, 73% of organizations have integrated microservices into their application architectures. If you’re one of them, we’d love to know how it’s going. Let us know in the comments.

The post What Are Microservices? appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

How to Connect Your QNAP NAS to Backblaze B2 Cloud Storage

Post Syndicated from Troy Liljedahl original https://www.backblaze.com/blog/guide-qnap-backup-b2-cloud-storage/

Network attached storage (NAS) devices are a popular solution for data storage, sharing files for remote collaboration purposes, syncing files that are part of a workflow, and more. QNAP, one of the leading NAS manufacturers, makes it incredibly easy to backup and/or sync your business or personal data for these purposes with the inclusion of its application, Hybrid Backup Sync (HBS). HBS consolidates backup, restoration, and synchronization functions into a single application.

Protecting your data with a NAS is a great first step, but you shouldn’t stop there. NAS devices are still vulnerable to any kind of on-premises disaster like fires, floods, and tornados. They’re also not safe from ransomware attacks that might hit your network. To truly protect your data, it’s important to back up or sync to an off-site cloud storage destination like Backblaze B2 Cloud Storage. Backblaze B2 offers a geographically distanced location for your data for $5/TB per month, and you can also embed it into your NAS-based workflows to streamline access across multiple locations.

Read on for more information on whether you should use backup or sync for your purposes and how to connect your QNAP NAS to Backblaze B2 step-by-step. We’ve even provided videos that show you just how easy it is—it typically takes less than 15 minutes!

➔ Download Our Complete NAS Guide

Should I Back Up or Sync?

It’s easy to confuse backup and sync. They’re essentially both making a copy of your data, but they have different use cases. It’s important to understand the difference so you’re getting the right protection and accessibility for your data.

Check out the table below. You’ll see that backup is best for being able to recover from a data disaster, including the ability to access previous versions of data. However, if you’re just looking for a mirror copy of your data, sync functionality is all you need. Sync is also useful as part of remote workflows: you can sync your data between your QNAP and Backblaze B2, and then remote workers can pull down the most up-to-date files from the B2 cloud.

A table comparing Backup vs. Sync

A table comparing Backup vs. Sync.

Because Hybrid Backup Sync provides both functions in one application, you should first identify which feature you truly need. The setup process is similar, but you will need to take different steps to configure backup vs. sync in HBS.

How to Set Up Your Backblaze B2 Account

Now that you’ve determined whether you want to back up or sync your data, it’s time to create your Backblaze B2 Cloud Storage account to securely protect your on-premises data.

If you already have a B2 Cloud Storage account, feel free to skip ahead. Otherwise, you can sign up for an account and get started with 10GB of free storage to test it out.

Ready to get started? You can follow along with the directions in this blog or take a look at our video guides. Greg Hamer, Senior Technical Evangelist, demonstrates how to get your data into B2 Cloud Storage in under 15 minutes using HBS for either backup or sync.

Video: Back Up QNAP to Backblaze B2 Cloud Storage with QNAP Hybrid Backup Sync

Video: Sync QNAP to Backblaze B2 Cloud Storage with QNAP Hybrid Backup Sync

How to Set Up a Bucket, Application Key ID, and Application Key

Once you’ve signed up for a Backblaze B2 Account, you’ll need to create a bucket, Application Key ID, and Application Key. This may sound like a lot, but all you need are a few clicks, a couple names, and less than a minute!

  1. On the Buckets page of your account, click the Create a Bucket button.
  2. An screenshot of the B2 Cloud Storage Buckets page.

  3. Give your bucket a name and enable encryption for added security.
  4. An image showing the Create a Bucket page with security features to be enabled.

  5. Click the Create a Bucket button and you should see your new bucket on the Buckets page.
  6. An image showing a successfully created bucket.

  7. Navigate to the App Keys page of your account and click Add a New Application Key.
  8. Name your Application Key and click the Create New Key button. Make sure that your key has both read and write permissions (the default option).
  9. Your Application Key ID and Application Key will appear on your App Keys page. Important: Make sure to copy these somewhere secure as the Application Key will not appear again!

How to Set Up QNAP’s Hybrid Backup Sync to Work With B2 Cloud Storage

To set up your QNAP with Backblaze B2 sync support, you’ll need access to your B2 Cloud Storage account. You’ll also need your B2 Cloud Storage account ID, Application Key, and bucket name—all of which are available after you log in to your Backblaze account. Finally, you’ll need the Hybrid Backup Sync application installed in QTS. You’ll need QTS 4.3.3 or later and Hybrid Backup Sync v2.1.170615 or later.

To configure a backup or sync job, simply follow the rest of the steps in this integration guide or reference the videos posted above. Once you follow the rest of the configuration steps, you’ll have a set-it-and-forget-it solution in place.

What Can You Do With Backblaze B2 and QNAP Hybrid Backup Sync?

With QNAP’s Hybrid Backup Sync software, you can easily back up and sync data to the cloud. Here’s some more information on what you can do to make the most of your setup.

Hybrid Backup Sync 3.0

QNAP and Backblaze B2 users can take advantage of Hybrid Backup Sync, as explained above. Hybrid Backup Sync is a powerful tool that provides true backup capability with features like version control, client-side encryption, and block-level deduplication. QNAP’s operating system, QTS, continues to deliver innovation and add thrilling new features. The ability to preview backed up files using the QuDedup Extract Tool, a feature first released in QTS 4.4.1, allowed QNAP users to save on bandwidth costs.

You can download the latest QTS update here and Hybrid Backup Sync is available in the App Center on your QNAP device.

Hybrid Mount and VJBOD Cloud

The Hybrid Mount and VJBOD Cloud apps allow QNAP users to designate a drive in their system to function as a cache while accessing B2 Cloud Storage. This allows users to interact with Backblaze B2 just like you would a folder on your QNAP device while using Backblaze B2 as an active storage location.

Hybrid Mount and VJBOD Cloud are both included in the QTS 4.4.1 versions and higher, and function as a storage gateway on a file-based or block-based level, respectively. Hybrid Mount enables Backblaze B2 to be used as a file server and is ideal for online collaboration and file-level data analysis. VJBOD Cloud is ideal for a large number of small files or singular massively large files (think databases!) since it’s able to update and change files on a block-level basis. Both apps offer the ability to connect to B2 Cloud Storage via popular protocols to fit any environment, including server message block (SMB), Apple Filing Protocol (AFP), network file sharing (NFS), file transfer protocol (FTP), and WebDAV.

QuDedup

QuDedup introduces client-side deduplication to the QNAP ecosystem. This helps users at all levels save on space on their NAS by avoiding redundant copies in storage. Backblaze B2 users have something to look forward to as well since these savings carry over to cloud storage via the HBS 3.0 update.

Why Backblaze B2?

QNAP continues to innovate and unlock the potential of B2 Cloud Storage in the NAS ecosystem. If you haven’t given B2 Cloud Storage a try yet, now is the time. You can get started with Backblaze B2 and your QNAP NAS right now, and make sure your NAS is synced securely and automatically to the cloud.

The post How to Connect Your QNAP NAS to Backblaze B2 Cloud Storage appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

What Is Kubernetes?

Post Syndicated from Molly Clancy original https://www.backblaze.com/blog/what-is-kubernetes/

Do you remember when “Pokémon Go” came out in 2016? Suddenly it was everywhere. It was a world-wide obsession, with over 10 million downloads in its first week and 500 million downloads in six months. System load rapidly escalated to 50 times the anticipated demand. How could the game architecture support such out-of-control hypergrowth?

The answer: At release time, Pokémon Go was “The largest Kubernetes deployment on Google Container Engine.” Kubernetes is a container orchestration tool that manages resources for dynamic web-scale applications, like “Pokémon Go.”

In this post, we’ll take a look at what Kubernetes does, how it works, and how it could be applicable in your environment.

What Is Kubernetes?

You may be familiar with containers. They’re conceptually similar to lightweight virtual machines. Instead of simulating computer hardware and running an entire operating system (OS) on that simulated computer, the container runs applications under a parent OS with almost no overhead. Containers allow developers and system administrators to develop, test, and deploy software and applications much faster than VMs, and most applications today are built with them.

But what happens if one of your containers goes down, or your ecommerce store experiences high demand, or if you release a viral sensation like “Pokémon Go”? You don’t want your application to crash, and you definitely don’t want your store to go down during the Christmas crush. Unfortunately, containers don’t solve those problems. You could implement intelligence in your application to scale as needed, but that would make your application a lot more complex and expensive to implement. It would be simpler and faster if you could use a drop-in layer of management—a “fleet manager” of sorts—to coordinate your swarm of containers. That’s Kubernetes.

Kubernetes Architecture: How Does Kubernetes Work?

Kubernetes implements a fairly straightforward hierarchy of components and concepts:

  • Containers: Virtualized environments where the application code runs.
  • Pods: “Logical hosts” that contain and manage containers, and potentially local storage.
  • Nodes: The physical or virtual compute resources that run the container code.
  • Cluster: A grouping of one or more nodes.
  • Control Plane: Manages the worker nodes and Pods in the cluster.

You have a few options to run Kubernetes. The minikube utility launches and runs a small single-node cluster locally for testing purposes. And you can control Kubernetes with any of several control interfaces: the kubectl command provides a command-line interface, and library APIs and REST endpoints provide programmable interfaces.

What Does Kubernetes Do?

Modern web-based applications are commonly implemented with “microservices,” each of which embodies one part of the desired application behavior. Kubernetes distributes the microservices across Pods. Pods can be used two ways—to run a single container (the most common use case) or to run multiple containers (like a pod of peas or a pod of whales—a more advanced use case). Kubernetes operates on the Pods, which act as a sort of wrapper around the container(s) rather than the containers themselves. As the microservices run, Kubernetes is responsible for managing the application’s execution. Kubernetes “orchestrates” the Pods, including:

  • Autoscaling: As more users connect to the application’s website, Kubernetes can start up additional Pods to handle the load.
  • Self-healing: If the code in a Pod crashes, or if there is a hardware failure, Kubernetes will detect it and restart the code in a new Pod.
  • Parallel worker processes: Kubernetes distributes the Pods across multiple nodes to benefit from parallelism.
  • Load balancing: If one server gets overloaded, Kubernetes can balance the load by migrating Pods to other nodes.
  • Storage orchestration: Kubernetes lets you automatically mount persistent storage, say a local device or cloud-based object storage.

The beauty of this model is that the applications don’t have to know about the Kubernetes management. You don’t have to write load-balancing functionality into every application, or autoscaling, or other orchestration logic. The applications just run simplified microservices in a simple environment, and Kubernetes handles all the management complexity.

As an example: You write a small reusable application (say, a simple database) on a Debian Linux system. Then you could transfer that code to an Ubuntu system and run it, without any changes, in a Debian container. (Or, maybe you just download a database container from the Docker library.) Then you create a new application that calls the database application. When you wrote the original database on Debian, you might not have anticipated it would be used on an Ubuntu system. You might not have known that the database would be interacting with other application components. Fortunately, you didn’t have to anticipate the new usage paradigm. Kubernetes and containers isolate your code from the messy details.

Keep in mind, Kubernetes is not the only orchestration solution—there’s Docker Swarm, Hashicorp’s Nomad, and others. Cycle.io, for example, offers a simple container orchestration solution that focuses on ease for the most common container use cases.

Implementing Kubernetes: External Storage Required

Kubernetes spins up and spins down Pods as needed. Each Pod can host its own internal storage (as shown in the diagram above), but that’s not often used. A Pod might get discarded because the load has dropped, or the process crashed, or for other reasons. The Pods (and their enclosed containers and volumes) are ephemeral, meaning that their state is lost when they are destroyed. But most applications are stateful. They couldn’t function in a transitory environment like this. In order to work in a Kubernetes environment, the application must store its state information externally, outside the Pod. A new instance (a new Pod) must fetch the current state from the external storage when it starts up, and update the external storage as it executes.

You can specify the external storage when you create the Pod, essentially mounting the external volume in the container. The container running in the Pod accesses the external storage transparently, like any other local storage. Unlike local storage, though, cloud-based object storage is designed to scale almost infinitely right alongside your Kubernetes deployment. That’s what makes object storage an ideal match for applications running Kubernetes.

When you start up a Pod, you can specify the location of the external storage. Any container in the Pod can then access the external storage like any other mounted file system.

Kubernetes in Your Environment

While there’s no doubt a learning curve involved (Kubernetes has sometimes been described as “not for normal humans”), container orchestrators like Kubernetes, Cycle.io, and others can greatly simplify the management of your applications. If you use a microservice model, or if you work with similar cloud-based architectures, a container orchestrator can help you prepare for success from day one by setting your application up to scale seamlessly.

The post What Is Kubernetes? appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.