We are heavy users of Amazon Web Services (AWS) Elastic Cloud Computing (EC2) and regularly work with clients to help them transition their on-premises systems to the cloud, build a brand-new infrastructure, or evaluate the security of their virtual data centers. Migrating servers to an Infrastructure as a Service (IaaS) like AWS makes sense for most of our clients.
Instead of the capital expense of servers and ongoing operating expenses needed to support an on-premises data center, using a cloud-based service like EC2 or Microsoft Azure is a win for most organizations.
- AWS Pricing Calculator lets you explore AWS services, and create an estimate for the cost of your use cases on AWS. Select your cookie preferences We use cookies and similar tools to enhance your experience, provide our services, deliver relevant advertising, and make improvements.
- Using Amazon S3 as an Image Hosting Service In Reducing Your Website's Bandwidth Usage, I concluded that my best outsourced image hosting option was Amazon's S3 or Simple Storage Service. S3 is a popular choice for startups. For example, SmugMug uses S3 as their primary data storage source.
- AWS S3 billing gets complicated, such as navigating the different types of storage and operations that can determine a user's cloud storage costs. Identifying them all can help users make sense of extra charges beyond what it costs to store digital goods and improve cloud efficiency.
File storage offers shared storage for legacy applications using the standard server message block (SMB) protocol. File storage is used in a similar manner to EFS in the AWS platform. Glacier and Azure Storage. Azure Archive Blob Storage is comparable to AWS Glacier storage service. It is intended for rarely accessed data that is stored for at.
However, one factor that is often not taken into consideration is archival storage and backups of server disk images (i.e., EC2 volumes). There are plenty of best practice use cases for creating backups or archives of server disk images – for instance, full backups of gold system images, images that were created before patches were applied, forensic images made before investigating an information security incident, or backups of old servers that have been decommissioned.
Amazon's snapshot functionality makes creating these images easy, but these images are then locked into the AWS ecosystem. If you leave the images in your AWS account, monthly storage fees quickly accumulate. Snapshot storage is billed at $0.05 USD per gigabyte per month. While that might not sound like much, keeping just twenty 100GB images on hand will cost your organization $1,200 a year. In most cases, you'll never need to use these old images, but even so, there are many reasons to keep them around just in case.
Given that many cloud providers such as Dropbox and Google are now offering packages that include unlimited storage, it's likely that your organization already has access to a much less costly space to use to store these kinds of server images. The problem is that Amazon does not offer a native ability to export AWS volumes outside of the Amazon ecosystem.
In Part 1 of this series, I'll demonstrate a technique using standard Linux tools to export an AWS volume to another cloud provider or on-prem storage. In Part 2, I'll demonstrate how to move and reinstate the volume back into AWS. In Part 3, I'll demonstrate how to automate the export process.
Read the series:
Learning Objectives
After completing this unit, you'll be able to:
- Differentiate between file, block, and object storage.
- Explain how traditional on-premises storage technology relates to AWS storage options.
Note
This module was produced in collaboration with Amazon Web Services (AWS), which owns, supports, and maintains the AWS products, services, and features described here. Use of AWS products, services, and features is governed by privacy policies and service agreements maintained by AWS.
Before you start this module, complete AWS Networking. The work you do here builds on the concepts you learn there.
You have a cat photo application on Amazon Elastic Compute Cloud (EC2) that has been properly secured by a network. However, that doesn't mean that all your problems have gone away.
When you work with computers, you typically need to work with storage as well. With a laptop, the storage is either built-in internal storage or external drives that store operating systems, software, application and system files. Now that your cat photo application exists on a (virtual) computer, you can't help but wonder where the (virtual) drives are. The only thing you remember is selecting the default storage option when creating your EC2 instance.
How can you add more storage so that your cat photo application can continue to grow? Before you learn how to solve this issue, it's important to understand all the different types of storage that AWS supports. This way you can confidently choose the correct storage service to solve your problem.
AWS storage services are grouped into three different categories: block storage, file storage, and object storage.
Store Data in a Hierarchy with File Storage
You may be familiar with file storage if you've interacted with file storage systems like Windows File Explorer or Finder on MacOS. You place your files in a tree-like hierarchy that consists of folders and subfolders. For example, if you have hundreds of cat photos on your laptop, you may want to create a folder called Cat photos, and place those images inside that folder to organize them. Since you know these images will be used in a cat photo application, you may want to place the cat photos folder inside another folder called Application files.
Each file has metadata such as file name, file size, and the date the file was created. The file also has a path, for example, computer/Application_files/Cat_photos/cats-03.png. When you need to retrieve a file, your system can use the path to find it in the file hierarchy.
File storage is ideal when you require centralized access to files that need to be easily shared and managed by multiple host computers. Typically, this storage is mounted onto multiple hosts and requires file locking and integration with existing file system communication protocols.
Common use cases for file storage include:
- Large content repositories
- Development environments
- User home directories
Store Data in Chunks with Block Storage
While file storage treats files as a singular unit, block storage splits files into fixed-size chunks of data called blocks that have their own addresses. Since each block is addressable, blocks can be retrieved efficiently.
When data is requested, these addresses are used by the storage system to organize the blocks in the correct order to form a complete file to present back to the requestor. Outside of the address, there is no additional metadata associated with each block. So, when you want to change a character in a file, you just change the block, or the piece of the file, that contains the character. This ease of access is why block storage solutions are fast and use less bandwidth.
Since block storage is optimized for low-latency operations, it is a typical storage choice for high-performance enterprise workloads, such as databases or enterprise resource planning (ERP) systems, that require low-latency storage.
Store Data in a Flat Address Space with Object Storage
Objects, much like files, are also treated as a single unit of data when stored. However, unlike file storage, these objects are stored in a flat structure instead of a hierarchy. Each object is a file with a unique identifier. This identifier, along with any additional metadata, is bundled with the data and stored.
Changing just one character in an object is more difficult than with block storage. When you want to change one character in a file, the entire file must be updated.
Aws Storage Prices
With object storage, you can store almost any type of data, and there is no limit to the number of objects stored, making it easy to scale. Object storage is generally useful when storing large data sets, unstructured files like media assets, and static assets, such as photos.
Aws Image Storage
Perhaps this will come in handy for storing cat photos later. Continue reading to find out!
Relate Back to Traditional Storage Systems
If you've worked with storage on-premises, you may already be familiar with block, file, and object storage. Consider the following technologies and how they relate to systems you may have seen before.
- Block storage in the cloud is analogous to direct-attached storage (DAS) or a storage area network (SAN).
- File storage systems are often supported with a network attached storage (NAS) server.
Aws Docker Image Storage
Adding more storage in a traditional data center environment is a more rigid process, as you need to purchase, install, and configure these storage solutions. With cloud computing, the process is more flexible. You can create, delete, and modify storage solutions all within a matter of minutes.
Cloud Aws Image Storage
Wrap Up
You have now learned the different categories of storage. Throughout the rest of this module, you'll relate this information to AWS storage services. In the next unit, you apply what you learned to your cat photo application by choosing the correct storage service for the problem discussed earlier.
Resources
