EBS, Instance Store and EFS

An EC2 instance can store files in three places. An EBS volume is a network disk that outlives the instance. An instance store is a local disk that dies with it. An EFS file system is a shared network drive that many instances mount at once. Choosing between them decides whether your data survives a stop, a crash or an AZ outage. The general block vs file vs object ideas are in Block, File and Object Storage.

Amazon EBS#

Amazon Elastic Block Store (EBS) provides block storage volumes for EC2. You can think of a volume as a disk that you plug into an instance over the network.

Lifetime and scope#

• An EBS volume is not part of the instance. It connects over the network, and it keeps its data when the instance stops.
• A volume lives in one Availability Zone. You can attach it only to an instance in the same AZ.
• AWS replicates each volume inside its AZ, which protects against a single disk failure. It does not protect against losing the whole AZ.
• A volume attaches to one instance at a time in normal use. The exception is Multi-Attach on Provisioned IOPS (io1 and io2) volumes, where several instances in the same AZ share one volume. The application must then coordinate writes itself, for example with a cluster file system.
• You can detach a volume and attach it to another instance in the same AZ.
• An instance can have many volumes attached.
• You cannot delete a volume while it is attached.

Root volumes and termination#

Attaching and detaching is one half of a volume's life. The other half is what happens when the instance goes away. Most AMIs are EBS-backed: at launch, AWS creates the root volume from a snapshot referenced by the AMI.

Each volume has a delete on termination flag. By default it is on for the root volume and off for extra volumes you attach. So terminating an instance usually deletes its root disk and keeps its data disks. Check this flag before you rely on either behavior.

Snapshots#

Replication inside one AZ doesn't cover an AZ outage or a bad delete, so you need backups. A snapshot is a point-in-time backup of a volume, stored by AWS in S3 (you do not see it in your buckets).

• Snapshots are incremental. The first one copies all used blocks. Each later one stores only the blocks that changed since the previous snapshot.
• Each snapshot still restores the full volume, because AWS links it to the unchanged blocks from earlier snapshots.
• Example: a volume holds 100 GiB and 5 GiB changes after the first snapshot. The second snapshot adds about 5 GiB of storage, not 100.
• To move data to another AZ, create a snapshot and restore it as a new volume in that AZ. To move it to another Region, copy the snapshot to that Region first.
• AMIs use snapshots too. An AMI points to the snapshots of its volumes.

Volume types and cost#

Not every EBS volume performs the same way. EBS has SSD types (General Purpose such as gp3, and Provisioned IOPS such as io2) and HDD types for large sequential reads (Throughput Optimized and Cold). You pay for:

• The provisioned size, whether you use it or not.
• Provisioned IOPS and throughput above the baseline, on types that charge for them.
• Snapshot storage.
• Data transfer for snapshot copies across Regions.

You can grow a volume and change its type while it is in use. You then extend the file system inside the OS to use the new space. Size and performance limits differ by type and change over time, so check the current EBS documentation.

Encryption#

EBS encryption uses AWS KMS keys. On an encrypted volume, data at rest, data moving between the instance and the volume, and all snapshots are encrypted. You can turn on encryption by default for each Region in your account.

How EBS counts IOPS#

The volume type decides how many operations per second you get, so it helps to know how EBS counts them. IOPS (input/output operations per second) is how EBS measures and limits a volume's operation rate. Not every request counts as one operation. EBS counts by size, with a cap on the size of one I/O:

The counting rules follow from that cap:

This explains why HDD volumes suit large sequential reads, such as log processing, and SSD volumes suit small random I/O, such as databases. When a volume hits its IOPS or throughput limit, requests queue and latency goes up.

Instance store#

EBS trades a network hop for durability. The second option makes the opposite trade. An instance store is block storage on disks physically attached to the host computer that runs your instance. Only some instance types have it, and the type fixes the number and size of the disks.

Lifetime#

Instance store data is ephemeral: it lasts only while the instance keeps running on the same host. The table shows what each event does to it.

An instance whose root volume is an instance store (an instance-store-backed AMI) cannot be stopped at all. You can only reboot or terminate it. An EBS-backed instance can still have instance store disks; their data is lost on stop.

Why use it#

Losing data on stop sounds bad, but instance store has real uses:

• It is very fast. There is no network hop, and no other customer shares the path to the disk.
• It is included in the instance price.
• It suits data you can lose: caches, buffers, scratch files, and data that is already copied across several instances (for example, a replicated database cluster or a stateless web fleet).

Never keep the only copy of important data on an instance store. It has no snapshots. Back up anything you need to EBS or S3. On current instance types that use NVMe instance store disks, the hardware encrypts data at rest.

Amazon EFS#

EBS and instance store both serve one instance. The third option is built for sharing. Amazon Elastic File System (EFS) is a managed NFS file system. Many instances can mount it at the same time and see the same files.

• It speaks NFS v4.0 and v4.1, so Linux instances mount it with standard tools.
• It is elastic: it grows and shrinks as you add and remove files. You do not provision a size, and you pay for what you store.
• A Regional file system stores data in several AZs. Instances in any AZ of the Region can mount it through a mount target in their own AZ. (A cheaper One Zone option keeps data in a single AZ.)
• It supports encryption at rest and in transit.
• It has storage classes for cost, such as Standard and Infrequent Access. Lifecycle rules move files that nobody reads to the cheaper class.
• It has performance and throughput settings for different workloads. Check the current documentation for the options.
• It cannot be the root volume of an instance. Boot disks are EBS or instance store.

Since EFS speaks NFS, mounting it looks like mounting any network drive. Use the EFS mount helper from the amazon-efs-utils package (the file system ID below is a placeholder):

sudo mkdir -p /mnt/efs
sudo mount -t efs -o tls fs-0123456789abcdef0:/ /mnt/efs

Concurrent writes#

Sharing has a cost. EFS lets many instances write to the same file, but it does not stop two writers from overwriting each other. Use NFS file locks or coordinate in the application when more than one instance writes to one file.

EFS still needs backups#

It's tempting to treat multi-AZ storage as a backup, since it protects against hardware and AZ failure. It does not protect against a bad deploy, a script that deletes files, or ransomware. Those changes replicate everywhere at once. Back up EFS with AWS Backup, which takes scheduled, point-in-time backups.

Use cases#

• Web servers that share uploaded content, such as a CMS media folder.
• Shared code, build artifacts or configuration across a fleet.
• Home folders.
• Datasets read by many compute instances at once, for example for analytics or ML training.

Comparison#

With all three options covered, the table puts them side by side. They differ most in lifetime, scope and sharing.

Key takeaways#

• EBS is a persistent network disk in one AZ. Attach it to an instance in the same AZ, and back it up with incremental snapshots.
• Check the delete-on-termination flag: by default the root volume goes, extra volumes stay.
• EBS counts I/O in chunks up to 256 KiB (SSD) or 1,024 KiB (HDD). Contiguous small I/Os merge; random ones do not.
• Instance store is fast and free but loses its data on stop, terminate or hardware failure. Use it only for data you can rebuild.
• EFS is a shared, elastic NFS file system across AZs. It is not a root volume, and it still needs backups.