Instance Metadata and User Data

A new EC2 instance needs to know things about itself, such as its ID, its Availability Zone and its IAM credentials. It also needs instructions for what to install on first boot. The Instance Metadata Service (IMDS) answers the first need, and user data answers the second. Both matter for automation. IMDS is also a common target in attacks, so you need to know how to lock it down.

Instance metadata#

Instance metadata is data about the running instance that the instance can read over HTTP. It is available only from inside the instance, at a fixed link-local address: 169.254.169.254. No login to the console and no API keys are needed.

Everything sits under http://169.254.169.254/latest/. These are the paths you'll use most:

Here's how those paths get used in practice:

• A boot script reads the instance ID or zone and writes it into logs or config.
• An app finds its own private IP to register with a service registry.
• AWS SDKs read the IAM role credentials from IMDS automatically, so your code never stores access keys.
• A Spot instance polls meta-data/spot/instance-action to catch the 2-minute interruption notice.

IMDSv2: session tokens#

That last credentials path is also why IMDS needs protecting, and the fix comes in its second version. There are two versions of the metadata service. IMDSv1 answers any plain GET. IMDSv2 first requires a session token, which you get with a PUT request and send back as a header on every read.

1. Send PUT /latest/api/token with a header that sets the token's lifetime in seconds.
2. IMDS returns a token.
3. Send each GET with the header X-aws-ec2-metadata-token: <token>.

# IMDSv2: get a token, then use it to read metadata.
TOKEN=$(curl -sX PUT "http://169.254.169.254/latest/api/token" \
  -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")

curl -s -H "X-aws-ec2-metadata-token: $TOKEN" \
  http://169.254.169.254/latest/meta-data/instance-id

Why the token matters#

The extra token step exists to stop one attack: server-side request forgery (SSRF), where a bug lets an attacker make your web app fetch a URL of their choice. With IMDSv1, they ask the app to fetch the IAM credentials path and steal the role's keys.

IMDSv2 blocks most of these attacks:

• Most SSRF bugs can only make GET requests and cannot set custom headers, so they cannot get a token.
• The PUT response has a network hop limit (1 by default), so the token cannot travel past the instance through a proxy or router. Containers that sit one hop further away may need a limit of 2.
• IMDS rejects token requests that carry an X-Forwarded-For header, which open proxies usually add.

Set new instances to require IMDSv2 (HttpTokens=required) in the launch template or account defaults. Recent Amazon Linux AMIs already do this. You can also turn IMDS off for instances that never need it.

Reading metadata from Python#

AWS SDKs handle IMDSv2 for you. If you call it yourself, the flow is the same as with curl: one PUT for the token, then a GET with the header.

import urllib.request

IMDS = "http://169.254.169.254/latest"


def imds_token(ttl: int = 21600) -> str:
    req = urllib.request.Request(
        f"{IMDS}/api/token",
        method="PUT",
        headers={"X-aws-ec2-metadata-token-ttl-seconds": str(ttl)},
    )
    with urllib.request.urlopen(req, timeout=2) as resp:
        return resp.read().decode()


def imds_get(path: str, token: str) -> str:
    req = urllib.request.Request(
        f"{IMDS}/meta-data/{path}",
        headers={"X-aws-ec2-metadata-token": token},
    )
    with urllib.request.urlopen(req, timeout=2) as resp:
        return resp.read().decode()


if __name__ == "__main__":
    token = imds_token()
    print(imds_get("placement/availability-zone", token))

The short timeout matters: off EC2 (for example on your laptop) the address does not answer, and you want the call to fail fast.

User data#

Metadata tells an instance about itself. User data is the second half: a script or config file you pass when you launch an instance. It runs at boot and turns a generic AMI into a server set up for its job.

• Size: user data is limited to 16 KB before encoding. For more, have the script download a larger file from S3.
• Encoding: the API takes user data as base64. The console and CLI encode it for you.
• Privileges: on Linux, the script runs as root.
• When it runs: by default, only on the instance's first boot. Rebooting does not run it again.

Something on the instance has to read that user data and run it. On Linux AMIs, that's cloud-init, which reads the user data and acts on it. Windows AMIs use an AWS launch agent (EC2Launch) instead.

Example: a bootstrap script#

Here the two halves of the note meet. This script installs nginx on Amazon Linux 2023 and writes the instance ID into the home page. It uses IMDSv2, so it works with tokens required.

#!/bin/bash
set -euo pipefail

dnf install -y nginx

TOKEN=$(curl -sX PUT "http://169.254.169.254/latest/api/token" \
  -H "X-aws-ec2-metadata-token-ttl-seconds: 300")
ID=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" \
  http://169.254.169.254/latest/meta-data/instance-id)

echo "<h1>Served by $ID</h1>" > /usr/share/nginx/html/index.html
systemctl enable --now nginx

If the script fails, check /var/log/cloud-init-output.log on the instance. It holds the script's output and errors.

Changing user data later#

Because user data runs only on first boot, changing it later takes more work. To change user data on an existing instance, you must stop it, edit the attribute and start it again. Even then, cloud-init does not re-run first-boot scripts by default. You have three options:

• Run the new script by hand over SSH or AWS Systems Manager.
• Configure cloud-init to run the script on every boot (for example, a script in the per-boot directory or a #cloud-config setting).
• Prefer replacing the instance: update the launch template and launch fresh instances.

The last option fits immutable, auto-scaled fleets best.

Pitfalls#

The common mistakes here are about security and boot time.

• Secrets in user data are not secret. Any process on the instance can read them from IMDS, and anyone with permission to describe the instance can read them too. Fetch secrets from Secrets Manager or Parameter Store at boot instead.
• IMDSv1 left on keeps the SSRF path to your credentials open. Require IMDSv2.
• Long user data scripts slow every scale-out and fail when a package mirror is down. Bake slow steps into the AMI.
• Assuming user data re-runs on reboot leads to config drift. It runs once unless you configure otherwise.

Key takeaways#

• Instance metadata lives at 169.254.169.254 and is reachable only from inside the instance.
• IMDS serves instance facts, user data and temporary IAM role credentials.
• IMDSv2 needs a token from a PUT request; this defeats most SSRF attacks. Make it required.
• User data is a boot script or #cloud-config file, run as root by cloud-init on the first boot only.
• Keep secrets out of user data, and log boot output to debug failed scripts.