Converting an Opscode Chef EC2 AMI to EBS-backed instance
At MediaPiston, we work hard to eliminate plagiarism from content produced by our writers. We have a large dataset of plagiarized articles that we run a full test suite against to make sure that we are always improving our quality as opposed to regressing in quality.
However, our test suite was taking multiple hours to run. After squeezing out a number of performance improvements from the actual code as well as caching expensive calls to disk, I decided to cheaply provision a high-performance machine on Amazon EC2 to run the suite as quick as possible.
I wanted to unpause an EC2 machine when I needed to run the full regression suite, and then pause the machine when it wasn’t in use to save some money. Amazon’s EBS-based EC2 machines let you do exactly this. However, we use Opscode’s Chef AMIs to bootstrap our machines, and they unfortunately don’t come pre-made as EBS-ready images.
These instructions assume that you’ve booted one of Opscode’s AMI images successfully, and now are looking to convert it to an EBS-backed image:
Creating and configuring a new EBS volume
First, take note of your currently running EC2 machine, and check which availability zone it’s running in. Mine was running in us-east-1c. In Elasticfox, create a new EBS volume in that same availability zone, at least 10GB in size:
Next, attach it to your currently running EC2 instance. I chose to attach the device to /dev/sdh.
We’re now going to format and mount this new volume. While the Opscode AMIs are formatted as ext3 partitions, I’d rather format this volume as xfs, to take advantage of better snapshotting.
Copying data
We’ll use rsync to copy over the data, since dd will overwrite the new XFS formatting.
Next, be sure to uncomment the /mnt volume in /mnt/ebs/etc/fstab. This would normally be your ephemeral EC2 storage, but EBS-based instances do not have ephemeral storage turned on by default.
Unmount the EBS drive.
Finally, detach this drive from the running instance:
Registering and booting a new AMI
First, create a snapshot of your new EBS drive.
You can use ec2-describe-snapshots snap-XXXXXXXX (replace with your snapshot ID) to check on the snapshotting status.
Once the snapshot has completed, you’ll want to register an AMI based off that snapshot. Replace the –snapshot parameter with your snapshot ID. You can get the correct values for –ramdisk, –kernel, and -a from the original Opscode image you used.
This command will give you an AMI image ID back, in the form of ami-XXXXXXXX. You can now start an EBS-backed image (replace all specific IDs with your own):
At this point you should have a running Opscode instance that you can pause/unpause at will!