2007-06-17

The Challenges of EC2

I've recently been working on a project building out the development process, environment and tools for a startup client. This includes things like configuration management, release engineering, automated testing, version control, etc. In doing so, I've been wanting to create as part of this several images including:
  • A bootable and optionally installable build system (see this post)
  • A bootable development environment for each developer (preconfigured application server, version control data, preconfigured database, etc)
  • A bootable QA and Beta environment
I was hoping to use AWS for this, particularly EC2 for the development, QA and Beta environments. If you're not familiar with EC2, it is Amazon's Elastic Compute Cloud, and it allows you to essentially boot and run OS images in their cloud. You pay for hourly CPU usage and data transfer from the cloud and to the cloud. Let's compare a fully hosted, dedicated server solution from the planet with an EC2 image.
 The PlanetAmazon EC2
OSCentOSLinux w/2.6 Kernel
Data Transfer1500GBUnlimited
Disk Space250GB160GB
RAM2048MB1792MB
Cost$147/month$312/month
So, the above for bandwidth assumes that you use your full 1500MB a month (and the same in EC2) and that inbound traffic is only 25% of the total traffic, that number comes from my own experience and is probably overly generous. If you decreased the percentage of inbound traffic, the price increases. Also, it assumes 24/7 operation of a machine over a 30 day period. We also a amortized a $225 setup fee at the planet over a 12 month period. So, EC2 is more expensive for a front-end web server then a hosted environment however your downtime due to hardware related failures decreases to almost zero. You also have no setup fee and you can literally bring images up in minutes (so I'm told, more on that later). However, for an N-tier system, EC2 is a very inexpensive solution for your middleware application servers and backend servers since traffic between EC2 systems costs you nothing. Your cost for operating an EC2 image on a 24/7 basis and only doing inter-image traffic? $72.00/month. So, purely on a cost basis, EC2 seems like a platform for at least middleware and backend systems. Also from an downtime perspective, not having hardware to deal with should increase availability. In doing some research, I found the following troubling points. First, the internet is apparently full. This limited beta is currently full, a helpful error message tells me, and I will be notified when there is availability. Okay, that sucks. Second, EC2 doesn't natively have persistent storage for images. That is, if an image fails/aborts/shutsdown, any data stored on the local disk is lost to you. Apparently you can mount an S3 partition in an EC2 image, but S3 isn't really meant for random IO like you might have in a database for instance. I hope that EC2 opens up to new users in the near future, because it does solve many common problems (rapid horizontal scaling for the web tier, ephemeral environments for dev or qa). It is not however a silver bullet. Bandwidth to and from the cloud is expensive, and the lack of persistent storage makes many jobs impractical for the platform. If I ever get a chance to test, I'll put more information up.

No comments: