Drupal Community Collaborating on Project Mercury Amazon EC2 Image

As we all know, Amazon's EC2 service is really beginning to mature as a web platform. Acquia is using it for their managed hosting, and many other companies have proven that with the right architecture and setup, "the Cloud" is more than just a buzzword.

Here at Chapter Three, we're on-board the cloud hosting train. While this is still a developing technology -- and there's a lot more to it than just Amazon -- the paradigmatic advantages of switching to elastic architecture make it a frontier worth braving. This is, we think, the future. As a way of learning more, and of helping the dream of "Drupal in the cloud" begin to materialize into reality, we're developing free/public AMIs (Amazon Machine Images) which anyone with an Amazon Web Services account can "spin up" to try out.

Our first AMI is called "Mercury", because it's meant to be waaaaay fast. For an idea of how fast (and why) check out my initial blog post.

Last week I posted to the Amazon Web Services and High Performance Drupal groups groups that a new Alpha 4 version of Project Mercury was up. It's starting to get exciting. Here's what's new and interesting:

• The single most exciting thing to me is that there's real interest in developer collaboration on this image, and the practices in general. I think "Drupal in the Cloud" is an obvious win for everyone in the future, and expanding the population of engineers in the know (as well as carving out good steps on the way up the learning curve) is going to be great for the whole community.
• On that note, I got some encouraging words from Mike Carper about his work on the Boost module developing active cache invalidation logic, which looks like it can port over to the Varnish system which Mercury uses for page caching. This would give us true best-practice caching performance. Hence, I started a Varnish module, which can handle the above, plus status integration and some amount of auto-configuration.
• I've also gotten some really exciting feedback from Sean Bannister, who has experience using AMI boot scripts to integrate the user-data feature, which allows passing parameters in when the instance first spins up. This is key for a production environment, as you will at a minimum need an Elastic IP for steady domain name hosting, and an Elastic Block Volume for MySQL and /files storage. Stay tuned for more here soon I hope.
• I made a number of other nuts and bolts steps towards an image I'd be ok with giving a "beta" stability level. These include more solid Drupal caching w/APC, including an MTA on the image, and some other software updates. All the details and an extensive writeup of how it all works are in the g.d.o. post.

This development feels pretty exciting and novel, and it's really energizing to have the community get on board. I think we could be onto something big here, but to once again reference Sir Issac Newton, "if I have seen a little further it is by standing on the shoulders of Giants."

If you're interested in being a part of this (or just want to know where to look to follow along), we'll be doing most of our work in the wide-open on existing infrastructure for now. Definitely keep an eye on the Amazon Web Services Drupal group. We'll also probably release code into modulespace (e.g. with the new Varnish module), and of course keep it up with blog posts/comments. However, you can always ping me at josh@chapterthree.com or in IRC (I'm josh_k) if you have a direct inquiry or question.