I gave a presentation about using rubber to deploy rails apps to ec2 to the Boston Ruby Group on Tuesday July 8th, 2008. I’m not quite sure how good my speaking skills are, but it seemed to go well. See the slides on slideshare.
We use capistrano to deploy our rails applications from a subversion repository that is accessed using svn+ssh. Recently we ran into a problem where Capistrano was unable to connect to subversion from some of the deployment hosts with an error like:
*** [err :: host.domain.com] ssh_exchange_identification: Connection closed by remote host
Since capistrano tries to fire off N svn checkouts simultaneously, it was running into a connection limit for ssh on the the subversion server. Turns out there is a MaxStartups setting for sshd_config which effectively determines how many clients can be trying to connect simultaneously. It defaults to 10, and our failures started happening when we went above 10. Changing the value to 50 on our svn/ssh server cleared up the problem.
One of the drawbacks of EC2 is that there are no static IPs to be had, so short of using an external (to ec2) solution, the only way to have redundant load balancers is to use dns round robin. If a load balancer instance goes down, we should ideally remove it from the round robin so that clients don’t continue to use the dead IP.
I’ve created yet another system for deploying rails apps to ec2 using capistrano. None of the others were doing it for me, primarily because I needed a way to deploy/configure/scale-up to multiple instances as needed.
(more…)
I setup a xen based system for our production environment at work. I’m using LVM throughout for all the benefits it provides (resizing, snapshot backups, etc). Each domU gets its own logical volume on dom0’s volume group. This logical volume actually looks like an entire physical disk to the domU, and the domU then further partitions it into a boot partition and a nested LVM partition (swap + root fs). Aren’t the layers of unix file abstractions a thing of beauty to behold? 
In retrospect I probably should have mapped LVM logical volumes direct from dom0 as partitions in domU rather than having domU add the extra LVM layer on top - though reading around shows that this _may_ not be supported in the future, so… (more…)