Automating gists with QuickSilver

I like to use gist for sharing code snippets. I like to use QuickSilver for automating common tasks. To combine the two is fairly simple:

1. Install the gist helper script
curl http://github.com/evaryont/gist/tree/master%2Fgist.rb?raw=true > gist
chmod 755 gist
sudo mv gist /usr/local/bin/gist

2. Enable the Terminal Module in QuickSilver Plug-ins
3. Create a new HotKey Trigger
4. Hit "." then type in "pbpaste | gist | pbcopy" in the Select Item field
5. Tab over to Action field, and select "Run Command in Shell"
6. Save the trigger, then set whatever hotkey you want - I use Cmd-Shift-G - you might need to restart quicksilver for this to take effect

Now you can select some text, copy to clipboard, then hit Cmd-Shift-G to create a gist from the clipboard contents, ending up with a url to that gist now in the clipboard.




Mysql connect/read/write timeouts in rails

Ran into a problem in production recently where mysql/OS ran out of memory, didn't die, but didn't actually do anything, so connections to it were still open. As a result, the rails mysql adapter would just hang indefinitely in the middle of a request to mysql, with no exception being thrown, thereby not triggering our db failover and causing our entire site to come to a screeching halt!

In order to prove my hypothesis correct, not to mention to have a way to test that I'd fixed the problem, I first had to be able to duplicate this problem, i.e. I had to simulate mysql hanging without actually killing it. Fortunately *nix has a good way of doing this via kill:

To stop a process: kill -s STOP mysqld_pid
To resume it: kill -s CONT mysqld_pid

This duplicated the problem perfectly - any ActiveRecord::Base.execute would hang indefinitely.

Digging around some, I discovered that the ruby mysql gem does have a way to set timeouts, but this config is not exposed in database.yml, so I dutifully forked rails (yay github!), patched it, and solved my problem.

Here's the Rails issue from which you can get the patch

Added EBS support to rubber

I recently committed a change to rubber that adds some basic support for Amazon EC2's persistent volumes (Elastic Block Store, EBS). The way it works is that you tell rubber the host volume configurations, and those volumes will get created/formatted/mounted automatically on those hosts. If the host gets destroyed and recreated, the volume will get remounted without being formatted.

For example, in rubber.yml you'd have something like:

hosts:
db01:
  availability_zone: us-east-1a
  ec2_volumes:
    - size: 100 # size of vol in GBs
    zone: us-east-1a # zone to create volume in, needs to match host's zone
    device: /dev/sdh # OS device to attach volume to
    mount: /mnt/mysql_data # The directory to mount this volume to
    filesystem: ext3 # the filesystem to create on volume

When you first create/bootstrap the db01 hostname, the volume will automatically get created, formatted and mounted to the given path on the instance. If you then destroy db01, you're given the option to cleanup the volume, but if you don't, recreating db01 will automatically re-attach and remount the volume (without formating it).

This addresses the use case (mine) where you assign persistent storage to say a DB server, and want to be able to quickly rebuild that instance if it dies, without having to figure out which ec2 volume IDs map to which instance IDs. Just make sure you answer NO to destroying the instance if you do a rubber:destroy and want to keep the volume around for when you recreate the instance.

I also changed Elastic IPs to work in a similar fashion - you specify you want an instance to have an elastic ip, and rubber allocates and assigns the static ip to that instance, remembering the IP/instance mapping so that if you destroy/recreate the instance, it will get back the same IP.

Similar to the volumes support, when you destroy an instance you get prompted to cleanup the IP, but if you answer NO it keeps the mapping around to reassign it to the instance at recreation time.
e.g.

hosts:
 web01:
   use_static_ip: true

Time Capsule restores from a secure shared disk

My macbook pro died this week. Turns out my logic board went caput, so they had to put in a new one. In the interim, I had a spare I could use, but first I had to do a full system restore from my backup on our Time Capsule.

We have our time capsule setup for secure shared disks with accounts so that each user has their own password protected private share so that no other user can snoop on their backup.

Unfortunately, the Full System Restore available when booting the Leopard DVD failed to see my backup even though I used my private time capsule account credentials when connecting to it. All I could see were other backups that were available on the public share, leading me to believe that my credentials were working to mount the image, but the restore app wasn't smart enough to mount my private mount point instead of (or in addition to) the public one. I tried calling apple support, and after escalating the issue, all they could suggest was to copy my backup from the private share to the public one. Needless to say, it takes a really long time to a couple of hundred GB even when connected directly to the time capsule using an ethernet cable.

Fortunately, when booting from the Leopard DVD, one also has access to a Terminal. By manually mounting the private share from Terminal, the full system restore UI was then able to see my private share and the restore was able to take place.

To Summarize, Boot from the Leopard DVD, select Utilities->Terminal, then use the mount command to mount your private time capsule share. In my case, my username and share are named the same - mconway - I think this is just the way time capsule works, so the pattern should be the same for you. Also, I had to add the .local to the end of the time capsule's name for the mount to work.

# Create directory to mount to
mkdir -p /Volumes/mconway
# Do the mount
mount -t afp 'afp://mconway:mypassword@my-time-capsule-name.local/mconway' /Volumes/mconway

Once mounted, you can verify with a "ls /Volumes/mconway" and you should see the sparseimage for your backup. Quit Terminal, go to Utilities->Full System Restore, and your backup should be visible, though you might have to lead the UI through the hoops by connecting to your time capsule and typing in your credentials again.

Hopefully this will save someone else the hours of grief I was faced with.

Converting to blogger from wordpress

The HTPC I was running wordpress on got replaced with an appletv+boxee (boxee rocks btw), and rather than waste electricity just to run my blog, I figured I'd convert it to a hosted solution. I don't want to have to do this again any time soon, so I wanted to use a hosted service thats pretty reliable and will be around for a while. I didn't feel like paying wordpress for the privilege of mapping a custom domain, so blogger it was. Problem was, I couldn't get my wordpress export imported into blogger. I tried wxr2blogger, but while the script ran fine, blogger wouldn't actually import the results. I also found a ruby script which seemed to work, but wouldn't pull my comments across, so I ended up tweaking it so that it would work with comments as well. Here is my improved wordpress2bloggger.rb script.

Rubber presentation at Boston Ruby Group

I gave a presentation about using rubber to deploy rails apps to ec2 to the Boston Ruby Group on Tuesday July 8th, 2008.  I'm not quite sure how good my speaking skills are, but it seemed to go well.  See the slides on slideshare.

RailsConf 2008

In Portland, Oregon all this weekend for RailsConf 2008, any like-minded readers interested in chatting over a beer?I gave a lightning talk on rubber today, if anyone wangts a more detailed demo, seek me out (twitter: mattconway).