Ran into a problem in production recently where mysql/OS ran out of memory, didn't die, but didn't actually do anything, so connections to it were still open. As a result, the rails mysql adapter would just hang indefinitely in the middle of a request to mysql, with no exception being thrown, thereby not triggering our db failover and causing our entire site to come to a screeching halt!

In order to prove my hypothesis correct, not to mention to have a way to test that I'd fixed the problem, I first had to be able to duplicate this problem, i.e. I had to simulate mysql hanging without actually killing it. Fortunately *nix has a good way of doing this via kill:

To stop a process: kill -s STOP mysqld_pid
To resume it: kill -s CONT mysqld_pid

This duplicated the problem perfectly - any ActiveRecord::Base.execute would hang indefinitely.

Digging around some, I discovered that the ruby mysql gem does have a way to set timeouts, but this config is not exposed in database.yml, so I dutifully forked rails (yay github!), patched it, and solved my problem.

Here's the Rails issue from which you can get the patch