Make ruby-mysql create less garbage
Posted 05 Oct 2006
During the preparation of my RailsConf2006 talk I did some comparative benchmarks with Mysql and Postgresql. To my astonishment, I observed that Rails with Mysql created many more objects. After digging into the C part of ruby bindings of Mysql, I discovered that the Mysql bindings create a copy of the columns name string for each record retrieved from the database. This obviously leads to quadratic memory use and calls for correction.
I’ve written a small patch for the ruby-mysql bindings and submitted it to the author, but never got a response, although a tried several times.
You can get the patch here.
The patch actually goes a little farther than just correcting the n*m problem. I have added a Mysql::Result.all_hashes method, so that the entire array of hashes creation runs in C.
For pages that retrieve small datasets, you will probably see only a small increase. I measured around 5% increase for pages that load 25 objects. Which is neat. But the patch really shines when you retrieve large data sets. For example, for loading 1000 objects, I got this perf data:
|page||c1 total||c2 total||c1 r/s||c2 r/s||c1 ms/r||c2 ms/r||c1/c2|
|GC statistics||c1 total||c2 total||c1 #gc||c2 #gc||c1 gc%||c2 #gc%||c1/c2|
But more important, a lot less gargabe gets created (in the example GC runs twice as often with the unpatched bindings).
You can take advantage of this by creating your own garbage ;-)