Rails 1.2-stable Performance Report (vs. 1.1-stable)

Short Summary for the Impatient

Rails 1.2-stable is somewhat slower than Rails 1.1-stable, especially on action cached pages. But the slowdown is nowhere near the numbers which have been reported by others. In fact, two recent patches to speed up session creation and pstore session retrieval, have resulted in improved performance for one of the benchmarked actions.

The following performance data table shows the speed difference for the fastest availabe configuration for the tested application.

page c1 totalc2 total c1 r/sc2 r/s c1 ms/rc2 ms/r c1/c2
empty 11.4403410.91962 437.0457.9 2.292.18 1.05
welcome 13.0464814.29709 383.2349.7 2.612.86 0.91
recipes 10.8361713.51635 461.4369.9 2.172.70 0.80
my_recipes 10.8013713.48035 462.9370.9 2.162.70 0.80
show 24.1672227.37878 206.9182.6 4.835.48 0.88
cat 26.5603829.88683 188.3167.3 5.315.98 0.89
cat_page5 27.2495430.78081 183.5162.4 5.456.16 0.89
letter 26.9906730.17507 185.2165.7 5.406.04 0.89
all requests 151.09216170.43488 264.7234.7 3.784.26 0.89
GC statistics c1 totalc2 total c1 #gcc2 #gc c1 gc%c2 #gc% c1/c2
10.4279012.90426 120.0135.0 6.907.57 0.81

Railsbench generated diagrams (for people who don't like tables ;-)

Environment

Each benchmarked action was run 5000 times using railsbench on an Athlon64 3000+ with 2G of memory using the latest Fedora Core 6, ruby 1.8.4 and Mysql 4.1, with RAILS_PERF_RUNS=3 and Rails logging disabled.

All tests where run with my garbage collector patch applied. For 1.2-stable, the heap size was increased from 600.000 to 650.000 slots, since my measurements have shown that the working set size for 1.2-stable differs from 1.1-stable by around 50.000 slots (probably mainly due to increased code size; remember that Ruby stores the code as abstract syntax trees on the heap).

Configuration options

A number of configuration options were tested:

pstore, arstore, sqlstore
Use either the default session storage container (pstore), ActiveRecordStore or SqlSessionStore
pstore.links, arstore.links, sqlstore.links
use link_to and url_for calls instead of using hand coded URL generation methods

Test Data

For the tests, I selected a number of pages from my recipe database application:

empty a simple render_text, creating a new session per request
welcome a welcome page, action cached, new session per request
recipes application front page, user dependent, action cached
my_recipes my recipes, user dependent, action cached
show shows a single recipe with comments of various users
cat show all recipes of a certain category, paginated
cat_page5 page 5 of action benchmark cat
letter all recipes with a title starting with a given letter

Performance data for each configuration can be found here.

One on one comparisons produced by railsbench for each configuration are listed in this file.

Performance Charts

All benchmarks for all tested configurations.

The numbers are requests per second.

Actions with database queries separated

Conclusions

I have profiled the Rails-1.2-stable code against my benchmarks using railsbench and Ruby Performance Validator, but could not find sweet spots for optimization (at least given my current time constraints). It seems that many small changes add up to the performance loss shown by the benchmark data.