Rails 1.2-stable Performance Report (vs. 1.1-stable)

Short Summary for the Impatient

Rails 1.2-stable is somewhat slower than Rails 1.1-stable, especially on action cached pages. But the slowdown is nowhere near the numbers which have been reported by others. In fact, two recent patches to speed up session creation and pstore session retrieval, have resulted in improved performance for one of the benchmarked actions.

The following performance data table shows the speed difference for the fastest availabe configuration for the tested application.

page	c1 total	c2 total	c1 r/s	c2 r/s	c1 ms/r	c2 ms/r	c1/c2
empty	11.44034	10.91962	437.0	457.9	2.29	2.18	1.05
welcome	13.04648	14.29709	383.2	349.7	2.61	2.86	0.91
recipes	10.83617	13.51635	461.4	369.9	2.17	2.70	0.80
my_recipes	10.80137	13.48035	462.9	370.9	2.16	2.70	0.80
show	24.16722	27.37878	206.9	182.6	4.83	5.48	0.88
cat	26.56038	29.88683	188.3	167.3	5.31	5.98	0.89
cat_page5	27.24954	30.78081	183.5	162.4	5.45	6.16	0.89
letter	26.99067	30.17507	185.2	165.7	5.40	6.04	0.89
all requests	151.09216	170.43488	264.7	234.7	3.78	4.26	0.89
GC statistics	c1 total	c2 total	c1 #gc	c2 #gc	c1 gc%	c2 #gc%	c1/c2
	10.42790	12.90426	120.0	135.0	6.90	7.57	0.81

Railsbench generated diagrams (for people who don't like tables ;-)

Environment

Each benchmarked action was run 5000 times using railsbench on an Athlon64 3000+ with 2G of memory using the latest Fedora Core 6, ruby 1.8.4 and Mysql 4.1, with RAILS_PERF_RUNS=3 and Rails logging disabled.

All tests where run with my garbage collector patch applied. For 1.2-stable, the heap size was increased from 600.000 to 650.000 slots, since my measurements have shown that the working set size for 1.2-stable differs from 1.1-stable by around 50.000 slots (probably mainly due to increased code size; remember that Ruby stores the code as abstract syntax trees on the heap).

Configuration options

A number of configuration options were tested:

pstore, arstore, sqlstore

Use either the default session storage container (pstore), ActiveRecordStore or SqlSessionStore

pstore.links, arstore.links, sqlstore.links

use link_to and url_for calls instead of using hand coded URL generation methods

Test Data

For the tests, I selected a number of pages from my recipe database application:

empty	a simple render_text, creating a new session per request
welcome	a welcome page, action cached, new session per request
recipes	application front page, user dependent, action cached
my_recipes	my recipes, user dependent, action cached
show	shows a single recipe with comments of various users
cat	show all recipes of a certain category, paginated
cat_page5	page 5 of action benchmark cat
letter	all recipes with a title starting with a given letter

Performance data for each configuration can be found here.

One on one comparisons produced by railsbench for each configuration are listed in this file.

Performance Charts

All benchmarks for all tested configurations.

The numbers are requests per second.

Actions with database queries separated

Conclusions

I have profiled the Rails-1.2-stable code against my benchmarks using railsbench and Ruby Performance Validator, but could not find sweet spots for optimization (at least given my current time constraints). It seems that many small changes add up to the performance loss shown by the benchmark data.

• Previous reports on Rails 1.2 being twice as slow as Rails 1.1 are peformance FUD
• Action caching could use some attention
• Never use pstore as your session container
• With the current Ruby interpreter it seems impossible to avoid slowdowns as the code base of Rails grows, due to slow GC and large interpreter overhead. YARV might change this.