Using memcached for Ruby on Rails session storage
Posted 24 Jan 2006
A few days ago Eric Hodel announced the availability of a new pure Ruby memcached client implementation (memcache-client-1.0.3) with performance improvements over the older Ruby-Memcache-0.4 implementation.
I had measured the old version previously, for use as a Ruby on Rails fragment cache storage container, but found its performance to be abysmal and completely unusable for my purposes.
memcache-client-1.0.3 provides much better performance: much faster than either the old implementation, pstore or ActiveRecordStore, but also faster than my optimized SQLSessionStore using MysqlSession (see Roll your own SQL session store).
In order to determine the relative performance I ran my usual benchmarks using railsbench against memcache-client-1.0.3 and SQLSessionStore using MyslSession.
All components resided on a single machine. Before running the tests, both DB tables and memcached were populated with 10000 sessions.
The first test was run with Mysql query caching disabled:
| page | c1 total | c2 total | c1 r/s | c2 r/s | c1 ms/r | c2 ms/r | c1/c2 |
|---|---|---|---|---|---|---|---|
| /empty/index | 1.23318 | 0.96322 | 810.9 | 1038.2 | 1.23 | 0.96 | 1.28 |
| /welcome/index | 1.44314 | 1.15051 | 692.9 | 869.2 | 1.44 | 1.15 | 1.25 |
| /rezept/index | 1.52869 | 1.19616 | 654.2 | 836.0 | 1.53 | 1.20 | 1.28 |
| /rezept/myknzlpzl | 1.52159 | 1.19022 | 657.2 | 840.2 | 1.52 | 1.19 | 1.28 |
| /rezept/show/713 | 4.14906 | 3.95939 | 241.0 | 252.6 | 4.15 | 3.96 | 1.05 |
| /rezept/cat/Hauptspeise | 9.48101 | 9.29202 | 105.5 | 107.6 | 9.48 | 9.29 | 1.02 |
| /rezept/cat/Hauptspeise?page=5 | 9.84404 | 9.65393 | 101.6 | 103.6 | 9.84 | 9.65 | 1.02 |
| /rezept/letter/G | 5.42370 | 5.20227 | 184.4 | 192.2 | 5.42 | 5.20 | 1.04 |
| c1: DB sessions, c2: memcached sessions, r/s: requests per second, ms/r: milliseconds per request | |||||||
On the second run, Mysql query caching was enabled:
| page | c1 total | c2 total | c1 r/s | c2 r/s | c1 ms/r | c2 ms/r | c1/c2 |
|---|---|---|---|---|---|---|---|
| /empty/index | 1.26804 | 0.95403 | 788.6 | 1048.2 | 1.27 | 0.95 | 1.33 |
| /welcome/index | 1.47070 | 1.13807 | 679.9 | 878.7 | 1.47 | 1.14 | 1.29 |
| /rezept/index | 1.55976 | 1.18066 | 641.1 | 847.0 | 1.56 | 1.18 | 1.32 |
| /rezept/myknzlpzl | 1.55402 | 1.17650 | 643.5 | 850.0 | 1.55 | 1.18 | 1.32 |
| /rezept/show/713 | 3.50876 | 3.20310 | 285.0 | 312.2 | 3.51 | 3.20 | 1.10 |
| /rezept/cat/Hauptspeise | 4.31720 | 4.00799 | 231.6 | 249.5 | 4.32 | 4.01 | 1.08 |
| /rezept/cat/Hauptspeise?page=5 | 4.41150 | 4.11811 | 226.7 | 242.8 | 4.41 | 4.12 | 1.07 |
| /rezept/letter/G | 4.31190 | 4.02345 | 231.9 | 248.5 | 4.31 | 4.02 | 1.07 |
| c1: DB sessions, c2: memcached sessions, r/s: requests per second, ms/r: milliseconds per request | |||||||
What can we learn from this data?
- Pages which force the creation of new sessions (empty, welcome) and action cached pages (rezept/index and rezept/knzlpzl) experience a speedup between 25% and 33%.
- Pages involving DB queries other than session retrieval don’t see as much speedup.
- If query caching is disabled and the query is really DB expensive (/rezept/cat issues a LIKE %Hauptspeise%), the speed improvement is negligible.
- Enabling the Mysql query cache will result in slightly slower creation of new sessions, but can speed up complex queries tremendously (table below).
| page | c1 total | c2 total | c1 r/s | c2 r/s | c1 ms/r | c2 ms/r | c1/c2 |
|---|---|---|---|---|---|---|---|
| /empty/index | 1.23318 | 1.26804 | 810.9 | 788.6 | 1.23 | 1.27 | 0.97 |
| /welcome/index | 1.44314 | 1.47070 | 692.9 | 679.9 | 1.44 | 1.47 | 0.98 |
| /rezept/index | 1.52869 | 1.55976 | 654.2 | 641.1 | 1.53 | 1.56 | 0.98 |
| /rezept/myknzlpzl | 1.52159 | 1.55402 | 657.2 | 643.5 | 1.52 | 1.55 | 0.98 |
| /rezept/show/713 | 4.14906 | 3.50876 | 241.0 | 285.0 | 4.15 | 3.51 | 1.18 |
| /rezept/cat/Hauptspeise | 9.48101 | 4.31720 | 105.5 | 231.6 | 9.48 | 4.32 | 2.20 |
| /rezept/cat/Hauptspeise?page=5 | 9.84404 | 4.41150 | 101.6 | 226.7 | 9.84 | 4.41 | 2.23 |
| /rezept/letter/G | 5.42370 | 4.31190 | 184.4 | 231.9 | 5.42 | 4.31 | 1.26 |
| c1: DB sessions without query cache, c2: DB sessions with query cache | |||||||
Like all benchmarks, these results have to be taken with a grain of salt. Choice of either option involves more than just looking at the above numbers.
Database and memcached have quite different scaling properties: adding more memcached daemons is easy, whereas DB based session storage scales mostly by buying a bigger DB machine or running the session DB on a separate machine using a session database.
On the other hand, DB session storage makes it easy to get application usage statistics using SQL queries, e.g. displaying the number of active sessions or checking whether a particular user is currently logged in. I don’t know an easy way to do this using the memcachd API.
It seems to me that using memcached for session storage is a good choice if your DB server experiences a very high load and your scaling options are already exhausted. I wouldn’t recommend to use it per default.
Using memcachd for Ruby on Rails fragment cache storage is a completely different story, on which I hope to report in the future.
