Musings on cache-money – Part I

So, I always wanted to find a way of memcaching via ActiveRecord, without having to re-invent the wheel 😉 My investigations initially took me via CachedModel, cache_fu and finally I settled on cache-money. Seems to be *almost exactly* what I wanted – any lookup goes via memcache, any update /edit goes via ActiveRecord + memcache and then if needed to the database.

This is the first of my stunts:

1. Create a basic rails project with a simple posts controller. Being as lazy as I am, I used the ./script/generate scaffold help contents to create my Posts controller! 😉

2. Install the cache-money gem

3. Install memcachd server and configure the rails project (all from the github README of cache-money)

— config/memcache.yml —

    ttl: 604800
    namespace: 'josh1'
    sessions: false
    debug: false
    servers: localhost:11211

— environments/production.rb —

# Use a different cache store in production
config.cache_store = :mem_cache_store
memcache_options = {
   :c_threshold => 10000,
   :compression => true,
   :debug => false,
   :namespace => 'josh1',
   :readonly => false,
 :urlencode => false

# require the new gem, this will load up latest memcache
# instead of using the built in 1.5.0
require 'memcache'

# make a CACHE global to use in your controllers instead of
# Rails.cache, this will use the new memcache-client 1.7.2
CACHE = memcache_options

# connect to your server that you started earlier
CACHE.servers = ''

# this is where you deal with passenger's forking
 PhusionPassenger.on_event(:starting_worker_process) do |forked|
 if forked
 # We're in smart spawning mode, so...
 # Close duplicated memcached connections - they will open themselves

# In case you're not running under Passenger (i.e. devmode with mongrel)
rescue NameError => error

— config/initializers/cache_money.rb

require 'cache_money'

config = YAML.load(, "config",
$memcache =
$memcache.servers = config['servers']

$local =$memcache)
$lock =$memcache)
$cache =$local, $lock)

class ActiveRecord::Base
 is_cached :repository => $cache

4. Benchmarking – Instead of running benchmarking, I wrote a few lines of code myself to create a 1000 posts with random text ranging from 1 to 100000 letters.

Setup: Mac OS 1.5 (Leopard) with nginx + passenger + memcached on a MacBook Pro laptop. I used Ruby 1.8.6 (default Mac OS 1.5 Version) and Rails 2.3.3

[term1] $ memcached -uroot -vv

$ ./script/console production
Loading production environment (Rails 2.3.3)
>>  alphanumerics = [('0'..'9'),('A'..'Z'),('a'..'z')].map {
?>       |range| range.to_a}.flatten
=> ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "A", "B", "C", "D",
"E","F", "G", "H", "I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S",
"T", "U", "V", "W", "X", "Y", "Z", "a", "b", "c", "d", "e", "f", "g", "h",
"i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w",
"x", "y", "z"]
>>  1000.times do |i|
?>    Post.create(:title => i.to_s, :body => (0...100000).map {
?>      alphanumerics[Kernel.rand(alphanumerics.size)] }.join)
>>  end

Results & Analysis

1. Memcache quickly scaled upto 64MB (default setting) which was cool. I could see the memcache screen scrolling fast to update all objects – in reality it will tries an LRU style purging and gets rid of the oldest objects once its actual memory is ‘full’. Memcache did not crash or stall or thrash — which was great!

2. For each object created in cache, we see the following memcache update:

<22 add lock/Post:1/id/15 0 30 6
<22 set Post:1/id/15 0 86400 280
<22 delete lock/Post:1/id/15 0

A GET for any posts results in:

<23 get Post:1/id/15
>23 sending key Post:1/id/15
>23 END

An UPDATE / CREATE / DELETE for any posts results in:

<23 add lock/Post:1/id/15 0 30 6
<23 set Post:1/id/15 0 86400 361
<23 delete lock/Post:1/id/15 0


2. To confirm if we were getting it right, I checked the logs:

First time for GET:

Processing PostsController#show (for at 2009-10-10 16:06:05) [GET]
 Parameters: {"id"=>"15"}
Rendering template within layouts/posts
Rendering posts/show
Completed in 5ms (View: 2, DB: 95) | 200 OK [http://josh1.local/posts/15]

Second Time for the same GET request:

Processing PostsController#edit (for at 2009-10-10 16:06:42) [GET]
 Parameters: {"id"=>"15"}
Rendering template within layouts/posts
Rendering posts/edit
Completed in 6ms (View: 5, DB: 0) | 200 OK [http://josh1.local/posts/15/edit]


2. Just to push the pedal, I ran another iteration of 1000 posts with random body text and saw that memcache memory stayed put at 62-64mb. I could see expired cache objects hitting the database and get cached and objects already in the cache NOT hitting the database. Exactly what I wished for.

Some caveats:

  1. a. Every ‘find’ request hits the database. Understandable but I wonder if this too can hit via ‘cache’ – its not safe or synch’ed but I wonder.
  2. money-cache still does not support joins or includes or nested attributes. (Time to contribute!! )


Next Steps in Part II:

  • Test money-cache on a live project with about 1 million records.
  • It has currently ~35 Requests Per Minute.
  • Some controllers calls take almost 50% of the time. Gotta reduce that to 5% (hopefully)


One thought on “Musings on cache-money – Part I

  1. Hey, I followed each step you mentioned above and landed with up the following error :
    `load_missing_constant’: uninitialized constant MemCache (NameError)

    I googled a bit and then installed the latest ‘memcache-client’ gem, which worked for me.
    btw nice tut. Thanks.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.