Moving from backgrounDrb to DelayedJob

In my earlier post regarding DelayedJob setup I mentioned how to setup DelayedJob and create news tasks in your existing code. This post provides details of how to move from backgrounDrb to delayed_job. First of all its important to know why ?

– BackgroundDrb takes in a lot of memory. It spawns worker at start-up. You can use MiddleMan for dynamically spawing backgrounDrb tasks but in any case, it slows thing down a little. I have 4 workers and overall the parent process consumes over 120MB — which is huge considering my linode.

– Monitoring these jobs is a little bit of a pain. Moreover, running in development / production mode requires a Drb server which adds more memory overhead.

As we speak, github has introduced Resque, which is inspired from DelayedJob but I plan to  currently continue with DelayedJob because I dont use Redis (yet). The blog post for Resque has a LOT of details about issues with current background job schedulers. Worth reading!

OK – so you’re now convinced we should use DelayedJob instead of backgrounDrb but have a lot of tasks already configured. These are the steps to follow:

1. Convert your backgrounDrb workers to standard ruby classes:

class BulkUploadWorker < BackgrounDRb::MetaWorker
  set_worker_name :bulk_upload_worker

  def upload(args)


class BulkUploadWorker

  def perform

2. If you earlier used arguments to be passed to the backgrounDrb jobs, you need to tweak the code a little.

Suppose I have an upload method which takes ‘arg’ as the Hash parameter, it would be invoked in the controller for backgrounDrb like this:

MiddleMan.worker(:bulk_upload_worker).async_upload(:arg => {
  'correction' => correction, 'file' => in_file, 'user' =>} )

Simple change to DelayedJob

Delayed::Job.enqueue, infile,

And change the worker to have a perform method (which is the one which gets called on the job):

BulkUploadTask =, :infile, :user_id) do
   def perform
     file =
     user = User.find(user_id)

If you look closely at the code above, even for an experienced Ruby coder – its no piece of cake. Now, I tried the original approch that was on github of

class BulkUploadWorker < (:correction, :infile, :user)

but this gives me a type mis-match error. After some searching on the net.. I found the answer to this, quite understandably from one of the ruby greats JEG2 Here James clearly explains how a returns a Class and accepts a block of code for all the methods. Note the use of Symbol :infile in the declaration but data member infile in the perform method.

Since my file was in lib/workers/bulk_upload_worker.rb, we need to explicitly require this file for DelayedJob. At this in the config/initializers/delayed_job.rb. Now, before I can get down to brass tags and incorporate it, I really need to know if this works . First ensure that the task works — directly from console:

RAILS_ENV=production ./script/console
>> task -, 'test_file', 3)
>> task.perform

Once the task performs as per expectation, start up delayed_job server and test it out from your Web App. If there are errors or exception, delayed_job stores this in the database. So, its a good idea for command-line users with access to the server, to keep an eye out for errors and exceptions in the database.


4 thoughts on “Moving from backgrounDrb to DelayedJob

  1. Weird error I get when trying to work with ferret via DelayedJob:

    closed stream
    /usr/lib/ruby/1.8/drb/drb.rb:961:in `select’
    /usr/lib/ruby/1.8/drb/drb.rb:961:in `alive?’
    /usr/lib/ruby/1.8/drb/drb.rb:1211:in `alive?’
    /usr/lib/ruby/1.8/drb/drb.rb:1168:in `open’
    /usr/lib/ruby/1.8/drb/drb.rb:1166:in `each’
    /usr/lib/ruby/1.8/drb/drb.rb:1166:in `open’
    /usr/lib/ruby/1.8/drb/drb.rb:1163:in `synchronize’
    /usr/lib/ruby/1.8/drb/drb.rb:1163:in `open’
    /usr/lib/ruby/1.8/drb/drb.rb:1092:in `method_missing’
    /usr/lib/ruby/1.8/drb/drb.rb:1110:in `with_friend’
    /usr/lib/ruby/1.8/drb/drb.rb:1091:in `method_missing’
    /usr/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.3/lib/remote_index.rb:31:in `<<'
    /usr/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.3/lib/instance_methods.rb:90:in `ferret_update'

    The task works fine when I run it off console, so it has got to be something to do with environment loading ! Have posted on stackoverflow ( ) but no answers yet! 😦

  2. I was wondering, whats the simplest way to put a delay in each task in a job. For example, lets say im fetching information for 5 entries in a file. The file is the job, and each entry is the task. How can i put a delay after the first entry is parsed and committed to the db?


    btw, great tutorial!

  3. @Ashish Using the Shopify’s extract of delayed job: you have a world of different possibilities. If your file is parsed in the proper way, you can so any of the following:

    – Schedule a job later:
    Delayed::Job.enqueue‘job name’, ),0, 5.minutes.from_now

    – Schedule a job at an interval:
    Delayed::Job.schedule‘job name’, ), :every => 3.hours

    – Schedule a job at a particular time
    Delayed::Job.schedule‘job name’, ), :every => :first_of_month
    Delayed::Job.schedule‘job name’, ), :every => :last_of_month

    Does this help you out?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.