I recently changed SmallPayroll to use Beanstalkd instead of delayed_job for background processing. delayed_job is an awesome tool and makes asynchronous processing so simple. However I wanted to have multiple queues so that I could have different workers processing different queues, and have some upcoming needs to process the jobs quicker than the 5 second polling time.
After watching railscasts on beanstalkd and stalker I decided to use that. Beanstalkd is a lightweight job processor, and stalker makes it very simple to use from the client end.
I used to have an observer that said
class UserObserver
This became:
class UserObserver user.id)
end
end
delayed_job was nice in that the job would just run against the model, but now I have to process the job in config/worker.rb:
require 'stalker'
include Stalker
require File.expand_path("../environment", __FILE__)
job 'default' do |args|
puts "I don't support the default queue"
end
job 'email.new_user' do |args|
user = User.find(args["user_id"])
UserMailer.deliver_welcome_email(user)
UserMailer.deliver_notify_admin(user)
end
One thing about stalker is that it wants you to pass simple objects instead of ActiveRecord objects, so I queue the user_id instead of the user model.
The script above monitors the default tube, which I don't use, because the nagios plugin for beanstalk expects someone to monitor it (after setting it all up, I guess I could have set it up to ignore that tube). I'm also using the munin plugin for beanstalkd to graph the activity in the daemon.
Then, script/worker uses the daemons gem to start the job and restart it if it crashes:
#!/usr/bin/env ruby
require 'rubygems'
require 'daemons'
pwd = File.dirname(File.expand_path(__FILE__))
file = pwd + '/../config/worker.rb'
Daemons.run_proc(
'payroll-generic-worker', # name of daemon
:dir_mode => :normal,
:dir => File.join(pwd, '../tmp/pids'),
:backtrace => true,
:monitor => true,
:log_output => true
) do
exec "stalk #{file}"
end
Finally, some capistrano magic to start the worker with the deploy inside config/deploy.rb
namespace :beanstalk do
desc "Start beanstalk process"
task :start, :roles => :app do
run "cd #{current_path}; RAILS_ENV=production script/worker start"
end
desc "Stop beanstalk process"
task :stop, :roles => :app do
run "cd #{current_path}; RAILS_ENV=production script/worker stop"
end
desc "Restart beanstalk process"
task :restart, :roles => :app do
run "cd #{current_path}; RAILS_ENV=production script/worker restart"
end
end
after "deploy:start", "beanstalk:start"
after "deploy:stop", "beanstalk:stop"
after "deploy:restart", "beanstalk:restart"
The only problem I've run into so far is that my HTML email seems to go out without the text/html content-type. Fixing that was a simple matter of putting content_type 'text/html'
inside my mailer, which wasn't needed when I was using delayed_job.
a