>Delayed Job & Monit configuration
We were struggling through how to monit delayed_job from past few months because monit doesn’t work seamlessly with delayed_job start/stop commands and finally we got able to monit delayed_job.
Here is our old configuration that wasn’t working anyhow-
check process delayed_job with pidfile /home/sandip/shared/pids/delayed_job.pid stop program = "/bin/bash -c 'cd /home/sandip/current && RAILS_ENV=production script/delayed_job stop'" start program = "/bin/bash -c 'cd /home/sandip/current && RAILS_ENV=production script/delayed_job start'" if totalmem > 100.0 MB for 3 cycles then restart if cpu usage > 95% for 3 cycles then restart
After doing google & looking at stackoverflow, we found different solutions to work with but none of them found useful to me. 😦
After reading google group someone (not remembering exactly) directed to write a init script for delayed_job server and that perfectly worked for me and my headache of self moniting delayed_job ended up 😉
Here is delayed_job init script
## /etc/init.d/deloyed_job
#! /bin/sh set_path="cd /home/sandip/current" case "$1" in start) echo -n "Starting delayed_job: " su - root -c "$set_path; RAILS_ENV=production script/delayed_job start" >> /var/log/delayed_job.log 2>&1 echo "done." ;; stop) echo -n "Stopping delayed_job: " su - root -c "$set_path; RAILS_ENV=production script/delayed_job stop" >> /var/log/delayed_job.log 2>&1 echo "done." ;; *) echo "Usage: $N {start|stop}" >&2 exit 1 ;; esac exit 0
finally here is the working monit delayed_job configuration
check process delayed_job with pidfile /home/sandip/shared/pids/delayed_job.pid stop program = "/etc/init.d/delayed_job stop" start program = "/etc/init.d/delayed_job start if totalmem > 100.0 MB for 3 cycles then restart if cpu usage > 95% for 3 cycles then restart
Thinking Sphinx monit configuration
check process sphinx with pidfile /home/sandip/shared/pids/searchd.pid stop program = "/bin/bash -c 'cd /home/sandip/current && /usr/bin/rake RAILS_ENV=production ts:stop'" start program = "/bin/bash -c 'cd /home/sandip/current && /usr/bin/rake RAILS_ENV=production ts:start'" if totalmem > 85.0 MB for 3 cycles then restart if cpu usage > 95% for 3 cycles then restart
Adhearsion (ahn) monit confiuration
check process ahn with pidfile /home/josh/shared/pids/ahnctl.pid stop program = "/bin/bash -c 'cd /home/sandip/current && /usr/bin/ahnctl stop adhearsion'" start program = "/bin/bash -c 'cd /home/sandip/current && /usr/bin/ahnctl start adhearsion'" if totalmem > 100.0 MB for 3 cycles then restart if cpu usage > 95% for 3 cycles then restart
Nginx monit configuration
check process nginx with pidfile /opt/nginx/logs/nginx.pid start program = "/opt/nginx/sbin/nginx" stop program = "/opt/nginx/sbin/nginx -s stop" if cpu > 70% for 3 cycles then alert if cpu > 80% for 5 cycles then restart if 10 restarts within 10 cycles then timeout
Learn to format your blog posts. This crap is useless regardless of the actual information in it.
Thanks Mike – we sometimes need to push to format older posts. Cheers!
I added the code to restart DJ script in /etc/monit/monitrc as u specified. But when i do “sudo monit reload”, i get the below warnings.
/etc/monit/monitrc:188: Warning: Program is not executable: ‘/etc/init.d/delayed_job’
/etc/monit/monitrc:189: Warning: Program is not executable: ‘/etc/init.d/delayed_job’
Hence, i rebooted my machine and confirmed using ‘ps -ef’ that DJ and monit are running. How to:
1) test that if accidently DJ process is killed, it will be restarted by monit?
2) in my deployment script, i am restarting is DJ (stop n then start). wont monit restart another DJ process resulting in 2 DJ processes?
Hi Prasad,
1) Use “kill -9 ” to stop delayed Job process and then observe that monit again starts delayed job process.
2) There are no chances of having 2 delayed job processes running because it will warn something like “There is already delayed job server running”.
–
Sandip
i have problem
/usr/bin/env ruby: no such file or directory
and
Monit does not want to restart delayed job
Does anyone can provide a solution for this?