Rails development and Sidekiq go hand in hand. One or the other time you must have gone into problem where jobs go missing. Wondering why?
Recently, in one of my projects, I came across a situation wherein I was required to parse a CSV uploaded by the end user. For performance reasons, I choose to create a new Sidekiq worker for each row in my CSV. All these rows (workers) were running in parallel (standard Sidekiq behaviour). I wanted to execute some logic ONLY after all rows (workers) had finished processing. Sidekiq Pro let’s you do this by creating a batch of workers. Sidekiq has a Batch API that provides success and failure callbacks for such scenarios.
Everything was working well in development and staging environment. Until a day when a customer reported that he is not receiving success/failure report for his jobs. I tried to reproduce the scenario described by the user but, couldn’t. Not even for the same customer account. But, then I was able to reproduce it and again I couldn’t.
View original post 529 more words