Sidekiq can optionally send notifications when a Batch is complete. A batch is complete when all of its jobs have executed once. If a batch has five jobs and four finish successfully and one raises an error, that batch is considered complete, even though that one job might be retried for the next few days.

batch = Sidekiq::Batch.new
batch.on(:complete, Sidekiq::Notifications::Campfire)
batch.on(:complete, Sidekiq::Notifications::Email, :to => '[email protected]')
batch.on(:complete, Sidekiq::Notifications::Webhook, url: 'https://acmecorp.com/batch/done')
batch.jobs do
  # you can mix and match workers as necessary
  10.times do |idx|
    IndexWorker.perform_async(idx)
  end
  SomeClass.delay.some_method
end

This will ping Campfire, send an email to [email protected] and POST a JSON payload to a URL when a batch is complete.

Configuration

Global notification configuration can be done in an initializer:

Sidekiq.configure_server do |config|
  config.options[:web] = 'https://example.org/sidekiq' # optional location of Sidekiq-Web for linking to batch status
  config.options[:notifications][:campfire] = { 
    :subdomain => 'acmecorp', 
    :roomid => 111111,
    :token => '417',
  }
  config.options[:notifications][:hipchat] = { 
    :roomid => 111111,
    :token => '0a01238109238102380123',
  }
  config.options[:notifications][:webhook] = { 
    :url => 'https://default/url', 
    :username => 'mike', # optional, for Basic Auth support
    :password => 'xyzzy',
  }
  config.options[:notifications][:email] = { 
    :to => '[email protected]',
    :via => :smtp,
    :via_options => {
      :address        => 'smtp.yourserver.com',
      :port           => '25',
      :user_name      => 'user',
      :password       => 'password',
      :authentication => :plain, # :plain, :login, :cram_md5, no auth by default
      :domain         => "localhost.localdomain" # the HELO domain provided by the client to the server
    }
  }
end  

Sidekiq uses Pony to send emails. The options hash above is passed directly to the Pony.mail method so any configuration options for Pony can easily be changed. See the Pony documentation for further details. You must add Pony to your Gemfile manually for email notification to work.

Job-Specific Configuration

batch.on(:complete, Sidekiq::Notifications::Campfire)
batch.on(:complete, Sidekiq::Notifications::Email, :to => '[email protected]')

If you want to override the global configuration, just pass the same options into the Batch#on method. Sidekiq will merge the options so they override any global defaults.

Pub/Sub

The result of every batch and batch job is sent to the batch-#{bid} channel in Redis. If you want to follow the progress of a batch in real-time, your code can subscribe to that channel and update the user. Sidekiq sends the following tokens:

• + - a job succeeded
• - - a job failed (and might be retried)
• ! - the batch is considered complete now, all jobs have executed
• $ - the batch has succeeded, all jobs executed successfully

# NB: this is a blocking, infinite loop.
Sidekiq.redis do |conn|
  conn.psubscribe("batch-*") do |on|
    on.pmessage do |pattern, channel, msg|
      # channel = 'batch-123456789'
      # msg = '-', '+', '$' or '!'
      if msg == "$"
        conn.punsubscribe
        # a batch has succeeded, do something with it.
        bid = channel.match(/batch-(.+)/)[1]
        finalize_batch(bid)
      end
    end
  end
end

Example Webhook Payload

Here's an example POST body for a complete message. The batch had 50 jobs; 2 failed and are still pending retry.

{"is_complete"=>true,
 "bid"=>"887f4eea1c92",
 "total"=>50,
 "pending"=>2,
 "description"=>nil,
 "failures"=>2,
 "created_at"=>1238624459.0,
 "fail_info"=>
  [{"jid"=>"abcdef",
    "error_class"=>"RuntimeError",
    "error_message"=>"Something failed",
    "backtrace"=>nil},
   {"jid"=>"187635",
    "error_class"=>"ArgumentError",
    "error_message"=>"Oops, failed",
    "backtrace"=>nil}]}