ZipmakerJob failures - sul-dlss/preservation_catalog GitHub Wiki

The below code can help to safely automate cleanup of failed ZipmakerJobs. Since ZipmakerJob (via DruidVersionZip) now attempts to delete any bad zip files it creates, this should rarely be necessary. But for cases where the cleanup failed, and the zip file hasn't aged out of temp space, some manual cleanup may be needed._

We can build on the above generalized code for wrangling failure queue info with some code for dealing with entries in zipmaker_failed.

failure_queue_arg_lists = dump_and_return_failure_queue_entries('zipmaker_failed').map { |failure| failure[:args][0]['arguments'] }

zm_druid_version_zips = failure_queue_arg_lists.map { |args| DruidVersionZip.new(args[0], args[1], args[2]) } ; nil
bad_druid_version_zips = zm_druid_version_zips.map do |dvz|
  if File.exist?(dvz.file_path)
    if dvz.send(:zip_size_ok?)
      puts "ok: zip exists at #{dvz.file_path}, but size is ok"
      nil
    else
      puts "ERROR: zip exists at #{dvz.file_path} but is smaller than moab version"
      dvz
    end
  else
    puts "ok: no cached zip exists at #{dvz.file_path}"
    nil
  end
end.compact ; nil
# it's possible that not all entries in the failure queue will have bad zips sticking around:  for example, ZipmakerJob might've detected the moab
# was unreadable before trying to create the zip, the zip may have already aged out of temp space if the failed job is old enough, etc.

# use DruidVersionZip#cleanup_zip_parts! to handle the cleanup for us
bad_druid_version_zips.each { |bad_dvz| bad_dvz.send(:cleanup_zip_parts!) }

# now it should be fine to retry everything in queue.
# re-running the above `bad_druid_version_zips = zm_druid_version_zips.map do |dvz|...` code should result in an empty bad_druid_version_zips, since the cleanup loop should've removed any busted zip files.