Delaying File Processing & Uploading with DelayedJob and PaperClip

I’m going to say it right now. I love DelayedJob. I’ve been working on a project that involves uploading assets, mostly images, to S3 and resizing the images to three sizes. With the addition of DelayedJob, this can happen much faster.

Before I go on, I should mention that I wouldn’t have been able to delay paperclip without this post, and I wouldn’t have been able to delay S3 uploads without this post. Thanks go out to both bloggers!

The original DelayedJob was done by tobi. I’ll be using the version created by collectiveidea, which has a few extra nice features.

So the first step is to install it. I had to install both the plugin and the gem. The plugin was unable to generate the database migration, and the gem was unable to run the rake task. Hopefully, it was either a mistake on my part, or they’ll fix it soon.

script/plugin install git://github.com/collectiveidea/delayed_job.git
sudo gem install collectiveidea-delayed_job

Collectiveidea’s delayed_job has a nice generation script which will make your delayed_jobs table for you.

script/generate delayed_job
rake db:migrate

You’ll also want to add a processing column to your asset table.

class AddProcessingToAsset < ActiveRecord::Migration
  def self.up
    add_column :assets, :processing, :boolean, :default => true
  end

  def self.down
    remove_column :assets, :processing
  end
end

Your next step is to move into the code. To prevent the S3 upload, we subclass our Asset model with TempAsset, and just save locally.

class TempAsset < Asset
  has_attached_file :media, :path => ":rails_root/tmp/uploads/:id/:basename.:extension"
end

In your controller, save your TempAsset instead of Asset, and call a method that will queue up the processing.

class AssetsController < ApplicationController
  def create
    @asset = TempAsset.new(params[:asset])
    @asset.save
    @asset.queue_move_to_s3
    redirect_to @asset
  end
end

From there, we want to make our queue_move_to_s3 method. I’m using send_later in this example, but enqueue works just as well. While we’re at it, we’ll write the method that will perform the saving.

# In temp_asset.rb
def queue_move_to_s3
  self.send_later(:perform)
end

def perform
  asset = Asset.find(self.id) # This is the same db record as self
  asset.media = self.media.to_file
  asset.processing = false
  asset.save!  # This will re-upload & re-size your image as per your has_attached_file method in Asset

  path = self.media.path
  self.media.to_file.close
  File.delete(path)
end

One last thing you may want to do is have a “We’re Still Processing This File” image. Thanks to the processing column we added earlier, it’s a piece of cake.

# In asset.rb
def url(style=:thumb)
  if processing
    still_processing_image_path
  else
    self.media.url(style)
  end
end

There are unfortunately a few hoops to jump through to get this working, but in the end you have a much faster response time which leads to a much better user experience. Hope this helps!