Delaying File Processing & Uploading with DelayedJob and PaperClip

I’m going to say it right now. I love DelayedJob. I’ve been working on a project that involves uploading assets, mostly images, to S3 and resizing the images to three sizes. With the addition of DelayedJob, this can happen much faster.

Before I go on, I should mention that I wouldn’t have been able to delay paperclip without this post, and I wouldn’t have been able to delay S3 uploads without this post. Thanks go out to both bloggers!

The original DelayedJob was done by tobi. I’ll be using the version created by collectiveidea, which has a few extra nice features.

So the first step is to install it. I had to install both the plugin and the gem. The plugin was unable to generate the database migration, and the gem was unable to run the rake task. Hopefully, it was either a mistake on my part, or they’ll fix it soon.

script/plugin install git://github.com/collectiveidea/delayed_job.git
sudo gem install collectiveidea-delayed_job

Collectiveidea’s delayed_job has a nice generation script which will make your delayed_jobs table for you.

script/generate delayed_job
rake db:migrate

You’ll also want to add a processing column to your asset table.

class AddProcessingToAsset < ActiveRecord::Migration
  def self.up
    add_column :assets, :processing, :boolean, :default => true
  end

  def self.down
    remove_column :assets, :processing
  end
end

Your next step is to move into the code. To prevent the S3 upload, we subclass our Asset model with TempAsset, and just save locally.

class TempAsset < Asset
  has_attached_file :media, :path => ":rails_root/tmp/uploads/:id/:basename.:extension"
end

In your controller, save your TempAsset instead of Asset, and call a method that will queue up the processing.

class AssetsController < ApplicationController
  def create
    @asset = TempAsset.new(params[:asset])
    @asset.save
    @asset.queue_move_to_s3
    redirect_to @asset
  end
end

From there, we want to make our queue_move_to_s3 method. I’m using send_later in this example, but enqueue works just as well. While we’re at it, we’ll write the method that will perform the saving.

# In temp_asset.rb
def queue_move_to_s3
  self.send_later(:perform)
end

def perform
  asset = Asset.find(self.id) # This is the same db record as self
  asset.media = self.media.to_file
  asset.processing = false
  asset.save!  # This will re-upload & re-size your image as per your has_attached_file method in Asset

  path = self.media.path
  self.media.to_file.close
  File.delete(path)
end

One last thing you may want to do is have a “We’re Still Processing This File” image. Thanks to the processing column we added earlier, it’s a piece of cake.

# In asset.rb
def url(style=:thumb)
  if processing
    still_processing_image_path
  else
    self.media.url(style)
  end
end

There are unfortunately a few hoops to jump through to get this working, but in the end you have a much faster response time which leads to a much better user experience. Hope this helps!

Features I’m Currently Infatuated with in Rails

I’ve recently started a new project in Rails and I’m trying to follow all the best practices. Sometimes at my company they have fallen by the wayside, but I’m really putting an effort into enforcing them on the team. In doing so, I’ve started using a few methods that I’ve known about for awhile, but never really had a chance to use.

Nested Routes

In a blog, Post has_many Comments, that’s a given. Wouldn’t it be nice if your URLs reflected this relationship? Now they can! In your routes file, simply add map.resources :post, :has_many => [:comments]. Then, change your paths from new_comment_path to new_post_comment_path(@post). In your CommentsController, you’ll have access to both params[:id] and params[:post_id] now.

While we’re talking about related objects, rather than creating objects like this:

@post = Post.find(params[:post_id])
@comment = Comment.new(params[:comment])
@comment.post_id = @post.id
@comment.save

Do something like this:

@post = Post.find(params[:post_id])
@comment = @post.comments.build(params[:comment])
@comment.save

This way, your association is automatically built.

The new and improved render method

This has been in Rails for awhile now, but I’m just now getting a chance to use it. I always hated doing this:

<% @comments.each do |comment| %>
  <%= render :partial => "comments/comment", :locals => { :comment => comment } %>
<% end %>

It seemed so overly verbose, having the singular and plural form of ‘comment’ in there six times. As of Rails 2.2 (I believe) there’s been a handy helper, though.

<%= render @comments %>

The loops through all of your comments and renders them into the partial ‘comments/_comment.html.erb’ and give them a variable name of comment. But what do you do when you’re in the partial? Well that brings me to my next point…

The joy of div_for

You’re in a comment partial. You want to wrap the comment in a div with a unique id. You want to give all your comments the same class. Don’t do this:

<div id="comment_<%= comment.id -%>" class="comment">
  <!-- display the comment --></div>

when you can do this:

<% div_for(comment) do %>
  <!-- display the comment -->
<% end %>

Nice and easy.

Installing Ruby 1.9 from sources on Ubuntu

I found a few posts on how to do this, and all of them over-complicate things. Here’s a simple, step-by-step method to install ruby 1.9.1 from source.

First, ensure that the latest stable version is still 1.9.1-p129. If it isn’t, make sure to change the appropriate file names.

Step 1: Download the source.
wget ftp://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.1-p129.tar.gz

Step 2: Un-Tar
tar -zvxf ruby-1.9.1-p129.tar.gz

Step 3: Configure (aka, Make the makefile)
cd ruby-1.9.1-p129/
./configure

Step 4: Make
make

Step 5: Run the tests. This is optional if you hate tests.
make test

Step 6: Install Ruby
sudo make install

And that’s it! You’re done!

Project Euler #3

Here’s the next one. I’m not happy with the way I determine if it’s a prime number (checking the modulo of every possible number), but it works.

The prime factors of 13195 are 5, 7, 13 and 29.

What is the largest prime factor of the number 600851475143 ?

def find_largest_prime_factor(n)
  if n % 2 == 0
    find_largest_prime_factor(n/2)
  else
    x = find_divisor(n, 3)
  end
  puts x
end

def find_divisor(number, divisor)
  return divisor if divisor >= number
  if number % divisor == 0
    find_divisor(number/divisor, divisor)
  else
    find_divisor(number, divisor+2)
  end
end

find_largest_prime_factor(600851475143)

And the golf…

# Score: 98
def a n;p n%2==0?a(n/2):b(n,3);end;def b(n, d);d>=n ?d:n%d==0?b(n/d,d):b(n,d+2);end;a 600851475143

Project Euler #2

Here we go, Project Euler #2!

Each new term in the Fibonacci sequence is generated by adding the previous two terms. By starting with 1 and 2, the first 10 terms will be:

1, 2, 3, 5, 8, 13, 21, 34, 55, 89, …

Find the sum of all the even-valued terms in the sequence which do not exceed four million.

one_back, two_back, current_fib, total = 1, 1, 0, 0</code>

while current_fib &lt; 4_000_000 do
two_back = one_back
one_back = current_fib
current_fib = one_back + two_back
total += current_fib if current_fib % 2 == 0
end

puts total

And the golf…

# Score: 69
a,b,c,t=1,0,0,0;while(c<4000000):b=a;a=c;c=a+b;t+=c if c%2==0;end;p t

EDIT!

I knew there was a way to do it without that stupid current_fib variable, but I couldn’t get it to work at first. Then, as soon as I post, I realised what I was doing wrong.

one_back, two_back, total = 1, 1, 0

while one_back < 4_000_000 do
  total += one_back if one_back % 2 == 0
  current = one_back
  one_back += two_back
  two_back = current
end

puts total

And, even better, it improved my golf score!

# Score: 63
a,b,t=1,1,0;while a<4000000:t+=a if a%2==0;c=a;a+=b;b=c;end;p t

Jim Weirich on Connascence

I was recently at RailsConf in Las Vegas. For me, the stand-out best talk was Jim Weirich’s session on Connascence, “Writing Modular Applications.” I, unfortunately, couldn’t find a video of the talk he gave at RailsConf, but he gave the same talk at Mountain West Ruby Conf 2009, and you can watch a video of that here. I truly believe that this is one of those things that every programmer should watch. What are you doing still reading this? Go watch.

Project Euler #1

I recently signed up for Project Euler. I’ve decided to see how far I can get, both just solving the problems and “golfing” them. If you’re not familiar with programming golf, it’s when you try to write as few lines of code and characters as possible.

So far, I’ve only solved the first problem, as I’ve just started. I’m looking forward to doing more in the future. If you’re interested, here’s my solution, along with my golf solution.

# If we list all the natural numbers below 10 that are
# multiples of 3 or 5, we get 3, 5, 6 and 9. The sum
# of these multiples is 23.
# Find the sum of all the multiples of 3 or 5 below 1000.

total = 0
1000.times do |i|
  if i % 3 == 0 or i % 5 == 0
    total += i
  end
end
puts total

# Score: 45
t=0;1000.times{|i|t+=i if i%3==0||i%5==0};p t

Single Quotes or Double Quotes?

Josh Susser recently had an interesting article about Symbols. I had noticed that in many places where I thought I should be able to use symbols, I had to use strings (the routes file comes to mind).

Reading that post got me thinking about the differences between ’strings’ and “strings.” I knew that “strings” allowed you to do mid-string insertions:

puts "Your variable is #{my_var}."

On the other hand, with ’strings’ you had to use plus signs:

'Your variable is ' + my_var + '.'

I began to wonder if one was faster than the other. I had been told that single quotes were faster, but I had no idea if this was true, or by how much. I decided to run some benchmarks to see how quickly a string could be saved using the different methods.

require 'benchmark'
test_variable = 'a string'

Benchmark.bm do |x|
  x.report("Single Quotes, no insertion") {
    10000.times do
      str = 'Text without insertion'
    end
  }

  x.report("Double Quotes, no insertion") {
    10000.times do
      str = "Text without insertion"
    end
  }

  x.report("Single Quotes, with insertion") {
    10000.times do
      str = 'Inserting ' + test_variable + ' into a string.'
    end
  }

  x.report("Double Quotes, with insertion") {
    10000.times do
      str = "Inserting #{test_variable} into a string."
    end
  }
end

It turns out that single quotes are, in fact, slower than double quotes. Here’s my results:

                               user       system     total    real
Single Quotes, no insertion    0.000000   0.000000   0.000000 (  0.003485)
Double Quotes, no insertion    0.010000   0.000000   0.010000 (  0.002694)
Single Quotes, with insertion  0.010000   0.000000   0.010000 (  0.011286)
Double Quotes, with insertion  0.010000   0.000000   0.010000 (  0.009462)


Now keep in mind, my benchmarks were based on saving 10,000 strings. That means for every string using single quotes you have is wasting over 0.00000018 seconds. In other words, you would need to convert about 5.4 million single-quoted strings to double-quoted string to save one second of running time.

Happy coding!