Working on Ruby on Rails we come across many situations were we need to run programs in the background. There are several methods to approach the problem; each of them having its share of advantages and disadvantages. But before going into those methods itself, let us first look at the problems faced.

The Problem

Rails is pretty much self sufficient when it comes to web frameworks, but when it comes to long running background process, it runs into problems if not handled properly. For example – a large file needs to be uploaded from the browser and the web application processes it in the backend. It ends up in a UI that is unresponsive, a clogged up server which can’t take up any more requests etc etc.

The Solution

The solution is simple in theory, we have to change the program such that interface will just initiate the process and the actual processing is done in background. But this generates new problems. How do you now show the status and result of the operations in the user interface?. Luckily there are gems like delayed_job, Backgroun Job aka bj and workling which does exactly that, so you don’t really need to reinvent the wheel. We’ll take a look at one of these gems – delayed_job

delayed_job

Say we need an app which will upload a file, then do some time consuming operations on it and finally display a message “File uploaded” on success. Let’s say the method which will be called in the controller is processfile(filename), which will take a lot of time to execute.

This is how controller will look

class UploadController < ApplicationController
  def upload
    Upload.processfile(params[:dump][:tempfile])
  end
end

And Upload class as below

class Upload
  def processfile(file)
    sleep(30)
  end
end

Supposing the processfile method takes 30 seconds to complete (which it does in the above example), all further URL calls to the server after the file upload will be on hold for 30 seconds – not good. This is where delayed_job comes in. So how do we integrate delayed_job here? Let’s go step-by-step.

So first thing we need to do is add the gem to our Gemfile and run bundle install.

gem 'delayed_job_active_record'

After bundle install finishes, in the app home folder, run the below command:

rails generate delayed_job

This will generate the migration files required for delayed_job. Now we have to run database migration.

rake db:migrate

This will create the tables required by the gem. That lays the groundwork for using delayed_job. Let’s now start looking into the changes we need to make in code.

Luckily, we just need a small change in code as below:

Upload.processfile(filename)

to

Upload.delay.processfile(filename)

As you have noticed the only change in the code is that a “delay” is added in between the object name and the object’s method. Viola! Your job is now properly delayed and will not block other requests to the server. And if the delayed method fails for some reason you can check the last_error field in delayed_job table which shows the error encountered while running the method.

How does it work then? What happens here is that delayed_job added a method delay to all the objects. The delay method will create an entry in the delayed_jobs table with serialized object as yaml. There is a delayed_job daemon running in the background which picks up this job from the table and runs the appropriate method that is called after delay.

delayed_job uses either activerecord or mongoid to store the delayed job details. But what if you don’t want to any of these and only have Redis to work with. In this case we can use the MobME async_service gem which is easy to install and use. And it’s open source! Let’s take a look.

async_service

async_service uses redis queue as backend and we have to add items to be processed to a queue. async_service gives us the freedom of using any queue name and what to do after we pop an item from the queue. Let’s look in detail

First you have to add async_service to the Gemfile and run bundle install

gem "async_service"

After installation of the gem we can straight away start using it. Compared to delayed_job configuration of async_service is the easiest. So now lets take a look on how to use the gem with the help of the example given in async_service homepage.

class CalculatorMachine < AsyncService::Worker
  # You need to set a service name @service_name = 'in.mobme.calculator_machine'
  # This function is called on run
  def work
    loop do
      queue.remove("work_queue") do |item|
        result = item[:a] + item[:b]
        queue.add("result_queue", result)
      end
      sleep 5
    end
  end
end

Inorder to use the gem the target class has to inherit from the AsyncService::Worker. Inside this class we have to define a method “work” which decides what is to be done with the data that is present in queue. The method work is triggered internally when we call the run method (inherited from AsyncService::Worker). Since this example is for a calculator it pops an element from the queue “work_queue”, adds the elements and saves them to another queue “result_queue”.

To use this class make an instance of the class and call the run method on that instance.

calculator = CalculatorMachine.new
calculator.run

So to add two numbers we will insert the numbers to “work_queue” and we can read the output, which is the sum of the numbers, from the “result_queue” of Redis.

More complex tasks

For most projects this would be sufficient. But how about a requirement (that we had internally) that needs to run a maintenance job daily which takes hours to complete. Add to that, it was not initiated by user, i.e there won’t be a user who asks for it to be run and it needs to start by itself everyday. First solution was to set a cron job which does the same. But it outlived its usefulness when multiple processes needed to be run at multiple times daily, which meant to create multiple cron jobs or create a single script which can handle this. We went in for the latter solution since that was more maintainable.

Rails Runner

Enter Rails Runner – this was the perfect solution to our earlier problem. Rails Runner helps us to run scripts in the Rails application environment. All supporting files and gems needed for the application environment is taken care of automatically and we don’t need to tinker with all that too much and rather can concentrate mostly on the job at hand. And the script that we came up with, once started will wait for the assigned time and will start the jobs as required by specification which gives us more freedom to schedule any number of maintenance jobs at any time. And we don’t have to worry about configuring the cron in the server every time a new requirement comes, we just have to restart our rails runner.

The command for that looks like below:

~/app/folder$ rails runner path/to/file.rb