Since the introduction of ActiveJob in Ruby on Rails 4.2 many years ago, it’s been a foregone conclusion that most Rails apps will need to rely on a job queue of some sort. At a very minimum you’ll probably want to use a queue to take necessary slow-running tasks like email delivery outside of the request lifecycle.

Mike Perham’s Sidekiq is probably the most widely-used and de-facto choice for many teams. There is, however, another queueing solution that I want to highlight because, for many apps, it cuts down on moving parts while increasing both integrity and performance (with a couple compromises that should be well-understood).

Enter Que

Que is a job queue that leverages PostgreSQL’s native advisory locks to optimize interactions with the queue’s persistence store. Sidekiq and many other job queues are built atop Redis, which, though it does have a level of durability (see https://redis.io/topics/persistence) and is certainly fast, is not fully durable by default (many default configurations sync to disk once a second). Que relies on PostgreSQL as its persistence store, which means guarantees of durability in addition to one fewer dependency. With its use of advisory locks, it also provides remarkably good performance.

There is of course a tradeoff you are making here which is that, if you are using your primary application database to house Que’s jobs table, you may subject it to a lot of writes if you have a high volume of jobs. This can be mitigated by using a secondary and dedicated database instance for Que’s jobs table. In anything short of a very large scale though this is likely a premature optimization.

Another killer feature of Que is the ability to run your worker pool within your web process. This may not be something you will do in your particular case depending on your job volume, but again, at something short of very large scale it is an incredible benefit with respect to simplifying your infrastructure and deployment. Essentially you no longer have separate queue workers that you need to configure and integrate with your deployment process. Deploying your web application code will implicitly mean that your job queues also get a code update and seamless restart.

You can find details about configuring an in-process worker pool here: https://github.com/chanks/que/blob/master/docs/advanced_setup.md

The basic configuration for Phusion Passenger is a simple matter of adding a few lines to your config.ru file:

Installing Que

Installing Que for your Rails app is very straightforward. Simply add it to your Gemfile:

Then bundle install and, finally, use the generator supplied by the gem to create a migration that will add the que_jobs table to your database schema:

 ActiveJob support

Que’s ActiveJob support is not as well documented as I’d like, but the necessary configuration is relatively simple.

First you need to set :que as the QueueAdapter. You will also need to set config.action_mailer.deliver_later_queue_name = '' if you want Que to pick up any emails enqueued with Rails’ stock deliver_later functionality (or add a special Que worker configuration to pick up jobs enqueued with the queue name “mailers”, since this is the default queue name used if not overridden with the deliver_later_queue_name configuration option).

You can now define your jobs as subclasses of ActiveJob::Base/ApplicationJob and they should work seamlessly with Que. Note that as Que is a first-class ActiveJob integration, GlobalID is supported, so you can pass actual ActiveRecord instances into jobs rather than IDs and trust that they will be correctly deserialized when the job is picked up. Let’s take a simple job that might invoke a method to reach out over the network and fetch a user avatar from an external service to store locally. This might look as follows:

You would enqueue one of these via ActiveJob’s perform_later method:

This will drop it into Que’s Postgres-backed job queue to be worked by the next available worker.

Give it a try

I hope this guide has shown how quick and easy it is to get Que set up with a Ruby on Rails application. If the idea of a fast job queue with fewer moving parts and great durability appeals to you, I recommend giving Que a try.