A week or so back we started porting projects to Rails 3 and all new projects were created using Rails 3. The project that I was working on (name irrelevant) needed some sort of concurrency. In the past we already tried a few existing Gems but there was always something missing or there was simply too much of something. Is most cases it was also hard to customize the gem or it simply didn’t live up to our expectations.
In the past we had also tried to create our own concurrency implementation but these either-way failed or were just too specifically designed. If you wanted to do something else than the feature it was designed for; you would have to rewrite parts of the code. You may call this design error, but it was simply not the intention to create a flexible concurrency implementation.
So our search for a new concurrency Gem went on; we tried two and came to the conclusion we would have to write something our self. I was given the task to create a new Gem. A new concurrency (queueing) Gem which would have to be flexible, maintainable and would have to be not as specific as we have been in the past. Or in fact this concurrency Gem had to be able to do all sort of tasks. From sending out mails to importing users from a CSV file.
I’d tried delayed_jobs and liked their implementation very much. The one thing we all agreed at here at the office was we didn’t want to have a separate Rake task which would require to be run in order to do some sort of concurrency. We wanted to be able to have the task been done within the application with whatever concurrency we specified; be it Forking or Threading or a simple yield, it all had to be possible. So with delayed_job in mind I started to design the Gem and came up with the name Runner (obvious name!).
I took me two days to implement a very basic version of the Gem. The only thing it could do back then was fork a method in the background without the need of an additional Rake task. At the current time of writing it has the following features:
- Supports various implementations of Concurrency (forking, threading and yielding. But the last isn’t really concurrency)
- Developers using the Gem may add different kind of concurrency to gem by simply giving the concurrency class the Runner initializer
- It has the ability to queue tasks which means you more or less park the tasks for later.
- It may immediately run tasks in the background using the various concurrency methods.
- Threading tasks requires no changes to your current classes.
- Extending the backend (currently comes with ActiveRecord) only requires you to implement database specific code (scopes, etc.)

