How to run a really long task from a Rails web request

Ruby On Rails Data Integration Stream Processing

Sidekiq Landing Page
1

Sidekiq

Sidekiq is a simple, efficient framework for background job processing in Ruby
Pricing:
- Open Source
So how do we trigger such a long-running process from a Rails request? The first option that comes to mind is a background job run by some of the queuing back-ends such as Sidekiq, Resque or DelayedJob, possibly governed by ActiveJob. While this would surely work, the problem with all these solutions is that they usually have a limited number of workers available on the server and we didn’t want to potentially block other important background tasks for so long.

#Ruby On Rails #Ruby #Background Processing 21 social mentions
Resque Landing Page

2

Resque

Resque is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later.

So how do we trigger such a long-running process from a Rails request? The first option that comes to mind is a background job run by some of the queuing back-ends such as Sidekiq, Resque or DelayedJob, possibly governed by ActiveJob. While this would surely work, the problem with all these solutions is that they usually have a limited number of workers available on the server and we didn’t want to potentially block other important background tasks for so long.

#Data Integration #Stream Processing #Web Service Automation 5 social mentions
delayed_job Landing Page

3

delayed_job

Database based asynchronous priority queue system -- Extracted from Shopify - collectiveidea/delayed_job

So how do we trigger such a long-running process from a Rails request? The first option that comes to mind is a background job run by some of the queuing back-ends such as Sidekiq, Resque or DelayedJob, possibly governed by ActiveJob. While this would surely work, the problem with all these solutions is that they usually have a limited number of workers available on the server and we didn’t want to potentially block other important background tasks for so long.

#Data Integration #Stream Processing #Web Service Automation 4 social mentions