I had no problem with awswrangler (https://github.com/aws/aws-sdk-pandas) and it supports reading and writing partitions which was really helpful and a few other optimizations that made it a great tool. Source: 6 months ago

Awslabs has developed their own package for this and given it's for their product, seem likely to maintain it. https://github.com/awslabs/aws-data-wrangler. Source: over 2 years ago

AWS data wrangler works well. it's a wrapper on pandas: https://github.com/awslabs/aws-data-wrangler. Source: over 2 years ago

Yep, agreed. Go is a great language for AWS Lambda type workflows. Python isn't as great (Python Lambda Layers built on Macs don't always work). AWS Data Wrangler (https://github.com/awslabs/aws-data-wrangler) provides pre-built layers, which is a work around, but something that's as portable as Go would be the best solution. - Source: Hacker News / about 3 years ago

So how do we trigger such a long-running process from a Rails request? The first option that comes to mind is a background job run by some of the queuing back-ends such as Sidekiq, Resque or DelayedJob, possibly governed by ActiveJob. While this would surely work, the problem with all these solutions is that they usually have a limited number of workers available on the server and we didn’t want to potentially... - Source: dev.to / about 2 years ago

Several gems support job queues and background processing in the Rails world — Delayed Job and Sidekiq being the two most popular ones. - Source: dev.to / about 2 years ago

Back in the day, before Sidekiq and such, we used Delayed Job https://github.com/collectiveidea/delayed_job. Source: over 2 years ago

There are a few of popular systems. A few need a database, such as Delayed::Job, while others prefer Redis, such as Resque and Sidekiq. - Source: dev.to / about 3 years ago