Based on our record, Dask should be more popular than Azkaban. It has been mentiond 16 times since March 2021. We are tracking product recommendations and mentions on various public social media platforms and blogs. They can help you identify which product is more popular and what people think of it.
We're using a lot of Python. In addition to these, gridMET, Dask, HoloViz, and kerchunk. Source: about 2 years ago
I wrote this for speeding up the RPC messaging in dask, but figured it might be useful for others as well. The source is available on github here: https://github.com/jcrist/msgspec. Source: about 2 years ago
Dask: Distributed data frames, machine learning and more. - Source: dev.to / over 2 years ago
To do that, we are efficiently using Dask, simply creating on-demand local (or remote) clusters on task run() method:. - Source: dev.to / over 2 years ago
I’m quite sure dask helps and has a pandas like api though will use disk and not just RAM. Source: over 2 years ago
Not sure if https://azkaban.github.io/ would fit your use case. Source: almost 2 years ago
I used this once, was pretty nice: https://azkaban.github.io/. Source: about 2 years ago
Apache Azkaban is a batch workflow job scheduler to help developers run Hadoop jobs. The open-sourced platform “resolves ordering through job dependencies” and offers an intuitive web interface to help users maintain and track workflows. Source: about 2 years ago
Pandas - Pandas is an open source library providing high-performance, easy-to-use data structures and data analysis tools for the Python.
Apache Airflow - Airflow is a platform to programmaticaly author, schedule and monitor data pipelines.
Luigi - Luigi is a Python module that helps you build complex pipelines of batch jobs.
NumPy - NumPy is the fundamental package for scientific computing with Python
RunDeck - RunDeck is an open source automation service with a web console, command line tools and a WebAPI.
Apache Spark - Apache Spark is an engine for big data processing, with built-in modules for streaming, SQL, machine learning and graph processing.