Software Alternatives & Reviews


Recommended and mentioned products

  1. An easy to use, powerful, and reliable system to process and distribute data.

    Is Apache Airflow HIPAA Compliant & Common Healthcare Stacks? about about 2 months ago

    As part of your analysis take a look at Apache NiFi. It may not be right for you but focus on the features. It is more mature, spun out of the NSA, and has a pretty decent ecosystem. It is a Java focused stack with some Python friendly capabilities, eg can execute Jython if you want it too. Airflow is a really neat tool and I am not trying to persuade you away from exploring its use just offering a different...
  2. StreamSets provides Continuous Ingest technology for the next generation of big data applications.

  3. Airflow is a platform to programmaticaly author, schedule and monitor data pipelines.

    BigQuery vs Relational Databases about 13 days ago

    However, my typical go-to is to utilize something like [DBT]( or [Airflow]( to orchestrate sets of related queries. There are a lot of powerful patterns you can adopt by using these kind of orchestration services in conjunction with BigQuery.
  4. AWS Data Pipeline is a cloud-based data workflow service that helps you process and move data between different AWS services and on-premise.

    Any data engineers familiar with building pipelines in AWS? about 5 months ago

    Unfortunately there's just so many options for data ingest. Any programming language could be used, and there's plenty of off-the-shelf software and SaaS solutions to do it too. For example it could be done with AWS Data Pipeline ( or maybe there's just a EC2 virual machine running some custom python code that is doing it.
  5. Fully managed extract, transform, and load (ETL) service

    Data Factory about 7 months ago

    Looks like that is a ETL system, so