Software Alternatives & Reviews

AWS Glue VS AWS Data Pipeline

Compare AWS Glue VS AWS Data Pipeline and see what are their differences

AWS Glue logo AWS Glue

Fully managed extract, transform, and load (ETL) service

AWS Data Pipeline logo AWS Data Pipeline

AWS Data Pipeline is a cloud-based data workflow service that helps you process and move data between different AWS services and on-premise.
AWS Glue Landing Page
AWS Glue Landing Page
AWS Data Pipeline Landing Page
AWS Data Pipeline Landing Page

AWS Glue details

Categories
Big Data Tools ETL Data Workflow
Website aws.amazon.com  

AWS Data Pipeline details

Categories
Data Pipelines ETL Data Migration Databases Automation Data Workflow
Website aws.amazon.com  

AWS Glue videos

Build ETL Processes for Data Lakes with AWS Glue - AWS Online Tech Talks

More videos:

  • Review - Getting Started with AWS Glue Data Catalog
  • Review - Bajaj Housing Finance Limited: Serverless Data Pipelines with AWS Glue and Amazon Aurora PGSQL

AWS Data Pipeline videos

AWS re:Invent BDT 201: AWS Data Pipeline: A guided tour

Category Popularity

0-100% (relative to AWS Glue and AWS Data Pipeline)
63
63%
37%
37
69
69%
31%
31
69
69%
31%
31
100
100%
0%
0

Reviews

These are some of the external sources and on-site user reviews we've used to compare AWS Glue and AWS Data Pipeline

AWS Glue Reviews

Top 7 ETL Tools for 2021
Notably, AWS Glue is serverless, which means that Amazon automatically provisions a server for users and shuts it down when the workload is complete. AWS Glue also includes features such as job scheduling and “developer endpoints” for testing AWS Glue scripts, improving the tool’s ease of use.
Source: www.xplenty.com

AWS Data Pipeline Reviews

We have no reviews of AWS Data Pipeline yet.
Be the first one to post

Social recommendations and mentions

Based on our record, AWS Glue should be more popular than AWS Data Pipeline. It has been mentiond 10 times since March 2021. We are tracking product recommendations and mentions on Reddit, HackerNews and some other platforms. They can help you identify which product is more popular and what people think of it.

AWS Glue mentions (10)

  • Deploying a Data Warehouse with Pulumi and Amazon Redshift
    So in the next post, we'll do that: We'll take what we've done here, add a few more components with Pulumi and AWS Glue, and wire it all up with a few magical lines of Python scripting. - Source: dev.to / 3 days ago
  • Serverless Event Driven AI as a Service
    Once it's in a Data Lake then you have different options depending on the analytics you need. For more advanced constant analytics then you could look into Amazon Kinesis Data Analytics instead of Firehose to S3, but for Ad-Hoc queries then this is where Glue and Athena come in. - Source: dev.to / about 1 month ago
  • Well-Architected Review - Part II - Operational Excellence
    You will want to use metrics based on operations outcomes to gain useful insights. Now you want to do analytics on your logs and use Cloudwatch Logs Insights or store the logs in Amazon S3, which then triggers an AWS Glue crawler to create an AWS Glue Data Catalog that then can be queried using Amazon Athena using standard SQL. The results can be visualized in Amazon Quicksight. - Source: dev.to / about 2 months ago
  • AWS Lambda storage options
    Storing data in S3 has an additional benefit, given how well it integrates with other AWS services. For instance, you can use Amazon Athena to query your S3 data, or Amazon Rekognition to analyze it. Additionally you can use AWS Glue to perform extract, transform, and loan (ETL) operations. To create ad hoc visualizations and business analysis reports, Amazon QuickSight can connect to your S3 buckets and produce... - Source: dev.to / 2 months ago
  • Keep Athena up to date with only the most recent data from partitions
    Not 100% if this is what you need, but look into integrating AWS Glue. It should be able to keep the data source that Athena uses up to date real time or close to it, from what I understand. - Source: Reddit / 5 months ago
View more

AWS Data Pipeline mentions (2)

  • Ingestion of live data
    Also, if you're doing this for an employer, and they have some deeper pockets, there is also AWS Data Pipeline. - Source: Reddit / 8 months ago
  • Any data engineers familiar with building pipelines in AWS?
    Unfortunately there's just so many options for data ingest. Any programming language could be used, and there's plenty of off-the-shelf software and SaaS solutions to do it too. For example it could be done with AWS Data Pipeline (https://aws.amazon.com/datapipeline) or maybe there's just a EC2 virual machine running some custom python code that is doing it. - Source: Reddit / over 1 year ago

What are some alternatives?

When comparing AWS Glue and AWS Data Pipeline, you can also consider the following products

Xplenty - Xplenty is the #1 SecurETL - allowing you to build low-code data pipelines on the most secure and flexible data transformation platform. No longer worry about manual data transformations. Start your free 14-day trial now.

AWS Database Migration Service - AWS Database Migration Service allows you to migrate to AWS quickly and securely. Learn more about the benefits and the key use cases.

Skyvia - Free cloud data platform for data integration, backup & management

Talend Data Integration - Talend offers open source middleware solutions that address big data integration, data management and application integration needs for businesses of all sizes.

Starfish ETL - The Starfish ETL (Extract Transform Load) Suite is a CRM integration and migration tool.

Apache Airflow - Airflow is a platform to programmaticaly author, schedule and monitor data pipelines.

User reviews

Share your experience with using AWS Glue and AWS Data Pipeline. For example, how are they different and which one is better?
Log in or Post with