Software Alternatives, Accelerators & Startups

Shipyard VS AWS Glue

Compare Shipyard VS AWS Glue and see what are their differences

Shipyard logo Shipyard

Shipyard’s your go-to cloud-based DataOps platform for secure workflow orchestration, extraction, transformation, reverse ETL, monitoring, alerting, and more.

AWS Glue logo AWS Glue

Fully managed extract, transform, and load (ETL) service
  • Shipyard Empower Your Data Team to Outpace Business Requests
    Empower Your Data Team to Outpace Business Requests //
    2024-05-29

Introducing Shipyard – a cloud-based DataOps platform for data extraction, transformation, reverse ETL, monitoring, alerting, and workflow orchestration.

Security is our top priority. We ensure your data's safety with HIPAA and SOC 2 Type II compliance, ephemeral file storage, SSO, and role-based access control (RBAC).

Scalability is built into our platform, adapting seamlessly to your growing needs.

Simplicity is key. Shipyard caters to all technical backgrounds, from data experts to less technical users, thanks to our pioneering hybrid and custom code approach.

Our goal is to streamline your data operations, making them secure, scalable, and accessible to everyone.

  • AWS Glue Landing page
    Landing page //
    2022-01-29

Shipyard videos

Shipyard Overview

AWS Glue videos

Build ETL Processes for Data Lakes with AWS Glue - AWS Online Tech Talks

More videos:

  • Review - AWS re:Invent BDT 201: AWS Data Pipeline: A guided tour
  • Review - Getting Started with AWS Glue Data Catalog
  • Review - Bajaj Housing Finance Limited: Serverless Data Pipelines with AWS Glue and Amazon Aurora PGSQL

Category Popularity

0-100% (relative to Shipyard and AWS Glue)
Web Service Automation
24 24%
76% 76
Data Integration
0 0%
100% 100
Automation
100 100%
0% 0
ETL
0 0%
100% 100

Questions and Answers

As answered by people managing Shipyard and AWS Glue.

What makes your product unique?

Shipyard's answer

Shipyard deploys in days, not weeks or months.

Shipyard is built for people of all technical backgrounds thanks to no code, open source low code, and full code options. Someone less technical, like a data analyst can deploy and build workflows with Shipyard just as easily as a data engineer who may choose to partially if not fully code within Shipyard.

Support is immediate. Our team is always available, as opposed to open source tools that primarily relies on communities for answers. The delay from the latter can prove to be an issue for many organizations.

You can test and deploy from your local environment in Shipyard, unlike all other orchestrators.

YAML configuration that syncs with drag and drop.

150 and counting open source low code blueprints allowing you to get started faster and not have to start from scratch.

Shipyard provides the infrastructure which allows for faster scaling, more elasticity, and a reduction in costs.

How would you describe your primary audience?

Shipyard's answer

Today our primary audience is data engineers, analytics engineers, data analysts, BI analysts, and heads of data/CDOs. We're currently a best fit for organizations with 500 of fewer employees, but by the end of Q1 2024 our primary audience will shift to large mid-market/small enterprises. We're product led growth on one hand, we offer a forever free trial, but we compliment it with a sales team to work with customers, which are primarily large, which want to work with a salesperson and other data experts.

Why should a person choose your product over its competitors?

Shipyard's answer

Lack of complexity, ease of use, and speed.

While just as powerful as any of our competitors, Shipyard is significantly easier to deploy, use, and to onboard others to Shipyard regardless of their technical background.

Shipyard is also significantly less expensive. Open source is free in that you don't have to pay to access the tools. But you do need the manpower and budget to code and built upon the open source to get the orchestration solution that is specific to the needs of your organizations.

You have a partner in Shipyard, one who's always available whether it's technical in nature or you just need to bounce ideas off or get another perspective from another data expert. We're strong believers in giving back to our data community and supporting our customers and partners, and you can always count on us exactly when you need us.

We're extraordinarily nimble. Whether you need a new integration, blue print, or other addition to satisfy your needs, we have the team to meet your needs, fast.

Who are some of the biggest customers of your product?

Shipyard's answer

  • Apache Airflow
  • Prefect
  • Dagster
  • Mage

What's the story behind your product?

Shipyard's answer

Our founding story aligns with our product philosophy. Our story is very much our strategy.

Our co-founder, Blake Burch, was a full-time data practitioner.

He had a love-hate relationships with low-code tools.

On the one hand, they made his life so much easier. Blake could click a few buttons and get access to data that he needed in five minutes instead of spending a week of work fiddling with new API endpoints.

On the other hand, using any of these tools meant that Blake had a few inevitable tradeoffs: - He always seemed to run into situations where the tool didn't support something that he needed for his specific use case. As a result, Blake would be stuck writing his own scripts that had to live on a separate platform, disconnected from his other solutions. This meant that Blake was then responsible for managing solutions across multiple tools. - If the low-code tool offered the ability to run code, it was always limited in some way. For some tools, you could only use pre-installed packages. For others, you were required to format the data in a very specific format. Some even restricted to small runtimes, small data, and small memory. - While the low-code tools were easy to use, the more Blake relied on them, the more of a black box his business' solutions became. Nobody knew exactly how they worked under the hood which created an extreme level of lock-in.

Blake and his co-founder, Eric Elsken, designed Shipyard to address these issues. Shipyard believes that low-code and your code should be able to work together seamlessly without any tradeoffs. To maximize your impact, data practitioners deserve a platform that affords them the flexibility to solve problems how they need. They have options.

That's why with Shipyard: - You can run your own Python, Node, or Bash code directly in the platform without any changes. There are no limitations on the script functionality that can be executed. - You can run our low-code Library Blueprints in the platform by providing a few simple inputs. These Blueprints are 100 percent open-source Python packages that you can dig into and even help contribute to. If you ever need to tweak the functionality - go for it. The code works outside of Shipyard too, so you can test and port the functionality as needed. - You can turn your own scripts into low-code Blueprints that look and feel exactly like our Library Blueprints. This is perfect for proprietary business logic that needs to be reusable by anyone in the organization.

In the ideal situation, a workflow should be a mixture of low-code solutions plus your own code when absolutely necessary.

You have more flexibility with Shipyard than any data orchestration tool that's ever been built.

User comments

Share your experience with using Shipyard and AWS Glue. For example, how are they different and which one is better?
Log in or Post with

Reviews

These are some of the external sources and on-site user reviews we've used to compare Shipyard and AWS Glue

Shipyard Reviews

We have no reviews of Shipyard yet.
Be the first one to post

AWS Glue Reviews

10 Best ETL Tools (October 2023)
AWS Glue is an end-to-end ETL offering intended to make ETL workloads easier and more integratable with the larger AWS ecosystem. One of the more unique aspects of the tool is that it is serverless, meaning Amazon automatically provisions a server and shuts it down following the completion of the workload.
Source: www.unite.ai
15+ Best Cloud ETL Tools
AWS Glue is a serverless data integration service designed to streamline analytics, machine learning, and app development tasks. It discovers, prepares, and moves data from a myriad of sources and offers a seamless integration experience. AWS Glue's inclusive toolset and automatic scaling let you focus on gaining insights from data instead of managing infrastructure.
Source: estuary.dev
Top 14 ETL Tools for 2023
Notably, AWS Glue is serverless, which means that Amazon automatically provisions a server for users and shuts it down when the workload is complete. AWS Glue also includes features such as job scheduling and “developer endpoints” for testing AWS Glue scripts, improving the tool’s ease of use.
A List of The 16 Best ETL Tools And Why To Choose Them
Better yet, when interacting with AWS Glue, practitioners can choose between a drag-and-down GUI, a Jupyter notebook, or Python/Scala code. AWS Glue also offers support for various data processing and workloads that meet different business needs, including ETL, ELT, batch, and streaming.
Top 10 AWS ETL Tools and How to Choose the Best One | Visual Flow
The AWS Glue Data Catalog contains table and job definitions, and other control information. It automatically generates statistics and registers partitions, so data queries can run more efficiently. The catalog also supports an extended history for schema versions, allowing you to see how data has changed over time.
Source: visual-flow.com

Social recommendations and mentions

Based on our record, AWS Glue seems to be more popular. It has been mentiond 13 times since March 2021. We are tracking product recommendations and mentions on various public social media platforms and blogs. They can help you identify which product is more popular and what people think of it.

Shipyard mentions (0)

We have not tracked any mentions of Shipyard yet. Tracking of Shipyard recommendations started around Mar 2021.

AWS Glue mentions (13)

  • Build Your Movie Recommendation System Using Amazon Personalize, MongoDB Atlas, and AWS Glue
    AWS Glue is a fully managed extract, transform, and load (ETL) service that makes it easy to prepare and load data for analysis. It helps bridge the gap between our MongoDB Atlas data and the services we'll use for recommendation. - Source: dev.to / 3 months ago
  • Using Snowflake data hosted in GCP with AWS Glue
    AWS Glue is a fully managed extract, transform, and load (ETL) service provided by Amazon Web Services (AWS). It is designed to make it easy for users to prepare and load their data for analysis. AWS Glue simplifies the process of building and managing ETL workflows by providing a serverless environment for running ETL jobs. - Source: dev.to / 4 months ago
  • How to check for quality? Evaluate data with AWS Glue Data Quality
    It is serverless data integration service to allow you to easily scale your workloads in preparing data and moving transformed data into a target location. - Source: dev.to / 12 months ago
  • Deploying a Data Warehouse with Pulumi and Amazon Redshift
    So in the next post, we'll do that: We'll take what we've done here, add a few more components with Pulumi and AWS Glue, and wire it all up with a few magical lines of Python scripting. - Source: dev.to / over 1 year ago
  • Serverless Event Driven AI as a Service
    Once it's in a Data Lake then you have different options depending on the analytics you need. For more advanced constant analytics then you could look into Amazon Kinesis Data Analytics instead of Firehose to S3, but for Ad-Hoc queries then this is where Glue and Athena come in. - Source: dev.to / over 1 year ago
View more

What are some alternatives?

When comparing Shipyard and AWS Glue, you can also consider the following products

Zapier - Connect the apps you use everyday to automate your work and be more productive. 1000+ apps and easy integrations - get started in minutes.

Xplenty - Xplenty is the #1 SecurETL - allowing you to build low-code data pipelines on the most secure and flexible data transformation platform. No longer worry about manual data transformations. Start your free 14-day trial now.

n8n.io - Free and open fair-code licensed node based Workflow Automation Tool. Easily automate tasks across different services.

AWS Database Migration Service - AWS Database Migration Service allows you to migrate to AWS quickly and securely. Learn more about the benefits and the key use cases.

Zenaton - Powerful & Easy Automation for Developers

Skyvia - Free cloud data platform for data integration, backup & management