Meltano VS Singer

Meltano Reviews

Singer Reviews

Meltano mentions (26)

• AI product development is being held back by data engineering

  Hey HN, Arch CEO here! Our team has been working at the intersection of data engineering and software engineering for a few years now with Meltano (https://meltano.com), and this year, the rise in Generative AI has made it clear that the bottleneck in unlocking the potential value of data has shifted from data integration on data teams to data engineering on software teams, so we’ve decided to do something about... - Source: Hacker News / 6 months ago

• How useful is Airbytes in production pipelines?

  We use Meltano for (EL) and Prefect for scheduling. Is not click-ops, but works very well for us! Behind the scenes Meltano wraps up Singer spec similarly like Airbyte does with its connectors. Before that we tried Airbyte (~5 months ago?) and it was so bad.. We could not choose the columns to replicate and the connectors were unstable i.e. Skipping data, all sort of odd errors and so on.. Source: 12 months ago

• Ask HN: Who is hiring? (May 2023)

  Meltano's all-remote team and community of thousands are on a mission to enable everyone to realize the full potential of their data. To this end, we are bringing software engineering best practices to data teams in the form of an open-source DataOps platform that we envision becoming the foundation of every team's ideal data stack. Our public company handbook (https://handbook.meltano.com/) has all the details on... - Source: Hacker News / about 1 year ago

• Ask HN: Who is hiring? (April 2023)

  Meltano | Full-Time | Remote | https://meltano.com Meltano's all-remote team and community of thousands are on a mission to enable everyone to realize the full potential of their data. To this end, we are bringing software engineering best practices to data teams in the form of an open-source DataOps platform that we envision becoming the foundation of every team's ideal data stack. Our public company handbook... - Source: Hacker News / about 1 year ago

• If dbt is the "T" part of an "ELT", what do you use for "EL"?

  We switched from AWS Glue to Meltano for the EL part of ELT and it's been a joy to use. We're moving so much faster now. Source: about 1 year ago

Singer mentions (7)

• Why do companies still build data ingestion tooling instead of using a third-party tool like Airbyte?

  Coincidently, I saw a presentation today on a nice half-way-house solution: using embeddable Python libraries like Sling and dlt - both open-source. See https://www.youtube.com/watch?v=gAqOLgG2iYY There is also singer.io which is more of a protocol than a library, but can also be installed although it looks like it is a true community effort and not so well maintained. Source: 5 months ago

• Data sources episode 2: AWS S3 to Postgres Data Sync using Singer

  Singer is an open-source framework for data ingestion, which provides a standardized way to move data between various data sources and destinations (such as databases, APIs, and data warehouses). Singer offers a modular approach to data extraction and loading by leveraging two main components: Taps (data extractors) and Targets (data loaders). This design makes it an attractive option for data ingestion for... - Source: dev.to / 12 months ago

• CDC (Change Data Capture) with 3rd party APIs

  Or you could build your own such system and run it on Airflow, Prefect, Dagster, etc. Check out the Singer project for a suite of Python packages designed for such a task. Quality varies greatly, though. Source: over 1 year ago

• Looking to build a database for BI reports

  This is good advice and I think Airbyte created a great product here. I tried singer.io and pipewise but Airbyte is much better in my opinion and I love the UI. Source: over 2 years ago

• Recommendation for approach for populating and refreshing new data lake

  Suspect my question should have been regarding FREE systems, rather than BUYING a system. Sounds like singer.io will do what I need. Source: almost 3 years ago