Meltano Reviews and details

Social recommendations and mentions

• AI product development is being held back by data engineering

  Hey HN, Arch CEO here! Our team has been working at the intersection of data engineering and software engineering for a few years now with Meltano (https://meltano.com), and this year, the rise in Generative AI has made it clear that the bottleneck in unlocking the potential value of data has shifted from data integration on data teams to data engineering on software teams, so we’ve decided to do something about... - Source: Hacker News / 6 months ago

• How useful is Airbytes in production pipelines?

  We use Meltano for (EL) and Prefect for scheduling. Is not click-ops, but works very well for us! Behind the scenes Meltano wraps up Singer spec similarly like Airbyte does with its connectors. Before that we tried Airbyte (~5 months ago?) and it was so bad.. We could not choose the columns to replicate and the connectors were unstable i.e. Skipping data, all sort of odd errors and so on.. Source: 11 months ago

• Ask HN: Who is hiring? (May 2023)

  Meltano's all-remote team and community of thousands are on a mission to enable everyone to realize the full potential of their data. To this end, we are bringing software engineering best practices to data teams in the form of an open-source DataOps platform that we envision becoming the foundation of every team's ideal data stack. Our public company handbook (https://handbook.meltano.com/) has all the details on... - Source: Hacker News / 12 months ago

• Ask HN: Who is hiring? (April 2023)

  Meltano | Full-Time | Remote | https://meltano.com Meltano's all-remote team and community of thousands are on a mission to enable everyone to realize the full potential of their data. To this end, we are bringing software engineering best practices to data teams in the form of an open-source DataOps platform that we envision becoming the foundation of every team's ideal data stack. Our public company handbook... - Source: Hacker News / about 1 year ago

• If dbt is the "T" part of an "ELT", what do you use for "EL"?

  We switched from AWS Glue to Meltano for the EL part of ELT and it's been a joy to use. We're moving so much faster now. Source: about 1 year ago

• Continuously import Aurora MySQL data into BigQuery

  There are other providers of CDC software. Notably Fivetran, Meltano and Airbyte. There are a few open source I personally recommend you use Google's platform since you are already going to have billing setup with them because of BigQuery. My experience is that Fivetran is expensive and Airbyte is buggy. But rather than take my opinion on the matter, consult the many threads on this sub referencing them. Source: over 1 year ago

• What's the best thing/library you learned this year ?

  Meltano for extracting and loading data. Source: over 1 year ago

• PostgreSQL to DuckDB - There and Quack Again

  I built my data pipeline to Extract some data from websites and CSV files, Load it into my database, and Transform it into a reporting-ready schema. I used Python and Pandas to extract and load some of the data and Meltano to load some additional supporting data. All of that data went into a PostgreSQL database hosted in the cloud on Azure where I then used dbt to create data models in the database optimized for... - Source: dev.to / over 1 year ago

• What should be the main point of a personal project?

  I'm learning https://meltano.com/ right now, so am building custom Taps, mostly for fun. I'm enjoying it. I'm pulling in a variety of data from https://www.geonames.org/ and Canadian weather/climate data into BigQuery. Source: over 1 year ago

• Reference Data Stack for Data-Driven Startups

  We chose Airbyte because of their open source self-hosted offering and their connector flexibility. There are other open source tools in the market like StitchData or Meltano but have limitations as discussed in this post by the Airbyte team. If you don’t want to self host extraction, consider Fivetran. - Source: dev.to / about 2 years ago

• Best ELT tool for nonprofits?

  Depending on your workload, you could give Meltano a shot, which includes some connectors / extractors out of the box. Source: over 2 years ago

• Best ELT tool for nonprofits?

  You could look at something like Meltano. Lots of connectors if you don’t mind getting hands dirty. Source: over 2 years ago

• Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

  Is https://meltano.com/ a more general version of this ? I am wondering how this compares to it? - Source: Hacker News / over 2 years ago

• Ask HN: Who is hiring? (November 2021)

  Meltano | All positions are full time and remote | https://meltano.com (follow the code blocks on our website for a demo!) We're a seed stage startup that recently spun out of GitLab. A bit about us: Meltano is an open source DataOps platform that brings together best-in-class open source tools and technologies for the data lifecycle, including the Singer standard for data integration, dbt for transformation,... - Source: Hacker News / over 2 years ago

• Manipulating data from various sources

  I suggest looking into some ETL tools for syncing into your PostgreSQL database, e.g. Tools like Meerschaum, Airbyte, and Meltano. Source: over 2 years ago

• How to deploy a python data integration to AWS?

  Also I'd recommend looking at https://meltano.com/ , built on python and uses the Singer spec to standardize how data flows. Since you're a developer you'd probably really like https://gitlab.com/meltano/sdk which is an sdk that makes writing taps/targets much easier. Source: almost 3 years ago

• Fivetan vs. Stitch vs. Singer vs. Airbyte vs. Meltano

  We'll be sharing in a bunch of places - the Meltano website, twitter, slack, and we're doing a special office hours on zoom/youtube that day too. Source: almost 3 years ago

• Python use cases for devops

  Ansible for DevOps. I'm looking into adding taps/targets for Ops to https://meltano.com/ (open source - MIT, and free) if you're interested in hoping in on the early days please let me know trying to crawl first before making a post to invite a bunch of people to join! Source: almost 3 years ago

• Matillion cost estimator

  I don't have much info about the pricing, but I would definitely suggest checking out some other (what I would consider newer) options. I am not sure what your use case is so they may not be a good fit, but you can use tools like Fivetran or Tray, or open-source specs like Meltano/Airbyte. There are also some SaaS tools like Paragon Connect or hotglue. Source: almost 3 years ago

• How does your startup handle data?

  Https://meltano.com/ and singer.io dumping data to Amazon Redshift + apache Superset for reporting (any BI tool will do). Source: almost 3 years ago

• From 0 to ELT in 90 seconds with Meltano, tap-gitlab, and target-postgres

  I thought it was time for a speedrun showing off Meltano! Source: almost 3 years ago