Apache Camel VS Singer

Apache Camel Reviews

Singer Reviews

Apache Camel mentions (12)

• Ask HN: What is the correct way to deal with pipelines?

  "correct" is a value judgement that depends on lots of different things. Only you can decide which tool is correct. Here are some ideas: - https://camel.apache.org/ - https://www.windmill.dev/ Your idea about a queue (in redis, or postgres, or sqlite, etc) is also totally valid. These off-the-shelf tools I listed probably wouldn't give you a huge advantage IMO. - Source: Hacker News / 8 months ago

• Why messaging is much better than REST for inter-microservice communications

  This reminds me more of Apache Camel[0] than other things it's being compared to. > The process initiator puts a message on a queue, and another processor picks that up (probably on a different service, on a different host, and in different code base) - does some processing, and puts its (intermediate) result on another queue This is almost exactly the definition of message routing (ie: Camel). I'm a bit doubtful... - Source: Hacker News / about 1 year ago

• Can I continuously write to a CSV file with a python script while a Java application is continuously reading from it?

  Since you're writing a Java app to consume this, I highly recommend Apache Camel to do the consuming of messages for it. You can trivially aim it at file systems, message queues, databases, web services and all manner of other sources to grab your data for you, and you can change your mind about what that source is, without having to rewrite most of your client code. Source: over 1 year ago

• S3 to S3 transform

  For a simple sequential Pipeline, my goto would be Apache Camel. As soon as you want complexity its either Apache Nifi or a micro service architecture. Source: over 1 year ago

• 🗞️ We have just released our JBang! catalog 🛍️

  🐪 Apache Camel : Camel JBang, A JBang-based Camel app for easily running Camel routes. - Source: dev.to / over 1 year ago

Singer mentions (7)

• Why do companies still build data ingestion tooling instead of using a third-party tool like Airbyte?

  Coincidently, I saw a presentation today on a nice half-way-house solution: using embeddable Python libraries like Sling and dlt - both open-source. See https://www.youtube.com/watch?v=gAqOLgG2iYY There is also singer.io which is more of a protocol than a library, but can also be installed although it looks like it is a true community effort and not so well maintained. Source: 5 months ago

• Data sources episode 2: AWS S3 to Postgres Data Sync using Singer

  Singer is an open-source framework for data ingestion, which provides a standardized way to move data between various data sources and destinations (such as databases, APIs, and data warehouses). Singer offers a modular approach to data extraction and loading by leveraging two main components: Taps (data extractors) and Targets (data loaders). This design makes it an attractive option for data ingestion for... - Source: dev.to / about 1 year ago

• CDC (Change Data Capture) with 3rd party APIs

  Or you could build your own such system and run it on Airflow, Prefect, Dagster, etc. Check out the Singer project for a suite of Python packages designed for such a task. Quality varies greatly, though. Source: over 1 year ago

• Looking to build a database for BI reports

  This is good advice and I think Airbyte created a great product here. I tried singer.io and pipewise but Airbyte is much better in my opinion and I love the UI. Source: over 2 years ago

• Recommendation for approach for populating and refreshing new data lake

  Suspect my question should have been regarding FREE systems, rather than BUYING a system. Sounds like singer.io will do what I need. Source: almost 3 years ago