Apache Camel VS Hadoop

Apache Camel Reviews

Hadoop Reviews

Apache Camel mentions (12)

• Ask HN: What is the correct way to deal with pipelines?

  "correct" is a value judgement that depends on lots of different things. Only you can decide which tool is correct. Here are some ideas: - https://camel.apache.org/ - https://www.windmill.dev/ Your idea about a queue (in redis, or postgres, or sqlite, etc) is also totally valid. These off-the-shelf tools I listed probably wouldn't give you a huge advantage IMO. - Source: Hacker News / 8 months ago

• Why messaging is much better than REST for inter-microservice communications

  This reminds me more of Apache Camel[0] than other things it's being compared to. > The process initiator puts a message on a queue, and another processor picks that up (probably on a different service, on a different host, and in different code base) - does some processing, and puts its (intermediate) result on another queue This is almost exactly the definition of message routing (ie: Camel). I'm a bit doubtful... - Source: Hacker News / about 1 year ago

• Can I continuously write to a CSV file with a python script while a Java application is continuously reading from it?

  Since you're writing a Java app to consume this, I highly recommend Apache Camel to do the consuming of messages for it. You can trivially aim it at file systems, message queues, databases, web services and all manner of other sources to grab your data for you, and you can change your mind about what that source is, without having to rewrite most of your client code. Source: over 1 year ago

• S3 to S3 transform

  For a simple sequential Pipeline, my goto would be Apache Camel. As soon as you want complexity its either Apache Nifi or a micro service architecture. Source: over 1 year ago

• 🗞️ We have just released our JBang! catalog 🛍️

  🐪 Apache Camel : Camel JBang, A JBang-based Camel app for easily running Camel routes. - Source: dev.to / over 1 year ago

Hadoop mentions (15)

• Getting thousands of files of output back from a container

  Did you check out tools like https://hadoop.apache.org/ ? Source: about 1 year ago

• 5 Best Practices For Data Integration To Boost ROI And Efficiency

  There are different ways to implement parallel dataflows, such as using parallel data processing frameworks like Apache Hadoop, Apache Spark, and Apache Flink, or using cloud-based services like Amazon EMR and Google Cloud Dataflow. It is also possible to use parallel dataflow frameworks to handle big data and distributed computing, like Apache Nifi and Apache Kafka. Source: about 1 year ago

• Data Engineering and DataOps: A Beginner's Guide to Building Data Solutions and Solving Real-World Challenges

  There are several frameworks available for batch processing, such as Hadoop, Apache Storm, and DataTorrent RTS. - Source: dev.to / over 1 year ago

• Effortlessly Set Up a Hadoop Multi-Node Cluster on Windows Machines with Our Step-by-Step Guide

  A copy of Hadoop installed on each of these machines. You can download Hadoop from the Apache website, or you can use a distribution like Cloudera or Hortonworks. - Source: dev.to / over 1 year ago

• In One Minute : Hadoop

  The Apache™ Hadoop™ project develops open-source software for reliable, scalable, distributed computing. - Source: dev.to / over 1 year ago