Apache Camel VS Apache Spark

Apache Camel Reviews

Apache Spark Reviews

Apache Camel mentions (12)

• Ask HN: What is the correct way to deal with pipelines?

  "correct" is a value judgement that depends on lots of different things. Only you can decide which tool is correct. Here are some ideas: - https://camel.apache.org/ - https://www.windmill.dev/ Your idea about a queue (in redis, or postgres, or sqlite, etc) is also totally valid. These off-the-shelf tools I listed probably wouldn't give you a huge advantage IMO. - Source: Hacker News / 7 months ago

• Why messaging is much better than REST for inter-microservice communications

  This reminds me more of Apache Camel[0] than other things it's being compared to. > The process initiator puts a message on a queue, and another processor picks that up (probably on a different service, on a different host, and in different code base) - does some processing, and puts its (intermediate) result on another queue This is almost exactly the definition of message routing (ie: Camel). I'm a bit doubtful... - Source: Hacker News / about 1 year ago

• Can I continuously write to a CSV file with a python script while a Java application is continuously reading from it?

  Since you're writing a Java app to consume this, I highly recommend Apache Camel to do the consuming of messages for it. You can trivially aim it at file systems, message queues, databases, web services and all manner of other sources to grab your data for you, and you can change your mind about what that source is, without having to rewrite most of your client code. Source: about 1 year ago

• S3 to S3 transform

  For a simple sequential Pipeline, my goto would be Apache Camel. As soon as you want complexity its either Apache Nifi or a micro service architecture. Source: over 1 year ago

• 🗞️ We have just released our JBang! catalog 🛍️

  🐪 Apache Camel : Camel JBang, A JBang-based Camel app for easily running Camel routes. - Source: dev.to / over 1 year ago

Apache Spark mentions (56)

• Groovy 🎷 Cheat Sheet - 01 Say "Hello" from Groovy

  Recently I had to revisit the "JVM languages universe" again. Yes, language(s), plural! Java isn't the only language that uses the JVM. I previously used Scala, which is a JVM language, to use Apache Spark for Data Engineering workloads, but this is for another post 😉. - Source: dev.to / about 2 months ago

• 🦿🛴Smarcity garbage reporting automation w/ ollama

  Consume data into third party software (then let Open Search or Apache Spark or Apache Pinot) for analysis/datascience, GIS systems (so you can put reports on a map) or any ticket management system. - Source: dev.to / 3 months ago

• Go concurrency simplified. Part 4: Post office as a data pipeline

  Also, this knowledge applies to learning more about data engineering, as this field of software engineering relies heavily on the event-driven approach via tools like Spark, Flink, Kafka, etc. - Source: dev.to / 4 months ago

• Five Apache projects you probably didn't know about

  Apache SeaTunnel is a data integration platform that offers the three pillars of data pipelines: sources, transforms, and sinks. It offers an abstract API over three possible engines: the Zeta engine from SeaTunnel or a wrapper around Apache Spark or Apache Flink. Be careful, as each engine comes with its own set of features. - Source: dev.to / 4 months ago

• Spark – A micro framework for creating web applications in Kotlin and Java

  A JVM based framework named "Spark", when https://spark.apache.org exists? - Source: Hacker News / 11 months ago