Apache NiFi Reviews

Social recommendations and mentions

• S3 to S3 transform

  For a simple sequential Pipeline, my goto would be Apache Camel. As soon as you want complexity its either Apache Nifi or a micro service architecture. - Source: Reddit / about 2 months ago

• Analysing Github Stars - Extracting and analyzing data from Github using Apache NiFi®, Apache Kafka® and Apache Druid®

  Spencer Kimball (now CEO at CockroachDB) wrote an interesting article on this topic in 2021 where they created spencerkimball/stargazers based on a Python script. So I started thinking: could I create a data pipeline using Nifi and Kafka (two OSS tools often used with Druid) to get the API data into Druid - and then use SQL to do the analytics? The answer was yes! And I have documented the outcome below. Here’s... - Source: dev.to / 2 months ago

• Read database each 5 sec and dispatch event

  Pretty sure you can do this with a NiFi connector. Https://nifi.apache.org. - Source: Reddit / 5 months ago

• Watch folder and zip new files that appear?

  It's kind of a big tool for the job, but you could do this with Nifi. - Source: Reddit / 8 months ago

• Using Apache NiFi with Redpanda for your Kafka workloads

  Apache NiFi is a tool for moving and processing data between systems. It started as an internal project from the National Security Agency (NSA), where it was known by the codename NiagaraFiles. Through a technology transfer program, the NSA declassified it as an open source project under the Apache Software Foundation (ASF). NiFi models streaming data as a series of flows through a directed acyclic graph (DAG).... - Source: dev.to / 12 months ago

• When should Dataflow be used? Are there examples or guidelines on when we should use it?

  Tangentially, if you like Dataflow you might also want to check out Apache NiFi. Same basic idea, but with a user interface that's pretty easy to use and fairly sophisticated. - Source: Reddit / over 1 year ago

• NIFI Cluster on ECS

  Presuming you mean this: https://nifi.apache.org/ ? Why not go with AWS Glue or similar AWS service directly? - Source: Reddit / over 1 year ago

• Is Apache Airflow HIPAA Compliant & Common Healthcare Stacks?

  As part of your analysis take a look at Apache NiFi. It may not be right for you but focus on the features. It is more mature, spun out of the NSA, and has a pretty decent ecosystem. It is a Java focused stack with some Python friendly capabilities, eg can execute Jython if you want it too. Airflow is a really neat tool and I am not trying to persuade you away from exploring its use just offering a different... - Source: Reddit / over 1 year ago

• Best Practices for Data Pipeline Error Handling in Apache NiFi

  Fortunately, we have tools such as Apache NiFi, which allow us to design and manage our data pipelines, reducing the amount of programming and increasing overall efficiency. Yet, when it comes to creating them, a key and often neglected aspect is minimizing potential errors. - Source: dev.to / almost 2 years ago

• Ask HN: I built it, nobody came, now what?

  How does it differ from Apache NiFi? https://nifi.apache.org/. - Source: Hacker News / almost 2 years ago