For a simple sequential Pipeline, my goto would be Apache Camel. As soon as you want complexity its either Apache Nifi or a micro service architecture. - Source: Reddit / about 2 months ago
Spencer Kimball (now CEO at CockroachDB) wrote an interesting article on this topic in 2021 where they created spencerkimball/stargazers based on a Python script. So I started thinking: could I create a data pipeline using Nifi and Kafka (two OSS tools often used with Druid) to get the API data into Druid - and then use SQL to do the analytics? The answer was yes! And I have documented the outcome below. Here’s... - Source: dev.to / 2 months ago
Pretty sure you can do this with a NiFi connector. Https://nifi.apache.org. - Source: Reddit / 5 months ago
It's kind of a big tool for the job, but you could do this with Nifi. - Source: Reddit / 8 months ago
Apache NiFi is a tool for moving and processing data between systems. It started as an internal project from the National Security Agency (NSA), where it was known by the codename NiagaraFiles. Through a technology transfer program, the NSA declassified it as an open source project under the Apache Software Foundation (ASF). NiFi models streaming data as a series of flows through a directed acyclic graph (DAG).... - Source: dev.to / 12 months ago
Tangentially, if you like Dataflow you might also want to check out Apache NiFi. Same basic idea, but with a user interface that's pretty easy to use and fairly sophisticated. - Source: Reddit / over 1 year ago
Presuming you mean this: https://nifi.apache.org/ ? Why not go with AWS Glue or similar AWS service directly? - Source: Reddit / over 1 year ago
As part of your analysis take a look at Apache NiFi. It may not be right for you but focus on the features. It is more mature, spun out of the NSA, and has a pretty decent ecosystem. It is a Java focused stack with some Python friendly capabilities, eg can execute Jython if you want it too. Airflow is a really neat tool and I am not trying to persuade you away from exploring its use just offering a different... - Source: Reddit / over 1 year ago
Fortunately, we have tools such as Apache NiFi, which allow us to design and manage our data pipelines, reducing the amount of programming and increasing overall efficiency. Yet, when it comes to creating them, a key and often neglected aspect is minimizing potential errors. - Source: dev.to / almost 2 years ago
How does it differ from Apache NiFi? https://nifi.apache.org/. - Source: Hacker News / almost 2 years ago
Do you know an article comparing Apache NiFi to other products?
Suggest a link to a post with product alternatives.