NiFi at first glance sometimes just looks like a glorified GUI for building out a data-delivery application. But NiFi doesn't just compile an application to be

NiFi at first glance sometimes just looks like a glorified GUI for building out a data-delivery application. But NiFi doesn't just compile an application to be deployed on your network. Instead, the "power" of NiFi is that it allows an operations staff to perform the regular day-in-day-out task of monitoring, regulating and if needed modifying the delivery of data to an enterprise. NiFi gives insight to your enterprise data streams in a way that allows "active" dataflow management. If a system is down, NiFi allows dataflow operations to make changes and deal with problems directly, right at tier 1 support. It's often the case that an enterprise software developer has an ongoing role of ensuring the healthy state of the applications from their team. They don't just develop, they are frequently on call and must ensure that data is flowing properly. NiFi helps decouple those roles, so that the operations of dataflow can be actively managed by a dedicated support team that is more tightly integrated with the "mission" of their dataflow. NiFi additionally offers some features that most programmers skip to help with the resiliency of the application. For example: - the concept of "back pressure" is baked into NiFi. This helps ensure that downstreams systems don't get overrun by data, allowing NiFi to send upstream signals to slow or buffer the stream. - data provenance, the ability to see where every piece of data in the system originated and was delivered (the pedigree of the data). Includes the ability to "replay" data as needed. - dynamic routing, allowing a dataflow operator to actively manage a stream, splicing it, or stopping delivery to one source and delivering to another. Sources and Sinks can be temporarily stopped and queued data placed into another route. Representational forms can be changed (csv -> xml -> json, avro), and even schemas can be changed based on stream. Anyone can write a shell script that uses curl to connect with a data source, piping to grep/sed/awk and sending to a database. NiFi is more about visualizing that dataflow, seeing it in real-time, and making adjustments to it as needed. It also helps answer the "what happens when things go wrong" question, the ability to back-off if under contention, or replay in case of failure. (disclaimer: affiliated with NiFi)