4 ms·
No, I run a small nifi cluster (8 nodes) on aws and it is very stable in my opinion. You need a zookeeper cluster, which is also very stable. I made the setup
by pinopinopino 6y ago
No, I run a small nifi cluster (8 nodes) on aws and it is very stable in my opinion. You need a zookeeper cluster, which is also very stable.
I made the setup scalable, which is a nice feature, but you can do without it. Nifi has no problems being scaled down or up.
The biggest problem is when your zookeeper cluster goes down. Nifi happily goes one with processing, but now it can't update the state of its processors, so it keeps doing the same stuff over and over again. Perhaps this is already fixed, dunno. I shut the cluster down now, if I detect zookeeper is down and then start everything up again if zookeeper is ok. But this happens about once in the two year.
It can also talk with LDAP, which is nice in corporate organizations. And it has a shit ton of processors, that do the work for you. If you have lots of etl flows, I can warmly advice to have a look at it.
I update it once every 6 months or so, it is an internal tool. I haven't got a hassle with that yet.