4 ms·

> I've seen customers fall into the trap of thinking they don't need expensive developers because you can drag and drop, just people who can use a mouse can cra

by zokier 7y ago

> I've seen customers fall into the trap of thinking they don't need expensive developers because you can drag and drop, just people who can use a mouse can crack on with NiFi. That is sort of the key problem I see with NiFi (and equivalents). The heavy emphasis on graphical UI and visual paradigm sort of implies that its oriented towards non-developers, but problem is that it doesn't make non-developers suddenly expert system architects or developers even if they manage to click through the UI. And many developers probably prefer just defining stuff in code instead of having fancy UIs. So it sort of falls between these to categories. Of course there is huge spectrum of skill in people, and there are probably plenty of "semi-technical" persons to whom this is perfect match, especially if supported by some more techy people.

dmix 7y ago
Doesn't Microsoft Access have a similar interface? It's been a long time since I've used it.
- popinman322 7y ago
  To me, honestly, this seems more similar to Azure Data Factory.
- EvanAnderson 7y ago
  Based on a quick look at NiFi, SQL Server Integration Services (SSIS) seems similar. The configuration management tooling for SSIS is pretty good and it's amenable to version control better than, it sounds like, NiFi is. SSIS still suffers from the potential "gotchas" the top-level poster mentions.
  - drbw 7y ago
    I haven't used SSIS since about SQL2008 - back then it terrible to use with version control - not only was it a huge blob of XML, it had more xml escaped and shoved into attributes of the main document! Whats more it seemed to re-allocate the GUIDs of the elements every time you opened a diagram, so there were always changes... Sounds like it's improved a bit since then.
mistrial9 7y ago
it is genuinely odd to see this prejuidice against graphical tools persist in 2020 .. text file dot-conf and graphical tools are different but.. is it controversial that a graphical tool can emit performant code ? I have no experience with NiFi itself.. just that graphical tools are not inherently "non-technical"
- cameronh90 7y ago
  I don't think it's about whether a graphical tool is performant. Many programming languages aren't performant either. It's more that graphical tools don't generally serve their stated purpose of making it possible for non-programmers to get things done because it appears that the main skill behind programming is actually problem decomposition and modelling rather than syntax. Additionally, graphical tools tend to have a bunch of problems inherent to them, such as being unable to write comments and harder to store in git and work on collaboratively and only having one single editor, which is usually much buggier than a text editor and compiler. So: graphical "languages" don't make most things much easier, and make other things much harder.
  - ibejoeb 7y ago
    > it appears that the main skill behind programming is actually problem decomposition and modelling rather than syntax Totally. This is where declarative and intention-oriented systems shine. Take something like SQL, where, in the majority of cases, the end-user needs to know next to nothing about algorithmic complexity and can still achieve excellent performance and correct results. It'd be neat to see a system that was actually designed toward problem decomposition. What's the state of the art around this kind of stuff?
    - tonyarkles 7y ago
      We have very obviously worked on different types of end-user SQL. One that springs to mind had 30 or 40 tables, evolved over the years as “special cases” popped up, and by time I was asked to look at it, it was maybe too far gone. I tried. Maybe if the schema had been tended like a garden, the queries wouldn’t have been so crazy, but... Mind, though, that this system had made a ton of money for 15 years, so in a way it was a huge win for them! They just eventually painted themselves into a corner and performance started to get worse faster than Moore’s law could save them. Last I heard, the great untangling is still in progress, and I stepped away from that project a few years ago. They likely have still come out on top when you take the sum of revenue they made from the system, but they incurred a huge unanticipated cost and got backed into a pretty bad corner. Sales people were still selling features that didn’t exist, and the team was desperately trying to build new things while also fixing the ever degrading performance.

by zokier 7y ago

dmix 7y ago

Doesn't Microsoft Access have a similar interface? It's been a long time since I've used it.

popinman322 7y ago
To me, honestly, this seems more similar to Azure Data Factory.
EvanAnderson 7y ago
Based on a quick look at NiFi, SQL Server Integration Services (SSIS) seems similar. The configuration management tooling for SSIS is pretty good and it's amenable to version control better than, it sounds like, NiFi is. SSIS still suffers from the potential "gotchas" the top-level poster mentions.
- drbw 7y ago
  I haven't used SSIS since about SQL2008 - back then it terrible to use with version control - not only was it a huge blob of XML, it had more xml escaped and shoved into attributes of the main document! Whats more it seemed to re-allocate the GUIDs of the elements every time you opened a diagram, so there were always changes... Sounds like it's improved a bit since then.

mistrial9 7y ago

it is genuinely odd to see this prejuidice against graphical tools persist in 2020 .. text file dot-conf and graphical tools are different but.. is it controversial that a graphical tool can emit performant code ? I have no experience with NiFi itself.. just that graphical tools are not inherently "non-technical"

cameronh90 7y ago
I don't think it's about whether a graphical tool is performant. Many programming languages aren't performant either. It's more that graphical tools don't generally serve their stated purpose of making it possible for non-programmers to get things done because it appears that the main skill behind programming is actually problem decomposition and modelling rather than syntax. Additionally, graphical tools tend to have a bunch of problems inherent to them, such as being unable to write comments and harder to store in git and work on collaboratively and only having one single editor, which is usually much buggier than a text editor and compiler. So: graphical "languages" don't make most things much easier, and make other things much harder.
- ibejoeb 7y ago
  > it appears that the main skill behind programming is actually problem decomposition and modelling rather than syntax Totally. This is where declarative and intention-oriented systems shine. Take something like SQL, where, in the majority of cases, the end-user needs to know next to nothing about algorithmic complexity and can still achieve excellent performance and correct results. It'd be neat to see a system that was actually designed toward problem decomposition. What's the state of the art around this kind of stuff?
  - tonyarkles 7y ago
    We have very obviously worked on different types of end-user SQL. One that springs to mind had 30 or 40 tables, evolved over the years as “special cases” popped up, and by time I was asked to look at it, it was maybe too far gone. I tried. Maybe if the schema had been tended like a garden, the queries wouldn’t have been so crazy, but... Mind, though, that this system had made a ton of money for 15 years, so in a way it was a huge win for them! They just eventually painted themselves into a corner and performance started to get worse faster than Moore’s law could save them. Last I heard, the great untangling is still in progress, and I stepped away from that project a few years ago. They likely have still come out on top when you take the sum of revenue they made from the system, but they incurred a huge unanticipated cost and got backed into a pretty bad corner. Sales people were still selling features that didn’t exist, and the team was desperately trying to build new things while also fixing the ever degrading performance.