4 ms·

Yes, it does exist. In the domain of "Drag & Drop" ETL tools (looking at you, Matillion, Talend, Alteryx etc.), I've seen horrible setups. No version control,

Yes, it does exist. In the domain of "Drag & Drop" ETL tools (looking at you, Matillion, Talend, Alteryx etc.), I've seen horrible setups. No version control, no centralized overview of queries, dependencies, code linting or auto-completion etc. Normally these tools are chosen by unexperienced, rather non-technical data or business people, to whom the initial ease of development is highly appealing. Of course compelxity quickly spirals out of control as LoC across all kinds of different transformation steps increase.

cm2187 5y ago
What I don't get is that most of these tools have their own steep learning curve. You have massive menus with dozens of actions, and your workflow look like complex electrical diagrams. It will put off the least technical people as well as SQL. So in the end you end up with really motivated, kind-of-geeky business users who might as well learn SQL and perhaps a little bit of python for the same effort. I would love to get every office users to do more with their computer than copy-pasting all day, but I came to belive that the problem is mentality, not tooling.
- fragmede 5y ago
  It might be hard to believe, but statisticians and data scientists can be really good at their job while being bad at traditional programming that would lend themselves to learning Python and SQL. I've seen some do transforms on data that are beyond my abilities, while also having difficulty writing functions. If the abstractions in their tools make more sense to them, who are we to question them? Learning SQL is a non-trivial investment, and while it may seem like being an, eg, expert user of Alteryx (and nothing else) may be a career limiting move, there are also tons of programmers who only know Java, don't want to learn anything else, and still have really good careers. I'm sure there's the same for an expert Alteryx user.
  - cm2187 5y ago
    I agree but you are starting with a user who is already an Alteryx expert. I am starting with users who are expert at nothing and are wasting time with manual work. What I say is that those users will either learn nothing, or if they do, they might as well learn SQL rather than learn to use a complicated, no-code software like Alteryx for the same effort than SQL.
    - hermitcrab 5y ago
      SQL seems pretty straightforward to people who done a fair bit of coding. Totally opaque (possibly terrifyingly so) to people with no coding experience. There is a reason why no-code data wrangling tools are a multi-billion dollar industry.

cm2187 5y ago

What I don't get is that most of these tools have their own steep learning curve. You have massive menus with dozens of actions, and your workflow look like complex electrical diagrams. It will put off the least technical people as well as SQL. So in the end you end up with really motivated, kind-of-geeky business users who might as well learn SQL and perhaps a little bit of python for the same effort. I would love to get every office users to do more with their computer than copy-pasting all day, but I came to belive that the problem is mentality, not tooling.

fragmede 5y ago
It might be hard to believe, but statisticians and data scientists can be really good at their job while being bad at traditional programming that would lend themselves to learning Python and SQL. I've seen some do transforms on data that are beyond my abilities, while also having difficulty writing functions. If the abstractions in their tools make more sense to them, who are we to question them? Learning SQL is a non-trivial investment, and while it may seem like being an, eg, expert user of Alteryx (and nothing else) may be a career limiting move, there are also tons of programmers who only know Java, don't want to learn anything else, and still have really good careers. I'm sure there's the same for an expert Alteryx user.
- cm2187 5y ago
  I agree but you are starting with a user who is already an Alteryx expert. I am starting with users who are expert at nothing and are wasting time with manual work. What I say is that those users will either learn nothing, or if they do, they might as well learn SQL rather than learn to use a complicated, no-code software like Alteryx for the same effort than SQL.
  - hermitcrab 5y ago
    SQL seems pretty straightforward to people who done a fair bit of coding. Totally opaque (possibly terrifyingly so) to people with no coding experience. There is a reason why no-code data wrangling tools are a multi-billion dollar industry.