9 ms·

Generally speaking, IMHO, making the interface easier, in practice, doesn’t actually make using statistical methods correctly easier and that’s scary. SPSS is

Generally speaking, IMHO, making the interface easier, in practice, doesn’t actually make using statistical methods correctly easier and that’s scary. SPSS is a classic example - and the social sciences have had a series of discredited papers over recent years due to poor application of statistical methods. Just because you can put a dataset in and get values out - even values that look significant, it doesn’t mean they are - did all of the assumptions and requirements of the methods/tests you used hold true/pass with your data? So, while I haven’t looked closely at this tool, all I saw was talk of the interface and the ease of getting results even if “you don’t know where to start”. That scares me. Especially when you start talking about applications in domains like medicine. That could be lives in the balance. Would we want civil engineers using tools like this to build bridges? Or people designing nuclear reactors? To me, this is worse, not better unless it somehow helps you actually understand when, why and how to correctly use these tools.

jakobegger 9y ago
Making tools easier to use frees up mental capacity for understanding what your are doing. Just because someone has spent a lot of time learning how to use a difficult tool doesn‘t mean they understand what they are doing any better.
- daveFNbuck 9y ago
  Making tools easier to use also allows you to get to a statistical result without the help of an expert who knows how to properly interpret it. People will launch an A/B test and start checking p-values for 20 different variables within hours when they have easy tooling available.
wanderfowl 9y ago
Agreed. It's odd to charge money for this at all, given that there are such robust statistical tools for free (read: R). But on top of what you said, the fact that trial versions can't export p-values is tacit endorsement of the "p < 0.0x is all that matters" camp, further promoting toxic statistical thinking. Promoting ease of use of tools and simplification of harder problems is great, but this is a really, really dangerous thing to make easy and oversimplified. Nope nope nope nope nope.
- wenc 9y ago
  To be fair, R isn't really that accessible to most social scientists. Heck, R isn't really that accessible to most programmers either. The tidyverse and R notebooks have definitely improved things by leaps and bounds, but R at its core is an unusual language. (I say this as a somewhat proficient R user) Its programmatic nature does aid in coming up with reproducible data analyses though. SPSS has, shall we say, a less than savory reputation. All this to say that there is a market for something much friendlier than R. R is used by pure statisticians, data scientists and the like, but most social scientists prefer Stata, which has pretty legit statistical routines as well as a point-and-click UI.
  - Fomite 9y ago
    I'm fairly fond of JMP, which in my mind manages to thread the gap between "Easy UI" and "Push buttons until you get a p-value" decently.
    - singingfish 9y ago
      Yeah I quite like JMP. JMP for the easy routine stuff. R for when shit gets real. I don't do much stats these days though, so not having a JMP license, I just reach straight for R on the rare occasions I need it.
  - catnaroek 9y ago
    > Heck, R isn't really that accessible to most programmers either. R is objectively a bad programming language. However, it is by no means inaccessible. I have no statistics background whatsoever, and I managed to learn enough R to be dangerous in a mere week. Other than the 1-based indexing and the utterly disgusting dynamic dispatch mechanism (you could simply not to use the latter), R is surprisingly pleasant to use. What I enjoyed the most is that vectors and matrices are first-class values, not objects that are referred to through pointers. It's probably copy-on-write under the hood, but I don't need to care. Hallelujah!
    - singingfish 9y ago
Fomite 9y ago
This. I tooled around with it a little bit using a data set I often use for teaching (from Klein and Moeschberger's survival analysis text) and it was remarkably easy to get an answer without any clue as to what was happening, and that's...not reassuring. As you note, getting an answer and getting the statistically correct answer aren't the same thing.
- rywebb 9y ago
  Yes, sadly having lots of fancy sounding stats creates a smoke-screen of believability.
Angostura 9y ago
I’ll give you a counter example. I’m a barely stats-literate biology graduate. I recognise that’s misuse of stats is dangerous but would like to become more expert. If this program can help me by handling the mechanical bits in a elegant fashion, while leaving me to worry about the appropriate tests and methodologies, I’m all for it.
kome 9y ago
wow, this attitude is really patronizing and completely misled. The difficulty of learning the software doesn't says much about the quality of the ideas, of the research design, of data gathering and interpretation. Like it or not, SPSS made science much easier. And we should work in that direction. Not creating more complex tools for no reason. A software can and should also be a pedagogic tool, guiding people through...
- incongruity 9y ago
  >Like it or not, SPSS made science much easier. No - it didn’t. Science isn’t running stats tools. It’s knowing what tools to use and using them properly so that you get valid answers. If you haven’t been paying attention, the social sciences have been plagued by an epidemic of failures of reproducibility - see: https://www.nature.com/news/over-half-of-psychology-studies-fail-reproducibility-test-1.18248 https://www.nature.com/news/over-half-of-psychology-studies-... for reference. Only 39% of studies were found to be replicable and much of the blame falls to poor methods and techniques like p-hacking. > And we should work in that direction. Yes, we should - but dumbing down interfaces in ways that don’t actually provide good guardrails/handholding to help with proper application is not making good science easier. > Not creating more complex tools for no reason. Nowhere have I or others suggested that. But a tool that makes it simple to get an answer is not the same as one that makes it simple to get a correct and valid answer. Science is a means to an end in many cases. Medicine, for example, is about better human lives. Shoddy research methods could do anything from giving false hope to actually endangering lives.
fnordsensei 9y ago
This is a well known problem in cognitive systems engineering, which sometimes seem to promote design which might seem backwards to an interaction designer focused on creating for everyday consumption. You don't want to design the control room for a nuclear reactor like you design a webpage, because you can under no circumstances sacrifice correctness for ease. A lot of times, it's about making the wrong thing really hard to do, with multiple layers of fail-safes. The potential gain in productivity when designing for ease is vastly overshadowed by the risks associated with making an error. When we're not talking about nuclear reactors, but—for example—statistics, where consequences are abstract, it might get tempting to err on the side of ease. In fact, any domain you don't understand well enough well tempt you to design for ease over correctness. Ideally, the designer should understand the domain at hand well enough to create a design that makes it easy to be correct, and hard to make an error. There's a lot of stuff out there that has been neither designed for ease nor correctness, however, but which has a design arbitrarily dictated by the table layout of a database, or some other random technical constrain that has nothing to do with the problem domain.
- incongruity 9y ago
  Thank you - I think you said much of what I was trying to say in a much better way.
vatys 9y ago
I use Wizard all the time. I'm not a data scientist, but I often need to make engineering decisions based on trends hidden in large datasets. Wizard makes it easy to find what I'm looking for. I'm generally using it for a first pass at data, and when I find the trend, outlier, correlation, etc, that I'm looking for, I'll then move on to a tool with more features like Tableau or JMP to really dig in. What is wrong with having a fast and easy to use tool which makes data analysis accessible? If people misuse their data or their tools, that's their fault.
- incongruity 9y ago
  Excuse a brief bit of hyperbole, but what’s your stance on gun control? How do you feel about seatbelt and helmet laws? All of those things and more exist for the protection of the users and/or general public - but many people hold the attitude that it should be up to the user to do or not so it right. We have a history of banning or socially stigmatizing dangerous products and practices - see 3 wheel ATVs and lawn darts, for easy examples. Similarly, and less extreme, standards and best practices exist exactly because it’s often pragmatically dangerous to expect the end user of a product to be an expert enough to avoid hidden or infrequent but high risk dangers, much less very common and high risk dangers. (See examples everywhere from medicine to aviation or the national elictrical code). Making it easier to shoot yourself in the foot isn’t a virtue on its own...

jakobegger 9y ago

Making tools easier to use frees up mental capacity for understanding what your are doing. Just because someone has spent a lot of time learning how to use a difficult tool doesn‘t mean they understand what they are doing any better.

daveFNbuck 9y ago
Making tools easier to use also allows you to get to a statistical result without the help of an expert who knows how to properly interpret it. People will launch an A/B test and start checking p-values for 20 different variables within hours when they have easy tooling available.

wanderfowl 9y ago

Agreed. It's odd to charge money for this at all, given that there are such robust statistical tools for free (read: R). But on top of what you said, the fact that trial versions can't export p-values is tacit endorsement of the "p < 0.0x is all that matters" camp, further promoting toxic statistical thinking. Promoting ease of use of tools and simplification of harder problems is great, but this is a really, really dangerous thing to make easy and oversimplified. Nope nope nope nope nope.

wenc 9y ago
To be fair, R isn't really that accessible to most social scientists. Heck, R isn't really that accessible to most programmers either. The tidyverse and R notebooks have definitely improved things by leaps and bounds, but R at its core is an unusual language. (I say this as a somewhat proficient R user) Its programmatic nature does aid in coming up with reproducible data analyses though. SPSS has, shall we say, a less than savory reputation. All this to say that there is a market for something much friendlier than R. R is used by pure statisticians, data scientists and the like, but most social scientists prefer Stata, which has pretty legit statistical routines as well as a point-and-click UI.
- Fomite 9y ago
  I'm fairly fond of JMP, which in my mind manages to thread the gap between "Easy UI" and "Push buttons until you get a p-value" decently.
  - singingfish 9y ago
    Yeah I quite like JMP. JMP for the easy routine stuff. R for when shit gets real. I don't do much stats these days though, so not having a JMP license, I just reach straight for R on the rare occasions I need it.
- catnaroek 9y ago
  > Heck, R isn't really that accessible to most programmers either. R is objectively a bad programming language. However, it is by no means inaccessible. I have no statistics background whatsoever, and I managed to learn enough R to be dangerous in a mere week. Other than the 1-based indexing and the utterly disgusting dynamic dispatch mechanism (you could simply not to use the latter), R is surprisingly pleasant to use. What I enjoyed the most is that vectors and matrices are first-class values, not objects that are referred to through pointers. It's probably copy-on-write under the hood, but I don't need to care. Hallelujah!
  - singingfish 9y ago

Fomite 9y ago

This. I tooled around with it a little bit using a data set I often use for teaching (from Klein and Moeschberger's survival analysis text) and it was remarkably easy to get an answer without any clue as to what was happening, and that's...not reassuring. As you note, getting an answer and getting the statistically correct answer aren't the same thing.

rywebb 9y ago
Yes, sadly having lots of fancy sounding stats creates a smoke-screen of believability.

Angostura 9y ago

I’ll give you a counter example. I’m a barely stats-literate biology graduate. I recognise that’s misuse of stats is dangerous but would like to become more expert. If this program can help me by handling the mechanical bits in a elegant fashion, while leaving me to worry about the appropriate tests and methodologies, I’m all for it.

kome 9y ago

wow, this attitude is really patronizing and completely misled. The difficulty of learning the software doesn't says much about the quality of the ideas, of the research design, of data gathering and interpretation. Like it or not, SPSS made science much easier. And we should work in that direction. Not creating more complex tools for no reason. A software can and should also be a pedagogic tool, guiding people through...

incongruity 9y ago
>Like it or not, SPSS made science much easier. No - it didn’t. Science isn’t running stats tools. It’s knowing what tools to use and using them properly so that you get valid answers. If you haven’t been paying attention, the social sciences have been plagued by an epidemic of failures of reproducibility - see: https://www.nature.com/news/over-half-of-psychology-studies-fail-reproducibility-test-1.18248 https://www.nature.com/news/over-half-of-psychology-studies-... for reference. Only 39% of studies were found to be replicable and much of the blame falls to poor methods and techniques like p-hacking. > And we should work in that direction. Yes, we should - but dumbing down interfaces in ways that don’t actually provide good guardrails/handholding to help with proper application is not making good science easier. > Not creating more complex tools for no reason. Nowhere have I or others suggested that. But a tool that makes it simple to get an answer is not the same as one that makes it simple to get a correct and valid answer. Science is a means to an end in many cases. Medicine, for example, is about better human lives. Shoddy research methods could do anything from giving false hope to actually endangering lives.

fnordsensei 9y ago

This is a well known problem in cognitive systems engineering, which sometimes seem to promote design which might seem backwards to an interaction designer focused on creating for everyday consumption. You don't want to design the control room for a nuclear reactor like you design a webpage, because you can under no circumstances sacrifice correctness for ease. A lot of times, it's about making the wrong thing really hard to do, with multiple layers of fail-safes. The potential gain in productivity when designing for ease is vastly overshadowed by the risks associated with making an error. When we're not talking about nuclear reactors, but—for example—statistics, where consequences are abstract, it might get tempting to err on the side of ease. In fact, any domain you don't understand well enough well tempt you to design for ease over correctness. Ideally, the designer should understand the domain at hand well enough to create a design that makes it easy to be correct, and hard to make an error. There's a lot of stuff out there that has been neither designed for ease nor correctness, however, but which has a design arbitrarily dictated by the table layout of a database, or some other random technical constrain that has nothing to do with the problem domain.

incongruity 9y ago
Thank you - I think you said much of what I was trying to say in a much better way.

vatys 9y ago

I use Wizard all the time. I'm not a data scientist, but I often need to make engineering decisions based on trends hidden in large datasets. Wizard makes it easy to find what I'm looking for. I'm generally using it for a first pass at data, and when I find the trend, outlier, correlation, etc, that I'm looking for, I'll then move on to a tool with more features like Tableau or JMP to really dig in. What is wrong with having a fast and easy to use tool which makes data analysis accessible? If people misuse their data or their tools, that's their fault.

incongruity 9y ago
Excuse a brief bit of hyperbole, but what’s your stance on gun control? How do you feel about seatbelt and helmet laws? All of those things and more exist for the protection of the users and/or general public - but many people hold the attitude that it should be up to the user to do or not so it right. We have a history of banning or socially stigmatizing dangerous products and practices - see 3 wheel ATVs and lawn darts, for easy examples. Similarly, and less extreme, standards and best practices exist exactly because it’s often pragmatically dangerous to expect the end user of a product to be an expert enough to avoid hidden or infrequent but high risk dangers, much less very common and high risk dangers. (See examples everywhere from medicine to aviation or the national elictrical code). Making it easier to shoot yourself in the foot isn’t a virtue on its own...