5 ms·

I agree, in this case, you’re right on (and I have another comment saying as much) but couldn’t there be a UI that guides you safely through some of those dange

I agree, in this case, you’re right on (and I have another comment saying as much) but couldn’t there be a UI that guides you safely through some of those dangers? I have yet to see anything close to one but I could imagine a system that would at least ask a few questions that would quickly disqualify a dataset or intended analysis for some use cases. Imagine, if you will, turbo-tax for (basic) statistical analyses. As I said elsewhere, a simple interface doesn’t necessarily mean a correct outcome - In short, most stats software is solving the wrong problem. Many (especially this one) make it easy to get an answer, right or wrong. I’d rather see them make it hard to get a wrong answer - or, perhaps, hard to get an answer when you’re using it wrong.

ACow_Adonis 9y ago
Hmmm. I understand that by answering in the negative (no, its not possible), i would put myself into the "64k is enough for anybody" type comments. Which is to say, since its not a logical impossibility and not a well defined concept and our technology/understanding is increasing, odds are we'll at least make headways towards it to the point that its already pretty good. One could argue that we're already there compared to statistical environments of the 80's and 90's. Honestly, I think this is a tricky human problem, not a tech problem. My reasoning is thus. I've been known to argue that even R is bad from this perspective: I view its success (apart from its free/OS nature) due to the fact that it carries with it libraries, a functional flavour, tied around a core engine/philosophy of implicit actions preference to result production rather than bothering the user. These enable a person (with just enough knowledge to be dangerous) to load an externally authored package (that has neither been tested nor verified) with one line, load a dataset with one line (which silently corrupted or changed something during import), and apply a function in one line (which silently coerced objects/values in the background and unreliably expressed/suppressed errors and warnings). Where it does express warnings/errors, it does so unreliably/unhelpfully, so amateurs are led down the path of excessive warnings/errors where things continue on regardless: ignore them. To many, what I've written up there is not a bad thing: you get statistical analysis in three simple lines for free, all while copying and pasting scripts from stack overflow or the internet. Now, i'm actually working on my own hobby project of designing a language/library for my own use which is designed around fixing those principals: still function based, interactive, fast, no implicit coercion, allowing flexibility while imposing restrains and guarantees. But I'm under no illusion that it would necessarily be popular if i ever realised it to the public. The attraction of R is that you get a model out in three lines, rather than 14 errors and no result telling you that there are issues involving realms of thought you didn't know you were ignorant in and you'll have to go away and study before you continue, or even that your data might not be suitable for what you're doing. It might not be a good piece of work, but cynically, for the type of person buying into such a mind-set of quick analysis via "darts thrown at an analytical wall", i'm not convinced that quality genuinely matters to them (even though it matters in the effects it has on the public further down the line). And i've not even gotten into the problem that in the real world 90% of work is not analysis/modelling but in data cleaning/munging, critical thinking and technical problem solving, and that there's another layer of problems below that one which most academics and professionals rarely engage in, which is questioning the systematic/contextual nature of the data before it got to your data set (which no software/interface i'm aware of currently addresses and most people just blithely ignore).

ACow_Adonis 9y ago

Hmmm. I understand that by answering in the negative (no, its not possible), i would put myself into the "64k is enough for anybody" type comments. Which is to say, since its not a logical impossibility and not a well defined concept and our technology/understanding is increasing, odds are we'll at least make headways towards it to the point that its already pretty good. One could argue that we're already there compared to statistical environments of the 80's and 90's. Honestly, I think this is a tricky human problem, not a tech problem. My reasoning is thus. I've been known to argue that even R is bad from this perspective: I view its success (apart from its free/OS nature) due to the fact that it carries with it libraries, a functional flavour, tied around a core engine/philosophy of implicit actions preference to result production rather than bothering the user. These enable a person (with just enough knowledge to be dangerous) to load an externally authored package (that has neither been tested nor verified) with one line, load a dataset with one line (which silently corrupted or changed something during import), and apply a function in one line (which silently coerced objects/values in the background and unreliably expressed/suppressed errors and warnings). Where it does express warnings/errors, it does so unreliably/unhelpfully, so amateurs are led down the path of excessive warnings/errors where things continue on regardless: ignore them. To many, what I've written up there is not a bad thing: you get statistical analysis in three simple lines for free, all while copying and pasting scripts from stack overflow or the internet. Now, i'm actually working on my own hobby project of designing a language/library for my own use which is designed around fixing those principals: still function based, interactive, fast, no implicit coercion, allowing flexibility while imposing restrains and guarantees. But I'm under no illusion that it would necessarily be popular if i ever realised it to the public. The attraction of R is that you get a model out in three lines, rather than 14 errors and no result telling you that there are issues involving realms of thought you didn't know you were ignorant in and you'll have to go away and study before you continue, or even that your data might not be suitable for what you're doing. It might not be a good piece of work, but cynically, for the type of person buying into such a mind-set of quick analysis via "darts thrown at an analytical wall", i'm not convinced that quality genuinely matters to them (even though it matters in the effects it has on the public further down the line). And i've not even gotten into the problem that in the real world 90% of work is not analysis/modelling but in data cleaning/munging, critical thinking and technical problem solving, and that there's another layer of problems below that one which most academics and professionals rarely engage in, which is questioning the systematic/contextual nature of the data before it got to your data set (which no software/interface i'm aware of currently addresses and most people just blithely ignore).