12 ms·

For the same reasons, testing cannot be automated. The only people who routinely claim that it can are not those who study and enjoy doing software testing. In

For the same reasons, testing cannot be automated. The only people who routinely claim that it can are not those who study and enjoy doing software testing. In fact, anyone who claims that any human activity has been automated faces a testing problem: how do you know? And a construct validity problem: how do you know that the new process is equivalent to the old in every way we care about? Anything can be automated if you are allowed to ignore its true nature and make unchallenged claims.

inter_netuser 5y ago
what if the AI is smarter than humans? even if narrowly, only within the sw dev domain?
- moltar 5y ago
  I think same logic applies. AI is developed by humans to be better than a human. But how can you test it?
  - manojlds 5y ago
    By having known activities that humans have done and asking it to repeat them?
    - bryanrasmussen 5y ago
      Let us say we have 1000 activities humans have done, the AI does them all correctly and of course in seconds instead of days. You then conclude that the AI is better than the humans - but you cannot prove that there is not some slight change in inputs that humans would consider inessential that would cause the AI to fail at a particular batch of tasks.
      - strbean 5y ago
        Is there any point to such speculation? Just as much as that is true, so too could the AI succeed in similar scenarios where humans would fail. The same question could be asked about two humans being contrasted against each other, and just as fruitlessly.
        bryanrasmussen 5y ago
        https://www.bbc.com/future/article/20170410-how-to-fool-artificial-intelligence https://www.bbc.com/future/article/20170410-how-to-fool-arti... AI and humans fail in different ways.
        ithkuil 5y ago
        Our own psychology and moral intuition will cause us to attribute a higher responsibility to the AI failures even of we can prove numerically that humans would have done more harm in average; all it takes is a few cases where a human would/could have saved the day
        ClumsyPilot 5y ago
        We understand human failure and how to guard against it - we dont let drunk people drive, we require multiple authorisations to move a billion dollar or launch nukes, etc. We don't understand AI failure, so it can cause collosal damage at the worst possible time
        catlifeonmars 5y ago
        And yet, drunk people drive and kill themselves and other people every day. We may understand human failure, but we’re still really bad at mitigating it.
        ithkuil 5y ago
        Here is where our moral instincts make the difference. When mitigation fails we fallback on taking solace in a mechanism that plays a role in the mitigation aspect but its psychological effect takes a life of its own: punishment. When a human makes a fatal mistake it bears the responsibility of that mistake and we have protocols to put in place to deal with that (including having somebody to blame and direct our anger against). When a machine does a mistake it becomes much much unclear against whom direct our anger. The engineers who built it? The corporation that profited from it and could have invested more?
      - luckylion 5y ago
        You can also not prove that a human would consider that change the same way another human would. An AI doesn't have to be better than a focused human with lots of experience, great education, at peak health, with a good night of sleep and no distractions. It just has to be better than the average human to be useful, and that's a very, very different scenario.
        bryanrasmussen 5y ago
        >You can also not prove that a human would consider that change the same way another human would. actually by the condition of my predicate I can - I postulated a 'slight change in inputs that humans would consider inessential' if a human considers it essential it is not a change that is allowed into our reasoning here. Now of course I said this because in the past, as noted by the article I linked in another response, AIs have failed because of changes in inputs that people discussing the matter thought inessential. So while the logic of my statement does not allow for humans considering the changes essential, the actual examples I was thinking of when I made my statement might very well have had changes that humans would consider essential. So perhaps we should consider the actual rather than the theoretical. What I cannot prove is whether or not there is any person anywhere that would consider these actual changes essential - I suspect that such a person does not exist but the inductive gap being what it is I might be wrong. Bayesian analysis was originally formulated as a way to bridge the inductive gap, although that is not the way it is generally used, still I wonder at this point what the chance is that some of the data variances that have led to differences in AI conclusions are variances that any human would find inessential.
        satisfice 5y ago
        But you can never know, so no sensible person would ever trust such an AI. AI stands for "automated irresponsibility." Didn't you know that? Maybe AI's are really being created to serve a special class of gullible human who don't understand basic epistemology.
  - tomerv 5y ago
    Eventually - when the field is mature enough - it might be tested like you would test any new hire. At first you keep a close eye on it/them, and eventually you trust them to do the job.
    - ClumsyPilot 5y ago
      You cannot tell how good is your lawyer untill its too late, or your bank, or the AI. It is also prone to sudden and unexpected failures, like tesla that was driving fine on the same road for 400 days and one day decided turns ito a barrier and kills the driver
      - inter_netuser 5y ago
        was that an actual tesla incident? links? interested to learn more about it, tia.
- magpi3 5y ago
  Defining the sw dev domain is a problem. Because I would argue that includes the social/human interaction domain as well.
  - inter_netuser 5y ago
    and why can't AI outperform here as well?
- satisfice 5y ago
  "What if...?" My answer is you could never know, because that is a testing problem, not an AI problem, and there is no test or set of tests that could settle this. We certainly can have AI that seems to claim it is smarter than humans. But we also have idiot humans who claim they are smarter than humans, too. So what? Anyone or anything can make any claim, however untrue. Imagine an AI that is smarter than most humans in terms of some technical ability but is stupider than most humans in terms of moral judgment. In other words, and automated sociopath. Please don't give automated sociopaths any power.
singingfish 5y ago
Testing can be automated, but it's highly unlikely that your automated testing is going to catch every problem. We got involved in an emergency "replace this component" job at work recently. When the inevitable "how are we going to test this" questions started cropping up we did describe how we'd made half a million automated tests and that we had a reasonable understanding of the < 1% failure rate.
- pfdietz 5y ago
  Testing is different from creation, though. In creating software, you make exclusive choices between options for how it will be made. But in testing, you can apply every testing strategy and they will complement each other. As one exhausts itself, mining out its favored part of bug space, the others become relatively better at finding the remaining bugs.
  - singingfish 5y ago
    Oh, the actual project was an emergency hack where we accidentally reverse engineered what the component we were trying to replace, and then went on to outperform the original 100 fold. Very creative enterprise.
- catlifeonmars 5y ago
  The converse is also true: “testing can be purely manual, but it’s highly unlikely that your manual testing is going to catch every problem”
bambax 5y ago
True, except I think we refer to the application of tests when we talk about automated testing, not their writing. It's important that a whole software suite be tested automatically for non regression with every change, even it the tests themselves have been written"by hand". Yet it is surprising how rarely this is put in plave.
- ithkuil 5y ago
  Yeah we seem to be conflating the automated execution of manually written tests with the automated crafting of such tests.
- bambax 5y ago
  *place
threatripper 5y ago
If you have good tests you can automate programming of code that passes these tests.
- agent327 5y ago
  There's a wealth of videos on youtube showing AIs finding unexpected ways to cheat at any given task. There's no reason to believe they wouldn't cheat at programming either.
  - IggleSniggle 5y ago
    Exactly. And once you’ve got your tests made to be specific enough to make it impossible for them to be evaded? You don’t have a testing suite anymore, you have a compiler! Which, honestly, is a great argument for choosing languages with highly specific and derivable type systems: I’d rather deal with compile time errors than with writing runtime tests.
  - gifnamething 5y ago
    That's a sentence that sounds strange - how can an AI cheat? It can't ignore the rules it's programmed with. Well, because there are additional, unspoken, unconsidered rules that it wasn't programmed with.
    - Akronymus 5y ago
      If there is a case that isn't properly tested, the AI is likely to use it. For example, if the test always uses 1,2,3,4 and 5 as inputs, the AI could just generate a list that returns the results for those 5 but not any other.
- satisfice 5y ago
  There are no sure tests to know if someone is clueless about testing, but one of many indicators is when he suggests that with "good tests" you can automate programming.
Scarblac 5y ago
I work in water management in the Netherlands. It used to be that water levels were registered by someone manually noting the level at a gauge, writing it down, and bicycli g to the next. He'd visit all of them in a region in a day. Now they're of course automated sensors. They are cheaper, there are more of them, they are more accurate, immediate available online, and with 5-minute resolution. Clearly better. But the person doing the rounds each day would also see the whole system each day, notice vandalism, know local conditions on growth of pest water plants by heart, et cetera. Now there is nobody with such detailed first hand knowledge of the system. The informal part of the job is lost, and the loss can't easily be quantified as some amount of money either. It didn't even exist on the org chart, but it still feels like it was essential. So we replace human work with different automated processes, and yes we ignore the true nature of the work.
- thaumasiotes 5y ago
  > But the person doing the rounds each day would also see the whole system each day, notice vandalism, know local conditions on growth of pest water plants by heart, et cetera. Not exactly the same, but this is strikingly similar to some of the observations made in this essay: https://desystemize.substack.com/p/desystemize-1 https://desystemize.substack.com/p/desystemize-1 When you reduce field work to a set of data, there's always something that got left out. You just have to hope it wasn't important. Sometimes it was.
- ithkuil 5y ago
  Would it make sense to employ one order of magnitude less people doing the same routine every two weeks?
- agent327 5y ago
  >I work in water management in the Netherlands. Wim Lex, is that you!?
- TomSwirly 5y ago
  Hey, I'm sitting in Watergraafsmeer, where I happen to know our front door is about 60cm below sea level, so I'd like to give you a shout-out for help keeping my feet dry. (And also, I agree with what you wrote.)
Florin_Andrei 5y ago
> how do you know that the new process is equivalent to the old in every way we care about? How could you possibly ever know or prove anything like that?
- satisfice 5y ago
  It's not that you can prove that it is exactly the same, which you can't, it's that we have strong and specific reasons to know that it is NOT materially the same, in most situations. In the area of testing, for instance, I study testers, and I document many things they do which require social competence to achieve. This is tacit knowledge. Tacit knowledge is not automatable. But when you don't know much about testing you don't see any of that tacit knowledge in action. So you think testers are just button pushers whose minds contribute nothing. When OUTSIDERS to a process claim that they've automated it, they are always wrong; and not just wrong but absurdly wrong.
suifbwish 5y ago
This is essentially rephrasing the Turing test. If something can be automated to where you can’t tell the difference between an automated and a human product then it doesn’t matter if there is one. It’s likely that even super advanced AI pretending to be human will be detectable by a human observer well past a singularity point, not that it would matter much.

inter_netuser 5y ago

what if the AI is smarter than humans? even if narrowly, only within the sw dev domain?

moltar 5y ago
I think same logic applies. AI is developed by humans to be better than a human. But how can you test it?
- manojlds 5y ago
  By having known activities that humans have done and asking it to repeat them?
  - bryanrasmussen 5y ago
    Let us say we have 1000 activities humans have done, the AI does them all correctly and of course in seconds instead of days. You then conclude that the AI is better than the humans - but you cannot prove that there is not some slight change in inputs that humans would consider inessential that would cause the AI to fail at a particular batch of tasks.
    - strbean 5y ago
      Is there any point to such speculation? Just as much as that is true, so too could the AI succeed in similar scenarios where humans would fail. The same question could be asked about two humans being contrasted against each other, and just as fruitlessly.
      - bryanrasmussen 5y ago
        https://www.bbc.com/future/article/20170410-how-to-fool-artificial-intelligence https://www.bbc.com/future/article/20170410-how-to-fool-arti... AI and humans fail in different ways.
        ithkuil 5y ago
        Our own psychology and moral intuition will cause us to attribute a higher responsibility to the AI failures even of we can prove numerically that humans would have done more harm in average; all it takes is a few cases where a human would/could have saved the day
        ClumsyPilot 5y ago
        We understand human failure and how to guard against it - we dont let drunk people drive, we require multiple authorisations to move a billion dollar or launch nukes, etc. We don't understand AI failure, so it can cause collosal damage at the worst possible time
        catlifeonmars 5y ago
        And yet, drunk people drive and kill themselves and other people every day. We may understand human failure, but we’re still really bad at mitigating it.
        ithkuil 5y ago
        Here is where our moral instincts make the difference. When mitigation fails we fallback on taking solace in a mechanism that plays a role in the mitigation aspect but its psychological effect takes a life of its own: punishment. When a human makes a fatal mistake it bears the responsibility of that mistake and we have protocols to put in place to deal with that (including having somebody to blame and direct our anger against). When a machine does a mistake it becomes much much unclear against whom direct our anger. The engineers who built it? The corporation that profited from it and could have invested more?
    - luckylion 5y ago
      You can also not prove that a human would consider that change the same way another human would. An AI doesn't have to be better than a focused human with lots of experience, great education, at peak health, with a good night of sleep and no distractions. It just has to be better than the average human to be useful, and that's a very, very different scenario.
      - bryanrasmussen 5y ago
        >You can also not prove that a human would consider that change the same way another human would. actually by the condition of my predicate I can - I postulated a 'slight change in inputs that humans would consider inessential' if a human considers it essential it is not a change that is allowed into our reasoning here. Now of course I said this because in the past, as noted by the article I linked in another response, AIs have failed because of changes in inputs that people discussing the matter thought inessential. So while the logic of my statement does not allow for humans considering the changes essential, the actual examples I was thinking of when I made my statement might very well have had changes that humans would consider essential. So perhaps we should consider the actual rather than the theoretical. What I cannot prove is whether or not there is any person anywhere that would consider these actual changes essential - I suspect that such a person does not exist but the inductive gap being what it is I might be wrong. Bayesian analysis was originally formulated as a way to bridge the inductive gap, although that is not the way it is generally used, still I wonder at this point what the chance is that some of the data variances that have led to differences in AI conclusions are variances that any human would find inessential.
      - satisfice 5y ago
        But you can never know, so no sensible person would ever trust such an AI. AI stands for "automated irresponsibility." Didn't you know that? Maybe AI's are really being created to serve a special class of gullible human who don't understand basic epistemology.
- tomerv 5y ago
  Eventually - when the field is mature enough - it might be tested like you would test any new hire. At first you keep a close eye on it/them, and eventually you trust them to do the job.
  - ClumsyPilot 5y ago
    You cannot tell how good is your lawyer untill its too late, or your bank, or the AI. It is also prone to sudden and unexpected failures, like tesla that was driving fine on the same road for 400 days and one day decided turns ito a barrier and kills the driver
    - inter_netuser 5y ago
      was that an actual tesla incident? links? interested to learn more about it, tia.
magpi3 5y ago
Defining the sw dev domain is a problem. Because I would argue that includes the social/human interaction domain as well.
- inter_netuser 5y ago
  and why can't AI outperform here as well?
satisfice 5y ago
"What if...?" My answer is you could never know, because that is a testing problem, not an AI problem, and there is no test or set of tests that could settle this. We certainly can have AI that seems to claim it is smarter than humans. But we also have idiot humans who claim they are smarter than humans, too. So what? Anyone or anything can make any claim, however untrue. Imagine an AI that is smarter than most humans in terms of some technical ability but is stupider than most humans in terms of moral judgment. In other words, and automated sociopath. Please don't give automated sociopaths any power.

singingfish 5y ago

Testing can be automated, but it's highly unlikely that your automated testing is going to catch every problem. We got involved in an emergency "replace this component" job at work recently. When the inevitable "how are we going to test this" questions started cropping up we did describe how we'd made half a million automated tests and that we had a reasonable understanding of the < 1% failure rate.

pfdietz 5y ago
Testing is different from creation, though. In creating software, you make exclusive choices between options for how it will be made. But in testing, you can apply every testing strategy and they will complement each other. As one exhausts itself, mining out its favored part of bug space, the others become relatively better at finding the remaining bugs.
- singingfish 5y ago
  Oh, the actual project was an emergency hack where we accidentally reverse engineered what the component we were trying to replace, and then went on to outperform the original 100 fold. Very creative enterprise.
catlifeonmars 5y ago
The converse is also true: “testing can be purely manual, but it’s highly unlikely that your manual testing is going to catch every problem”

bambax 5y ago

True, except I think we refer to the application of tests when we talk about automated testing, not their writing. It's important that a whole software suite be tested automatically for non regression with every change, even it the tests themselves have been written"by hand". Yet it is surprising how rarely this is put in plave.

ithkuil 5y ago
Yeah we seem to be conflating the automated execution of manually written tests with the automated crafting of such tests.
bambax 5y ago
*place

threatripper 5y ago

If you have good tests you can automate programming of code that passes these tests.

agent327 5y ago
There's a wealth of videos on youtube showing AIs finding unexpected ways to cheat at any given task. There's no reason to believe they wouldn't cheat at programming either.
- IggleSniggle 5y ago
  Exactly. And once you’ve got your tests made to be specific enough to make it impossible for them to be evaded? You don’t have a testing suite anymore, you have a compiler! Which, honestly, is a great argument for choosing languages with highly specific and derivable type systems: I’d rather deal with compile time errors than with writing runtime tests.
- gifnamething 5y ago
  That's a sentence that sounds strange - how can an AI cheat? It can't ignore the rules it's programmed with. Well, because there are additional, unspoken, unconsidered rules that it wasn't programmed with.
  - Akronymus 5y ago
    If there is a case that isn't properly tested, the AI is likely to use it. For example, if the test always uses 1,2,3,4 and 5 as inputs, the AI could just generate a list that returns the results for those 5 but not any other.
satisfice 5y ago
There are no sure tests to know if someone is clueless about testing, but one of many indicators is when he suggests that with "good tests" you can automate programming.

Scarblac 5y ago

I work in water management in the Netherlands. It used to be that water levels were registered by someone manually noting the level at a gauge, writing it down, and bicycli g to the next. He'd visit all of them in a region in a day. Now they're of course automated sensors. They are cheaper, there are more of them, they are more accurate, immediate available online, and with 5-minute resolution. Clearly better. But the person doing the rounds each day would also see the whole system each day, notice vandalism, know local conditions on growth of pest water plants by heart, et cetera. Now there is nobody with such detailed first hand knowledge of the system. The informal part of the job is lost, and the loss can't easily be quantified as some amount of money either. It didn't even exist on the org chart, but it still feels like it was essential. So we replace human work with different automated processes, and yes we ignore the true nature of the work.

thaumasiotes 5y ago
> But the person doing the rounds each day would also see the whole system each day, notice vandalism, know local conditions on growth of pest water plants by heart, et cetera. Not exactly the same, but this is strikingly similar to some of the observations made in this essay: https://desystemize.substack.com/p/desystemize-1 https://desystemize.substack.com/p/desystemize-1 When you reduce field work to a set of data, there's always something that got left out. You just have to hope it wasn't important. Sometimes it was.
ithkuil 5y ago
Would it make sense to employ one order of magnitude less people doing the same routine every two weeks?
agent327 5y ago
>I work in water management in the Netherlands. Wim Lex, is that you!?
TomSwirly 5y ago
Hey, I'm sitting in Watergraafsmeer, where I happen to know our front door is about 60cm below sea level, so I'd like to give you a shout-out for help keeping my feet dry. (And also, I agree with what you wrote.)

Florin_Andrei 5y ago

> how do you know that the new process is equivalent to the old in every way we care about? How could you possibly ever know or prove anything like that?

satisfice 5y ago
It's not that you can prove that it is exactly the same, which you can't, it's that we have strong and specific reasons to know that it is NOT materially the same, in most situations. In the area of testing, for instance, I study testers, and I document many things they do which require social competence to achieve. This is tacit knowledge. Tacit knowledge is not automatable. But when you don't know much about testing you don't see any of that tacit knowledge in action. So you think testers are just button pushers whose minds contribute nothing. When OUTSIDERS to a process claim that they've automated it, they are always wrong; and not just wrong but absurdly wrong.

suifbwish 5y ago

This is essentially rephrasing the Turing test. If something can be automated to where you can’t tell the difference between an automated and a human product then it doesn’t matter if there is one. It’s likely that even super advanced AI pretending to be human will be detectable by a human observer well past a singularity point, not that it would matter much.