6 ms·

What part of computer science is experimentation?

deleted 11y ago
[deleted]
mpweiher 11y ago
I am tempted to write: "what part isn't?" Seriously, just about everything we do in computer science is experimentation, at least if it involves...physical computers. Every time we run a program, we run an experiment. At least if we have some expectation of what we want the program to do. Every time we write a test, or run a test suite, we are running an experiment. Every time we implement some sort of abstraction, every time we vary a parameter, every time we optimize a program, we are running experiments. At least if we're doing it right. If you're not doing experiments, I have a hard time categorizing what you are doing as computer science.
- ggreer 11y ago
  You're describing software engineering, not computer science. CS is big-O, Chomsky hierarchy, finite automata, type theory, and computability classes. It's the study of algorithms & data structures, not their implementations. In other words, CS is a branch of math.
  - mpweiher 11y ago
    You're describing theoretical computer science: Theoretical computer science is a division or subset of general computer science and mathematics that focuses on more abstract or mathematical aspects of computing and includes the theory of computation. https://en.wikipedia.org/wiki/Theoretical_computer_science https://en.wikipedia.org/wiki/Theoretical_computer_science
- TeMPOraL 11y ago
  That's not the kind of experiment I meant. In physics, we do experiments because we don't know the rules of the natural world. That's the only way we can learn them. In CS/programming, if we're doing any experiments, they're done because we're too stupid to infer the right answer beforehand. Insofar as you're dealing with the theoretical part of the problem, you shouldn't need any experiments, because the results are purely mathematic and should be perfectly predictable. And if you're doing hardware experiments, it's not computer science either - that's electrical engineering or applied physics. Whatever other tests you may be doing to save some time are still the engineering type. The point is, computer science doesn't have any area in which it could use experiments for discovery. It's fair to say then, that it's not really a science. (It's also fair to point out that the line between science and engineering is getting blurry nowadays.)
  - mpweiher 11y ago
    > In CS/programming, if we're doing any experiments, they're > done because we're too stupid to infer the right answer That's the same as for physics. In theory, there is no difference between theory and practice. In practice, there is. More specifically, for CS: what you're describing is an alternate universe where the halting problem[1] doesn't exist, or rather has been solved. In short, there is no way to predict, in the general case, whether a program will terminate or not, other than running it (= doing the experiment). More generally, if you have a mechanism that predicts with perfect accuracy what the output of my program is, more quickly than running the program, then I will just call that new mechanism "the program", and use that instead of the original program. And then you're back to running the program to find out what it does. UPDATE: To me, this is a (if not THE) fundamental results of theoretical computer science: a precise description of its limits. And since this is an article by Knuth: "Beware of bugs in the above code; I have only proved it correct, not tried it"[2] [1] https://en.wikipedia.org/wiki/Halting_problem https://en.wikipedia.org/wiki/Halting_problem [2] http://www-cs-faculty.stanford.edu/~uno/faq.html http://www-cs-faculty.stanford.edu/~uno/faq.html
- rdtsc 11y ago
  > at least if it involves...physical computers. Yeah you "explore stuff" in any field. Science, art, math, every day life, when learning a craft (welding etc). Just because you explore doesn't mean that is what the core of what computer science is. > Every time we implement some sort of abstraction, every time we vary a parameter, every time we optimize a program, we are running experiments. Ok, what in life then isn't science because well everything is running experiments? I am going to buy milk -- it's an experiment to see what happens. Maybe I won't find parking. I am drinking coffee in the morning, it's an experiment to see if it will still keep me awake. By that token we've diluted the word science to not really mean anything anymore.
  - mpweiher 11y ago
    > Yeah you "explore stuff" in any field. And you do experiments in many fields! "An experiment is a procedure carried out to verify, refute, or establish the validity of a hypothesis. Experiments provide insight into cause-and-effect by demonstrating what outcome occurs when a particular factor is manipulated. Experiments vary greatly in goal and scale, but always rely on repeatable procedure and logical analysis of the results." https://en.wikipedia.org/wiki/Experiment https://en.wikipedia.org/wiki/Experiment Particularly debugging, if you aren't completely random, usually takes on the form of repeated experiments. http://c2.com/cgi/wiki?DebuggingAndTheScientificMethod http://c2.com/cgi/wiki?DebuggingAndTheScientificMethod I have done a lot of performance work in my career, and there you run experiments almost continuously. At least you do if you actually want to have an effect. Admittedly, many people try to optimize without running experiments. That tends to not be very effective. And yes, computer systems are easily sufficiently complex that reasoning it out ahead of time without running the experiment tends to be completely futile except for the most trivial arrangements. In fact, even in some situations that seem completely trivial. > I am going to buy milk -- it's an experiment to see what happens. If you have a hypothesis and you're trying to find out if it is true or not, sure, that can be an experiment. Usually it probably won't be. > I am drinking coffee in the morning, it's an experiment to see if it will still keep me awake. That certainly can be an experiment, for example if you vary the amount of coffee to see how much of will be keep you awake. Though it seems unlikely that you will be isolating the variables sufficiently. With computers, isolating the variables is often a lot simpler. > By that token we've diluted the word science to not really mean anything anymore. Going with your straw-men: probably.
VLM 11y ago
This is stretching it, but if you accept automata theory as a part of CS (not very controversial) and accept cellular automata like the game of life as a branch of automata theory (not very controversial) and accept some research methods involving experimentation and observation of the behavior of random seeds as part of cellular automata theory (not very controversial) then the very extended argument stretches from hard core CS to 'stick a random seed in the CA, run it, and ta da, we observe the un predicted existence of gliders!'. Its stretching it but I think its acceptable. I was going to list some formal percolation theory studies in as an applied version of imaginary network routing protocols or could be seen as a very weird emulation of a CA, kinda, but that's probably going way too far. Practical simulation of a specific algo is obviously cheating, but simulation of an abstract concept is maybe not cheaty. Possibly there is, or will be, an analytic topological pure math theory that could be applied to mathematically prove constants currently only discovered by experiment in perc theory. If you're ever bored and want to run some simulations, perc theory is very project euler like in that a short(ish) question quickly results in "I guess I gotta run it and see". Another "stretch the limits" is there is no unclassified explanation of some peculiar corners of some crypto algos. Hard to know if they're truly random, determined by experimental runs against possibly classified cracking techniques, or intentional back doors, or intentional classified design techniques. But relying on military intel classification "we can't rule out experimental methods being used in secret" is kinda cheaty in the spirit of the discussion.
carapace 11y ago
The interaction of humans with computers.

deleted 11y ago

[deleted]

I am tempted to write: "what part isn't?" Seriously, just about everything we do in computer science is experimentation, at least if it involves...physical computers. Every time we run a program, we run an experiment. At least if we have some expectation of what we want the program to do. Every time we write a test, or run a test suite, we are running an experiment. Every time we implement some sort of abstraction, every time we vary a parameter, every time we optimize a program, we are running experiments. At least if we're doing it right. If you're not doing experiments, I have a hard time categorizing what you are doing as computer science.

ggreer 11y ago
You're describing software engineering, not computer science. CS is big-O, Chomsky hierarchy, finite automata, type theory, and computability classes. It's the study of algorithms & data structures, not their implementations. In other words, CS is a branch of math.
- mpweiher 11y ago
  You're describing theoretical computer science: Theoretical computer science is a division or subset of general computer science and mathematics that focuses on more abstract or mathematical aspects of computing and includes the theory of computation. https://en.wikipedia.org/wiki/Theoretical_computer_science https://en.wikipedia.org/wiki/Theoretical_computer_science
TeMPOraL 11y ago
That's not the kind of experiment I meant. In physics, we do experiments because we don't know the rules of the natural world. That's the only way we can learn them. In CS/programming, if we're doing any experiments, they're done because we're too stupid to infer the right answer beforehand. Insofar as you're dealing with the theoretical part of the problem, you shouldn't need any experiments, because the results are purely mathematic and should be perfectly predictable. And if you're doing hardware experiments, it's not computer science either - that's electrical engineering or applied physics. Whatever other tests you may be doing to save some time are still the engineering type. The point is, computer science doesn't have any area in which it could use experiments for discovery. It's fair to say then, that it's not really a science. (It's also fair to point out that the line between science and engineering is getting blurry nowadays.)
- mpweiher 11y ago
  > In CS/programming, if we're doing any experiments, they're > done because we're too stupid to infer the right answer That's the same as for physics. In theory, there is no difference between theory and practice. In practice, there is. More specifically, for CS: what you're describing is an alternate universe where the halting problem[1] doesn't exist, or rather has been solved. In short, there is no way to predict, in the general case, whether a program will terminate or not, other than running it (= doing the experiment). More generally, if you have a mechanism that predicts with perfect accuracy what the output of my program is, more quickly than running the program, then I will just call that new mechanism "the program", and use that instead of the original program. And then you're back to running the program to find out what it does. UPDATE: To me, this is a (if not THE) fundamental results of theoretical computer science: a precise description of its limits. And since this is an article by Knuth: "Beware of bugs in the above code; I have only proved it correct, not tried it"[2] [1] https://en.wikipedia.org/wiki/Halting_problem https://en.wikipedia.org/wiki/Halting_problem [2] http://www-cs-faculty.stanford.edu/~uno/faq.html http://www-cs-faculty.stanford.edu/~uno/faq.html
rdtsc 11y ago
> at least if it involves...physical computers. Yeah you "explore stuff" in any field. Science, art, math, every day life, when learning a craft (welding etc). Just because you explore doesn't mean that is what the core of what computer science is. > Every time we implement some sort of abstraction, every time we vary a parameter, every time we optimize a program, we are running experiments. Ok, what in life then isn't science because well everything is running experiments? I am going to buy milk -- it's an experiment to see what happens. Maybe I won't find parking. I am drinking coffee in the morning, it's an experiment to see if it will still keep me awake. By that token we've diluted the word science to not really mean anything anymore.
- mpweiher 11y ago
  > Yeah you "explore stuff" in any field. And you do experiments in many fields! "An experiment is a procedure carried out to verify, refute, or establish the validity of a hypothesis. Experiments provide insight into cause-and-effect by demonstrating what outcome occurs when a particular factor is manipulated. Experiments vary greatly in goal and scale, but always rely on repeatable procedure and logical analysis of the results." https://en.wikipedia.org/wiki/Experiment https://en.wikipedia.org/wiki/Experiment Particularly debugging, if you aren't completely random, usually takes on the form of repeated experiments. http://c2.com/cgi/wiki?DebuggingAndTheScientificMethod http://c2.com/cgi/wiki?DebuggingAndTheScientificMethod I have done a lot of performance work in my career, and there you run experiments almost continuously. At least you do if you actually want to have an effect. Admittedly, many people try to optimize without running experiments. That tends to not be very effective. And yes, computer systems are easily sufficiently complex that reasoning it out ahead of time without running the experiment tends to be completely futile except for the most trivial arrangements. In fact, even in some situations that seem completely trivial. > I am going to buy milk -- it's an experiment to see what happens. If you have a hypothesis and you're trying to find out if it is true or not, sure, that can be an experiment. Usually it probably won't be. > I am drinking coffee in the morning, it's an experiment to see if it will still keep me awake. That certainly can be an experiment, for example if you vary the amount of coffee to see how much of will be keep you awake. Though it seems unlikely that you will be isolating the variables sufficiently. With computers, isolating the variables is often a lot simpler. > By that token we've diluted the word science to not really mean anything anymore. Going with your straw-men: probably.

VLM 11y ago

This is stretching it, but if you accept automata theory as a part of CS (not very controversial) and accept cellular automata like the game of life as a branch of automata theory (not very controversial) and accept some research methods involving experimentation and observation of the behavior of random seeds as part of cellular automata theory (not very controversial) then the very extended argument stretches from hard core CS to 'stick a random seed in the CA, run it, and ta da, we observe the un predicted existence of gliders!'. Its stretching it but I think its acceptable. I was going to list some formal percolation theory studies in as an applied version of imaginary network routing protocols or could be seen as a very weird emulation of a CA, kinda, but that's probably going way too far. Practical simulation of a specific algo is obviously cheating, but simulation of an abstract concept is maybe not cheaty. Possibly there is, or will be, an analytic topological pure math theory that could be applied to mathematically prove constants currently only discovered by experiment in perc theory. If you're ever bored and want to run some simulations, perc theory is very project euler like in that a short(ish) question quickly results in "I guess I gotta run it and see". Another "stretch the limits" is there is no unclassified explanation of some peculiar corners of some crypto algos. Hard to know if they're truly random, determined by experimental runs against possibly classified cracking techniques, or intentional back doors, or intentional classified design techniques. But relying on military intel classification "we can't rule out experimental methods being used in secret" is kinda cheaty in the spirit of the discussion.

carapace 11y ago

The interaction of humans with computers.