7 ms·

Interesting that they started with data. Some of the most interesting discoveries I've made have started with me saying "that's weird" while looking at data. Fr

• ajkjk 5y ago
  I've long felt that the American school system's version of the scientific method (formulate hypothesis, run experiment, collect data, draw conclusions) is totally wrong and turns people off of science. The real procedure that we should teach kids is: collect data, ask questions about it, hypothesize some answers, and only then do a experiment to test your theory. The question formulation, based on observational evidence, is where all of the interesting science really happens.
  • icelancer 5y ago
    Both lead to common biases - the former in fraudulent experimentation, the latter (your suggestion) in fraudulent data snooping and anecdotal case-finding. Both have their merits done correctly, however, but the latter is only possible because of the former - the data has to come from somewhere, and experimentation was how it was historically done... since there wasn't a global database upon instantiation of the planet.
    • diegs 5y ago
      IANAPOSE (I am not a philosophy of science expert) but I think a happy medium is to observe, collect data, and look for things that are interesting, and then perform properly formulate experiments to test those hypotheses. I think that focusing on either side alone can lead to biases and problematic science, and focusing on both is actually the most "fun"
      • sitkack 5y ago
        Both center around observation for all definitions of the word. Before we had microscopes or MRI machines, we still have those but we now have powerful statistical machines called ML models. In all cases though, we need perceive the world. You can't see what do you don't look for.
    • srcreigh 5y ago
      I argue that there is no such thing as formulating a hypothesis with no prior knowledge. We all have prior observations - hypotheses don't appear out of nowhere. In this respect I think GP's comment is admirable - it means teaching kids to use their experiences to formulate hypotheses, rather than forcing them to guess what you want them to formulate. (As an aside, I set out to comment here because I tire of the point that XYZ can be done incorrectly... Everything can be - it's not very interesting - I wish we could discuss less about that, more about new things to do correctly.)
      • makeitdouble 5y ago
        Formulating hypothesis first, even with little knowledge, is fine as long as the point is to invalidate them (basically you assume they are wrong and you’re fine spending time to prove it). It can help focus the data gathering phase, and you get two results: you can ignore your first hypothesis from now on, and you have further data to look at to formulate a decent hypothesis to expand on.
        • srcreigh 5y ago
          I agree in practice, but I think my point is to resist the dichotomy you are implying - that there is a "first hypothesis with little data"... vs ... the other ones..? Isn't all our knowledge little knowledge - even if it seems like a lot? Wouldn't your advice apply equally as well to your most knowledgable topic? The hard part isn't having knowledge, it's finding the hypothesis to show how your knowledge is little knowledge. (Aside: I would love to hear the experiences you are forming your comment around! I'm sure there's a lot of interesting stuff there.)
          • makeitdouble 5y ago
            I think we fundamentally agree, and probably have the same approach, with just a different “marketing”. For the background of these, on a professional level I try to sell the experiments with some angle, whatever it is, even if it is intuitively wrong. An exemple of that was a discussion around wording for a relatively delicate part of our site. We didn’t know how to approach it, and it hasn’t been touch for a very long time. So, the random hypothesis taken was that nobody reads the text anyway. We set hooks and tracking in and out of the sections to have a decent view of what’s happening, and we got enough data to come with new insights and propositions; Next phase was AB testing two changes, with expectations to have positive effect, and the tracking we set in place was to validate it went as planned. This also didn’t go fully as expected, but not that far from the mark, and we iterated. In hindsight we could surely have just done the data collection and make hypothesis from it, but I feel the data would have been more vague and less exploitable. I don‘t trust “let’s just collect data” approaches much.
  • floatingatoll 5y ago
    This model is especially valuable when paired with a teacher who can mentor the students in expanding the horizons of kids during the "ask questions about it" step, because you'll end up with a class full of people who reflexively cross-pollinate problems and solutions across multiple domains.
  • CobrastanJorji 5y ago
    Isn't that exactly how we teach the scientific method? I've been out of school for a good while, but I'm pretty sure I learned "Observe, Hypothesis, Experiment, Conclusion, Tri-Fold Poster Session."
    • ajkjk 5y ago
      I'm sure it's taught differently in different places, but what I want to see is: the science fair projects that everyone does should consist mostly of data-gathering _before_ the hypothesis step. Instead of mostly being about verifying a hypothesis, it should mostly be about _constructing_ an interesting hypothesis (and then verifying it afterwards).
  • AussieWog93 5y ago
    >The real procedure that we should teach kids is: collect data, ask questions about it, hypothesize some answers, and only then do a experiment to test your theory. Apparently lots of interesting discoveries have followed the formula of "get drunk with other PhD students, wait for someone to ask "what if...", do experiment, pretend idea wasn't suggested in a bar at conference"
  • stirfish 5y ago
    I went to school in America. When we learned the scientific method, formulating a hypothesis was the fourth or fifth step. I believe the first step we were taught was to make an observation.
  • Jerrrry 5y ago
    You are 100% correct. The boogeyman of confirmation bias paralyzes researchers.
    • sidr 5y ago
      The reality of not being able to publish negative results, and the publish or perish system of academia paralyzes researchers.
  • version_five 5y ago
    I think this depends on the field. In physics and engineering, there is a body of underlying theory from which one can derive a hypothesis, based on some causal mechanism, before seeing any data. You can then go out and collect some data to confirm - in many cases this is basically just a sanity check, occasionally the data is in conflict with the underlying theory and you learn something big. Outside of the hard sciences, your procedure could make more sense. We do this in business consulting (not always a science but it can be if you do it well) by looking through financial, operational and HR data and asking questions or trying to draw out interesting relationships that can lead to hypothesis for deeper examination. I would expect the same is true in lots of health science as well or any empirical field where we have data on outcomes and the characteristics that may influence then. Even in these fields though, it doesn't preclude starting with a causal mechanism based on underlying theory and then devising data collection and experiments to go out and conform or refute your hypothesis. Personally, I've seen lots of cases where people are quick to rely on statistics and empirical studies, without looking for the mechanism that must exist behind the relationship they're claiming. Such findings are much weaker than results that may have less data or a bigger p-value but are grounded in actual theory and not just reporting a model-free relationship. So to summarize, there is certainly a place for the order you suggest, but if done right, a stronger formulation can be made based on what you call the American school system method.
    • XorNot 5y ago
      Physics and engineering though are absolutely full of interesting effects we observed first though, then hypothesized as to why. University level physics labs are almost wall to wall "observe the original effect that was weird" as a result.
  • kristopolous 5y ago
    General scientific education should be focused on methodological scrutiny and skepticism. Our biggest issue in general scientific literacy is in people being convinced by quacks, charlatans, and well intentioned incompetence. I'll use examples to demonstrate the fix. Let's start with a recent bogus claim: "vaccine causes biomagnetism" The experiment people use is to stick a smooth magnetic object on their oily skin. Let's see if this works using what we teach people "collect data, ask questions about it, hypothesize some answers" Those could lead people to conclude the biomagnetism theory. That's the problem. Instead, scientific literacy needs to focus on scrutiny. Karl Popper style testing, verifying, falsifying, validating. For example: trying to stick magnets to unvaccinated people or similar shaped non-magnets to either group, see if they respond to a compass, or use a piece of paper in between the skin and the magnet, or some talc to smooth the skin (Randi debunked human magnetism with this). But instead, there's hundreds of videos of the smooth object on oily skin test followed by the conclusion that vaccines cause biomagnetism. A friend of mine showed me an hour compilation of countless people repeating the same flawed experiment as if a larger quantity was a valid refutation of my methodological objections. I watched it regardless. The same flawed experiment repeated. A large stream of commenters claimed this is what convinced them. Why? What we teach in school won't help us identify how this is wrong. In fact, they're taught that repeatability is good, and it is, but it doesn't necessarily support a hypothesis without sufficient variation. That's Robert Boyle, 1600s. That's why instead students should poke holes (or support) in increasingly difficult arguments as they progress through school. --- Second example, let's stick with vaccines because why not. The hogwash: vaccines cause X. The graphs are pretty convincing until you ask how things are defined, detected and the frequency of testing. Then everything is easily explained. --- Third example: homoeopaths. They pass all the current methods taught in school. That's the reason medicine invented double blind studies, to show homoeopaths were nonsense. (See the salt tests of 1835 https://www.jameslindlibrary.org/articles/inventing-the-randomized-double-blind-trial-the-nurnberg-salt-test-of-1835/ https://www.jameslindlibrary.org/articles/inventing-the-rand...) --- If students learned valid scrutiny and real scientific skepticism (not the sloppy hand waving nonsense that climate denying propagandists put out for instance) we'd have addressed climate change. We'd have more effective social programs, better economic policy, not be wasteful as a society and not have half the people getting boondoggled by grifters and charlatans. --- Let me belabor the point with one final example. The extremely dangerous advice given by Norman Vincent Peale in his famous 1952 text, "The power of positive thinking". He uses the word "scientific" 40 times and produces exactly zero science beyond unverifiable anecdotes. Some of his uses of "scientific" include "scientific use of prayer" and "spiritually scientific principles (used 3 times)". He makes remarkable and totally unsubstantiated claims throughout the book on things such as clairvoyance, ESP, and just straight up magic such as miraculous healings and physically impossible events. His techniques have actually led to severe psychological illness and many suicides. Translated into 42 languages, sold 15 million copies. People seem to be on board, just like they kill their children with homeopaths and themselves because they refuse to take a vaccine that will supposedly make them magnetic. What they've been taught is science backs them up on every one of these beliefs. Ladies and Gentlemen, we have a problem here.
  • jacobolus 5y ago
    "The best geologist is he who has seen the most rocks." – Herbert Harold Read
  • studentrob 5y ago
    To me the observe step is implied. The scientific method is just meant to say it's okay to be wrong by prioritizing hypothesis over the conclusion. It allows you to be wrong and still fact-based in a world where people naturally don't want to be wrong. It's okay to prefix this with an explicit "observe" step but the absence of that step in teaching doesn't mean we haven't already spent time observing.
  • bryceacc 5y ago
    people don't make a hypothesis in a vacuum though, they should always coming from a starting point of observations. I think the real problem is motivation to critically think about a hypothesis and follow through with investigating it. People are too lazy to find out random truths
  • Ericson2314 5y ago
    I'm reminded of https://blog.khinsen.net/posts/2020/12/10/the-structure-and-interpretation-of-scientific-models/ https://blog.khinsen.net/posts/2020/12/10/the-structure-and-... the opening paragraph is worth quoting in (almost) full: > It is often said that science rests on two pillars, experiment and theory. Which has lead some to propose one or two additional pillars for the computing age: simulation and data analysis. However, the real two pillars of science are observations and models. Observations are the input to science, in the form of numerous but incomplete and imperfect views on reality. Models are the inner state of science. They represent our current understanding of reality, which is necessarily incomplete and imperfect, but understandable and applicable. Simulation and data analysis are tools for interfacing and thus comparing observations and models. They don’t add new pillars, but transforms both of them.
  • tasogare 5y ago
    This is where Western and Japanese linguistics show their biggest difference. The former has a tendency to try making heavy generalizations (finding the universal properties of language/s) and prove it by crafting a few examples fitted for the task (peak armchair linguistics). In the latter, a huge amount of data is collected. I've seen at library a book with hundred maps, relatively precise, about the fine variation in Japanese dialect. It's however sometimes lacking explanation. So both approaches have their strengths and weaknesses and I think that more ground breaking word can be done if relying on works from the two traditions.
    • simiones 5y ago
      I don't think this is a difference between "Western" linguistics and Japanese linguistics. Collecting examples with relatively little explanation is traditional linguistics as it was practiced academically in Europe and Japan and probably most other places as well. You had etymology, and grammar, and many many examples for each. However, modern/computational linguistics has started asking new questions that were simply not considered in traditional linguistics: why and how do the rules of grammar arise? How are they learned by infants? Why do human beings use language at all? How are human languages different from animal communication? For many such questions, looking at hundreds of regional variations of certain words or grammar rules is not particularly helpful, unfortunately, though there is some work that was done based on that as well.
• hyperpallium2 5y ago
  When you figure out a library by reading the source.
• deleted 5y ago
  [deleted]