10 ms·

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science b

• DeathArrow 3y ago
  Still, the guys developing ML algorithms like Neural nets, Linear regression, Decision tree, SVM, Naive Bayes, KNN etc., weren't doing science?
  • varispeed 3y ago
    It's like asking whether a carpenter is a scientist because they developed a cabinet...
    • moolcool 3y ago
      I think that you could genuinely make that argument though.
    • DeathArrow 3y ago
      Well, Wikipedia might be wrong, but this is how they describe Computer Science: "Computer science is the study of computation, information, and automation. Computer science spans theoretical disciplines (such as algorithms, theory of computation, and information theory) to applied disciplines (including the design and implementation of hardware and software). Though more often considered an academic discipline, computer science is closely related to computer programming." You might argue, though, that Computer Science is not "real" science.
      • npteljes 3y ago
        I honestly feel like that, as a "Software Engineer". As the digital world is new, and full of similar concepts to the old analogue world, it's natural that we borrow names along with the concepts. Similar to how a lot of things in the sea are named after land things, with a marine prefix, like the sea horse, sea star, sea cucumber, and so on. And now in IT we have engineers, architects, rockstars, tribes, and science, even though, very often, they have no relation to the original profession or concept, and especially doesn't have the responsibility or impact of that.
        • grepfru_it 3y ago
          Computer science is the study, that is looking at how computers and the use of computers can benefit mankind (in my short lay version). Software engineering is a subset of computer science where you build a real world application of the studies. The algorithms behind the Facebook feed and your favorite AI/ML product is the science. The code that powers the feed and chatgpt is the engineering. Wikipedia is not wrong IMHO, but then I studied computer science 20 years ago so maybe I’m out of touch
          • npteljes 3y ago
            I agree with you. There's a lot of engineering, and science going into IT. But I'm a Software Engineer too, and while I'm clever and do clever things, I can assure you that there's no rigor or reason to call it engineering. This version of engineering is at most the re-use of the word, like how the word art is not just covering artistic expressions, but skillfulness too, even though when you apply a skill artfully, you don't create art. Similarly, I do a lot that takes ideas of engineering, but really I'm just writing code and kinda design a smallish system maybe.
            • grepfru_it 3y ago
              Engineering is at all levels, I’ve designed systems that span global infrastructure and systems that run in embedded systems. I still think that design process is a key part of engineering. Figuring out how to put things together is part of design. I agree the more jr engineer you are, the less “engineering” you see.. but you can’t run until you learn to walk. I talk to my friends who sent projects into space and friends who design deep sea drilling rigs. The engineering process is very similar.
    • blueprint 3y ago
      I hate to break it to you but that is part of science. Perhaps the major part of it too. Hypothesis: I can build a cabinet with these materials which will bear some load range. Experiment: I built it and it obviously works. Now change cabinet to particle accelerator that a giant team of other theorists and engineers designed. Am I not doing science by participating in building it? So experimental scientists arent scientists?
      • dylan604 3y ago
        To me, you're describing the differences between a cook and a chef. Just because you've built something with well known methods doesn't make you a scientist. You didn't come up with anything new. In fact, we're starting to sound a lot like Apple. Apple is (in)famous for taking ideas that someone else did all of the hard work of developing and proving to work, and then take various ideas like that to combine into an actual useful working something. (They also do a lot of pure deep research as well, so don't think I'm going too far with the analogy.)
        • varispeed 3y ago
          This image is burned into my retina https://en.meming.world/images/en/a/aa/Something_of_a_Scientist.jpg https://en.meming.world/images/en/a/aa/Something_of_a_Scient...
        • blueprint 3y ago
          semantics, I say. cooks are "chiefs" of certain things otherwise they'd have no value on a team. unless you're saying that chefs setting the menu means theyre more of a scientist. at worst you're saying e.g. scientists are like chefs in that their used their creativity to make up the standard model. at best you're saying chefs are theorists and cooks are experimental scientists which is , again, to my own point.
          • dylan604 3y ago
            i'm saying that chefs are more likely to be aware of what is happening when doing the whipping, baking, cooling, etc and why that's important vs just doing it as a step. so when they experiment with the menus, there's a bit more than basic understanding in why they think the experiments might work. cooks are just the kids in chemistry class following the directions, and depending on how well they follow the directions (and to a point how well the instructions are written), they might not blow up the lab (or burn the m.f. soufflé)
  • agumonkey 3y ago
    Yeah to my newb eyes, the core of machine learning can (and probably is) as scientific as any mathematics goes (granted they try novel ideas, and not applying old solutions). Now openai sits more on the scaling/engineering to make his business.
  • tensor 3y ago
    There is actually an old argument about whether Computer Science is math or science. The lack of hypotheses and physicality drives the "it's math" camp. I tend to fall into that camp. Most computer science is either math or applied math. That said, most of machine learning is not just engineering. If anything, there is little engineering done at all.
  • mdp2021 3y ago
    The note is interesting and in a way a time taking puzzle, but do not forget the apparent point in the original poster's is the lacking approach below the implementations of above said techniques. The techniques you listed in a substantial way expanded our knowledge ("oh, we can do this ... and solve more problems") while raising little more questions. The said implementations on topic give results but raise many more unanswered questions. (This post is unfortunately an "immature" reply - just a provisional tentative point. The matter of "ML algorithms ∩ science" requires more time and concentration and consultation of the texts of the Great Ones.)
    • YeGoblynQueenne 3y ago
      >> but do not forget the apparent point in the original poster's is the lacking approach below the implementations of above said techniques. That's right. I think the approach that is lacking is best explained in the submission guidelines for the Journal of AI Research: >> Papers describing systems should clearly describe the contributions or the principles underlying the system. Papers describing theoretical results should also discuss their practical utility. In general, it should be clear how the work advances the current state of understanding and why the advance matters. Papers should report on what was learned in doing the work, rather than merely on what was done. https://www.jair.org/index.php/jair/about/submissions https://www.jair.org/index.php/jair/about/submissions Doing stuff is not science, even if it's doing stuff that's never been done before. In science, we experiment to understand why stuff can be done, and how, not just that it can be done. And of course it matters what is being done, so for example experimenting to find the best combination of spices for a dish is not an instance of science.
• amelius 3y ago
  It's more a science than pure math or CS, since they are producing falsifiable models.
  • ethanbond 3y ago
    In what sense are they falsifiable?
    • e-neko 3y ago
      if they're wrong, they don't work.
      • swexbe 3y ago
        If my BST implementation isn't sorted, it also doesn't work.
        • amelius 3y ago
          The difference is that you can reason about why it didn't work. With deep learning models not so. They are too big to reason about in the same sense as your BST algorithm. Hence, you need a scientific approach to construct them. I.e., with lots of experimenting, hypotheses, etc.
          • ethanbond 3y ago
            That seems like it pushes it further from science, no? The point of a well-crafted hypothesis is that if it doesn’t bear out, you know that it’s because one+ of your assumptions was wrong. Your ability to continue your scientific inquiry is pretty much == your ability to then identify which assumption was wrong.
            • amelius 3y ago
              Not sure what you are saying here. Perhaps an analogy helps. Psychology is a science. You can make falsifiable statements about the human brain. You will need experiments to build and test theories. It's the same with deep learning. With computer "science" (and math) it's not the same. You can reason completely about your subjects, i.e. you can determine if something will or will not work just by reasoning, no experiments needed. For more information on the differences between math and science I recommend reading: https://en.wikipedia.org/wiki/Scientific_method#Relationship_with_mathematics https://en.wikipedia.org/wiki/Scientific_method#Relationship...
• steppi 3y ago
  I don't think things are quite as dire as that. I have experience as a machine learning scientist, have actually never worked on the kinds of papers you're talking about, but have helped apply machine learning to scientific problems. First in a systems biology lab for tasks upstream of drug discovery, and then in industry for credit risk modeling. I've mostly just used classical machine learning or Bayesian machine learning models though. My interest has always been in using machine learning as a tool to help understand some underlying phenomena, not in trying to push to the top of the leaderboard on benchmark datasets. I think this kind of attitude isn't uncommon in academic labs focused on doing real science, though the methods used aren't necessarily state of the art, and ML typically plays only a supporting role. Curiously, I'm going to present at a conference later today and will give a toy example showing why those leaderboards are not necessarily reliable for distinguishing between the quality of different models.
• bionhoward 3y ago
  Pretty sure the idea of scaling is under test here, just because it’s relatively straightforward to use larger models doesn’t make it less of an application of the scientific method. Also, process supervision, devops is experimental for a startup like OpenAI, and the mixture of experts really hasn’t been used with MHA models at scale before. I’d argue it’s pedantic to assume all science has to be completely novel or revolutionary. More neurons good, is a perfectly reasonable reason to experiment. I think you’re being a bit “snooty” in setting such a high bar to gatekeep science…
• hprotagonist 3y ago
  Are you secretly Ali Rahimi? :-D
• mensetmanusman 3y ago
  {Hypothesis, test, loop} is the scientific method, and I can guarantee it is being used when fine tuning an LLM.
  • mdp2021 3y ago
    ...but in a technical context, not in a scientific one. One thing is identifying (age of copper; age of bronze) the best ways of smelting ore to obtain the metal through trial and error, another is to try and understand the nature of materials.
    • mensetmanusman 3y ago
      Those steps are both necessary and usually take decades to build understanding.
      • mdp2021 3y ago
        But the activity of "attempting to find explanations" is specific. The attempt to formulate laws (the "better faring hypoteses") can bring to "universal laws" or to more humble "patterns". It will contribute to the scientific effort - it does not exhaust it.
      • visarga 3y ago
        Next time we get the urge to complain that GPT-n is just applying patterns seen in the training corpus we should remember humans don't do much better. We are just language agents with rich feedback from outside. We can't even write software top-down in one go without running the code.
        • mdp2021 3y ago
          > patterns Except we consciously (intentionally) work on those matchings. > are Opinions should be offered in proper places. Polling is not debating.
        • mdp2021 3y ago
• mdp2021 3y ago
  > as it is practiced today Reminder that such situation - science being out of focus, below the attempt to obtain practical results "empirically" - is not necessary but contingent. The effort towards "science" is just postponed and/or relegated to other researchers in the same field. And anyway, it is not that the effort towards "explanation" is completely absent. The situation is that many are working on the prospect of achieving big results through bets.
• immibis 3y ago
  Science is knowing how the model scales when you throw this much processing power at it. They won't even tell us how much processing power that is.
• jeron 3y ago
  > racing cool-looking sports cars. I like the analogy attempt but there's a lot of science behind Formula 1
• hyperthesis 3y ago
  I love this diatribe! But to be fair, this kind of engineering historically can at least lead to science.
• jncfhnb 3y ago
  Sure. But remember also that the standard “science” is largely a broken model. Academia sucks. It’s honestly bad. It’s overflowing with papers that are misleading, irrelevant, or fraudulent due to a mix of poor stats knowledge, bad incentives, and a failure to organize effectively. While people praise the scientific method, the majority of achievements we attribute to “science” are not derived from guess and check grad students doing their thing. The standard paper is not very good. And a lot of the reason is that our scientific model is largely designed to find tiny effect sizes. Which is fine for like… medical stuff. But the juice is in large effect sizes. Stuff so obvious you don’t really need the stats to evaluate it. It doesn’t really matter if you followed the scientific method or not when you discover penicillin. It just works, clearly. And you can demonstrate it working again.
  • gmarx 3y ago
    I think the search for tiny effects and the belief that they should drive treatment, is one of the biggest things wrong with medical research. Doctors will put millions of people on a pill that hits a vital metabolic pathway, for life, based on tiny (to my thinking) statistically significant (i.e. only a 5% chance they aren't based on a real effect) results. These days I only believe large effects- like smoking causes heart disease
    • jncfhnb 3y ago
      Medical standards aren’t that bad. It’s just a complicated system with lots of interacting effects that only work for specific circumstances. Coupled with the fact that many medicines are not cures, but something that makes you die slower.
  • gitfan86 3y ago
    Right. In historical context the scientific method was a huge improvement over the previous method of understanding the world, which was mostly religion. But today, academia is a victim of organizational and political capture, making it less competitive for talent.
    • 3y ago
• machina_ex_deus 3y ago
  I'm tired of science as a religion. People treat it as some gospel, like if you check out some criterions you're suddenly "scientific" and instantly get a sense of validity and authority that you shouldn't logically get. I judge things as "what you can do", not "what can you predict". The only demonstration of knowledge and understanding is being able to do something. Not predict. Not "scientific method" and ridiculous "peer review" (actually peer pressure), not blind trials and not rigorous statistical analysis. In the end of the day, you either manage to do something or you don't. So much of so called science had lost all contact with reality because our judgement of success isn't successfully doing something, it is successfully jumping through "scientific" hoops. Look at string theory and social sciences. The scientific process, instead of being a tool, became the purpose. It became a stamp of validity to seek. A stamp of validity with gatekeepers in the academia, in the peer review process, in the media coverage afterwards all the way to social media censorship and "fact checkers". What used to be the frontier of creative people became a stagnant beurocratic machine worshipped like a new religion. The side of the heretics burned at the stake became the ones crying out heresy. Enjoy your new brand of science. I'll stick to the older brand of heretics and mad men which did whatever it was the prevailing orthodoxy told them to avoid doing and thinking, and I'll remind you that the only real reason those are remembered is because they did something useful, not because of the social traditions they adhered to or the rigorous scientific standards they followed.
  • dcow 3y ago
    You would love Thomas Kuhn.
  • JKCalhoun 3y ago
    Science as a rear-view mirror.
  • lorepieri 3y ago
    This is the reason why I left academia for startups... Seems like a better way (not perfect by any means) to innovate, actually doing things. PS: I would be happy to connect, you can find my socials in my bio.
  • deleted 3y ago
• deleted 3y ago
  [deleted]
• dataangel 3y ago
  > Machine learning, as it is practiced today, is not science. There is no scientific theory behind it and there is no scientific method applied. There are no scientific questions asked, or attempted to be answered. Total horseshit. There are tons of scientific papers on ML published. In fact it is MORE like traditional science than typical CS, because it is trying to reverse engineer how something we encountered in the real world works. We know NNs do amazing things, and we don't fully yet understand how.
  • deleted 3y ago
    [deleted]
• hamburga 3y ago
  They are doing computer science.
• javajosh 3y ago
  Obligatory xkcd: https://xkcd.com/1838/ https://xkcd.com/1838/
• dekhn 3y ago
  Of course, 99.9% of science is engineering, not "science". We do science all the time without asking questions or attempting to come up with generalized answers. Is a scientist doing a linear regression not doing science? Anyway, alphafold, while not particularly "scientific", did answer one scientific question: it is possible to predict the structure of most proteins thru a combination of limited structural and extensive sequence information, combined with a sophisticated (and "non-scientific") algorithm. That was an open question for some time and their results convinced the community that their methods were right. What's amazing is that while it's entirely nonscientific, the results have been absolutely blockbuster in the scientific field. And even better, the only reason Alphafold was able to show this is because there was a well-defined protein structure leaderboard.