4 ms·

So much that you write is blatantly false, or stretch to the point of being as good as false. But you seem well read, so I don't think there is a point in tryi

• deleted 5y ago
  [deleted]
• deleted 5y ago
  [deleted]
• YeGoblynQueenne 5y ago
  >> From my perspective, I don't care for theoretical advances. I care about results and if scale provides results, I'm happy. Like I say you can sort ever larger lists with ever bigger computers and bubblesort. But you can go a lot further if you use the brain and come up with a better sorting algorithm. But I think you would be really surprised to find out how much humans have achieved by using their brains rather than ever bigger computers. Here's a small example that happened to be on my mind recently: https://en.wikipedia.org/wiki/Copernican_Revolution https://en.wikipedia.org/wiki/Copernican_Revolution That is an example of the difference between expensive toys like Dall-E and world-changing scientific work. Though I appreciate that the difference can sometimes appear somewhat mudddled and a machine that can generate a "hamster dragon" may seem as an important scientific breakthrough. Depends on what you're more interested in: hamster dragons, or understanding how the world works. Oh well. As to this: >> It's also interesting coming from someone researching Inductive Logic Programming that provides no results at all. That's an obvious -and odious- attempt at trolling me, but unlike yours, my HN profile is linked to my real-world identity and so I am forced to respond with politenes and to represent my field with dignity. Therefore, I will point out a famous achievement of ILP with which you are sadly unfamiliar and that I expect you to immediately discount on the grounds that you didn't know about it, so it can't have been all that important. I'm linking to an article in the popular press that will be more accessible. It doesn't go into depth over the machine learning approach used but it's ILP; I link to the scholarly article immediately after: Robot Makes Scientific Discovery All by Itself For the first time, a robotic system has made a novel scientific discovery with virtually no human intellectual input. Scientists designed “Adam” to carry out the entire scientific process on its own: formulating hypotheses, designing and running experiments, analyzing data, and deciding which experiments to run next. https://www.wired.com/2009/04/robotscientist/ https://www.wired.com/2009/04/robotscientist/ And the scholarly article: Functional genomic hypothesis generation and experimentation by a robot scientist https://www.nature.com/articles/nature02236 https://www.nature.com/articles/nature02236
  • Isinlor 5y ago
    >> That is an example of the difference between expensive toys like Dall-E and world-changing scientific work. There isn't really that much difference. DeepMind took some 3 years to max out CASP protein-folding benchmark running from 1994 on a problem known since ~1960. Protein folding is exactly the type of world-changing scientific work. But it is just a tip of the iceberg. DL is being successfully applied to PDE solving or electron density predictions and more. The psychedelic hamster dragon is an example of the fact that DL is capable of 0-shot generalization. It is a significant scientific observation in itself. And with regard to your example. We are discussing whether deep learning has staled. I give you mostly examples from last year like EfficientZero (30 Oct) ~500x improvement in sample efficiency over 2013 DQN, some literally not older than a month: GLIDE (20 Dec), Player of Games (6 Dec). You are giving me example from 2004 (2009?). I'm not trying to troll you. I just think you are biased in you assessment of significant results and what it means to be stalled.
    • YeGoblynQueenne 5y ago
      >> Protein folding is exactly the type of world-changing scientific work. Nope. Not at all. That's a misunderstanding of the goal of science which is to explain how the world works. AlphaFold, like all neural network models is a predictive model but has no explanatory power. It can predict the structure of proteins from sequences but it cannot explain why, or how, proteins fold. Scientists can still not explain how proteins fold, certainly not by interacting with AlphaFold. Why is that a problem? You said you care about results. I linked to the wikipedia article about the Copernican Revolution. I recommend you read the wikipedia article on epicycles [1]. To summarise, for hundreds of years, from Hipparchus of Nicaea, through Claudius Ptolemy and all the way up to Copernicus, astronomers used systems of epicycles (circles-upon-circles) to explain the apparent movements of the planets. Epicycle-based models matched observations very well and predicted the movement of the planets very accuratey because it is always possible to fit a smooth curve with arbitrary precision given a sufficient number of epicycles. However, even Copernicus' model that had the sun in the center of the universe, unlike many earlier models, could not explain why the planets moved the way they did. A true explanation only became available when Isaac Newton developed his laws of universal gravitation. Newton's theory (itself later superseded by Einstein's theory of general relativity) provided a true explanation of observable phenomena and a way to calculate better models and obtain more accurate predictions than were ever possible before. That is the power of explaining and understanding the world and simply modelling an incomprehensible set of observations, which is all that neural networks can do, doesn't even come close. >> I'm not trying to troll you. I just think you are biased in you assessment of significant results and what it means to be stalled. ILP is a tiny field with maybe a few dozen people that consistently publish on the subject. By comparison, there are many tens of thousands of papers published on deep learning every year, the deep learning conferences receive many thousands of submissions and research is funded with many millions of dollars by governments, militaries, and the largest of large technology corporatations, the Googles and Facebooks etc, and training state-of-the-art models requires many terrabytes of data, many petaflops of computing power, to the extent that training such models is now only feasible for the aforementioned large corporations. Normally, I'd complain that you are comparing apples to oranges. How can a few people with meager resources be expected to outdo DeepMind and OpenAI? Yet the algorithms that I study run on a student laptop, are trained in a weakly supervised manner, take seconds to train, and generalise robustly from single examples without any sort of pre-training whatsoever (for instance, the approach I study, Meta-Interpetive Learning, comfortably and routinely learns recursive prorgams with arbitrary structure from a single example, without an example of the "base-case"). These are capabilities unheard of in neural networks that must be trained on hundreds of millions of examples [2] of actual programs (rather than examples of inputs and outputs, as in ILP) in order to generate programs and cannot solve programming tasks, nor generate solutions, unlike the examples tasks and solutions in their training sets; while in the case of ILP every programming task solved is, by definition unseen (hence, "weakly supervised" because examples are programming tasks without examples of the solutions). That is what it means for a field to be stalled. When a single graduate student with a seven-year old laptop can solve problems that the dominant approach cannot even attetmpt (in this case, learning from a single example, without pretraining and with weak supervision). ____________ [1] https://en.wikipedia.org/wiki/Deferent_and_epicycle https://en.wikipedia.org/wiki/Deferent_and_epicycle [2] https://arxiv.org/abs/1703.07469v1 https://arxiv.org/abs/1703.07469v1 See Section 4.4. Hyperparameters and Training: "Each minibatch was re-sampled so the model saw 256 million random programs and 1024 million random I/O examples during training.".