16 ms·

You might not need machine learning

437 points by chmaynard 6y ago

coopsmgoops 6y ago
While true I don't think this driving problem is used as an example for something you should use ML for rather an example to show how you could apply it.
- theaeolist 6y ago
  Nevertheless, it is impossible to say why a 'neural network' would be better than some other model function with a large number of parameters. It's true, there are some significant successes achieved with NNs, but there is not a lot of work separating necessities from contingencies. Basically, a lot of it is trial and error.
deleted 6y ago
[deleted]
snemvalts 6y ago
The approach they show in the end - it's still machine learning though? Exploring a space and finding parameters to optimize for a loss function (speed around the track), just not deep learning with neural nets.
- TheUndead96 6y ago
  An important observation. People forget that genetic algorithms and a whole swathe of classical machine learning strategies exist.
- dhruvdh 6y ago
  Isn't that just searching a space? Machine learning generally refers to a fancy way of searching a space. Here the guy searched randomly so I'd say not machine learning. But perhaps using a cost function or a loss function is enough to call it machine learning. A machine just used an algorithm to learn another algorithm after all.
  - mlthoughts2018 6y ago
    Stochastic gradient descent and other things like genetic algorithms and simulated annealing are random search techniques specifically created and taught in the context of machine learning.
    - kgwgk 6y ago
      Simulated annealing goes back to the seventies and was definitely not "specifically created in the context of machine learning". Many (most?) optimization techniques have their origin in Operations Research.
      - mlthoughts2018 6y ago
        Simulated annealing was developed for purposes of parameter fitting in physics modeling, based on Metropolis Hastings which was likewise developed in the context of parameter inference for model fitting. Simulated annealing for eg traveling salesman problem came later. I do agree some optimization algorithms are rooted in other fields. I wasn’t trying to say that machine learning is the only historic field from which optimization methods were developed. I just wanted to point out it is a major historic field where some highly respected search and optimization procedures were first created, since people often overlook how old machine learning is and the vast set of modeling procedures apart from neural networks that make up the core of machine learning.
    - cupofcoffee 6y ago
      Lol. Are all optimization problems specifically created and taught in the context of machine learning? Do you know any other context than ML?
      - mlthoughts2018 6y ago
        Your comment is not coherent. You ask, “ Are all optimization problems specifically created and taught in the context of machine learning?” but this has no logical or semantic connection to my comment in any way. It fails to be a valid response or question. Instead it seems you falsely believe you are writing with some sarcasm that endows rhetorical flair to undercut my comment. It’s very rude and juvenile in addition to being wholly ineffective.
- tgv 6y ago
  It might not be a hard boundary, but I think the perception of ML vs. optimization is how much of a model you have. If all you have is a black box, then it's ML; if you know how the system you are studying works, it's (parameter) optimization.
  - mlthoughts2018 6y ago
    That’s a very unfair distinction, almost like a No True Scotsmam fallacy to say machine learning is only bad and other stuff is only good (in terms of transparency). But machine learning has predated neural networks by hundreds of years. The core mathematical basis of all machine learning coursework linear regression and decision trees. Other models like SVMs, Bayesian models, nearest neighbor indexes, TFIDF text search, naive Bayes classifier, etc., are basically like machine learning 101, and they have many different properties regarding interpretability depending on the problem to solve.
    - username90 6y ago
      Saying that linear regression is machine learning is like saying that newtons laws is chemistry. There was no machine learning before computers, just regular old optimization algorithms.
      - disgruntledphd2 6y ago
        Yes, there was. It was just called statistical modelling.
        tgv 6y ago
        There were no machines in the sense of ML in 1740.
        disgruntledphd2 6y ago
        But ML is essentially just function approximation, and that definitely existed back then. I know this is super pedantic, but it's important to remember the roots of things, and that even things which appear new have precursors that are much older than a lot of people realise.
- mjlee 6y ago
  I think Arthur Samuel would agree. This approach has a loss function, parameters, and inputs that feed in to a model to optimise the parameters. The big difference between this and the other approach mentioned in the article is the model is a simple one that's easy to understand instead of a many layered neural network which is rather opaque. I think the article may be better titled "You might not need neural networks."
  - 0-_-0 6y ago
    > I think the article may be better titled "You might not need neural networks." Since a linear model is essentially a single layer neural network with linear activation, we can't even say that. The athor was using a neural network without realising it :)
    - SomeoneFromCA 6y ago
      I'd argue that a NN cannot have linear activation. I mean, if the activation function is linear, it is not an NN anymore.
  - username90 6y ago
    Machine learning is a set of techniques developed to attack modeling problems that traditional algorithms couldn't solve. So if the algorithm was developed and used before computers it definitely isn't machine learning. Everything done in this article was known and used before computers existed hence not machine learning. Doing automatically controlled systems was still possible before computers just that it required a bit more creative use of analog components.
    - disgruntledphd2 6y ago
      > Machine learning is a set of techniques developed to attack modeling problems that traditional algorithms couldn't solve. So if the algorithm was developed and used before computers it definitely isn't machine learning I think that it's important to note here that this was the set of problems that computer science researchers didn't know how to solve. In the early days, they mostly ended up re-inventing statistical methods. And to be fair, a neural net is just a bunch of linear models joined by a non-linearity. In that case, it's essentially stacked logistic regression, which was invented before computers.
- Der_Einzige 6y ago
  Correct. I was waiting for someone to point this out. He optimized the paramaters of a polynomial equation. The machine learned to effectively race around the track through this optimization. The machine learned
_hoysala_ 6y ago
I think links like this are really helpful. Too often I suffer from the (common on HN) mindset of, "I know nothing about this field, but I'm intelligent enough to know this is stupid". Posts like this are not only interesting, but give me a small dose of humility.
- zachrose 6y ago
  Where have I heard that before? https://news.ycombinator.com/item?id=25196754 https://news.ycombinator.com/item?id=25196754
  - square_usual 6y ago
    Wow, how bizarre. Am I weird in jumping to the conspiracy theorist conclusion that this is some sort of experiment on us, like an A/B test to see what kinds of comment works on what kind of thread?
    - core-questions 6y ago
      No, this is someone getting a bot some karma so it can eventually contribute to the manipulation of posts (i.e. getting things upvoted to the front page).
      - feanaro 6y ago
        Does karma play a role in this on HN? I wasn't aware of that.
        93po 6y ago
        Yes, either accounts need a certain amount of karma to make their votes matter or higher karma gives more weight to the vote. Then when you post links that promote your own content or business you can upvote them with your army of bots
      - sgt 6y ago
raverbashing 6y ago
Yes ML is overkill in this case, now test it with sensor noise, track obstacles, and the simple polynomial version might break down
- thomasfromcdnjs 6y ago
  And doing the track the fastest.
- jonny383 6y ago
  This is a pointless comment that misses the point of the post completely. It says "you _might_ not". The given example in the opening paragraph describes a scenario where you don't need it.
  - struxure 6y ago
    The parent commenter didn't say that everybody should use ML in all circumstances. Real world tasks are often more complex than invented toy examples, and quite often hand crafted policies don't work so well. And it was not elaborated in the original blog post. So I think it was a valid point.
moonbug 6y ago
If only there were some rigorous mathematical frameworks we could use for dealing with uncertainty.
jiofih 6y ago
That’s really not a good example. The polynomial-driven cars behave erratically and slow down for no reason - fixable, but the key is that improving that behaviour will take many man hours of trial-and-error work and math, where the ML version will just improve itself based on the goals.
- midjji 6y ago
  The problems you mention are likely an artifact of him using a bad optimzation method, not inherently a limitation in the function approximator. Ironically, its most similar to genetic search, a method most commonly associated with machine learning. If he used a standard optimzation method instead, convergence would be fast and the result much better. A similar problem, using splines to set force inputs for a robot that travels through a maze with barriers optimized using start and end position and force minimization, was a lab in a course I took last year. The question becomes one of hyper parameter search, i.e. what kind of model/function approximator is sufficient. Here the problem is easy enough that its easy to find a sufficient simple model. The huge networks are nice for more general problems because they tend to work moderately well for everything... in the dataset.
deleted 6y ago
[deleted]
sdoering 6y ago
It is a nice, albeit problematic example. Nonetheless I find many such examples in my daily work as a Data Analyst. Regarding recommender systems I see many companies trying neural nets and so many other fancy ML stuff for things that - in AB-tests are always outperformed by basic rules. I understand the fun it is to build stuff and to use the new hot stuff. And at least for many analysts and marketing people as well as shop product owners this is new hot shit. I also understand, that it is way more easy to get management to hand out the big bucks for something that is the new rage, as they tend to read the respective soundbites in their manager magazines. But as said - I see it underperforming in tests nearly all the time. Not only, but esp. if you take the costs of development and maintenance into account. These systems cost more to build, more to enhance, more to run and bring in less real business value 70 - 90 percent of all the times I have seen them. But they are presented to management in shiny presentations from agencies that need to sell the new hot shit to their clients to show that they are relevant. Because, as said, management thinks they need it and are often not open to agencies telling them, that the business value could be better served otherwise. Because in the end for a manager it is often times more valuable to show a fancy state of the art project to his/her higher ups than creating real business value.
- mlthoughts2018 6y ago
  > “ Regarding recommender systems I see many companies trying neural nets and so many other fancy ML stuff for things that - in AB-tests are always outperformed by basic rules.” I work on large scale recommender systems for an ecommerce company and in my career I’ve seen only the exact opposite. Don’t get me wrong, sometimes simpler ML models, like clustering LSA vectors or nearest neighbors, work better than complex models like neural nets. But I have never seen plain rule systems work better for any problem even remotely at scale. Rule systems give an illusory sense of control and understanding, yet are rife with complex interaction effects and edge cases that typically make them intractable to change.
  - sdoering 6y ago
    From big automotive clients to small-ish fashion eCommerce. From publishing to food-delivery (with upselling in the checkout process) - I found the gains in using rules -> simple ML techniques -> complex systems like NN in most cases not to warrant the costs. The quality of recommendations nearly always increased from a revenue as well as perceived quality standpoint. However, it almost never had a positive impact on the profit margin that would have justified the necessary investments. As said - from a quality standpoint most simple systems were just "good enough". One would need to know the business case and environment. But take automotive (new cars) as an example: The goal is nearly always to get the user to request some form of contact from a physical dealership near them. For that you nearly never need a perfekt, fully configured recommendation. I know of an example (a car manufacturer) where the search space of all configurable variants (including various things the car owner would never register because they are specific screw variants) is of a size where even the number of visitors to the website per year is some orders of magnitude less than the number of options. The way to go here was to reduce the search space and the number of variants. Here it turned out that you can, quite fast, reach that goal with few specific questions (active learning) to lead the user to vehicle variants, which correspond to its interests and led to a disproportionately high contact behavior. And yes: ML techniques were used for the analysis and reduction of the search space. For the concept people to then develop specific questions to get to these reduced attributes. But in the end the recommender now works rule-based. I don't imply that this holds true for every scale of company/problem. And I know some counter examples - but most companies do not operate on that scale. If you are ebay, Zalando (Germany) and the likes: I would probably get different results from testing the revenue validity of the different approaches.
st1x7 6y ago
More accurate title: You probably don't need a neural network if your model has 5 inputs.
deleted 6y ago
[deleted]
karolkozub 6y ago
I love the article, but I don't agree with the premise that machine learning equals neural nets. In my understanding machine learning is a very broad term that just as well could be applied to the polynomial model if the constants were optimized algorithmically. I feel like the presented argument is more for transparent vs opaque models rather than machine learning vs something else. Also one could argue that the polynomial model is just a perceptron[0]. [0]: https://en.wikipedia.org/wiki/Perceptron https://en.wikipedia.org/wiki/Perceptron
- lmilcin 6y ago
  Well... I guess most people equal ML with AI and use these terms interchangeably. If you just replace ML with AI everywhere in this article it is going to make sense. The article has other problems, one being the main premise. The problem isn't to drive a car around track (which is what the polynomials did) but rather write a program that can figure out how to drive a car without you knowing how to solve it.
  - maweki 6y ago
    That's not symbolic AI though. That's only statistical methods. The statistical methods are all the rage now, but explainable AI that can reason is an important area of computer science (and research) and uses formal methods. Edit: yeah, you can downvote this, but current AI research splits right along this line, whether it's symbolic or statistical. Some AI courses will use NNs, others will use Prolog and ASP. You can't just dismiss a whole field of research by reducing AI to statistical methods.
    - laichzeit0 6y ago
      When I see "symbolic AI" I immediately think of Gary Marcus and immediately feel disdain towards the topic because of his behaviour on Twitter and other places.
      - maweki 6y ago
        I don't know the dude. I "only" know that my field of research is deductive reasoning in interactive applications and that this area falls under "Logic Programming" and LP is an area of AI. I know that AI researchers are usually a bit dismissive about the other area. I don't like statistics either. Reducing the whole of AI research to statistical approaches (and NNs are one of those) is disingenious and dismisses hundreds of researchers doing important work. You may not want to have rule-based image recognition, but if your car decides to run over somebody, I feel we better have an explanation for this behaviour based on reasoning and logic.
huffmsa 6y ago
"You might not need neural networks" would be a more appropriate title. The non-nn example given here is one for epoch in epochs loop to take care of picking the best coefficients away from being a ML implementation.
ccortes 6y ago
As a former data scientist I feel we try solve problems the hard way because one of two reasons (maybe both): 1. To feel smart 2. To justify our paycheck Most of the time simple solutions like the one in the link will be more than enough but we just can't resist the urge to implement this new paper we just found. I remember thinking about using a NN for a problem we had, after looking it closely for two days all I needed was a simple linear regression and got the extra benefits of being able to explain what was going on. Eventually I started looking for the "elegant" solution because now that was the thing that made me feel smart. Although some times using brute force is the best approach. There's a balance you find once you gain enough experience... I think.
- GarethGonzales 6y ago
  I agree, but as a data scientist myself I've always known that part of my job is to choose the right tool out of the box. If that tool is a linear fit, or heck even adding a few columns in a spreadsheet, it's my job to realise that and choose appropriately. If it's a complex custom-built NN, then as long as the cost-benefit analysis justifies the build time, I'll choose that. Of course, as you mention, there's always the business-political aspects - of explaining or justifying your choice to people who don't understand any of those tools, and who often want to pretend that they're part of a "smart data-driven AI" company.
- spratzt 6y ago
  Genuinely curious as to what you moved into after working as a data scientist. I'm a data scientist and desperate to get out.
  - Flashtoo 6y ago
    As a data scientist at the start of my career: why?
    - st1x7 6y ago
      The list is so long we might as well start another thread for it.
stfwn 6y ago
> A key feature of neural networks is that the outputs are a nonlinear function of the inputs. However, steering a 2D car is simple enough that a linear function is more than sufficient, and neural networks are unnecessary. This depends entirely on the definition of 'steering a 2D car'. In the model used, throttle is simply proportional to the distance to the nearest wall in front of the car. This means the agent will never accelerate coming out of a corner, because it can't know it has the headroom to steer away from the wall as it's coming out. Similarly, the model for steering inherently steers the car towards the middle of the track. I would expect the car to wobble from left to right if the road's edges are ragged, make up its own corners if the track edges describe a 'fake' turn on a straight bit, and the car would likely crash if it were to encounter a Y junction or a pit stop. The neural network agents showed smarter behavior here because it is able to capture more complex cross-dependence between different inputs. On the topic of junctions, if the track were to include them, perhaps it'd be nice if the car chose the quickest route to optimize for lap times. But maybe that stretches the problem statement too much. > Instead of doing anything fancy, my program generates the coefficients at random to explore the space. If I wanted to generate a good driver for a course, I’d run a few thousand of these and pick the coefficients that complete the course in the shortest time. In theory this is more random and less efficient than an evolutionary algorithm, which searches the problem space in a structured way. If the author really wanted to hammer the point home, a least squares method to one-shot the coefficients would be more convincing. All in all, the author doesn't make any hard claims that are false. But I would nuance the point of "neural networks are unnecessary" to "simpler models will do for simpler objectives".
- blackbrokkoli 6y ago
  I see the article more as a broader metaphor for the AI hype. Take, I dunno, video recommendation. Sure, YouTube itself proably built insane stuff in their engine you could never replicate with classic methods (ignoring whether the YT algo is any good). However, if we are just talking about the Vlog of your real estate company, you should probably A/B-test whether your viewers prefer order by time or clicks and implement a decent title search bar. And kick the consultant hyping you up about ML out, now. So my takeaway is that is not about whether AI is never useful or about 2D steering, but about using the right tool for the right job. And building on that, I have to give the author probs to demonstrating an alternative solution to I problem wich I would have definitely solved via AI.
johbjo 6y ago
It goes without saying that if you know (or guess) the model or function you are trying to implement, then you don't need machine learning. Toy problems like this are still interesting because they demonstrate techniques that are applicable to bigger problems. I'm guessing the network in the car game doesn't need more than one layer and two outputs (acceleration vector), but that is besides the point. The technique being demonstrated is the ability of GA (or maybe particle filters), to find "optimal" weights for a network given whatever simulator. This is always interesting, especially when done with graphics like this.
ZephyrBlu 6y ago
Can someone who knows C help me understand the source code? https://gist.github.com/skeeto/da7b2ac95730aa767c8faf8ec309815c https://gist.github.com/skeeto/da7b2ac95730aa767c8faf8ec3098... The concepts and equations he has are pretty simple but I'm really not seeing how they translate into code. The code also isn't the easiest to try and understand for me. Multiple 1-3 letter or otherwise (Seemingly) poorly named variables, some C jargon I'm not familiar with (Mostly the -> operator) and the fact it's a relatively large chunk of code. It seems like the core logic loop is here: https://gist.github.com/skeeto/da7b2ac95730aa767c8faf8ec309815c#file-aidrivers-c-L428 https://gist.github.com/skeeto/da7b2ac95730aa767c8faf8ec3098... And the position and acceleration of the car is altered each iteration through randomization? It's hard to piece together.
- organian 6y ago
  The arrow operator is a field access through a pointer. That is, if you have a pointer to a struct, it's the same as dereferencing the pointer and accessing the field: "(*foo).bar" corresponds to "foo->bar". I think the main driving logic is on line 228. It could definitely do with some more descriptive variable names, though - "a" for angle, and "s" for sense inputs seems a bit too short.
- devdas 6y ago
  -> is a dereference operator. On line 65 (function ppm_create), f is declared as a pointer to a structure of type ppm (a reference to an object of type ppm). f is the actual structure/object. (f).w is the 'w' member of the structure. The brackets are needed for operator priority. f->w is another way to write (*f).w
wokwokwok 6y ago
I feel like there’s something more interesting going on here than the author is giving credit to. Most tools you can start by solving simple problems with, and gradually work up the complexity of the problem you’re addressing until it does “something useful”. This is a good way to learn what you can, can’t and should use a tool for. Deep learning is problematic though; solving trivial tasks is actually quite difficult, and often the “something useful” level of sophistication means copy pasting someone else’s paper and tweaking it a bit and kind of vaguely hoping something you do makes any difference. Being able to solve trivial problems with neural networks is really important, and useful; not because it’s a good solution, but because it means you can see what happens when you try it out. The problem with this post isn’t the conclusion; the author is quite right. You can solve this problem in many ways, maybe NN aren’t the best for this kind of trivial problem. ...but, if you want to solve harder problems with more than random trial and error tweaking parameters, solving easy problems with the same tool is a good way to learn how. ...and we have like 20 years of proof that using hand crafted models has been proven not to scale effectively.
- roenxi 6y ago
  I can only agree. It is also notable that pretty much the entire repertoire of debugging techniques from programming land are useless. Attempting to implementing a paper from scratch is an interesting experience. (1) implement what they describe (2) it doesn't work (3) ... well, that was fun. Project over. Very different experience from trying to implement quicksort, even though the extensions to a basic neural net often aren't conceptually much more complicated.
- disgruntledphd2 6y ago
  > and we have like 20 years of proof that using hand crafted models has been proven not to scale effectively. What do you mean by this? In what context? Like, the twenty year old models are still running in credit risk and insurance, so I'm confused if you mean in ML/statistical modelling.
  - wokwokwok 6y ago
    You know, driving cars, like the article was talking about? ...or, NLP, audio & image recognition, recommendations... come on. It’s not controversial.
jokoon 6y ago
Can you "download" trained neural nets? Aren't there popular formats for those? For example, if I want to recognize pictures of animals, with reasonable accuracy, how much space would such neural net take? I'm not very taught about ML, but in my mind, since ML is generally heavy on computation, it should be possible to distribute neural nets for specific applications and re-use them.
- chewxy 6y ago
  Yes you can. The "universal" format is ONNX. You can even do crazy things like this: https://blog.owulveryck.info/2018/06/11/recurrent-neural-network-serverless-with-webassembly-and-s3.html https://blog.owulveryck.info/2018/06/11/recurrent-neural-net...
- tuvan 6y ago
  Yes, most of the large models that are not feasible to be trained by individuals are downloadable. Frameworks make it easy to import and export snapshots of model weights. See https://github.com/tensorflow/models/tree/master/research/slim#pre-trained-models https://github.com/tensorflow/models/tree/master/research/sl...
erichahn 6y ago
Neural networks are just regressors. Yes you can learn the weights with genetic algorithms. Is this advised? Not so much : 99.99% of neural networks are trained with some variation of gradient descent on a specified loss function. I don't even know if I agree to the statement. Polynomial regression solves basically the same problem as neural networks but performs way, way, way worse on big datasets. But nonetheless I would say that polynomial regression is machine learning too.
- Der_Einzige 6y ago
  This post isn't about learning NN weights with GAs, Also the reinforcement learning community does a significant amount of it's work with neural networks which are not trained using gradients. Gradient free training is a highly active research area. More like 90% of neural networks are trained with gradient descent
  - 0-_-0 6y ago
    It's a GA without crossover applied to a single layer neural network with a linear activation
  - erichahn 6y ago
    No but this post is trying to generalize on an experiment where NNs are trained with GAs which is - in my opinion - not where NN shine.
villgax 6y ago
You don't expect anyone to write up code to create image classifier for obscure category or the daily?
cupofcoffee 6y ago
I simply do not understand the ML hype. It's absolutely ridiculous. On top of that you have Elon Musk thinking we are a few months away from SkyNet even though we are decades and decades away from AGI. Each and every time I see neural networks on some "techies" blog, I wanna vomit.
- mckirk 6y ago
  Decades and decades? I wouldn't be so sure in such a bold prediction. One breakthrough in meta-learning and we could be well on our way. (Which is not to say that it will happen next year -- just pointing out that a development like this is nigh impossible to predict with the certainty your comment expressed.)
- luc4sdreyer 6y ago
  > On top of that you have Elon Musk thinking we are a few months away from SkyNet even though we are decades and decades away from AGI. The actual problem is that we don't know how long it'll be until we build an AGI. Experts put the range somewhere between 10 years and never. Building an unconstrained AGI is an existential risk, so it's important to try and narrow the confidence bands on these questions. That's one of the reasons why Musk pledged $1 billion to OpenAI.
  - cambalache 6y ago
    The answer is "Not any time soon", you could also worry about giant lasers on sharks. Trying to regulate now a supposedly imminent AGI is like Morse and Edison regulating the Internet. We are doing an spectacularly lousy job of regulating what it already exists.
  - qayxc 6y ago
    > Building an unconstrained AGI is an existential risk But why? So far no-one has been able to explain this to me. An AGI in at of itself is nothing but a brain-in-a-vat. The exponential increase in scientific knowledge was based on the scientific method, which replaced the ancient Greece discussion-based epistemology with a cycle of observation-hypothesis->test->observation->... An isolated AGI cannot test its predictions about the world. Since the unconstrained space of possible explanations is bigger than the space of explanations constrained by observational evidence, there is no way an AGI can gain useful knowledge without access to external observations. This means we are in full control of how "intelligent" (by whatever metric) an AGI can even get by restricting its access to information. But even unlimited access to (passive) information only gets you so far, as some models require data that cannot be obtained passively (i.e. they require deliberate controlled experiments). The final nail in the coffin of dangerous AGI is interaction with the physical world. Yes, even a toddler is a terrifying menace to millions of people if I place a button right next it that sets off a thermonuclear bomb in a city centre. But how about we just don't do that? An AGI with limited or no physical interaction with the world (directly via robot body or indirectly through remote access) can't be any more harmful and menacing than the late Stephen Hawking. There's no need to put AI on a leash, since there's a final naturally limiting factor: energy. Switch off the cooling system and your AGI has to throttle down lest it faces fiery death.
yes_man 6y ago
Generally it is easy to agree with the sentiment that machine learning is being used to solve problems where it isn't really needed. But on the other hand, the same argument could have been used when computers initially came along: "you don't need a supercomputer to calculate this function, I can do it with pen and paper faster". So the real value in projects like this is not that you can now drive 2d car game with a neural network, but that neural networks just took another tiny step forward and demonstrate that their capability to minimize error functions is in principle transferrable to real life problems no matter how stupid, and one day those small steps will have accumulated and neural nets will (and already have) supercede many limits that humans have.
sails 6y ago
I'd guess that a NN could be extended to "racing" multiple cars, where if that was your goal, a polynomial wouldn't be very competitive?
svntid 6y ago
I love that :)
alanfranz 6y ago
s/machine learning/deep learning/ That's what the author means. There are a lot of "traditional" ML and AI techniques that can work very well for a lot of problems.
LoSboccacc 6y ago
well yeah anything you can do with NN you can do chaining enough functions what NN allows is to test millions of permutations of thousands these functions to find the one that works if your problem is intractably hard to figure out functions for, NN provide a larger hammer to brute-force an approximate solution.
yters 6y ago
How is this not machine learning? I guess because there is no gradient descent to pick the parameters, just random space exploration?
- username90 6y ago
  Gradient decent has been known for hundreds of years, it isn't machine learning. Optimizing parameters is an entire field in itself. Calling all cases of parameter optimization "machine learning" is just a fad because it sounds cooler. Before computers people still optimized thing but by hand, doing the same calculations with a computer is no more machine learning than doing them by hand is human learning.
viviansolide 6y ago
What about Deep Learning? :)
mlthoughts2018 6y ago
This is a farcical idea if the author actually is serious. The polynomial method is deep, dangerous overfitting. It’s the same reason why you don’t make a hand-made decision tree: you don’t know which features of the data are important or not. Presuming you do know, like basing the car’s steering on a polynomial fit of the three directional probes, will be a disaster in non-toy problems. This example shows why you _do_ need machine learning. The toy steering problem is a place for you to work out how the car can learn to drive with as little structure or assumption baked in as possible. Putting less a priori structure on it directly means you need models with higher capacity to figure it out. To put this all more succinctly, “never use machine learning when business logic would do” is a statement about how commonly needed machine learning is, not business logic.
yalogin 6y ago
I understand it’s not needed but is it wrong? I see machine learning as a general purpose black box function approximation tool. It could be a polynomial curve or something else entirely, as long as we have enough data we can approximate it using ML. So in one way I see it as a “lazy man’s easy way out” tool.
astrea 6y ago
All he proved was you don't need a neural network for MicroMouse. We've known that for years. Some problems do require it. In my job experience, facial recognition can't be easily done with traditional signals processing. Stereo vision can be, but the algorithms are slow, expensive, and very error prone when compared to deep learning models.
- laquition 6y ago
  The author is pretty much attacking a strawman he cherry picked. He took one toy example of someone learning ML and used it to attack the newest trend. Instead of taking the time to dive into the topic he's trying to discount it as less relevant. It's obvious he's stuck in the C/Unix era of programming, which has long since passed it's heyday.
curiousllama 6y ago
The best time series prediction model is often “Use the previous day (or week’s) value”. The best classification model is often “pretend everything is the same.” The best regression model is often “assume it’s the mean”. This should not be surprising: a core premise of data science (empiricism in general? All logic and human knowledge?) is that intuition is often wrong. Initial hypotheses/problem definitions are just intuition.
- edgyquant 6y ago
  This is incorrect and I’m not sure the point you’re trying to make.
xthestreams 6y ago
> Instead of doing anything fancy, my program generates the coefficients at random to explore the space. If I wanted to generate a good driver for a course, I’d run a few thousand of these and pick the coefficients that complete the course in the shortest time. It is worth pointing out that this strategy can likely overfit on the data that you have used for training: when you change the track, your car may not behave as good as before. In other words: the coefficients are only good for that specific track(s). The author is still using Machine Learning, even if not with neural networks: the need for rigorous strategies for model selection doesn't disappear.
- firethief 6y ago
  > It is worth pointing out that this strategy can likely overfit on the data that you have used for training: when you change the track, your car may not behave as good as before. In other words: the coefficients are only good for that specific track(s). More generally, I think this approach is only suited to "static" courses, which only matter in contrived demos. Any real use of a steering algorithm requires reacting to conditions that can't be predicted a priori; e.g. if you have to avoid collisions with other cars, and one car is controlled by a human player, every run is effectively a different track and overfitting like this would not be an option.
ben7799 6y ago
There's another aspect of this overhype of machine learning right now. Manager - We need to do X, we need a ML Data Scientist. Engineering Team - there's a simple solution we can use instead of ML. Manager - No we need a ML Data Scientist, we need to do this right. Time Passes, hiring a ML Data Scientist is hard, and the problem never gets solved.
bayeslaw 6y ago
I thought hn has an intelligent and well informed crowd. ok I get it that engineers are not scientist but judging from the responses here it's quite staggering to me how clueless some ppl are here.. do yourself a favour and maybe learn a bit about ml before forming an opinion. massaging JavaScript for a living is cool I guess but maybe there's a reason faang is paying 5-10x your salaries to ml researchers. maybe just maybe it's not just hype but billions and billions of dollars worth of innovation potential. it's quite clear to me that a lot of cs ppl are super bitter that they are not the top paid ppl anymore but supply and demand dictates these things and turns out inventing the latest transformer architecture is more valuable (and factors of magnitude less ppl can do it) than writing trivial front end or backend stuff..
- laquition 6y ago
  Basic programming is quickly becoming antiquated and articles like this, and many of the commenters are stuck in the past. The efficiency of ML is in cost, not overall computation. Throwing machine resources at a problem is cheaper than hiring some guru with the necessary math and CS background to solve the problem. We've seen the same thing with frameworks/libraries. Before it took specialized knowledge to do basic things like networking, media creation, etc. in code. Now there are existing tools that do everything for you. The same has happened with optimization/algorithmic knowledge, and the genie is not going back in the bottle. There will definitely be specialized cases where that particular expertise is needed, but that is no longer the norm.
tel 6y ago
It's always a disappointing state of affairs when people skip simple models. If nothing else, a nice baseline like the one described in this article should be constructed before using ML techniques to serve as a comparison point.
offtop5 6y ago
I wasted roughly a month trying to get machine learning working for a game. Unless you seriously need it, and have a massive budget to match, basic boolean logic will probably get you where you want to be faster. I wouldn't advise any solo developer to use machine learning unless that's the entire product. It's very easy to make an insanely difficult game without it
plg 6y ago
I love love love the plain C code, making use of unix pipes to produce (real time) video. The subtext is also that you might not need fancy tools.
amatic 6y ago
I've read a wonderful paper on what information people use when driving, it is pretty similar to this driving simulation. The basic cue is "where you will be in T milliseconds", extrapolated using the current vehicle velocity, and trying to keep that point on the road by steering and throttling. Here is that paper, A unifying theory of driver perception and steering control (2019) https://www.researchgate.net/publication/337024514_A_Unifying_Theory_of_Driver_Perception_and_Steering_Control_on_Straight_and_Winding_Roads https://www.researchgate.net/publication/337024514_A_Unifyin...
ExcavateGrandMa 6y ago
This guy is EPIC at coding! I'd have loved to have such an enthusiast as teacher!
giardini 6y ago
Statistical sampling has been reported to work in many cases.
poxwole 6y ago
Machine Learning is basically the new Snake Oil Cure All. Everyone is slapping it on everything
fractionalhare 6y ago
I agree with the premise. But to dig into the specifics here, because it's not clear in the article: is this model generalizable to arbitrary tracks, or will the author have to generate new coefficients for each track? If you have to generate new coefficients for each track, your polynomial regression reduces to a polynomial interpolation of two-dimensional points which represent the track path on a plane. Which is fine and still accomplishes the specific goal, but doesn't solve what would generally be considered the actual research problem. But then again I don't know if the neural network actually achieves this. It's a little unclear in the video: I don't know if the model is being able to learn from the human guiding the vehicle on n iterations or if the model is generated by the human guiding the vehicle on n iterations. Presumably the research goal is to develop a model which learns tracks (in this circumstance, that would be akin to the model choosing the coefficients rather than being the coefficients).
hour_glass 6y ago
In cases like this and many other cases i feel like the purpose of the machine learning algorithm should be to find a simplified polynomial function. I remember reading a paper in which an “AI physicist” was created and it would reduce complex machine learning models into simple equations that would explain the physics of a simulation. It would reduce the complex simulation into human readable and potentially more useful and predictive “laws” of physics.
ummonk 6y ago
I'd make two broad points against this article: A) In most domains, a nonlinear model is far superior to a linear model. At the very least, generally you need to at least apply a transformation (e.g. logistic) to create a nonlinear model from a linear model, because most problem spaces are nonlinear. A nonlinear model doesn't have to be particularly complex though. B) Machine learning (i.e. automatic tuning of parameters) is far simpler for the user than manual tuning of parameters. It's not a question of whether you "need" machine learning, but whether it will save you work. In fact, the author here is in denial - he actually says he would do machine learning, but doesn't realize that is what it is: "Instead of doing anything fancy, my program generates the coefficients at random to explore the space. If I wanted to generate a good driver for a course, I’d run a few thousand of these and pick the coefficients that complete the course in the shortest time."
didibus 6y ago
> If I wanted to generate a good driver for a course, I’d run a few thousand of these and pick the coefficients that complete the course in the shortest time. I might be pedantic here, but wouldn't this then be a machine learning algorithm? Since the machine is learning the most appropriate coefficient based on some heuristic (best of X random). Wouldn't it be better (and more honest) to say then that, simpler ML models and learning techniques can often yield good enough result that don't justify going to more advanced model and learning techniques?
- bastawhiz 6y ago
  > I might be pedantic here, but wouldn't this then be a machine learning algorithm? I would argue no. Simply trying a bunch of inputs and choosing the most effective ones based on the output is like running a single round of training on a machine learning model. It's hardly machine learning by any stretch—there's no feedback loop, the machine doesn't "learn" anything. If I build a compiler and test a thousand constants for default configuration options and choose the ones that perform the best, or would be a very bold claim to call my compiler "powered by machine learning".
  - didibus 6y ago
    > If I build a compiler and test a thousand constants for default configuration options and choose the ones that perform the best What if that was automated in the build? I had assumed the quote meant that it would be, not that they'd manually run a few random cases and manually pick the best. I thought it be automated, like given a new course, you'd run some training routine where the machine would play the course say 100 times each time choosing random coefficients and at the end, it'll take the ones from the pass that resulted in the quickest playthrough, and those would say go in some config file and become the coefficient used by the AI CPU race cars for that course. To me this is machine learning. Especially if you crank up that 100 to 1 million or 1 billion. At that point, it's still something that only a machine could do, I couldn't realistically try 1 billion random coefficients for the ones that result in the fastest playthrough. So in effect, I see the machine is learning which coefficients perform better for a given course. If it tried 1 billion, it learned that out of 1 billion different possible coefficients, some particular set was the best. > there's no feedback loop, the machine doesn't "learn" anything So that's interesting, because if my prior statement is not to be considered machine learning. My next question is what's the criteria to go from the above to machine learning proper? So it seems that it could be the learning has to be a feedback loop. So each attempt at learning must take something away from the previous. I'd be okay with this definition. And now I'm thinking what's the most minimal modification I can make to meet this new definition. What if as it tried random coefficients, it remembered the ones it tried prior? And what if it made sure that no new attempt at a random set of coefficients had already been attempted before? This isn't super refined, no heuristic to what are the best coefficients to try next given the ones tried prior like say what linear regression would accomplish. But it still meets the definition. It starts random on the first round, and the next round is no longer truly random, since it can't pick the previous's round coefficients again. So at least it's descending along the set of possible coefficients through a path that will eventually try all combinations. Would this be enough to be considered machine learning? I'm also thinking this can start to sound a lot like Evolutionary Computation. Oh boy, in all honesty, I've always been confused about differences between stochastic, metaheuristc, and machine learning optimization techniques.
adontz 6y ago
I like how these cars on the last animation move like a school of fish.
danans 6y ago
ML is useful for domains where we don't have a good analytical model of real world behavior of systems. Of course, that is a lot of problems, including many useful things that biological intelligence appears to solve. But it is true that ML - in particular neural networks - often gets thrown at problems that traditional analysis and modeling could serve more efficiently, and also more predictably. The other benefit of first trying traditional analysis is that it helps the implementer of the system better understand the domain first. Pertinently, I've heard some scientists working on medical imaging describe to me how there has been a trend away from the basic science of understanding the medical phenomena behind the image being analyzed and instead relying excessively on ML based pattern recognition to make predictions/inferences.
collyw 6y ago
Repeat for pretty much every technology that is "in fashion" at the time.
anatoly_b 6y ago
Hi, Please try implement formula calculations here: https://youtu.be/eBaHqKVWriU https://youtu.be/eBaHqKVWriU Use neuromorphic networks and be happy!
anatoly_b 6y ago
Use neuromorhic and be happy https://youtu.be/eBaHqKVWriU https://youtu.be/eBaHqKVWriU

437 points by chmaynard 6y ago

While true I don't think this driving problem is used as an example for something you should use ML for rather an example to show how you could apply it.

theaeolist 6y ago
Nevertheless, it is impossible to say why a 'neural network' would be better than some other model function with a large number of parameters. It's true, there are some significant successes achieved with NNs, but there is not a lot of work separating necessities from contingencies. Basically, a lot of it is trial and error.

deleted 6y ago

[deleted]

snemvalts 6y ago

The approach they show in the end - it's still machine learning though? Exploring a space and finding parameters to optimize for a loss function (speed around the track), just not deep learning with neural nets.

TheUndead96 6y ago
An important observation. People forget that genetic algorithms and a whole swathe of classical machine learning strategies exist.
dhruvdh 6y ago
Isn't that just searching a space? Machine learning generally refers to a fancy way of searching a space. Here the guy searched randomly so I'd say not machine learning. But perhaps using a cost function or a loss function is enough to call it machine learning. A machine just used an algorithm to learn another algorithm after all.
- mlthoughts2018 6y ago
  Stochastic gradient descent and other things like genetic algorithms and simulated annealing are random search techniques specifically created and taught in the context of machine learning.
  - kgwgk 6y ago
    Simulated annealing goes back to the seventies and was definitely not "specifically created in the context of machine learning". Many (most?) optimization techniques have their origin in Operations Research.
    - mlthoughts2018 6y ago
      Simulated annealing was developed for purposes of parameter fitting in physics modeling, based on Metropolis Hastings which was likewise developed in the context of parameter inference for model fitting. Simulated annealing for eg traveling salesman problem came later. I do agree some optimization algorithms are rooted in other fields. I wasn’t trying to say that machine learning is the only historic field from which optimization methods were developed. I just wanted to point out it is a major historic field where some highly respected search and optimization procedures were first created, since people often overlook how old machine learning is and the vast set of modeling procedures apart from neural networks that make up the core of machine learning.
  - cupofcoffee 6y ago
    Lol. Are all optimization problems specifically created and taught in the context of machine learning? Do you know any other context than ML?
    - mlthoughts2018 6y ago
      Your comment is not coherent. You ask, “ Are all optimization problems specifically created and taught in the context of machine learning?” but this has no logical or semantic connection to my comment in any way. It fails to be a valid response or question. Instead it seems you falsely believe you are writing with some sarcasm that endows rhetorical flair to undercut my comment. It’s very rude and juvenile in addition to being wholly ineffective.
tgv 6y ago
It might not be a hard boundary, but I think the perception of ML vs. optimization is how much of a model you have. If all you have is a black box, then it's ML; if you know how the system you are studying works, it's (parameter) optimization.
- mlthoughts2018 6y ago
  That’s a very unfair distinction, almost like a No True Scotsmam fallacy to say machine learning is only bad and other stuff is only good (in terms of transparency). But machine learning has predated neural networks by hundreds of years. The core mathematical basis of all machine learning coursework linear regression and decision trees. Other models like SVMs, Bayesian models, nearest neighbor indexes, TFIDF text search, naive Bayes classifier, etc., are basically like machine learning 101, and they have many different properties regarding interpretability depending on the problem to solve.
  - username90 6y ago
    Saying that linear regression is machine learning is like saying that newtons laws is chemistry. There was no machine learning before computers, just regular old optimization algorithms.
    - disgruntledphd2 6y ago
      Yes, there was. It was just called statistical modelling.
      - tgv 6y ago
        There were no machines in the sense of ML in 1740.
        disgruntledphd2 6y ago
        But ML is essentially just function approximation, and that definitely existed back then. I know this is super pedantic, but it's important to remember the roots of things, and that even things which appear new have precursors that are much older than a lot of people realise.
mjlee 6y ago
I think Arthur Samuel would agree. This approach has a loss function, parameters, and inputs that feed in to a model to optimise the parameters. The big difference between this and the other approach mentioned in the article is the model is a simple one that's easy to understand instead of a many layered neural network which is rather opaque. I think the article may be better titled "You might not need neural networks."
- 0-_-0 6y ago
  > I think the article may be better titled "You might not need neural networks." Since a linear model is essentially a single layer neural network with linear activation, we can't even say that. The athor was using a neural network without realising it :)
  - SomeoneFromCA 6y ago
    I'd argue that a NN cannot have linear activation. I mean, if the activation function is linear, it is not an NN anymore.
- username90 6y ago
  Machine learning is a set of techniques developed to attack modeling problems that traditional algorithms couldn't solve. So if the algorithm was developed and used before computers it definitely isn't machine learning. Everything done in this article was known and used before computers existed hence not machine learning. Doing automatically controlled systems was still possible before computers just that it required a bit more creative use of analog components.
  - disgruntledphd2 6y ago
    > Machine learning is a set of techniques developed to attack modeling problems that traditional algorithms couldn't solve. So if the algorithm was developed and used before computers it definitely isn't machine learning I think that it's important to note here that this was the set of problems that computer science researchers didn't know how to solve. In the early days, they mostly ended up re-inventing statistical methods. And to be fair, a neural net is just a bunch of linear models joined by a non-linearity. In that case, it's essentially stacked logistic regression, which was invented before computers.
Der_Einzige 6y ago
Correct. I was waiting for someone to point this out. He optimized the paramaters of a polynomial equation. The machine learned to effectively race around the track through this optimization. The machine learned

_hoysala_ 6y ago

I think links like this are really helpful. Too often I suffer from the (common on HN) mindset of, "I know nothing about this field, but I'm intelligent enough to know this is stupid". Posts like this are not only interesting, but give me a small dose of humility.

zachrose 6y ago
Where have I heard that before? https://news.ycombinator.com/item?id=25196754 https://news.ycombinator.com/item?id=25196754
- square_usual 6y ago
  Wow, how bizarre. Am I weird in jumping to the conspiracy theorist conclusion that this is some sort of experiment on us, like an A/B test to see what kinds of comment works on what kind of thread?
  - core-questions 6y ago
    No, this is someone getting a bot some karma so it can eventually contribute to the manipulation of posts (i.e. getting things upvoted to the front page).
    - feanaro 6y ago
      Does karma play a role in this on HN? I wasn't aware of that.
      - 93po 6y ago
        Yes, either accounts need a certain amount of karma to make their votes matter or higher karma gives more weight to the vote. Then when you post links that promote your own content or business you can upvote them with your army of bots
    - sgt 6y ago

raverbashing 6y ago

Yes ML is overkill in this case, now test it with sensor noise, track obstacles, and the simple polynomial version might break down

thomasfromcdnjs 6y ago
And doing the track the fastest.
jonny383 6y ago
This is a pointless comment that misses the point of the post completely. It says "you _might_ not". The given example in the opening paragraph describes a scenario where you don't need it.
- struxure 6y ago
  The parent commenter didn't say that everybody should use ML in all circumstances. Real world tasks are often more complex than invented toy examples, and quite often hand crafted policies don't work so well. And it was not elaborated in the original blog post. So I think it was a valid point.

moonbug 6y ago

If only there were some rigorous mathematical frameworks we could use for dealing with uncertainty.

jiofih 6y ago

That’s really not a good example. The polynomial-driven cars behave erratically and slow down for no reason - fixable, but the key is that improving that behaviour will take many man hours of trial-and-error work and math, where the ML version will just improve itself based on the goals.

midjji 6y ago
The problems you mention are likely an artifact of him using a bad optimzation method, not inherently a limitation in the function approximator. Ironically, its most similar to genetic search, a method most commonly associated with machine learning. If he used a standard optimzation method instead, convergence would be fast and the result much better. A similar problem, using splines to set force inputs for a robot that travels through a maze with barriers optimized using start and end position and force minimization, was a lab in a course I took last year. The question becomes one of hyper parameter search, i.e. what kind of model/function approximator is sufficient. Here the problem is easy enough that its easy to find a sufficient simple model. The huge networks are nice for more general problems because they tend to work moderately well for everything... in the dataset.

deleted 6y ago

[deleted]

sdoering 6y ago

It is a nice, albeit problematic example. Nonetheless I find many such examples in my daily work as a Data Analyst. Regarding recommender systems I see many companies trying neural nets and so many other fancy ML stuff for things that - in AB-tests are always outperformed by basic rules. I understand the fun it is to build stuff and to use the new hot stuff. And at least for many analysts and marketing people as well as shop product owners this is new hot shit. I also understand, that it is way more easy to get management to hand out the big bucks for something that is the new rage, as they tend to read the respective soundbites in their manager magazines. But as said - I see it underperforming in tests nearly all the time. Not only, but esp. if you take the costs of development and maintenance into account. These systems cost more to build, more to enhance, more to run and bring in less real business value 70 - 90 percent of all the times I have seen them. But they are presented to management in shiny presentations from agencies that need to sell the new hot shit to their clients to show that they are relevant. Because, as said, management thinks they need it and are often not open to agencies telling them, that the business value could be better served otherwise. Because in the end for a manager it is often times more valuable to show a fancy state of the art project to his/her higher ups than creating real business value.

mlthoughts2018 6y ago
> “ Regarding recommender systems I see many companies trying neural nets and so many other fancy ML stuff for things that - in AB-tests are always outperformed by basic rules.” I work on large scale recommender systems for an ecommerce company and in my career I’ve seen only the exact opposite. Don’t get me wrong, sometimes simpler ML models, like clustering LSA vectors or nearest neighbors, work better than complex models like neural nets. But I have never seen plain rule systems work better for any problem even remotely at scale. Rule systems give an illusory sense of control and understanding, yet are rife with complex interaction effects and edge cases that typically make them intractable to change.
- sdoering 6y ago
  From big automotive clients to small-ish fashion eCommerce. From publishing to food-delivery (with upselling in the checkout process) - I found the gains in using rules -> simple ML techniques -> complex systems like NN in most cases not to warrant the costs. The quality of recommendations nearly always increased from a revenue as well as perceived quality standpoint. However, it almost never had a positive impact on the profit margin that would have justified the necessary investments. As said - from a quality standpoint most simple systems were just "good enough". One would need to know the business case and environment. But take automotive (new cars) as an example: The goal is nearly always to get the user to request some form of contact from a physical dealership near them. For that you nearly never need a perfekt, fully configured recommendation. I know of an example (a car manufacturer) where the search space of all configurable variants (including various things the car owner would never register because they are specific screw variants) is of a size where even the number of visitors to the website per year is some orders of magnitude less than the number of options. The way to go here was to reduce the search space and the number of variants. Here it turned out that you can, quite fast, reach that goal with few specific questions (active learning) to lead the user to vehicle variants, which correspond to its interests and led to a disproportionately high contact behavior. And yes: ML techniques were used for the analysis and reduction of the search space. For the concept people to then develop specific questions to get to these reduced attributes. But in the end the recommender now works rule-based. I don't imply that this holds true for every scale of company/problem. And I know some counter examples - but most companies do not operate on that scale. If you are ebay, Zalando (Germany) and the likes: I would probably get different results from testing the revenue validity of the different approaches.

st1x7 6y ago

More accurate title: You probably don't need a neural network if your model has 5 inputs.

deleted 6y ago

[deleted]

karolkozub 6y ago

I love the article, but I don't agree with the premise that machine learning equals neural nets. In my understanding machine learning is a very broad term that just as well could be applied to the polynomial model if the constants were optimized algorithmically. I feel like the presented argument is more for transparent vs opaque models rather than machine learning vs something else. Also one could argue that the polynomial model is just a perceptron[0]. [0]: https://en.wikipedia.org/wiki/Perceptron https://en.wikipedia.org/wiki/Perceptron

lmilcin 6y ago
Well... I guess most people equal ML with AI and use these terms interchangeably. If you just replace ML with AI everywhere in this article it is going to make sense. The article has other problems, one being the main premise. The problem isn't to drive a car around track (which is what the polynomials did) but rather write a program that can figure out how to drive a car without you knowing how to solve it.
- maweki 6y ago
  That's not symbolic AI though. That's only statistical methods. The statistical methods are all the rage now, but explainable AI that can reason is an important area of computer science (and research) and uses formal methods. Edit: yeah, you can downvote this, but current AI research splits right along this line, whether it's symbolic or statistical. Some AI courses will use NNs, others will use Prolog and ASP. You can't just dismiss a whole field of research by reducing AI to statistical methods.
  - laichzeit0 6y ago
    When I see "symbolic AI" I immediately think of Gary Marcus and immediately feel disdain towards the topic because of his behaviour on Twitter and other places.
    - maweki 6y ago
      I don't know the dude. I "only" know that my field of research is deductive reasoning in interactive applications and that this area falls under "Logic Programming" and LP is an area of AI. I know that AI researchers are usually a bit dismissive about the other area. I don't like statistics either. Reducing the whole of AI research to statistical approaches (and NNs are one of those) is disingenious and dismisses hundreds of researchers doing important work. You may not want to have rule-based image recognition, but if your car decides to run over somebody, I feel we better have an explanation for this behaviour based on reasoning and logic.

huffmsa 6y ago

"You might not need neural networks" would be a more appropriate title. The non-nn example given here is one for epoch in epochs loop to take care of picking the best coefficients away from being a ML implementation.

ccortes 6y ago

As a former data scientist I feel we try solve problems the hard way because one of two reasons (maybe both): 1. To feel smart 2. To justify our paycheck Most of the time simple solutions like the one in the link will be more than enough but we just can't resist the urge to implement this new paper we just found. I remember thinking about using a NN for a problem we had, after looking it closely for two days all I needed was a simple linear regression and got the extra benefits of being able to explain what was going on. Eventually I started looking for the "elegant" solution because now that was the thing that made me feel smart. Although some times using brute force is the best approach. There's a balance you find once you gain enough experience... I think.

GarethGonzales 6y ago
I agree, but as a data scientist myself I've always known that part of my job is to choose the right tool out of the box. If that tool is a linear fit, or heck even adding a few columns in a spreadsheet, it's my job to realise that and choose appropriately. If it's a complex custom-built NN, then as long as the cost-benefit analysis justifies the build time, I'll choose that. Of course, as you mention, there's always the business-political aspects - of explaining or justifying your choice to people who don't understand any of those tools, and who often want to pretend that they're part of a "smart data-driven AI" company.
spratzt 6y ago
Genuinely curious as to what you moved into after working as a data scientist. I'm a data scientist and desperate to get out.
- Flashtoo 6y ago
  As a data scientist at the start of my career: why?
  - st1x7 6y ago
    The list is so long we might as well start another thread for it.

stfwn 6y ago

> A key feature of neural networks is that the outputs are a nonlinear function of the inputs. However, steering a 2D car is simple enough that a linear function is more than sufficient, and neural networks are unnecessary. This depends entirely on the definition of 'steering a 2D car'. In the model used, throttle is simply proportional to the distance to the nearest wall in front of the car. This means the agent will never accelerate coming out of a corner, because it can't know it has the headroom to steer away from the wall as it's coming out. Similarly, the model for steering inherently steers the car towards the middle of the track. I would expect the car to wobble from left to right if the road's edges are ragged, make up its own corners if the track edges describe a 'fake' turn on a straight bit, and the car would likely crash if it were to encounter a Y junction or a pit stop. The neural network agents showed smarter behavior here because it is able to capture more complex cross-dependence between different inputs. On the topic of junctions, if the track were to include them, perhaps it'd be nice if the car chose the quickest route to optimize for lap times. But maybe that stretches the problem statement too much. > Instead of doing anything fancy, my program generates the coefficients at random to explore the space. If I wanted to generate a good driver for a course, I’d run a few thousand of these and pick the coefficients that complete the course in the shortest time. In theory this is more random and less efficient than an evolutionary algorithm, which searches the problem space in a structured way. If the author really wanted to hammer the point home, a least squares method to one-shot the coefficients would be more convincing. All in all, the author doesn't make any hard claims that are false. But I would nuance the point of "neural networks are unnecessary" to "simpler models will do for simpler objectives".

blackbrokkoli 6y ago
I see the article more as a broader metaphor for the AI hype. Take, I dunno, video recommendation. Sure, YouTube itself proably built insane stuff in their engine you could never replicate with classic methods (ignoring whether the YT algo is any good). However, if we are just talking about the Vlog of your real estate company, you should probably A/B-test whether your viewers prefer order by time or clicks and implement a decent title search bar. And kick the consultant hyping you up about ML out, now. So my takeaway is that is not about whether AI is never useful or about 2D steering, but about using the right tool for the right job. And building on that, I have to give the author probs to demonstrating an alternative solution to I problem wich I would have definitely solved via AI.

johbjo 6y ago

It goes without saying that if you know (or guess) the model or function you are trying to implement, then you don't need machine learning. Toy problems like this are still interesting because they demonstrate techniques that are applicable to bigger problems. I'm guessing the network in the car game doesn't need more than one layer and two outputs (acceleration vector), but that is besides the point. The technique being demonstrated is the ability of GA (or maybe particle filters), to find "optimal" weights for a network given whatever simulator. This is always interesting, especially when done with graphics like this.

ZephyrBlu 6y ago

Can someone who knows C help me understand the source code? https://gist.github.com/skeeto/da7b2ac95730aa767c8faf8ec309815c https://gist.github.com/skeeto/da7b2ac95730aa767c8faf8ec3098... The concepts and equations he has are pretty simple but I'm really not seeing how they translate into code. The code also isn't the easiest to try and understand for me. Multiple 1-3 letter or otherwise (Seemingly) poorly named variables, some C jargon I'm not familiar with (Mostly the -> operator) and the fact it's a relatively large chunk of code. It seems like the core logic loop is here: https://gist.github.com/skeeto/da7b2ac95730aa767c8faf8ec309815c#file-aidrivers-c-L428 https://gist.github.com/skeeto/da7b2ac95730aa767c8faf8ec3098... And the position and acceleration of the car is altered each iteration through randomization? It's hard to piece together.

organian 6y ago
The arrow operator is a field access through a pointer. That is, if you have a pointer to a struct, it's the same as dereferencing the pointer and accessing the field: "(*foo).bar" corresponds to "foo->bar". I think the main driving logic is on line 228. It could definitely do with some more descriptive variable names, though - "a" for angle, and "s" for sense inputs seems a bit too short.
devdas 6y ago
-> is a dereference operator. On line 65 (function ppm_create), f is declared as a pointer to a structure of type ppm (a reference to an object of type ppm). f is the actual structure/object. (f).w is the 'w' member of the structure. The brackets are needed for operator priority. f->w is another way to write (*f).w

wokwokwok 6y ago

I feel like there’s something more interesting going on here than the author is giving credit to. Most tools you can start by solving simple problems with, and gradually work up the complexity of the problem you’re addressing until it does “something useful”. This is a good way to learn what you can, can’t and should use a tool for. Deep learning is problematic though; solving trivial tasks is actually quite difficult, and often the “something useful” level of sophistication means copy pasting someone else’s paper and tweaking it a bit and kind of vaguely hoping something you do makes any difference. Being able to solve trivial problems with neural networks is really important, and useful; not because it’s a good solution, but because it means you can see what happens when you try it out. The problem with this post isn’t the conclusion; the author is quite right. You can solve this problem in many ways, maybe NN aren’t the best for this kind of trivial problem. ...but, if you want to solve harder problems with more than random trial and error tweaking parameters, solving easy problems with the same tool is a good way to learn how. ...and we have like 20 years of proof that using hand crafted models has been proven not to scale effectively.

roenxi 6y ago
I can only agree. It is also notable that pretty much the entire repertoire of debugging techniques from programming land are useless. Attempting to implementing a paper from scratch is an interesting experience. (1) implement what they describe (2) it doesn't work (3) ... well, that was fun. Project over. Very different experience from trying to implement quicksort, even though the extensions to a basic neural net often aren't conceptually much more complicated.
disgruntledphd2 6y ago
> and we have like 20 years of proof that using hand crafted models has been proven not to scale effectively. What do you mean by this? In what context? Like, the twenty year old models are still running in credit risk and insurance, so I'm confused if you mean in ML/statistical modelling.
- wokwokwok 6y ago
  You know, driving cars, like the article was talking about? ...or, NLP, audio & image recognition, recommendations... come on. It’s not controversial.

jokoon 6y ago

Can you "download" trained neural nets? Aren't there popular formats for those? For example, if I want to recognize pictures of animals, with reasonable accuracy, how much space would such neural net take? I'm not very taught about ML, but in my mind, since ML is generally heavy on computation, it should be possible to distribute neural nets for specific applications and re-use them.

chewxy 6y ago
Yes you can. The "universal" format is ONNX. You can even do crazy things like this: https://blog.owulveryck.info/2018/06/11/recurrent-neural-network-serverless-with-webassembly-and-s3.html https://blog.owulveryck.info/2018/06/11/recurrent-neural-net...
tuvan 6y ago
Yes, most of the large models that are not feasible to be trained by individuals are downloadable. Frameworks make it easy to import and export snapshots of model weights. See https://github.com/tensorflow/models/tree/master/research/slim#pre-trained-models https://github.com/tensorflow/models/tree/master/research/sl...

erichahn 6y ago

Neural networks are just regressors. Yes you can learn the weights with genetic algorithms. Is this advised? Not so much : 99.99% of neural networks are trained with some variation of gradient descent on a specified loss function. I don't even know if I agree to the statement. Polynomial regression solves basically the same problem as neural networks but performs way, way, way worse on big datasets. But nonetheless I would say that polynomial regression is machine learning too.

Der_Einzige 6y ago
This post isn't about learning NN weights with GAs, Also the reinforcement learning community does a significant amount of it's work with neural networks which are not trained using gradients. Gradient free training is a highly active research area. More like 90% of neural networks are trained with gradient descent
- 0-_-0 6y ago
  It's a GA without crossover applied to a single layer neural network with a linear activation
- erichahn 6y ago
  No but this post is trying to generalize on an experiment where NNs are trained with GAs which is - in my opinion - not where NN shine.

villgax 6y ago

You don't expect anyone to write up code to create image classifier for obscure category or the daily?

cupofcoffee 6y ago

I simply do not understand the ML hype. It's absolutely ridiculous. On top of that you have Elon Musk thinking we are a few months away from SkyNet even though we are decades and decades away from AGI. Each and every time I see neural networks on some "techies" blog, I wanna vomit.

mckirk 6y ago
Decades and decades? I wouldn't be so sure in such a bold prediction. One breakthrough in meta-learning and we could be well on our way. (Which is not to say that it will happen next year -- just pointing out that a development like this is nigh impossible to predict with the certainty your comment expressed.)
luc4sdreyer 6y ago
> On top of that you have Elon Musk thinking we are a few months away from SkyNet even though we are decades and decades away from AGI. The actual problem is that we don't know how long it'll be until we build an AGI. Experts put the range somewhere between 10 years and never. Building an unconstrained AGI is an existential risk, so it's important to try and narrow the confidence bands on these questions. That's one of the reasons why Musk pledged $1 billion to OpenAI.
- cambalache 6y ago
  The answer is "Not any time soon", you could also worry about giant lasers on sharks. Trying to regulate now a supposedly imminent AGI is like Morse and Edison regulating the Internet. We are doing an spectacularly lousy job of regulating what it already exists.
- qayxc 6y ago
  > Building an unconstrained AGI is an existential risk But why? So far no-one has been able to explain this to me. An AGI in at of itself is nothing but a brain-in-a-vat. The exponential increase in scientific knowledge was based on the scientific method, which replaced the ancient Greece discussion-based epistemology with a cycle of observation-hypothesis->test->observation->... An isolated AGI cannot test its predictions about the world. Since the unconstrained space of possible explanations is bigger than the space of explanations constrained by observational evidence, there is no way an AGI can gain useful knowledge without access to external observations. This means we are in full control of how "intelligent" (by whatever metric) an AGI can even get by restricting its access to information. But even unlimited access to (passive) information only gets you so far, as some models require data that cannot be obtained passively (i.e. they require deliberate controlled experiments). The final nail in the coffin of dangerous AGI is interaction with the physical world. Yes, even a toddler is a terrifying menace to millions of people if I place a button right next it that sets off a thermonuclear bomb in a city centre. But how about we just don't do that? An AGI with limited or no physical interaction with the world (directly via robot body or indirectly through remote access) can't be any more harmful and menacing than the late Stephen Hawking. There's no need to put AI on a leash, since there's a final naturally limiting factor: energy. Switch off the cooling system and your AGI has to throttle down lest it faces fiery death.

yes_man 6y ago

Generally it is easy to agree with the sentiment that machine learning is being used to solve problems where it isn't really needed. But on the other hand, the same argument could have been used when computers initially came along: "you don't need a supercomputer to calculate this function, I can do it with pen and paper faster". So the real value in projects like this is not that you can now drive 2d car game with a neural network, but that neural networks just took another tiny step forward and demonstrate that their capability to minimize error functions is in principle transferrable to real life problems no matter how stupid, and one day those small steps will have accumulated and neural nets will (and already have) supercede many limits that humans have.

sails 6y ago

I'd guess that a NN could be extended to "racing" multiple cars, where if that was your goal, a polynomial wouldn't be very competitive?

svntid 6y ago

I love that :)

alanfranz 6y ago

s/machine learning/deep learning/ That's what the author means. There are a lot of "traditional" ML and AI techniques that can work very well for a lot of problems.

LoSboccacc 6y ago

well yeah anything you can do with NN you can do chaining enough functions what NN allows is to test millions of permutations of thousands these functions to find the one that works if your problem is intractably hard to figure out functions for, NN provide a larger hammer to brute-force an approximate solution.

yters 6y ago

How is this not machine learning? I guess because there is no gradient descent to pick the parameters, just random space exploration?

username90 6y ago
Gradient decent has been known for hundreds of years, it isn't machine learning. Optimizing parameters is an entire field in itself. Calling all cases of parameter optimization "machine learning" is just a fad because it sounds cooler. Before computers people still optimized thing but by hand, doing the same calculations with a computer is no more machine learning than doing them by hand is human learning.

viviansolide 6y ago

What about Deep Learning? :)

mlthoughts2018 6y ago

This is a farcical idea if the author actually is serious. The polynomial method is deep, dangerous overfitting. It’s the same reason why you don’t make a hand-made decision tree: you don’t know which features of the data are important or not. Presuming you do know, like basing the car’s steering on a polynomial fit of the three directional probes, will be a disaster in non-toy problems. This example shows why you _do_ need machine learning. The toy steering problem is a place for you to work out how the car can learn to drive with as little structure or assumption baked in as possible. Putting less a priori structure on it directly means you need models with higher capacity to figure it out. To put this all more succinctly, “never use machine learning when business logic would do” is a statement about how commonly needed machine learning is, not business logic.

yalogin 6y ago

I understand it’s not needed but is it wrong? I see machine learning as a general purpose black box function approximation tool. It could be a polynomial curve or something else entirely, as long as we have enough data we can approximate it using ML. So in one way I see it as a “lazy man’s easy way out” tool.

astrea 6y ago

All he proved was you don't need a neural network for MicroMouse. We've known that for years. Some problems do require it. In my job experience, facial recognition can't be easily done with traditional signals processing. Stereo vision can be, but the algorithms are slow, expensive, and very error prone when compared to deep learning models.

laquition 6y ago
The author is pretty much attacking a strawman he cherry picked. He took one toy example of someone learning ML and used it to attack the newest trend. Instead of taking the time to dive into the topic he's trying to discount it as less relevant. It's obvious he's stuck in the C/Unix era of programming, which has long since passed it's heyday.

curiousllama 6y ago

The best time series prediction model is often “Use the previous day (or week’s) value”. The best classification model is often “pretend everything is the same.” The best regression model is often “assume it’s the mean”. This should not be surprising: a core premise of data science (empiricism in general? All logic and human knowledge?) is that intuition is often wrong. Initial hypotheses/problem definitions are just intuition.

edgyquant 6y ago
This is incorrect and I’m not sure the point you’re trying to make.

xthestreams 6y ago

> Instead of doing anything fancy, my program generates the coefficients at random to explore the space. If I wanted to generate a good driver for a course, I’d run a few thousand of these and pick the coefficients that complete the course in the shortest time. It is worth pointing out that this strategy can likely overfit on the data that you have used for training: when you change the track, your car may not behave as good as before. In other words: the coefficients are only good for that specific track(s). The author is still using Machine Learning, even if not with neural networks: the need for rigorous strategies for model selection doesn't disappear.

firethief 6y ago
> It is worth pointing out that this strategy can likely overfit on the data that you have used for training: when you change the track, your car may not behave as good as before. In other words: the coefficients are only good for that specific track(s). More generally, I think this approach is only suited to "static" courses, which only matter in contrived demos. Any real use of a steering algorithm requires reacting to conditions that can't be predicted a priori; e.g. if you have to avoid collisions with other cars, and one car is controlled by a human player, every run is effectively a different track and overfitting like this would not be an option.

ben7799 6y ago

There's another aspect of this overhype of machine learning right now. Manager - We need to do X, we need a ML Data Scientist. Engineering Team - there's a simple solution we can use instead of ML. Manager - No we need a ML Data Scientist, we need to do this right. Time Passes, hiring a ML Data Scientist is hard, and the problem never gets solved.

bayeslaw 6y ago

I thought hn has an intelligent and well informed crowd. ok I get it that engineers are not scientist but judging from the responses here it's quite staggering to me how clueless some ppl are here.. do yourself a favour and maybe learn a bit about ml before forming an opinion. massaging JavaScript for a living is cool I guess but maybe there's a reason faang is paying 5-10x your salaries to ml researchers. maybe just maybe it's not just hype but billions and billions of dollars worth of innovation potential. it's quite clear to me that a lot of cs ppl are super bitter that they are not the top paid ppl anymore but supply and demand dictates these things and turns out inventing the latest transformer architecture is more valuable (and factors of magnitude less ppl can do it) than writing trivial front end or backend stuff..

laquition 6y ago
Basic programming is quickly becoming antiquated and articles like this, and many of the commenters are stuck in the past. The efficiency of ML is in cost, not overall computation. Throwing machine resources at a problem is cheaper than hiring some guru with the necessary math and CS background to solve the problem. We've seen the same thing with frameworks/libraries. Before it took specialized knowledge to do basic things like networking, media creation, etc. in code. Now there are existing tools that do everything for you. The same has happened with optimization/algorithmic knowledge, and the genie is not going back in the bottle. There will definitely be specialized cases where that particular expertise is needed, but that is no longer the norm.

tel 6y ago

It's always a disappointing state of affairs when people skip simple models. If nothing else, a nice baseline like the one described in this article should be constructed before using ML techniques to serve as a comparison point.

offtop5 6y ago

I wasted roughly a month trying to get machine learning working for a game. Unless you seriously need it, and have a massive budget to match, basic boolean logic will probably get you where you want to be faster. I wouldn't advise any solo developer to use machine learning unless that's the entire product. It's very easy to make an insanely difficult game without it

plg 6y ago

I love love love the plain C code, making use of unix pipes to produce (real time) video. The subtext is also that you might not need fancy tools.

amatic 6y ago

I've read a wonderful paper on what information people use when driving, it is pretty similar to this driving simulation. The basic cue is "where you will be in T milliseconds", extrapolated using the current vehicle velocity, and trying to keep that point on the road by steering and throttling. Here is that paper, A unifying theory of driver perception and steering control (2019) https://www.researchgate.net/publication/337024514_A_Unifying_Theory_of_Driver_Perception_and_Steering_Control_on_Straight_and_Winding_Roads https://www.researchgate.net/publication/337024514_A_Unifyin...

ExcavateGrandMa 6y ago

This guy is EPIC at coding! I'd have loved to have such an enthusiast as teacher!

giardini 6y ago

Statistical sampling has been reported to work in many cases.

poxwole 6y ago

Machine Learning is basically the new Snake Oil Cure All. Everyone is slapping it on everything

fractionalhare 6y ago

I agree with the premise. But to dig into the specifics here, because it's not clear in the article: is this model generalizable to arbitrary tracks, or will the author have to generate new coefficients for each track? If you have to generate new coefficients for each track, your polynomial regression reduces to a polynomial interpolation of two-dimensional points which represent the track path on a plane. Which is fine and still accomplishes the specific goal, but doesn't solve what would generally be considered the actual research problem. But then again I don't know if the neural network actually achieves this. It's a little unclear in the video: I don't know if the model is being able to learn from the human guiding the vehicle on n iterations or if the model is generated by the human guiding the vehicle on n iterations. Presumably the research goal is to develop a model which learns tracks (in this circumstance, that would be akin to the model choosing the coefficients rather than being the coefficients).

hour_glass 6y ago

In cases like this and many other cases i feel like the purpose of the machine learning algorithm should be to find a simplified polynomial function. I remember reading a paper in which an “AI physicist” was created and it would reduce complex machine learning models into simple equations that would explain the physics of a simulation. It would reduce the complex simulation into human readable and potentially more useful and predictive “laws” of physics.

ummonk 6y ago

I'd make two broad points against this article: A) In most domains, a nonlinear model is far superior to a linear model. At the very least, generally you need to at least apply a transformation (e.g. logistic) to create a nonlinear model from a linear model, because most problem spaces are nonlinear. A nonlinear model doesn't have to be particularly complex though. B) Machine learning (i.e. automatic tuning of parameters) is far simpler for the user than manual tuning of parameters. It's not a question of whether you "need" machine learning, but whether it will save you work. In fact, the author here is in denial - he actually says he would do machine learning, but doesn't realize that is what it is: "Instead of doing anything fancy, my program generates the coefficients at random to explore the space. If I wanted to generate a good driver for a course, I’d run a few thousand of these and pick the coefficients that complete the course in the shortest time."

didibus 6y ago

> If I wanted to generate a good driver for a course, I’d run a few thousand of these and pick the coefficients that complete the course in the shortest time. I might be pedantic here, but wouldn't this then be a machine learning algorithm? Since the machine is learning the most appropriate coefficient based on some heuristic (best of X random). Wouldn't it be better (and more honest) to say then that, simpler ML models and learning techniques can often yield good enough result that don't justify going to more advanced model and learning techniques?

bastawhiz 6y ago
> I might be pedantic here, but wouldn't this then be a machine learning algorithm? I would argue no. Simply trying a bunch of inputs and choosing the most effective ones based on the output is like running a single round of training on a machine learning model. It's hardly machine learning by any stretch—there's no feedback loop, the machine doesn't "learn" anything. If I build a compiler and test a thousand constants for default configuration options and choose the ones that perform the best, or would be a very bold claim to call my compiler "powered by machine learning".
- didibus 6y ago
  > If I build a compiler and test a thousand constants for default configuration options and choose the ones that perform the best What if that was automated in the build? I had assumed the quote meant that it would be, not that they'd manually run a few random cases and manually pick the best. I thought it be automated, like given a new course, you'd run some training routine where the machine would play the course say 100 times each time choosing random coefficients and at the end, it'll take the ones from the pass that resulted in the quickest playthrough, and those would say go in some config file and become the coefficient used by the AI CPU race cars for that course. To me this is machine learning. Especially if you crank up that 100 to 1 million or 1 billion. At that point, it's still something that only a machine could do, I couldn't realistically try 1 billion random coefficients for the ones that result in the fastest playthrough. So in effect, I see the machine is learning which coefficients perform better for a given course. If it tried 1 billion, it learned that out of 1 billion different possible coefficients, some particular set was the best. > there's no feedback loop, the machine doesn't "learn" anything So that's interesting, because if my prior statement is not to be considered machine learning. My next question is what's the criteria to go from the above to machine learning proper? So it seems that it could be the learning has to be a feedback loop. So each attempt at learning must take something away from the previous. I'd be okay with this definition. And now I'm thinking what's the most minimal modification I can make to meet this new definition. What if as it tried random coefficients, it remembered the ones it tried prior? And what if it made sure that no new attempt at a random set of coefficients had already been attempted before? This isn't super refined, no heuristic to what are the best coefficients to try next given the ones tried prior like say what linear regression would accomplish. But it still meets the definition. It starts random on the first round, and the next round is no longer truly random, since it can't pick the previous's round coefficients again. So at least it's descending along the set of possible coefficients through a path that will eventually try all combinations. Would this be enough to be considered machine learning? I'm also thinking this can start to sound a lot like Evolutionary Computation. Oh boy, in all honesty, I've always been confused about differences between stochastic, metaheuristc, and machine learning optimization techniques.

adontz 6y ago

I like how these cars on the last animation move like a school of fish.

danans 6y ago

ML is useful for domains where we don't have a good analytical model of real world behavior of systems. Of course, that is a lot of problems, including many useful things that biological intelligence appears to solve. But it is true that ML - in particular neural networks - often gets thrown at problems that traditional analysis and modeling could serve more efficiently, and also more predictably. The other benefit of first trying traditional analysis is that it helps the implementer of the system better understand the domain first. Pertinently, I've heard some scientists working on medical imaging describe to me how there has been a trend away from the basic science of understanding the medical phenomena behind the image being analyzed and instead relying excessively on ML based pattern recognition to make predictions/inferences.

collyw 6y ago

Repeat for pretty much every technology that is "in fashion" at the time.

anatoly_b 6y ago

Hi, Please try implement formula calculations here: https://youtu.be/eBaHqKVWriU https://youtu.be/eBaHqKVWriU Use neuromorphic networks and be happy!

anatoly_b 6y ago

Use neuromorhic and be happy https://youtu.be/eBaHqKVWriU https://youtu.be/eBaHqKVWriU