13 ms·
Darpa Goes “Meta” with Machine Learning for Machine Learning (2016)
- zitterbewegung 10y agoShould have a 2016 in the title. I know that Mathematica tries to do this with machine learning functions and its available in Mathematica 10 https://www.wolfram.com/mathematica/new-in-10/highly-automated-machine-learning/ https://www.wolfram.com/mathematica/new-in-10/highly-automat...
- Cacti 10y agoThis isn't about automating model generation, which is an old goal of many, but of DARPA's involvement in it and the start of a new program to encourage work in this area.
- amelius 10y agoSo after truck drivers, data scientists are next on the list of people who will lose their jobs to ML. It seems that this new technology impacts the poorly and highly educated alike.
- nicklovescode 10y agoSimilarly, I think it's important to think of jobs in terms of behavior requirement rather than income levels. For instance, it's well documented that radiologists are going to be one of the first jobs that suffer as a result of ML, due to it being image recognition.
- Namrog84 10y agoHe didn't reference income levels. But that of education level. Education level isn't always correlated with income. Just look at a lot of PhD that don't make a ton compared to less "formally" educated
- deleted 10y ago[deleted]
- sjg007 10y agoYes but it will be fought.. also radiology is also outsourced.
- fest 10y agoI was surprised to see that ECG test results are still analyzed/summarized manually[0]. I have seen ECG machines at hospitals try to summarize the test automatically but doctors don't seem to trust them and still manually analyze the result using measuring compass and specially graduated rulers. It seems to me that the companies manufacturing these machines used pure signal processing techniques which don't give the results doctors are expecting. 0: http://www.ecglibrary.com/ecgs/norm.png http://www.ecglibrary.com/ecgs/norm.png
- felippee 10y agoNot a single truck driver has yet lost his job to ML. Just a side note.
- digler999 10y agoDriver jobs have been lost to autonomous container movers at shipping ports. Not expecting ML to have been integral to that, but driver jobs were indeed lost due to automation.
- argonaut 10y agoYou must be thinking of longshoremen, not drivers.
- renesd 10y agoMining, shipping, and human transport driver jobs have all been lost to robots. There's giant trucks in WA Australia, the ports (again in Aus), and trains in airports. The big new train line in Sydney also has no drivers. (Not just these, and not just in Australia -- these are just used as examples you can look up).
- mastazi 10y agoYour statement is true with regards to long-haul trucks, but not in other cases e.g. the port of Rotterdam has had driverless container trucks for some years now https://www.youtube.com/watch?v=xhZBalESrXU https://www.youtube.com/watch?v=xhZBalESrXU Also confirmed by this article https://www.naiop.org/en/Magazine/2015/Summer-2015/Business-Trends/Driverless-Trucks.aspx https://www.naiop.org/en/Magazine/2015/Summer-2015/Business-...
- taneq 10y agoI don't know if it's "ML" but ore trucks at all the new Western Australian iron ore mines are autonomous. Instead of a driver per truck, you have a couple of operators in the control room giving orders to the fleet.
- drcode 10y agoI disagree- The last humans to be jobless will be those that specialize in massaging data to feed it to computers.
- emorse 10y agoMay not be true as unsupervised learning improves.
- deelowe 10y agoYou sure about that? I assumed it would be celebrities, fine dining chefs, sommeliers, brew masters, etc... Basically, people who are recognized just as much for their notoriety as much as the products they produce. My humidor after all, has not a single machine made cigar despite one every so often being improperly rolled and unsmokable.
- mastazi 10y agoOn a general level I agree, the less quantitative-based is the job, the more chances there are that it won't be done by machines. However let's not forget that we have (and we have had for some time now) algorithmic composition[1] and more recently we have seen style transfer through neural networks[2] and if artificial intelligence is able to produce visual art and music, that makes me think that there are very few human activities that may not be automated. What will "save" some human activities is either technological shortcomings, which are probably temporary (e.g. the hand made cigars in your humidor: the difficulty is that machines are currently not very good at rolling cigars with a whole leaf filler, but I think eventually we will get there) or consumer's preference, which is arbitrary (e.g. you may think "OK so now, in 2025, machines are able to roll as well as the best Cuban rollers, but I choose to buy only human made cigars, just because I can"). [1] https://en.wikipedia.org/wiki/Algorithmic_composition https://en.wikipedia.org/wiki/Algorithmic_composition [2] https://research.googleblog.com/2016/10/supercharging-style-transfer.html https://research.googleblog.com/2016/10/supercharging-style-...
- pmyjavec 10y ago
- deepnotderp 10y agoLiterally no one hires data scientists/ML engineers only to make a basic feedforward ConvNet.
- benevol 10y agoWe're about to really get to know and feel the exponential property of technology.
- phaemon 10y agoDespite having thought of the dangers of AI, this literally never occurred to me. Humans don't create AI, machines do.
- LesZedCB 10y agoImagine if we had the unambiguous source code for our own bodies? AI will be able to have that without any work, and can modify and recompile with little to no effect and all benefit in the matter of seconds, where humans have to wait years. This is referred to as the intelligence explosion[0] or AI Foom[1]. [0] https://en.wikipedia.org/wiki/Intelligence_explosion https://en.wikipedia.org/wiki/Intelligence_explosion [1] https://wiki.lesswrong.com/wiki/The_Hanson-Yudkowsky_AI-Foom_Debate https://wiki.lesswrong.com/wiki/The_Hanson-Yudkowsky_AI-Foom...
- mastazi 10y ago> Imagine if we had the unambiguous source code for our own bodies? We sequenced human DNA, so I think we already have something kinda like it. I'm not a scientist but my basic understanding is that while we have the whole sequence (in your source code analogy, we can do git clone or git pull) we are not really good at tweaking it ("oops this codebase uses architectural solutions we're not familiar with, we will have to study it extensively before we can deploy those proposed changes to the production environment")
- jcranmer 10y agoDNA is not really source code. At best, it's kind of like... an algorithm textbook. Only a very tiny portion of DNA (like, ~1-2% in Homo sapiens) codes for proteins in the manner that you would have learned in biology class. A much larger fraction is used somehow, but how much and what its functions are is still a matter of major debate. The effort to sequence the human genome has also led us to discover that there are other ways in which traits can be heritable that don't involve changes to DNA. The focus now is largely on epigenetics, although explanations invoking microbiomes of bacteria inside our organs was definitely popular for a while. In short, we really don't know a whole lot about our own molecular biology, and a lot of the research in the past 60 years since the discovery of DNA has tended to show "there's more going on than we thought." Where things involve just DNA, we have very good tools for reading (sequencing) and writing (CRISPR/Cas9) it. What we don't have good tools for is modifying our epigenetics, and we don't have a good handle on what comes from DNA and what comes from epigenetics.
- deleted 10y ago[deleted]
- malux85 10y agoTopology generation is the new feature engineering, it's not surprising that it's the first thing to go. I have been working on advancing HyperNEAT for this very purpose - similar, but not quite the same to: http://blog.otoro.net/2016/05/07/backprop-neat/ http://blog.otoro.net/2016/05/07/backprop-neat/ Non-sequential and non-hierarchical topologies are the future, and it makes sense that machines should generate them
- aaronsnoswell 10y agoCan you expand on what you mean by non-sequential and non-hierarchical?
- siavosh 10y agoI'm curious to hear from current researchers: Back when I did some work in neural networks (10-15 years ago), the field had become saturated with 'meta' papers on topology generation, and also on optimization techniques, (slightly better gradient descent, pruning techniques etc). Neural networks had become more art than science. At the end of that period, I got the distinct sense that these were signs that the field had stalled in any significant break throughs. Soon after they fell out favor for well known reasons. I'm curious outside of the current hype, if there's a sense of that now with the current field practitioners and saturation of papers.
- malux85 10y agoTopology generation requires the evaluation of many models - For non-trivial datasets/model sizes the evaluation of a model can be hours, days or weeks, even on current GPUs. There's much better tooling in place, and hardware in place( modern GPUs) now to distribute the evaluation over a cluster of GPU machines, so it's becoming practical to push the boundaries further edit: I also forgot to mention, older methods often required the manual specification of heuristics for mutation and crossover rules, this is no longer the case, because the machines can be trained with objective functions to prevent overfitting, optimize execution speed or memory usage and can use techniques like dropout, skip connections, layer / node hyperparameters and so on to direct the topology search
- hellofunk 10y agoIs NEAT among the methods you refer to?
- malux85 10y agoYes, but take it way further. i.e. How can you use a Deep Network to direct NEAT? Can you model the topology generation as a latent space, and what would some of the properties of that space be? Can we integrate the best parts of transfer learning to speed up the generate -> evaluate cycle. That sort of thing
- xiphias 10y agoThe idea is nothing new..everything in machine learning is being automated. The hard problem is to find.the next step that can be easily...just randomly training new models and adjusting doesn't work, because training time goes up too fast.
- adamnemecek 10y agoI've been trying to find an answer to this for some time. Is it possible that ML could be implemented much better on analog computers as they have a "native support" for differential calculus?
- kelvin0 10y agoOK, so how do you store your model? In analog or digital format? If it's digital you might have to convert the analog signals into digital, with all the constraints involved with such a conversion. Also when you replay your model to process some data, you need to convert back from digital to analog. Please correct me if I misunderstood your point. Otherwise you might need to clarify your idea a bit more.
- adamnemecek 10y agoNote that my arguments are pretty likely wrong in some fundamental ways but here it goes. > OK, so how do you store your model? In analog or digital format? This is a good question. Do you think that conversion back and forth would diminish the gains from analog computing?
- kelvin0 10y agoConversion from analog to digital will necessarily involve the loss of information. Now the question is, is the data lost crucial to the operation of your system or not? If losing some data is not that big a deal, you might as well build it digitally from the ground up? On the other hand if you can't afford to lose any original information from your analog system, you might have quite a few obstacles to overcome and design an entire system which would be analog based (kinda like the human brain on some level). But then again, if you can build a perfect functioning human brain a nobel prize awaits you.
- adamnemecek 10y ago> Conversion from analog to digital will necessarily involve the loss of information. Now the question is, is the data lost crucial to the operation of your system or not? If losing some data is not that big a deal, you might as well build it digitally from the ground up? I think of it as the output being the solution of an equation/dynamic system. It's ok you lose the equations, you care about the solution. I've been doing some light internet research into this not too long ago and came across this title https://www.amazon.com/Neuromorphic-Photonics-Paul-R-Prucnal/dp/1498725228/ref=as_li_ss_tl?ie=UTF8&qid=1484086290&sr=8-3&keywords=photonics+neural+networks&linkCode=ll1&tag=akhn-20&linkId=32c9eac37d5cf1c5815e59f1098bbfb9 https://www.amazon.com/Neuromorphic-Photonics-Paul-R-Prucnal... that seems to suggest that analog photonics might be the way forward. But I can't really comment on the state of this. Photonics is definitely becoming more and more viable it appears.
- kelvin0 10y ago>"...The goal of D3M is to help overcome the data-science expertise gap by enabling non-experts to construct complex empirical models through automation of large parts of the model-creation process..." This makes me think of the premise behind Excel, and Access: provide a way for (mostly) non-programmers a nice wizard like tool to analyze data. Of course, the (painful) shortcomings of such tools are known all too well. Hope this Darpa project fares better. On the plus side it democratizes access to a technology, the good and the bad.
- radarsat1 10y agoWith recent successes in specific games (Go, arcade games), I'd love to see a resurgence of interest in "General Game Playing" research, i.e., the idea that an AI is given a description of a game and learns how to play it. Doing a search shows that it's a subject that is still being taught, but I rarely see recent articles about it.
- bluetwo 10y agoI'm also very interested in this area of AI. It does seem like it is due for a resurgence. I wonder a bit if there is a bias against the gaming aspect of it. The jumps in poker have been kind of neat lately. Would like to see more variations played.
- mastazi 10y agoThis is completely OT but I just googled about the topic of "solving" Texas Hold'em Poker and other imperfect information games and discovered that one of the contributors to the research on Counterfactual Regret Minimization algorithms, developed at the University of Alberta[0], is Oskari Tammelin[1] who also created, in the late 90s, my favourite piece of music software, Jeskola Buzz[2]. His personal website[3] contains pages on both topics. [0] http://poker.srv.ualberta.ca/about http://poker.srv.ualberta.ca/about [1] https://arxiv.org/abs/1407.5042 https://arxiv.org/abs/1407.5042 [2] https://en.wikipedia.org/wiki/Jeskola_Buzz https://en.wikipedia.org/wiki/Jeskola_Buzz [3] http://jeskola.net/ http://jeskola.net/
- radarsat1 10y agoThat's pretty awesome! I used to use Buzz quite a bit too! In fact his harddrive mishap is what pushed me over the edge to become an open source advocate ;) I wish I could find the details about that accident, but if I can remember correctly he lost the source code for Buzz and people spent quite a bit of time reverse engineering the .exe just to be able to continue writing "gear" for Buzz..
- 10y ago
- hellofunk 10y agoDah, that's nothing. We have models that we built specifically to determine the best way to train other models. After 2 years of tweaking these models, we learned that the best way to optimize our learning was to build new models that train these models for determining how to train our other models.
- beachbum8029 10y agoSo it's models all the way down?
- randcraw 10y agoOf course. Even Plato knew that. But since most models are wrong... there's life yet in us 'bags of mostly water'.
- Capt-RogerOver 10y agoModels are not wrong. They are just models. Ie, != reality.
- notgood 10y agoUntil we realize such matrix of models analyzing each other is what we tend to call "consciousness"
- joshvm 10y agoThe obvious citation is the wonderfully titled Learning to learn by gradient descent by gradient descent https://arxiv.org/abs/1606.04474 https://arxiv.org/abs/1606.04474
- yellowapple 10y agoI'm pretty sure this is exactly how Skynet starts. ;) More seriously, as someone who's interested in machine learning but doesn't really know where to start in terms of really understanding it beyond "it takes input, does some weird statistical magic, and gives you an output": would one of these D3M tools (should they become readily available and less theoretical) be a decent starting point? Or would it still be better from a learning perspective to start with something more fundamental or "basic building block"? In other words: does it help to know how to build these models from scratch even if you're using some tool to do it automatically?
- Florin_Andrei 10y ago> doesn't really know where to start - Take Andrew Ng's ML class on Coursera. - Install TensorFlow and TFLearn. - git clone something based on TF from GitHub and hack it. - Do some pet project from scratch. - profit
- jgalloway___ 10y agoI pitched something similar a few years back to a client and the client's architect agreed there was value in it but we couldn't make it fit in our sprint schedule so went with some other user stories.
- slantaclaus 10y agoUpvote because title
- lowglow 10y agoThis is epic and nice to see more people thinking this way. We're actually tackling something similar at Asteria right now and we're about to open source our device client software in a much needed RFC. Gitter here for anyone interested: https://gitter.im/getasteria/General https://gitter.im/getasteria/General
- zopf 10y agoFirst off: isn't this from June of 2016? Second: I don't get it. The primary example they use to illustrate the need has almost nothing to do with model building or selection, and everything to do with selecting and painstakingly cleaning data. This mirrors my experience with data science so far. "A recent exercise conducted by researchers from New York University illustrated the problem. The goal was to model traffic flows as a function of time, weather and location for each block in downtown Manhattan, and then use that model to conduct “what-if” simulations of various ride-sharing scenarios and project the likely effects of those ride-sharing variants on congestion. The team managed to make the model, but it required about 30 person-months of NYU data scientists’ time and more than 60 person-months of preparatory effort to explore, clean and regularize several urban data sets, including statistics about local crime, schools, subway systems, parks, noise, taxis, and restaurants." So - the meta part isn't such a big deal. But if DARPA has found a way to properly automate the painstaking process of selecting, cleaning, validating, and normalizing data, well THEN we'll really have something to be impressed about.
- eruditely 10y agoIs Darpa just a really good program or something? Across the years I have yet to hear any governmental rachet toward decay, they have been seemingly involved and doing it will for quite some time. Any reason for this?
- randcraw 10y agoI suspect the goal here is to automate the task of intelligence analysts. As the quantity of signal data has gone up exponentially, the DoD needs ever more subject matter experts who are also schooled in data science techniques. But that's a rare combo of skills, and given the high demand for these folks in industry and the better pay there, I suspect Uncle Sam is suffering a dire unmet need. Ergo the proposed solution: automate them. But the example in the article of traffic modeling with 30 person-months of analysis and 90 p-months of cleansing illustrates how hard such analysis actually is. Ain't no way you can automate it. Useful intel signal is hard to find in noise, especially in what by now must be zottabytes. Enlisting anyone to do analysis without deep domain expertise, especially someone as dumb as a computer, is not a promising strategy for success. But it sounds like a great way for beltway bandits to get funding for long term blue sky R&D contracts...
- mark_l_watson 10y agoFrom my limited experience of just working on a couple of DARPA projects (decades ago), DARPA is interested in new technology focused on specific problems. The idea of automating aspects of data science like data cleansing and model building makes sense. A good goal. There is obviously a lot of excitement in deep learning success stories but I would like to see more effort also put into both fusing more traditional AI with deep learning, and also better interactive UIs for data science and machine learning work flows. A little off topic, but I started using Pharo Smalltalk again this week after not touching it in a long time. I was thinking about using Pharo like environments instead of tools like iPython for organizing workflows. I admit this is likely not such a good idea because most of the great libraries for data science and AI are in C++, Java, Python, and Scala.
- deleted 10y ago[deleted]