9 ms·
Honestly, as someone who has been playing around with/working with AI in some form (previously bayes nets etc...) since the 90s and trying since 2012 to build o
by AndrewKemendo 10y ago
Honestly, as someone who has been playing around with/working with AI in some form (previously bayes nets etc...) since the 90s and trying since 2012 to build our computer vision company - every keynote from AFGAM makes me want to give up a little more each time.
It's like even with a world class AI team the majors just blow past everyone with their on demand scale, access to data, hardware, people and distribution.
In the past three weeks the products that were shown at conferences (GTC, Build, I/O) would have been 100 different independent companies, with very highly trained, specialized PhD level researchers and developers only 5 years ago. Now this stuff is just baked in to the top platforms.
This has been happening for a while, and I've been saying it for a while too. No clue where to go from here honestly.
edit: The point I didn't make here but is subtext is that, IMO ML/AI is the last frontier for technology (an IMO humanity) so if a few dominate it, its kind of game over for the existence of smaller players generally.
- blhack 10y agoKeep in mind you're watching a marketing demo.
- wsinks 10y agoI know this to be the case. It's all careful shown for the moment.
- jboggan 10y agoLook at the flipside - for the people who want to focus on applications of ML instead of futzing with platforms and implementations this sort of environment is a godsend. Compare what it took to keep a server/website/database up in the late 90's compared to something turnkey like Heroku today. It frees up a lot of developer time and energy for building the next layer of innovative things!
- AndrewKemendo 10y agoThe "flipside" is that some of the best minds in AI basically have only 2 choices coming out of school: - AFGAM - Military The period of being able to build your own competitive AI startup was...what, like 3 years between 2010 and 2013?
- nailer 10y agoAs of 2016 investors were still throwing money at 'ML for x' startups.
- pas 10y agoYeah, why not, Google will buy them if they are okay.
- ballenf 10y agoIf AI is the high end of programming, then it's the equivalent of saying that a recent manufacturing grad can't build their own car assembly line and has to go to work for Toyota, Honda, et al. Barriers to entry at the "top" of industries are generally higher than lower down. I'm sympathetic to the sentiment. Consumers might benefit from the conglomeration more than from a lot of independent groups each reinventing the wheel or spending resources on duplicative infrastructure. There are trade offs of course. In particular, I really don't want one single ML platform to have all my data and know me that well.
- AndrewKemendo 10y agoThe metaphor would be that no combination of people will ever be able to compete with Toyota, Honda etc... and if they tried would be acquired or beaten to death by Honda because they are leveraging their massive scale.
- inlineint 10y agoBut what about Tesla? It managed somehow to compete with Toyota, Honda, etc, although it had to create a new market (market of electric cars) to do it.
- neovive 10y agoI agree with this point. One can view this is as an extension of the current trend towards platform/infrastructure outsourcing. If AI and ML are deemed essential to future apps, we should be just as comfortable consuming it via a PaaS as we are with leveraging other parts of a cloud infrastructure. A valid concern, as pointed out, is when the large platform providers begin moving into domain-specific markets, leveraging their scale and closing out start-ups/small business. However, I believe nimbler startups will always be able to carve out a successful niche as long as they are closer to the customers.
- phil21 10y ago> However, I believe nimbler startups will always be able to carve out a successful niche as long as they are closer to the customers. The difference is where the majority of the "idle wealth" is created. Sure, there will be great business doing consulting work leveraging cloud providers - you can make a good business simply consulting on such things. However in the end you're simply enriching a large company. Whenever an "amazon cloud consultant" transitions a closet of servers to AWS - they made a bit of money, but the large bulk of the value of that customer is now in a giant corporation's hands vs. being spread out how it was traditionally (either in house, or via much smaller infrastructure providers that generally competed on service without much lock-in). The lack of competition in the web space is astounding to me, and will be incredibly damaging to innovation. How quickly people forget why decentralization used to be considered such an important goal. Yet it appears the entire industry is rushing head long into this model.
- dlwdlw 10y agoOne thing though is that although 1:1 specialized and individual service is always superior, the providers arent well paid. This means that startups, though better than large companies in a certain respect, can no longer become rich because they no longer are capable of creating a barrier to entry. A butler provides good service to you individually yes, but he can also be replaced fairly easily. (Relearning your quirks and stuff) Startups then return to what they are in any other countries. Just a way to make a living rather than the potential path to riches it is in the bay area. As a side effect you get smaller elites controlling more of the pie. That has other implications as well.
- omarforgotpwd 10y agoYes, access to data is everything.
- snarf21 10y agoI think you need to pick a different niche. I don't know what your computer vision focuses on but try to pick something the big players aren't focusing or has less mass appeal. It also says to me that if you become a domain expert for this niche and develop the models, etc. that you are ripe to get acquired by one of the major players as automation (and software) continues to eat the world.
- ska 10y agoIt's always been about the data. Algorithms talent and effort is at best a 2nd order effect - and this has always been true. As a result, you would success in this area is mostly driven by the data. In other words: this is rolling out exactly how you would expect, no?
- AndrewKemendo 10y agoOh it totally is, it's just one of those things that I think most of us expected to be more distributed rather than relegated to a few silos of data. That's not to say there aren't opportunities, it's just that if anyone makes significant progress in one of those other areas, it's either on the back of AFGAM or will eventually be absorbed by them through acquisition/targeting.
- acchow 10y ago> No clue where to go from here honestly Would you be opposed to joining one of their AI teams? Clearly you believe they are building great stuff...
- webmaven 10y agoNot the GP, but many people in this situation cannot move to one of the few places the AFGAMs have offices (let alone the fewer places they've located their AI/ML teams), and working remotely doesn't seem to really be an option.
- hota_mazi 10y agoIt's called commoditization. It's a pretty common phenomenon that follows the same well known cycle of first disrupting and then ushering the whole industry into a new era. Nothing to worry about, unless you are trying to create these commoditized items, which is pretty much a doomed undertaking.
- return0 10y agoNot really, deep learning is still a research subject. It's not commodity for small companies, it's probably commodity for google, (e.g. they can use deep learning to design optimal deep learning systems)
- turnip1979 10y agoI feel for you man. Same stuff is happening in IoT.
- mahyarm 10y agoI think IoT is even worse, since the actual tech isn't new or innovative, just being applied everywhere because it has reached a certain price point now. It's also a badly integrated insecure mess currently that has a lot of bugs. Which screams for a big player to take over.
- quotemstr 10y agoI've been wondering for a long time whether machine learning will break the long-time tech industry lifecycle we've all come to regard as some kind of immutable law of nature. Traditionally, 1) we start with big, established companies in tech. 2) A few people, dissatisfied with the trammels that big companies place on their employees, leave, found a new company, and then, 3) through superior grit, gumption, execution, flexibility, and speed, steal market share from the big companies. As the new company grows, 4) it accumulates organizational scar tissue and is slowly infiltrated by incompetent careerists, and finally, 5) becomes the very company that the founders set out to create. We've been going through this cycle since the late 1950s, when the "traitorous eight" left Shockley Semiconductor Laboratory to found "Fairchild Semiconductor". Machine learning might stop this cycle by breaking step #3. Big companies have access to huge data sets that no group of human developers, no matter how talented, can match. Even slow, hidebound, and barely-competent use of these data sets can squash upstart smaller companies. It'll be like a planet, full of water and life, with a core that cools to the point where it can no longer sustain plate tectonics. Without constant replenishment, the crust seizes up, the oceans boil off, and the life dies. The tech industry will become Mars.
- AndrewKemendo 10y agoMachine learning might stop this cycle by breaking step #3. Yea that's exactly what I think is happening, and in fact that's one of the core promises of being first into AI - you use it to move faster than anyone else can because it gives you better intelligence and prediction. Even more so with AGI, which Deepmind (Alphabet company) has explictly stated they are trying to create.
- thwd 10y agoOn the other hand, #3 might in this case be exactly that: Small companies creating accurate & precise modeling techniques using only a limited amount of data, while giants continue to rely on the sheer and blunt volume of it.
- trhway 10y ago>Small companies creating accurate & precise modeling techniques using only a limited amount of data, while giants continue to rely on the sheer and blunt volume of it. until of course sheer and blunt volume allows to build more accurate and precise models. To me this is exactly at the core of the current AI wave - the models emerging inside deep representational layers are more complex, accurate and precise way beyond whatever human analytical thinking could have built or even just come up with. My friend just couple days ago was talking about one specific multi-year effort few years ago to constructively build a practical model of sentiment analysis which i in turn contrasted with that recent result from OpenAI of "sentiment neuron" emerging from [unsupervised] learning of next character in Amazon reviews.
- stevofolife 10y agoTo be honest, your sentiment comes from selfishness. You want a fair playing field so that you can rise up but at the stake of slowing down progress for the entire world. Some fields are meant to be dominated and that's part of life man.
- AndrewKemendo 10y agoThere are many reasons monopolies/duopolies are bad, it's not some selfish game.
- stevofolife 10y agoThat's not really the case here but let's say even it is, these big guys are producing most of the work for everyone to lean on. Look at all the open source initiatives out there. Don't stop progress for the sake of money.
- ashark 10y agoThe stupid-money from spying-heavy web/mobile advertising is suppressing other business models that would be friendlier to smaller companies. Until it's gone (likely require a law—please oh please) we're (entrepreneurs and consumers alike) stuck with a handful of free-but-we-spy-on-you solutions, either provided by one of the big tech companies or soon-to-be acquired by them, and little else.
- pishpash 10y ago"IMO ML/AI is the last frontier for technology (an IMO humanity)" This is too extreme. It is the last frontier of some set of problems, but it just introduces a new set of problems. This always happens. Even the implausible singularity still introduces more problems to solve.
- psyc 10y agoI think the point is that our role in solving those problems diminishes. Certainly some, and maybe a lot.
- zitterbewegung 10y agoYou can out compete Google by simply keeping your services not deprecate themselves and make them available and having good customer service . Look at Clarfai when benchmarked it against Google's API and Microsoft Clarifai was more accurate .
- hasch 10y agono, this ML level still statistical inference. next step is about structure and generalization.
- killjoywashere 10y agoHere's the central issue for someone outside the CS industry: I have large scale, high value data in a competitive, highly regulated, exciting market, something that bends GDP: defense, aerospace, healthcare, transportation. It's not like I got a bunch bored tweens in the Valley to share pics of their food. I have to sell my organization, which may be looking at a decade or more of data, that my CS collaborator can a) handle the data at scale, b) secure the data, c) hold on long enough to get to market. Because a couple years won't even get you to the FDA's or FAA's doorstep. NASA, DoD? FRA, NHTSA? You're looking at a 10 year lead time. You're looking at hosting tens of millions, hundreds of millions of dollars of data, running an equal amount of compute, and doing it for the next 10 years. Not the last 10 years. The next 10. I can't, cannot, convince the org to bet the future on the ability and moral grounding of two random guys in a garage. They wouldn't even be able to wrap their heads around it, the difference in scales is too big: "You want to give our 10 PB of data to two guys in a garage? What? Get out. In fact, go get a new job, you're done here." These deals are being signed by people who worry about capitalized words, like Gross National Product or Personal Health Information or Mars. We're talking orgs with tens of thousands of people who have committed their lives to a thing. We're not trying to use Dutch actions to steer 20-somethings in Orange County to the right sushi restaurant. Which is a sweet trick, don't get me wrong. But it's a bit different when we're trying to protect nations and their economies, and maybe just save the world or the human race. Do you know what the US Government defines as "a large firm" for contracting purposes? 200 people. That's about right. You and I both know that those same two guys working for AFGAM are still the same two guys. But I need to contract with an entity wherein, if those guys I'm collaborating with go south, I have recourse with the Inc to keep the project on track. They can fire the guys, bring in new guys, subcontract some guys, whatever. But I've got to convince serious people that I can maintain a 10 year trajectory in a white hot, globally competitive market.
- dmoy 10y agoSomeone convinced the CIA to drop a big fraction of a billion dollars on AWS, like 3-4 years ago. So it appears to be within the realm of feasibility.
- 10y ago
- killjoywashere 10y agoTo balance out my other comment, check out Indica Labs. It's one guy in New Mexico: http://www.indicalab.com/ http://www.indicalab.com/
- psyc 10y agoI'm glad to see this comment at the top and setting the direction of the discussion. A year or two ago, this thread would have been dominated by ML is 'just' this or that, ML can only do XYZ, it's not AI, it's certainly not a path to AGI, and nothing is.
- narrator 10y agoThe nice thing that saves ML from being only another way for AFGAM to dominate and force all the innovation to a highly concentrated group of geniuses is transfer learning! You can download Imagenet trained brains and get them to classify stuff just as good as in the competitions and then adapt them to various different applications. Google has been nice and releasing some of these models and they'll form the basis of open robot brains, etc. in the future. There will probably be a Open Robot distro pre-packaged with a bunch of these brains and applications in the future. Little companies can adapt these brains to niches that are too small for the big guys.
- Eridrus 10y agoTransfer learning is great, but is still pretty limited; the main success stories are retraining CNNs from ImageNet to other datasets and word2vec/GloVE. In the space of all the things you would want to do with ML, this is pretty limited. [EDIT]: I stumbled upon a transfer learning for NER paper a second ago: https://arxiv.org/abs/1705.06273v1 https://arxiv.org/abs/1705.06273v1 And the results are not conclusive. You can can improve your results in truly data impoverished regimes, but you get diminishing returns as you add more data, so the only way to catch up is to get more data. So, you may be able to get reasonable performance with fewer samples, but you will still get better performance with more samples, and the more samples you get, the less benefit you get from transfer learning. This may not always hold, and I wonder if active learning paired with transfer learning would do better, but at the moment it's not convincing.
- thr0waway1239 10y agoExcept, their AI is simultaneously too stupid to preempt problems like the story we had with the Firebase customer. To generalize this a bit more, their AI is certainly too stupid to preempt similar problems with over-billing in general (though you can certainly create a little conspiracy around that one). And then to generalize even one layer higher, their AI is also too stupid to identify candidates who might get frustrated with their customer support and turn to the HN bully pulpit. And lastly, come on, they can't even predict the election winner! Otherwise why would they throw their weight behind the losing candidate? :-)
- pas 10y agoBecause that was not AI, and that was not a problem for them they set out to solve.