9 ms·
Google Sprints Ahead in AI Building Blocks, Leaving Rivals Wary
- kartD 10y agoI'd really like to see what Apple is doing, they usually come late to the table but they get a lot of things right, even though they may not be the most technologically advanced. Granted this may not be applicable since this is more about AI infrastructure.
- danpalmer 10y agoI definitely agree with your point when it comes to consumer electronics and software, but developer tools/services? I'm not sure. Their investment in AI technology so far looks very small as well.
- ethbro 10y agoI'm kind of curious too. They've got the revenue to fund it, but I wouldn't say "theoretically advanced software" has historically been Apple's notable quality (subject to a few periods and exceptions). Incredible design, hardware engineering, and product fit, yes. But the software typically seemed a more polished version of mainstream ideas.
- surfmike 10y agoAgreed. Took Apple a long time for their cloud based services to work out their kinks.
- honkhonkpants 10y agoApple "worked out the kinks" of iCloud by outsourcing it to actual cloud providers. They may become a cloud services powerhouse but they are way behind today.
- kartD 10y agoMaps too, they're adding some really cool features that google doesn't have
- s3r3nity 10y agoThis is actually where I'm worried Apple is falling behind, _because_ they are a bit private & come late to to the table. Microsoft, Google, and others very openly share and learn from the AI community, but Apple has been known to be only a "lurker" at best (for lack of a better term.)
- spacemanmatt 10y agoThey have almost constantly innovated by acquisition. That explains their lurker status, I think.
- kartD 10y agoThey're also very secretive. They only say they do something new publicly when they have a product to show it off.
- dharma1 10y agoWell, I doubt they are going to open source a machine learning framework to rival TensorFlow anytime soon, but they recently added CNN API's to iOS and macOS, so you can do optimised inference on Apple devices. https://developer.apple.com/videos/play/wwdc2016/715/ https://developer.apple.com/videos/play/wwdc2016/715/
- cm2012 10y agoApple is not really a leader in software the same way they are in hardware.
- objectivistbrit 10y agoBrett Victor mentioned Apple experimenting with many futuristic interfaces, but not developing them further after realising they didn't work from a UI pov. I suspect they may have similar feelings about AI. Many 'clever' systems which aim to predict what the user wants can be frustrating in practice. People prefer tool-like UIs where they can learn the mapping between their actions and the software's behaviour. Personally, I think there should be more focus on AI augmenting human capabilities, and maybe Apple is working on something like that.
- cs702 10y agoThe way I see it, TensorFlow has already won, even if competing frameworks don't yet see it that way. New ideas and research breakthroughs will spread faster to the framework that has the most users and contributors, thereby attracting more users and contributors, in a feedback loop. Some of the other frameworks will continue to be used (for example, FaceBook's AI team is unlikely to back away from its commitment to Torch), but the TensorFlow ecosystem and its network effects appear to have reached 'escape velocity' in a winner-take-all space. New higher-level services and products will be built atop the framework most widely used by the most software developers, almost by definition. Expect a shakeout: https://medium.com/@mjhirn/tensorflow-wins-89b78b29aafb#.bzlyyk3ed https://medium.com/@mjhirn/tensorflow-wins-89b78b29aafb#.bzl... -- EDITS: Added the "new higher-level services and products" sentence.
- astazangasta 10y agoIt's been less than a year, eh. A bit early to declare it has "already won"? There's decades of research ahead of us.
- AJ007 10y agoHow many of those decades will be human driven?
- astazangasta 10y agoSince no one even in concept has figured out how to give an AI motivations or goals other than ones specified by humans: all of them.
- cmaury 10y agoWhile Google may be the leader in Machine Learning tools, they are way behind when it comes to Machine Learning services. They've only just now released a Natural Language Understanding service in beta, and it is more limited compared to other NLU services from Microsoft/others. While tooling is important, the market for low level tools is much smaller than for the services built using those tools. The vast majority of businesses who could benefit from Machine learning don't have the expertise to run RNNs using Tensor Flow, but do have engineers who can integrate API's that leverage trained classifiers.
- nevir 10y agoThe market for services is several orders of magnitude less than the market for products that make use of ML. It's probably ok if they don't focus to heavily on the services side Google makes heavy use of these tools internally to build a wide range of products (and enhance existing ones)
- cmaury 10y agoHow do you see those two as being different? In my mind, products that make use of ML are using ML services as the back end. An example would be the recent wave of bot companies: many are not rolling their own NLU system but rather leveraging services like wit.ai or Microsoft's LUIS.
- shakil 10y ago> they are way behind when it comes to Machine Learning services. Umm, there's five separate managed services within the Google ML family: Vision (GA), Translate (GA), Natural Language (Beta), Speech (Beta), and Cloud ML (Alpha) https://cloud.google.com/vision/ https://cloud.google.com/vision/ https://cloud.google.com/translate/ https://cloud.google.com/translate/ https://cloud.google.com/natural-language/ https://cloud.google.com/natural-language/ https://cloud.google.com/speech/ https://cloud.google.com/speech/ https://cloud.google.com/ml/ https://cloud.google.com/ml/
- AIMunchkin 10y agoGoogle has done a far better job of abstracting the underlying math of machine learning into vertical-oriented services like the CloudVision API and the Cloud Natural Language API. In contrast, the AWS Machine Learning service provides a 100% vendor-locked interface to logistic regression and that's it. You can't even import or export models. They just hired Alex Smola to do something about that. We'll see what comes of that.
- ISL 10y agoThe article makes the argument that, by opening up TensorFlow, other companies are at risk of having their TensorFlow-fluent employees poached by Google. Trouble is, if other companies are using the same tooling as Google, it's equally easy for the outside company to poach a Google employee. Opening up the code to the world is more likely to be a net-good act, propelling everyone forward. https://github.com/tensorflow/tensorflow/blob/master/LICENSE https://github.com/tensorflow/tensorflow/blob/master/LICENSE
- jessriedel 10y ago> it's equally easy for the outside company to poach a Google employee. The effect doesn't become symmetric until we reach an equilibrium where TensorFlow is ubiquitous. > Opening up the code to the world is more likely to be a net-good act, propelling everyone forward. These things aren't contradictory. It can be a net good for the world, and also a net good for Google, and also a net bad for Facebook.
- gcr 10y agoThe hard parts of these frameworks isn't the framework itself. The hard part is understanding what's going on; the same properties common to all frameworks. CNNs are hard to understand. Understanding why gradient descent improves the model is very hard to understand. And understanding the right architecture to solve a problem takes a significant chunk of your research career unless you're already an expert. But languages and frameworks are simple. If you know Caffe, Torch will be easy to understand. If you know Torch, Tensorflow will be easy to understand. If you know Tensorflow, the others will be easy to understand. It's quite possible to get up to speed with a different framework in a few weeks.
- guelo 10y agoIn the corporate world even a few weeks of ramp-up time can create lock-in and network effects. Most programming languages can be learned in a few weeks by an experienced programmer, but there are still huge network effects. Part of it is that a lot of times you just don't have a few weeks to learn. But the other part of it is that the more popular it gets the library and tooling ecosystem also starts growing exponentially, so then you have to add the learning time of the tooling and library to the switching cost.
- paulsutter 10y agoIt's interesting that none of the quotes from Torch and Theano folks indicate any shortcomings of Tensorflow, their sole counterargument is that using tensorflow might give Google more control.
- mappingbabeljc 10y agoHiya, I wrote the article. What might not be captured here is that this is above all a community of people all working on AI so it's not like anyone is really emotional and/or aggressive about this issue. It's more that it's always better for an ecosystem to have multiple dominant players who are all relatively equal. Torch & Theano people were saying it's non-optimal if TF becomes the majority framework, which is a fair point. Everyone I spoke to for this article said TF was engineered well, albeit a bit of a black box in certain aspects.
- garbage_stain 10y agoNumber of stars on Github is not a great metric of the health of a project. TensorFlow, in particular, has received a lot of attention, so a lot of people have starred it, but this does not measure the number of non-Google contributors or what would happen to the project if Google abandoned it. Note that I'm not saying TensorFlow is going to fail in the future; I'm saying that the measures used in this article are dubious at best. Does nobody remember Elefant, Alex Smola's machine learning library that had a lot of hype then suddenly died after he left NICTA? The machine learning world is littered with tons of dead projects, and many of these died without a lot of warning. I'm concerned that this effect is only going to get worse now that machine learning libraries are often company-controlled instead of academic.
- BooneJS 10y agoPoliticians quote the number of "Likes" they get on social media, too. Stars are no different. It's a measure of interest, where the error bars are quite significant.
- balls187 10y agoAgreed it's not a great metric, but it is a good proxy especially if you suplement it with another source, such as the number of HN front page articles discussing TensorFlow. Given those two signals, I imagine Tensorflow will continue to gain momentum, at least in the short term. However, like all SDKs (is that still a thing?), it can be easily be supplanted when something that is easier to use, fixes the flaws of, and is better documented comes out. This is doubly true given that the AI space is still (imo) in its infancy, wrt to developer tooling.
- mappingbabeljc 10y agoHi there, I wrote the article. We also looked at data on things like number of forks and contributors on GitHub as well as massive rise in questions on StackOverflow (thanks to Delip Rao's great post here http://deliprao.com/archives/168 http://deliprao.com/archives/168 and invaluable ecosystem analysis from Francois Chollet). These other things were cut for purposes of length and because we felt a mainstream reader (which is more Bloomberg's audience) would be able to understand the 'stars' figure most easily of all metrics. Obviously I agree with you that this measure isn't definitive, but we felt it was relatively easy to understand. And the rest of the reporting we did for this story bore out the idea that TF has gathered an unusual level of both enthusiasm and commitment in a short amount of time. Thanks for reading!
- skybrian 10y ago"If the company changes the software too much, then other companies that have adopted it will need to make a copy of the software and rewrite it to suit their own needs -- an expensive and time-consuming process known as forking." Seems like it's the opposite? Forking a project is technically easy. Writing something equivalent from scratch is much harder. (Of course there are marketing considerations - it's "just a fork" so it might not get much attention.)
- Smerity 10y agoSome people are noting that TensorFlow evens the playing field, allowing other companies to poach Google's employees. Whilst that is true to some light degree, there are amazingly large caveats - Google still have the major advantage when it comes to poaching. The other important note is that open sourcing TensorFlow is likely more important for attracting recent graduates in machine learning than it is for attracting employees of other companies. Most machine learning engineers and researchers get excited by two things: underlying hardware and underlying tooling. Google have enough GPUs to throw 50 K80s for a few weeks into the hands of a single researcher for a single paper[1]. That's impressive. It gets even more impressive considering Google are making custom made ASICs that are more efficient - both power and computation - than GPUs too. None of that hardware is open source and even if it was, I don't see anyone getting them into general production in the near future. Nervana[2] are the closest to custom hardware I know about yet still have many steps until production. TensorFlow is also just the beginnings of the tooling Google has internally. The original TensorFlow whitepaper talks about their internal distributed training stack. A tiny recreation of this has reached the wild but it misses many of the interesting optimizations. (1) Automated efficient allocation of tensor operations to components (does this op go on GPU? CPU? Across the network to another GPU machine?). (2) Minor optimizations, such as truncating 32 bit floats to 16, then re-expanding - they noted it in the paper but I've not seen external verification. (3) Google DeepMind moved to TensorFlow but given they do a lot of reinforcement learning, they likely have a modified TF internally that's better tailored for RL. (4) etc etc etc Given the majority of the contributors to TensorFlow are also from Google, that gives them the strongest say in the future of the framework. This is especially important given that Google would likely be trying to ensure their internal modifications and variations of TensorFlow remain in line with the open version. For that reason, I don't expect to see TensorFlow handed off to a third party caretaker any time soon. Releasing TensorFlow is a net win for Google, both in "making the world a better place" and "making Google a better place (for attracting talent)". [1]: https://arxiv.org/abs/1607.05691 https://arxiv.org/abs/1607.05691 [2]: https://www.nervanasys.com/ https://www.nervanasys.com/