11 ms·
Apple to Start Publishing AI Research
- deleted 10y ago[deleted]
- deepnotderp 10y agoYay! I never thought I'd ever see this though.
- deleted 10y ago[deleted]
- amelius 10y agoOf course! What researcher would want to work for a company that prohibits scientific publications? I've always been amazed by their attitude. Look at Microsoft Research, and their enormous scientific output over the years. IBM and Google look bleak by comparison, and Apple is not even on the chart.
- deelowe 10y agoNot sure how this stacks up against MS, but Google's research is posted here: https://research.google.com/pubs/papers.html https://research.google.com/pubs/papers.html
- adenadel 10y agoFrom that page, it looks like Google has published 4359 papers. From this page, https://www.microsoft.com/en-us/research/about/ https://www.microsoft.com/en-us/research/about/ Microsoft claims to have published over 10000 papers.
- deelowe 10y agoAnd of course, Microsoft has been in business 41 years where as Google has been 18. I'm guessing publication rates between the two are comparable. I wouldn't be surprised if IBM were similar. All three are highly committed to research.
- Cyph0n 10y agoRegardless, comparing the number of papers is meaningless. A better metric is comparing how many significant advances resulted from research funded by the company. Using the second metric, the winner in all cases - without negotiation - would be AT&T Bell Labs.
- abritinthebay 10y agoTrue, tho Xerox has to be up there as well.
- serge2k 10y agoNah, the transistor is more groundbreaking than anything xerox did.
- aylons 10y agoThe transistor is more groundbreaking than most developments in the 20th century, combined.
- sangnoir 10y agoMore groundbreaking than other developments singly - probabably; combined? not a chance. Other 20th century developments include flight, rocketry, the United Nations, quantum physics, splitting the atom, the M.A.D. principle, assembly line production, robotics, analog computers (including those used for AA targeting systems in WWII, predating the transistor), among many, many others.
- Cyph0n 10y ago
- etrain 10y agoMicrosoft is 23 years older than Google and it usually takes time for a corporation to create an R&D lab. Given Google's rich academic roots this happened faster there than usual. Google is doing a good job given that it's a relatively young company. I hope their research output continues to increase in quality and volume. If you go by publication count alone, I suspect that IBM Research is still near the top. They have a large research organization that made some fundamental contributions in computer science in everything from speech recognition to databases.
- serge2k 10y agoMicrosoft research was only formed in 1991, according to wiki.
- lettergram 10y agoI'd also assume Google hires MANY more PhD's than Microsoft did and does. Could be wrong though.
- deleted 10y ago[deleted]
- JamilD 10y agoIs numbers of papers published a good metric for determining the contribution of a company to public research?
- Cyph0n 10y agoIf you consider the number of papers published over the lifetime of IBM, and the fact that IBM Research does work in all levels of computing (materials science -> software), Microsoft's research output looks minuscule in comparison. But if you're considering only CS research, then you might be right. Here is a nice snapshot of the major contributions that came out of IBM Research over the past 60 years: http://www.research.ibm.com/featured/history/ http://www.research.ibm.com/featured/history/. Edit: the invention of copper interconnects is enough for me to be frank (http://www-03.ibm.com/ibm/history/ibm100/us/en/icons/copperchip/ http://www-03.ibm.com/ibm/history/ibm100/us/en/icons/copperc...). On a similar note, Intel is another company that is very active in research and has published a significant amount of peer-reviewed publications. Samsung Research is yet another; they have an amazing presence at circuit design conferences, for instance.
- AndrewKemendo 10y agoIntel is also 13 years older, has roots in research and basically started with HUGE projects like the Apollo missions. Class of their own IMO.
- dekhn 10y agointel started with huge projects like missles. The Air Force was their largest and most demanding customer.
- j1vms 10y agoAnd all of these, in terms of quality of research output, pale by about an order of magnitude in comparison to Bell Labs for much of its existence (particularly in the era ~1940 to mid 1980s). They were the R&D benchmark for most improvements to IBM Research from the 1960s onward. Additionally IBM sought out its alumni and populated their ranks with notable management and research minds from the Bell Labs stable.
- Cyph0n 10y agoYou could probably bundle all of the achievements of the aforementioned companies into a single decade of research output from Bell Labs. It's not a fair comparison at all.
- KKKKkkkk1 10y agoAccording to Craig Federighi, senior VP of software engineering, only losers who want to pad their resumes at the expense of their company care about publishing: “Our practices tend to reinforce a natural selection bias — those who are interested in working as a team to deliver a great product versus those whose primary motivation is publishing,” says Federighi. https://backchannel.com/an-exclusive-look-at-how-ai-and-machine-learning-work-at-apple-8dbfb131932b#1c65 https://backchannel.com/an-exclusive-look-at-how-ai-and-mach...
- ced 10y agoWas it worth it for Microsoft?
- hiddencost 10y ago# of papers published is a poor metric (google's published like 40% as many papers, over a shorter period?) MSR has done cool work and important work, but google's publications have been often been major paradigm shifts. In a lot of ways they're currently way out ahead of everyone else.
- paulddraper 10y agoMicrosoft created Haskell. Google created Go.
- Chinjut 10y agoMicrosoft didn't create Haskell, though they did ultimately hire many of the movers and shakers who did. (I agree that Go is a horrific mediocrity in comparison.)
- paulddraper 10y agoYou're correct. Microsoft has been Haskell's unofficial benefactor (maybe like Google for Python?)
- hiddencost 10y agoGo isn't even a product of Google research. We're in a thread talking about AI research. I'm comparing Google's ML work to Microsofts.
- deleted 10y ago[deleted]
- pjmlp 10y agoReally? MSR was one of the first companies bringing FP into mainstream developers tools with LINQ and F#. Investing into dependent type programming via F*. Making JavaScript scale via TypeScript (adopted by Google teams instead of Dart). Sponsors Haskell and OCaml research. Researches OS design that aren't yet another UNIX clone, namely memory safe OSes with Singularity, Midori, Drawbrige, theorem provers for device driver validation, P language, micro-kernels.
- KKKKkkkk1 10y agoUnfortunately, my prior reply here was flagged, but I'd like to try again. Here's a quote from Craig Federighi, senior VP of software engineering, less than half a year ago: “Our practices tend to reinforce a natural selection bias — those who are interested in working as a team to deliver a great product versus those whose primary motivation is publishing,” says Federighi. https://backchannel.com/an-exclusive-look-at-how-ai-and-machine-learning-work-at-apple-8dbfb131932b#1c65 https://backchannel.com/an-exclusive-look-at-how-ai-and-mach... In my opinion, this gives you a view of the thinking that goes inside Apple. Publishing papers correlates negatively with being a team player and delivering great products, and here Apple is on record as saying that they do not want that sort of people working there.
- leonroy 10y ago> those who...deliver a great product versus those whose primary motivation is publishing Interesting quote. Federighi seems a tad condescending - in that producing great products and producing publications are mutually exclusive. Whilst Apple might expect their engineers to toil away anonymously under NDA without much recognition outside the organization, the world of academic research does not work like that at all. I guess proof of the pudding is in the eating and in my experience with Apple's AI efforts they are far behind their competitors. I for one am glad Apple are going to start publishing research - it should attract talent, foster sharing and ultimately result in better AI products.
- coldtea 10y ago>Of course! What researcher would want to work for a company that prohibits scientific publications? Depends on the compensation and several other factors. Lots of military/intelligence related research for example is not published either...
- bitmapbrother 10y ago>Look at Microsoft Research, and their enormous scientific output over the years. IBM and Google look bleak by comparison, and Apple is not even on the chart. Do you have any metrics to back up this dubious claim? I find it very hard to believe Microsoft is even in the same ballpark as IBM considering all of the research centers IBM has throughout the world and the number of years they've been in business.
- deleted 10y ago[deleted]
- ktamiola 10y agoand what would it be? ... What have Apple developed that goes beyond currently available peer-reviewed work, done by Deep Mind?
- serge2k 10y ago> Amazon.com Inc.’s Alexa So Alexa is really the biggest thing in the market so far right? How many papers does Amazon publish?
- SEJeff 10y agoNot sure how many papers they publish, but I do really use this paper[1] from James Hamilton (literally the Architect of AWS) as the defacto way of building large distributed services: [1] http://perspectives.mvdirona.com/2014/07/challenges-in-designing-at-scale-formal-methods-in-building-robust-distributed-systems/ http://perspectives.mvdirona.com/2014/07/challenges-in-desig...
- echosmith 10y agoAmazon's Alexa group has published 10 papers in the last 2 years. https://www.amazon.jobs/en-gb/landing_pages/alexa-machine-learning https://www.amazon.jobs/en-gb/landing_pages/alexa-machine-le... Dunno about the rest of the company.
- iverjo 10y agoJokingly, this was the first apple AI research I saw online, about a month ago: https://arxiv.org/abs/1610.08120v1 https://arxiv.org/abs/1610.08120v1
- melling 10y agoStory broke 6 hours ago: http://www.businessinsider.com/apple-is-finally-going-to-start-publishing-its-artificial-intelligence-research-2016-12?r=UK&IR=T http://www.businessinsider.com/apple-is-finally-going-to-sta... Mac site picks it up 5 hours ago and it gets reported on HN: https://9to5mac.com/2016/12/06/apple-ai-researchers-can-publish/ https://9to5mac.com/2016/12/06/apple-ai-researchers-can-publ... Bloomberg writes the same story an hour ago and it finally gets huge traction on HN. My guess is that many people ignore the "new" page and it's all a matter of luck that 3 or 4 people get it to the front page where a story takes off.
- JamilD 10y agoBusinessInsider and Mac blogs tend to be less reputable sources than Bloomberg, whose articles regularly make it to the HN front page. Might have contributed to this particular submission's success.
- zackkatz 10y agoFirst paper: "Snarky Reply and Joke Generation by a Virtual Assistant Using Deep Learning"
- deleted 10y ago[deleted]
- themodelplumber 10y agoDidn't MS already try that on Twitter and it turned out to be a racist meme generator?
- mtgx 10y agoMy hope is that Apple will continue research on privacy-protecting and privacy-enhancing machine learning, because out of all the big tech companies using machine learning, they may just be the only ones to do that. Some random research for privacy technologies may come out of Google, too, but they are much less likely to actually use them at scale, especially if they conflict with ad revenue.
- numbas 10y agoActually Microsoft has been publishing a lot on privacy enhancing Machine Learning too. For instance: https://www.microsoft.com/en-us/research/publication/cryptonets-applying-neural-networks-to-encrypted-data-with-high-throughput-and-accuracy/ https://www.microsoft.com/en-us/research/publication/crypton... https://www.microsoft.com/en-us/research/publication/private-computation-on-encrypted-genomic-data/ https://www.microsoft.com/en-us/research/publication/private... https://youtu.be/xsaXMUelOEA https://youtu.be/xsaXMUelOEA As for Apple: seeing is believing.
- euyyn 10y agoAnd Google started publishing on differential privacy in 2014, way before Apple used WWDC's marketing to paint themselves as the only ones using it.
- givinguflac 10y agoGenuinely curious, where does google actually use differential privacy?
- delroth 10y agoChrome telemetry is one example I know of: https://www.chromium.org/developers/design-documents/rappor https://www.chromium.org/developers/design-documents/rappor
- deleted 10y ago[deleted]
- maciejgryka 10y agoI'm just at NIPS and Apple does indeed have a decent presence, which is a good sign (even if there are no actual publications yet). Super interesting to see how this will play out compared to their past. There's lots of criticism that we can throw at big tech companies, but the fact they many of them are so open about their research and thus are forcing others to do the same is pretty cool.
- hota_mazi 10y ago> Researchers say among the reasons Apple has failed to keep pace is its unwillingness to allow its AI engineers to publish scientific papers, stymieing its ability to feed off wider advances in the field. I don't follow. How would preventing employees from writing papers would stop them from reading papers?
- jedberg 10y agoIf the could publish their research other researchers would take their work in a new direction that they didn't think of, which they could then bring back. Basically they can't provide that seed insight that motivates others.
- leereeves 10y agoI think they mean that Apple isn't offering researchers the incentives they desire, and may have trouble recruiting them.
- freyir 10y agoYes. Researchers like to publish their research because their publication record the metric they're evaluated on by the outside world. If you can't publish your research, that will significantly reduce your ability to get hired by another top research group, industrial or academic, in the future.
- argonaut 10y agoWould you rather work for a company that has their engineers actively contribute to open source, or a company where all of your code is private?
- sytelus 10y agoModern AI research heavily depends on building work of others as well as having others build on your work. It's this virtuous cycle that is bringing all the advances we are seeing. Let's say someone at Apple had invented recurrent networks but they never published it. That would mean they would miss out on literally hundreds of researchers working on this, enhancing techniques in very critical ways and writing thousands of papers over the period of just few years that would feed in to this cycle. Many of these papers would contribute absolutely important enhancements to productionize things. Much of the AI research currently starts with some seed idea like GAN which looks cool in small experiments but have tons of blind spots that needs to iron out. Unlike typical product efforts where you can probably work your way out through brute force engineering, advances in most AI related areas requires massive amount of collaboration, mathematical acrobatics, trial-and-error and cross-pollination across fields consuming multi-person-years before yielding fruits. In theory, Apple can still keep any silver bullets they find in the field secret but in practice such silver bullets are rare and advances are very incremental spanning over many years and many people.
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- deepnotderp 10y agoJust goes to show you the market power of ml researchers, even apple has to bend the knee.
- pinouchon 10y agoI completely agree. But still I find it surprising that apple managed to hire Salakhutdinov. He is a huge name in Deep Learning (with a focus on math, unsupervised learning and autoencoders)
- ejz 10y agoI have an advance copy of their paper. Available at this link: [redacted]
- ticotico 10y agoTook a while! As somebody who made the transition from academia to industry this is a huge deal for me. I would not want to work for a company that does not allow scientific publications. Moreover, I think a closed policy would shun off a lot people I admire and would want to work with.
- home_boi 10y agoThe free market wins. All the AI/ML engineers were going to FB/Google/etc. (and on some rare occasions M$)
- _1 10y ago> and on some rare occasions M$ Are we still doing that?
- k_lander 10y agoSo glad that this is happening. It is really inspiring to see the knowledge sharing spirit become the expected default in the community. This is only going to be great for progress in the field!
- Animats 10y agoHas Apple published anything yet?
- brudgers 10y agoConference slides have been tweeted.
- turingbook 10y agoActually, Apple is one of sponsors of this year's NIPS conference: https://nips.cc/Conferences/2016/Sponsors https://nips.cc/Conferences/2016/Sponsors
- dovdovdov 10y agoBased on my Siri experience, I imagine a chimp pulling strings in a control room. :)