13 ms·
A Year at Google Brain (2020)
- SoylentOrange 6y agoThis is incredibly well-written and a great peek behind the curtain of industrial vs academic ML research. One question: can you elaborate a little on your point about intellectual diversity?
- akshayka 6y agoSure! While Google Brain is focused specifically on AI, Stanford as a whole conducts research across a very wide range of disciplines. As a result, my network at Stanford has a more diverse collection of interests. As a PhD candidate I have worked with or am connected to people interested in neuroscience, radiation treatment planning, finance, control, economics, computer systems, and (yes) machine learning, among other topics. Of course there's nothing wrong with focusing specifically on AI. It's just that I wanted to develop my research sensibilities in a more diverse environment. edit: & thank you for the kind words!
- a-dub 6y agohow much real-world collaboration do you get to do with students, postdocs and faculty in those other disciplines? (ie; what is the experience like when one does a phd in computational tools as opposed to a specific stem domain?)
- akshayka 6y agoI think it depends on many things, such as whether people at your university are generally collaborative, and whether your advisor or you are well connected. Most of my time is spent on tools or fundamental research, since that's my primary interest; but sometimes people will reach out to me (or vice versa) about starting a collaboration. But even just exchanging notes and chatting with researchers in other disciplines has been fun. But that is just my experience. Some of my labmates had PhD's in which the majority of their work was collaborative, across disciplines.
- ramraj07 6y agoGreat post. An interesting aphorism I used to spout was “to never listen to a first year PhD student sing praise of their PhD experience.” Though that was aimed at commodity phd students in regular universities not at people who can just choose to work at Google Brain if they feel like it. But ignoring that one persons warning about PhD still sounded like he had the typical arrogance contained by every person who ever did a PhD (including me). If someone says “don’t do it” and your reaction is to not even follow up with questions perhaps you should acknowledge you already know why they’re saying that! It’s still not clear to me a person of this cadre need a PhD in a technical field in this day and age; they might contribute just as much faster if they chose any other route.
- akshayka 6y agoI'm now in my third year of my PhD and enjoying it more than ever, so there's that at least! In all seriousness, I totally acknowledge that the variance in satisfaction with a PhD is very large. I'm lucky to have a great advisor and incredible labmates; my happiness with the program is largely due to them.
- MobileVet 6y ago> I'm lucky to have a great advisor and incredible labmates; my happiness with the program is largely due to them. This. So much this. As in just about every profession, people make or break places. Your advisor and lab mates are everything... choose wisely, and choose them vs a school for its name.
- kkylin 6y agoAs others have noted, it can be harder to move around academia than in industry. Your immediate community matters a lot more, whether you're a PhD student, a postdoc, or faculty. When things work it's great; when they don't, it's correspondingly worse.
- Waterluvian 6y ago>aimed at commodity phd students in regular universities Woof.
- arduinomancer 6y ago> While I’m interested in machine learning, I’m not convinced that today’s AI is anywhere near as profound as electricity or fire This seems a not too uncommon experience among CS people where machine learning loses its interest once they understand the magic
- jimbokun 6y agoOnce you have a handle on the deep learning tools, it seems like it’s more like configuration and less like programming.
- iujjkfjdkkdkf 6y agoI'm a big ML proponent, both because of work and personal interest. Saying it is like electricity is idiotic, and only makes false expectations about ML worse. But I would not say that makes it uninteresting. Nor does it make it irrelevant from a business perspective. There are still lots of beneficial ways to use ML, and lots of different angles from which to derive intellectual satisfaction. I suppose if someone went into the field expecting I Robot type stuff, they might be disappointed, but it's hard to imagine someone with a CS background having such unrealistic expectations of what ML actually is.
- nkozyra 6y ago> This seems a not too uncommon experience among CS people where machine learning loses its interest once they understand the magic Maybe. I went into my program with no preconceived ideas that there was 'magic' happening here. What I got out of it was a real curiosity about how humans think and how that may not be a magical as we think either. Long-term I cannot see AI being anything but the paradigm-smashing artifact it's sold as. But progress is (relatively) slow on a human scale, so we find incremental progress ultimately lackluster. If you take it through that lens it becomes more reasonable and you find yourself less skeptical that, just as we have no flying cars, we have no AGI or that all work has been replaced.
- jonas21 6y agoIn 1850, few people felt that electricity was particularly profound either. That year, William Gladstone, Chancellor of the Exchequer, asked Michael Faraday why electricity was valuable. Faraday replied, "One day sir, you may tax it." [1]. The electric motor had been invented (by Faraday), but it couldn't do any useful work and had to sit in a pool of mercury to function. Electric lighting had been invented, but the filaments were dim and burned out too quickly to be practical. There was no way to generate or distribute electricity at scale. To most people, electricity looked like a toy (and a terrifying toy at that). It would take decades of invention and refinement to get to the point where electricity was useful, reliable, and ubiquitous. But the seeds for doing so were there in 1850. So when Sundar Pichai says AI is more profound than electricity, it's possible he's right. It's just that we're still in its 1850, so we can't see it yet. [1] https://en.wikipedia.org/wiki/Electricity#Cultural_perception https://en.wikipedia.org/wiki/Electricity#Cultural_perceptio...
- akshayka 6y agoI didn't expect to see this here! Happy to answer questions. (There's also a discussion on Twitter: https://twitter.com/akshaykagrawal/status/1368276916587794433 https://twitter.com/akshaykagrawal/status/136827691658779443...)
- texasbigdata 6y agoRead this on the weekend, and found a interesting branch of mathematics from your links. Thanks so much!
- sytelus 6y agoI am sure you need to stay politically correct but do you see places where Google Brain can do better or improve further?
- supersereneblue 6y agoI have nothing of value to add except that seeing someone do what they love and enjoy, greatly warms my heart. Good luck with everything Akshay!!
- rejectedandsad 6y agoReally? If anything it makes me really angry. I generally despise seeing people happy, especially people that have gotten everything in life (as someone that has definitely not). What an interesting way to view the world...
- exdsq 6y agoI always get so excited when reading about Brain, X, Parc, etc... But sadly I just don’t have the academic background to get into one of those places :(
- ajayka 6y agoIncredible post and very useful!
- LolWolf 6y agoAgreed! It’s awesome :)
- deleted 6y ago[deleted]
- vincentmarle 6y ago> I’m not convinced that today’s AI is anywhere near as profound as electricity or fire @akshayka can you elaborate a bit more on why you think that
- benrbray 6y agoThe answer is simple: electricity is a prerequisite to artificial intelligence, and fire was necessary to provide us with the nutrition we needed to harness the power of electricity.
- akshayka 6y agoModern ML certainly has some cool applications, but none seem revolutionary. Can you list any that are?
- snejad123 6y agoOff the top of my head: When paired with a robotic medium, we'll be able to automate most pattern driven jobs like construction, welding, surgery, anything really that doesn't require human element to it and is more of a pattern matching behavior. This is extremely revolutionary and will put a ton of people out of work. Any pattern-dependent job that does not require human interaction will be at risk, even coding (as we've seen by GPT-3's ability to generate code). Lawyers, sales people, cops, these jobs will be ok, but there will be less demand for humongous medical teams, software teams, etc. Bc all the entry-level work can be handled by AI, and just need some QA from a few engineers.
- ubercore 6y agoThis is a wildly optimistic take on what AI can accomplish, and a misunderstanding of the difficulties in all those tasks. If AI can do the first set of jobs, it will be able to replace sales people, cops, and lawyers too. But there's nothing, yet, to indicate that there isn't a wide gulf between the current state of the art and the kind of generalized intelligence you're talking about.
- meetups323 6y ago> For example, my coworkers let me take the lead in writing an academic paper about TensorFlow 2, even though my contributions to the technology were smaller than theirs. Sounds like some folks didn't like writing! :)
- TeMPOraL 6y agoThe relevant question isn't who wrote the bulk of a paper - it's whose name appears first on the list of authors :).
- EvgeniyZh 6y agoThis guy is [1], if I found the right paper. Though the order appears to be alphabetical. [1] https://arxiv.org/abs/1903.01855 https://arxiv.org/abs/1903.01855
- pertymcpert 6y agoIt's not alphabetical.
- EvgeniyZh 6y agoAkshay A..., Akshay N..., Ale..., All..., Ash..., Asi..., I..., J..., M..., R..., S... Looks alphabetical for me
- meetups323 6y agoIt literally says "Authors listed in alphabetical order." right in the paper. I just don't get some HN assertions...
- blt 6y ago> I suspect that convex optimization has the potential to become a powerful, widely-used technology. I would say it has achieved that status already :) Deep learning is getting more than its share of hype. Convex optimization is a jewel of computational science.
- JCDenton2052 6y agoOf particular note: "Google’s CEO Sundar Pichai (who believes AI is “more profound than electricity or fire”) has emphasized that Google is an “AI-first” company, with the company seeking to implement machine learning in nearly everything do" Nearly everything they do with your data.
- inakarmacoma 6y agoNot sure if a part of your post got cut off? As framed, it seems rather inflammatory without much direction or practical addition to the conversation. Perhaps I'm missing something.
- JCDenton2052 6y agoNope, nothing got cut. My post stands as I wrote it. It is not ad-hominem and not what-aboutery. It is very much a reasonable question to ask what they use their much-touted AI on and how this relates to the well documented privacy concerns around Google.
- eitland 6y agoThe original is actually more correct it seems. IMO they are clearly betting (parts of) the farm on this as can be seen by the (often) ridiculous results many of us have witnessed in both search results quality and ad matching quality.
- Yahskapar 6y agoFantastic blog post! Thanks for sharing. I always find fairly positive accounts of graduate school to be quite interesting. Without a doubt, you can always find someone in the comments (if it's something posted in a format such as this) with a chip on their shoulder from their own graduate school experience (or lack thereof). Also I find it interesting to think about places similar in impact to Xerox PARC, which people usually admire after everything unfolds and becomes a part of history. A good chunk of people from today would probably avoid a place like PARC back then in order to opt for a more stable or known job.
- dougb5 6y ago> [Sundar Pichai] has emphasized that Google is an “AI-first” company Maybe I need to lighten up here, but I find this "AI-first" branding to be really off-putting. What I fear it means is "We're going to add the world's biases and a bit more unpredictability into every single one of our features." And in some ways it's like like a carpenter deciding to become a "wrench-first" operation. Shouldn't the first priority be the people affected by the things that you do, rather than a particular tool that you inflict upon them? And, great as it can be, is it really the right tool for every job? Maybe some of the problems the AI fairness team has exposed (or tried to) will push Google to realize that it need not be present in everything, let alone first.
- rurounijones 6y ago> I find this "AI-first" branding to be really off-putting Honestly my first reaction was a snarky "Well, your moderation and support channels certainly are" given all the horror stories we hear about people getting locked out of their entire google accounts with no recourse or access to humans to appeal / investigate.
- haskellandchill 6y agoI kind of love how google has destroyed their reputation on the customer support side and doesn't seem to care. I believe at a certain point it will affect their bottom line. They must be betting that they develop passable AI support before that happens.
- pocketgrok 6y agoDid Google ever have a good reputation for customer support?
- killtimeatwork 6y agoIt reminds me of my managers talking about "focusing on moving to the cloud", without any real analysis whether it makes sense from technical or cost standpoint. It's purely hype driven.
- seaman1921 6y agoThank you for the writeup and don't be put off by all the snarky comments here :)
- petercerno 6y agoI once did Convex Optimization course with Stephen Boyd and it was totally amazing! I think it is still available here: https://www.edx.org/course/convex-optimization https://www.edx.org/course/convex-optimization and the e-book https://web.stanford.edu/~boyd/cvxbook/ https://web.stanford.edu/~boyd/cvxbook/ is also freely available. I am quite jealous. :) Good luck with your PhD!
- wilperkins 6y agoThis course was great
- aroundtown 6y agoI'm always a little skeptical of someone who gets all their degrees from the same university, even it it is Stanford. They aren't exposed to as diverse of a group of thoughts and ideas as someone who goes else where and proves themselves to other people with differing opinions.