10 ms·
Groq, a Stealthy Startup Founded by Google’s TPU Team, Is Raising $60M
- lquist 8y agoIs this competitive to NVIDIA's chips? How worried should they be?
- writepub 8y agonot the least bit worried. It'snot about performance, it's reliability. When you're a big cloud provider, only a miracle can get you to use a chip from a no-name company that no one else has dog food-ed. Then there's the actual number being marketed, there's been no independent verification. Right now, this is a fancy P.R hit piece
- sytelus 8y agoNo, NVidia should be very worried. There is a huge uproar in the community with some of the practices NVidia has forced like you must use 3X expensive version of same GPUs in data center. Even consumer GPUs are in short supply. Most of the people are not using massive complex rendering pipelines that these GPUs have but they are paying in terms of price and wattage. There is a huge demand for consumer version of TPU like chips and the market is going to eat up any similarly performing alternatives. Lot of gain in NVidia's revenue comes from blockchain and deep learning segment. Much of these gain is at risk by chips like TPU or Groq. It is quite surprising that NVidia hasn't announced any competing product and I hope they don't get sleeping at the wheel while this big wave is about to hit the market.
- twtw 8y ago> Most of the people are not using massive complex rendering pipelines that these GPUs have but they are paying in terms of price and wattage. This is not how compute on GPUs works now, or since G80 was released in 2006. The "massive complex rendering pipeline" doesn't even light up.
- sanxiyn 8y agoNVIDIA should be worried, if true. That's a big if though.
- anonymous5133 8y agoDefinetly seems interesting.
- woodrowbarlow 8y agofounded by _former members_ (albeit founding members) of Google's TPU team. i wonder how they negotiated this from a legal standpoint. every employment contract i've ever signed certainly would not allow for starting a project this similar to my employer's core business.
- ttul 8y agoYou can’t - in California anyhow - really stop employees from quitting and then competing with you. Most non compete forms have been made unlawful, which is a really good thing for competitiveness. In any case, I don’t see any confirmation that this startup is pursuing something that would be competitive with Google. The whole stealth thing leaves us with little to go on.
- ryandrake 8y agoBut companies can (and Google does) prevent current employees from working on side projects—even on their own time and equipment. So I suppose the relevant question is: did these guys leave and then start with a completely blank slate? Presumably yes! Any IP ownership ambiguity that would arise from moonlighting would have been flushed out during funding due diligence.
- kornish 8y agoIs this true? Moonlighting is legal in CA [1]. [1]: https://danashultz.com/2016/05/31/moonlighting-employees-protected-california-labor-code/ https://danashultz.com/2016/05/31/moonlighting-employees-pro...
- ryandrake 8y agoThey can prevent moonlighting if it creates a conflict of interest, for example if the work is similar to what the employer does or could do. For large companies, the definition of that can amount to “every potential side project”.
- 8y ago
- scottlegrand2 8y agoIf it's FP32 or FP16/32, it's interesting. If it's INT8/32 it's incrementally better than a 2080TI GPU, and if it's INT4/32, it's stillborn.
- monocasa 8y agoIt depends. If the chips are for training, I agree with you. If they're for inference, I think the jury's still out.
- sanxiyn 8y agoIt's an inference chip.
- deepnotderp 8y agoThey say "TOPS" which usually refers to INT8
- writepub 8y agoHardware is hard. For Gorq to be successful - - The boards using their chips need to fit into commodity interfaces (PCIE? DIMM? ) in Open Compute hardware - Someone needs to buy hundreds of thousands of these to even minutely impact their bottom line - Multiple such big volume wins need to consistently happen. - Their IP needs to actually be defensible. Otherwise, an Intel/Samsung whose manufacturing prowess & channel reach is multiple times that of Gorq will undercut the pricing with almost the same performance/watt. Oh, and they'll happily play nice with Open Compute, other standards bodies. - Most of all, their product needs to work as advertised, at scale, in a reliable fashion. This is easier said than done in semiconductors, especially for the kind of performance gains they're marketing. If they'd gotten here with ~$30M capital, and demonstrated traction in the marketplace, I'd give them a chance, but expecting Google's pay while working at an independent chip startup, with $0 in revenues portends financial doom. It's not the founder's fault though - hardware is capital intensive, not compatible with agile development & an MVP just won't cut it - it needs to be fully functional & reliable right out of the bat. I sincerely hope I'm wrong, as I'd like to see the silicon put back in silicon valley - having been in the semiconductor industry for 20 years, it just seems unlikely.
- singularity2001 8y ago- The boards using their chips need to fit into commodity interfaces USB? There are USB GPU boxes too.
- writepub 8y agoUSB? USB 3.1 is 16X slower than PCIE and maybe 100X slower than DIMM. USB is not an OCP recommended peripherals interface, it's for consumer removable products. Then again, I'm not entirely sure about the requirement for a high speed interface in AI. I'm assuming a fast interface is needed as the amount of data ingested is typically large
- akhilcacharya 8y agoPeople use DIMM as a general purpose interface?
- syntaxing 8y agoI'm kind of curious how many electrical engineers and talent that can design ASICs are out there? Most electrical engineers that I have met that designed ASICs at one point or another were for mainly military use or in the Semi industry. But the team that designed the chips were always super small compared to the mechanical or electrical team.
- deepnotderp 8y agoIf you mean full custom, very few, and most are in i/o. If you mean synthesis, a lot.
- syntaxing 8y agoYeah it seems like full custom ASICs kind of died off a little when affordable robust MCUs came out. Kind of nice to see it picking up again!
- nobrains 8y agoAre they planning to create cryptocurrency miners?
- nicodjimenez 8y agoDelivering an incremental speed boost vs nvidia chips seems like a tough way to win. Even if they can be 2x faster, people will still stick to cuda. They either have to be 5-10x faster or find a new market. Definitely possible. Would be pretty embarrassing for nvidia if that happened.
- ur-whale 8y agoA large part of accelerating AI loads in H/w lives in the software stack, not in the chip itself. If people can run TensorFlow loads directly on the chip via qroq's s/w stack, who cares about Cuda. That will be the differentiator for this company, not their h/w manufacturing prowess.
- sanxiyn 8y agoApparently they have a person who was involved in TensorFlow TPU port.
- sytelus 8y agoNope. People aren't addicted to CUDA. In fact its hell to work with CUDA. Just to even download the binaries you have to go through registration process, docs are shit and version dependency is nightmare. The only reason CUDA is in use is because in old days it was the only game in town and Caffee framework integrated it from some very early code researchers wrote. Then people kept using that baseline code all the way to TF and PyTorch. Thanks to TPU, frameworks are already being to forced to be agnostic and new alternatives will be much easier to integrate. If Groq chip delivers what its promising then you can bet that it would be integrated within few months in most frameworks and people will soon forget about CUDA. Most people who work with deep learning neither write code specific to cuda nor do they care that cuda is being used under the hood as long as things are being massively parallelized.
- sanxiyn 8y ago> If Groq chip delivers what its promising then you can bet that it would be integrated within few months in most frameworks and people will soon forget about CUDA. Groq seems careful not to promise any price point. Even if Groq delivers every promise, if it's expensive its adoption will be chancy.
- Soundest 8y agoThis looks interesting. Whilst I agree with other commenters that it's hard to compete in hardware I think there's a good niche for this product. Google isn't going to start selling TPUs so something off-the-shelf for machine learning might make some real traction. Best case, lot's of sales to cloud providers (amazon, microsoft etc.) and lots of custom houses. Worst case would be acquisition by MS or Amazon. Having said that, it's certainly true that Nvidia are tough to beat. But right now we're in a bubble, VC will throw millions at companies and big corporations will throw billions at acquisitions. So I think it's probably a very profitably move in general.
- boulos 8y agoDisclosure: I work on Google Cloud. Local inference can be important and even a requirement, so at NEXT we announced our intent to start shipping our Edge TPUs: https://cloud.google.com/edge-tpu/ https://cloud.google.com/edge-tpu/
- perrohunter 8y agoI’m still waiting patiently for those to arrive so I can order a couple
- person_of_color 8y agoAre they hiring?
- jasondrowley 8y agoAuthor of the article here. Yes, they are apparently hiring, or so suggests some of my internet research. One of the founders' linkedin profiles said they're mostly building in Haskell.
- person_of_color 8y agoThanks. They must be using www.clash-lang.org/
- sanxiyn 8y agoIt's probably Bluespec instead.
- css 8y ago> a company with a very spartan website What does the word "spartan" mean in this context? "Serious?" "Utilitarian?" I do not know this usage of the word. Edit (from Webster): > 2 b often not capitalized: marked by simplicity, frugality, or avoidance of luxury and comfort. "A spartan room"
- jasondrowley 8y agoAuthor of the article here. If you look at the company's website, it is very, very plain.
- hyperpape 8y agoaustere
- sytelus 8y agoThe numbers on their website is quite stunning if they are true: - 16X more power efficient than TitanX - 3X more ops than TitanX - 25K images/sec vs 5K images/sec inference on nVidia I'm completely bewildered why NVidia hasn't came up with deep learning specific chips yet that doesn't have crud of massive rendering pipeline. https://blog.groq.com/2017/11/09/69/ https://blog.groq.com/2017/11/09/69/
- sanxiyn 8y agoNVIDIA already created such a chip: http://nvdla.org/ http://nvdla.org/
- HNNewer 8y agoI believe they got so much funding because they are coming from Google, not for the product itself. They could have sold even crap hardware.
- jmunjr 8y agoThis is troubling: https://seekingalpha.com/article/4206948-nvidias-inference-problem-alarming-sell-side-ai-iq https://seekingalpha.com/article/4206948-nvidias-inference-p...