5 ms·
Etched Is Making the Biggest Bet in AI
- deleted 2y ago[deleted]
- CommieBobDole 2y ago"In five years, AI models became smarter than humans on most standardized tests." This seems to be a novel definition of "smarter" - one could also argue that the printed answer key for a standardized test is smarter than most humans.
- ilaksh 2y agoWow.. I wonder how this compares to groq? I made a request to access their developer cloud. Anyone have any idea when they start processing those requests and how many slots they might have?
- samspenc 2y agoWas curious about this as well, seems like the closest comparison today is to Groq, wondering how it holds up.
- Havoc 2y agoThere are a couple now. Groq , sambanova and tenstorrent come to mind. The more the merrier. If we do need power plants worth of juice then it may as well be dedicated custom hardware to optimise this to the max
- ilaksh 2y agoThanks for telling me about Sambanova. Too bad they don't have an API. The speed is absolutely incredible. They actually have a demo where it just keeps re-running the chat completion as you are typing your query.
- seany62 2y agoA coworker of mine recently got access to Groq. Not sure how many slots there are but we have a pretty strict rate limit for what thats worth
- ilaksh 2y agoI have access to grow, was asking about sambanova
- candiddevmike 2y ago> Specialized chips are inevitable I mean, are they? Seems like the industry would prefer these things to become commodities, especially if it helps with portability and reproducibility.
- jononor 2y agoThe specialized chips may very well become commoditized over time. It is the way things go if the market is large and competitive enough - and that there is sufficient competitive advantage over the existing types of devices (GPUs in this case). For a rather recent example, see Bitcoin mining ASICs. Of course GPUs (and CPUs, and pre-transformer/LLM type NPU/TPUs) are also going to respond to better suit LLM workloads. So we may see a convergence in capabilities. What is for sure is that the LLM chip architecture is not yet settled. Currently available chips are mostly designed pre LLM craze, with slight adaptations. And have large areas of potential improvement, at least 10-100x (maybe 1000x) in power efficiency. And power efficiency is a key element (in addition to up from hardware investment). Which is the opportunity this startup (and others) have identified.
- osnium123 2y agoGiven that Moore’s law is slowing down, given that circuit design is getting more automated and given the rise of chiplet architecture, I expect that you will have more specialized chips.
- returnInfinity 2y agoHow will these companies replace the software? Don't they need something similar to CUDA?
- arjvik 2y ago(I work at Etched.) You need something as complex as CUDA only to support general-purpose programmability; Sohu is built for one thing and one thing only: transformers. So while we certainly need a software stack to harness the chip, it’s much easier to do so, and even easier then to adapt existing LLM serving tools (vLLM, etc.) to use this stack.
- samspenc 2y ago> Sohu is built for one thing and one thing only: transformers Thanks for clarifying this. Could you clarify whether your chip supports the transformer architecture in general, or only specific models for e.g. Llama 70B? In case of the latter, would your ASIC have to be reprogrammed for each model?
- arjvik 2y agoTransformers in general. There’s no reprogramming of the ASIC needed, just applying a different sequence of layers, and that’s exactly what our software stack is meant to support.
- jeffs4271 2y agoEtched has an interesting approach, I am not convinced in their public info because it is vague. Could be great or could be bullshit. Well certainly some marketing.