7 ms·

Tenstorrent and the State of AI Hardware Startups

224 points by zdw 2y ago

alkh 2y ago
Kudos to the Tenstorrent team for being so open to having a discussion. I wish more companies would be like that, as constructive criticism is very useful, especially for a startup
Mathnerd314 2y ago
I almost forgot about that ARM-Qualcomm dispute, the fireworks are only a few days away.
latchkey 2y ago
I have a business renting high performance compute. I want to democratize compute to make it more easily available on short term basis to anyone who wants access. I talked to TT about a year ago; also one of their customers as well. I was really impressed with everyone I talked to and I'd love to work with them. What I realized though is that as much as I'd like to buy and host it and make it available, I'm not sure the economics or interest are there yet. The focus today is so dedicated to Nvidia that "fringe" gear is still just that. People who really want to play with it, will just buy it themselves. They've probably been doing that with hardware for a long time. So, it is a bit of a catch-22 for me right now. Hopefully interest grows in these sorts of offerings and demand will pick up. The world needs alternatives to just Nvidia dominance of all of AI hardware and software.
- choppaface 2y ago
  Can you sell or re-sell colo space to a handful of customers who might put TT or other “weird” hardware there? Doesn’t scale, but it hedges your own business. Requires the right customer though, like somebody who might buy Nervana / Xeon Phi but then buy NVidia from you when it blows up.
  - latchkey 2y ago
    Kind of. It is easier for us to be the capex/opex (meaning buy and run equipment) and then rent it out. I have the backing to make large investments without too much drama. We can do that with any hardware as long as we have longer (1-2 year) contracts in place. We have the space/power available in a top tier data center (Switch). https://hotaisle.xyz/cluster/ https://hotaisle.xyz/cluster/
- kouteiheika 2y ago
  > Hopefully interest grows in these sorts of offerings and demand will pick up. Well, looking at their (as far as I can see highest end) accelerator the n300s we get: - 24GB of memory - 576GB/s of memory bandwidth - $1400 As a hobbyist this is still not compelling enough to get excited and port my software - same amount of memory as a 4090/3090, half the bandwidth as a 4090/3090, slightly cheaper (than a 4090), more expensive (than a 3090), much worse software support. Why would I buy it over NVidia? This might be more compelling to bigger fish customers who would buy thousands of these (so then the lower price makes a difference), but you really need small fish people to use it too if you want to achieve good, widespread software support. However if they'd at least double the amount of memory at the same price, now we'd be talking...
  - A_D_E_P_T 2y ago
    Yeah, clearly the low-hanging fruit is simply to offer more memory. In a recent thread, just about everybody was talking about how Intel should have released its "Battlemage" line of cards with 48GB+ and how they'd actually have a very compelling offering on their hands had they done so. https://news.ycombinator.com/item?id=42308590 https://news.ycombinator.com/item?id=42308590 Missed opportunities. That said, I know some of the Tenstorrent guys and they're extremely smart and capable. They'll figure it out, and the company is probably going to 10x.
    - kouteiheika 2y ago
      > Yeah, clearly the low-hanging fruit is simply to offer more memory. Yep. And just to be clear, this isn't necessarily a strategy to directly make money, as the market for people running these things locally is probably not very big (as a lot of people in the Intel thread said). But it's a strategy to beat the CUDA monopoly and get everyone and their dog to support your hardware with their software. I know I would be porting my software to their hardware if it was actually compelling for small fish, and I know plenty of other people who would too. AMD also somewhat falls into this trap. They're slightly cheaper than NVidia, and their hardware is roughly good enough, but because their software stack and their hardware support sucks (just look at their ROCm compatibility list, it's a complete joke with, like, 3 models of GPUs, vs NVidia where I can use any GPU from the last 8 years) no one bothers. Being good enough is not good enough, you need to offer some sort of a unique selling point for people to tolerate the extra software issues they'll have with your hardware.
- steeve 2y ago
  Look at what we (ZML) are doing: https://github.com/zml/zml https://github.com/zml/zml Any model, any hardware, zero compromise. TT support is on the roadmap and we talk to them.
  - thangngoc89 2y ago
    Do you have any plan to allow conversion from Pytorch to your format?
C-programmer 2y ago
> I genuinely believe Groq is a fraud. There is no way their private inference cloud has positive gross margins. > Llama 3.1 405B can currently replace junior engineers I'd like more exposition on these claims.
- refulgentis 2y ago
  Today, I wrote a full YouTube subtitle downloader in Dart. 52 minutes from starting to google anything about it, to full implementation and tests, custom formatting any of the 5 obscure formats it could be in to my exact whims. Full coverage of any validation errors via mock network responses. I then wrote a web AudioWorklet for playing PCM in 3 minutes, which complied to the same interface as my Mac/iOS/Android versions, ex. Setting sample rate, feedback callback, etc. I have no idea what an AudioWorklet is. Two days ago, I stubbed out my implementation of OpenAI's web socket based realtime API, 1400 LOC over 2 days, mostly by hand while grokking and testing the API. In 32 minutes, I had a brand spanking new batch of code, clean, event-based architecture, 86% test coverage. 1.8 KLOC with tests. In all of these cases, most I needed to do was drop in code files and say, nope wrong a couple times to sonnet, and say "why are you violating my service contract and only providing an example solution" to o1. Not llama 3.1 405B specifically, I haven't gone to the trouble of running it, but things turned some sort of significant corner over the last 3 months, between o1 and Sonnet 3.5. Mistakes are rare. Believable 405B is on that scale, IIRC it went punch for punch with the original 3.5 Sonnet. But I find it hard to believe a Google L3, and third of L4s, (read: new hires, or survived 3 years) are that productive and sending code out for review at a 1/5th of that volume, much less on demand. So insane-sounding? Yes. Out there? Probably, I work for myself now. I don't have to have a complex negotiation with my boss on what I can use and how. And I only saw this starting ~2 weeks ago, with full o1 release. Wrong? Shill? Dilletante? No. I'm still digesting it myself. But it's real.
  - nightski 2y ago
    Most software is not one off little utilities/scripts, greenfield small projects, etc... That's where LLMs excel, when you don't have much context and can regurgitate solutions. It's less to do with junior/senior/etc.. and more to do with the types of problems you are tackling.
benreesman 2y ago
It would be pretty awesome if AMD sold aggressively into the data center market, or if NVIDIA sold aggressively into the supercompute market. All discussion on this is so much sports team fandom until Jensen and Lisa stop pillow fighting on the biggest pile of money since that 50 Cent cover and the highest margins since the Dutch East India Company. I just spent a few days getting FlashAttention and TransformerEngine built with bleeding edge drivers and all the wheels to zero typing install on three generations of NVIDIA and shit (PS you’re build advertises working without ninja but fucks up CMake INSTALL paths): this is the unassailable awesomeness that no one can compete with? ptxas spinning a core for longer than it takes to build Linux and sometimes crashing in the process? No, this is central committee grift sanctioned at the highest levels to the applause of HN folks long NVDA.
- benreesman 2y ago
  And if you really think about who is getting insulted here? It's the NVIDIA and AMD driver and toolchain hackers. I read diffs from those folks: they are polymath ballers. The average HN comment including this one has more bugs in it than a patch from an NVIDIA/AMD driver author. But no, they're supposedly just too incompetent to run PyTorch or whatever. Fuck that, the hackers kick ass. This shit is broken because shareholders get more rent when it's broken.
  - lhl 2y ago
    Having spent a fair amount of time over the past few years as someone working on top of both Nvidia and AMD toolchains (and having spent decades working up and down the stack from mcus, mobile device, to webscale stacks), I have an alternative take. I'm smart and experienced enough, and the people working on the toolchains are capable and even smarter, but these systems are just extremely hard/complex. The hardware is buggy, the firmware is buggy, the drivers are buggy, every layer above it (compilers, kernels, multiple layers of frameworks) also all buggy. Add on top of that everyone is implementing as fast as they can, every part is changing constantly, and also a good chunk of the code being used is being written by researchers and grad students. Oh, and the hardware changes dramatically every couple years. Sure all tech is swiss cheese, but this is a particularly unstable/treacherous version of that atm. BTW, shareholders obviously have no idea whether what's being sold works well or not, but there's no financial incentive (in terms of selling stuff/profiting) if your product doesn't even work/work well enough to compete. And if it were so easy, you would see any number of hardware startups take over from the incumbents. Sure there are network effects, but if I could run PyTorch (or really, just train and inference my models easily) faster and cheaper, me, and I assume lots of other people would switch in a heartbeat. The fact that no one (besides Google arguably if you count having to switch all your implementations over for TPUs) has been able to do it, whether they be startups or multi-billion dollar companies (including Amazon, Meta, and Microsoft, all who have direct financial incentive to and have been trying), points to the problem being elsewhere.
lanza 2y ago
> Llama 3.1 405B can currently replace junior engineers lol
- FridgeSeal 2y ago
  Plain grift, or are they high on their own supply?
  - datadrivenangel 2y ago
    Both? Both is good.
- ponector 2y ago
  Can LLM join a standup call? Can LLM create a merge request? At the moment it looks like an experienced engineer can pressure LLM to hallucinate a junior level code.
  - Philpax 2y ago
    The argument is that, instead of hiring a junior engineer, a senior engineer can simply produce enough output to match what the junior would have produced and then some. Of course, that means you won't be able to train them up, at least for now. That being said, even if they "only" reach the level of your average software developer, they're already going to have pretty catastrophic effects on the industry. As for automated fixes, there are agents that _can_ do that, like Devin (https://devin.ai/ https://devin.ai/), but it's still early days and bug-prone. Check back in a year or so.
    - newsclues 2y ago
      Not training new workers and relying on senior engineers with tools is short sighted and foolish. LLMs seem to be accelerating the trend
      - baobabKoodaa 2y ago
ksec 2y ago
>Given that ARM is aggressively raising license prices and royalty rates and has (metaphorically) nuked Qualcomm, RISC-V CPU IP clearly has a bright future. It is interesting the world ( or may be RISC-V world ) seems to think any company can simply break the contract and agreement others signed, while pretending they are still righteous. At the same time thinking this wont happen to their IP. Well I think we will all know soon.
- klelatti 2y ago
  It is informative to note the contrast between the fact that 22 companies (Microsoft, Google, Samsung, etc, etc) were all given enough notice in advance of the Nuvia acquisition, to be able to come up with a quote endorsing it. [1] And the treatment of the 'partner' upon whose IP the deal depends: > Neither Qualcomm nor Nuvia provided prior notice of this transaction to Arm [2] [1] https://www.qualcomm.com/news/releases/2021/01/qualcomm-acquire-nuvia https://www.qualcomm.com/news/releases/2021/01/qualcomm-acqu... [2] From Arm's initial court submission.
  - brookst 2y ago
    I read that part differently. I didn’t see it as a statement about the legal merits of ARM’s choices so much as the pragmatic business outcomes. It is possible for an action to be 100% legally correct and a bad business move (see: creative IP companies that sue their biggest fans). Even granting that ARM is completely within their right, this is a pretty bug reminder of the dangers of building on sole-source licensed IP when there’s an open alternative. I have little doubt that ARM drove some number of startups into choosing RISC-V. Which may or may not matter in the long run. But it is rolling the dice and by definition a short term maximization strategy.
    - klelatti 2y ago
      My comment wasn’t really around legal or business merits but rather the bad faith nature of these actions which would apply whether using open or closed source ip. However, the ‘dangers of building on sole-source’ licensed IP is true in general but a bit overdone in this case. Qualcomm had a large number of options using Arm IP including developing their own cores in house. However they chose to do something that was - on the face of it - in breach of license agreements that Nuvia had signed.
tails4e 2y ago
One concern Ive heard is that Llama can be a great accelerator for senior engineers who know what they want. However for junior engineers it could hold back their learning as they just use whatever it gives without really understanding what or why something was done that way. It seems plausible to me. At least using stack overflow people get explanations of why you do x to get y, it's a bit copy paste, sure, but the pace brought learning. Now if you type a comment and get code, a junior engineer may not even read what's generated as 'theu don't need to, it usually works' .... Any ideas how we can ensure junior engineers do effectively learn and understand what is going on, while still getting the LLM benefit?
- ripped_britches 2y ago
  I’ve been coding for a long time and just became very proficient in dart this year just by using LLM and asking it to explain anything I don’t know. I haven’t been on Stack Overflow in 2 years and it’s not because I’m taking shortcuts by letting the LLM write my code, it’s because it is the best teacher / paired programmer that will just sit with you with endless patience. Especially using @docs with cursor and @directory context.
  - zkry 2y ago
    > I’ve been coding for a long time... I think having been coding for a long time, I don't think you fall into the same category. Dart having paradigms not too different from other standard languages, a lot of these skills are probably transferable. I've seen beginners on the other hand using LLMs who couldn't even write a proper for-loop without AI assistance. They lack the fundamental ability to "run code in their head." This type of person I feel would be utterly limited by the capabilities of the AI model and would fail in lockstep with it.
    - brookst 2y ago
      This is kind of the classic “kids these days” argument: that because we understand something from what we consider the foundational level up to what we consider the goal level, anyone who comes along later and starts at higher levels will be limited / less capable / etc. It is true, but also irrelevant. Just like most programmers today do not need to understand CPU architecture or even assembly language, programmers in the future will not need to understand for loops the way we do. They will get good at writing LLM-optimized specifications that produce the desired results. And that will be fine, even if we old-timers bemoan that they don’t really program the way we do. Yes, the abstractions required will be inefficient. And we will always be able to say that when those kids solve our kinds of problems, our methods are better. Just like assembly programmers can look at simple python programs and be astounded at the complexity “required” to solve simple problems.
zozbot234 2y ago
It's not really "AI hardware", it's just HPC on a slightly smaller scale than the traditional sort. And it's still going to be useful for plenty of workloads regardless of the current AI frenzy.
- latchkey 2y ago
  Like what workloads? Genuinely curious what you'd run on it.
Animats 2y ago
> Tenstorrent has an implicate view that the future of AI is mixed-workload, not pure linear algebra spam. Yes, MATMUL go BRRRRRR is valuable, but CPU workloads will be needed in the future. That is the hope. So far, this has not played out. At least not in the training end of machine learning, which is the big money sink. What striking about this whole business is that it's still mostly hammering on a decades-old idea with more and more hardware and data. That worked great for a few years, but now seems to be reaching its limits. What's next?
arisAlexis 2y ago
beware of the "expert" rating other companies

224 points by zdw 2y ago

alkh 2y ago

Kudos to the Tenstorrent team for being so open to having a discussion. I wish more companies would be like that, as constructive criticism is very useful, especially for a startup

Mathnerd314 2y ago

I almost forgot about that ARM-Qualcomm dispute, the fireworks are only a few days away.

latchkey 2y ago

I have a business renting high performance compute. I want to democratize compute to make it more easily available on short term basis to anyone who wants access. I talked to TT about a year ago; also one of their customers as well. I was really impressed with everyone I talked to and I'd love to work with them. What I realized though is that as much as I'd like to buy and host it and make it available, I'm not sure the economics or interest are there yet. The focus today is so dedicated to Nvidia that "fringe" gear is still just that. People who really want to play with it, will just buy it themselves. They've probably been doing that with hardware for a long time. So, it is a bit of a catch-22 for me right now. Hopefully interest grows in these sorts of offerings and demand will pick up. The world needs alternatives to just Nvidia dominance of all of AI hardware and software.

choppaface 2y ago
Can you sell or re-sell colo space to a handful of customers who might put TT or other “weird” hardware there? Doesn’t scale, but it hedges your own business. Requires the right customer though, like somebody who might buy Nervana / Xeon Phi but then buy NVidia from you when it blows up.
- latchkey 2y ago
  Kind of. It is easier for us to be the capex/opex (meaning buy and run equipment) and then rent it out. I have the backing to make large investments without too much drama. We can do that with any hardware as long as we have longer (1-2 year) contracts in place. We have the space/power available in a top tier data center (Switch). https://hotaisle.xyz/cluster/ https://hotaisle.xyz/cluster/
kouteiheika 2y ago
> Hopefully interest grows in these sorts of offerings and demand will pick up. Well, looking at their (as far as I can see highest end) accelerator the n300s we get: - 24GB of memory - 576GB/s of memory bandwidth - $1400 As a hobbyist this is still not compelling enough to get excited and port my software - same amount of memory as a 4090/3090, half the bandwidth as a 4090/3090, slightly cheaper (than a 4090), more expensive (than a 3090), much worse software support. Why would I buy it over NVidia? This might be more compelling to bigger fish customers who would buy thousands of these (so then the lower price makes a difference), but you really need small fish people to use it too if you want to achieve good, widespread software support. However if they'd at least double the amount of memory at the same price, now we'd be talking...
- A_D_E_P_T 2y ago
  Yeah, clearly the low-hanging fruit is simply to offer more memory. In a recent thread, just about everybody was talking about how Intel should have released its "Battlemage" line of cards with 48GB+ and how they'd actually have a very compelling offering on their hands had they done so. https://news.ycombinator.com/item?id=42308590 https://news.ycombinator.com/item?id=42308590 Missed opportunities. That said, I know some of the Tenstorrent guys and they're extremely smart and capable. They'll figure it out, and the company is probably going to 10x.
  - kouteiheika 2y ago
    > Yeah, clearly the low-hanging fruit is simply to offer more memory. Yep. And just to be clear, this isn't necessarily a strategy to directly make money, as the market for people running these things locally is probably not very big (as a lot of people in the Intel thread said). But it's a strategy to beat the CUDA monopoly and get everyone and their dog to support your hardware with their software. I know I would be porting my software to their hardware if it was actually compelling for small fish, and I know plenty of other people who would too. AMD also somewhat falls into this trap. They're slightly cheaper than NVidia, and their hardware is roughly good enough, but because their software stack and their hardware support sucks (just look at their ROCm compatibility list, it's a complete joke with, like, 3 models of GPUs, vs NVidia where I can use any GPU from the last 8 years) no one bothers. Being good enough is not good enough, you need to offer some sort of a unique selling point for people to tolerate the extra software issues they'll have with your hardware.
steeve 2y ago
Look at what we (ZML) are doing: https://github.com/zml/zml https://github.com/zml/zml Any model, any hardware, zero compromise. TT support is on the roadmap and we talk to them.
- thangngoc89 2y ago
  Do you have any plan to allow conversion from Pytorch to your format?

C-programmer 2y ago

> I genuinely believe Groq is a fraud. There is no way their private inference cloud has positive gross margins. > Llama 3.1 405B can currently replace junior engineers I'd like more exposition on these claims.

refulgentis 2y ago
Today, I wrote a full YouTube subtitle downloader in Dart. 52 minutes from starting to google anything about it, to full implementation and tests, custom formatting any of the 5 obscure formats it could be in to my exact whims. Full coverage of any validation errors via mock network responses. I then wrote a web AudioWorklet for playing PCM in 3 minutes, which complied to the same interface as my Mac/iOS/Android versions, ex. Setting sample rate, feedback callback, etc. I have no idea what an AudioWorklet is. Two days ago, I stubbed out my implementation of OpenAI's web socket based realtime API, 1400 LOC over 2 days, mostly by hand while grokking and testing the API. In 32 minutes, I had a brand spanking new batch of code, clean, event-based architecture, 86% test coverage. 1.8 KLOC with tests. In all of these cases, most I needed to do was drop in code files and say, nope wrong a couple times to sonnet, and say "why are you violating my service contract and only providing an example solution" to o1. Not llama 3.1 405B specifically, I haven't gone to the trouble of running it, but things turned some sort of significant corner over the last 3 months, between o1 and Sonnet 3.5. Mistakes are rare. Believable 405B is on that scale, IIRC it went punch for punch with the original 3.5 Sonnet. But I find it hard to believe a Google L3, and third of L4s, (read: new hires, or survived 3 years) are that productive and sending code out for review at a 1/5th of that volume, much less on demand. So insane-sounding? Yes. Out there? Probably, I work for myself now. I don't have to have a complex negotiation with my boss on what I can use and how. And I only saw this starting ~2 weeks ago, with full o1 release. Wrong? Shill? Dilletante? No. I'm still digesting it myself. But it's real.
- nightski 2y ago
  Most software is not one off little utilities/scripts, greenfield small projects, etc... That's where LLMs excel, when you don't have much context and can regurgitate solutions. It's less to do with junior/senior/etc.. and more to do with the types of problems you are tackling.

benreesman 2y ago

It would be pretty awesome if AMD sold aggressively into the data center market, or if NVIDIA sold aggressively into the supercompute market. All discussion on this is so much sports team fandom until Jensen and Lisa stop pillow fighting on the biggest pile of money since that 50 Cent cover and the highest margins since the Dutch East India Company. I just spent a few days getting FlashAttention and TransformerEngine built with bleeding edge drivers and all the wheels to zero typing install on three generations of NVIDIA and shit (PS you’re build advertises working without ninja but fucks up CMake INSTALL paths): this is the unassailable awesomeness that no one can compete with? ptxas spinning a core for longer than it takes to build Linux and sometimes crashing in the process? No, this is central committee grift sanctioned at the highest levels to the applause of HN folks long NVDA.

benreesman 2y ago
And if you really think about who is getting insulted here? It's the NVIDIA and AMD driver and toolchain hackers. I read diffs from those folks: they are polymath ballers. The average HN comment including this one has more bugs in it than a patch from an NVIDIA/AMD driver author. But no, they're supposedly just too incompetent to run PyTorch or whatever. Fuck that, the hackers kick ass. This shit is broken because shareholders get more rent when it's broken.
- lhl 2y ago
  Having spent a fair amount of time over the past few years as someone working on top of both Nvidia and AMD toolchains (and having spent decades working up and down the stack from mcus, mobile device, to webscale stacks), I have an alternative take. I'm smart and experienced enough, and the people working on the toolchains are capable and even smarter, but these systems are just extremely hard/complex. The hardware is buggy, the firmware is buggy, the drivers are buggy, every layer above it (compilers, kernels, multiple layers of frameworks) also all buggy. Add on top of that everyone is implementing as fast as they can, every part is changing constantly, and also a good chunk of the code being used is being written by researchers and grad students. Oh, and the hardware changes dramatically every couple years. Sure all tech is swiss cheese, but this is a particularly unstable/treacherous version of that atm. BTW, shareholders obviously have no idea whether what's being sold works well or not, but there's no financial incentive (in terms of selling stuff/profiting) if your product doesn't even work/work well enough to compete. And if it were so easy, you would see any number of hardware startups take over from the incumbents. Sure there are network effects, but if I could run PyTorch (or really, just train and inference my models easily) faster and cheaper, me, and I assume lots of other people would switch in a heartbeat. The fact that no one (besides Google arguably if you count having to switch all your implementations over for TPUs) has been able to do it, whether they be startups or multi-billion dollar companies (including Amazon, Meta, and Microsoft, all who have direct financial incentive to and have been trying), points to the problem being elsewhere.

lanza 2y ago

> Llama 3.1 405B can currently replace junior engineers lol

FridgeSeal 2y ago
Plain grift, or are they high on their own supply?
- datadrivenangel 2y ago
  Both? Both is good.
ponector 2y ago
Can LLM join a standup call? Can LLM create a merge request? At the moment it looks like an experienced engineer can pressure LLM to hallucinate a junior level code.
- Philpax 2y ago
  The argument is that, instead of hiring a junior engineer, a senior engineer can simply produce enough output to match what the junior would have produced and then some. Of course, that means you won't be able to train them up, at least for now. That being said, even if they "only" reach the level of your average software developer, they're already going to have pretty catastrophic effects on the industry. As for automated fixes, there are agents that _can_ do that, like Devin (https://devin.ai/ https://devin.ai/), but it's still early days and bug-prone. Check back in a year or so.
  - newsclues 2y ago
    Not training new workers and relying on senior engineers with tools is short sighted and foolish. LLMs seem to be accelerating the trend
    - baobabKoodaa 2y ago

ksec 2y ago

>Given that ARM is aggressively raising license prices and royalty rates and has (metaphorically) nuked Qualcomm, RISC-V CPU IP clearly has a bright future. It is interesting the world ( or may be RISC-V world ) seems to think any company can simply break the contract and agreement others signed, while pretending they are still righteous. At the same time thinking this wont happen to their IP. Well I think we will all know soon.

klelatti 2y ago
It is informative to note the contrast between the fact that 22 companies (Microsoft, Google, Samsung, etc, etc) were all given enough notice in advance of the Nuvia acquisition, to be able to come up with a quote endorsing it. [1] And the treatment of the 'partner' upon whose IP the deal depends: > Neither Qualcomm nor Nuvia provided prior notice of this transaction to Arm [2] [1] https://www.qualcomm.com/news/releases/2021/01/qualcomm-acquire-nuvia https://www.qualcomm.com/news/releases/2021/01/qualcomm-acqu... [2] From Arm's initial court submission.
- brookst 2y ago
  I read that part differently. I didn’t see it as a statement about the legal merits of ARM’s choices so much as the pragmatic business outcomes. It is possible for an action to be 100% legally correct and a bad business move (see: creative IP companies that sue their biggest fans). Even granting that ARM is completely within their right, this is a pretty bug reminder of the dangers of building on sole-source licensed IP when there’s an open alternative. I have little doubt that ARM drove some number of startups into choosing RISC-V. Which may or may not matter in the long run. But it is rolling the dice and by definition a short term maximization strategy.
  - klelatti 2y ago
    My comment wasn’t really around legal or business merits but rather the bad faith nature of these actions which would apply whether using open or closed source ip. However, the ‘dangers of building on sole-source’ licensed IP is true in general but a bit overdone in this case. Qualcomm had a large number of options using Arm IP including developing their own cores in house. However they chose to do something that was - on the face of it - in breach of license agreements that Nuvia had signed.

tails4e 2y ago

One concern Ive heard is that Llama can be a great accelerator for senior engineers who know what they want. However for junior engineers it could hold back their learning as they just use whatever it gives without really understanding what or why something was done that way. It seems plausible to me. At least using stack overflow people get explanations of why you do x to get y, it's a bit copy paste, sure, but the pace brought learning. Now if you type a comment and get code, a junior engineer may not even read what's generated as 'theu don't need to, it usually works' .... Any ideas how we can ensure junior engineers do effectively learn and understand what is going on, while still getting the LLM benefit?

ripped_britches 2y ago
I’ve been coding for a long time and just became very proficient in dart this year just by using LLM and asking it to explain anything I don’t know. I haven’t been on Stack Overflow in 2 years and it’s not because I’m taking shortcuts by letting the LLM write my code, it’s because it is the best teacher / paired programmer that will just sit with you with endless patience. Especially using @docs with cursor and @directory context.
- zkry 2y ago
  > I’ve been coding for a long time... I think having been coding for a long time, I don't think you fall into the same category. Dart having paradigms not too different from other standard languages, a lot of these skills are probably transferable. I've seen beginners on the other hand using LLMs who couldn't even write a proper for-loop without AI assistance. They lack the fundamental ability to "run code in their head." This type of person I feel would be utterly limited by the capabilities of the AI model and would fail in lockstep with it.
  - brookst 2y ago
    This is kind of the classic “kids these days” argument: that because we understand something from what we consider the foundational level up to what we consider the goal level, anyone who comes along later and starts at higher levels will be limited / less capable / etc. It is true, but also irrelevant. Just like most programmers today do not need to understand CPU architecture or even assembly language, programmers in the future will not need to understand for loops the way we do. They will get good at writing LLM-optimized specifications that produce the desired results. And that will be fine, even if we old-timers bemoan that they don’t really program the way we do. Yes, the abstractions required will be inefficient. And we will always be able to say that when those kids solve our kinds of problems, our methods are better. Just like assembly programmers can look at simple python programs and be astounded at the complexity “required” to solve simple problems.

zozbot234 2y ago

It's not really "AI hardware", it's just HPC on a slightly smaller scale than the traditional sort. And it's still going to be useful for plenty of workloads regardless of the current AI frenzy.

latchkey 2y ago
Like what workloads? Genuinely curious what you'd run on it.

Animats 2y ago

> Tenstorrent has an implicate view that the future of AI is mixed-workload, not pure linear algebra spam. Yes, MATMUL go BRRRRRR is valuable, but CPU workloads will be needed in the future. That is the hope. So far, this has not played out. At least not in the training end of machine learning, which is the big money sink. What striking about this whole business is that it's still mostly hammering on a decades-old idea with more and more hardware and data. That worked great for a few years, but now seems to be reaching its limits. What's next?

arisAlexis 2y ago

beware of the "expert" rating other companies