3 ms·

I was with Amazon but wasn't part of Alexa. I was working closely with the Alexa team however. I remember vividly the challenge of building centralized infra f

by yandie 1y ago

I was with Amazon but wasn't part of Alexa. I was working closely with the Alexa team however. I remember vividly the challenge of building centralized infra for ML at Amazon: we had to align with our organization's "success metrics" and while our central team got ping ponged around, and our goals had to constantly change. This was exhausting when you're trying to build infra to support scientists across multiple organizations and while your VP is saying the team isn't doing enough for his organization. Sadly our team got disbanded eventually since Amazon just can't justify funding a team to build infra for their ML.

didip 1y ago
Isn't this a challenge for any big tech companies. Success metrics tend to be attached to a particular product and yet... central infra is a necessary foundational block for everyone, but it is without a specific success metrics.
- yandie 1y ago
  We were working with folks from other central ML infra teams in other companies. Amazon is the one that doesn't have any funding for central ML infra (they have the central infra for software). Having interacted with many companies after leaving Amazon, they are just really bad in terms of investing in central ML.
  - foota 1y ago
    Is this unique to ML, or does Amazon have an issue funding central infra?
    - Jensson 1y ago
      Amazon did AWS, so they can do central infra. Possibly the massive success of AWS makes them expect the same level of self funding for other infra projects even if those would be better served as internal only with internal funding.
giancarlostoro 1y ago
> Amazon just can't justify funding a team to build infra for their ML. Sounds like they didn't plan it out correctly. It should have been done in phases, one team at a time, starting with the Alexa team, or the smallest team with the smallest amount of effort as a test bed, while keeping the other teams informed in case they have suggestions or feedback for when their turn comes along.
laidoffamazon 1y ago
For me the big problem was how hard Hoverboard was to use - they built nicer tooling around it eventually but getting onboarded took weeks to months, getting GPUs usable enough to do LLM training would be nigh-impossible, an _extremely rigorous_ dedication to customer security meant transferring any data into and out of it for analysis was a major pain.... I remember being in the office when GPT2 dropped and thinking the entire Alexa Engine/skill routing codebase became outmoded overnight. That didn't really happen, but now that MCP servers are so easy to build I'm surprised Alexa doesn't just use tool-calling (unless it does in Alexa+?)
artyom 1y ago
I was at Amazon but wasn't part of Alexa. I remember taking a look at their code. It was an endless spaghetti of if statements. The top one started like this if(tenant == "spotify") { ... and everything else was downhill from there. The rest of the description on how Amazon operates is quite accurate. Impossible for anyone to do anything meaningful anymore.

by yandie 1y ago

didip 1y ago

Isn't this a challenge for any big tech companies. Success metrics tend to be attached to a particular product and yet... central infra is a necessary foundational block for everyone, but it is without a specific success metrics.

yandie 1y ago
We were working with folks from other central ML infra teams in other companies. Amazon is the one that doesn't have any funding for central ML infra (they have the central infra for software). Having interacted with many companies after leaving Amazon, they are just really bad in terms of investing in central ML.
- foota 1y ago
  Is this unique to ML, or does Amazon have an issue funding central infra?
  - Jensson 1y ago
    Amazon did AWS, so they can do central infra. Possibly the massive success of AWS makes them expect the same level of self funding for other infra projects even if those would be better served as internal only with internal funding.

giancarlostoro 1y ago

> Amazon just can't justify funding a team to build infra for their ML. Sounds like they didn't plan it out correctly. It should have been done in phases, one team at a time, starting with the Alexa team, or the smallest team with the smallest amount of effort as a test bed, while keeping the other teams informed in case they have suggestions or feedback for when their turn comes along.

laidoffamazon 1y ago

For me the big problem was how hard Hoverboard was to use - they built nicer tooling around it eventually but getting onboarded took weeks to months, getting GPUs usable enough to do LLM training would be nigh-impossible, an _extremely rigorous_ dedication to customer security meant transferring any data into and out of it for analysis was a major pain.... I remember being in the office when GPT2 dropped and thinking the entire Alexa Engine/skill routing codebase became outmoded overnight. That didn't really happen, but now that MCP servers are so easy to build I'm surprised Alexa doesn't just use tool-calling (unless it does in Alexa+?)

artyom 1y ago

I was at Amazon but wasn't part of Alexa. I remember taking a look at their code. It was an endless spaghetti of if statements. The top one started like this if(tenant == "spotify") { ... and everything else was downhill from there. The rest of the description on how Amazon operates is quite accurate. Impossible for anyone to do anything meaningful anymore.