23 ms·
Implementing search correctly would solve the problem too, without requiring a square kilometer of GPUs.
by regularjack 3y ago
Implementing search correctly would solve the problem too, without requiring a square kilometer of GPUs.
- Spivak 3y agoRight but implementing search correctly often means humans combing through the data and applying common sense to tag the products with their actual facets. So really you're just choosing between a square kilometer of office space for cheap labor or GPUs.
- cratermoon 3y ago> you're just choosing between a square kilometer of office space for cheap labor or GPUs. No, with LLMs you're getting both, because a. LLMs need the human-curated data to ingest and b. after training the model is going to get RLHF. Also, a square kilometer of office space still uses much less electricity than a square kilometer of GPUs.
- pixl97 3y agoI'm guessing we're at or very near the point where LLMs could tag the data better and cheaper than humans.
- lukevp 3y agoIf you have >100k items, “implementing search correctly” is an intractable problem if you don’t have a hundred person team of search relevance optimizers and taggers. Search is terrible until you gather tons of metadata about each item. Search is not nearly as simple as text matching within product titles. An LLM by comparison is peanuts. You could run it overnight and just have it build tags and metadata for all your items and it would probably be 75% accurate and a hell of a lot cheaper. Things can then be fine tuned by people later.
- lightbendover 3y agoAt Amazon scale that last x% is a multi-decade problem and still going. It’s intractable full stop.