2 ms·
Chat works because humans are really impressed by natural language responses, irregardless of the actual correctness/quality. Once you build it into a product
by rafaelmn 2y ago
Chat works because humans are really impressed by natural language responses, irregardless of the actual correctness/quality.
Once you build it into a product the failure modes become obvious.
- kazinator 2y ago'irregardless' of correctness, LOL. Best thing I read all day.
- rafaelmn 2y agoIs it double negative ? English not a first language so maybe doesn't translate exactly. But kind of proves my point - if that was grammatically correct bullshit you're way more impressed than getting an awkwardly structured reply. When you actually build stuff with AI into products (I've been a part of several integrations), failure modes and reliability become obviously lacking. Models failing to respond to a simple RAG question with relevant context a significant percentage of time, meanwhile they are solving PHD problems on some benchmark's. Then you find out they have to sample 10s of times to get the right answer in scenarios where evaluation is simple, or include tests in training data and then suddenly the lack of product integration makes sense.