4 ms·

> But there is something dangerous and seductive about things that almost work This is what makes me extremely anxious about investing time in AI. I've seen th

> But there is something dangerous and seductive about things that almost work This is what makes me extremely anxious about investing time in AI. I've seen things that make me want to keep going, but I also see the dragons in the corners. Would 97% correct be good enough? I am not actually sure. Even with direct understanding of specifics for our business and use cases, I can't say for certain. Humans don't get it right 100% of the time either. I think the nuance is things like "Use AI (LLM) for summarization & classification, not direct English->SQL conversions". If you scope the system correctly, you can maybe push the failure modes into a "meh, not a big deal" bucket.

deleted 3y ago
[deleted]
PaulHoule 3y ago
It is the biggest fight in product development. A product that has 97% of the originally planned features might completely satisfy users. A sort algorithm that gets 97% of the items in the right place is useless. (A big part of the problem in Python packaging as that people have long tolerated dependency solving algorithms in pip that aren’t always right…. make a project complex enough and you go to a phase transition from ‘I can just add packages with my IDE’ to forever pushing a bubble around under the rug. ‘Just use Docker’ and now you have two problems.) Recommender systems, full text search and such have always been “subjective” in that everyone knows there is not really a right answer. There is a large toolbox of methods for deciding if one of these is better than another, if it is “good enough” and when I called up 20 or so major vendors of full text search systems (e.g. OpenText) I found these were rarely used in practice because they made sales on the basis of having 387 connectors to suck in data from every product and not on the basis of the results being any good. The trouble w/ summarization is it may all seem OK until it screws up bad enough to get passed around in screenshots, get in the newspaper, and cause somebody important in the organization to cross the chain of command to expedite doing ‘something” about it.

deleted 3y ago

[deleted]

PaulHoule 3y ago

It is the biggest fight in product development. A product that has 97% of the originally planned features might completely satisfy users. A sort algorithm that gets 97% of the items in the right place is useless. (A big part of the problem in Python packaging as that people have long tolerated dependency solving algorithms in pip that aren’t always right…. make a project complex enough and you go to a phase transition from ‘I can just add packages with my IDE’ to forever pushing a bubble around under the rug. ‘Just use Docker’ and now you have two problems.) Recommender systems, full text search and such have always been “subjective” in that everyone knows there is not really a right answer. There is a large toolbox of methods for deciding if one of these is better than another, if it is “good enough” and when I called up 20 or so major vendors of full text search systems (e.g. OpenText) I found these were rarely used in practice because they made sales on the basis of having 387 connectors to suck in data from every product and not on the basis of the results being any good. The trouble w/ summarization is it may all seem OK until it screws up bad enough to get passed around in screenshots, get in the newspaper, and cause somebody important in the organization to cross the chain of command to expedite doing ‘something” about it.