4 ms·
> why can't it read a PDF document and transform it into a machine-readable format? It can definitely do that, but you might not like the cost/benefit analysi
by syllogism 6y ago
> why can't it read a PDF document and transform it into a machine-readable format?
It can definitely do that, but you might not like the cost/benefit analysis, depending on how many such documents you want to process. The costs are coming down steadily though as the tech improves. If you need to do millions of such documents, yeah a model will probably be worth it. But if you need to do a few hundred you probably should just do them manually.
The thing is, reality has surprisingly high resolution. When you give out a task like this to a person, they will likely come back to you for clarification about how you want to deal with some of the examples. Your initial requirements will be underspecified, or incorrect, in some details. When you are dealing with a person, these minor adjustments are pretty inconsequential, and so you don't really notice it happening. The worker might also have enough context to guess what you want and not ask, and just tell you the summary when they deliver the work.
If you're training a model, you need to work through all these annoying details about what you want, just as you would when you're creating any other sort of program. This adds some overhead, and places a lower bound on how many examples you'll need to have annotated -- you'll always need enough examples of annotation to actually specify your requirements, including various corner-cases. You need enough contact with the data to realise which of your initial expectations about the task were wrong.
So there will always be a lower scaling limit, where the automation isn't worthwhile for some small volume of work. The threshold is getting lower but there will always be a trade-off.
- dwighttk 6y ago> The thing is, reality has surprisingly high resolution. When you give out a task like this to a person, they will likely come back to you for clarification about how you want to deal with some of the examples. Why isn’t this generally solved, though?
- syllogism 6y agoWhat could that even look like? In the extreme case this is like asking, "Why do I have to write programs, can't the computer just do what I tell it?". Writing the program is telling it what you want. In supervised machine learning, annotating the data is, instead. If you tell me, "Go to ebay and get me a list of the prices of washing machines", that sounds simple, but then I'm faced with some washer/dryer combo or some hand-crank contraption. These are things you didn't think of. I can either ask you, or take a guess and hand you something that needs to be cleaned up later. If I'm instead training a model, I need to encounter these tricky examples in order to ask you for a policy on them. If I'm collecting an unbiased sample of the training data, this could take a very long time. If there's some sampling strategy maybe it's faster, but there's still a minimum number of examples we need to think about, no matter what.
- cmendel 6y agoBecause people aren't necessarily great at clearly communicating their needs.