3 ms·

> No. Sign up and look at the current missions. A lot of what they want transcribed is totally straightforward to OCR --- not even LLM, OCR. Whatever's going on

> No. Sign up and look at the current missions. A lot of what they want transcribed is totally straightforward to OCR --- not even LLM, OCR. Whatever's going on, and I'm not second-guessing them, a pretty big chunk of their problem appears to be well within the state of the art. If it's that easy, then do it and be the hero they want. Or maybe, just maybe, "a pretty big chunk of their problem appears to be well within the state of the art" is a sweeping generalization lacking understanding of the difficulties involved.

tptacek 2y ago
Go ahead and find something hard, and relate back the steps you took to find it.
- AdieuToLogic 2y ago
  > Go ahead and find something hard, and relate back the steps you took to find it. This is a strawman[0] argument. You proclaimed: A lot of what they want transcribed is totally straightforward to OCR And I replied: If it's that easy, then do it and be the hero they want. So do it or do not. Nowhere does my finding "something hard" have any relevance to your proclamation. 0 - https://en.wikipedia.org/wiki/Straw_man https://en.wikipedia.org/wiki/Straw_man
  - tptacek 2y ago
    I did in fact do it, and what I got was much, much easier than the samples in the article, which 4o did fine with. I'm sorry, but I declare the burden of proof here to be switched. Can you find a hard one? (I don't think you need to Wikipedia-cite "straw man" on HN).
    - AdieuToLogic 2y ago
      > I did in fact do it, and what I got was much, much easier than the samples in the article, which 4o did fine with. Awesome. Can you guarantee its results are completely accurate every time, with every document, and need no human review? > I'm sorry, but I declare the burden of proof here to be switched. If you are referencing my stating: If it's that easy, then do it and be the hero they want. Then I don't really know how to respond. Otherwise, if you are referencing my statement: > Perhaps "random humans" can perform tasks which could reshape your belief: >> OCR is VERY good To which I again ask, can you guarantee the correctness of OCR results will exceed what "random humans" can generally provide? What about "non-random motivated humans"? My point is that automated approaches to tasks such as what the National Archives have outlined here almost always require human review/approval, as accuracy is paramount. > (I don't think you need to Wikipedia-cite "straw man" on HN). I do so for two purposes. First, if I misuse a cited term someone here will quickly correct me. Second, there is always a probability of someone new here which is unaware of the cited term(s).
      - Dylan16807 2y ago
        > If you are referencing my stating: > > If it's that easy, then do it and be the hero they want. > Then I don't really know how to respond. If someone says a thing is easy, and you respond by demanding they do it a million times to prove that it's easy, you are the one that has screwed up the burden of proof.
      - tptacek 2y ago
        Can I ask, did you sign up and look at what they're actually looking for? Show of good faith: can you give 3 of the headers for the top-level "missions" they have for transcriptions?
        AdieuToLogic 2y ago
        I have failed to express myself clearly and for that I apologize. What I was trying to convey is that no matter what tooling is employed for this specific problem, a human will have to review the result to ensure accuracy. Since the problem is defined as reading cursive handwriting, people who can do so intrinsically obviate the need for tooling. Whether or not automation can produce reasonable results is moot when the requirement is to have as accurate as possible a one-time transcription of cursive handwriting documents.
  - Dylan16807 2y ago
    There are two claims. The main one is that all of these documents are easy to individually transcribe by machine. The other is that a whole lot can be OCR'd, which is pretty simple to check. That's not a claim that processing the entire archive would be trivial. And even if it was, whether that would make someone the "hero they want" is part of what's being called into question. So your silly demand going unmet proves nothing. Also, "give me an example please" is not a strawman! If you actually want to prove something, you need to show at least one document in the set that a human can do but not a machine, or to really make a good point you need to show that a non-neglibile fraction fit that description.
    - AdieuToLogic 2y ago
      > So your silly demand going unmet proves nothing. I made demands of no one. > Also, "give me an example please" is not a strawman! My identification of the strawman was that it referenced "find something hard" when I had said "be the hero they want" and that what is needed in this specific problem domain may be more difficult than what a generalization addresses. > If you actually want to prove something, you need to show at least one document in the set that a human can do but not a machine, or to really make a good point you need to show that a non-neglibile fraction fit that description. Maybe this is the proof you demand. LLM's are statistical prediction algorithms. As such, they are nondeterministic and, therefore, provide no guarantees as to the correctness of their output. The National Archives have specific artifacts requiring precise textual data extraction. Use of nondeterministic tools known to produce provably incorrect results eliminate their applicability in this workflow due to all of their output requiring human review. This is an unnecessary step and can be eliminated by the human reading the original text themself. Does that satisfy your demand?
      - Dylan16807 2y ago
        > I made demands of no one. Whatever you want to call "If it's that easy, then do it" > LLM's [...] Does that satisfy your demand? That's a different argument from the one above where you were trying to contradict tptacek. And that argument is flawed itself. In particular, humans don't have guarantees either. > provably incorrect results This gets back to the actual request from earlier, which is showing an example where the machine performs below some human standard. Just pointing out that LLMs make mistakes is not enough proof of incorrectness in this specific use case.
  - idlewords 2y ago
    Please don't link to wikipedia definitions of elementary terms. It's condescending and obnoxious.
    - AdieuToLogic 2y ago
      > Please don't link to wikipedia definitions of elementary terms. It's condescending and obnoxious. I respect what you have chosen to contribute to this conversation, but neither need nor seek your approval of mine.
      - tptacek 2y ago
        For the record, it mostly comes across as you only recently learning what a straw man argument is, which, in the context of a message board argument, is a rough thing to have to admit, so you have my sympathies.

tptacek 2y ago

Go ahead and find something hard, and relate back the steps you took to find it.

AdieuToLogic 2y ago
> Go ahead and find something hard, and relate back the steps you took to find it. This is a strawman[0] argument. You proclaimed: A lot of what they want transcribed is totally straightforward to OCR And I replied: If it's that easy, then do it and be the hero they want. So do it or do not. Nowhere does my finding "something hard" have any relevance to your proclamation. 0 - https://en.wikipedia.org/wiki/Straw_man https://en.wikipedia.org/wiki/Straw_man
- tptacek 2y ago
  I did in fact do it, and what I got was much, much easier than the samples in the article, which 4o did fine with. I'm sorry, but I declare the burden of proof here to be switched. Can you find a hard one? (I don't think you need to Wikipedia-cite "straw man" on HN).
  - AdieuToLogic 2y ago
    > I did in fact do it, and what I got was much, much easier than the samples in the article, which 4o did fine with. Awesome. Can you guarantee its results are completely accurate every time, with every document, and need no human review? > I'm sorry, but I declare the burden of proof here to be switched. If you are referencing my stating: If it's that easy, then do it and be the hero they want. Then I don't really know how to respond. Otherwise, if you are referencing my statement: > Perhaps "random humans" can perform tasks which could reshape your belief: >> OCR is VERY good To which I again ask, can you guarantee the correctness of OCR results will exceed what "random humans" can generally provide? What about "non-random motivated humans"? My point is that automated approaches to tasks such as what the National Archives have outlined here almost always require human review/approval, as accuracy is paramount. > (I don't think you need to Wikipedia-cite "straw man" on HN). I do so for two purposes. First, if I misuse a cited term someone here will quickly correct me. Second, there is always a probability of someone new here which is unaware of the cited term(s).
    - Dylan16807 2y ago
      > If you are referencing my stating: > > If it's that easy, then do it and be the hero they want. > Then I don't really know how to respond. If someone says a thing is easy, and you respond by demanding they do it a million times to prove that it's easy, you are the one that has screwed up the burden of proof.
    - tptacek 2y ago
      Can I ask, did you sign up and look at what they're actually looking for? Show of good faith: can you give 3 of the headers for the top-level "missions" they have for transcriptions?
      - AdieuToLogic 2y ago
        I have failed to express myself clearly and for that I apologize. What I was trying to convey is that no matter what tooling is employed for this specific problem, a human will have to review the result to ensure accuracy. Since the problem is defined as reading cursive handwriting, people who can do so intrinsically obviate the need for tooling. Whether or not automation can produce reasonable results is moot when the requirement is to have as accurate as possible a one-time transcription of cursive handwriting documents.
- Dylan16807 2y ago
  There are two claims. The main one is that all of these documents are easy to individually transcribe by machine. The other is that a whole lot can be OCR'd, which is pretty simple to check. That's not a claim that processing the entire archive would be trivial. And even if it was, whether that would make someone the "hero they want" is part of what's being called into question. So your silly demand going unmet proves nothing. Also, "give me an example please" is not a strawman! If you actually want to prove something, you need to show at least one document in the set that a human can do but not a machine, or to really make a good point you need to show that a non-neglibile fraction fit that description.
  - AdieuToLogic 2y ago
    > So your silly demand going unmet proves nothing. I made demands of no one. > Also, "give me an example please" is not a strawman! My identification of the strawman was that it referenced "find something hard" when I had said "be the hero they want" and that what is needed in this specific problem domain may be more difficult than what a generalization addresses. > If you actually want to prove something, you need to show at least one document in the set that a human can do but not a machine, or to really make a good point you need to show that a non-neglibile fraction fit that description. Maybe this is the proof you demand. LLM's are statistical prediction algorithms. As such, they are nondeterministic and, therefore, provide no guarantees as to the correctness of their output. The National Archives have specific artifacts requiring precise textual data extraction. Use of nondeterministic tools known to produce provably incorrect results eliminate their applicability in this workflow due to all of their output requiring human review. This is an unnecessary step and can be eliminated by the human reading the original text themself. Does that satisfy your demand?
    - Dylan16807 2y ago
      > I made demands of no one. Whatever you want to call "If it's that easy, then do it" > LLM's [...] Does that satisfy your demand? That's a different argument from the one above where you were trying to contradict tptacek. And that argument is flawed itself. In particular, humans don't have guarantees either. > provably incorrect results This gets back to the actual request from earlier, which is showing an example where the machine performs below some human standard. Just pointing out that LLMs make mistakes is not enough proof of incorrectness in this specific use case.
- idlewords 2y ago
  Please don't link to wikipedia definitions of elementary terms. It's condescending and obnoxious.
  - AdieuToLogic 2y ago
    > Please don't link to wikipedia definitions of elementary terms. It's condescending and obnoxious. I respect what you have chosen to contribute to this conversation, but neither need nor seek your approval of mine.
    - tptacek 2y ago
      For the record, it mostly comes across as you only recently learning what a straw man argument is, which, in the context of a message board argument, is a rough thing to have to admit, so you have my sympathies.