3 ms·

DocQuery (https://github.com/impira/docquery https://github.com/impira/docquery), a project I work on, allows you to do something similar, but search over seman

DocQuery (https://github.com/impira/docquery https://github.com/impira/docquery), a project I work on, allows you to do something similar, but search over semantic information in the PDF files (using a large language model that is pre-trained to query business documents). For example: $ docquery scan "What is the due date?" /my/invoices/ /my/invoices/Order1.pdf What is the due date?: 4/27/2022 /my/invoices/Order2.pdf What is the due date?: 9/26/2022 ... It's obviously a lot slower than "grepping", but very powerful.

nsomaru 4y ago
I’d like a way to index a lot of case law and then ask it questions.
pugio 4y ago
Wow this is exactly what I've been looking for, thank you! I just wish with these transformer models it was possible to extract a structured set of what the model "knows" (for e.g. easy search indexing ). These natural language question systems are a little too fuzzy sometimes.
- ankrgyl 4y ago
  Can you tell me a bit more about your use case? A few things that come to mind: - There are some ML/transformer-based methods for extracting a known schema (e.g. NER) or an unknown schema (e.g. relation extraction). - We're going to add a feature to DocQuery called "templates" soon for some popular document types (e.g. invoices) + a document classifier which will automatically apply the template based on the doc type. - Our commercial product (http://impira.com/ http://impira.com/) supports all of this + is a hosted solution (many of our customers use us to automate accounts payable, process insurance documents, etc.)
  - pugio 4y ago
    Your commercial product looks very cool, but my use case is in creating an offline-first local document storage system (data never reaches a cloud). I'd like to be enable users to search through all documents for relevant pieces of information. The templates sound very cool - are they essentially just using a preset list of (natural language) queries tied to a particular document class? It seems like you're using a version of donut for your document classification?
    - ankrgyl 4y ago
      > but my use case is in creating an offline-first local document storage system (data never reaches a cloud). Makes sense -- this is why we OSS'd DocQuery :) > The templates sound very cool - are they essentially just using a preset list of (natural language) queries tied to a particular document class? It seems like you're using a version of donut for your document classification? Yes that's the plan. We've done extensive testing with other approaches (e.g. NER) and realized that the benefits of using use-case specific queries (customizability, accuracy, flexibility for many use cases) outweigh the tradeoffs (NER only needs one execution for all fields). Currently, we support pre-trained Donut models for both querying and classification. You can play with it by adding the --classify flag to `docquery scan`. We're releasing some new stuff soon that should be faster and more accurate.
      - pugio 4y ago
        Sweet! I'll keep an eye on the repo. Thank you for open sourcing DocQuery. I agree with your reasoning: my current attempts to find an NER model that covers all my use cases have come up short
  - mbb70 4y ago
    Since you mention insurance documents, could you speak to how well this would extract data from a policy document like https://ahca.myflorida.com/medicaid/Prescribed_Drug/drug_criteria_pdf/Spinraza_Criteria.pdf https://ahca.myflorida.com/medicaid/Prescribed_Drug/drug_cri... ? The unstoppable administrative engine that is the American Healthcare system produces hundreds of thousands of continuously updated documents like this with no standardized format/structure. Manually extracting/normalizing this data into a querable format is an industry all its own.
    - ankrgyl 4y ago
      It's very easy to try! Just plug that URL here: https://huggingface.co/spaces/impira/docquery https://huggingface.co/spaces/impira/docquery. I tried a few questions: What is the development date? -> June 20, 2017 What is the medicine? -> SPINRAZA® (nusinersen) How many doses -> 5 doses Did the patient meet the review criteria? -> Patient met initial review criteria. Is the patient treated with Evrysdi? -> not
- a1369209993 4y ago
  > to extract a structured set of what the model "knows" To be fair, that's impossible in the general case, since the model can know things (ie be able to answer queries) without knowing that it knows them (ie being able to produce a list of anserable queries by any means significantly more efficient than trying every query and seeing which ones work). As a reducto ad absurdum example, consider a 'model' consisting of a deniably encrypted key-value store, where it's outright cryptographically guaranteed that you can't effiently enumerate the queries. Neural networks aren't quite that bad, but (in the general-over-NNs case) they at least superficially appear to be pretty close. (They're definitely not reliably secure though; don't depend on that.)
ultrasounder 4y ago
This is so epic. I was just ruminating about this particular use-case. who are your typical customers. Supply chain or purchasing? Also I notice that you do text extraction from Invoices? Are you using something similar to CharGRID or its derivate BERTGRID? Wish you and your team more success!
- ankrgyl 4y ago
  Thank you ultrasounder! Supply chain, construction, purchasing, insurance, financial services, and healthcare are our biggest verticals. Although we have customers doing just about anything you can imagine with documents! For invoices, we have a pre-trained model (demo here: https://huggingface.co/spaces/impira/invoices https://huggingface.co/spaces/impira/invoices) that is pretty good at most fields, but within our product, it will automatically learn about your formats as you upload documents and confirm/correct predictions. The pre-trained model is based on LayoutLM and the additional learning we do uses a generative model (GMMs) that can learn from as little as one example. LMK if you have any other questions.
  - tarasglek 4y ago
    Can this work like AI readability extension? What's the content of this page? What are the navigation options on this page?

nsomaru 4y ago

I’d like a way to index a lot of case law and then ask it questions.

pugio 4y ago

Wow this is exactly what I've been looking for, thank you! I just wish with these transformer models it was possible to extract a structured set of what the model "knows" (for e.g. easy search indexing ). These natural language question systems are a little too fuzzy sometimes.

ankrgyl 4y ago
Can you tell me a bit more about your use case? A few things that come to mind: - There are some ML/transformer-based methods for extracting a known schema (e.g. NER) or an unknown schema (e.g. relation extraction). - We're going to add a feature to DocQuery called "templates" soon for some popular document types (e.g. invoices) + a document classifier which will automatically apply the template based on the doc type. - Our commercial product (http://impira.com/ http://impira.com/) supports all of this + is a hosted solution (many of our customers use us to automate accounts payable, process insurance documents, etc.)
- pugio 4y ago
  Your commercial product looks very cool, but my use case is in creating an offline-first local document storage system (data never reaches a cloud). I'd like to be enable users to search through all documents for relevant pieces of information. The templates sound very cool - are they essentially just using a preset list of (natural language) queries tied to a particular document class? It seems like you're using a version of donut for your document classification?
  - ankrgyl 4y ago
    > but my use case is in creating an offline-first local document storage system (data never reaches a cloud). Makes sense -- this is why we OSS'd DocQuery :) > The templates sound very cool - are they essentially just using a preset list of (natural language) queries tied to a particular document class? It seems like you're using a version of donut for your document classification? Yes that's the plan. We've done extensive testing with other approaches (e.g. NER) and realized that the benefits of using use-case specific queries (customizability, accuracy, flexibility for many use cases) outweigh the tradeoffs (NER only needs one execution for all fields). Currently, we support pre-trained Donut models for both querying and classification. You can play with it by adding the --classify flag to `docquery scan`. We're releasing some new stuff soon that should be faster and more accurate.
    - pugio 4y ago
      Sweet! I'll keep an eye on the repo. Thank you for open sourcing DocQuery. I agree with your reasoning: my current attempts to find an NER model that covers all my use cases have come up short
- mbb70 4y ago
  Since you mention insurance documents, could you speak to how well this would extract data from a policy document like https://ahca.myflorida.com/medicaid/Prescribed_Drug/drug_criteria_pdf/Spinraza_Criteria.pdf https://ahca.myflorida.com/medicaid/Prescribed_Drug/drug_cri... ? The unstoppable administrative engine that is the American Healthcare system produces hundreds of thousands of continuously updated documents like this with no standardized format/structure. Manually extracting/normalizing this data into a querable format is an industry all its own.
  - ankrgyl 4y ago
    It's very easy to try! Just plug that URL here: https://huggingface.co/spaces/impira/docquery https://huggingface.co/spaces/impira/docquery. I tried a few questions: What is the development date? -> June 20, 2017 What is the medicine? -> SPINRAZA® (nusinersen) How many doses -> 5 doses Did the patient meet the review criteria? -> Patient met initial review criteria. Is the patient treated with Evrysdi? -> not
a1369209993 4y ago
> to extract a structured set of what the model "knows" To be fair, that's impossible in the general case, since the model can know things (ie be able to answer queries) without knowing that it knows them (ie being able to produce a list of anserable queries by any means significantly more efficient than trying every query and seeing which ones work). As a reducto ad absurdum example, consider a 'model' consisting of a deniably encrypted key-value store, where it's outright cryptographically guaranteed that you can't effiently enumerate the queries. Neural networks aren't quite that bad, but (in the general-over-NNs case) they at least superficially appear to be pretty close. (They're definitely not reliably secure though; don't depend on that.)

ultrasounder 4y ago

This is so epic. I was just ruminating about this particular use-case. who are your typical customers. Supply chain or purchasing? Also I notice that you do text extraction from Invoices? Are you using something similar to CharGRID or its derivate BERTGRID? Wish you and your team more success!

ankrgyl 4y ago
Thank you ultrasounder! Supply chain, construction, purchasing, insurance, financial services, and healthcare are our biggest verticals. Although we have customers doing just about anything you can imagine with documents! For invoices, we have a pre-trained model (demo here: https://huggingface.co/spaces/impira/invoices https://huggingface.co/spaces/impira/invoices) that is pretty good at most fields, but within our product, it will automatically learn about your formats as you upload documents and confirm/correct predictions. The pre-trained model is based on LayoutLM and the additional learning we do uses a generative model (GMMs) that can learn from as little as one example. LMK if you have any other questions.
- tarasglek 4y ago
  Can this work like AI readability extension? What's the content of this page? What are the navigation options on this page?