6 ms·

I am a complete noob to the AI space but I was wondering whether the following is possible (in AWS). I have a million scanned images of court documents. Some

I am a complete noob to the AI space but I was wondering whether the following is possible (in AWS). I have a million scanned images of court documents. Some are briefs, some are motions, some are court orders, etc... Given that I have images and their types, could I "train" the AI with these million documents to recognize a new image that might come in?

aerovistae 10y ago
Is that possible with AI? Yes, almost trivially. Is that possible with AWS AI? > Finally, we provide AI engines, a collection of open-source, deep learning frameworks for academics and data scientists who want to build cutting edge, sophisticated intelligent systems, pre-installed configured on a convenient machine image. Which is to say anything that TensorFlow can do, AWS AI can do. So yes. But is it possible without implementing it yourself? It looks like Amazon Rekognition may be able to do what you're looking for, but I'm not certain. You'll have to research that one.
- starik36 10y ago
  > almost trivially If not AWS, how could I get started elsewhere? Every tutorial I've looked at deals with a CVS data file, not an image. Could you point me to a resource that has an example of how to train AI with image documents?
  - cobookman 10y ago
    https://cloud.google.com/products/machine-learning/ https://cloud.google.com/products/machine-learning/ Look at the text API which parses text from an image and gives you it's pixel location.
  - allenz 10y ago
    You would just use pixel values as inputs, see https://www.tensorflow.org/get_started/mnist/beginners https://www.tensorflow.org/get_started/mnist/beginners You can make it more performant by downsizing the documents, but obviously this is incredibly GPU-intensive to train. I second the suggestions to just use OCR.
- ranman 10y ago
  Rekognition doesn't currently support OCR.
bhaavan 10y ago
Principally, yes. However, the approach may be more nuisanced than that. If I were you, I would first pick a character recognition engine (which might have already been well trained) to convert the image to text. Once the text is there, that might serve as a better feature to classify the content. Furthermore, I had recommend converting words in the text to word-embeddings/ vectors using a suitable Glove or Word2Vec dataset similar to your content. While there are many benefits of end-to-end training, I don't think it might be best suited for this case. This is because we already know that the only useful feature in the document is the text, and not the contours or textures. Theefore wasting neurons in your neural network to learn the wastefulness of this features is just waste of resources. Furthermore, you benefit from even a larger corpus of learned data, which the char recognition engine has been trained on.
- starik36 10y ago
  OCR is my current approach. I am not really happy with it. The quality of OCRing leaves much to be desired, probably due to the documents themselves being haphazardly handled by the court personnel. OCR itself is a pretty CPU intensive activity and takes a significant time to complete for many documents. Thus, I was looking for a more advanced approach.
  - jononor 10y ago
    What do you want to recognize? OCR is a better understood problem than a general neural net, so I think it's likely easier to improve its quality that to superseded the quality with image-based recognition.
    - starik36 10y ago
      Ideally, I would like to get all the information from the page. Phone numbers, who is suing whom, case caption, etc... With OCR, you get bits and pieces of information, but because I don't know what the type of the document is, it is difficult to determine where, structurally speaking, this information resides on a page. If I could use AI to determine the type of the doc, I would know the structure of the document and I could then use OCR to pinpoint specific information on the page. Most court documents are created from templates.
cobookman 10y ago
Yep it's possible to do this with ml. I've dealt with some partners who've built solutions in this space. More than happy to chat with you
skewart 10y ago
AWS doesn't offer any high-level services for training your own custom model. You'd have to build the neural network yourself and deploy EC2 boxes to run it. I've had success with Clarifai's [0] custom CV model API in the past. You basically upload batches of labeled images to train a model, and then you can submit new images for classification. Of course, I have no idea how effective it would be for your documents. Obviously it depends on how visually distinct the different types are. [0] https://www.clarifai.com https://www.clarifai.com
- starik36 10y ago
  Wow, this is literally exactly what I was looking for. Thank you. Hopefully, it works well. From https://developer.clarifai.com/quick-start/ https://developer.clarifai.com/quick-start/ Seems simple to train // add inputs with concepts app.inputs.create([{ "url": "https://samples.clarifai.com/dog1.jpeg", "concepts": [ { "id": "cat", "value": false }, { "id": "dog", "value": true } ] }, { "url": "https://samples.clarifai.com/dog2.jpeg", "concepts": [ { "id": "cat", "value": false }, { "id": "dog", "value": true } ] Then you predict what the another image is: // predict the contents of an image by passing in a url app.models.predict(Clarifai.GENERAL_MODEL, 'https://samples.clarifai.com/metro-north.jpg').then( function(response) { console.log(response); }, function(err) { console.error(err); } );
  - skewart 10y ago
    I'm glad I could help! I'd actually be really curious to hear how well it works for you.
    - starik36 10y ago
      I'll post results here once I get permission to bring those docs into the cloud - they are all mostly private.
IanCal 10y ago
People keep recommending things and approaches, but I'm really not clear on what it is you're actually trying to solve, so most of the responses might not help. > Given that I have images and their types, could I "train" the AI with these million documents to recognize a new image that might come in? Do I understand this right: 1. You have lots of documents as images, and their type (brief, motion, court order) 2. You get a new document, as an image. 3. You want to assign a type to this new document. Is that right? What's your goal in terms of quality? A key way of thinking about this is: 1. What's the risk/cost if you mis-classify a document? 2. What's the risk/cost if you fail to classify a document? Are the documents typically very structurally different? Could I probably tell them apart without wearing glasses? Or are they largely the same, but with nuanced differences in the text?
- starik36 10y ago
  > Do I understand this right That is exactly right. As far as quality, if I classify the document wrong today (or fail to classify it), it's not that big of a deal to the system as a whole - but users will be "very" annoyed and have to correct it. > Are the documents typically very structurally different? Could I probably tell them apart without wearing glasses? Or are they largely the same, but with nuanced differences in the text? The documents are structurally reasonably different, but not "very" different. If a trained human looks at it, they would be able to tell them apart. There are exceptions, of course. For instance, if the attorney doesn't follow accepted convention, but those are rare.
  - Jarwain 10y ago
    There's probably a way to determine the document type based on the differing structures (would have to see it to be sure). Alternately, is it not possible for a user to specify what type the document is?
moeadham 10y ago
We have a similar project we are working on, it might be interesting to collaborate based on your needs. Send me a message: contact - @ - omen.ai

aerovistae 10y ago

Is that possible with AI? Yes, almost trivially. Is that possible with AWS AI? > Finally, we provide AI engines, a collection of open-source, deep learning frameworks for academics and data scientists who want to build cutting edge, sophisticated intelligent systems, pre-installed configured on a convenient machine image. Which is to say anything that TensorFlow can do, AWS AI can do. So yes. But is it possible without implementing it yourself? It looks like Amazon Rekognition may be able to do what you're looking for, but I'm not certain. You'll have to research that one.

starik36 10y ago
> almost trivially If not AWS, how could I get started elsewhere? Every tutorial I've looked at deals with a CVS data file, not an image. Could you point me to a resource that has an example of how to train AI with image documents?
- cobookman 10y ago
  https://cloud.google.com/products/machine-learning/ https://cloud.google.com/products/machine-learning/ Look at the text API which parses text from an image and gives you it's pixel location.
- allenz 10y ago
  You would just use pixel values as inputs, see https://www.tensorflow.org/get_started/mnist/beginners https://www.tensorflow.org/get_started/mnist/beginners You can make it more performant by downsizing the documents, but obviously this is incredibly GPU-intensive to train. I second the suggestions to just use OCR.
ranman 10y ago
Rekognition doesn't currently support OCR.

bhaavan 10y ago

Principally, yes. However, the approach may be more nuisanced than that. If I were you, I would first pick a character recognition engine (which might have already been well trained) to convert the image to text. Once the text is there, that might serve as a better feature to classify the content. Furthermore, I had recommend converting words in the text to word-embeddings/ vectors using a suitable Glove or Word2Vec dataset similar to your content. While there are many benefits of end-to-end training, I don't think it might be best suited for this case. This is because we already know that the only useful feature in the document is the text, and not the contours or textures. Theefore wasting neurons in your neural network to learn the wastefulness of this features is just waste of resources. Furthermore, you benefit from even a larger corpus of learned data, which the char recognition engine has been trained on.

starik36 10y ago
OCR is my current approach. I am not really happy with it. The quality of OCRing leaves much to be desired, probably due to the documents themselves being haphazardly handled by the court personnel. OCR itself is a pretty CPU intensive activity and takes a significant time to complete for many documents. Thus, I was looking for a more advanced approach.
- jononor 10y ago
  What do you want to recognize? OCR is a better understood problem than a general neural net, so I think it's likely easier to improve its quality that to superseded the quality with image-based recognition.
  - starik36 10y ago
    Ideally, I would like to get all the information from the page. Phone numbers, who is suing whom, case caption, etc... With OCR, you get bits and pieces of information, but because I don't know what the type of the document is, it is difficult to determine where, structurally speaking, this information resides on a page. If I could use AI to determine the type of the doc, I would know the structure of the document and I could then use OCR to pinpoint specific information on the page. Most court documents are created from templates.

cobookman 10y ago

Yep it's possible to do this with ml. I've dealt with some partners who've built solutions in this space. More than happy to chat with you

skewart 10y ago

AWS doesn't offer any high-level services for training your own custom model. You'd have to build the neural network yourself and deploy EC2 boxes to run it. I've had success with Clarifai's [0] custom CV model API in the past. You basically upload batches of labeled images to train a model, and then you can submit new images for classification. Of course, I have no idea how effective it would be for your documents. Obviously it depends on how visually distinct the different types are. [0] https://www.clarifai.com https://www.clarifai.com

starik36 10y ago
Wow, this is literally exactly what I was looking for. Thank you. Hopefully, it works well. From https://developer.clarifai.com/quick-start/ https://developer.clarifai.com/quick-start/ Seems simple to train // add inputs with concepts app.inputs.create([{ "url": "https://samples.clarifai.com/dog1.jpeg", "concepts": [ { "id": "cat", "value": false }, { "id": "dog", "value": true } ] }, { "url": "https://samples.clarifai.com/dog2.jpeg", "concepts": [ { "id": "cat", "value": false }, { "id": "dog", "value": true } ] Then you predict what the another image is: // predict the contents of an image by passing in a url app.models.predict(Clarifai.GENERAL_MODEL, 'https://samples.clarifai.com/metro-north.jpg').then( function(response) { console.log(response); }, function(err) { console.error(err); } );
- skewart 10y ago
  I'm glad I could help! I'd actually be really curious to hear how well it works for you.
  - starik36 10y ago
    I'll post results here once I get permission to bring those docs into the cloud - they are all mostly private.

IanCal 10y ago

People keep recommending things and approaches, but I'm really not clear on what it is you're actually trying to solve, so most of the responses might not help. > Given that I have images and their types, could I "train" the AI with these million documents to recognize a new image that might come in? Do I understand this right: 1. You have lots of documents as images, and their type (brief, motion, court order) 2. You get a new document, as an image. 3. You want to assign a type to this new document. Is that right? What's your goal in terms of quality? A key way of thinking about this is: 1. What's the risk/cost if you mis-classify a document? 2. What's the risk/cost if you fail to classify a document? Are the documents typically very structurally different? Could I probably tell them apart without wearing glasses? Or are they largely the same, but with nuanced differences in the text?

starik36 10y ago
> Do I understand this right That is exactly right. As far as quality, if I classify the document wrong today (or fail to classify it), it's not that big of a deal to the system as a whole - but users will be "very" annoyed and have to correct it. > Are the documents typically very structurally different? Could I probably tell them apart without wearing glasses? Or are they largely the same, but with nuanced differences in the text? The documents are structurally reasonably different, but not "very" different. If a trained human looks at it, they would be able to tell them apart. There are exceptions, of course. For instance, if the attorney doesn't follow accepted convention, but those are rare.
- Jarwain 10y ago
  There's probably a way to determine the document type based on the differing structures (would have to see it to be sure). Alternately, is it not possible for a user to specify what type the document is?

moeadham 10y ago

We have a similar project we are working on, it might be interesting to collaborate based on your needs. Send me a message: contact - @ - omen.ai