4 ms·
Looking for something similar for .pdf academic papers in my field but nothing really useful to automatise the extraction process exists, so the best path is st
by DrNuke 10y ago
Looking for something similar for .pdf academic papers in my field but nothing really useful to automatise the extraction process exists, so the best path is still to extract data manually, homogenise data in a standard protocol, fed ML algos. Once a data protocol becomes a widespread standard and maybe a ISO or similar, there is a chance automated extraction will work at the finest level, as necessary for complex information.
- dartwing 10y agoI found these two projects so far: https://cloud.google.com/natural-language/ https://cloud.google.com/natural-language/ https://www.textrazor.com/demo https://www.textrazor.com/demo Do you expect the approach of extracting entities and relations between them to be limited? I hope that it can be boosted for specific domain with predefined entities and facts structures.
- DrNuke 10y agoI do not know how these tools work today but I can imagine a dictionary becoming standard for each given field and being provided with every future .pdf paper as a companion in the form of a .csv sparse matrix or similar format. Such dictionaries will be put forward by researchers for exchanging data in the smoothest way, agreed as a standard after some time and then forced into adoption worldwide. We will get there for sure, some fields before others, some industries have started already (aeronautics, automotive).
- lwhsiao 10y agoOn system for extracting information from PDFs is Fonduer[1], which is built on the Snorkel framework from Stanford. It may be worth checking out for your use case. Here's a blog post introducing it [2]. Disclosure: I worked on the project. [1] https://arxiv.org/abs/1703.05028 https://arxiv.org/abs/1703.05028 [2] https://hazyresearch.github.io/snorkel/blog/fonduer.html https://hazyresearch.github.io/snorkel/blog/fonduer.html
- dartwing 10y agoThank you! Will look through.