16 ms·

I used pdf2json to scrape all the data from the GFCO PDF, which they publish annually. So mostly tool-assisted manual labor :D However, I'm now negotiating wit

I used pdf2json to scrape all the data from the GFCO PDF, which they publish annually. So mostly tool-assisted manual labor :D However, I'm now negotiating with GFCO to get continually up-to-date information. I'm also working with another major GF certification and will soon be adding their products too.

rambojazz 9y ago
What's the license of the data in GFCO?
- flybayer 9y ago
  There's no license because it's facts and can't be copyrighted.
  - semi-extrinsic 9y ago
    For the US, correct [1]. For other countries, YMMV. Edit: e.g. in the UK, sui generis rights apply to a database [2], such that if creation of a database required "substantial investment in obtaining, verifying or presenting the contents of the database" then someone else cannot copy it and re-publish in another form. This is very different from US law, which requires some "creative originality" to be involved. [1] https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._Rural_Telephone_Service_Co https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._R.... [2] https://en.wikipedia.org/wiki/Sui_generis_database_right https://en.wikipedia.org/wiki/Sui_generis_database_right
    - rambojazz 9y ago
      Would this also apply to a database such as IMDB, since movie information is just facts? So, like, I can crawl their website and make a public-domain database of movies/actors? This would be cool, since I'm not aware of any free database for movies/actors.
DanBC 9y ago
Do different certification organisations have similar requirements? GFCO seem to require less than 10 PPM, and no barley. Is that consistent with other orgs?
- flybayer 9y ago
  GFCO (glutenfreecert.com) holds the manufacturer to whatever they self-declare (20ppm, 10ppm, 5ppm, etc and even undetectable). GFCO certifies individual products with testing. GFCP certifies facilities with required systems and processes. These folks basically do the certification at a higher place in the chain and is where the industry is headed.
- code_duck 9y ago
  Definitely no barley, because barley is just as much a gluten containing grain as wheat. Barley, rye, wheat and other grains each have their own protein. For those they are hordein, secalin, and gliadin, and they are all considered gluten.
nickk2006 9y ago
Really cool project! Forwarded link to friend who’s alergic to gluten. Is the information regarding where to purchase the products added manually?
- flybayer 9y ago
  Thank you! :) Purchase links are added a few different ways. We're an affiliate for Nuts.com, so we have access to their entire product list. It takes a bit of manual work to match up the data, but I'll be able to automate it as soon as it makes sense to do so.
code_duck 9y ago
My concern in devoting time to a project like this would be that GFCO could easily make their own data searchable at any time, given that they already have their own database, and thus search isn’t very difficult to implement. It’s the same problem faced by any developer relying on one company or API for all of their data. What would their motivation be to work with you rather than just enhance their own website?

rambojazz 9y ago

What's the license of the data in GFCO?

flybayer 9y ago
There's no license because it's facts and can't be copyrighted.
- semi-extrinsic 9y ago
  For the US, correct [1]. For other countries, YMMV. Edit: e.g. in the UK, sui generis rights apply to a database [2], such that if creation of a database required "substantial investment in obtaining, verifying or presenting the contents of the database" then someone else cannot copy it and re-publish in another form. This is very different from US law, which requires some "creative originality" to be involved. [1] https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._Rural_Telephone_Service_Co https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._R.... [2] https://en.wikipedia.org/wiki/Sui_generis_database_right https://en.wikipedia.org/wiki/Sui_generis_database_right
  - rambojazz 9y ago
    Would this also apply to a database such as IMDB, since movie information is just facts? So, like, I can crawl their website and make a public-domain database of movies/actors? This would be cool, since I'm not aware of any free database for movies/actors.

DanBC 9y ago

Do different certification organisations have similar requirements? GFCO seem to require less than 10 PPM, and no barley. Is that consistent with other orgs?

flybayer 9y ago
GFCO (glutenfreecert.com) holds the manufacturer to whatever they self-declare (20ppm, 10ppm, 5ppm, etc and even undetectable). GFCO certifies individual products with testing. GFCP certifies facilities with required systems and processes. These folks basically do the certification at a higher place in the chain and is where the industry is headed.
code_duck 9y ago
Definitely no barley, because barley is just as much a gluten containing grain as wheat. Barley, rye, wheat and other grains each have their own protein. For those they are hordein, secalin, and gliadin, and they are all considered gluten.

nickk2006 9y ago

Really cool project! Forwarded link to friend who’s alergic to gluten. Is the information regarding where to purchase the products added manually?

flybayer 9y ago
Thank you! :) Purchase links are added a few different ways. We're an affiliate for Nuts.com, so we have access to their entire product list. It takes a bit of manual work to match up the data, but I'll be able to automate it as soon as it makes sense to do so.

code_duck 9y ago

My concern in devoting time to a project like this would be that GFCO could easily make their own data searchable at any time, given that they already have their own database, and thus search isn’t very difficult to implement. It’s the same problem faced by any developer relying on one company or API for all of their data. What would their motivation be to work with you rather than just enhance their own website?