4 ms·

I think there is a far more effective method in this situation: the FDA has mercifully made this data available for download as plain text files: https://www.fd

I think there is a far more effective method in this situation: the FDA has mercifully made this data available for download as plain text files: https://www.fda.gov/medical-devices/510k-clearances/downloadable-510k-files https://www.fda.gov/medical-devices/510k-clearances/download... So I would download all 6 files, then write a script/app to fuzzy search/filter/transform those files as needed. No need to manually scrape using the web form! For example, here is line 65457 from PMNLSTMN.ZIP. It seems the OP is interested in searching the last column and outputting the first column: K202167|Cordis Corporation|...|N||Brite Tip Radianz Guiding Sheath

wcedmisten 3y ago
Author here! That's true, and I actually used that dump[1] and OCR to automate about 80% of this project. The manual data entry was only required for the 20% that couldn't be found automatically. Since this was just to fill in the gaps, I figured there would be diminishing returns on re-implementing fuzzy search on the data, but adding shortcuts to my manual workflow took less time for a reasonable speedup. [1] I also used the JSON dump of this data that the FDA provides here: https://open.fda.gov/apis/device/510k/download/ https://open.fda.gov/apis/device/510k/download/
- Leftium 3y ago
  Thank goodness~ I imagined the author doing nothing but manually scraping data from this web form for an entire year.
thefatboy 3y ago
why be fuzzy like Google when you can be exact?
- Leftium 3y ago
  It was a limitation of the web form the author mentioned[1]: > The device and company names are not standardized, and may have abbreviations, acronyms, or plain old typos. The website's search functionality does not provide fuzzy string matching, so finding a device often takes some trial and error. [1]: https://wcedmisten.fyi/post/keyboard-shortcuts-userscripts/#:~:text=The%20device%20and%20company%20names%20are%20not%20standardized%2C%20and%20may%20have%20abbreviations%2C%20acronyms%2C%20or%20plain%20old%20typos.%20The%20website%27s%20search%20functionality%20does%20not%20provide%20fuzzy%20string%20matching%2C%20so%20finding%20a%20device%20often%20takes%20some%20trial%20and%20error https://wcedmisten.fyi/post/keyboard-shortcuts-userscripts/#....

wcedmisten 3y ago

Author here! That's true, and I actually used that dump[1] and OCR to automate about 80% of this project. The manual data entry was only required for the 20% that couldn't be found automatically. Since this was just to fill in the gaps, I figured there would be diminishing returns on re-implementing fuzzy search on the data, but adding shortcuts to my manual workflow took less time for a reasonable speedup. [1] I also used the JSON dump of this data that the FDA provides here: https://open.fda.gov/apis/device/510k/download/ https://open.fda.gov/apis/device/510k/download/

Leftium 3y ago
Thank goodness~ I imagined the author doing nothing but manually scraping data from this web form for an entire year.

thefatboy 3y ago

why be fuzzy like Google when you can be exact?

Leftium 3y ago
It was a limitation of the web form the author mentioned[1]: > The device and company names are not standardized, and may have abbreviations, acronyms, or plain old typos. The website's search functionality does not provide fuzzy string matching, so finding a device often takes some trial and error. [1]: https://wcedmisten.fyi/post/keyboard-shortcuts-userscripts/#:~:text=The%20device%20and%20company%20names%20are%20not%20standardized%2C%20and%20may%20have%20abbreviations%2C%20acronyms%2C%20or%20plain%20old%20typos.%20The%20website%27s%20search%20functionality%20does%20not%20provide%20fuzzy%20string%20matching%2C%20so%20finding%20a%20device%20often%20takes%20some%20trial%20and%20error https://wcedmisten.fyi/post/keyboard-shortcuts-userscripts/#....