4 ms·
There's a deep connection between what I do and text retrieval in general. Take a look at the early work in IR in the 1940s and 1950s, at https://en.wikipedia.
by dalke 6y ago
There's a deep connection between what I do and text retrieval in general.
Take a look at the early work in IR in the 1940s and 1950s, at https://en.wikipedia.org/wiki/Information_retrieval#Timeline https://en.wikipedia.org/wiki/Information_retrieval#Timeline
1947, "Hans Peter Luhn (research engineer at IBM since 1941) began work on a mechanized punch card-based system for searching chemical compounds"
1950s, "invention of citation indexing (Eugene Garfield)" - Garfield's earlier work was with chemical structure data sets, and his PhD work was on the linguists of chemical compound names.
1950: "The term "information retrieval" was coined by Calvin Mooers." - that was presented at an American Chemical Society (ACS) meeting that year, and in the 1940s Mooers developed an early version of what is now called a connection table, hand-waved a substructure search algorithm which was implemented a few years later. (I'm a Mooers fanboy!)
Many of the early IR events were at ACS meetings - the concept of an "inverted index" was presented at one, as I recall.
This is because in the 1940s, chemical search was Big Data, with >1 million records containing many structured data search fields, and demand for chemical record search from many organizations.
So many of the core concepts are the same, though in cheminformatics we've switched to a lossy encoding of molecular features to a bitstring fingerprint since we tend to look more at global similarity than local similarity, and there are a lot of possible features to encode.
Thank you for writing that it was a very readable paper. I have received very little feedback of any sort about the publication, and have been worried that it was too verbose, pedantic, or turgid.
- dekhn 6y agoIts a bit verbose, and I really think it's several papers (the technical details of the package is one, the open source positioning is another). But it's readable- a person outside the field (say, a search engineer at Google) could sit down, read this and immediately recognize what you were trying to achieve ("implement popcnt" used to be a popular question), and then immediately suggest ways to get the output results faster by using a cluster :)
- dalke 6y agoIndeed, it is several papers. There are two journals in my field - one I can't read because it's behind a paywall and one that's expensive to publish in. I choose the latter, but couldn't afford multiple months of rent in order to publish several papers. :( A blog post I wrote years ago use to part of the "implement popcnt" literature - http://www.dalkescientific.com/writings/diary/archive/2008/07/03/hakmem_and_other_popcounts.html http://www.dalkescientific.com/writings/diary/archive/2008/0... . It's now outdated, and actual low-level programmers have done better, but it still gets mentioned in-passing in postings like the one referenced on HN last year at https://news.ycombinator.com/item?id=20914479 https://news.ycombinator.com/item?id=20914479 .
- dekhn 6y agoIt's really extraordinary how tightly coupled modern innovation in scientific fields is to processor implementations. I suspect you and I share a keen interest in the path by which we got to this enviable situation.