8 ms·

Announcing the New PubMed

120 points by vo2maxer 7y ago

nahikoa 7y ago
PubMed is an incredibly useful resource. I've found that it is fairly common for my psychiatrist and GP to be relatively unaware of the latest studies and treatments. Of course, reading through abstracts and papers requires a critical eye into the statistics, methodology, and funding. The Cochrane Reviews are similarly awesome.
- deleted 7y ago
  [deleted]
- softwaredoug 7y ago
  Same here. It’s pretty shocking how many different opinions you’ll get from doctors. I end up doing my own research quite a lot via public.
  - GordonS 7y ago
    And same here too (in the UK), at least within the NHS - it seems doctors train, start working, and then don't spend any time learning about new developments within their field. It's very disappointing. I'm sure it can't be like that for all doctors, but it's been my (unfortunately vast and varied) experience.
    - Engineering-MD 7y ago
      Doctors have to show continued professional development every year for Revalidation of their license. It’s hard to keep on top of all the research, there is so much of it. Much of it of arguable quality, and contradictory making it hard to interpret. Generally people follow guidelines and advice from experts unless they are the expert in that are due to this. And doctors are culturally quite cautious, preferring proven treatments/ideas than new ones, partly from being burnt by previous “great” ideas (e.g. thalidomide), and due to the risk involved.
      - softwaredoug 7y ago
        In my experience as a patient trying to deal with a chronic illness. Doctors, like many professionals, seem to have a strong preference for methods that seem to work for them. Crucially I don’t think there’s a strong feedback loop for specialists. I was frustrated with my urologist. I just got a 2nd (and 3rd) opinion. But the information of “what you were doing hasn’t been working for me” didn’t make it back to my doctors. In part because it takes months to schedule an appointment. They also probably assume if they don’t see me, it just got better. This is partially because medicine is hard and there’s so much subjectivity in patient experiences. Crucially these specialists are in high demand, so they will find work, even if they really are just marginally effective. Maybe there’s more “feedback” than I see, but That’s just my 2 cents from the patient perspective.
        Engineering-MD 7y ago
        What you say is also true. Often, if patients don’t return we assume it has resolved itself, partly because it sometimes does (especially minor complaints) and partly as pragmatically its hard enough to keep up with current workload. I agree doctors do what works for them, and think both of what’s good for the patient and what they could be sued/penalised for and how to avoid that harm too. I think this sometimes leads to worse outcomes but you have to work in the system you are in. I guess part of the difference between a good doctor and a great one is knowing when to pry more, find out things aren’t working and try something different. Sorry for your bad experiences. I think there are more good doctors than we think out there, it’s just the system and workload ruin it for everyone.
- davycro 7y ago
  Medicine moves fast and it’s impossible to stay up to date on everything. In general I do not incorporate the latest discoveries into my daily practice unless the evidence is absurdly overwhelming, which rarely happens. Usually new therapies have marginal evidence to support them. I prefer to wait until multiple studies support a treatment before it becomes my standard of care, and by that time it’s likely going to be summarized in a Cochrane Review. Should a patient come in with a preference for a treatment they read about on PubMed (which has never occurred in my practice) then I would likely accommodate their preference unless I thought it would put them at serious harm. A patient knowing more than I do about a particular disease happens more than you’d expect, especially if they have a rare chronic disorder. This does not diminish my ego or confidence in my knowledge base.
  - danieltillett 7y ago
    You are a very unusual doctor. I wish there were more like you.
    - deleted 7y ago
      [deleted]
    - jazoom 7y ago
      Doesn't seem unusual to me. I could have written the same thing myself. Most of my doctor colleagues would be similar. Maybe you live in a different part of the world where doctors jump on every latest trend/study, but I'm not sure where that would be. Most of us like to let the water settle first.
      - danieltillett 7y ago
        It is not the first paragraph that makes the OP unusual, but the second.
        jazoom 7y ago
        That also doesn't seem unusual to me. I'd say most doctors recognise their limitations.
abhisuri97 7y ago
Yay. Big plus for keeping search result page load sizes roughly the same.
akinhwan 7y ago
I'm actually part of the team developing the new PubMed. Very curious and interested to know what the hacker news community thinks and feels about their experience. https://pubmed.gov/labs https://pubmed.gov/labs
- erikrothoff 7y ago
  Will RSS feeds work as before?
- seriesf 7y ago
  Happy Entrez user for many years. Please don't break it :-)
- degenerate 7y ago
  Two things that stick out to me: 1) The timeline graph is very cool. However I thought clicking "download csv" would give me a CSV of all the search results, which I was very excited for! I was sad to see it only downloaded a CSV detailing the number of results per year, not the results themselves. 2) Search history (found on the "advanced" search page) is something I am usually against, but on a site like PubMed, it's very nice to have. Great job implementing that in a clear and usable way. I recommend linking to it from somewhere on the basic search page... something like "View your search history" which will also help them discover the advanced page once they land on it.
- westurner 7y ago
  This looks great: I like the search timeline, the ability to easily search for free full-text meta-analyses (a selection bias we should all be aware of), the MeSH term listing in a reasonably-sized font, and that there's schema.org/ImageObject metadata within the page, but there's no [Medical]ScholarlyArticle metadata? I've worked with Google Scholar (:o) [1], Semantic Scholar (Allen Institute for AI) [2], Meta (Chan Zuckerberg Institute) [3], Zotero, Mendeley and a number of other tools for indexing and extracting metadata and graph relations from https://schema.org/ScholarlyArticle https://schema.org/ScholarlyArticle and MedicalScholarlyArticles . Without RDFa (or Microdata, or JSON-LD) in PDF, there's a lot of parsing that has to go down in order to get a graph from the citations in the article. Each service adds value to this graph of resources. Pushing forward on publishing linked research that's reproducible (#LinkedResearch, #LinkedReproducibility) is a worthwhile investment in meta-research that we have barely yet addressed: > http://Schema.org/NewsArticle http://Schema.org/NewsArticle .citation: https://schema.org/citation https://schema.org/citation ... Wouldn't it be great if NewsArticles linked to the ScholarlyArticle and/or Notebook CreativeWorks that they're .about (with reified relations)? > A practical use case: Alice wants to publish a ScholarlyArticle [1] (in HTML with structured data, as a PDF) predicated upon Datasets [2] (as CSV, CSVW JSONLD, XLSX (DataDownload)) with static HTML (and no special HTTP headers). 1 https://schema.org/ScholarlyArticle https://schema.org/ScholarlyArticle 2 https://schema.org/Dataset* https://schema.org/Dataset* > B wants to build a meta analysis: to collect a # of ScholarlyArticles and Dataset DataDownloads; review study controls and data; merge, join, & concatenate Datasets if appropriate, and inductively or deductively infer a conclusion and suggestions for further studies of variance* The Linked Open Data Cloud shows the edges, the relations, the structured data links between very many (life sciences) datasets: https://lod-cloud.net/ https://lod-cloud.net/ . https://5stardata.info/en/ https://5stardata.info/en/ lists TimBL's suggested 5-start deployment schema for Open Data; which culuminates in publishing linked open data in non-proprietary formats that uses URIs to describe and link to things. Could any of these [1][2][3][4][5] services cross-link the described resources, given a common URI identifier such as https://schema.org/identifier https://schema.org/identifier and/or https://schema.org/url https://schema.org/url ? ORCID is a service for generating stable identifiers for researchers and publishers who have names in common but different emails. W3C DID solves for this need in a different way. When I check an article result page with the OpenLink OSDS extension (or any of a number of other tools for extracting structured data from HTML pages (and documents!) https://github.com/CodeForAntarctica/codeforantarctica.github.io/pull/3 https://github.com/CodeForAntarctica/codeforantarctica.githu... ), there could be quite a bit more data there for search engines, browser extensions, and meta-research tools. Is this something like ElasticSearch on the backend? It is possible to store JSON-LD documents in the search index. I threw together elasticsearchjsonld to "Generate JSON-LD @contexts from ElasticSearch JSON Mappings" for the OpenFDA FAERS data a few year ago. That's not GraphQL or SPARQL, but it's something and it's Linked Data. re: "Canada's Decision To Make Public More Clinical Trial Data Puts Pressure On FDA" https://news.ycombinator.com/item?id=21232183 https://news.ycombinator.com/item?id=21232183 > We really could get more out of this data through international collaboration and through linked data (e.g. URIs for columns). See: "Open, and Linked, FDA data" https://github.com/FDA/openfda/issues/5#issuecomment-539296649 https://github.com/FDA/openfda/issues/5#issuecomment-5392966... and "ENH: Adverse Event Count / 'Use' Count Heatmap" https://github.com/FDA/openfda/issues/49 https://github.com/FDA/openfda/issues/49 . With sales/usage counts, we'd have a denominator with which we could calculate relative hazard. W3C Web Annotations handle threaded comments and highlights; reviewing the reviewers is left as an exercise for the reader. Does Zotero still make it easy to save the bibliographic metadata for one or more ScholarlyArticles from PubMed to a collection in the cloud (and add metadata/annotations)? Sorry to toot my own horn here. Great job on this. This opens up many new opportunities for research. [1] https://scholar.google.com https://scholar.google.com [2] https://www.semanticscholar.org/ https://www.semanticscholar.org/ [3] https://www.meta.org/ https://www.meta.org/ [4] https://zotero.org/ https://zotero.org/ [5] https://mendeley.org/ https://mendeley.org/
owaty 7y ago
Great, pubmed and other NCBI resources are long due for a redesign. I also wish they got a shorter url.
- jszymborski 7y ago
  pubmed.gov redirects to the full URL :)
drannex 7y ago
Been using the labs for the better part of a year (over a year?) Incredibly great work!
jszymborski 7y ago
I usually am skeptical of redesigns that look like this because it typically signals that literally hundreds of 1MB JS dependencies are to be downloaded to display simple text. The new pubmed, much like the old pubmed, is speedy! Pages are ~800kb on first visit, subsequent pages visits avg 30kb transferred data in my very unscientific tests. Congratulations on making a great resource greater :)
bransonf 7y ago
Even if you don’t find yourself reading biomedical research on a frequent basis, you should expose yourself to PubMed just to see how much care went into its development. Seriously, PubMed is one of my favorite websites for its simplicity, speed and incredible usability. It has a querying system for hundreds of thousands of articles that makes sense, it saves these queries and allows you to edit them. It allows you to save articles to lists and makes them portable. While it may be easy to just think of this as just another clever website redesign, I can tell you from personal experience that these increases in usability have a non-zero impact on research productivity. And since it’s PubMed, there’s a lot of important research at stake.
kirillbobyrev 7y ago
Really happy to see PubMed improving over time! One thing I'm missing is a recommendation service of articles that might be relevant for me. I used to read a lot of Machine Learning/Math papers on arXiv and I have found arXiv-sanity [0] to be extremely useful, maybe something similar would be great as a part of PubMed functionality - being able to keep track of research which is relevant to the papers I've added to my favorites. Also, I'm slightly confused by the sort order in the search. I found four options for sorting the items while searching: * Default order * Pub Date * Journal * PMC Live Date IIUC "Default order" is not "sort by the number of citations/citation index", but that's what I typically want when I'm trying to figure out what are the most important/influential research in the field. [0]: http://www.arxiv-sanity.com/ http://www.arxiv-sanity.com/
cheviana 7y ago
Thanks a lot for warm words and useful feedback from PubMed Labs team! To make sure your feedback is heard, please use "Feedback button" found in bottom right corner of https://pubmed.gov/labs https://pubmed.gov/labs

120 points by vo2maxer 7y ago

PubMed is an incredibly useful resource. I've found that it is fairly common for my psychiatrist and GP to be relatively unaware of the latest studies and treatments. Of course, reading through abstracts and papers requires a critical eye into the statistics, methodology, and funding. The Cochrane Reviews are similarly awesome.

deleted 7y ago
[deleted]
softwaredoug 7y ago
Same here. It’s pretty shocking how many different opinions you’ll get from doctors. I end up doing my own research quite a lot via public.
- GordonS 7y ago
  And same here too (in the UK), at least within the NHS - it seems doctors train, start working, and then don't spend any time learning about new developments within their field. It's very disappointing. I'm sure it can't be like that for all doctors, but it's been my (unfortunately vast and varied) experience.
  - Engineering-MD 7y ago
    Doctors have to show continued professional development every year for Revalidation of their license. It’s hard to keep on top of all the research, there is so much of it. Much of it of arguable quality, and contradictory making it hard to interpret. Generally people follow guidelines and advice from experts unless they are the expert in that are due to this. And doctors are culturally quite cautious, preferring proven treatments/ideas than new ones, partly from being burnt by previous “great” ideas (e.g. thalidomide), and due to the risk involved.
    - softwaredoug 7y ago
      In my experience as a patient trying to deal with a chronic illness. Doctors, like many professionals, seem to have a strong preference for methods that seem to work for them. Crucially I don’t think there’s a strong feedback loop for specialists. I was frustrated with my urologist. I just got a 2nd (and 3rd) opinion. But the information of “what you were doing hasn’t been working for me” didn’t make it back to my doctors. In part because it takes months to schedule an appointment. They also probably assume if they don’t see me, it just got better. This is partially because medicine is hard and there’s so much subjectivity in patient experiences. Crucially these specialists are in high demand, so they will find work, even if they really are just marginally effective. Maybe there’s more “feedback” than I see, but That’s just my 2 cents from the patient perspective.
      - Engineering-MD 7y ago
        What you say is also true. Often, if patients don’t return we assume it has resolved itself, partly because it sometimes does (especially minor complaints) and partly as pragmatically its hard enough to keep up with current workload. I agree doctors do what works for them, and think both of what’s good for the patient and what they could be sued/penalised for and how to avoid that harm too. I think this sometimes leads to worse outcomes but you have to work in the system you are in. I guess part of the difference between a good doctor and a great one is knowing when to pry more, find out things aren’t working and try something different. Sorry for your bad experiences. I think there are more good doctors than we think out there, it’s just the system and workload ruin it for everyone.
davycro 7y ago
Medicine moves fast and it’s impossible to stay up to date on everything. In general I do not incorporate the latest discoveries into my daily practice unless the evidence is absurdly overwhelming, which rarely happens. Usually new therapies have marginal evidence to support them. I prefer to wait until multiple studies support a treatment before it becomes my standard of care, and by that time it’s likely going to be summarized in a Cochrane Review. Should a patient come in with a preference for a treatment they read about on PubMed (which has never occurred in my practice) then I would likely accommodate their preference unless I thought it would put them at serious harm. A patient knowing more than I do about a particular disease happens more than you’d expect, especially if they have a rare chronic disorder. This does not diminish my ego or confidence in my knowledge base.
- danieltillett 7y ago
  You are a very unusual doctor. I wish there were more like you.
  - deleted 7y ago
    [deleted]
  - jazoom 7y ago
    Doesn't seem unusual to me. I could have written the same thing myself. Most of my doctor colleagues would be similar. Maybe you live in a different part of the world where doctors jump on every latest trend/study, but I'm not sure where that would be. Most of us like to let the water settle first.
    - danieltillett 7y ago
      It is not the first paragraph that makes the OP unusual, but the second.
      - jazoom 7y ago
        That also doesn't seem unusual to me. I'd say most doctors recognise their limitations.

abhisuri97 7y ago

Yay. Big plus for keeping search result page load sizes roughly the same.

akinhwan 7y ago

I'm actually part of the team developing the new PubMed. Very curious and interested to know what the hacker news community thinks and feels about their experience. https://pubmed.gov/labs https://pubmed.gov/labs

erikrothoff 7y ago
Will RSS feeds work as before?
seriesf 7y ago
Happy Entrez user for many years. Please don't break it :-)
degenerate 7y ago
Two things that stick out to me: 1) The timeline graph is very cool. However I thought clicking "download csv" would give me a CSV of all the search results, which I was very excited for! I was sad to see it only downloaded a CSV detailing the number of results per year, not the results themselves. 2) Search history (found on the "advanced" search page) is something I am usually against, but on a site like PubMed, it's very nice to have. Great job implementing that in a clear and usable way. I recommend linking to it from somewhere on the basic search page... something like "View your search history" which will also help them discover the advanced page once they land on it.
westurner 7y ago
This looks great: I like the search timeline, the ability to easily search for free full-text meta-analyses (a selection bias we should all be aware of), the MeSH term listing in a reasonably-sized font, and that there's schema.org/ImageObject metadata within the page, but there's no [Medical]ScholarlyArticle metadata? I've worked with Google Scholar (:o) [1], Semantic Scholar (Allen Institute for AI) [2], Meta (Chan Zuckerberg Institute) [3], Zotero, Mendeley and a number of other tools for indexing and extracting metadata and graph relations from https://schema.org/ScholarlyArticle https://schema.org/ScholarlyArticle and MedicalScholarlyArticles . Without RDFa (or Microdata, or JSON-LD) in PDF, there's a lot of parsing that has to go down in order to get a graph from the citations in the article. Each service adds value to this graph of resources. Pushing forward on publishing linked research that's reproducible (#LinkedResearch, #LinkedReproducibility) is a worthwhile investment in meta-research that we have barely yet addressed: > http://Schema.org/NewsArticle http://Schema.org/NewsArticle .citation: https://schema.org/citation https://schema.org/citation ... Wouldn't it be great if NewsArticles linked to the ScholarlyArticle and/or Notebook CreativeWorks that they're .about (with reified relations)? > A practical use case: Alice wants to publish a ScholarlyArticle [1] (in HTML with structured data, as a PDF) predicated upon Datasets [2] (as CSV, CSVW JSONLD, XLSX (DataDownload)) with static HTML (and no special HTTP headers). 1 https://schema.org/ScholarlyArticle https://schema.org/ScholarlyArticle 2 https://schema.org/Dataset* https://schema.org/Dataset* > B wants to build a meta analysis: to collect a # of ScholarlyArticles and Dataset DataDownloads; review study controls and data; merge, join, & concatenate Datasets if appropriate, and inductively or deductively infer a conclusion and suggestions for further studies of variance* The Linked Open Data Cloud shows the edges, the relations, the structured data links between very many (life sciences) datasets: https://lod-cloud.net/ https://lod-cloud.net/ . https://5stardata.info/en/ https://5stardata.info/en/ lists TimBL's suggested 5-start deployment schema for Open Data; which culuminates in publishing linked open data in non-proprietary formats that uses URIs to describe and link to things. Could any of these [1][2][3][4][5] services cross-link the described resources, given a common URI identifier such as https://schema.org/identifier https://schema.org/identifier and/or https://schema.org/url https://schema.org/url ? ORCID is a service for generating stable identifiers for researchers and publishers who have names in common but different emails. W3C DID solves for this need in a different way. When I check an article result page with the OpenLink OSDS extension (or any of a number of other tools for extracting structured data from HTML pages (and documents!) https://github.com/CodeForAntarctica/codeforantarctica.github.io/pull/3 https://github.com/CodeForAntarctica/codeforantarctica.githu... ), there could be quite a bit more data there for search engines, browser extensions, and meta-research tools. Is this something like ElasticSearch on the backend? It is possible to store JSON-LD documents in the search index. I threw together elasticsearchjsonld to "Generate JSON-LD @contexts from ElasticSearch JSON Mappings" for the OpenFDA FAERS data a few year ago. That's not GraphQL or SPARQL, but it's something and it's Linked Data. re: "Canada's Decision To Make Public More Clinical Trial Data Puts Pressure On FDA" https://news.ycombinator.com/item?id=21232183 https://news.ycombinator.com/item?id=21232183 > We really could get more out of this data through international collaboration and through linked data (e.g. URIs for columns). See: "Open, and Linked, FDA data" https://github.com/FDA/openfda/issues/5#issuecomment-539296649 https://github.com/FDA/openfda/issues/5#issuecomment-5392966... and "ENH: Adverse Event Count / 'Use' Count Heatmap" https://github.com/FDA/openfda/issues/49 https://github.com/FDA/openfda/issues/49 . With sales/usage counts, we'd have a denominator with which we could calculate relative hazard. W3C Web Annotations handle threaded comments and highlights; reviewing the reviewers is left as an exercise for the reader. Does Zotero still make it easy to save the bibliographic metadata for one or more ScholarlyArticles from PubMed to a collection in the cloud (and add metadata/annotations)? Sorry to toot my own horn here. Great job on this. This opens up many new opportunities for research. [1] https://scholar.google.com https://scholar.google.com [2] https://www.semanticscholar.org/ https://www.semanticscholar.org/ [3] https://www.meta.org/ https://www.meta.org/ [4] https://zotero.org/ https://zotero.org/ [5] https://mendeley.org/ https://mendeley.org/

owaty 7y ago

Great, pubmed and other NCBI resources are long due for a redesign. I also wish they got a shorter url.

jszymborski 7y ago
pubmed.gov redirects to the full URL :)

drannex 7y ago

Been using the labs for the better part of a year (over a year?) Incredibly great work!

jszymborski 7y ago

I usually am skeptical of redesigns that look like this because it typically signals that literally hundreds of 1MB JS dependencies are to be downloaded to display simple text. The new pubmed, much like the old pubmed, is speedy! Pages are ~800kb on first visit, subsequent pages visits avg 30kb transferred data in my very unscientific tests. Congratulations on making a great resource greater :)

bransonf 7y ago

Even if you don’t find yourself reading biomedical research on a frequent basis, you should expose yourself to PubMed just to see how much care went into its development. Seriously, PubMed is one of my favorite websites for its simplicity, speed and incredible usability. It has a querying system for hundreds of thousands of articles that makes sense, it saves these queries and allows you to edit them. It allows you to save articles to lists and makes them portable. While it may be easy to just think of this as just another clever website redesign, I can tell you from personal experience that these increases in usability have a non-zero impact on research productivity. And since it’s PubMed, there’s a lot of important research at stake.

kirillbobyrev 7y ago

Really happy to see PubMed improving over time! One thing I'm missing is a recommendation service of articles that might be relevant for me. I used to read a lot of Machine Learning/Math papers on arXiv and I have found arXiv-sanity [0] to be extremely useful, maybe something similar would be great as a part of PubMed functionality - being able to keep track of research which is relevant to the papers I've added to my favorites. Also, I'm slightly confused by the sort order in the search. I found four options for sorting the items while searching: * Default order * Pub Date * Journal * PMC Live Date IIUC "Default order" is not "sort by the number of citations/citation index", but that's what I typically want when I'm trying to figure out what are the most important/influential research in the field. [0]: http://www.arxiv-sanity.com/ http://www.arxiv-sanity.com/

cheviana 7y ago

Thanks a lot for warm words and useful feedback from PubMed Labs team! To make sure your feedback is heard, please use "Feedback button" found in bottom right corner of https://pubmed.gov/labs https://pubmed.gov/labs