5 ms·
I came across this company called OpenEvidence. They seem to be offering semantic search on medical research. Founded in 2021. How could it possibly keep up wi
by simianwords 9mo ago
I came across this company called OpenEvidence. They seem to be offering semantic search on medical research. Founded in 2021.
How could it possibly keep up with LLM based search?
- dnw 9mo agoIt is a little more than semantic search. Their value prop is curation of trusted medical sources and network effects--selling directly to doctors. I believe frontier labs have no option but to go into verticals (because models are getting commoditized and capability overhang is real and hard to overcome at scale), however, they can only go into so many verticals.
- simianwords 9mo ago> Their value prop is curation of trusted medical sources Interesting. Why wouldn't an LLM based search provide the same thing? Just ask it to "use only trusted sources".
- tacoooooooo 9mo agoThey're building a moat with data. They're building their own datasets of trusted sources, using their own teams of physicians and researchers. They've got hundreds of thousands of physicians asking millions of questions everyday. None of the labs have this sort of data coming in or this sort of focus on such a valuable niche
- simianwords 9mo ago> They're building their own datasets of trusted sources, using their own teams of physicians and researchers. Oh so they are not just helping in search but also in curating data. > They've got hundreds of thousands of physicians asking millions of questions everyday. None of the labs have this sort of data coming in or this sort of focus on such a valuable niche I don't take this too seriously because lots of physicians use ChatGPT already.
- some_random 9mo agoLots of physicians use ChatGPT but so do lots of non-physicians and I suspect there's some value in knowing which are which
- palmotea 9mo ago> Why wouldn't an LLM based search provide the same thing? Just ask it to "use only trusted sources". Is that sarcasm?
- simianwords 9mo agowhy?
- Rygian 9mo agoHow does the LLM know which sources can be trusted?
- simianwords 9mo agoyeah it can avoid blogspam as sources and prioritise research from more prestigious journals or more citations. it will be smart enough to use some proxy.
- palmotea 9mo agoYou can also tell it to just not hallucinate, right? Problem solved. I think what you'll end up is a response that still relies on whatever random sources it likes, but it'll just attribute it to the "trusted sources" you asked for.
- simianwords 9mo agoyou have an outdated view on how much it hallucinates.
- palmotea 9mo agoThe point was: will telling it to not hallucinate make it stop hallucinating?
- 9mo ago
- otikik 9mo agoI don't think you can use an LLM for that. For the same reason you can't just ask it to "Make the app secure and fast"
- simianwords 9mo agoThis is completely incorrect. This is exactly what LLMs can do better.
- sjsjshzhhz 9mo agoSomebody should tell the Claude code team then. They’ve had some perf issues for awhile now. More seriously, the concept of trust is extremely lossy. The LLM is gonna lean in one direction that may or may not be correct. At the extreme, it wound likely refute a new discovery that went against what we currently know. In a more realistic version, certain AIs are more pro Zionist than others.
- simianwords 9mo agoI meant that LLMs can be trusted to do searches and not hallucinate while doing it. You’ve taken that to mean it can comply with anything. The thing is, LLMs are quite good at search and probably way way more strong that whatever RAG setup this company has. What failure mode are you looking at from a search perspective? Will ChatGPT just end up providing random links?
- otikik 9mo agoNo, it is "absolutely right". The chatbots will say they can do it, but they can't. See the Openclaw debacle for a recent example.
- Esophagus4 9mo agoHave you tried it? Or are you just grasping at the latest straw you can find?
- dnw 9mo agoYes, they can. We have gotten better at grounding LLMs to specific sources and providing accurate citations. Those go some distance in establishing trust. There is trust and then there is accountability. At the end of the day, a business/practice needs to hold someone/entity accountable. Until the day we can hold an LLM accountable we need businesses like OpenEvidence and Harvey. Not to say Anthropic/OpenAI/Google cannot do this but there is more to this business than grounding LLMs and finding relevant answers.
- Rygian 9mo ago> We have gotten better at grounding LLMs to specific sources and providing accurate citations And how does the LLM know which specific sources to ground itself to?
- rune-dev 9mo ago> Just ask it to "use only trusted sources". This is pure LLM brain rot. You can’t “just ask” an LLM to be more reliable.
- AlexeyBelov 9mo agoLook at their other comments. They must be trolling at this point.
- olliepro 9mo agoMuch of the scientific medical literature is behind paywalls. They have tapped into that datasource (whereas ChatGPT doesn't have access to that data). I suspect that were the medical journals to make a deal with OpenAI to open up the access to their articles/data etc, that open evidence would rely on the existing customers and stickiness of the product, but in that circumstance, they'd be pretty screwed. For example, only 7% of pharmaceutical research is publicly accessible without paying. See https://pmc.ncbi.nlm.nih.gov/articles/PMC7048123/ https://pmc.ncbi.nlm.nih.gov/articles/PMC7048123/
- simianwords 9mo agoDo you think maybe ~10B USD to should cover all of them? For both indexing and training? Seems highly valuable. Edit: seems like it is ~10M USD.
- Der_Einzige 9mo agobtw - OpenEvidence is also the name that competitive debaters used for their giant archive of policy debate, LD debate, and (small amounts of) PF debate evidence. That project has been going on for decades now. We turned that into a proper, ready-for-use-in-AI dataset and contributed it to the mainstream AI community under the name OpenDebateEvidence. Presented at NeurIPS 2024 Dataset and Benchmark track. https://neurips.cc/virtual/2024/poster/97854 https://neurips.cc/virtual/2024/poster/97854 https://huggingface.co/datasets/Yusuf5/OpenCaselist https://huggingface.co/datasets/Yusuf5/OpenCaselist
- OJFord 9mo agoOpenEvidence does use an LLM. It's ChatGPT for doctors/medical research, tuned to give references in respected journals etc. Hospitals (at least US, Canada, UK) allow and even encourage (US) staff to use it as a quick lookup, the way they'd otherwise Google for a dosage say, it just does that better. (My wife's a hospital doctor & author and introduced me to it; other family in other countries.)
- thepotatodude 9mo agohttps://arxiv.org/abs/2512.01191 https://arxiv.org/abs/2512.01191 See this. I use OpenEvidence. It has access to full text from some of the major medical journals. But generalist models seem to outperform it. Not sure what is going on there.