3 ms·
these are awesome projects. thanks for sharing. it would accelerate research so much if LLM accuracy increased on biomedical papers. very much agreed on the p
by panabee 3y ago
these are awesome projects. thanks for sharing.
it would accelerate research so much if LLM accuracy increased on biomedical papers.
very much agreed on the potential to extract signal from paper structures.
two questions if you don't mind:
1. did you post a summary of your chunking analysis somewhere? i'm curious which method maximized accuracy, and which sentence-overlap methods were most effective.
2. do you think general tokenization methods limit LLMs on scientific/biomedical papers?
- dmezzetti 3y agoAppreciate it! > 1. did you post a summary of your chunking analysis somewhere? i'm curious which method maximized accuracy, and which sentence-overlap methods were most effective. Good idea on this but nothing posted. In general, grouping by sections of a paper has worked best (i.e. methods, conclusions, results etc). GROBID is helpful with arbitrary papers. > 2. do you think general tokenization methods limit LLMs on scientific/biomedical papers? Possibly. For vectorization, specifically with medical, I do have this model (https://huggingface.co/NeuML/pubmedbert-base-embeddings https://huggingface.co/NeuML/pubmedbert-base-embeddings) which is a fine-tuned sentence embeddings model using this base model (https://huggingface.co/microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext https://huggingface.co/microsoft/BiomedNLP-BiomedBERT-base-u...). The base model does have a custom vocabulary. In terms of LLMs, I've found that this model (https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca) works well but haven't experimented with domain specific LLMs.