5 ms·
> Satisfying HIPAA rules around patient privacy alone may take many years, or decades, to resolve themselves before we could even contemplate directly using pro
by amstan 3y ago
> Satisfying HIPAA rules around patient privacy alone may take many years, or decades, to resolve themselves before we could even contemplate directly using programs like ChatGPT in a medical theater.
This is easily solved by not using someone else's API.
Here's $500 worth of hardware you can get on ebay running Vicuna-13B, locally:
> IV fluids may not be the best treatment for someone with severe pulmonary edema and respiratory distress because it could make their condition worse. This is because when a person has severe pulmonary edema, their lungs are already filled with fluid, which makes it harder for them to breathe. Adding more fluid to their body could increase the pressure in their lungs and make it even harder for them to breathe. This could be life-threatening.
> Instead, the best course of action would be to focus on treating the underlying cause of the pulmonary edema, such as heart failure or a lung infection. This may involve giving medications to help the heart pump more effectively or to reduce inflammation in the lungs. The patient may also need oxygen therapy to help them breathe more easily.
> It's important to remember that every case is unique, and the best course of treatment will depend on the individual patient's condition. The healthcare team will do their best to provide the most appropriate care for the patient based on their specific needs.
- cced 3y agoHow much time does this take to get produced locally?
- smoldesu 3y agoI run a similar Vicuna bot on a free ARM VPS from Oracle. Inference takes ~20 seconds for ~200 tokens on the CPU, so it should stream results about half as fast as ChatGPT. ...however, that was on cheapo ARM hardware. On a hospital budget I bet you could beat OpenAI with a local model pre-mapped in GPU memory on a 3090.
- cced 3y agoDo you guys have any reference material on how to get this started?
- smoldesu 3y agoJust one guy, me :) But there are a lot of different routes to go. If you want to do what I've done though - 1. Sign up for Oracle Cloud and try to get their free 4 core ARM VPS (these are at-capacity very often, but free if you can get them) 2. Install the Ampere Pytorch runtime: https://cloudmarketplace.oracle.com/marketplace/en_US/adf.task-flow?tabName=O&adf.tfDoc=%2FWEB-INF%2Ftaskflow%2Fadhtf.xml&application_id=125935163&adf.tfId=adhtf https://cloudmarketplace.oracle.com/marketplace/en_US/adf.ta... (can also use the ONNX or Transformers acceleration image if you need it) 3. Install your Pytorch/ONNX/Transformers software, link in system libraries and enjoy! Whole thing works super smooth in my experience. 7B and 13B models are very usable for chatbot type applications.
- amstan 3y agohttps://github.com/lm-sys/FastChat https://github.com/lm-sys/FastChat The hardest part is downloading the 30GB of weights.
- thatcherc 3y agoNot OP, and I'd have to see the prompt to run it on my set up, but I but together a refurbished HP Chromebox with some extra RAM and an SSD for a total of $250 and it'll run all the 13B models about as fast as I can type on my phone, just as a reference point. I'm consistently impressed by it even if it's not blazing fast.
- cced 3y agoDo you guys have any reference material on how to get this started?
- doug_durham 3y agoYes, but this is adding in treatments not listed in the original prompt. There was nothing mentioned about treating heart failure or reducing immflamuation. Vicuna is extrapolating here and not simplifying and making the message more compassionate. It is an interesting application of local inference though.
- aiappreciator 3y agoYou really, really don't want to use inferior models where accuracy is a life or death issue.
- Narkov 3y agoIn this example, it is not a diagnosis tool. It is a communication tool with "stakeholders".
- dr_dshiv 3y agoAnthropic announce HIPAA compliance yesterday.