2 ms·
I'm curious to know what Rad AI ended up doing? IIRC the initial problem was how do you turn this set of radiology notes into some summary radiology notes, with
by rahimnathwani 3y ago
I'm curious to know what Rad AI ended up doing? IIRC the initial problem was how do you turn this set of radiology notes into some summary radiology notes, with a specific format. Is that right?
If you were approaching this problem anew today, you'd probably try with GPT-4 and Claude, and then see what you could achieve by finetuning GPT-3.5.
And, yes, for a given level of quality, the finetuned GPT-3.5 will likely be cheaper than the GPT-4 version. But for radiology notes, perhaps you'd be happy to pay 10x per even if it were to give only a tiny improvement?
- tedivm 3y agoI guess a question to ask is "What is GPT-4". Is it the algorithm, the weights, the data, or a combination of them all? To put it another way, the researchers at Rad AI consumed every paper that was out there including very cutting edge stuff. This included reimplimenting GPT-2 in house, as well as many other systems. However, we didn't have the same data that was used by OpenAI. We also didn't have their hyperparameters (and since our data was different it's not a guarantee that those would have been the best ones anyways). So with that in mind it's possible that Rad AI could today be using their own in house GPT-4, but specialized with their radiology data. In other words them using a specialized model, and them using GPT-4, wouldn't be contradictory. I do want to toss out a disclaimer that I left there in 2021, so I have no insights into their current setup other than what's publicly released. However I have no reason to believe they aren't still doing cutting edge work and building out custom stuff taking advantage of the latest papers and techniques.