4 ms·
Problem being its not a Research Paper, which they where doing previously. This is very bad state as you're not detailing anything that external parties can rec
by amrb 4y ago
Problem being its not a Research Paper, which they where doing previously.
This is very bad state as you're not detailing anything that external parties can recreate or prove the scientific method.
They can exclaim the model says 40% less "xbox live gamer words" which people outside the company couldn't validate.
tl:dr OpenAi is now a business
Worth watch Yannic talk about the problem and other cool ML topics too.
https://www.youtube.com/watch?v=2zW33LfffPc https://www.youtube.com/watch?v=2zW33LfffPc
- siva7 4y agoIt's not like a closed model only available to scientists you can't benchmark yourself. Benchmarking should also be done by a 3rd party otherwise we have a conflict of interest.
- amrb 4y agoIf this was a cpu/graphic cards sure lets benchmark it, worst case you getting less frames. Here we'd need to see more about its design and safety, else you may be getting recipes for veggie dishes when what you really wanted was fried chicken.
- brookst 4y agoHow would knowing the architecture or safety mechanisms help you decide if it’s going to give incorrect results more than actual testing would? I’m no LLM expert, but I don’t think you can eyeball the arch and say “that’s going to confuse veggies for fried chicken”.