3 ms·
This is secret sauce that people have been hoarding for the last year or so. With the deepseek open source releases this is now worth a lot less and companies
by noosphr 2y ago
This is secret sauce that people have been hoarding for the last year or so.
With the deepseek open source releases this is now worth a lot less and companies are cashing out on reputational increases instead of being scooped.
I have done this exact thing in September 2023 with llama2 finetunes but couldn't get approval to share it with anyone.
- EMIRELADERO 2y agoInteresting! What results did you get with that? Also, do you think this is what O3 is doing?
- noosphr 2y agoAbove sota on logical reasoning for grounding text. LLMs at the time were so bad at it that even fornier models when given A, not B would derive A and B in the output about half the time.
- bloomingkales 2y agoWhat do you use to benchmark?
- noosphr 2y agoIt was a SAT solver with the symbolic expression fed into the LLM. Using the SAT solver I'd rate how well the LLM was able to solve a given boolean formula, if it got it wrong it would get a second prompt to ask it to split the boolean expression at the main connective into two sub expressions. If it got that right it would be asked to solve the sub expressions etc.. Then reward/punish based on how well it did overall. There was meant to be a lot more to the system than that, but I never had the training budget to do anything but the first draft of the system.
- bloomingkales 2y agoI was hoping for something more reproducible. Free work, I know.
- noosphr 2y agoThis was done as part of a contract and like I said I never got permission to write a paper about it, let alone share code.
- bloomingkales 2y agoNot even a hint?
- NitpickLawyer 2y ago> couldn't get approval to share it with anyone. sounds like MS :( They had some killer research projects with various teams around the world, but eventually they all got snuffed.