pact_inference

3 ms·

1. 1.▲
   A Beginning for Mathematics(proofsandprompts.com)

   28 points by pact_inference 1mo ago|4 comments
2. 2.▲
   Evals Aren't Dead, but They're Low Res(hackbot.dad)

   2 points by pact_inference 2mo ago|0 comments
3. 3.▲
   by pact_inference 1y ago

   whoops, definitely a typo! It should be 5e-4 for as the base "pretraining" LR, you're absolutely correct. your intuition is sound, but my fingers are not.
4. 4.▲
   by pact_inference 1y ago

   > Are you going to rely on the LLM for pseudorandom numbers? Definitely! I'm going to start with instruction tuning it for basic question answering, and then add tools to allow it to search the markdown source to cite answers to rul
5. 5.▲
   by pact_inference 1y ago

   definitely! However, my intuition is that correctly interpreting the rules pulled in context will require some basic understanding of the game system that pretraining would help with. Ultimately after training this base model for instructio
6. 6.▲
   Domain Adaptation of Base Models + ShadowdarkQA Bench(gygaxtest.com)

   17 points by pact_inference 1y ago|6 comments