3 ms·
> It's a genuinely insightful summary of the last few months of activity in open source models Yes this is an amazing summary! Just for its summary alone, it i
by ftxbro 4y ago
> It's a genuinely insightful summary of the last few months of activity in open source models
Yes this is an amazing summary! Just for its summary alone, it is probably one of the top five writings I saw on LLMs and I read every one!
> because I have trouble imagining anyone writing something this good and deciding NOT to take credit for the analysis themselves.
Not everyone has a substack that they spam onto hacker news every time a thought enter their head. Or imagine that INTP exist. In my opinion the best take ever on LLMs is the simulators essay https://generative.ink/posts/simulators/ https://generative.ink/posts/simulators/ and the author is so shy they went pseudonymous and put their twitter as private and don't even want their name in conferences.
- heliophobicdude 4y agoI think the another great paper is RLHF article from Chip Huyen https://huyenchip.com/2023/05/02/rlhf.html https://huyenchip.com/2023/05/02/rlhf.html
- ftxbro 4y agoThanks for sharing it but I'm sorry I don't agree that it's so important. In my opinion almost every interesting thing about LLMs comes from the raw base model, which is before the RLHF is applied. For example the simulators paper was written before ChatGPT was even released, based on research with the GPT-3 base model that only had text completion and no instruction tuning or any kind of RLHF or lobotomization. In another example, in the interviews with the people who had access to the base model of GPT-4 like the red teamers and the ones at microsoft who integrated it with bing, they consistently explain that the raw base pretrained model has the most raw intelligence which is deadened as they put RLHF and guardrails onto it.