13 ms·
Despite what people say about ϕ-2, I never liked its responses. It clearly lacks depth and consistency.
by behnamoh 3y ago
Despite what people say about ϕ-2, I never liked its responses. It clearly lacks depth and consistency.
- eightysixfour 3y agoI feel like Phi-2 is all about the context of the fact that it is a <3b parameter model. It is incredible in that context, it isn't incredible in others.
- make3 3y agoit's definitely what it's about. it's insanely strong for its size
- yufeng66 3y agoPhi-2 basically demonstrated that you don't need a very large model to figure out language. It not very smart but speaks perfect English. it's not obvious the best way to gain IQ is to have a larger language model. some other structure might be needed.
- behnamoh 3y agoBut isn't that something that even smaller GPT-2 models demonstrated already?
- erichocean 3y agoNo, and this is covered in the various Phi papers, as well as TinyStories [0]: > Models with around 125M parameters such as GPT-Neo (small) or GPT-2 (small) can rarely generate coherent and consistent English text beyond a few words even after extensive training. [0] https://arxiv.org/abs/2305.07759 https://arxiv.org/abs/2305.07759
- visarga 3y agomaybe it's useful for fine-tuning jobs, not for free prompting I have had a similar bad experience with it trying to prompt it
- m3kw9 3y agoIt’s a model for low rM usage 4gigs? I heard it would work well for RAG to speak to your docs, instead of relying purely on its parameters to answer.
- coder543 3y agoDid you try Dolphin Phi-2? The Dolphin fine-tune seems better to me in this case.