3 ms·
> I believe GPT-4 was trained on a server farm, not a single computer. Yes a server farm of Nvidia A100 with supercomputing performance. > It is easy to see h
by xcv123 3y ago
> I believe GPT-4 was trained on a server farm, not a single computer.
Yes a server farm of Nvidia A100 with supercomputing performance.
> It is easy to see how this stuff works. .. you can train a Hidden Markov Model..
No. GPT is not a Markov model.
- YeGoblynQueenne 3y agoA HMM is one way to train a language model that optimises the conditional objective I note above. A transformer is another. Working with a HMM will give you an insight into how language modelling works, and it's something you can do easily and cheaply, unlike training a giant transformer architecture.
- xcv123 3y agoThose primitive obsolete models are not the subject of the article. I already have a degree in applied mathematics and computer science and know the basics of machine learning. My questions to you were rhetorical. I wasn't asking you for academic guidance.
- YeGoblynQueenne 3y agoI've already posted in another comment that I don't think there's a reason for me to insist, but I'd just like to point out that it's OK to not be an expert in everything but one should not have strong opinions on matters not of one's expertise. It's not a shame to not know everything, but it's unseemly to act like one does.
- xcv123 3y agoThe exact same would apply to you. Projection. You are extremely arrogant with your opinions that no one else really agrees with and you talk down to people who disagree. Essentially you are insisting that your own personal definition of "world model" is true and the normal meaning of the term is not. Your argument is totally subjective and philosophical.