3 ms·
Decoder only architecture? What is this? That doesn't sound like a transformer at all, are you saying gpt4 uses a totally different algorithm?
by revelio 3y ago
Decoder only architecture? What is this? That doesn't sound like a transformer at all, are you saying gpt4 uses a totally different algorithm?
- newhouseb 3y agoNope, a decoder only transformer is a variant of the original architecture proposed by Google [1]. All variants of GPT that we know about (1 through 3) all roughly use this same architecture which takes only the decoder stack from the original Google paper and drops the encoder [2] [1] Original Google Paper - https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 [2] Original GPT Paper - https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf https://s3-us-west-2.amazonaws.com/openai-assets/research-co...
- flangola7 3y agoHow can it work without an encoder?