3 ms·
How many transformers are in a typical LLM? Or is the whole thing considered a transformer?
by bilsbie 3y ago
How many transformers are in a typical LLM? Or is the whole thing considered a transformer?
- dfgtyu65r 3y agoNormally, the LLM is composed of multiple transformer blocks, where each block consists of the (mutli-head) attention and fully-connected feedforward components. These are then stacked on top of each other to give the final output of the network.
- jerpint 3y agoThe architecture as a whole is referred to as a transformer (autoregressive decoder-only architecture in the case of gpt style LLMs). Note that there can be other types of LLMs too that are not necessarily transformer based (SSM, RNN, etc)