3 ms·

You described v1. Just wait for v2. You can’t possibly think these things will remain clowns forever. These things aren’t “Markov chains” - the architecture i

• grbsh 3y ago
  A Markov Chain is a mathematical structure, not a machine learning architecture. If there are a finite number of states, and a function to determine the probability to transition to any given state from any other state, then it’s a Markov chain. Transformers with finite block size have a finite number of states, so they are Markov chains.
• leifross 3y ago
  GPT3 released four years ago, in terms of iterations beyond that, we are at ~v5, and progress has only been incremental relative to that milestone. The transformer models can only be scaled so far before not even VC money can sustain training. I believe we will get there eventually, but transformer based LLMs have been hitting a roof for a long time, and we need to think differently.