2 ms·
Eventually better understanding of the mechanisms should allow us to make better models. Right now OpenAI is still exploring the boundaries of "how much does it
by wongarsu 4y ago
Eventually better understanding of the mechanisms should allow us to make better models. Right now OpenAI is still exploring the boundaries of "how much does it get better if we just make it bigger", but that is just the first phase of making LLMs. Remember that transformer models are basically brand new, having been invented just six years ago. There are bound to be lots of improvements left that will later be described as low-hanging fruit.