4 ms·
OpenAI’s moat is not _that_ big in the grand scheme of things. You can train GPT2 in a couple weeks on a dozen GPUs. Sure, it’s a lot worse than GPT4, but not
by whiplash451 4y ago
OpenAI’s moat is not _that_ big in the grand scheme of things.
You can train GPT2 in a couple weeks on a dozen GPUs. Sure, it’s a lot worse than GPT4, but not insanely behind and the community is fast.
Same for RLHF. It is for sure fine engineering, but not unreachable for a motivated and competent bunch.