4 ms·
The point is that you can split the work in two: the one you mentioned, creating and training the model, and a separate step of shaping it with reinforcement le
by radu_floricica 4y ago
The point is that you can split the work in two: the one you mentioned, creating and training the model, and a separate step of shaping it with reinforcement learning. Which is quite expensive, because it uses human operators in one of its steps. But apparently you can duplicate easily enough this with an API of an already educated engine, which makes the second step not be a moat anymore.
- Imnimo 4y agoHow expensive do you imagine collecting human feedback data is compared to the cost of doing the original language modelling training?
- cubefox 4y agoBut that's not a new point?! The second, expensive step (RLHF) was already automated by Anthropic with their Constitutional AI approach several months ago.