3 ms·
Other than by affecting the output length the complexity of the input won’t affect how likely it is to succeed, as each token always takes a fixed time to gener
by circuit10 3y ago
Other than by affecting the output length the complexity of the input won’t affect how likely it is to succeed, as each token always takes a fixed time to generate and there’s no way for the AI model itself to crash
- hiddencost 3y agoAt sufficient scale everything becomes a daily occurrence. Hardware failure e.g.
- circuit10 3y agoYes, but it’s not related to the complexity of the input
- buildbot 3y agoGiven deterministic everything sure, but nothing really is. Perhaps GPU 7 has a very toasty neighbor and is slower.
- circuit10 3y agoWhat I’m saying is it’s not correlated to the complexity of the input, because GPT models don’t have a built in way to say “let me stop and think about this”, it’s a fixed amount of computation per token