3 ms·

Website is better, has technical information and much more detail and benchmarks. Claims performance between GPT-3.5 and GPT-4. https://x.ai/ https://x.ai/ The

by wilg 3y ago

Website is better, has technical information and much more detail and benchmarks. Claims performance between GPT-3.5 and GPT-4. https://x.ai/ https://x.ai/ The direct link to the application is https://grok.x.ai/ https://grok.x.ai/

ShamelessC 3y ago
Thanks! The tweet doesn’t even load for me.
- schleck8 3y ago
  And neither did the web app because the infrastructure appears to have crashed after launch under excessive load. Couldn't have seen this coming woth Elon at the Helm of engineering and also tweeting about it to his 100 million followers
calmworm 3y ago
[Sign in with X] … no, thanks.
nojvek 3y ago
You know what they say about LLM benchmarks nowadays? Pre-training on the test set is all you need. https://arxiv.org/abs/2309.08632 https://arxiv.org/abs/2309.08632 Many of the modern LLMs take entire copy of internet which includes the test set for many of these benchmarks. So if someone claims to beat ChatGPT and their model is trained on the test set, ofcourse they’ll do better. Even ChatGPT is likely trained on the test set. Even a hash table will get stellar results if trained on the test set. Their website provides no evidence that it did not train on the test set. Until we get to play with it, their claims are to be taken with a grain of salt.
dragonwriter 3y ago
Claims Grok is better than other models in its compute class, by which it explicitly includes GPT-3. 5-turbo but excluded GPT-4. But how is getting any idea of “compute class”? Neither of the OpenAI models have official published information sufficient to judge this; 3.5 was rumored to be 175 billion parameters, though Microsoft's CodeFusion paper indicated that its a 20 billion parameter model. Lots of people throw around a 1.5 trillion parameter guess for GPT-4, but that doesn't seem to be grounded in anything but speculation (in part itself based on the 175 billion parameter estimate for gpt-3.5-turbo.)

by wilg 3y ago

ShamelessC 3y ago

Thanks! The tweet doesn’t even load for me.

schleck8 3y ago
And neither did the web app because the infrastructure appears to have crashed after launch under excessive load. Couldn't have seen this coming woth Elon at the Helm of engineering and also tweeting about it to his 100 million followers

calmworm 3y ago

[Sign in with X] … no, thanks.

nojvek 3y ago

You know what they say about LLM benchmarks nowadays? Pre-training on the test set is all you need. https://arxiv.org/abs/2309.08632 https://arxiv.org/abs/2309.08632 Many of the modern LLMs take entire copy of internet which includes the test set for many of these benchmarks. So if someone claims to beat ChatGPT and their model is trained on the test set, ofcourse they’ll do better. Even ChatGPT is likely trained on the test set. Even a hash table will get stellar results if trained on the test set. Their website provides no evidence that it did not train on the test set. Until we get to play with it, their claims are to be taken with a grain of salt.

dragonwriter 3y ago

Claims Grok is better than other models in its compute class, by which it explicitly includes GPT-3. 5-turbo but excluded GPT-4. But how is getting any idea of “compute class”? Neither of the OpenAI models have official published information sufficient to judge this; 3.5 was rumored to be 175 billion parameters, though Microsoft's CodeFusion paper indicated that its a 20 billion parameter model. Lots of people throw around a 1.5 trillion parameter guess for GPT-4, but that doesn't seem to be grounded in anything but speculation (in part itself based on the 175 billion parameter estimate for gpt-3.5-turbo.)