letitgo12345

7 ms·

31.
by letitgo12345 3y ago
Yup. I use GPT for unit test generation and Wizard Coder is nowhere near even GPT-3.5 let alone GPT-4. HumanEval performance is not representative of general model capability.
32.
by letitgo12345 3y ago
Similar to https://arxiv.org/abs/2307.12950
33.
by letitgo12345 3y ago
Some of it is randomness for sure. However the prompts make sense usually intuitively and there's a big deterministic component -- as which prompts work are likely a function of the training data which has a huge overlap between models
34.
by letitgo12345 3y ago
Maybe this indicates that they are finding new text added to the web to be of little use
35.
by letitgo12345 3y ago
For EB1 - probably Oct. For EB2/3 -- likely 15 years or more...
36.
by letitgo12345 3y ago
They ran out of GC numbers for the year. It will probably jump back to 2022 in Oct
37.
by letitgo12345 3y ago
TN visa
38.
by letitgo12345 3y ago
You should look into securing your priority date via EB2-NIW and applying through EB1-B -- that's far easier to get approval for than EB1-A.
39.
by letitgo12345 3y ago
If you're releasing open source models but not profiting from them, how are you competing with OAI. I don't think that's a violation of ToS.
40.
by letitgo12345 3y ago
Probably legal if you use third party collected data (as you personally haven't agreed to OAIs ToS and AI generations can't be copyrighted so aren't owned by OAI) but I guess corps are too wary to risk a court battle. Directl
41.
by letitgo12345 3y ago
I'm talking about RLHF on their chatbot data, not ChatGPT data on the web.
42.
by letitgo12345 3y ago
? adding on ChatGPT data into existing data is not lessening the amount of data...
43.
by letitgo12345 3y ago
the idea is it wld learn how to do reasoning better based on this
44.
by letitgo12345 3y ago
FTC needs to declare that any company using copyrighted data cannot forbid its users from using its API to develop competing products
45.
by letitgo12345 3y ago
It's been literally just 3 months since GPT-4 was released. I think you're gonna see a lot of changes especially once GPT starts being trained on ChatGPT data
46.
by letitgo12345 3y ago
OAI revenue is 100m+ at least right now iirc and probably gonna grow quite a bit
47.
by letitgo12345 3y ago
Several people in the community have confirmed it so while we don't "know know", it does seem like the dominant plausible rumor going around
48.
by letitgo12345 3y ago
The next iteration will be trained on your own data where "when it's a little wrong, you (the expert) can fix the issue and move on without friction" so that case will become "when it's right" and some amount o
49.
by letitgo12345 3y ago
The optimal way right now seems to be to train a gigantic model to get strong capabilities and use distillation to make a much smaller model stronger. But you seem to still need that gigantic model and we still don't know if we're
50.
by letitgo12345 3y ago
What's 16 iter inference?
51.
by letitgo12345 3y ago
Otoh, that ad driven navel gazing ultimately delivered Generative AI which is likely going to lead to superhuman AI. What equivalent value did HFT deliver?
52.
by letitgo12345 3y ago
Doesn't the Stack contain HumanEval? So you're basically comparing numbers on the pretraining data.
53.
by letitgo12345 3y ago
PPO for RL. Plus a lot of the people behind the innovations you mention are now at OAI - Lukas who was one of the Transformer authors, Bert paper author, chain of thought prompting author, etc. But ya their strong point is execution and doi
54.
by letitgo12345 3y ago
They present it as an article about transformers in general, not ones using Flash Attention. Anyway maybe they're presenting per token memory requirement instead of the requirement for the entire sequence at once.
55.
by letitgo12345 3y ago
Shouldn't the memory needed scale quadratically with sequence length rather than the linear scaling they have in their equations?
56.
by letitgo12345 4y ago
Difference being that Generative AI is something you can actually make money from as opposed the old voice tech
57.
by letitgo12345 4y ago
Good chance so is OAI's https://arxiv.org/abs/2110.13711 (paper has the long context lead at OAI as a co-author)
58.
by letitgo12345 4y ago
That's possible ya. Hope it happens
59.
by letitgo12345 4y ago
Asking for your preferences to figure out your product choices or searching for a product for you based on your given requirements is a lot more than just FAQs and Q&As. You can even imagine a fancier version where you describe say what
60.
by letitgo12345 4y ago
1. ToS make it hard for a commercial entity to do so. So some third parties would have to collect the data first 2. You won't be able to get the hundreds of millions or more interactions that OAI will have (both due to cost of API as w

More ›