3 ms·

companies are not publishing anything because they have nothing to publish. not because they are inventing anything relevant. The article is spot on. It is jus

companies are not publishing anything because they have nothing to publish. not because they are inventing anything relevant. The article is spot on. It is just a better-mechanical-turk that spew things that were already written but screwing some words on happy little accidents. The image generation are more impressive because it is harder to judge how those things fall into place. but it's mostly a perception thing versus text/code. Nobody publishes anything because the only edges now are access to better training data (which would be better monetized making a better google at this point) or tweaking the constraints (the new version of big-data-analyst pool of jobs, now titled as prompt-engineer or "scientist" where all you do is add more layers to the opponent model)

ftxbro 3y ago
> companies are not publishing anything because they have nothing to publish This is just very uninformed. Before GPT-4 and the 'AI race' the various LLM-adjacent research was usually done by teams with members in both academia and industry and they published all their details as a point of pride and to get prestige and high value publication venues. After GPT-4 and the 'AI race' this immediately stopped. Here is one example where OpenAI deliberately didn't publish details of GPT-4. Instead of publishing a 'research paper' they published a 'technical report' https://arxiv.org/abs/2303.08774 https://arxiv.org/abs/2303.08774 "Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar" When Google published their rival PaLM 2 results, they followed the OpenAI GPT-4 lead by publishing only a 'technical report' https://ai.google/static/documents/palm2techreport.pdf https://ai.google/static/documents/palm2techreport.pdf that looks mainly at the capabilities of the system and not at the details of 'the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.'

ftxbro 3y ago

> companies are not publishing anything because they have nothing to publish This is just very uninformed. Before GPT-4 and the 'AI race' the various LLM-adjacent research was usually done by teams with members in both academia and industry and they published all their details as a point of pride and to get prestige and high value publication venues. After GPT-4 and the 'AI race' this immediately stopped. Here is one example where OpenAI deliberately didn't publish details of GPT-4. Instead of publishing a 'research paper' they published a 'technical report' https://arxiv.org/abs/2303.08774 https://arxiv.org/abs/2303.08774 "Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar" When Google published their rival PaLM 2 results, they followed the OpenAI GPT-4 lead by publishing only a 'technical report' https://ai.google/static/documents/palm2techreport.pdf https://ai.google/static/documents/palm2techreport.pdf that looks mainly at the capabilities of the system and not at the details of 'the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.'