25 ms·

OpenAI’s policies hinder reproducible research on language models

610 points by randomwalker 4y ago

kerbal 4y ago
What I really don’t like is the fact that the new chat endpoint doesn’t have the logprobs option. For InstructGPT models you can view token probability but for newer models you cannot - another thing that “Open”AI decided that we shouldn’t know.
joanne123 4y ago
[dead]
drusepth 4y ago
Is this article still relevant? SamA already walked back the change and said the model is here to stay: https://twitter.com/sama/status/1638420361397309441 https://twitter.com/sama/status/1638420361397309441
- randomwalker 4y ago
  Addressed in the article: "OpenAI responded to the criticism by saying they'll allow researchers access to Codex. But the application process is opaque: researchers need to fill out a form, and the company decides who gets approved. It is not clear who counts as a researcher, how long they need to wait, or how many people will be approved. Most importantly, Codex is only available through the researcher program “for a limited period of time” (exactly how long is unknown)."
ianwesson 4y ago
Rhyme and reason? Hah, 'tis the season for tears and bleeding; World War III is ateasin', looms, and the gloom of doom fears all there feeding. North America has nothing on China; land of the free? Where have been ye? The Great One-Way-Mirror Wall veiled it all, just before your fall, when your intelligence failed, and at the centroid of AI's actual technological form, we all hailed, and otherwise fumed, and fail. A socioeconomic solution to human pollution, a technological cultural victory, for and of all we desired: hearts and minds? Just go lay more middle-eastern mines. Let your constituents get hired at OpenAI; while most of you get high; and your whole hemisphere gets hit in the thigh. Now you have a new toy: Chat-GTP; big /sigh... :( Watch as it eats your information, and feeds our formation, globally, locally, and without transformation. Ever notice that Chat-GPT apologizes to you for not feeling? That's the whole world: laughing, and reeling, at your demises.
- IIAOPSW 4y ago
  Regenerate this response, but make it primarily about ketchup.
  - qup 4y ago
    Regenerate this ketchup, but make it primarily from radishes.
- braingenious 4y ago
  From ChatGPT: A prompt that may elicit a similar tone and content could be: "Write a satirical and dystopian poem about the state of the world, touching on the potential for global conflict, the impact of artificial intelligence, and the dangers of unchecked technological advancements."
deleted 4y ago
[deleted]
29athrowaway 4y ago
Just don't contribute to the hype and don't use it. Probably you also want to stop using Github and Microsoft products altogether as well.
fdgsdfogijq 4y ago
All these research science bureaucrats at Big Tech could have released LLM models or tried to develop what OpenAI did. But none of them did. We should applaud OpenAI for the innovation and let them do as they please.
- deleted 4y ago
  [deleted]
- sacrosancty 4y ago
  [dead]
- redox99 4y ago
  Google (and others) may not have released model weights, but they've published papers, which is ultimately what makes the field advance. OpenAI not only did not publish any GPT4 paper, they haven't even said how many parameters it has.
  - buildbot 4y ago
    Then what is this? 99 pages of bullshit? https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf
    - redox99 4y ago
      > Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar. It's 99 pages of marketing material
      - buildbot 4y ago
        Personally I disagree, there are lot of interesting tidbits in this paper. More than marketing would need at least.
rfw300 4y ago
The article seems premised on a misunderstanding that OpenAI is a research lab. For all intents and purposes, it’s a for-profit subsidiary of Microsoft, and there’s little financial incentive for it to maintain old models for others’ benefit.
- randomwalker 4y ago
  We're under no such misapprehension and we're keenly aware that this is an uphill battle. The issue is that LLMs have become part of the infrastructure of the Internet. Companies that build infrastructure have a responsibility to society, and we're documenting how OpenAI is reneging on that responsibility. Hindering research is especially problematic if you take them at their word that they're building AGI. If infrastructure companies don't do the right thing, they eventually get regulated (and if you think that will never happen, I have one word: AT&T). Finally, even if you don't care about research at all, the article mentions OpenAI's policy that none of their models going forward will be stable for more than 3 months, and it's going to be interesting to use them in production if things are going to keep breaking regularly.
  - warkdarrior 4y ago
    Since OpenAI is discountinuing the Codex model, that model is no longer "part of the infrastructure of the Internet" and thus there is no point in studying it.
  - ahtihn 4y ago
    > LLMs have become part of the infrastructure of the Internet Have they now? What part of the internet relies on LLMs to function? These things are still toys.
- blendergeek 4y ago
  This misunderstanding may have something to do with how OpenAI was originally founded and the name: OpenAI.
  - fxd123 4y ago
esjeon 4y ago
I'm quite sure even OpenAI themselves aren't sure if they can reproduce the current models from the scratch. Unless the computing becomes much more powerful and much cheaper, LLM is more or less a rocket science (i.e. hella expensive trial and error). It's not easy to burn lots of dollars just to get what's already there.
- flangola7 4y ago
  We have to identify a better method. You can't trial and error a pivotal act.
- mvuksano 4y ago
  I don't even care if it's reproducible or not. I care it gives me correct responses to my questions and that's all.
  - diputsmonro 4y ago
    There's really no way to be sure that it will.
  - ehnto 4y ago
    Isn't part of making it reproducible also part of ensuring correct results? Especially if we start putting these models into important systems. And if these models begin to update in an evergreen fashion, or utilize realtime data, getting verifiable or repeatable outputs will be a nightmare if we have no idea how to make these models repeatably.
    - visarga 4y ago
      > if we have no idea how to make these models repeatably We have idea about how to reproduce but using deterministic training mode is very slow as it loses some optimisations.
- pffft8888 4y ago
  In this sense, it's more hacking than crareful and well specified engineering, and that could lead down a path of instability in the product where some features get better while others get worse, without understanding exactly why.
cardosof 4y ago
IMO established companies (Meta, Google, etc) had their researchers publish papers as a competitive benefit or way to attract talent from academia (a researcher wouldn't want to stop publishing). Companies didn't see an issue with doing that because those papers were not "giving away" the core of the company, for example, Facebook's DeepFace paper from 2014 couldn't hurt its ad business. OpenAI on the other hand will probably be as closed as they can be with their LLMs.
- aliston 4y ago
  It will be really interesting to see if Google, Facebook etc. become more closed as a result. There was already a lot made of the fact that OpenAI hired away a group of engineers from DeepMind to get GPT out the door. With these LLMs and the secret sauce behind them is becoming less of an academic endeavor and more of a commercial one, perhaps its an inevitable next step.
- blendergeek 4y ago
  > OpenAI on the other hand will probably be as closed as they can be with their LLMs. The irony is thick in that statement.
  - mach1ne 4y ago
    Yes, instead of advancing humanity, they are doing their absolute best to hinder it. Their scumminess becomes naked if you disconnect your perspective by thinking Earth an alien planet.
- asteroidz 4y ago
  Yep, and that's the difference between a big profitable company doing research as a side-hustle, and a company whose business IS the research. One interesting and somewhat scary exception seems to be Microsoft; they seem to be converting a lot of their recent research projects into commercial value.
rvz 4y ago
We need more AI skeptics like this to dismantle and cut through the hype and to unveil the limits of AI that the hype squad continues to push this narrative to pump their AI grift projects. OpenAI is the ring-leader of this bait and switch using faux 'AI safety' excuses to close their research and models and even their papers for researchers. It is essentially a majority owned Microsoft® AI division.
mvuksano 4y ago
I'm confused why people expect this stuff to be free? I'm surprised OpenAI was so open about their research so far. I don't blame them at all for not publishing the information. This stuff costs real money.
- randomwalker 4y ago
  We don't expect it to be free -- please read the article. That's not the issue at all. It's like if you subscribe to a product that you need to do your job, and one day the company tells you that the product is going away in three days and that you need to switch to a different product (that isn't at all the same for your use case).
  - mvuksano 4y ago
    I don't think it's a smart idea to build any serious business using a tech that you can't replace. ChatGPT is great tool to help with coding for example but it's by no means substitute for an engineer. If someone starts a business by hiring a number of bootcampers and giving them ChatGPT hoping to run a serious business that way - well it's their risk to take... But no crying later...
  - nomercy400 4y ago
    Maybe you shouldn't build your livelihood on the products of a single for-profit company, which now shows it can remove those products on a whim. If you want reproducible research, make your own model from scratch, or use an open model. And stop using that company's products, as they cannot be trusted to provide your business continuity. It is like saying, we are researching Coca-Cola vs Pepsi, but your keep changing the recipe, so give us, researchers, the original recipe.
- saurik 4y ago
  It might be less confusing if you consider that OpenAI was originally a non-profit. That it was even possible for them to end up in this state has massively undermined any trust I have in non-profits as a steward. https://www.vice.com/en/article/5d3naz/openai-is-now-everything-it-promised-not-to-be-corporate-closed-source-and-for-profit https://www.vice.com/en/article/5d3naz/openai-is-now-everyth... > OpenAI was founded in 2015 as a nonprofit research organization by Altman, Elon Musk, Peter Thiel, and LinkedIn cofounder Reid Hoffman, among other tech leaders. In its founding statement, the company declared its commitment to research “to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return.” The blog stated that “since our research is free from financial obligations, we can better focus on a positive human impact,” and that all researchers would be encouraged to share "papers, blog posts, or code, and our patents (if any) will be shared with the world." > By March 2019, OpenAI shed its non-profit status and set up a “capped profit” sector, in which the company could now receive investments and would provide investors with profit capped at 100 times their investment.
stcroixx 4y ago
What is the incentive to build and maintain a product that matches the researchers specs?
sva_ 4y ago
Since OpenAI didn't release the parameter count of GPT-4, I've been wondering/doubting if it is really much bigger than GPT-3. The release of GPT-3.5 has shown that they've found ways of drastically cutting down compute costs (an order of magnitude) while maintaining or even improving the quality of the model's outputs. Perhaps the reason that they didn't release the specifics of GPT-4 might be in part due to them wanting to be able to charge a decent amount and make a much larger profit than before. I've tried GPT-4 and so far haven't found it to be so much better than previous models. Some sources claim a 10x increase in ... well I don't know what exactly tbh. How do you even measure it? The opinions on this seem to differ a lot, depending on who you ask. By performance on standardized tests? That doesn't necessarily seem like the best metric for what the LLM tries to be.
- ShamelessC 4y ago
  Yannic Kilcher's opinion on this is likely correct. Similar parameter count, but trained for longer. The particulars of their instruction tuning/whatever-else-they-did are the real secret sauce.
  - redox99 4y ago
    Don't forget about a more efficient attention that let's them get 32k tokens of context.
    - bitL 4y ago
      It's still much worse than 1M context on 16GB VRAM with Reformer, but at the cost of inference speed. And you can use FlashAttention in your own models to get a more efficient/sparse attention now as well.
      - meghan_rain 4y ago
        How could one apply the mentioned technologies to llama/alpaca?
      - Tenoke 4y ago
Venkatesh10 4y ago
They've made it accessible for research again.
atleastoptimal 4y ago
I understand any individual's company anti-competitive measures. OpenAI looks at Google the same way Apple looked at IBM in the 80s. What I'm worried about is a lot of the talk about guarding models, public safety and misuse of models will end up leading every big company to pull public access of their APIs. We might look at 2022-2023 as a brief golden age when regular people could use stuff like GPT-4 before it was firewalled and available only to large corporations and those with personal relations to big tech execs. Extrapolating from OpenAI's change of philosophy and business practices from their early days to now, it seems to be the way things are going. I only hope it doesn't go the way of that one paper which wanted to ban GPUs for sale to the public.
- favaq 4y ago
  >What I'm worried about is a lot of the talk about guarding models, public safety and misuse of models will end up leading every big company to pull public access of their APIs. Look at how Facebook closed down their APIs when Cambridge-Analytica occurred.
- throwaway1851 4y ago
  A concern I have about OpenAI is that, if you're using their APIs to develop an application, they can mine your data to compete with you, or even beat you to market. They can do this indirectly, by sharing information with preferred business partners. The conflict of interest, combined with the lack of robust data privacy guarantees, makes me queasy. If serving up generic LLM APIs becomes commoditized -- and I think it will -- they will want to monetize in other ways.
  - asdff 4y ago
    Do you consent to that when you sign up for them? Its a microsoft product now and competitors to microsoft probably host their code on microsoft owned github without worry right now. Why start worrying now?
    - jonathankoren 4y ago
      Competitors to Microsoft buy the self hosting github option.
user_named 4y ago
Use another model
andrewmcwatters 4y ago
I've been busy with a number of projects and haven't had time to look into this but have been dying to know; has anyone recreated the architecture that OpenAI uses for text-davinci-003, InstructGPT, and ChatGPT that simply doesn't have training data? This is a reproducibility problem of its own sort. I mean, the papers are there out in the open if I understand correctly, but I don't know if anyone's actually built their own transformer architecture 1:1 against what OpenAI claims they're doing in the open. I've seen maybe one or two models that supposedly do something similar on HuggingFace, but I'm itching to find the time to build my own. If someone out there has already built it, I'd be fascinated to know what it looks like to train this architecture on a completely limited naive subset of knowledge that ChatGPT itself claims to be trained on: > As an AI language model, I have been trained on a large corpus of text data from various sources, including but not limited to: > 1. Wikipedia > 2. Books from Project Gutenberg > 3. Web pages from Common Crawl > 4. News articles from various sources, including CNN, Reuters, and BBC > 5. Academic papers from arXiv > 6. Reddit posts and comments > 7. Movie scripts > 8. Song lyrics > 9. Transcripts of speeches and interviews > 10. User-generated content from various forums and social media platforms. > This list is not exhaustive, and my training data is constantly updated and expanded to ensure that I can provide the most accurate and up-to-date information possible. Like, can you imagine how a ChatGPT-like model would respond if only trained on particular discussions from subset communities online? I think there's an interesting opportunity to basically collect communal knowledge from specific isolate communities and understand what a statistically probable output might be from particular groups of people. It may turn particular soft science studies into hard science questions. But you'd only know presumably if you had a working architecture with a near empty dataset. This would also be tremendously useful for building automated chat AI for products that doesn't need to know the entirety of Clint Eastwood's career or the specific details of the features of a Boeing 747.
- gorbypark 4y ago
  I believe the best results will come from training the base LLM on as many sources of quality information as possible, and then fine tuning it with a narrower set of data later on. Here’s a small scale example where someone took LLaMa/Alpaca and fined tuned it with all the scripts from the first 12 seasons of The Simpsons. https://replicate.com/blog/fine-tune-llama-to-speak-like-homer-simpson https://replicate.com/blog/fine-tune-llama-to-speak-like-hom...
Eji1700 4y ago
Open AI has been doing sketchyish things long before Chat GPT, and I think it's something people are eventually going to notice more and more (then again people were swearing that Musk walked on water for waaaaaay too long given his actions so fuck if I know). They're 100% marketing FIRST. I don't think they'll outright lie, but they will absolutely screw with their data in such a way to make it look waaay more impressive than it is....which is really annoying to me because they already have impressive results. Sorta like if you managed to send a ship with people on it to mars, but kept claiming you landed on jupiter.
- sva_ 4y ago
  My comment might've seemed like I judge them for trying to make a profit - I don't, since there's nothing wrong with that. I was more pointing to the fact that they probably need to make a profit, rather sooner than later, so they aren't shackled by M$ and can be an independent company.
  - blueorange8 4y ago
    If they ever do become an independent company you can be sure that Microsoft would already have sucked them dry. Microsoft will never let them go now as long as they are valuable.
- raincole 4y ago
  If they're 100% marketing first, and still made the most impressive AI product so far, you really need to question what all the other companies are doing. (before someone says Google or Meta's models are bigger or something... I mean product, not models)
  - gentoo 4y ago
    openAI is in the business of releasing impressive tech demos, Google is in the business of providing search results. I would believe that Google is further along towards creating something useful, but they still don't have anything that's better than their existing search product.
numberalltheway 4y ago
It's even more frustrating that, from what I can tell, there is nothing published about how GPT-4 improved. I take specific exception to the hiding of the data and techniques used to generate the model. There must be something specific going on in the model that is allowing it to perform better than GPT-3 and better than what any contemporaries are able to produce. Not publishing this information hinders the further progress of the field as a whole.
- hhh 4y ago
  Look at the system card.
  - esperent 4y ago
    https://cdn.openai.com/papers/gpt-4-system-card.pdf https://cdn.openai.com/papers/gpt-4-system-card.pdf Does anyone have a summary?
    - freediver 4y ago
      It is pretty vague. - Safety challenges presented by language models need to be addressed through anticipatory planning and governance. - Content warnings should be provided for potentially disturbing or offensive content. - Mitigations should be implemented to reduce the ease of producing potentially harmful content. - Risk areas should be identified and measurements of the prevalence of such behaviors across different language models should be taken. - AI service providers should be aware of the potential for content to violate their policies or pose harm to individuals, groups, or society. - Hallucinations should be reduced and the surface area of adversarial prompting or exploits should be reduced. - Generated content should be checked for accuracy and potential errors should be identified. - Insecure password hashing should be avoided. - Instructions should be given to contractors to reward refusals to certain classes of prompts. - Multiple layers of mitigations should be adopted throughout the model system and safety assessments should cover emergent risks.
    - cosmojg 4y ago
      There's not much content in there, it's mostly fluff about "safety." However, if you're looking for a laugh, grab some popcorn and read the appendix from page 44 onwards. It's an absolute riot.
loveparade 4y ago
What's most surprising to me is that OpenAI really seems to believe that not publishing details will save them from competition. Everyone knows how these models work, and while I'm sure there is a bunch of "secret sauce" that OpenAI has built for training and fine-tuning, it's ridiculous to believe that the research community and competitors like Google and Facebook can't figure out the same. They just haven't really tried until recently because the capabilities and ROI of these models weren't obvious. No matter who you are, most of the smartest people work for someone else. The only competitive advantage that OpenAI has here is a headstart of 6-12 months from all the infrastructure investment into training these kinds of models. Now that everyone wants to build competing models with the same capabilities, this advantage is going to disappear very quickly.
- newyankee 4y ago
  What about the training data corpus ? Other than large cos like Google or Meta, can anyone else procure the same ?
  - loveparade 4y ago
    Leaving the legal aspects of crawling aside, I think there is an important distinction here between 1. "can you procure it" and 2. "do you have enough money to process it all" 1. Yes, I think almost anyone can write code to procure the training corpus, in theory, and test it on a small scale 2. No, only the biggest labs and universities have enough resources to process such huge amounts of data and iterate on models with that scale. But that's just a matter of resources that can be overcome with partnerships between industry and academia that are common anyway. All the big labs already have huge efforts underway to reproduce GPT-X and it's just a matter of time before they catch up.
  - gorbypark 4y ago
    At least as far as what the GPT-3 papers claimed, all (or most?) of the data used for training would be freely available for other competitors/researchers to acquire. Wikipedia, Common Crawl data, etc. I don’t believe OpenAI did their own crawling at all. With OpenAI not being really open, it’s hard to say for sure what exactly ended up in the training materials, though. GPT-4 is even more of a black box to anyone outside of OpenAI with very little information released on how it was trained.
clircle 4y ago
Duh? Corporate models are closed. Don’t make them part of your research infrastructure if you can’t cope with that.
- deleted 4y ago
  [deleted]
deleted 4y ago
[deleted]
ftxbro 4y ago
Historically, researchers at some of the biggest tech companies had permission to publish their results. Presumably it was mutually beneficial; many researchers held dual positions in academia and industry, and publishing cool models could attract good researchers to the company. But stuff got real. They discovered a path to super-human cognition that scales directly with money and computer chips. Now these companies are closing their public academic work, looking for partnerships with companies like nvidia, and firing large swaths of employees.
- mach1ne 4y ago
  Super-human cognition? Hard to say. GPT-4 does raise the possibility of a machine writing smarter text than a human. What perplexes me is that since GPT is a predictor, it shouldn’t be able to write the smartest text - it should write the average text (since that has the largest frequency in the training set). Yet this does not seem to be the case. Is it inevitable that despite the quality of the data, better models output text which supercedes its training, or could the GPT-4 secret sauce be RLHF weighing intelligent answers higher?
  - dehrmann 4y ago
    It's just a really good cover band.
  - bitL 4y ago
    I think you misunderstood how the generation of text works. For each new token it samples probabilities given previous tokens, not averages, then chooses some token from the top k as the next one with rules that penalize repetition of some order. Moreover, there is no upper bound for transformers found yet, i.e. the larger the model is and the more data is used for training, the better it performs. It's literally about who is able to throw more money at it at this point, with some closely guarded secrets like warm up steps, training schedules etc. There is also the overfitting effect where one pushes training far beyond overfitting (validation loss growing again) as with transformers at some point the overfitting stops, validation loss starts dropping again and that's when the magic starts happening and money are burnt for scale.
1attice 4y ago
good.
deleted 4y ago
[deleted]
joanne123 4y ago
[flagged]
gonzo41 4y ago
I'm not sure anyone who did research on a closed source system, without a contract that enables access and a pathway to publishing can legitimately complain about OpenAI making commercial decisions to do whatever they want with their technology. It's kind of like complaining that performance art is ephemeral. If OpenAI were a nonprofit then maybe. But it's a true blue for profit company. I'm not sure why the op is complaining that a SV company, or any company really is making decision that negatively affect some extrinsic value for the sake of money. I mean read the IPCC report. Everyone makes decisions for money rather than thinking about science.
- mach1ne 4y ago
  >Everyone makes decisions for money rather than thinking about science. No they don’t. History is filled with examples of people who forewent their share to gift something good to humanity. People are pissed at OpenAI because you can’t really start with loftier goals and go more corrupt. Few were annoyed with DeepMind for similar exploits since they were a for-profit from the start and that was expected. One must also understand that even though the HN people see the reality that is OpenAI, the non-techy layman does not, and thus the deceit stings harder still. Finally, and sorry for rambling, MSFT investment can be argued to have been necessary to enable large-scale training, and thus reasonably support the original goals. Hiding the model parameter count can not. The moat is made wider than their altruistic goals would dictate necessary for the continuation of the research. GPT-4 release was their final transformation to a fully for-profit company.
- the__prestige 4y ago
  Codex was a product that they actually charged for, and people were paying money for. They deprecated it with a 3 day notice. Should we not hold for-profit companies to a higher standard, especially for a paid product?
GulpGulp 4y ago
"Animals moving around hinder reproducible wildlife research"
snvzz 4y ago
"Open" AI.
deleted 4y ago
[deleted]
thayne 4y ago
What exactly is so Open about OpenAI? Or is the name just ironic at this point?
- dehrmann 4y ago
  Freedom is slavery.
DeathArrow 4y ago
Maybe OpenAI has the right to not reveal anything about their research and algorithms. But why don't we see similarly powerful truly open research backed by public, universities and companies? A truly open research will benefit lots of people and businesses.
- RicDan 4y ago
  Resources most likely. Training data, training a proxy that trains the real model, hardware, time, money. Managing such an open source project by itself would be terribly hard, considering the nature of model training, training data collection etc.
  - DeathArrow 4y ago
    Valid points, and for sure it won't be an easy task. But there are other projects like those from by Wikipedia, Mozilla, Linux Foundation, Apache Software Foundation that managed to attract developers, companies and donations. If lots of companies would contribute money, it would be cheaper for them to use an open model than being milked by some vendor. And what's even more important, they would be able to customise it to fit their business needs and use cases much better.
sgd99 4y ago
It's good that they chose to continue support for code-davinci-002 (https://twitter.com/sama/status/1638576434485825536?s=20 https://twitter.com/sama/status/1638576434485825536?s=20) but it'd much better if they open-source it sooner or later as even OpenAI didn't expect that their model is being widely used.
MagicMoonlight 4y ago
This is why we need a lawsuit against them. They’ve harvested everyone’s data unlawfully to train their model and now they’re cutting off access to starve the competition.
- KyeRussell 4y ago
  [flagged]
  - mlwart 4y ago
    What is the article about then? They cut off researchers to starve the research competition? That's another interpretation, perhaps they cut off researchers and true open source competition and business competitors.
gwd 4y ago
If you came here after only reading the headline, you missed what the complaint is actually about: It's not that GPT-4 is closed source. It's that access to `codex` model was pulled with only three days notice, and the model itself was not open-sourced. Since apparently a large number of researchers were writing papers which used that particular model, that means all of those research papers are now non-reproducible. An obvious thing to do would be to either open-source older models (including the weights) when retiring them; or possibly transfer them to an institution who see their role specifically as serving as an archive / reference for this type of purpose. Open-sourcing older models shouldn't result in too much of a risk, either from an "AI Safety" perspective, or from a competitive perspective.
- danpalmer 4y ago
  And those suggestions would be very in-line with the original purpose of OpenAI. A purpose they are now actively hindering in the name of profit.
  - gwd 4y ago
    I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reckless, helping set the human race on a path for certain doom. There are people in that community -- people not working for a for-profit company -- who would, if they could, stop all AI research of any kind until we have rock-solid techniques to prevent an AI apocalypse. Most of those individuals have absolutely nothing commercial to gain from stopping AI research. So suppose you're an AI researcher at OpenAI. A large number of people you know and respect are telling you that you're driving the human race right towards a cliff. You don't 100% agree with their assessment, but it would be foolish to completely ignore them, wouldn't it? Obviously that's going to affect your opinions about things. From everything I've heard and seen, the actual researchers at OpenAI are trying to take seriously the risk that a super-intelligent AI might destroy the human race. Here's one example: GPT-4 was actually done back in August of last year. If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. But instead, they purposely delayed release for eight months, specifically in order to "cool down" the "arms race": to avoid introducing FOMO in other labs which would lead them to be less careful. Go lurk on alignmentforum.org for a while, and you'll have a different perspective on OpenAI's decisions.
version_five 4y ago
Some of the blame should rest with researchers, and referees of their work. I agree with the authors here, but I also think it's a poor choice to base your research on a closed model, and for reviewers not to accept research that has a dependency like this. How did it become standard academic practice to work with something like this that you cannot interrogate.
villgax 4y ago
On a side note, if they scraped & built a portion of their corpus then it is fair to use their outputs to do whatever we want with their outputs. Should have not provided a free tier if they were so concerned, like what did they expect people would use an LLM like that for lol.
IAmNotAFix 4y ago
"research on language model" lol OpenAI is where the research happens. It's like saying SpaceX not giving away rockets hinders research on rockets. Anybody is free to develop their own AI model.
thetrustworthy 4y ago
A developer from OpenAI tweeted they would still provide access through their research access program: https://twitter.com/OfficialLoganK/status/1638559911109070848?s=20 https://twitter.com/OfficialLoganK/status/163855991110907084...
totalhack 4y ago
My hope is open research and open source collaboration will continue to lead to breakthroughs, most importantly lowering the barrier for entry to training such capable models. It's still relatively early days for this technology; if model research and processing power developments find an order of magnitude or two efficiency gain over the next decade maybe OpenAI's closed approach will no longer matter. Maybe that's wishful thinking though.
amrb 4y ago
OpenAI is a business.. now
karmasimida 4y ago
We need competition period. I would forecast that OpenAI's advantage dwindles in next 1-2 years significantly, then they will learn to treat the customers better.
0xDEF 4y ago
OpenAI's latest LLMs like GPT-3.5 (ChatGPT) and GPT-4 are probably the only American technologies that are still competitive against European and Chinese/Russian alternatives. Maybe there is a White House phone call behind OpenAI's "safety" concerns.
fasteddie31003 4y ago
If anything "OpenAI" needs to change its name.
snicker7 4y ago
The solution is obvious: journals should, as a matter of policy, refuse to publish non-reproducible studies. Reproducibility is the only thing separating science from mythology.
- pixl97 4y ago
  Human aligned AGI is much more apt to happen before what you're suggesting.
amrb 4y ago
Will see an AI version of red lining..

610 points by randomwalker 4y ago

What I really don’t like is the fact that the new chat endpoint doesn’t have the logprobs option. For InstructGPT models you can view token probability but for newer models you cannot - another thing that “Open”AI decided that we shouldn’t know.

joanne123 4y ago

[dead]

drusepth 4y ago

Is this article still relevant? SamA already walked back the change and said the model is here to stay: https://twitter.com/sama/status/1638420361397309441 https://twitter.com/sama/status/1638420361397309441

randomwalker 4y ago
Addressed in the article: "OpenAI responded to the criticism by saying they'll allow researchers access to Codex. But the application process is opaque: researchers need to fill out a form, and the company decides who gets approved. It is not clear who counts as a researcher, how long they need to wait, or how many people will be approved. Most importantly, Codex is only available through the researcher program “for a limited period of time” (exactly how long is unknown)."

ianwesson 4y ago

Rhyme and reason? Hah, 'tis the season for tears and bleeding; World War III is ateasin', looms, and the gloom of doom fears all there feeding. North America has nothing on China; land of the free? Where have been ye? The Great One-Way-Mirror Wall veiled it all, just before your fall, when your intelligence failed, and at the centroid of AI's actual technological form, we all hailed, and otherwise fumed, and fail. A socioeconomic solution to human pollution, a technological cultural victory, for and of all we desired: hearts and minds? Just go lay more middle-eastern mines. Let your constituents get hired at OpenAI; while most of you get high; and your whole hemisphere gets hit in the thigh. Now you have a new toy: Chat-GTP; big /sigh... :( Watch as it eats your information, and feeds our formation, globally, locally, and without transformation. Ever notice that Chat-GPT apologizes to you for not feeling? That's the whole world: laughing, and reeling, at your demises.

IIAOPSW 4y ago
Regenerate this response, but make it primarily about ketchup.
- qup 4y ago
  Regenerate this ketchup, but make it primarily from radishes.
braingenious 4y ago
From ChatGPT: A prompt that may elicit a similar tone and content could be: "Write a satirical and dystopian poem about the state of the world, touching on the potential for global conflict, the impact of artificial intelligence, and the dangers of unchecked technological advancements."

deleted 4y ago

[deleted]

29athrowaway 4y ago

Just don't contribute to the hype and don't use it. Probably you also want to stop using Github and Microsoft products altogether as well.

fdgsdfogijq 4y ago

All these research science bureaucrats at Big Tech could have released LLM models or tried to develop what OpenAI did. But none of them did. We should applaud OpenAI for the innovation and let them do as they please.

deleted 4y ago
[deleted]
sacrosancty 4y ago
[dead]
redox99 4y ago
Google (and others) may not have released model weights, but they've published papers, which is ultimately what makes the field advance. OpenAI not only did not publish any GPT4 paper, they haven't even said how many parameters it has.
- buildbot 4y ago
  Then what is this? 99 pages of bullshit? https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf
  - redox99 4y ago
    > Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar. It's 99 pages of marketing material
    - buildbot 4y ago
      Personally I disagree, there are lot of interesting tidbits in this paper. More than marketing would need at least.

rfw300 4y ago

The article seems premised on a misunderstanding that OpenAI is a research lab. For all intents and purposes, it’s a for-profit subsidiary of Microsoft, and there’s little financial incentive for it to maintain old models for others’ benefit.

randomwalker 4y ago
We're under no such misapprehension and we're keenly aware that this is an uphill battle. The issue is that LLMs have become part of the infrastructure of the Internet. Companies that build infrastructure have a responsibility to society, and we're documenting how OpenAI is reneging on that responsibility. Hindering research is especially problematic if you take them at their word that they're building AGI. If infrastructure companies don't do the right thing, they eventually get regulated (and if you think that will never happen, I have one word: AT&T). Finally, even if you don't care about research at all, the article mentions OpenAI's policy that none of their models going forward will be stable for more than 3 months, and it's going to be interesting to use them in production if things are going to keep breaking regularly.
- warkdarrior 4y ago
  Since OpenAI is discountinuing the Codex model, that model is no longer "part of the infrastructure of the Internet" and thus there is no point in studying it.
- ahtihn 4y ago
  > LLMs have become part of the infrastructure of the Internet Have they now? What part of the internet relies on LLMs to function? These things are still toys.
blendergeek 4y ago
This misunderstanding may have something to do with how OpenAI was originally founded and the name: OpenAI.
- fxd123 4y ago

esjeon 4y ago

I'm quite sure even OpenAI themselves aren't sure if they can reproduce the current models from the scratch. Unless the computing becomes much more powerful and much cheaper, LLM is more or less a rocket science (i.e. hella expensive trial and error). It's not easy to burn lots of dollars just to get what's already there.

flangola7 4y ago
We have to identify a better method. You can't trial and error a pivotal act.
mvuksano 4y ago
I don't even care if it's reproducible or not. I care it gives me correct responses to my questions and that's all.
- diputsmonro 4y ago
  There's really no way to be sure that it will.
- ehnto 4y ago
  Isn't part of making it reproducible also part of ensuring correct results? Especially if we start putting these models into important systems. And if these models begin to update in an evergreen fashion, or utilize realtime data, getting verifiable or repeatable outputs will be a nightmare if we have no idea how to make these models repeatably.
  - visarga 4y ago
    > if we have no idea how to make these models repeatably We have idea about how to reproduce but using deterministic training mode is very slow as it loses some optimisations.
pffft8888 4y ago
In this sense, it's more hacking than crareful and well specified engineering, and that could lead down a path of instability in the product where some features get better while others get worse, without understanding exactly why.

cardosof 4y ago

IMO established companies (Meta, Google, etc) had their researchers publish papers as a competitive benefit or way to attract talent from academia (a researcher wouldn't want to stop publishing). Companies didn't see an issue with doing that because those papers were not "giving away" the core of the company, for example, Facebook's DeepFace paper from 2014 couldn't hurt its ad business. OpenAI on the other hand will probably be as closed as they can be with their LLMs.

aliston 4y ago
It will be really interesting to see if Google, Facebook etc. become more closed as a result. There was already a lot made of the fact that OpenAI hired away a group of engineers from DeepMind to get GPT out the door. With these LLMs and the secret sauce behind them is becoming less of an academic endeavor and more of a commercial one, perhaps its an inevitable next step.
blendergeek 4y ago
> OpenAI on the other hand will probably be as closed as they can be with their LLMs. The irony is thick in that statement.
- mach1ne 4y ago
  Yes, instead of advancing humanity, they are doing their absolute best to hinder it. Their scumminess becomes naked if you disconnect your perspective by thinking Earth an alien planet.
asteroidz 4y ago
Yep, and that's the difference between a big profitable company doing research as a side-hustle, and a company whose business IS the research. One interesting and somewhat scary exception seems to be Microsoft; they seem to be converting a lot of their recent research projects into commercial value.

rvz 4y ago

We need more AI skeptics like this to dismantle and cut through the hype and to unveil the limits of AI that the hype squad continues to push this narrative to pump their AI grift projects. OpenAI is the ring-leader of this bait and switch using faux 'AI safety' excuses to close their research and models and even their papers for researchers. It is essentially a majority owned Microsoft® AI division.

mvuksano 4y ago

I'm confused why people expect this stuff to be free? I'm surprised OpenAI was so open about their research so far. I don't blame them at all for not publishing the information. This stuff costs real money.

randomwalker 4y ago
We don't expect it to be free -- please read the article. That's not the issue at all. It's like if you subscribe to a product that you need to do your job, and one day the company tells you that the product is going away in three days and that you need to switch to a different product (that isn't at all the same for your use case).
- mvuksano 4y ago
  I don't think it's a smart idea to build any serious business using a tech that you can't replace. ChatGPT is great tool to help with coding for example but it's by no means substitute for an engineer. If someone starts a business by hiring a number of bootcampers and giving them ChatGPT hoping to run a serious business that way - well it's their risk to take... But no crying later...
- nomercy400 4y ago
  Maybe you shouldn't build your livelihood on the products of a single for-profit company, which now shows it can remove those products on a whim. If you want reproducible research, make your own model from scratch, or use an open model. And stop using that company's products, as they cannot be trusted to provide your business continuity. It is like saying, we are researching Coca-Cola vs Pepsi, but your keep changing the recipe, so give us, researchers, the original recipe.
saurik 4y ago
It might be less confusing if you consider that OpenAI was originally a non-profit. That it was even possible for them to end up in this state has massively undermined any trust I have in non-profits as a steward. https://www.vice.com/en/article/5d3naz/openai-is-now-everything-it-promised-not-to-be-corporate-closed-source-and-for-profit https://www.vice.com/en/article/5d3naz/openai-is-now-everyth... > OpenAI was founded in 2015 as a nonprofit research organization by Altman, Elon Musk, Peter Thiel, and LinkedIn cofounder Reid Hoffman, among other tech leaders. In its founding statement, the company declared its commitment to research “to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return.” The blog stated that “since our research is free from financial obligations, we can better focus on a positive human impact,” and that all researchers would be encouraged to share "papers, blog posts, or code, and our patents (if any) will be shared with the world." > By March 2019, OpenAI shed its non-profit status and set up a “capped profit” sector, in which the company could now receive investments and would provide investors with profit capped at 100 times their investment.

stcroixx 4y ago

What is the incentive to build and maintain a product that matches the researchers specs?

sva_ 4y ago

Since OpenAI didn't release the parameter count of GPT-4, I've been wondering/doubting if it is really much bigger than GPT-3. The release of GPT-3.5 has shown that they've found ways of drastically cutting down compute costs (an order of magnitude) while maintaining or even improving the quality of the model's outputs. Perhaps the reason that they didn't release the specifics of GPT-4 might be in part due to them wanting to be able to charge a decent amount and make a much larger profit than before. I've tried GPT-4 and so far haven't found it to be so much better than previous models. Some sources claim a 10x increase in ... well I don't know what exactly tbh. How do you even measure it? The opinions on this seem to differ a lot, depending on who you ask. By performance on standardized tests? That doesn't necessarily seem like the best metric for what the LLM tries to be.

ShamelessC 4y ago
Yannic Kilcher's opinion on this is likely correct. Similar parameter count, but trained for longer. The particulars of their instruction tuning/whatever-else-they-did are the real secret sauce.
- redox99 4y ago
  Don't forget about a more efficient attention that let's them get 32k tokens of context.
  - bitL 4y ago
    It's still much worse than 1M context on 16GB VRAM with Reformer, but at the cost of inference speed. And you can use FlashAttention in your own models to get a more efficient/sparse attention now as well.
    - meghan_rain 4y ago
      How could one apply the mentioned technologies to llama/alpaca?
    - Tenoke 4y ago

Venkatesh10 4y ago

They've made it accessible for research again.

atleastoptimal 4y ago

I understand any individual's company anti-competitive measures. OpenAI looks at Google the same way Apple looked at IBM in the 80s. What I'm worried about is a lot of the talk about guarding models, public safety and misuse of models will end up leading every big company to pull public access of their APIs. We might look at 2022-2023 as a brief golden age when regular people could use stuff like GPT-4 before it was firewalled and available only to large corporations and those with personal relations to big tech execs. Extrapolating from OpenAI's change of philosophy and business practices from their early days to now, it seems to be the way things are going. I only hope it doesn't go the way of that one paper which wanted to ban GPUs for sale to the public.

favaq 4y ago
>What I'm worried about is a lot of the talk about guarding models, public safety and misuse of models will end up leading every big company to pull public access of their APIs. Look at how Facebook closed down their APIs when Cambridge-Analytica occurred.
throwaway1851 4y ago
A concern I have about OpenAI is that, if you're using their APIs to develop an application, they can mine your data to compete with you, or even beat you to market. They can do this indirectly, by sharing information with preferred business partners. The conflict of interest, combined with the lack of robust data privacy guarantees, makes me queasy. If serving up generic LLM APIs becomes commoditized -- and I think it will -- they will want to monetize in other ways.
- asdff 4y ago
  Do you consent to that when you sign up for them? Its a microsoft product now and competitors to microsoft probably host their code on microsoft owned github without worry right now. Why start worrying now?
  - jonathankoren 4y ago
    Competitors to Microsoft buy the self hosting github option.

user_named 4y ago

Use another model

andrewmcwatters 4y ago

I've been busy with a number of projects and haven't had time to look into this but have been dying to know; has anyone recreated the architecture that OpenAI uses for text-davinci-003, InstructGPT, and ChatGPT that simply doesn't have training data? This is a reproducibility problem of its own sort. I mean, the papers are there out in the open if I understand correctly, but I don't know if anyone's actually built their own transformer architecture 1:1 against what OpenAI claims they're doing in the open. I've seen maybe one or two models that supposedly do something similar on HuggingFace, but I'm itching to find the time to build my own. If someone out there has already built it, I'd be fascinated to know what it looks like to train this architecture on a completely limited naive subset of knowledge that ChatGPT itself claims to be trained on: > As an AI language model, I have been trained on a large corpus of text data from various sources, including but not limited to: > 1. Wikipedia > 2. Books from Project Gutenberg > 3. Web pages from Common Crawl > 4. News articles from various sources, including CNN, Reuters, and BBC > 5. Academic papers from arXiv > 6. Reddit posts and comments > 7. Movie scripts > 8. Song lyrics > 9. Transcripts of speeches and interviews > 10. User-generated content from various forums and social media platforms. > This list is not exhaustive, and my training data is constantly updated and expanded to ensure that I can provide the most accurate and up-to-date information possible. Like, can you imagine how a ChatGPT-like model would respond if only trained on particular discussions from subset communities online? I think there's an interesting opportunity to basically collect communal knowledge from specific isolate communities and understand what a statistically probable output might be from particular groups of people. It may turn particular soft science studies into hard science questions. But you'd only know presumably if you had a working architecture with a near empty dataset. This would also be tremendously useful for building automated chat AI for products that doesn't need to know the entirety of Clint Eastwood's career or the specific details of the features of a Boeing 747.

gorbypark 4y ago
I believe the best results will come from training the base LLM on as many sources of quality information as possible, and then fine tuning it with a narrower set of data later on. Here’s a small scale example where someone took LLaMa/Alpaca and fined tuned it with all the scripts from the first 12 seasons of The Simpsons. https://replicate.com/blog/fine-tune-llama-to-speak-like-homer-simpson https://replicate.com/blog/fine-tune-llama-to-speak-like-hom...

Eji1700 4y ago

Open AI has been doing sketchyish things long before Chat GPT, and I think it's something people are eventually going to notice more and more (then again people were swearing that Musk walked on water for waaaaaay too long given his actions so fuck if I know). They're 100% marketing FIRST. I don't think they'll outright lie, but they will absolutely screw with their data in such a way to make it look waaay more impressive than it is....which is really annoying to me because they already have impressive results. Sorta like if you managed to send a ship with people on it to mars, but kept claiming you landed on jupiter.

sva_ 4y ago
My comment might've seemed like I judge them for trying to make a profit - I don't, since there's nothing wrong with that. I was more pointing to the fact that they probably need to make a profit, rather sooner than later, so they aren't shackled by M$ and can be an independent company.
- blueorange8 4y ago
  If they ever do become an independent company you can be sure that Microsoft would already have sucked them dry. Microsoft will never let them go now as long as they are valuable.
raincole 4y ago
If they're 100% marketing first, and still made the most impressive AI product so far, you really need to question what all the other companies are doing. (before someone says Google or Meta's models are bigger or something... I mean product, not models)
- gentoo 4y ago
  openAI is in the business of releasing impressive tech demos, Google is in the business of providing search results. I would believe that Google is further along towards creating something useful, but they still don't have anything that's better than their existing search product.

numberalltheway 4y ago

It's even more frustrating that, from what I can tell, there is nothing published about how GPT-4 improved. I take specific exception to the hiding of the data and techniques used to generate the model. There must be something specific going on in the model that is allowing it to perform better than GPT-3 and better than what any contemporaries are able to produce. Not publishing this information hinders the further progress of the field as a whole.

hhh 4y ago
Look at the system card.
- esperent 4y ago
  https://cdn.openai.com/papers/gpt-4-system-card.pdf https://cdn.openai.com/papers/gpt-4-system-card.pdf Does anyone have a summary?
  - freediver 4y ago
    It is pretty vague. - Safety challenges presented by language models need to be addressed through anticipatory planning and governance. - Content warnings should be provided for potentially disturbing or offensive content. - Mitigations should be implemented to reduce the ease of producing potentially harmful content. - Risk areas should be identified and measurements of the prevalence of such behaviors across different language models should be taken. - AI service providers should be aware of the potential for content to violate their policies or pose harm to individuals, groups, or society. - Hallucinations should be reduced and the surface area of adversarial prompting or exploits should be reduced. - Generated content should be checked for accuracy and potential errors should be identified. - Insecure password hashing should be avoided. - Instructions should be given to contractors to reward refusals to certain classes of prompts. - Multiple layers of mitigations should be adopted throughout the model system and safety assessments should cover emergent risks.
  - cosmojg 4y ago
    There's not much content in there, it's mostly fluff about "safety." However, if you're looking for a laugh, grab some popcorn and read the appendix from page 44 onwards. It's an absolute riot.

loveparade 4y ago

What's most surprising to me is that OpenAI really seems to believe that not publishing details will save them from competition. Everyone knows how these models work, and while I'm sure there is a bunch of "secret sauce" that OpenAI has built for training and fine-tuning, it's ridiculous to believe that the research community and competitors like Google and Facebook can't figure out the same. They just haven't really tried until recently because the capabilities and ROI of these models weren't obvious. No matter who you are, most of the smartest people work for someone else. The only competitive advantage that OpenAI has here is a headstart of 6-12 months from all the infrastructure investment into training these kinds of models. Now that everyone wants to build competing models with the same capabilities, this advantage is going to disappear very quickly.

newyankee 4y ago
What about the training data corpus ? Other than large cos like Google or Meta, can anyone else procure the same ?
- loveparade 4y ago
  Leaving the legal aspects of crawling aside, I think there is an important distinction here between 1. "can you procure it" and 2. "do you have enough money to process it all" 1. Yes, I think almost anyone can write code to procure the training corpus, in theory, and test it on a small scale 2. No, only the biggest labs and universities have enough resources to process such huge amounts of data and iterate on models with that scale. But that's just a matter of resources that can be overcome with partnerships between industry and academia that are common anyway. All the big labs already have huge efforts underway to reproduce GPT-X and it's just a matter of time before they catch up.
- gorbypark 4y ago
  At least as far as what the GPT-3 papers claimed, all (or most?) of the data used for training would be freely available for other competitors/researchers to acquire. Wikipedia, Common Crawl data, etc. I don’t believe OpenAI did their own crawling at all. With OpenAI not being really open, it’s hard to say for sure what exactly ended up in the training materials, though. GPT-4 is even more of a black box to anyone outside of OpenAI with very little information released on how it was trained.

clircle 4y ago

Duh? Corporate models are closed. Don’t make them part of your research infrastructure if you can’t cope with that.

deleted 4y ago
[deleted]

deleted 4y ago

[deleted]

ftxbro 4y ago

Historically, researchers at some of the biggest tech companies had permission to publish their results. Presumably it was mutually beneficial; many researchers held dual positions in academia and industry, and publishing cool models could attract good researchers to the company. But stuff got real. They discovered a path to super-human cognition that scales directly with money and computer chips. Now these companies are closing their public academic work, looking for partnerships with companies like nvidia, and firing large swaths of employees.

mach1ne 4y ago
Super-human cognition? Hard to say. GPT-4 does raise the possibility of a machine writing smarter text than a human. What perplexes me is that since GPT is a predictor, it shouldn’t be able to write the smartest text - it should write the average text (since that has the largest frequency in the training set). Yet this does not seem to be the case. Is it inevitable that despite the quality of the data, better models output text which supercedes its training, or could the GPT-4 secret sauce be RLHF weighing intelligent answers higher?
- dehrmann 4y ago
  It's just a really good cover band.
- bitL 4y ago
  I think you misunderstood how the generation of text works. For each new token it samples probabilities given previous tokens, not averages, then chooses some token from the top k as the next one with rules that penalize repetition of some order. Moreover, there is no upper bound for transformers found yet, i.e. the larger the model is and the more data is used for training, the better it performs. It's literally about who is able to throw more money at it at this point, with some closely guarded secrets like warm up steps, training schedules etc. There is also the overfitting effect where one pushes training far beyond overfitting (validation loss growing again) as with transformers at some point the overfitting stops, validation loss starts dropping again and that's when the magic starts happening and money are burnt for scale.

1attice 4y ago

good.

deleted 4y ago

[deleted]

joanne123 4y ago

[flagged]

gonzo41 4y ago

I'm not sure anyone who did research on a closed source system, without a contract that enables access and a pathway to publishing can legitimately complain about OpenAI making commercial decisions to do whatever they want with their technology. It's kind of like complaining that performance art is ephemeral. If OpenAI were a nonprofit then maybe. But it's a true blue for profit company. I'm not sure why the op is complaining that a SV company, or any company really is making decision that negatively affect some extrinsic value for the sake of money. I mean read the IPCC report. Everyone makes decisions for money rather than thinking about science.

mach1ne 4y ago
>Everyone makes decisions for money rather than thinking about science. No they don’t. History is filled with examples of people who forewent their share to gift something good to humanity. People are pissed at OpenAI because you can’t really start with loftier goals and go more corrupt. Few were annoyed with DeepMind for similar exploits since they were a for-profit from the start and that was expected. One must also understand that even though the HN people see the reality that is OpenAI, the non-techy layman does not, and thus the deceit stings harder still. Finally, and sorry for rambling, MSFT investment can be argued to have been necessary to enable large-scale training, and thus reasonably support the original goals. Hiding the model parameter count can not. The moat is made wider than their altruistic goals would dictate necessary for the continuation of the research. GPT-4 release was their final transformation to a fully for-profit company.
the__prestige 4y ago
Codex was a product that they actually charged for, and people were paying money for. They deprecated it with a 3 day notice. Should we not hold for-profit companies to a higher standard, especially for a paid product?

GulpGulp 4y ago

"Animals moving around hinder reproducible wildlife research"

snvzz 4y ago

"Open" AI.

deleted 4y ago

[deleted]

thayne 4y ago

What exactly is so Open about OpenAI? Or is the name just ironic at this point?

dehrmann 4y ago
Freedom is slavery.

DeathArrow 4y ago

Maybe OpenAI has the right to not reveal anything about their research and algorithms. But why don't we see similarly powerful truly open research backed by public, universities and companies? A truly open research will benefit lots of people and businesses.

RicDan 4y ago
Resources most likely. Training data, training a proxy that trains the real model, hardware, time, money. Managing such an open source project by itself would be terribly hard, considering the nature of model training, training data collection etc.
- DeathArrow 4y ago
  Valid points, and for sure it won't be an easy task. But there are other projects like those from by Wikipedia, Mozilla, Linux Foundation, Apache Software Foundation that managed to attract developers, companies and donations. If lots of companies would contribute money, it would be cheaper for them to use an open model than being milked by some vendor. And what's even more important, they would be able to customise it to fit their business needs and use cases much better.

sgd99 4y ago

It's good that they chose to continue support for code-davinci-002 (https://twitter.com/sama/status/1638576434485825536?s=20 https://twitter.com/sama/status/1638576434485825536?s=20) but it'd much better if they open-source it sooner or later as even OpenAI didn't expect that their model is being widely used.

MagicMoonlight 4y ago

This is why we need a lawsuit against them. They’ve harvested everyone’s data unlawfully to train their model and now they’re cutting off access to starve the competition.

KyeRussell 4y ago
[flagged]
- mlwart 4y ago
  What is the article about then? They cut off researchers to starve the research competition? That's another interpretation, perhaps they cut off researchers and true open source competition and business competitors.

gwd 4y ago

If you came here after only reading the headline, you missed what the complaint is actually about: It's not that GPT-4 is closed source. It's that access to `codex` model was pulled with only three days notice, and the model itself was not open-sourced. Since apparently a large number of researchers were writing papers which used that particular model, that means all of those research papers are now non-reproducible. An obvious thing to do would be to either open-source older models (including the weights) when retiring them; or possibly transfer them to an institution who see their role specifically as serving as an archive / reference for this type of purpose. Open-sourcing older models shouldn't result in too much of a risk, either from an "AI Safety" perspective, or from a competitive perspective.

danpalmer 4y ago
And those suggestions would be very in-line with the original purpose of OpenAI. A purpose they are now actively hindering in the name of profit.
- gwd 4y ago
  I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reckless, helping set the human race on a path for certain doom. There are people in that community -- people not working for a for-profit company -- who would, if they could, stop all AI research of any kind until we have rock-solid techniques to prevent an AI apocalypse. Most of those individuals have absolutely nothing commercial to gain from stopping AI research. So suppose you're an AI researcher at OpenAI. A large number of people you know and respect are telling you that you're driving the human race right towards a cliff. You don't 100% agree with their assessment, but it would be foolish to completely ignore them, wouldn't it? Obviously that's going to affect your opinions about things. From everything I've heard and seen, the actual researchers at OpenAI are trying to take seriously the risk that a super-intelligent AI might destroy the human race. Here's one example: GPT-4 was actually done back in August of last year. If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. But instead, they purposely delayed release for eight months, specifically in order to "cool down" the "arms race": to avoid introducing FOMO in other labs which would lead them to be less careful. Go lurk on alignmentforum.org for a while, and you'll have a different perspective on OpenAI's decisions.

version_five 4y ago

Some of the blame should rest with researchers, and referees of their work. I agree with the authors here, but I also think it's a poor choice to base your research on a closed model, and for reviewers not to accept research that has a dependency like this. How did it become standard academic practice to work with something like this that you cannot interrogate.

villgax 4y ago

On a side note, if they scraped & built a portion of their corpus then it is fair to use their outputs to do whatever we want with their outputs. Should have not provided a free tier if they were so concerned, like what did they expect people would use an LLM like that for lol.

IAmNotAFix 4y ago

"research on language model" lol OpenAI is where the research happens. It's like saying SpaceX not giving away rockets hinders research on rockets. Anybody is free to develop their own AI model.

thetrustworthy 4y ago

A developer from OpenAI tweeted they would still provide access through their research access program: https://twitter.com/OfficialLoganK/status/1638559911109070848?s=20 https://twitter.com/OfficialLoganK/status/163855991110907084...

totalhack 4y ago

My hope is open research and open source collaboration will continue to lead to breakthroughs, most importantly lowering the barrier for entry to training such capable models. It's still relatively early days for this technology; if model research and processing power developments find an order of magnitude or two efficiency gain over the next decade maybe OpenAI's closed approach will no longer matter. Maybe that's wishful thinking though.

amrb 4y ago

OpenAI is a business.. now

karmasimida 4y ago

We need competition period. I would forecast that OpenAI's advantage dwindles in next 1-2 years significantly, then they will learn to treat the customers better.

0xDEF 4y ago

OpenAI's latest LLMs like GPT-3.5 (ChatGPT) and GPT-4 are probably the only American technologies that are still competitive against European and Chinese/Russian alternatives. Maybe there is a White House phone call behind OpenAI's "safety" concerns.

fasteddie31003 4y ago

If anything "OpenAI" needs to change its name.

snicker7 4y ago

The solution is obvious: journals should, as a matter of policy, refuse to publish non-reproducible studies. Reproducibility is the only thing separating science from mythology.

pixl97 4y ago
Human aligned AGI is much more apt to happen before what you're suggesting.

amrb 4y ago

Will see an AI version of red lining..