7 ms·

It's a shame that it has turned out to be necessary to externally re-make and re-train a model that has come out of company called `OPEN`AI. Wasn't one of the f

• spiderfarmer 6y ago
  I don't really care if OpenAI offers commercial licenses as long as the underlying research is truly open. This way alternative options will become available eventually.
  • querez 6y ago
    Arguably openAI is one of the most closed industry AI labs (among those that are still participating in the research community), on par only with deep mind (though deepmind at least publishes way more). Funnily enough, FAIR and Google Brain have a vastly better track record wrt. publishing not only papers but also code and models.
• wraptile 6y ago
  OpenAI turning out to be a total bait and switch. Especially true when your co-founder is actively calling you out on it[1] Remember kids: if it's not a non-profit organization it is a _for_ profit one! It was silly to expect anything else: > In 2019, OpenAI transitioned from non-profit to for-profit. The company distributed equity to its employees and partnered with Microsoft Corporation, who announced an investment package of US$1 billion into the company. OpenAI then announced its intention to commercially license its technologies, with Microsoft as its preferred partner [2] 1 - https://edition.cnn.com/2020/09/27/tech/elon-musk-tesla-bill-gates-microsoft-open-ai/index.html https://edition.cnn.com/2020/09/27/tech/elon-musk-tesla-bill... 2 - https://en.wikipedia.org/wiki/OpenAI https://en.wikipedia.org/wiki/OpenAI
  • person_of_color 6y ago
    So OpenAI employees get Microsoft RSUs?
    • unixhero 6y ago
      What is an RSU?
      • ourcat 6y ago
        Restricted Stock Units
      • agravier 6y ago
        It means restricted stock unit, and it's a kind of company stock unit that may be distributed to some "valued" employees. There is usually a vesting schedule, and you can't do whatever you want with it.
    • garmaine 6y ago
      Why would they? It's a separate company.
  • dvfjsdhgfv 6y ago
    It will be interesting to see the attitude of Microsoft towards this project in the light of their "Microsoft loves open source" propaganda.
    • Closi 6y ago
      I don't know where people got this idea that Microsoft can't participate positively in Open Source, and do that sincerely, without open sourcing absolutely everything. Of course they can - just because you contribute to open source, and do that because you also benefit from open source projects, doesn't mean you have to do absolutely everything under open source. Especially considering OpenAI isn't even Microsoft's IP or codebase.
      • 6y ago
• JacobiX 6y ago
  It's a recurring theme in OpenAI research, they become more and more closed. For instance their latest model called DALL·E hit the headlines before the release of the paper. Needless to say, the model is not available and no online demo has been published so far.
  • visarga 6y ago
    But they still released the CLIP model which is the complement of DALL-E and used in the DALL-E pipeline as a final filter. There are collabs with CLIP floating around and even a web demo.
    • JacobiX 6y ago
      Thank you for this info, as you mentioned CLIP is used for re-ranking DALL-E outputs, by itself it is just an (image, text) pairs classification network.
  • cbozeman 6y ago
    Because its winner-take-all in this research, not "winner-take-some". Andrew Yang talked about this and why breaking up Big Tech won't work. No one wants to use the second best search engine. The second best search engine is Bing and I almost never go there. Tech isn't like automobiles, where you might prefer a Honda over a Toyota, but ultimately they're interchangeable. A Camry isn't dramatically different and doesn't perform dramatically better than an Accord. Whoever builds the best AI "wins" and wins totally.
• mhuffman 6y ago
  But I was told GPT-3 was too powerful for mere mortal hands (unless you have an account!) and that it would be used for hate speech and to bring about skynet. How will this project avoid those terrible outcomes?
  • ForHackernews 6y ago
    New research has revealed that intelligence is not a prerequisite for generating hate speech on social media platforms.
  • visarga 6y ago
    By putting the cat back in the bag. Oh, it's too late ... useless to think about it - we can't de-invent an invention or stop people from replicating. Its like that time when NSA wanted to restrict crypto.
  • dvfjsdhgfv 6y ago
    I don't know a single intelligent person who believed this argument, it simply doesn't hold up.
    • thelastwave 6y ago
      Lots of people "believe" that, they just prefer to downvote anonymously rather than try to defend their position.
• dave_sullivan 6y ago
  Really. OpenAI assembled some of the best minds from the deep learning community. The problem isn't that they are a for-profit SaaS, the problem is they lied.
  • thelastwave 6y ago
    And ended up making an AI service that's really good at... lying.
• visarga 6y ago
  It was probably bait and switch to hire top researchers and get initial funding. Now that OpenAI is a household name, they don't have to pretend anymore.
  • b3kart 6y ago
    I buy the former, researchers might be happier knowing their work potentially benefits all of humanity, not just a bunch of investors. But wouldn’t it be more difficult to get funding as a non-profit?
    • littlestymaar 6y ago
      It's just never going to be difficult to get funding when you have Elon Musk and Sam Altman as founders (and even more so when founders put one billion of their own money into it).
      • b3kart 6y ago
        Sure, but that's OpenAI's particular set of circumstances. Generally speaking I struggle to see investors preferring a nebulous non-profit over a for-profit with a clear path to market.
        • littlestymaar 6y ago
          Sure, but we're explicitly talking about OpenAI here.
          • b3kart 6y ago
            Of course. It's just that the comment I've been responding to suggested OpenAI going the "open"/non-profit route was to 1) get top researchers and 2) get investment. I was arguing that this doesn't seem to (generally) be a good way to get investment, but I agree with you in that in their case investment just wasn't a consideration at all.
• Sambdala 6y ago
  Wild-Ass Guess (Ass-Guess) incoming: OpenAI was built to influence the eventual value chain of AI in directions that would give the funding parties more confidence that their AI bets would pay off. This value chain basically being one revolving around AI as substituting predictions and human judgement in a business process, much like cloud can be (oversimply) modeled as moving Capex to Opex in IT procurement. They saw that, like any primarily B2B sector, the value chain was necessarily going to be vertically stratified. The output of the AI value chain is as an input to another value chain, it's not a standalone consumer-facing proposition. The point of OpenAI is to invest/incubate a Microsoft or Intel, not a Compaq or Sun. They wanted to spend a comparatively small amount of money to get a feel for a likely vision of the long-term AI value chain, and weaponize selective openness to: 1) establish moats, 2) Encourage commodification of complementary layers which add value to, or create an ecosystem around, 'their' layer(s), and 3) Get insider insight into who their true substitutes are by subsidizing companies to use their APIs As AI is a technology that largely provides benefit by modifying business processes, rather than by improving existing technology behind the scenes, your blue ocean strategy will largely involve replacing substitutes instead of displacing direct competitors, so points 2 and 3 are most important when deciding where to funnel the largest slice of the funding pie. _Side Note: Becoming an Apple (end-to-end vertical integration) is much harder to predict ahead of time, relies on the 'taste' and curation of key individuals giving them much of the economic leverage, and is more likely to derail along the way._ They went non-profit to for-profit after they confirmed the hypothesis that they can create generalizeable base models that others can add business logic and constraints to and generate "magic" without having to share the underlying model. In turn, a future AI SaaS provider can specialize in tuning the "base+1" model, then selling that value-add service to the companies who are actually incorporating AI into their business processes. It turned out, a key advantage at the base layer is just brute force and money, and further outcomes have shown there doesn't seem to be an inherent ceiling to this; you can just spend more money to get a model which is unilaterally better than the last one. There is likely so much more pricing power here than cloud. In cloud, your substitute (for the category) is buying and managing commodity hardware. This introduces a large-ish baseline cost, but then can give you more favorable unit costs if your compute load is somewhat predictable in the long term. More importantly, projects like OpenStack and Kubernetes have been desperately doing everything to commodotize the base layer of cloud, largely to minimize switching costs and/or move the competition over profits up to a higher layer. You also have category buyers like Facebook, BackBlaze, and Netflix investing heavily into areas aimed at minimizing the economic power of cloud as a category, so they have leverage to protect their own margins. It's possible the key "layer battle" will be between the hardware (Nvidia/TPUs) and base model (OpenAI) layers. It's very likely hardware will win this for as long as they're the bottleneck. If value creation is a direct function of how much hardware is being utilized for how long, and the value creation is linear-ish as the amount of total hardware scales, the hardware layer just needs to let a bidding war happen, and they'll be capturing much of the economic profit for as long as that continues to be the case. However, the hardware appears (I'm no expert though) to be something that is easier to design and manufacture, it's mostly a capacity problem at this point, so over time this likely gets commoditized (still highly profitable, but with less pricing power) to a level where the economic leverage goes to the Base model layer, and then the base layer becomes the oligopsony buyer, and the high fixed investment the hardware layer made then becomes a problem. The 'Base+1' layer will have a large boom of startups and incumbent entrants, and much of the attention and excitement in the press will be equal parts gushing and mining schaudenfreude about that layer, but they'll be wholly dependent on their access to base models, who will slowly (and deliberately) look more and more boring apart from the occasional handwringing over their monopoly power over our economy and society. There will be exceptions to this who are able to leverage proprietary data and who are large enough to build their own base models in-house based on that data, and those are likely to be valuable for their internal AI services preventing an 'OpenAI' from having as much leverage over them and being much better matched to their process needs, but they will not be as generalized as the models coming from the arms race of companies who see that as their primary competitive advantage. Facebook and Twitter are two obvious ones in this category, and they will primarily consume their own models, rather than expose them as model-as-a-service directly. The biggest question to me is whether there's a feedback loop here which leads to one clear winning base layer company (probably the world's most well-funded startup to date due to the inherent upfront costs and potential long-term income), or if multiple large, incumbent tech companies see this as an existential enough question that they more or less keep pace with each other, and we have a long-term stable oligopoly of mostly interchangeable base layers, like we do in cloud at the moment. Things get more complex when you look to other large investment efforts such as in China, but this feels like a plausible scenario for the SV-focused upcoming AI wars.
• Tenoke 6y ago
  The research is open to the public. Here's the gpt3 paper https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165 Also gpt2 models and code at least were publicly released and so has a lot of their work. And yes, they realized they can achieve more by turning for profit and partnering with Microsoft. So true, they are not fully 'open' but pretending they don't release things to the public and making the constant 'more like closedai aimirite' comments is getting old.