5 ms·
Even with my rose-tinted glasses on about the future of AI, it's not clear who will be the "winner" here, or even if any business making them will be a winner.
by ben_w 3y ago
Even with my rose-tinted glasses on about the future of AI, it's not clear who will be the "winner" here, or even if any business making them will be a winner.
If open source models are good enough (within the category of image generators it looks like many Stable Diffusion clone models are), what's the business case for Stability AI or Midjourney Inc.?
Same for OpenAI and LLMs — even though for now they have the hardware edge and a useful RLHF training set from all the ChatGPT users giving thumbs up/down responses, that's not necessarily enough to make an investor happy.
- rebeccaskinner 3y agoEarly signals to me are that regulatory capture will end up being the moat that gets used here. I think it's a horrible outcome for society, but likely one that will make some companies a lot of money. Early grumblings around regulation for a lot of AI models seem at risk of making open source models (and even open-weigh models) effectively illegal. Training from scratch is also going to both remain prohibitively expensive for individuals and most bootstrapped startups, plus with more of the common sources of data locking out companies from using training data it's going to be hard for new entrants to catch up. I personally think the only way AI will end up being a benefit to society is if we end up with unencumbered free and open models that run locally and can be refined locally. Every financial incentive is pushing in the other direction though.
- benreesman 3y agoThis should be one of the highest voted comments in all of the AI threads this year. Meta is no doubt doing this because it’s in their best interest, but if both the quality and licensing of LLaMA 2 start a trend that’s a pretty effective counter-weight to eyeball scanner world. And there’s other stuff. George Hotz is pretty unpopular because he does kind of put the crazy in crazy smart (which I personally find a refreshing change to the safe space for relatively neurotypical people in the land of aspy nerds), but tinygrad is a fundamentally more optimizable design than its predecessors with an explicit technical emphasis on accelerator portability and an implicit idealistic agenda around ruining the whole day of The AI Cartel. And it runs the marquee models. Serious megacorp CEOs seem to be glancing nervously in his direction, which is healthy. It’s not locked-in yet.
- late2part 3y agoI've not met Mister Hotz, but from what I've read and heard from him on podcasts, I'm not sure there's a lot of smart in the crazy smart.
- benreesman 3y agoIf what you’ve read and listened to is a lot, please elaborate? If it isn’t a lot, I encourage you to look more closely. Let’s skip all the disputes about his achievements prior to this year (and you really need to take with a grain of salt anything bad you hear about someone who has been actively fucking up the afternoon of powerful people their entire life). Tinygrad is so obvious in retrospect, but hindsight is 20/20 and I missed it. Why the hell do we have kernels for any composite tensor operation? The machine economics are that starting a training run is who cares. Doing a training run is machine hours, days, centuries, millennia. Spend like a drunken sailor customizing the kernel, exploit the aforementioned “who cares” blank check to just dump the friggin CUDA or PTX or OpenCL or Metal or whatever into /tmp/ and hit it with a fork/execve to nvcc or whatever. You can fuse and columnize and whatever to your hears content in the IR. We spent how much time rigging up a trouś convolutions via IM2Col by hand with template instantiation in ATen? Well, we’re never getting that back. And while dating NVIDIA is fun, being chained in their soundproof basement not so fun. “It lives with the same 24Gb of GDDR6 on it paid for last year on its skin or it gets the p4d.24xlarge pricing again”. Nobody has jacked anyone so usuriously via API lock-in since Gates and Win32. Anyone who is doing a YouTube video called “Get in Losers, We’re building a Chatbot” and then live codes for 5 hours to turn 2700 lines of Python that can’t run LLaMA into 3200 lines of Python that can run it on three times the number of platforms that PyTorch? I’m struggling here.
- threeseed 3y agoI think many people's first introduction to George Hotz was with Twitter. Where his arrogance, ignorance and complete lack of experience with how teams and companies function effectively was on full display. You can spout off technical terms all day long but I've worked at two of the FAANG companies and met hundreds of incredible engineers and it wasn't their grasp of the technology that made them great. It was their ability to lead, deliver, communicate, relate etc which are skills that sound far less impressive but are infinitely more important.
- threeseed 3y agoSorry but most of this is wrong. History has shown time and time again that you need the guardrails that regulation/laws provides in order to channel progress in an effective direction. Otherwise the negatives of human nature destroy progress. We have seen this recently with the rampant criminality and negative behaviours permanently crippling the crypto space. In the AI space there is no evidence that allowing models with illegal/unethical content is going to magically translate to some boost in performance, capability or efficacy than would otherwise happen. In fact again history has shown that the opposite is likely to happen. That having models that don't exhibit negative effects e.g. racism, sexism will result in them being used in more places and exposed to wide audiences. Thus translating to a bigger net benefit for society.
- serjester 3y agoAgreed look at Jasper - a year ago the 600lb gorilla in the room and overnight their moat has dried up along with many of their customers.
- emadm 3y agoMake models usable is really valuable, for Stability AI I discussed business models with Sam Lessin here: https://www.youtube.com/watch?v=mOOYJONenWU https://www.youtube.com/watch?v=mOOYJONenWU but basically the edge is data and distribution given how widely used this technology will be. Open is its own area, proprietary general models are a race to zero vs OpenAI and Google who are non-economic actors. Most AI right now is just features tho, very basic without the real thinking needed. Next year we go enterprise.
- alfalfasprout 3y ago> Open is its own area, proprietary general models are a race to zero vs OpenAI and Google who are non-economic actors. Agreed > Make models usable is really valuable Sure... but is there really a moat here? Seems like OSS can address "making models usable" and without lockin from a startup.
- hgsgm 3y agoThat's like glorified rsync startups like Dropbox never got off the ground
- emadm 3y agoDon’t need lock in just support. Business model is called open core hundred of billions in market cap here as most folk don’t want to build/train their own models and will pay for support and help https://en.m.wikipedia.org/wiki/Open-core_model https://en.m.wikipedia.org/wiki/Open-core_model Can look at databricks and many kfheds
- mahathu 3y agoI know nothing about AI or stocks, so please correct me if I'm wrong here: Isn't NVIDIA a clear winner already (bar any major technological advances allowing all of us to run LLMs on our phones?) I just checked the stock on google and it went up 200% since the beginning of the year!
- yeck 3y agoI'd say they are a winner right now. Selling some of the best shovels and sieves in the gold rush hype. Edit: if AMD plays their cards right, I'd expect that they can get a lot more in on the action, too.
- pclmulqdq 3y agoIntel and AMD are both trying to win the supercomputer-scale AI battle, and both of them have chips that put up impressive paper results. Nvidia's user experience is just better, though, and they treat hobbyists with slightly less disdain than the other chip companies.