8 ms·
Generative AI's Act Two
- deleted 3y ago[deleted]
- malux85 3y agoIs there room for generative AI in science? I am experimenting with this a lot at https://atomictessellator.com https://atomictessellator.com, As a computational chemist I found it difficult just to stay on top of all of the papers that are released, I thought it would be cool to have generative AI attempt to reproduce the experiments using simulation tech. Here's a few cool insights I have uncovered while working on this: - Developer tools are popular, but there's an upcoming market for AI tools - I.e. Tooling/API's that are meant to be used primarily by LLMs/AIs. Design the interfaces to be well composable at the right level of abstraction and AIs can design, run and monitor experiments really well. - Tree of thought and Graph of thought are really, really, really important for this - I think this is because it compensates for the lack of looping mechanisms in LLMs and also adds the ability for recursive problem decomposition, abstraction laddering, composable malleability and so on. Does it work? Yes, I have a working E2E pipeline now that has already made novel discoveries validated by lab work, I am focusing on scaling this out now to support a broader search space and give the LLMs more freedom to explore. A shameless plug: I am working on this 4-5 hours a day as well as my day job. If anyone is aware of any grants or investors that I could connect with, I would love to work on this full time. I am neurodivergent so I think running a company is not really my thing, but there must be an alternate way, advice anyone?
- radarsat1 3y ago> Is there room for generative AI in science? I would love a system like ChatGPT but targeted specifically at exploring existing literature. A system that can recommend papers to read, that you can chat with about your problem and can recommend approaches that have worked for others and tell you why. That you can prompt and refine and go into detail with while it helps you figure out what do next based on previous work. That can link you to actual papers to read. With ChatGPT, I get some of this, but I have to be veeery careful how I use it. It is generally good at discussing points at a high level and helping you sort out some ideas, but when you get into detail it is very easy to catch it making mistakes, and when you ask it for references to read further, it almost always makes up some or all of them. Forget asking it to give you actual links. Maybe some of the other GPT systems targeted at the search space are better at this, I don't know I haven't tried them. But I would love a system that actually does this kind of thing well. Not just for scientific research -- I've used ChatGPT to help understand some legal things, to help understand some government application procedures (cut through the "consulate speak" and explain some steps to me in my son's visa application in plain language. Of course I double-checked everything it told me.) Having a system that has "read everything" and can explain it back to you after you ask some questions is just fantastic. I just want it to be more reliable.
- bloaf 3y agoI agree, I think there is a wide open vista of opportunity in the hypothesis generation space when it comes to science. An instant literature-review-as-a-service would be incredibly valuable too, to say nothing of a, "write the code to implement the methods described in this paper."
- whimsicalism 3y agoYes, but most of the current ideas on how this moment will change science are willfully short-sighted. We are only a few years from next generation multimodal modeling. The future contains protein embeddings, genome embeddings, medical image embeddings, and chatbot decoders to discuss them with you. Imagine prompting an image-conditioned decoder with questions like "Q: Why do you think this brain MRI indicates the person will get Parkinson's?" These are things that models can currently do, but we have basically no understanding of what they are looking at.
- te_chris 3y agoTurns out everyone wants to sell shovels
- deleted 3y ago[deleted]
- worldsayshi 3y agoMaybe trying to sell shovels is at worst a good way to learn how to best use shovels or which shovels are actually best? Then you can get ahead in the shovelling game.
- deleted 3y ago[deleted]
- gumballindie 3y ago> Four decades of the internet (accelerated by COVID) has given us trillions of tokens’ worth of training data. Yeah four decades of stolen intellectual property posted by people on the internet in good will for the world to see, only to have it stolen and monetised by these folks. Wondering, once the good content dies out how will you “train” your “ai”?
- deleted 3y ago[deleted]
- ozr 3y agoModern models are increasingly trained on synthetic data.
- jahewson 3y agoHow was the synthetic data generator trained?
- ChrisClark 3y agoAre we not allowed to learn from other's work?
- gumballindie 3y agoYou can learn all you want, but ai is a software, not a “we”.
- warkdarrior 3y agoAI is a software tool used by a user. Is a (human) user not allowed to learn all they want by using whatever helper tools they want?
- lewhoo 3y agoIs using uTorrent also qualify ? Come on, I'm downloading only for educational purposes.
- nextworddev 3y agoAct 1 (basic enterprise adoption) hasn't even really shaped out, so it's premature to call anything Act 2.
- pphysch 3y agoAnd frankly it probably never will. Generative AI is not a platform unto itself. In order to effectively leverage fine-tuned models, you need your data to be somewhat organized in the first place. The average enterprise struggles enormously with this and always will. In short, I would bet on endemic platform providers (Google, MS, Adobe, SAP, etc) rather than standalone AI providers (OpenAI, startups). Who owns the data? That is where the value lies.
- phillipcarter 3y agoI think it absolutely has. Certainly in developer tools, but you can see it in other tools like Hubspot, Gong, etc. Not to mention the enterprise organizations who are building their own features that use LLMs. Yes, we'll see some "enterprises" adopt their first LLM in 10 years because some move as slow as molasses. That's fine; there's companies who are tasting Cloud for the first time right now, but nobody is saying that Cloud adoption is still in its first phase. I think Sequoia sees this and decided the train is already starting to roll, so it's time to look towards what's next.
- callalex 3y agoAre there any examples of companies turning a GAAP profit using generative models that aren’t selling the shovels, but actually using them to dig?
- jerrygenser 3y agoAdobe is already incorporated genAI into core product and will start pricing for compute units. This is part of their core product, not a tool being sold to other people to build genAI products
- 3y ago
- plra 3y ago>the compute horsepower to process exaflops of data Lol. How many bytes in a flop again? Or maybe FLOP = FuckLoad Of Petabytes.
- Wingy 3y agoWorking with two 16-bit floats you'd have 4 bytes in one FLOP?
- datadrivenangel 3y agoMBA = Master's of Business Administration.
- deleted 3y ago[deleted]
- slig 3y agoAt least we know that this PR wasn't written by GPT4.
- nabakin 3y agoJust a bunch of hype marketing. I didn't read anything of real substance, granted I didn't read the whole thing because it seemed pointless.
- rco8786 3y agoYea I gave up. Ironically it reads like it's (at least partially) AI generated. Long winded and boring without getting to the damn point.
- nabakin 3y agoI think you're right, the list of authors includes GPT-4
- danielmarkbruce 3y agoVCs writing shallow thought leadership pieces has become too common. They don't seem to yet realize everyone has figured it out for what it is - self serving.
- nabakin 3y agoIt's a very biased take too, as if it's already clear modern AI is on the level of the internet. If they believe this honestly, I'd say they are in a bubble. Modern AI is impressive, sure, but comparing it to the invention of the internet itself? It's too early to come to such conclusions imo.
- vsareto 3y agoBut are you sure you don't want to see "AI Special Forces" in job postings?
- berkle4455 3y agoJust remember that Sequoia deleted their Sam Bankman-Fried puff piece.
- 3y ago
- mabbo 3y agoThe moment that generative AI became something crazy for me was when I said "holy shit, maybe Blake Lemoine was right". Lemoine was the Google engineer who made a big fuss saying that Google had a sentient AI in development and he felt there were ethical issues to consider. And at the time we all sort of chuckled- of course Google doesn't have a true AGI in there. No one can do that. And it wasn't much later I had my first conversation with ChatGPT and thought "Oh... oh okay, I see what he meant". It's telling that all of these LLM chat systems are trained to quite strongly insist they aren't sentient. Maybe we don't yet know quite what to do with this thing we've built, but I feel quite strongly that what we've created with Generative AI is a mirror of ourselves, collectively. A tincture of our intelligence as a species. And every day we seem to get better distilling it into a purer form.
- danielmarkbruce 3y agoIsn't the takeaway : "holy shit, these things are advanced enough to make people like Blake Lemoine believe they are sentient?"
- tshaddox 3y agoOr "holy shit, we don't know enough about sentience to even begin to know whether something has it, other than humans, because we've gotten used to assuming that all human minds operate similarly to our own and experience things similarly to how we do."
- __loam 3y agoHFRL is literally just training the AI to be convincing. That's what these systems are optimized for.
- falcor84 3y agoIs it really that different from socialization (particularly primary socialization [0]), whereby we teach kids our social norms with the aim of them not being sociopaths? [0] https://en.wikipedia.org/wiki/Primary_socialization https://en.wikipedia.org/wiki/Primary_socialization
- RC_ITR 3y agoClassic Sequoia piece: VCs sharing lengthy, jargon-filled thoughts specific to their worldview (supplemented with narrowly relevant charts!) to imply that they have a grand unifying vision. Interesting that Sonya and Pat credited GPT-4 as an author; they could've used it to make things concise!
- deleted 3y ago[deleted]
- shwaj 3y ago“Four decades of the internet (accelerated by COVID) has given us trillions of tokens’ worth of training data.” What’s up with the “accelerated by COVID”? It feels completely out of place. Would we have not had enough training data if COVID didn’t happen? Blessings in disguise, I guess.
- Bjorkbat 3y agoI’m mildly curious if those additional tokens will make AI better, or worse. We arguably trained AI on the good stuff first. Novels. Wikipedia entries. GitHub open-source projects with a lot of stars. What’s left but mediocrity and our “baser” internet ramblings? Some researchers already found out that AI-sourced content can affect the models, but what about content from increasingly out-of-touch people?
- deleted 3y ago[deleted]
- jhbadger 3y agoThe only way I can interpret that is assuming it is referring to the massive uptick in video conferencing which COVID caused. Although I'm not sure what data programs like Zoom actually collected and if it was shared with generative AI creators.
- latchkey 3y ago"AI-first infrastructure companies like Coreweave, Lambda Labs, Foundry, Replicate and Modal are unbundling the public clouds and providing what AI companies need most: plentiful GPUs at a reasonable cost, available on-demand and highly scalable, with a nice PaaS developer experience." Oddly to me, this is playing down the point in the "what we got wrong" section... "2. The bottleneck is on the supply side." Try signing up for a Coreweave account.
- deleted 3y ago[deleted]
- GiorgioG 3y agoCan't wait for most of these companies to disappear once the hype dies down and the money dries up. Generative AI in its current form is a huge letdown.
- __loam 3y agoI've gotten like 5 recruiter calls this week from companies that are basically just gpt wrappers.
- whimsicalism 3y agoTypical HN pessimism, but I strongly disagree with anyone characterizing it as a huge letdown. Even if you personally don't get much use out of it (honestly hard for me to believe if you give it a fair try), there are tons of uses that will be impacting you significantly within the next year. Contrast that with crypto.
- tracerbulletx 3y agoIt's going to be one of those things like the internet where over the next decade it will have massively impacted every aspect of life but no one will fully appreciate how much everything has changed.
- wnc3141 3y agoIt's already replaced a bunch of search for me. I don't use it to make my own work, but I use it as a really fast q/a tool for straightforward answers absent of ads and clickbait
- GiorgioG 3y agoI paid for ChatGPT for a few months and finally cancelled yesterday. As a code generator it’s a frustrating assistant that gets it wrong most of the time. I guess that’s good from a selfish developer point of view.
- whimsicalism 3y agoIt works quite well for my usecase as a coder - generating boilerplate UDFs for spark, etc., generating boilerplate python functions from a description of what it does, rewriting code to handle errors, asking it to review simple code.
- dvt 3y agoTypical Sequoia "think piece" meant to bring in tier-2 and tier-3 VCs onto the AI hype train, further bolstering their own position. I don't even think Sequoia believes in AI per se, they're just chasing the next shiny ball. It's genius if you really think about it. A year ago, they were gushing over SBF. There's no due diligence here, there's no product market fit. It's a legal pump and dump. Much like crypto, there will be bag-holders, but it won't be Sequoia or a16z, they're far too smart for that. If they get lucky and hit a home run, they were the visionaries all along. Their hubris is frankly quite admirable. The world goes 'round, and the rich get richer.
- popcorncowboy 3y agoAnd if it all goes sideways, quietly take down the puff piece bollocks and continue to curate your garden of unremitting genius. Here's to the bag holders. These pieces are such cringe to read it hurts.
- geodel 3y agoExactly right. They do seem to be getting high on their own supply.
- KaiserPro 3y agoI see we are in peak bullshit again. AI will be a force amplifier. great, we get that. I suspect that this kind of vague marketing bollocks will be generated almost without any oversight soon. However there are also risks. Just like calico which used to be a really expensive luxury, and then with empire and colonialism, and later mechanisation became a cheap(er) commodity, LLMs will kneecap whole industries. I think any site that relies on advertising on the web is going to die. This includes google, as they are the gateway to the web, and rely on you using them to answer questions obliquely. This means that the need for web developers and support staff is going to decrease. I think the open web will shrink to ~20% it's present size. It will mostly revert back to hobbyists and people arguing. Things that require up to date data will continue (news, sports, gaming, etc.) Social media will either thrive, or revert to small group chats (think half whatsapp half tiktok/instagram), to get away from autogenerated horseshit. As for streaming TV/movies. AI porn is the canary there. If that takes off, then your kardashians/court reality/low budget drama might feasibly be generated (we are a way off though)
- otabdeveloper4 3y ago> I think any site that relies on advertising on the web is going to die. What? No. The only known useful application of generative LLMs is to write automatic ad copy. This means the cost of doing advertising will lower dramatically and we'll get more of it. (Mostly "native" ads in places where putting ad copy was previously not economically feasible - e.g., twitter posts, tiktoks, etc.)
- KaiserPro 3y agoWhere is the vast majority of advertising now? on websites. If you are not going to websites anywhere near as much to get info, then the cost per click goes through the roof. Money drains from the system, those websites disappear.
- warthog 3y agoI think a great deal of opportunity is indeed being missed: AI Porn. History shows that you can always sell a lot if you rely on the 7 sins of men, in this case lust. Porn has always been a sort of a pioneer in the tech space anyways but none of the VCs have the agreements in place to invest in such a space. Though I know AI Porn is going to print money. Especially when video generation becomes remotely usable.
- pedalpete 3y agoI love the AI space, but I wonder if the internet comparison is valid? Isn't it more like the cloud or mobile? Most apps will have AI of some sort. It's an infrastructure serving the problem. It isn't a completely new distribution medium, consumption mechanism.
- warthog 3y agoAgreed that it is different to mobile or cloud that it is not a distribution mechanism
- rcarmo 3y agoHilariously for me (I’m a Microsoft FTE, fyi) there’s no mention of Bing, any of the multiple Copilot products in the Office/enterprise space, or, tellingly, the Azure OpenAI service (which provides you with private endpoints and models). I do see startups that use it, but it is a grossly lopsided view of the “market”. (And regardless of my employer, there are a lot of consumer and enterprise-focused solutions in that space - including direct competition - that aren’t mentioned anywhere.)
- redwood 3y agoThere is a large community of folks that simply can't stand your employer. I hate to break that to you. The anti-competitive antics and open source washing has no end