7 ms·
GPT-4 Outperforms Humans in Pitch Deck Effectiveness
- assbuttbuttass 3y agoThis says more about VC's than it does about AI
- moglito 3y agoI'm in agreement regarding more. But I think it also says something about clear messaging and how many people struggle with that. It's actually consistent with YC advice. They say most pitches fail to even convey what the hell they are doing and what problem they are solving. I suspect a language model will get that part across rather well.
- TechBro8615 3y agoI mean, if you're creating a pitch deck and not asking ChatGPT what it thinks about the messaging, you're being negligent. Of course you don't need to listen to its advice, but it takes five seconds to paste some text into ChatGPT and get additional suggestions that you might not have otherwise considered.
- yieldcrv 3y agolanguage model could, but did it in this experiment? we can all make decks that say things people want to hear and then go to jail 12 years later
- ftxbro 3y agowait isn't that the point of fake it until you make it though
- monnok 3y agoTo be fair to humans, 90% of the workday is spent using bizarre internal corporate speak that has evolved for indirectness approaching meaninglessness. It’s asking a lot to recognize a real transactional moment, codeswitch, and present effectively to the narrow goal. Even then, the narrow goal (maximize the chance of external funding) might be misaligned with the broad personal goal (don’t overcommit… on fucking anything). A robot can just maximize on the funding result. It didn’t spend its whole career being lectured in circles about the next “one-corpco initiative” that will replace everybody’s SAP front-ends or whatever.
- ftxbro 3y agoyeah i wont be impressed by ai until it comes and breaks down my door, pulls me out of my chair, and kicks my own ass
- famouswaffles 3y agoYawn. "This says more about [insert accomplishment] than it does about GPT" is getting tiring. It loses any potential meaning when that's the go to retort for any of its impressive achievements.
- Etheryte 3y ago> 1 in 5 investors and business owners pitched by GPT-4 would invest $10,000 or more. This is a completely useless metric and also a common pitfall. It's been shown many times that what people say they would invest/pay for something is completely unrelated to what they actually do when push comes to shove.
- seanhunter 3y agoThat in fact is pretty much the foundational insight behind capital markets- people say one thing and do another but when they vote with their wallets they say what they really mean.
- usaar333 3y agoYeah, but it's a comparison metric to the human ones, so is relevant to implying the GPT-4 set is "better".
- version_five 3y agoNot to mention, founder and team are basically the main criteria, at least early on. So people claiming to be making invest decisions without that are full of crap anyway.
- ThomPete 3y agoIn my experience having pitched to VC's it's not that there is a perfect deck but many really bad ones. I.e. there is diminishing returns on a "perfect" pitch deck. You just want to be in the category of not sucking and the decks level of perfection is no longer important. Today I would even go as long as to say the pitch deck is much less important than a demo. You can in theory pitch in an email, the rest is just to make everyone feel good about giving and receiving the money.
- TechBro8615 3y agoThe secret to raising money is getting the first offer. Everything falls into place after that, because investors are herd animals. Your pitch could go in one ear and out the other, but they'll suddenly start paying attention if you end it with "we've got an offer for $X at $Y cap, it's not from our ideal investor, and we'd prefer you lead the round, but we told them we'd get back to them by Wednesday and we just want to get back to work on building this thing."
- phas0ruk 3y ago100
- vidarh 3y ago> but many really bad ones. Indeed, having spent 4 years at a VC where we all took part in initial filtering of decks, the typical reasons decks failed to even make it to our investment committee, were: * Not describing the team, or not having a team, or thinking you could outsource key functions (e.g. we once rejected a "tech company" where the entire tech function including the CTO were outsourced to a consultancy with no link to the founders) * Not clearly explaining the idea. * Not setting out the size of the opportunity clearly. * Not setting out unit economics. * Not setting out a long enough plan (some might want 5, we wanted 10) that sets out an optimistic but possible case to a huge exit. If you go to an angel, they might be ok with a case for a 10m acquihire, but for a VC putting in $5m+, you better have a path towards a $100m+/year revenue business even if it's a long shot. It doesn't matter if the opportunity is huge if you plan to take a conservative approach that will hardly get you any of it.
- RcouF1uZ4gsC 3y agoI think this is actually a good thing. It seems there are a lot of people who have no talent but being able to hype up ideas. Often they have overshadowed the actual people who come up with and implement ideas. GPT-4 and LLMs in general democratizing generating BS may actually shift the advantage more to the actual doers. For example, There are probably on HN brilliant developers who have written at the core of really groundbreaking services and applications but will never be able to get consideration because they just don’t have the talent for making slick decks. Them being able to use GPT-4 to generate pitch decks would allow them to get their ideas in front of a lot more people.
- ftxbro 3y ago> "democratizing generating BS" I feel like this meaning of 'democratizing' is jumping the shark. Did that word always mean this? Is it going to keep meaning it?
- esjeon 3y ago> It seems there are a lot of people who have no talent but being able to hype up ideas. I get the sentiment, but PR is a critical skill in the free market. Also it's not easy to tell the difference b/w legit good ideas and empty decent PRs.
- Mordisquitos 3y agoI think that is GP commenter's point. If LLMs were to outperform humans in this area, such that anyone could produce a highly marketable pitch deck for any given business idea, then better PR would no longer provide a competitive edge with regards to getting funding, and would thus no longer be a critical skill in the free market. This could be a net positive, as whatever criterion investors may start using to decide which businesses to finance, at least it would no longer be a proxy of 'which startup had enough money to hire the best PR staff to sell their idea'.
- moffkalast 3y agoThe free market now sells this skill for $25/month.
- jeffbee 3y agoDemocratize ripping off clueless angels.
- dbfclark 3y agoThe key question here isn’t so much whether GPT-4 beats the actual human decks as much as what it had to fabricate to do so. The humans are probably constrained by things like “reality” and “what their business has done in it” while GPT-4 could make up anything it wanted. A fair comparison would be to humans given the same prompt and told to invent whatever facts they wanted.
- moglito 3y agoGreat point! Since they are not showing the decks, for all we know these GPT4 pitches may have been proposing they found a cure for cancer.
- EA-3167 3y ago"In addition, my plan guarantees every child a free kitten or puppy, along with the aforementioned immortality."
- SamBam 3y agoSure, but a human could write that too. Surely the onus is on the investor to assess how realistic they think the founders' claims are.
- EA-3167 3y agoThey fell for Juicero and Theranos, can we *really expect that from them?
- ftxbro 3y agois this some new hacker news poes law usually i can tell but this one is too spicy for me
- paulddraper 3y agoYes, but often in a later due diligence stage. And also the human will be on the hook for actually delivering, so sane founders will take that into account.
- usaar333 3y agoI find it weird to compare GPT-4 to not expert humans (which this did - they seemed to have just sampled pitch decks for funded companies). GPT-4 generally will beat non-experts. What you want to see is a comparison to say pitch deck consultants - it's also more useful to evaluate whether AI or a human should help you build one. This does suggest at minimum though that if you are a founder that GPT-4 will likely provide positive value in helping you relative to no one.
- golergka 3y agoIf a company was funded, doesn't it mean that pitch deck was successful?
- usaar333 3y agoOnly on a binary scale. There's even among actually funded companies better or worse pitch decks. You can often just get funding by having a good team matched to a VC that knows your industry. Pitch deck can be crap and other VCs might hard reject. Another potential methodology flaw is that they don't seem to group decks by VC industry. I definitely altered my deck for different VCs, in ways which could make it a worse deck for the "average VC". GPT-4 might be better at setting context that industry expert VCs don't need - meaning this isn't even comparing like and like.
- esjeon 3y ago> GPT-4 generally will beat non-experts. Just a nitpicking, but I would say GPT can easily beat untrained people. Clearly, GPT has already learned from all the good examples available on the internet, and it's nothing weird about that trained ones mostly outperform untrained ones.
- ftxbro 3y agoyeah nothing weird about these LLMs at all
- sailfast 3y agoNow analyze whether GPT-4 gets better returns than the VCs rating GPT-4 pitch decks!
- nivertech 3y agoAre there any examples? The ones I've seen looked more like low-quality ICO pitch decks ;)
- moneywoes 3y agoSeems like hallucinations would amount to fraud in these comparisons no?
- DotSauce 3y agoI bet I could improve your prompts and make the pitch decks even better. Let me know if you would like a prompt audit. I also have a detailed prompt for content analysis and scoring. The reason being is because I do tests like this as well to compare GPT-4 basic prompts to prompts enhanced by the SmartGPT AI tool I created.
- s1mon 3y agoWell, GPT is well known for being able to formulate very convincing bullshit.
- mkoubaa 3y agoBillion dollar bullshit generator is better at generating bullshit than humans
- croes 3y agoIsn't the same true for scammers?
- kernal 3y agoWe’ve reached the event AI horizon when we’re now scraping the barrel for chatGPT wins.
- mnd999 3y agoVCs are stupid. So what’s new?
- deleted 3y ago[deleted]
- phas0ruk 3y agoI would be interested to know how this was done. Unless they have access to Code Interpreter then gpt4 won’t generate graphs and images. Did they have it generate Python that made the graphs in matplotlib or something ? What did the prompts look like?
- arbuge 3y agoI had the same questions. The article was extremely light on how GPT-4 was actually used to generate the pitch decks. All it says is that various prompts were used to do so. That could mean lots of different things.
- jimsimmons 3y agoGah. You’d think people would’ve learnt to not fall for such things by now
- reducesuffering 3y agoDunking on VC is a useless takeaway. Ignoring snark, VC's and founders giving pitch decks are above-average educated intellectual demographics. This shows how much more convincing current LLM are than the majority of the population. Social media can have regular peoples' voices drowned out by a flood of more persuasive bots. Instead of actually talking issues out amongst each other, even on HN, it could be a deluge of output from "Give me the most convincing argument why they stole the election from Trump." Now imagine what will happen with state of the art LLM in a few years...
- ilaksh 3y agoReminds me of something I saw today. An AI-generated continuous "debate" between realistic Trump and Biden talking heads. It's basically 4th-grade level entertainment but seems like it could be tuned for something less ridiculous. https://m.twitch.tv/trumporbiden2024 https://m.twitch.tv/trumporbiden2024 Anyway, people with different political views generally do not "talk issues out".
- arisAlexis 3y agoOh maybe smarter than a dog huh Lecun?
- dist-epoch 3y agoHow about we try the other end too - have GPT-4 choose what to invest in, see how it compares with VCs.
- JimtheCoder 3y agoIsn't this where the whole "LLMs lack a worldview" type stuff comes in to play?
- z5h 3y ago“ Via a survey of 250 investors and 250 business owners…” Who cares what humans think. What did GPT-4 think about the pitches?
- moffkalast 3y ago"As an AI language model..."
- JimtheCoder 3y ago"What did GPT-4 think about the pitches?" Probably nothing.
- z5h 3y agoPoint being, why consider the outcome of GPT’s ability to dress up an idea to be persuasive, without considering its ability to cut through persuasive language and consider the core idea?
- ilaksh 3y agoThis is how we lose control. We won't have to wait until the AIs "wake up" and take over the nukes. Humans are very susceptible to bulls** which might be the most powerful force in the world. AI bulls** generation is already superhuman and continues to accelerate. The only way to counter it will be with your own AI bulls** generator going head to head with the competition. In the coming years, it will go faster and faster until it is dozens and then hundreds of times faster at generating high-level bulls** than humans. Due to their superior powers of persuasion, these AIs gather more and more power. Civilization ends in a hyperspeed barrage of superbulls** fighting superbulls**. It's already started. https://m.twitch.tv/trumporbiden2024 https://m.twitch.tv/trumporbiden2024
- JimtheCoder 3y agoBut if humans know the level of BS is increasing, and get burned by it once or twice (where it hurts), will they not adapt and become more adept at detecting BS?
- ilaksh 3y agoThe history of human civilization proves otherwise.
- neilv 3y agoBut does GPT-4 outperform a skilled human BSer. We sometimes see BSers of various degrees in business. Four thoughts: 1. A lot of things are seem easier for someone who knows what would be an advantageous thing to say in the moment, to that audience, and just says it, unconstrained by truth. 2. Some people are so good at BSing and complementary skills that they're incredibly slick at presentations. You might want to have them as a frontperson, until you consider... 3. Chickens tend to come home to roost. BSing is a scam tactic, and people tend to get burned by that. 4. Often the BSer themself seems immune to adverse repercussions. GPT-4 is Robo-BSer.
- superphil0 3y agoI personally can't really trust this. Their description of how they got the numbers leaves a lot of room for speculation
- ftxbro 3y agowell yeah thats because gpt shamelessly bullshits while the truthcel humans are afraid their tribe will exile them. still not agi but nice try.
- lee101 3y ago[dead]