13 ms·
[flagged]
by yosito 2y ago
[flagged]
- apsec112 2y agoWhat evidence or data, if you (hypothetically) saw it, do you think would disprove the thesis that "[LLMs] will ALWAYS be the wrong tool for the job"?
- yosito 2y agoYou're attempting to set goal posts for a logical argument, like we're talking about religion or politics, and you've skipped the part about mutually agreeing on definitions. Define what an LLM is, in technical terms, and you will have your answer about why it is not intelligent, and not capable of reasoning. It is a statistical language model that predicts the next token of a plausible response, one token at a time. No matter how you dress it up, that's all it can ever do, by definition. The evidence or data that would change my mind is if instead of talking about LLMs, we were talking about some other technology that does not yet exist, but that is fundamentally different than an LLM.
- apsec112 2y agoIf we defined "LLM" as "any deep learning model which uses the GPT transformer architecture and is trained using autoregressive next-token prediction", and then we empirically observed that such a model proved the Riemann Hypothesis before any human mathematician, it would seem very silly to say that it was "not intelligent and not capable of reasoning" because of an a-priori logical argument. To be clear, I think that probably won't happen! But I think it's ultimately an empirical question, not a logical or philosophical one. (Unless there's some sort of actual mathematical proof that would set upper bounds on the capabilities of such a model, which would be extremely interesting if true! but I haven't seen one.)
- yosito 2y agoLet's talk when we've got LLMs proving the Riemann Hypothesis (or any mathematical hypothesis) without any proofs in the training data. I'm confident in my belief that an LLM can't do that, and will never be able to. LLMs can barely solve elementary school math problems reliably.
- deleted 2y ago[deleted]
- valval 2y agoIf the cure for cancer arrived to us in the form of the most probable token being predicted one at a time, would your view on the matter change in any way? In other words, do you have proof that this medium of information output is doomed to forever be useless in producing information that adds value to the world? These are of course rhetorical questions that you nor anyone else can answer today, but you seem to have a weird sort of absolute position on this matter, as if a lot depended on your sentiment being correct.
- MuffinFlavored 2y ago> but I have a feeling that part of the reason they're charging so much is to give people more confidence in the model Or each user doing an o1 model prompt is probably like, really expensive and they need to charge for it until they can get cost down? Anybody have estimates on what a single request into o1 costs on their end? Like GPU, memory, all the "thought" tokens?
- yosito 2y agoPerplexity does reasoning and searching, for $10/mo, so I have a hard time believing that it costs OpenAI 20x as much to do the same thing. Especially if OpenAI's model is really more advanced. But of course, no one except internal teams have all of the information about costs.
- Max-q 2y agoI would say using performance of Perplexity as a benchmark for the quality of o1-pro is a stretch?
- yosito 2y agoFind third party benchmarks of the relevant models and then this discussion is worth having. Otherwise, it's just speculation.
- awongh 2y agoIs $200 a lot if you end up using it quite often? It makes me wonder why they don't want to offer a usage based pricing model. Is it because people really believe it makes a much worse product offering? Why not offer some of the same capability as pay-per-use?
- nemonemo 2y agoWouldn't you say the same thing for most of the people? Most of the people suck at verifying truth and reasoning. Even "intelligent" people make mistakes based on their biases. I think at least LLMs are more receptive to the idea that they may be wrong, and based on that, we can have N diverse LLMs and they may argue more peacefully and build a reliable consensus than N "intelligent" people.
- jazzyjackson 2y agoThe difference between a person and a bot is that a person has a stake in the outcome. A bot is like a person who's already put in their two weeks notice and doesn't have to be there to see the outcome of their work.
- MichaelZuo 2y agoThat’s still amazing quality output for someone working for under $1/hour?
- Smaug123 2y agoIt's not obvious that one should prefer that, versus not having that output at all.
- MichaelZuo 2y agoWhy does that matter? Even if it was a consensus opinion among all HN users, which hardly seems to be the case, it would have little impact on the other billion plus potential customers…
- lukan 2y agoIntelligent people will know they made a mistake, if given a hint and figure out what went wrong. A LLM will just pretend to care about the error and happily repeats the error over and over.
- sangeeth96 2y ago
- AlanYx 2y ago$200 a month is potentially a bargain since it comes with unlimited advanced voice. Via the API, $200 used to only get you 14 hours of advanced voice.
- htrp 2y agoyou'll be throttled and rate limited
- jerjerjer 2y agoDoes it give unlimited API access though?
- AlanYx 2y agoNo (naturally). But my thought process is that if you use advanced voice even half an hour a day, it's probably a fair price based on API costs. If you use it more, for something like language learning or entertaining kids who love it, it's potentially a bargain.
- yosito 2y agoI've got unlimited "advanced voice" with Perplexity for $10/mo. You're defining a bargain based on the arbitrary limits set by the company offering you said bargain.
- dumbmrblah 2y agoThe advanced voice of ChatGPT is miles ahead of the Perplexity one. I subscribe to both.
- crazygringo 2y ago> In other words, it's a con. A con like that wouldn't last very long. This is for people who rely enough on ChatGPT Pro features that it becomes worth it. Whether they pay for it because they're freelance, or their employer does. Just because an LLM doesn't boost your productivity, doesn't mean it doesn't for people in other lines of work. Whether LLM's help you at your work is extremely domain-dependent.
- matteoraso 2y ago>Whether LLM's help you at your work is extremely domain-dependent. I really doubt that, actually. The only thing that LLMs are truly good for is to create plausible-sounding text. Everything else, like generating facts, is outside of its main use case and known to frequently fail.
- rusticpenn 2y agoI use llms to do most of my dunki work.
- tiahura 2y agoLLMs have become indispensable for many attorneys. I know many other professionals that have been able to offload dozens of hours of work per month to ChatGPT and Claude.
- PittleyDunkin 2y agoWhat on earth is this work that they're doing that's so resilient to the fallible nature of LLMs? Is it just document search with a RAG?
- tiahura 2y agoEverything. Drafting correspondence, pleadings discovery, discovery responses. Reviewing all of the same. Reviewing depositions, drafting deposition outlines. Everything that is “word processing,” and that’s a lot.
- lm28469 2y ago> $200 a month for this is insane Losing $5-10b per year also is insane. People are still looking for the added value, it's been 2 whole years now
- MP_1729 2y agoMy new internet is in his 3rd day in the job and he's still behind o1-preview with less than 25 prompts.
- vessenes 2y agoI mean this in what I hope will be taken in the most helpful way possible: you should update your thinking to at least imagine that intelligent thoughtful people see some value in ChatGPT. Or alternately that some of the people who see value in ChatGPT are intelligent and thoughtful. That is, aim for the more intelligent "Interesting, why do so many people like this? Where is it headed? Given that, what is worth doing now, and what's worth waiting on?" over the "This doesn't meet my standards in my domain, ergo people are getting scammed." I'll pay $200 a month, no problem; right now o1-preview does the work for me of a ... somewhat distracted graduate student who needs checking, all for under $1 / day. It's slow for an LLM, but SUPER FAST for a grad student. If I can get a more rarely distracted graduate student that's better at coding for $7/day, well, that's worth a try. I can always cancel.
- yosito 2y ago[flagged]
- vessenes 2y agoI think you did make some strong inferences about others when you said "it's a con." But I'm actually not interested in the other people, or defending against an ad hominem attack - I'm comfortable making my own purchase decisions. My intent was to say "you seem like a smart person, but you seem to have a blind spot here, might benefit you to stay more open minded."
- yosito 2y agoLook in the mirror and read your comment back.
- ghshephard 2y agoIf you do a lot of work in an area that o1 is strong in - $200/month effectively rounds down to $0 - and a single good answer at the right time could justify that entire $200 in a single go.
- yosito 2y agoPresumably, this is what they want the marks buying the $200 plan to think. Whether it's actually capable of providing answers worth $200 and not just sweet talking is the whole question.
- daveguy 2y agoI feel like a single bad answer at the wrong time could cost a heck of a lot more than $200. And these LLMs are riddled with bad answers.
- amelius 2y agoThink of it as an intern. Don't trust everything they say.
- crackrook 2y agoI would hesitate to hire an intern that makes incorrect statements with maximum confidence and with no ability to learn from their mistakes.
- pie420 2y agonothing chatgpt says is with maximum confidence. the EULA and terms of use are riddled with "no guarantee of accuracy" and "use at own risk"
- albumen 2y agoNo they're right. ChatGPT (and all chargers) responds confidently while making simple errors. Disclaimers upon signup or in tiny corner text are so at odds with the actual chat experience.
- dubeye 2y agoIf i'm happy to pay 20 in retirement just for the odd bit of writing help, then i can easily imagine it being worth 200 to someone with a job
- josephg 2y agoYep. I’m currently paying for both Claude and chatgpt because they’re good at different things. I can’t tell whether this is extremely cheap or expensive - last week Claude saved me about a day of time by writing a whole lot of very complex sql queries for me. The value is insane.
- cryptoegorophy 2y agoyeah, as someone who is far from programming, the amount of time and money it saved me helping me make sql queries and making php code for wordpress is insane. It even helped me fix some wordpress plugins that had errors and you just copy paste or even screenshot those errors until they get fixed! If used correctly and efficiently the value is insane, I would say $20, $200 is still cheap for such an amazing tool.
- behringer 2y agoI kind of feel this is a kick in the face. Now I'll forever be using a second rate model because I'm not rich enough. If I'm stuck using a second rate model I may go find someone else's model to use.
- raincole 2y agoThe problem isn't whether ChatGPT Pro can save you $200/mo (for most programmers it can.) The problem is whether it can saves you $180/mo more than Claude does.
- choppaface 2y agoThey claim unlimited access, but in practice couldn't a user wrap an API around the app and use it for a service? Or perhaps the client effectively throttles use pretty aggressively? Interesting to compare this $200 pricing with the recent launch of Amazon Nova, which has not-equivalent-but-impressive performance for 1/10th the cost per million tokens. (Or perhaps OpenAI "shipmas" will include a competing product in the next few days, hence Amazon released early?) See e.g.: https://mastodon.social/@mhoye/113595564770070726 https://mastodon.social/@mhoye/113595564770070726
- wslh 2y ago> $200 a month for this is insane, but I have a feeling that part of the reason they're charging so much is to give people more confidence in the model. Is it possible that they have subsidized the infrastructure for free and paid users and they realized that OpenAI requires a higher revenue to maintain the current demand?
- yosito 2y agoYes, it's entirely possible that they're scrambling to make money. That doesn't actually increase the value that they're offering though.
- eigenvalue 2y agoCouldn't disagree more, I will be signing up for this as soon as I can, and it's a complete no brainer.
- talldayo 2y ago[flagged]
- yosito 2y ago[flagged]
- cdrini 2y agoWhat will you be using it for? Where do you think you'll see the biggest benefit over the cheaper plan?
- thelastparadise 2y agoThe megga disappointment is o1 is performing worse than o1-preview [1], and claude 3.6 had already nearly caught up to o1-preview. 1. https://x.com/nrehiew_/status/1864763064374976928 https://x.com/nrehiew_/status/1864763064374976928
- eigenvalue 2y agoFor programming. I've already signed up for it and it seems quite good (the o1 pro model I mean). I was also running into constraints on o1-preview before so it will be nice to not have to worry about that either. I wish I could get a similar more expensive plan for Claude 3.5 Sonnet that would let me make more requests.
- jrflowers 2y ago> In other words, it's a con. I'm a paying Perplexity user I love this back-to-back pair of statements. It is like “You can never win three card monte. I pay a monthly subscription fee to play it.”
- yosito 2y agoI pay $10/month for perplexity because I fully understand its limitations. I will not pay $200/month for an LLM.
- monkey_monkey 2y agoI am CERTAIN you do not FULLY understand its limitations.
- yosito 2y agomkay
- deleted 2y ago[deleted]
- monkey_monkey 2y agoyeah, that's what i thought.
- JCharante 2y agoit's literally the cost of a cup per coffee per day
- talldayo 2y agoOr the price of replacing your espresso machine on a monthly basis.
- yosito 2y agoWhen you put it this way, I think I need to finally buy that espresso machine.
- mwigdahl 2y agoNot if you make coffee at home.
- latexr 2y agoI don’t drink coffee. But even if I did, and I drank it everyday at a coffeehouse or restaurant in my country (which would be significantly higher quality than something like a Starbucks), it wouldn’t come close to that cost.
- tiltowait 2y agoThis argument only works in isolation, and only for a subset of people. “Cost of a cup of coffee per day” makes it sound horrifically overpriced to me, given how much more expensive a coffee shop is than brewing at home.
- tiahura 2y agoOr an Avacado Toast.
- specproc 2y agoIn America. If you drink your coffee from coffee shops.
- vunderba 2y agoNot to be glib, but where do you live such that a single cup of coffee runs you seven USD? Just to put that into perspective. I also really don't find comparisons like this to be that useful. Any subscription can be converted into an exchange rate of coffee, or meals. So what?
- brookst 2y agoI’d like to see more evidence that it’s a scam than just your feelings. Any data there? I certainly don’t see why mere prediction can’t validate reasoning. Sure, it can’t do it perfectly all the time, but neither can people.
- talldayo 2y ago> I’d like to see more evidence that it’s a scam Have you been introduced to their CEO yet? 5 minutes of Worldcoin research should assuage your curiosity.
- latexr 2y agohttps://www.technologyreview.com/2022/04/06/1048981/worldcoin-cryptocurrency-biometrics-web3/ https://www.technologyreview.com/2022/04/06/1048981/worldcoi... https://www.buzzfeednews.com/article/richardnieva/worldcoin-crypto-eyeball-scanning-orb-problems https://www.buzzfeednews.com/article/richardnieva/worldcoin-...
- brookst 2y agoSo you’ve got feelings and guilt by association. And I’ve got a year of using ChatGPT, which has saved tens to hundreds of hours of tedious work. Forgive me for not finding your argument persuasive.
- yosito 2y agoGuilt by association? It's literally the same guy.
- brookst 2y agoYou’re saying the company’s product has no value because another company by the same guy produced no value. That is the literal definition of guilt by association: you are judging the chatgpt produced based on the worldcoin product’s value. As a customer, I don’t care about the people. I’m not interested in either argument by authority (if Altman says it’s good it must be good) or ad hominem (that Altman guy is a jerk, nothing he does can have value). The actual product. Have you tried it? With an open mind?
- jack_riminton 2y agoIf a model is good enough (I’m not saying this one is that level) I could imagine individuals and businesses paying 20,000 a month. If they’re answering questions at phd level (again, not saying this one is) then for a lot of areas this makes sense
- yosito 2y agoLet me know when the models are actually, verifiably, this good. They're barely good enough to replace interns at this point.
- TeMPOraL 2y agoLet me know where you can find people that are individually capable at performing at intern level in every domain of knowledge and text-based activity known to mankind. "Barely good enough to replace interns" is worth a lot to businesses already. (On that note, a founder of a SAP competitor and a major IT corporation in Poland is fond of saying that "any specialist can be replaced by a finite number of interns". We'll soon get to see how true that is.)
- jll29 2y agoCześć! Since when does SAP have competitors? ;-P A friend of mine claims most research is nowadays done by undergraduates because all senior folks are too busy.
- etrautmann 2y agopostdocs but yeah
- ssl-3 2y agoLet me know what kind of intern you can keep around 24/7 for a total monthly outlay of $200, and then we can compare notes.
- 2y ago
- ren_engineer 2y agotarget market is probably people who will write it off as a business expense
- deleted 2y ago[deleted]
- fzeindl 2y ago> After awhile, I started realizing that these mistakes are present in almost all topics. A fun question I tried a couple of times is asking it to give me a list with famous talks about a topic. Or a list of famous software engineers and the topics they work on. A couple of names typically exist but many names and basically all talks are shamelessly made up.
- valval 2y agoIf you understood the systems you’re using, you’d know the limitations and wouldn’t marvel at this. Use search engines for searching, calculators for calculating, and LLMs for generating text.
- bowsamic 2y agoWhenever I’ve used ChatGPT for this exact thing it has been very accurate and didn’t make up anyone
- crowcroft 2y agoConsidering no one makes money in AI, maybe this is just economics.
- JSDevOps 2y agoExactly I thought this, People falsely equate high price == high quality. Basically with the $200 you are just donating to their cloud bills
- 8f2ab37a-ed6c 2y agoMy main concern with $200/mo is that, as a software dev using foundational LLMs to learn and solve problems, I wouldn't get that much incremental value over the $20/mo tier, which I'm happy to pay for. They'd have to do a pretty incredible job at selling me on the benefits for me to pay 10x the original price. 10x for something like a 5% marginal improvement seems sus.
- tippytippytango 2y agoIt’s like hiring an assistant. You could hire one for 60k/year. But you wouldn’t do it unless you knew how the assistant could help you make more than 60k per year. If you don’t know what to do with an employee then don’t hire them. If you don’t know what to do with expensive ai, don’t pay for it.
- metacritic12 2y agoDo you also think $40K a year for Hubspot is insane? What about people who pay $1k in order to work on a field for 4 hours hitting a small ball with a stick? The truth is that there are people who value the marginal performance -- if you think it's insane, clearly it's not for you.
- digitcatphd 2y agoTheir demo video was uploading a picture of a birdhouse and asking how to build it
- echelon 2y agoI'm extremely excited because this margin represents opportunity for all the other LLM startups.
- Barrin92 2y ago>What about people who pay $1k in order to work on a field for 4 hours hitting a small ball with a stick? Those people want to purchase status. Unless they ship you a fancy bow tie and a wine tasting at a wood cabin with your chatgpt subscription this isn't gonna last long. This isn't about marginal performance, it's an increasingly desperate attempt to justify their spending in a market that's increasingly commodified and open sourced. Gotta convince Microsoft somehow to keep the lights on if you blew tens of billions to be the first guy to make a service that 20 different companies are soon gonna sell for pennies.
- Salgat 2y agoThe performance difference seems minor, so this is a great way for the company to get more of its funding from whales versus increasing the base subscription fee.
- vbezhenar 2y agoI would pay $200 for GPT4o. Since GPT4, ChatGPT is absolutely necessary for my work and for my life. It changed every workflow like Google changed. I'm paying $20 to remove ads from youtube which I watch may be once a week, so $20 for ChatGPT was a steal. That said, my "issue" might be that I usually work alone and I don't have anyone to consult with. I can bother people on forums, but these days forums are pretty much dead and full of trolls, so it's not very useful. ChatGPT was that thing that allows me to progress in this environment. If you work in Google and can ask Rob Pike about something, probably you don't need ChatGPT as much.
- outside415 2y agothis is more or less my take too. if tomorrow all Claude and ChatGPT became $200/month I would still pay. The value they provide me with far, far exceeds that. so many cynics in this thread.
- throwaway314155 2y agoYou don't have to be a cynic to be annoyed with a $200/month price. Just make a normal amount of money.
- llm_trw 2y agoI'm singing up when I get home tonight.
- athrowaway3z 2y agoI've actually hit a interesting situation a few times that make use of this. If some language feature, argument, or configuration option doesn't exists it will hallucinate one. This hallucination is usually a very good choice to name the option / API.
- clutchdude 2y agoI've seen this before and it's frustrating to deal with chasing phantom APIs it invents. I wish it could just say "There is not a good approximation of this API existing - I would suggest reviewing the following docs/sources:....".
- tptacek 2y agoIs it insane? It's the cost of a new laptop every year. There are about as many people who won't blink at that among practitioners in our field as people who will. I think the ship has sailed on whether GPT is useful or a con; I've lost track of people telling me it's their first search now rather than Google. I'd encourage skeptics who haven't read this yet to check out Nicholas' post here: https://news.ycombinator.com/item?id=41150317 https://news.ycombinator.com/item?id=41150317
- rfoo 2y ago> It's the cost of a new laptop every year. It's the cost of a new, shiny, Apple laptop every year.
- yodsanklai 2y agoCould be a case of price discrimination [1], and a way to fuel the hype. [1] https://www.investopedia.com/terms/p/price_discrimination.asp https://www.investopedia.com/terms/p/price_discrimination.as...
- Kiro 2y ago> In other words, it's a con. Such a silly conclusion to draw based on a gut feeling, and to see all comments piggyback on it like it's a given feels like I'm going crazy. How can you all be so certain?
- yosito 2y agoYou don't have to be certain to be skeptical. But you should definitely be certain before you buy.
- DaveInTucson 2y agoRemember the whole "how many r's in strawberry" thing? Yeah, not really fixed: https://imgur.com/a/counting-letters-with-chatgpt-7cQAbu0 https://imgur.com/a/counting-letters-with-chatgpt-7cQAbu0
- nasmorn 2y agoI am a moderately successful software consultant and it is not even 1% of my revenue. So definitely not insane if it delivers the value. What I doubt though is that it can reach a mass market even in business. A good large high resolution screen is something that I absolutely consider to deliver the value it costs. Most businesses don’t think their employees deserve a 2k screen which will last for 6-10 years and thus costs just a fraction of this offering. Apparently the majority of businesses don’t believe in marginal gains