7 ms·
If you do a lot of work in an area that o1 is strong in - $200/month effectively rounds down to $0 - and a single good answer at the right time could justify th
by ghshephard 2y ago
If you do a lot of work in an area that o1 is strong in - $200/month effectively rounds down to $0 - and a single good answer at the right time could justify that entire $200 in a single go.
- yosito 2y agoPresumably, this is what they want the marks buying the $200 plan to think. Whether it's actually capable of providing answers worth $200 and not just sweet talking is the whole question.
- daveguy 2y agoI feel like a single bad answer at the wrong time could cost a heck of a lot more than $200. And these LLMs are riddled with bad answers.
- amelius 2y agoThink of it as an intern. Don't trust everything they say.
- crackrook 2y agoI would hesitate to hire an intern that makes incorrect statements with maximum confidence and with no ability to learn from their mistakes.
- pie420 2y agonothing chatgpt says is with maximum confidence. the EULA and terms of use are riddled with "no guarantee of accuracy" and "use at own risk"
- albumen 2y agoNo they're right. ChatGPT (and all chargers) responds confidently while making simple errors. Disclaimers upon signup or in tiny corner text are so at odds with the actual chat experience.
- daveguy 2y agoI think the point is that an LLM almost always responds with the appearance of high confidence. It will much quicker hallucinate than say "I don't know."
- Terr_ 2y agoAnd we, as humans, are having a hard time compartmentalizing and forgetting our lifetimes of language cues, which typically correlate with attention to detail, intelligence, time investment, etc. New echnology allows those signs to be counterfeited quickly and cheaply, and it tricks our subconscious despite our best efforts to be hyper-vigilant. (Our brains don't want to do that, it's expensive.) Perhaps a stopgap might be to make the LLM say everything in a hostile villainous way...
- Draiken 2y agoThey aren't talking about EULAs. It's how they give out their answers.
- crackrook 2y agoWhat I meant to say was that the model uses the verbiage of a maximally confident human. In my experience the interns worth having have some sense of the limits of their knowledge and will tell you "I don't know" or qualify information with "I'm not certain, but..." If an intern set their Slack status to "There's no guarantee that what I say will be accurate, engage with me at your own risk." That wouldn't excuse their attempts to answer every question as if they wrote the book on the subject.
- educasean 2y agoWhen you highlight only the negatives, yeah it does sound like no one should hire that intern. But what if the same intern happens to have an encyclopedia for a brain and can pour through massive documents and codebases to spot and fix countless human errors in a snap? There seems to be two camps: People who want nothing to do with such flawed interns - and people who are trying to figure out how to amplify and utilize the positive aspects of such flawed, yet powerful interns. I'm choosing to be in the latter camp.
- crackrook 2y agoThose are fair points, I didn't mean to imply that there are only negatives, and I don't consider myself to be in the former camp you describe as wanting nothing to do with these "interns". I shouldn't have stuck with the intern analogy at all since it's difficult for me to compare the two, with one being fairly autonomous and the other being totally reliant on a prompter. The only point I wanted to make was that an LLM's ability and propensity to generate plausible falsehoods should, in my opinion, elicit a much deeper sense of distrust than one feels for an intern, enough so that comparing the two feels a little dangerous. I don't trust an intern to be right about everything, but I trust them to be self aware, and I don't feel like I have to take a magnifying glass to every tidbit of information they provide.
- parthdesai 2y agoYeah, but you personally don't pay $200/month out of your pocket for the intern. Heck in Canada, govt. actually rebates for hiring interns and co-ops.
- crindy 2y agoIt's so strange to me that in a forum full of programmers, people don't seem to understand that you set up systems to detect errors before they cause problems. That's why I find ChatGPT so useful for helping me with programming - I can tell if it makes a mistake because... the code doesn't do what I want it to do. I already have testing and linting set up to catch my own mistakes, and those things also catch AI's mistakes.
- daveguy 2y agoYou assume programming software with an existing well-defined and correct test suite is all these will be used for.
- xandrius 2y agoThank you! I always feel so weird to actually use chatgpt without any major issues while so many people keep on claiming how awful it is; it's like people want it 100% perfect or nothing. For me if it gets me 80% there in 1/10 the time, and then I do the final 20%, that's still heck of a productivity boost basically for free.
- JamesBarney 2y agoWell now at $200 it's a little farther away from free :P
- thelastparadise 2y agoI could a car for that kind of money!
- xandrius 2y agoWhat do you mean? ChatGPT is free, the Pro version isn't. I'm talking of the generally available one, haven't had the chance to try this new version.
- crindy 2y agoYep, I’m with you. I’m a solo dev who never went to college… o1 makes far fewer errors than I do! No chance I’d make it past round one of any sort of coding tournament. But I managed to bootstrap a whole saas company doing all the coding myself, which involved setting up a lot of guard rails to catch my own mistakes before they reached production. And now I can consult with a programming intelligence the likes of which I could never afford to hire if it was a person. It’s amazing.
- stavros 2y agoIf I have to do the work to double-check all the answers, why am I paying $200?
- billti 2y agoWhy do companies hire junior devs? You still want a senior to review the PRs before they merge into the product right? But the net benefit is still there.
- stavros 2y agoWe hire junior devs as an investment, because at some point they turn into seniors. If they stayed juniors forever, I wouldn't hire them.
- jhgg 2y agoAre you implying this technology will remain static in its capabilities going forward despite it having seen significant improvement over the last few years?
- stavros 2y agoNo, I'm explicitly saying that gpt-4o-2024-11-20 won't get any smarter, no matter how much I use it.
- jhgg 2y agoDoes that matter when you can just swap it for gpt-5-whatever at some point in the future?
- stavros 2y agoSomeone asked why I hire juniors. I said I hire juniors because they get better. I don't need to use the model for it to get better, I can just wait until it's good and use it then. That's the argument.
- ZiiS 2y agoEven in this case loosing $200 + whatever vs a tiny bit higher chance of loosing $20 + whatever makes pro seem a good deal.
- ruszki 2y agoCompared to know things and not loosing whatever, both are pretty bad deals.
- daveguy 2y agoDoesn't that completely depend on those chances and the magnitude of +whatever? It just seems to me that you really need to know the answer before you ask it to be over 90% confident in the answer. And the more convincing sounding these things get the more difficult it is to know whether you have a plausible but wrong answer (aka "hallucination") vs a correct one. If you have a need for a lot of difficult to come up with but easy to verify answers it could be worth it. But the difficult to come up with answers (eg novel research) are also where LLMs do the worst.
- awestroke 2y agoEasy - don't trust the answers. Verify them
- malux85 2y agoThen the lesson you have learned is “don’t blindly trust the machine” Which is a very valuable lesson, worth more than $200
- deleted 2y ago[deleted]
- llm_trw 2y agoThat's why you have a human in the loop responsible for the answer.
- Kiro 2y agoWhat specific use cases are you referring to where that poses a risk? I've been using LLMs for years now (both directly and as part of applications) and can't think of a single instance where the output constituted a risk or where it was relied upon for critical decisions.