7 ms·
How much would it have cost if GPT-4 had written your code
- ldoughty 3y agoIsn't this akin to: "The cost of an iphone is $500" ? Sure, it's $500 in parts if you just quantify the final working result... but it ignores all prototyping, overhead, and manual work to link together code (e.g. assembling the phone)... Maybe I'm misunderstanding just how good GPT-4 is at making entire repos of code... or exactly how this tool functions?
- starbugs 3y agoYou aren't misunderstanding it. This is not a valid way to calculate the cost of your code.
- SketchySeaBeast 3y agoWell, that's a fun exercise, but it's not the volume of code that has value, it's how it's put together.
- magic_hamster 3y agoIt's how much it would cost to produce the text files in your code. I think that most software engineers will agree that writing the code is the easiest part of the job. The hard parts are figuring out the problem, figuring out your data modeling, what is going to work and what it would cost, considering tradeoffs in speed, scalability, maintainability, etc. And, you know, getting the thing to actually run. There's no doubt that AI is going to be a huge factor in coding. But we will have to change a lot of things for it to outright replace humans. I'm sure some people are already working on those changes...
- mackopes 3y agoI could imagine an LLM that brainstorms all the edge cases, data modelling, algorithm, etc... The final decision will still have to be done by a human, but having a companion LLM that gives you ideas, plays devils advocate and looks for problems would definitely speed up the decision process.
- alfalfasprout 3y agoYeah, but then you need the knowledge, experience, and decision making to act on that feedback. LLMs are also notoriously bad at things like edge cases, etc. and more training data won't necessarily help.
- starbugs 3y agoUnless it leads you astray and that costs more time than coming up with the code on your own in the first place.
- jbverschoor 3y agoLet's just all print $100 bills.. it can be done for under $5
- starbugs 3y agoThis seems to assume that GPT4 can actually generate the code of your repo. Good luck with that assumption.
- mackopes 3y agoIt seems that the package calculates the lower bound of how much would it cost to write the same code with GPT-4. It does not take into consideration that even GPT-4 cannot write code at god mode and write it correctly on the first try. However, it gets interesting when you realise that if writing the code with an LLM is dirt cheap, then 1000 iterations of writing the same code with the guidance of a skilled software engineer would still be cheap and probably faster. I can imagine a world where whole engineering teams are replaced by just one engineer with a code-generating LLM.
- wild_pointer 3y agoIt's even cheaper to get a monkey to type that code for you. Will probably cost a banana. Chances it will type the code you expect? Similar to GPT-4 for anything remotely non-trivial.
- planetpluta 3y agoIf a banana costs ~60¢ (depending where you get it!) and GPT-4 is 6¢ per 1k tokens… you’re better off using a monkey for everything > 10k tokens!
- KMnO4 3y agoWhat’s the purpose of putting this on the package index instead of just a GitHub repository?
- pmontra 3y agoA lot of conversation is needed to create that code. I guesstimate that the actual tokens are at least 100 times than the ones in the repository, even if one can convince ChatGPT not to explain every choice it makes.
- starbugs 3y ago> convince ChatGPT not to explain every choice it makes I assume this is a major issue, because it tends to be more talky than my ex which is a real feat.
- BeefySwain 3y agoCome on everyone, we all know that "most of the work that goes into software engineering isn't writing code", that "it's impossible to create a working system in one shot", etc etc etc, and therefore this tool will not give a realistic idea of what it would cost to build a system with nothing but GPT-4. No one here (including the author) is under the impression that you can just ask ChatGPT to "write a Unix-like kernel" and get Linux spit out the other side. That said, I am really curious what it would cost, as a theoretical lower bound, to pay OpenAI to write the Linux Kernel. 10 grand? 100 grand? I have no clue, but I'm glad someone wrote a tool to make it easier to find out!
- londons_explore 3y ago> No one here (including the author) is under the impression that you can just ask Chat GPT to "write a Unix-like kernel" and get Linux spit out the other side. I really don't see such a thing being far off. GPT-4 is already pretty good at writing small modules ("write a disk IO queue for my custom kernel in C"). With a little more work in allowing GPT-4 to test out code it has written and iteratively make changes, allowing it to use debuggers, benchmarks, and sanitizers, allowing it to write its own tools, and then put modules together, I think we could very soon be asking it to "write me a Unix-like kernel".
- gwoolhurme 3y agoIt’s that good in your experience? Enough that it is close to writing something as nice as the Linux Kernel we have now? How close is close? Reading comments like these makes me feel like I’m swallowing crazy pills. It legitimately makes me feel like I’m using it wrong. I do Android and iOS native development and some IoT and it gives me really wrong code very often. To the point that I don’t see much difference between chatgpt and gpt4. But you say it’s close to just “write me a unix-like kernel”
- starbugs 3y agoIt's not close and I also see no big difference between GPT 3.5 and 4. Don't get hyped. I am sure it will eventually happen, but calling it close is very optimistic.
- dathinab 3y agoall your mony, as you never would have reached a complient program maybe with GPT-6 or so, we will see, but not with GPT-4
- gwoolhurme 3y agoThe entire premise of this thing is wrong too because what if you need to regenerate something? Which even with the mighty gpt4? Still happens a lot… It’s almost akin to me asking a co-worker how fast he types and calculating how much of his salary it is for him to generate code based on that without any other factors. Also pointed out but why is this even a package? It seems like the definition of bloat when this should have just been a blog post tool or something. It’s all completely unnecessary and seems to try and cash in on hype.
- tr33house 3y agonow, was this written ... by GPT-4?
- Mistletoe 3y agoCan anyone give me some estimates instead of defending your jobs as coders?
- gwoolhurme 3y agoWhat? Nobody is defending their jobs here. It’s explaining how non-trivial this is. What is the point of your comment? If you wanted pure token to money amount and are too lazy to calculate it your self use the tool in the link. That’s not how the real world even with an LLM will work. Let’s assume the LLM writes everything, even gpt4 won’t get everything in 1 go now. Requirements of software are hard to put into pure English… why write this snarky message. Adds nothing to the conversation.
- Mistletoe 3y agoThese comments remind me of newspaper articles saying the internet will fail and horse breeders saying cars will fail. All want to deflect to how wrong this is when it should be like the Drake Equation and can still give you data and ranges. With time and AI evolution, the estimates will get more close to this. “It is difficult to get a man to understand something, when his salary depends on his not understanding it.” -Upton Sinclair
- gwoolhurme 3y agoYou misunderstood what so many people here wrote with even more snark and a bad quote. It could and probably will replace my job and I have no contingency plan in place leaving me at great risk. That’s a given it seems. What isn’t is giving an estimate on a dollar amount and of time. Especially with any scientific rigor. The tool in the link is silly and that is what everyone is talking about. Perhaps you should try to be less bias reading the comments here.
- vba616 3y ago>It could and probably will replace my job People tell stories about employees at big companies whose entire job is to send a small report to someone once a week. Even in cases like that, it's illogical to fear AI, because if it really was so simple to automate, it would've been done already. Either the storytellers are clueless about the true nature of the job, or else there are defenses against redundancy that aren't obvious.
- Jeff_Brown 3y agoHow much money could a restaurant save by replacing the food with dirt.
- jumploops 3y agoIt cost me $7.61 to build this library[0] using GPT-4, much more than the direct cost of tokens required to reimplement the library. The majority of the cost was the large context windows being proxied back and forth to OpenAI from my local machine, rather than just the additional code with each new message. Also, I had no idea how to do what I was doing, so even if I could tell GPT-4 exactly what to build, I wasn't even sure it was possible! [0] https://github.com/jumploops/magic https://github.com/jumploops/magic
- furyofantares 3y agoNice - now please share the prompt to get it to write my code.
- tpoacher 3y agoYou mean before, or after the bankruptcy? /s
- zubairq 3y agoAs a developer I “wish” chatgpt could take my job. Really hoping it happens sooner than later so I can get my product out into the world faster. Have tried many chatgpt and other A.I. tools to do this but all of these AI coding tools seem to have no understanding of the code for maintenance when dealing with edge cases and things that are hard to explain to A.I. tools. I wish we had these tools tomorrow but it looks like it is 50 years off . In fact using chatgpt makes a lot of products take longer to build i suspect as 99.999 % of the work is in understanding the code for small upgrades and bug fixes and these tools seem to produce alot of complex and unmaintainable code