13 ms·

Using Claude 3.5 Sonnet in Cursor Composer already shows huge benefits for coding. I'm more productive than ever before. The models are still getting better and

by JanSt 2y ago

Using Claude 3.5 Sonnet in Cursor Composer already shows huge benefits for coding. I'm more productive than ever before. The models are still getting better and better. I'm not saying AGI is right around the corner or that we will reach it, but the benefits are undeniable. o1 added test-time compute. No need to be snarky.

jsheard 2y ago
There's no accounting for taste, but keep in mind that all of these services are currently losing money, so how much would you actually be willing to pay for the service you're currently getting in order to let it break even? There was a report that Microsoft is losing $20 for every $10 spent on Copilot subscriptions, with heavy users costing them as much as $80 per month. Assuming you're one of those heavy users, would you pay >$80 a month for it? Then there's chain-of-thought being positioned as the next big step forwards, which works by throwing more inferencing at the problem, so that cost can't be amortized over time like training can...
- HPsquared 2y ago
  It's called investment. You need to spend money to make money. Their costs will certainly come down.
- JanSt 2y ago
  1) The costs will go down over time, much of the cost is the margin of NVIDIA and training new models 2) Absolutely. Thats like one hour of an engineer salary for a whole month.
  - sofixa 2y ago
    > The costs will go down over time, much of the cost is the margin of NVIDIA and training new models Isn't each new model bigger and heavier and thus requries more compute to train?
    - JanSt 2y ago
      Yes, but 1) you only need to train the model once and the inference is way cheaper. Train one great model (i.e. Claude 3.5) and you can get much more than $80/month worth out of it. 2) the hardware is getting much better and prices will fall drastically once there is a bit of a saturation of the market or another company starts putting out hardware that can compete with NVIDIA
      - sofixa 2y ago
        > Train one great model (i.e. Claude 3.5) and you can get much more than $80/month worth out of it Until the competition outcompetes you with their new model and you have to train a new superior one, because you have no moat. Which happens what, around every month or two? > the hardware is getting much better and prices will fall drastically once there is a bit of a saturation of the market or another company starts putting out hardware that can compete with NVIDIA Where is the hardware that can compete with NVIDIA going to come from? And if they don't have competition, which they don't, why would they bring down prices?
        JanSt 2y ago
        The point is not that every lab will be profitable. There only needs to be one model in the end to increase our productivity massively, which is the point I'm making. Huge margins lead to a lot of competition trying to catch up, which is what makes market economies so successful.
        ben_w 2y ago
        > Until the competition outcompetes you with their new model and you have to train a new superior one, because you have no moat. Which happens what, around every month or two? Eventually one of you runs out of money, but your customers keep getting better models until then; and if the loser in this race releases the weights on a suitable gratis license then your businesses can both lose. But that still leaves your customers with access to a model that's much cheaper to run than it was to create.
        Workaccount2 2y ago
- brookst 2y ago
  Is there any reason to believe costs won’t come down with scale and hardware iteration, just like they did for everything else? Short term pricing inefficiency is not relevant to long term impact.
  - HarHarVeryFunny 2y ago
    Of course, but every token generated by a 100B model is going to take minimally 100B FLOPS, and if this is being used as an IDE typing assistant then there is going to be a lot of tokens being generated. If there is a common shift to using additional runtime compute to improve quality of output, such as OpenAI's GPT-o1, then FLOPs required goes up massively (OpenAI has said it takes exponential increase in FLOPS/cost to generate linear gains in quality). So, while costs will of course decrease, those $20-30K NVIDEA chips are going to be kept burring, and are not going to pay for themselves ... This may end up like the shift to cloud computing that sounds good in theory (save the cost of running your own data center), but where corporate America balks when the bill comes in. It may well be that the endgame for corporate AI is to run free tools from the likes of Meta (or open source) in their own datacenter, or maybe even locally on "AI PCs".
    - wongarsu 2y ago
      Which is why the work to improve the results of small models is so important. Running a 3B or even 1B model as typing assistant and reserving the 100B model for refactoring is a lot more viable.
    - brookst 2y ago
      > but every token generated by a 100B model is going to take minimally 100B FLOPS Drop the S, I think. There’s no time dimension. And FLOP is a generalized capability meaning you can do any operation. Hardware optimizations for ML can deliver the same 100B computations faster and cheaper by not being completely generalized. Same way ray tracing acceleration works: it does not use the same amount of compute as ray tracing in general CPU’s.
      - 2y ago
- archerx 2y ago
  I can already pay $0 a month and use uncensored local models for both text and images. Llama, Mixtral, Stable diffusion and Flux are a lot of fun and free to run locally, you should try them out.
  - jsheard 2y ago
    You can pay $0 for those models because a company paid $lots to train them and then released them for free. Those models aren't going away now of course, but lets not pretend that being able to download the product of millions of dollars worth of training completely free of charge is sustainable for future developments. Especially when most of the companies releasing these open models are wildly unprofitable and will inevitably bankrupt themselves when investments dry up unless they change their trajectory.
    - likium 2y ago
      Much could be said about open source libraries that companies release for free to use (kubernetes, react, firecracker, etc). It might be strategically make sense for them so in the meantime we’ll just reap the benefits.
      - skydhash 2y ago
        All of these require maintenance, and mostly it's been a treadmill just applying updates to React codebases. Complex tools are brittle and often only makes sense at the original source.
    - archerx 2y ago
      You’re acting as if computing power isn’t going to get better. With time training the models will get faster. Let me use CG rendering as an example. Back in the day only the big companies could afford to do photoreal 3D rendering because only they had access to the compute and even then it would take days to render a frame. Eventually people could do these renders at home with consumer hardware but it still took forever to render. Now we can render photoreal with path tracing at near realtime speeds. If you could go back twenty years and show CG artists the Unreal Engine 5 and show them it’s all realtime they would lose their minds. I see the same for A.I., now it’s only the big companies that can do it, then we will be able to do it at home but it will be slow and finally we will be able to train it at home for quick and cheap.
- binocarlos 2y ago
  I would pay hundreds of dollars per month for the combination of cursor and claude - I could not get my head around it when my beginner lever colleague said "I just coded this whole thing using cursor". It was an entire web app, with search filters, tree based drag and drop GUIs, the backend api server, database migrations, auth and everything else. Not once did he need to ask me a question. When I asked him "how long did this take" and expected him to say "a few weeks" (it would have taken me - a far more experienced engineer - 2 months minimum). His answer was "a few days". What I'm not saying is "AGI is close" but I've seen tangible evidence (only in the last 2 months), that my 20 year software engineering career is about to change and massively for the upside. Everyone is going to be so much more productive using these tools is how I see this.
  - threeseed 2y ago
    > 20 year software engineering career is about to change I have also been developing for 20+ years. And have heard the exact same thing about IDEs, Search Engines, Stack Overflow, Github etc. But in my experience at least how fast I code has never been the limiting factor in my project's success. So LLMs are nice and all but isn't going to change the industry all that much.
    - pluc 2y ago
      There will be a whole industry of people who fix what AI has created. I don't know if it will be faster to build the wrong thing and pay to have it fixed or to build the right thing from the get go, but after having seen some shit, like you, I have a little idea.
      - dumbfounder 2y ago
        Correction: a whole industry of AI that will fix what AI has created.
        vocram 2y ago
        Will AI also be on call when things break in production?
- gotaran 2y ago
  $80 a month is a no brainer given the productivity multiplier.
- ema 2y ago
  If it makes software developers 10% more productive there sure would be many companies who'd pay $80 a month per seat.
  - apwell23 2y ago
    It actually makes them less productive and creates havoc in codebases with hidden bugs and verbose code that ppl are copy pasting.
  - renegade-otter 2y ago
    It's like saying "AI is going to replace book writers because they are so much more productive now". All you will get is more mediocre content that someone will have to fix later - the same with code. 10% more productive. What does that mean? If you mean lines of code, then it's an incredibly poor metric. They write more code, faster. Then what? What are the long-term consequences? Is it ultimately a wash, or even a detriment? https://stackoverflow.blog/2024/03/22/is-ai-making-your-code-worse/ https://stackoverflow.blog/2024/03/22/is-ai-making-your-code...
    - ben_w 2y ago
      LLMs set a new minimum level; because of this they can fill in the gaps in a skillet — if I really suck at writing unit tests, they can bring me up from "none" to "it's a start". Likewise all the other specialities within software. Personally I am having a lot of fun, as an iOS developer, creating web games. No market in that, not really, but it's fun and I wouldn't have time to update my CSS and JS knowledge that was last up-to-date in 1998.
  - HarHarVeryFunny 2y ago
    Maybe there are people out there working in coding sweatshops churning out boilerplate code 8 hours a day, 50 weeks a year - people who's job is 100% coding (not what I would call software engineers or developers - just coders). It's easy to imagine that for such people (but do they even exist?!) there could be large productivity gains. However, for a more typical software engineer, where every project is different, you have full lifecycle responsibility from design through coding, occasional production support, future enhancements, refactorings, updates for 3rd party library/OD updates, etc/etc, then how much of your time is actually spent pure coding (non-stop typing) ?! Probably closer to 10-25%, and certainly no-where near 100%. The potential overall time saving from a tool that saves, let's say, 10-25% of your code typing is going to be 1-5%, which is probably far less than gets wasted in meetings, chatting with your work buddies, or watching bullshit corporate training videos. IOW the savings is really just inconsequential noise. In many companies the work load is cyclic from one major project to the next, with intense periods of development interspersed with quieter periods in-between. Your productivity here certainly isn't limited by how fast you can type.
- mklepaczewski 2y ago
  I would, and I don't use chatgpt as much as other people. I would pay for it for each of my employees.
- christkv 2y ago
  Also at some point you can run the equivalent model locally. There is no long term moat here i think and facebook seems hellbent of ensuring there will be no new google from llms
  - KoolKat23 2y ago
    I think physics at some point will get in the way, well at least for a while. An H100 costs like $20k-$30k and there's only so much compression/efficiency they can gain without beginning to lose intelligence, purely because you can't compute out of thin air.
- ben_w 2y ago
  > There's no accounting for taste, but keep in mind that all of these services are currently losing money, so how much would you actually be willing to pay for the service you're currently getting in order to let it break even Ok models already run locally; that aside, as the hosted ones are kinda similar quality to interns (though varying by field), the answer is "what you'd pay an intern". Could easily be £1500/month, depending on domain.
- infecto 2y ago
  Definitely. My time is valuable and I would spend multiples more on the current subscription costs.
- fassssst 2y ago
  Why do you assume they’re losing money on inference?
- jejeyyy77 2y ago
  - it won't work. - ok it works, but it won't be useful. - ok it's useful, but it won't scale. - ok it scales, but it won't make any money. - ok it makes money, but it's not going to last. etc etc
  - pdinny 2y ago
    Retrospectively framing technologies that succeeded despite doubts at the time discounts those that failed. After all, you could have used the exact same response in defense of web3 tech. That doesn't mean LLMs are fated to be like web3, but similarly the outcome that the current expenditure can be recouped is far from a certainty just because there are doubters.
  - farts_mckensy 2y ago
    There certainly has been some goal post moving over the past few months. A lot of the people in here have some kind of psychological block when it comes to technology that may potentially replace them one day.
    - KoolKat23 2y ago
      Yeah currently the sentiment seems to be "okay fine it works for simple stuff but won't deal with my complex query so it can be dismissed outright." Save yourselves some time and use it for that simple stuff folks.
- chmod775 2y ago
  > There was a report that Microsoft is losing $20 for every $10 spent on Copilot subscriptions, with heavy users costing them as much as $80 per month. Assuming you're one of those heavy users, would you pay >$80 a month for it? I'm probably one of those "heavy users", though I've only been using it for a month to see how well it does. Here's my review: Large completions (10-15 lines): It will generally spit out near-working code for any codemonkey-level framework-user frontend code, but for anything more it'll be at best amusing and a waste of time. Small completions (complete current line): Usually nails it and saves me a few keystrokes. The downside is that it competes for my attention/screen space against good old auto-completion, which costs me productivity every time it fucks up. Having to go back and fix identifiers in which it messed up the capitalization/had typos, where basic auto-complete wouldn't have failed is also annoying. I'd pay about about $40 right now because at least it has some entertainment value, being technologically interesting.
  - jeremy151 2y ago
    I find tools where I am manually shepherding the context into an LLM to work much better than Copilot at current. If I think thru the problem enough to articulate it and give the model a clear explanation, and choose the surrounding pieces of context (the same stuff I would open up and look at as a dev) I can be pretty sure the code generated (even larger outputs) will work and do what I wanted, and be stylistically good. I am still adding a lot in this scenario, but it's heavier on the analysis and requirements side, and less on the code creation side. If what I give it is too open ended, doesn't have enough info, etc, I'll still get a low quality output. Though I find I can steer it by asking it to ask clarifying questions. Asking it to build unit tests can help a lot too in bolstering, a few iterations getting the unit tests created and passing can really push the quality up.
- jsemrau 2y ago
  Just a thought exercise. If we would have an AI with the intellectual capabilities of a Ph.D holding professor in a hard science. How much would it be worth for you to have access to that AI? 100,000 ? 500,000 ?
  - salawat 2y ago
    0 unless what I'm interested in is that Professor's very narrowly tailored niche. It's called Piled Higher and Deeper for a reason.
- refulgentis 2y ago
  CoT is not RL'ing over reasoning traces, costs have come down 87.5% since that article, and I agree generally that "free" is a bad price point
- BaculumMeumEst 2y ago
  I don't find this very compelling. Hardware is becoming more available and cheaper as production ramps up, and smaller models are constantly seeing dramatic improvements.
- lupire 2y ago
  People hate paying specifically for stuff. If Copilot came for free and Azure cost a tiny bit more, nobody would even blink.
- CamperBob2 2y ago
  There's no accounting for taste, but keep in mind that all of these services are currently losing money, so how much would you actually be willing to pay for the service you're currently getting in order to let it break even? For ChatGPT in its current state, probably $1K/month.
- lhl 2y ago
  When was this profitability report, because the cost per token generation has dropped significantly. When GPT4 was launched last year, the API cost was about $36/M blended tokens, but you can now get GPT4o tokens for about $4.4/M tokens, Gemini 1.5 Pro for $2.2/M or DeepSeek-V2 (as 21B A/236B W model that matches GPT4 on coding) for as low as $0.28/M tokens (over 100X cheaper for the same quality output over the course of about 1.5 years). The just released Qwen2.5-Coder-7B-Instruct (Apache 2.0 licensed) also basically matches/beats GPT4 on coding benchmarks and quantized can not only can run at a decent speed on just about any consumer gaming GPU, but on most new CPUs/NPUs as well. This is about a 250X smaller model than GPT4. There are now a huge array of open weight (and open source) models that are very capable and that can be run locally/on the edge.
ramblerman 2y ago
2 things can be true at the same time. Op is addressing the hype that there is some linear path of improvement here and chatgpt 8.5 will be AGI. To which people always seem to jump in with but it’s useful for me and makes me code faster. Which is fine and valid, just beside the point
lynx23 2y ago
While I get where your fascinaton comes from... > I'm more productive than ever before. You realize that another way to read that sentence is "I am a really bad coder".
- frankc 2y ago
  I think this is what the kids call "copium". To be honest, when people think like this it makes me smile. I'd rather compete against people programming on punchcards.
- JanSt 2y ago
  Maybe I am, but I'm getting pretty rich doing it, so there is that. :)
  - skwee357 2y ago
    The fact that you make money using AI, has nothing to do with its usefulness for society/humanity. There are people who are getting “pretty rich” by trafficking humans, or selling drugs. Would you want to live in a society where such activities are encouraged? In the end, we need to look at technological progress (or any progress for that matter) as where it will bring us to in the future, rather than what it allows you to do now. It also pisses me off that software engineering has such a bad reputation that everyone, from common folks to the CEO of nvidia, is shitting on it. You don’t hear phrases like “AI is going to change medicine/structural engineering”, because you would shit your pants if you had to sit in a dentist chair, while the dentist would ask ChatGPT how to perform a root canal; or if you had to live in a house designed by a structural engineer whose buddy was Claude. And yet, somehow, everyone is ready to throw software engineers under the bus and label them as "useless"/easily replaceable by AI.
    - JanSt 2y ago
      I‘m not making money because of AI, I make a lot of money because I‘m a good programmer. My current income has little to do with ai (99% build before GPT). So relax please.
  - 2y ago
aithrowawaycomm 2y ago
OP could have been more substantive, but there is no contradiction between "current AI tools are sincerely useful" and "overinflated claims about the supposed intelligence of these tools will lead to an AI winter." I am quite confident both are true about LLMs. I use Scheme a lot, but the 1970s MIT AI folks' contention that LISPs encapsulate the core of human symbolic reasoning is clearly ridiculous to 2020s readers: LISP is an excellent tool for symbolic manipulation and it has no intelligence whatsoever even compared to a jellyfish[1], since it cannot learn. GPTs are a bit more complicated: they do learn, and transformer ANNs seem meaningfully more intelligent than jellyfish or C. elegans, which apparently lack "attention mechanisms" and, like word2vec, cannot form bidirectional associations. Yet Claude-3.5 and GPT-4o are still unable to form plans, have no notions of causality, cannot form consistent world models[2] and plainly don't understand what numbers actually mean, despite their (misleading) successes in symbolic mathematics. Mice and pigeons do have these cognitive abilities, and I don't think it's because God seeded their brains with millions of synthetic math problems. It seems to me that transformer ANNs are, at any reasonable energy scale, much dumber than any bird or mammal, and maybe dumber than all vertebrates. There's a huge chunk we are missing. And I believe what fuels AI boom/bust cycles are claims that certain AI is almost as intelligent as a human and we just need a bit more compute and elbow grease to push us over the edge. If AI investors, researchers, and executives had a better grasp of reality - "LISP is as intelligent as a sponge", "GPT is as intelligent as a web-spinning spider, but dumber than a jumping spider" - then there would be no winter, just a realization that spring might take 100 years. Instead we see CS PhDs deluding themselves with Asimov fairy tales. [1] Jellyfish don't have brains but their nerve nets are capable of Pavlovian conditioning - i.e., learning. [2] I know about that Othello study. It is dishonest. Unlike those authors, when I say "world model" I mean "world."
- apsec112 2y ago
  What LLM abilities, if you saw them demonstrated, would cause you to change your mind?
  - aithrowawaycomm 2y ago
    Let's start with a multimodal[1] LLM that doesn't fail vacuously simple out-of-distribution counting problems. I need to be convinced that an LLM is smarter than a honeybee before I am willing to even consider that it might be as smart as a human child. Honeybees are smart enough to understand what numbers are. Transformer LLMs are not. In general GPT and Claude are both dramatically dumber than honeybees when it comes to deep and mysterious cognitive abilities like planning and quantitative reasoning, even if they are better than honeybees at human subject knowledge and symbolic mathematics. It is sensible to evaluate Claude compared to other human knowledge tools, like an encyclopedia or Mathematica, based on the LLM benchmarks or "demonstrated LLM abilities." But those do not measure intelligence. To measure intelligence we need make the LLM as ignorant as possible so it relies on its own wits, like cognitive scientists do with bees and rats. (There is a general sickness in computer science where one poorly-reasoned thought experiment from Alan Turing somehow outweighs decades of real experiments from modern scientists.) [1] People dishonestly claim LLMs fail at counting because of minor tokenization issues, but a) they can count just fine if your prompt tells them how, so tokenization is obviously not a problem b) they are even worse at counting if you ask them to count things in images, so I think tokenization is irrelevant!
dathinab 2y ago
idk. about Claude 3.5 but if you remove implicit subventions from the AI/AGI hype then for many such tools the cost to benefit calculation of creating and operating will become ... questionable furthermore the places where such tools tend to shine the most often places where the IT industry has somewhat failed, like unnecessary verbose and bothersome to use tools, missing tooling and troublesome code reuse (so you write the same code again and again). And this LLM based tools are not fixing the problem they just kinda hiding it. And that has me worried a bit because it makes it much much less likely for the problem to ever be fixed. Like I think there is a serious chance for this tooling causing the industry to be stuck on a quite sub-par plato for many many years. So while they clearly help, especially if you have to reinvent the wheel for a thousands time, it's hard to look at them favorably.
- mewpmewp2 2y ago
  Hopefully it will be able to also reduce boilerplate and do reasonable DRY abstractions if repetition becomes too much. E.g. I feel like it should be possible to first blast out a lot of repetitive code and then for LLM to go over all of it and abstract it reasonably, while tests are still passing.
  - JonChesterfield 2y ago
    Code generator in the editor has been around for ages and serves primarily to maximise boilerplate and minimise DRY. Expecting the opposite from a new code generator will yield disappointment.
    - mewpmewp2 2y ago
      I mean LLM can go through all the files in a source code and find repetitions that can be abstracted. Reorganize files into more appropriate structures etc. It just needs an optimal algorithm to provide optimal context for it.
- Demiurge 2y ago
  > And that has me worried a bit because it makes it much much less likely for the problem to ever be fixed. How will that ever get solved, in this universe? Look at what C++ does to C, what TypeScript does to JavaScript, what every standard does to the one before. It builds on top, without fixing the bottom, paving over the holes. If AI helps generate sane low level code, maybe it will help you make less buffer overflow mistakes. If AI can help test and design your firewall and network rules, maybe it will help you avoid exposing some holes in your CUPS service. Why not, if we're never getting rid of IP printing or C? Seems like part of the technological progress.
PaulHoule 2y ago
Usually I learn my way around the reference docs for most languages I use but CSS has about 50 documents to navigate. I’ve found Copilot does a great job with CSS questions though for Java I really do run into cases where it tells me that Optional doesn’t have a method that I know is there.
inoop 2y ago
"This is actually good for Bitcoin"
renegade-otter 2y ago
I have yet to watch people be THAT more productive using, say, Copilot. Outside of some annoying boilerplate that I did not have to write myself, I don't know what kind of code you are writing that makes it all so much easier. This gets worse if you are using less trendy languages. No offense, but I have only seen people who barely coded before describe being "very productive" with AI. And, sure, if you dabble, these systems will spit out scripts and simpler code for you, making you feel empowered, but they are not anywhere near being helpful with a semi-complex codebase.
- f1shy 2y ago
  I’ve tried enough times to generate code with AI: any attempt to generate non absolutely trivial piece of code that I can do intoxicated and sleep deprived, is just junk. It takes more time and effort to correct the AI output as starting from 0. Let’s see in some years… long winter ahead.
  - deleted 2y ago
    [deleted]
- surgical_fire 2y ago
  I tried many times. Things that AI is good at: - Generate boilerplate - Generate extremely simple code patterns. You need a simple CRUD API? Yeah, it can do it. - Generate solutions for established algorithms. Think of solutions for leetcode exercises. So yeah, if that's your job as a developer, that was a massive productivity boost. Playing with anything beyond that and I got varying degrees of failure. Some of which are productivity killers. The worst is when I am trying to do something in a language/framework I am not familiar with, and AI generates plausibly sounding but horribly wrong bullshit. It sends me in some deadends that take me a while to figure out, and I would have been better just looking it up by myself.
  - skydhash 2y ago
    And the solutions for these already existed: - Generate boilerplate : Snippets, templates, and code generators - Generate extremely simple code patterns : Frameworks - Generate solutions for established algorithms : Libraries.
rafaelmn 2y ago
It's a usefull coding tool - but at the same time it displays a lack of intelligence in the responses provided. Like it will generate code like `x && Array.isarray(x)` because `x && x is something` is a common pattern I guess - but it's completely pointless in this context. It will often do roundabout shit solutions when there's trivial stuff built into the tool/library when you ask it to solve some problem. If you're not a domain expert or search for better solutions to check it you'll often end up with slop. And the "reasoning" feels like the most generic answers while staying on topic, like "review this code" will focus on bullshit rather than prioritizing the logic errors or clearing up underlying assumptions, etc. That said it's pretty good at bulk editing - like when I need to refactor crufty test cases it saves a bunch of typing.
__alexs 2y ago
It's not snark, it's calling out a fundamental error of extrapolating a short term change in progress to infinity. It's like looking at the first version of an IDE that got intellisense/autocomplete and deciding that we'll be able to write entire programs by just pressing tab and enter 10,000 times.
aznumeric 2y ago
If you like Cursor, you should definitely check out ClaudeDev (https://github.com/saoudrizwan/claude-dev https://github.com/saoudrizwan/claude-dev) It's been a hit in the Ai dev community and I've noticed many folks prefer it over Cursor. It's free and open-source. You use your API credits instead of subscription and it supports other LLMs like DeepSeek too.
jetsetk 2y ago
how come MS Teams is still trash when everyone is being so much more productive? Shouldn't MS - sitting at the source - be able to create software wonders like all the weekend warriors using AI?
layer8 2y ago
I’ll be waiting for these developer benefits to translate into tangible end user benefits in software.
dangitman 2y ago
[dead]
gorjusborg 2y ago
Do you think AI companies will be able to afford running massive compute farms solely so coders can get suggestions? I do not claim to know what the future holds, but I do feel the clock is ticking on the AI hype. OpenAI blew people's minds with GPTs, and people extrapolated that mind-blowing experience into a future with omniscient AI agents, but those are nowhere to be seen. If investors have AGI in mind, and it doesn't happen soon enough, I can see another winter. Remember, the other AI winters were due to a disconnect between expectations and reality of the current tech. They also started with unbelievable optimism that ended when it became clear the expectations were not reality. The tech wasn't bad back then either, it just wasn't The General Solution people were hoping for.
gm3dmo 2y ago
I feel like these new tools have helped me get simple programming tasks done really quickly over the last 18 months. They seem like a faster, better and more accurate replacement for googling and Stackoverflow. They seem very good at writing SQL for example. All the commas are in the right place and exactly the right amount of brackets square curly and round. But when they get it wrong, it really shows up the lack of intelligence. I hope the froth and bubble in the marketing of these tools matures into something with a little less hyperbole because they really are great just not intelligent.
TheCondor 2y ago
It’s not snark, our industry is run on fear. If there is the tiniest flicker of potential, we will spend piles of money out of fear of being left behind. As you age, it becomes harder to deny.. 10 years ago, I was starting to believe that my kids would never learn to drive or possibly buy a car, here we are ten years later and not that much has changed, I know you can take a robotaxi in some cities but nearly all interstate trucking has someone driving. Coding AI assistants have done some impressive things, I’ve been amazed at how they sniffed out some repetitive tasks I was hacking on and I just tab completed pages of code that was pretty much correct. There is use. I pay for the feature. I don’t know if it’s worth 35% of the world’s energy consumption and all new fabrication resources over the next handful of years being dedicated to ‘ai chips.’ We arent looking for a better 2.0, we are expecting an exponentially better “2.0” and those are very rare.
- dmix 2y ago
  That doesn't mean this is a bad investment for VCs. GPT is being directly integrated in iOS and is a top app on both markets. We've also barely scrapped the surface for potential niche applications that go beyond just a generalist chatbox interface. API use will likely continue to explode as the mountain of startups building off it come online. Voice stuff will probably kill off Alexa/Google Home. I don't think the bulk of this VC money is predicated on AGI being around the corner. But the general trend hopping nature of big VC money is real. Still, VCs manage to continue to make a profit despite this, otherwise the industry would have died off or shrunk the 10 other years HN critiqued this behaviour, so on the whole they must be doing something right.
  - tdeck 2y ago
    VCs mostly make money by selling a narrative about investing in the next big thing and then collecting management fees, not by beating the market. If the public sours on AI we need a new hype to replace it and keep tech money flowing at the same rate. A lot of funding seems to follow fads and be disproportionate to value generated (I remember when there were a bazillion people building social networks because that was hot).
  - 2y ago
sufjkw 2y ago
> I'm more productive than ever before. Who are you and what are you being so productive in? These code assistants are wholly unable to help with the day to day work I do. Sometimes I use them to remind me what flags to use with a tarball[0], so replaced SO, but anything of consequence or creativity and they flounder. What are you getting out of this excess productivity? A pay raise? More time with your loved ones? [0] https://xkcd.com/1168/ https://xkcd.com/1168/ (addressing the tool tip, but hilariously, in regards the comics content that would be a circumstance where I would absolutely avoid trusting one of these ‘assistants’)
ashkankiani 2y ago
LLMs make mediocre engineers into slightly less mediocre engineers, and non-engineers into below mediocre engineers. They do nothing above the median. I've tried dozens of times to use them productively. Outside of very very short isolated template creation for some kind of basic script or poorly translating code from one language to another, they have wasted more time for me than they saved. The area they seem to help people, including me, the most in is giving me code for something I don't have any familiarity with that seems plausible. If it's an area I've never worked in before, it could maybe be useful. Hence why the less breadth of knowledge in programming you have, the more useful it is. The problem is that you don't understand the code it produces so you have to entirely be reliant on it, and that doesn't work long term. LLMs are not and will not be ready to replace programmers within the next few years, I guarantee it. I would bet $10k on it.
freejazz 2y ago
Nothing snarky about pointing out AGI is nowhere near
beefnugs 2y ago
The economics dont make sense at all: Either you pay more and more to keep your job as it gets better, or the company pays any amount for it so they can replace you over and over as a barely useful cog. The current state of it being cheap only exist as it is in beta and they need more info from you, the expert, until it no longer needs you

by JanSt 2y ago

jsheard 2y ago

There's no accounting for taste, but keep in mind that all of these services are currently losing money, so how much would you actually be willing to pay for the service you're currently getting in order to let it break even? There was a report that Microsoft is losing $20 for every $10 spent on Copilot subscriptions, with heavy users costing them as much as $80 per month. Assuming you're one of those heavy users, would you pay >$80 a month for it? Then there's chain-of-thought being positioned as the next big step forwards, which works by throwing more inferencing at the problem, so that cost can't be amortized over time like training can...

HPsquared 2y ago
It's called investment. You need to spend money to make money. Their costs will certainly come down.
JanSt 2y ago
1) The costs will go down over time, much of the cost is the margin of NVIDIA and training new models 2) Absolutely. Thats like one hour of an engineer salary for a whole month.
- sofixa 2y ago
  > The costs will go down over time, much of the cost is the margin of NVIDIA and training new models Isn't each new model bigger and heavier and thus requries more compute to train?
  - JanSt 2y ago
    Yes, but 1) you only need to train the model once and the inference is way cheaper. Train one great model (i.e. Claude 3.5) and you can get much more than $80/month worth out of it. 2) the hardware is getting much better and prices will fall drastically once there is a bit of a saturation of the market or another company starts putting out hardware that can compete with NVIDIA
    - sofixa 2y ago
      > Train one great model (i.e. Claude 3.5) and you can get much more than $80/month worth out of it Until the competition outcompetes you with their new model and you have to train a new superior one, because you have no moat. Which happens what, around every month or two? > the hardware is getting much better and prices will fall drastically once there is a bit of a saturation of the market or another company starts putting out hardware that can compete with NVIDIA Where is the hardware that can compete with NVIDIA going to come from? And if they don't have competition, which they don't, why would they bring down prices?
      - JanSt 2y ago
        The point is not that every lab will be profitable. There only needs to be one model in the end to increase our productivity massively, which is the point I'm making. Huge margins lead to a lot of competition trying to catch up, which is what makes market economies so successful.
      - ben_w 2y ago
        > Until the competition outcompetes you with their new model and you have to train a new superior one, because you have no moat. Which happens what, around every month or two? Eventually one of you runs out of money, but your customers keep getting better models until then; and if the loser in this race releases the weights on a suitable gratis license then your businesses can both lose. But that still leaves your customers with access to a model that's much cheaper to run than it was to create.
      - Workaccount2 2y ago
brookst 2y ago
Is there any reason to believe costs won’t come down with scale and hardware iteration, just like they did for everything else? Short term pricing inefficiency is not relevant to long term impact.
- HarHarVeryFunny 2y ago
  Of course, but every token generated by a 100B model is going to take minimally 100B FLOPS, and if this is being used as an IDE typing assistant then there is going to be a lot of tokens being generated. If there is a common shift to using additional runtime compute to improve quality of output, such as OpenAI's GPT-o1, then FLOPs required goes up massively (OpenAI has said it takes exponential increase in FLOPS/cost to generate linear gains in quality). So, while costs will of course decrease, those $20-30K NVIDEA chips are going to be kept burring, and are not going to pay for themselves ... This may end up like the shift to cloud computing that sounds good in theory (save the cost of running your own data center), but where corporate America balks when the bill comes in. It may well be that the endgame for corporate AI is to run free tools from the likes of Meta (or open source) in their own datacenter, or maybe even locally on "AI PCs".
  - wongarsu 2y ago
    Which is why the work to improve the results of small models is so important. Running a 3B or even 1B model as typing assistant and reserving the 100B model for refactoring is a lot more viable.
  - brookst 2y ago
    > but every token generated by a 100B model is going to take minimally 100B FLOPS Drop the S, I think. There’s no time dimension. And FLOP is a generalized capability meaning you can do any operation. Hardware optimizations for ML can deliver the same 100B computations faster and cheaper by not being completely generalized. Same way ray tracing acceleration works: it does not use the same amount of compute as ray tracing in general CPU’s.
    - 2y ago
archerx 2y ago
I can already pay $0 a month and use uncensored local models for both text and images. Llama, Mixtral, Stable diffusion and Flux are a lot of fun and free to run locally, you should try them out.
- jsheard 2y ago
  You can pay $0 for those models because a company paid $lots to train them and then released them for free. Those models aren't going away now of course, but lets not pretend that being able to download the product of millions of dollars worth of training completely free of charge is sustainable for future developments. Especially when most of the companies releasing these open models are wildly unprofitable and will inevitably bankrupt themselves when investments dry up unless they change their trajectory.
  - likium 2y ago
    Much could be said about open source libraries that companies release for free to use (kubernetes, react, firecracker, etc). It might be strategically make sense for them so in the meantime we’ll just reap the benefits.
    - skydhash 2y ago
      All of these require maintenance, and mostly it's been a treadmill just applying updates to React codebases. Complex tools are brittle and often only makes sense at the original source.
  - archerx 2y ago
    You’re acting as if computing power isn’t going to get better. With time training the models will get faster. Let me use CG rendering as an example. Back in the day only the big companies could afford to do photoreal 3D rendering because only they had access to the compute and even then it would take days to render a frame. Eventually people could do these renders at home with consumer hardware but it still took forever to render. Now we can render photoreal with path tracing at near realtime speeds. If you could go back twenty years and show CG artists the Unreal Engine 5 and show them it’s all realtime they would lose their minds. I see the same for A.I., now it’s only the big companies that can do it, then we will be able to do it at home but it will be slow and finally we will be able to train it at home for quick and cheap.
binocarlos 2y ago
I would pay hundreds of dollars per month for the combination of cursor and claude - I could not get my head around it when my beginner lever colleague said "I just coded this whole thing using cursor". It was an entire web app, with search filters, tree based drag and drop GUIs, the backend api server, database migrations, auth and everything else. Not once did he need to ask me a question. When I asked him "how long did this take" and expected him to say "a few weeks" (it would have taken me - a far more experienced engineer - 2 months minimum). His answer was "a few days". What I'm not saying is "AGI is close" but I've seen tangible evidence (only in the last 2 months), that my 20 year software engineering career is about to change and massively for the upside. Everyone is going to be so much more productive using these tools is how I see this.
- threeseed 2y ago
  > 20 year software engineering career is about to change I have also been developing for 20+ years. And have heard the exact same thing about IDEs, Search Engines, Stack Overflow, Github etc. But in my experience at least how fast I code has never been the limiting factor in my project's success. So LLMs are nice and all but isn't going to change the industry all that much.
  - pluc 2y ago
    There will be a whole industry of people who fix what AI has created. I don't know if it will be faster to build the wrong thing and pay to have it fixed or to build the right thing from the get go, but after having seen some shit, like you, I have a little idea.
    - dumbfounder 2y ago
      Correction: a whole industry of AI that will fix what AI has created.
      - vocram 2y ago
        Will AI also be on call when things break in production?
gotaran 2y ago
$80 a month is a no brainer given the productivity multiplier.
ema 2y ago
If it makes software developers 10% more productive there sure would be many companies who'd pay $80 a month per seat.
- apwell23 2y ago
  It actually makes them less productive and creates havoc in codebases with hidden bugs and verbose code that ppl are copy pasting.
- renegade-otter 2y ago
  It's like saying "AI is going to replace book writers because they are so much more productive now". All you will get is more mediocre content that someone will have to fix later - the same with code. 10% more productive. What does that mean? If you mean lines of code, then it's an incredibly poor metric. They write more code, faster. Then what? What are the long-term consequences? Is it ultimately a wash, or even a detriment? https://stackoverflow.blog/2024/03/22/is-ai-making-your-code-worse/ https://stackoverflow.blog/2024/03/22/is-ai-making-your-code...
  - ben_w 2y ago
    LLMs set a new minimum level; because of this they can fill in the gaps in a skillet — if I really suck at writing unit tests, they can bring me up from "none" to "it's a start". Likewise all the other specialities within software. Personally I am having a lot of fun, as an iOS developer, creating web games. No market in that, not really, but it's fun and I wouldn't have time to update my CSS and JS knowledge that was last up-to-date in 1998.
- HarHarVeryFunny 2y ago
  Maybe there are people out there working in coding sweatshops churning out boilerplate code 8 hours a day, 50 weeks a year - people who's job is 100% coding (not what I would call software engineers or developers - just coders). It's easy to imagine that for such people (but do they even exist?!) there could be large productivity gains. However, for a more typical software engineer, where every project is different, you have full lifecycle responsibility from design through coding, occasional production support, future enhancements, refactorings, updates for 3rd party library/OD updates, etc/etc, then how much of your time is actually spent pure coding (non-stop typing) ?! Probably closer to 10-25%, and certainly no-where near 100%. The potential overall time saving from a tool that saves, let's say, 10-25% of your code typing is going to be 1-5%, which is probably far less than gets wasted in meetings, chatting with your work buddies, or watching bullshit corporate training videos. IOW the savings is really just inconsequential noise. In many companies the work load is cyclic from one major project to the next, with intense periods of development interspersed with quieter periods in-between. Your productivity here certainly isn't limited by how fast you can type.
mklepaczewski 2y ago
I would, and I don't use chatgpt as much as other people. I would pay for it for each of my employees.
christkv 2y ago
Also at some point you can run the equivalent model locally. There is no long term moat here i think and facebook seems hellbent of ensuring there will be no new google from llms
- KoolKat23 2y ago
  I think physics at some point will get in the way, well at least for a while. An H100 costs like $20k-$30k and there's only so much compression/efficiency they can gain without beginning to lose intelligence, purely because you can't compute out of thin air.
ben_w 2y ago
> There's no accounting for taste, but keep in mind that all of these services are currently losing money, so how much would you actually be willing to pay for the service you're currently getting in order to let it break even Ok models already run locally; that aside, as the hosted ones are kinda similar quality to interns (though varying by field), the answer is "what you'd pay an intern". Could easily be £1500/month, depending on domain.
infecto 2y ago
Definitely. My time is valuable and I would spend multiples more on the current subscription costs.
fassssst 2y ago
Why do you assume they’re losing money on inference?
jejeyyy77 2y ago
- it won't work. - ok it works, but it won't be useful. - ok it's useful, but it won't scale. - ok it scales, but it won't make any money. - ok it makes money, but it's not going to last. etc etc
- pdinny 2y ago
  Retrospectively framing technologies that succeeded despite doubts at the time discounts those that failed. After all, you could have used the exact same response in defense of web3 tech. That doesn't mean LLMs are fated to be like web3, but similarly the outcome that the current expenditure can be recouped is far from a certainty just because there are doubters.
- farts_mckensy 2y ago
  There certainly has been some goal post moving over the past few months. A lot of the people in here have some kind of psychological block when it comes to technology that may potentially replace them one day.
  - KoolKat23 2y ago
    Yeah currently the sentiment seems to be "okay fine it works for simple stuff but won't deal with my complex query so it can be dismissed outright." Save yourselves some time and use it for that simple stuff folks.
chmod775 2y ago
> There was a report that Microsoft is losing $20 for every $10 spent on Copilot subscriptions, with heavy users costing them as much as $80 per month. Assuming you're one of those heavy users, would you pay >$80 a month for it? I'm probably one of those "heavy users", though I've only been using it for a month to see how well it does. Here's my review: Large completions (10-15 lines): It will generally spit out near-working code for any codemonkey-level framework-user frontend code, but for anything more it'll be at best amusing and a waste of time. Small completions (complete current line): Usually nails it and saves me a few keystrokes. The downside is that it competes for my attention/screen space against good old auto-completion, which costs me productivity every time it fucks up. Having to go back and fix identifiers in which it messed up the capitalization/had typos, where basic auto-complete wouldn't have failed is also annoying. I'd pay about about $40 right now because at least it has some entertainment value, being technologically interesting.
- jeremy151 2y ago
  I find tools where I am manually shepherding the context into an LLM to work much better than Copilot at current. If I think thru the problem enough to articulate it and give the model a clear explanation, and choose the surrounding pieces of context (the same stuff I would open up and look at as a dev) I can be pretty sure the code generated (even larger outputs) will work and do what I wanted, and be stylistically good. I am still adding a lot in this scenario, but it's heavier on the analysis and requirements side, and less on the code creation side. If what I give it is too open ended, doesn't have enough info, etc, I'll still get a low quality output. Though I find I can steer it by asking it to ask clarifying questions. Asking it to build unit tests can help a lot too in bolstering, a few iterations getting the unit tests created and passing can really push the quality up.
jsemrau 2y ago
Just a thought exercise. If we would have an AI with the intellectual capabilities of a Ph.D holding professor in a hard science. How much would it be worth for you to have access to that AI? 100,000 ? 500,000 ?
- salawat 2y ago
  0 unless what I'm interested in is that Professor's very narrowly tailored niche. It's called Piled Higher and Deeper for a reason.
refulgentis 2y ago
CoT is not RL'ing over reasoning traces, costs have come down 87.5% since that article, and I agree generally that "free" is a bad price point
BaculumMeumEst 2y ago
I don't find this very compelling. Hardware is becoming more available and cheaper as production ramps up, and smaller models are constantly seeing dramatic improvements.
lupire 2y ago
People hate paying specifically for stuff. If Copilot came for free and Azure cost a tiny bit more, nobody would even blink.
CamperBob2 2y ago
There's no accounting for taste, but keep in mind that all of these services are currently losing money, so how much would you actually be willing to pay for the service you're currently getting in order to let it break even? For ChatGPT in its current state, probably $1K/month.
lhl 2y ago
When was this profitability report, because the cost per token generation has dropped significantly. When GPT4 was launched last year, the API cost was about $36/M blended tokens, but you can now get GPT4o tokens for about $4.4/M tokens, Gemini 1.5 Pro for $2.2/M or DeepSeek-V2 (as 21B A/236B W model that matches GPT4 on coding) for as low as $0.28/M tokens (over 100X cheaper for the same quality output over the course of about 1.5 years). The just released Qwen2.5-Coder-7B-Instruct (Apache 2.0 licensed) also basically matches/beats GPT4 on coding benchmarks and quantized can not only can run at a decent speed on just about any consumer gaming GPU, but on most new CPUs/NPUs as well. This is about a 250X smaller model than GPT4. There are now a huge array of open weight (and open source) models that are very capable and that can be run locally/on the edge.

ramblerman 2y ago

2 things can be true at the same time. Op is addressing the hype that there is some linear path of improvement here and chatgpt 8.5 will be AGI. To which people always seem to jump in with but it’s useful for me and makes me code faster. Which is fine and valid, just beside the point

lynx23 2y ago

While I get where your fascinaton comes from... > I'm more productive than ever before. You realize that another way to read that sentence is "I am a really bad coder".

frankc 2y ago
I think this is what the kids call "copium". To be honest, when people think like this it makes me smile. I'd rather compete against people programming on punchcards.
JanSt 2y ago
Maybe I am, but I'm getting pretty rich doing it, so there is that. :)
- skwee357 2y ago
  The fact that you make money using AI, has nothing to do with its usefulness for society/humanity. There are people who are getting “pretty rich” by trafficking humans, or selling drugs. Would you want to live in a society where such activities are encouraged? In the end, we need to look at technological progress (or any progress for that matter) as where it will bring us to in the future, rather than what it allows you to do now. It also pisses me off that software engineering has such a bad reputation that everyone, from common folks to the CEO of nvidia, is shitting on it. You don’t hear phrases like “AI is going to change medicine/structural engineering”, because you would shit your pants if you had to sit in a dentist chair, while the dentist would ask ChatGPT how to perform a root canal; or if you had to live in a house designed by a structural engineer whose buddy was Claude. And yet, somehow, everyone is ready to throw software engineers under the bus and label them as "useless"/easily replaceable by AI.
  - JanSt 2y ago
    I‘m not making money because of AI, I make a lot of money because I‘m a good programmer. My current income has little to do with ai (99% build before GPT). So relax please.
- 2y ago

aithrowawaycomm 2y ago

OP could have been more substantive, but there is no contradiction between "current AI tools are sincerely useful" and "overinflated claims about the supposed intelligence of these tools will lead to an AI winter." I am quite confident both are true about LLMs. I use Scheme a lot, but the 1970s MIT AI folks' contention that LISPs encapsulate the core of human symbolic reasoning is clearly ridiculous to 2020s readers: LISP is an excellent tool for symbolic manipulation and it has no intelligence whatsoever even compared to a jellyfish[1], since it cannot learn. GPTs are a bit more complicated: they do learn, and transformer ANNs seem meaningfully more intelligent than jellyfish or C. elegans, which apparently lack "attention mechanisms" and, like word2vec, cannot form bidirectional associations. Yet Claude-3.5 and GPT-4o are still unable to form plans, have no notions of causality, cannot form consistent world models[2] and plainly don't understand what numbers actually mean, despite their (misleading) successes in symbolic mathematics. Mice and pigeons do have these cognitive abilities, and I don't think it's because God seeded their brains with millions of synthetic math problems. It seems to me that transformer ANNs are, at any reasonable energy scale, much dumber than any bird or mammal, and maybe dumber than all vertebrates. There's a huge chunk we are missing. And I believe what fuels AI boom/bust cycles are claims that certain AI is almost as intelligent as a human and we just need a bit more compute and elbow grease to push us over the edge. If AI investors, researchers, and executives had a better grasp of reality - "LISP is as intelligent as a sponge", "GPT is as intelligent as a web-spinning spider, but dumber than a jumping spider" - then there would be no winter, just a realization that spring might take 100 years. Instead we see CS PhDs deluding themselves with Asimov fairy tales. [1] Jellyfish don't have brains but their nerve nets are capable of Pavlovian conditioning - i.e., learning. [2] I know about that Othello study. It is dishonest. Unlike those authors, when I say "world model" I mean "world."

apsec112 2y ago
What LLM abilities, if you saw them demonstrated, would cause you to change your mind?
- aithrowawaycomm 2y ago
  Let's start with a multimodal[1] LLM that doesn't fail vacuously simple out-of-distribution counting problems. I need to be convinced that an LLM is smarter than a honeybee before I am willing to even consider that it might be as smart as a human child. Honeybees are smart enough to understand what numbers are. Transformer LLMs are not. In general GPT and Claude are both dramatically dumber than honeybees when it comes to deep and mysterious cognitive abilities like planning and quantitative reasoning, even if they are better than honeybees at human subject knowledge and symbolic mathematics. It is sensible to evaluate Claude compared to other human knowledge tools, like an encyclopedia or Mathematica, based on the LLM benchmarks or "demonstrated LLM abilities." But those do not measure intelligence. To measure intelligence we need make the LLM as ignorant as possible so it relies on its own wits, like cognitive scientists do with bees and rats. (There is a general sickness in computer science where one poorly-reasoned thought experiment from Alan Turing somehow outweighs decades of real experiments from modern scientists.) [1] People dishonestly claim LLMs fail at counting because of minor tokenization issues, but a) they can count just fine if your prompt tells them how, so tokenization is obviously not a problem b) they are even worse at counting if you ask them to count things in images, so I think tokenization is irrelevant!

dathinab 2y ago

idk. about Claude 3.5 but if you remove implicit subventions from the AI/AGI hype then for many such tools the cost to benefit calculation of creating and operating will become ... questionable furthermore the places where such tools tend to shine the most often places where the IT industry has somewhat failed, like unnecessary verbose and bothersome to use tools, missing tooling and troublesome code reuse (so you write the same code again and again). And this LLM based tools are not fixing the problem they just kinda hiding it. And that has me worried a bit because it makes it much much less likely for the problem to ever be fixed. Like I think there is a serious chance for this tooling causing the industry to be stuck on a quite sub-par plato for many many years. So while they clearly help, especially if you have to reinvent the wheel for a thousands time, it's hard to look at them favorably.

mewpmewp2 2y ago
Hopefully it will be able to also reduce boilerplate and do reasonable DRY abstractions if repetition becomes too much. E.g. I feel like it should be possible to first blast out a lot of repetitive code and then for LLM to go over all of it and abstract it reasonably, while tests are still passing.
- JonChesterfield 2y ago
  Code generator in the editor has been around for ages and serves primarily to maximise boilerplate and minimise DRY. Expecting the opposite from a new code generator will yield disappointment.
  - mewpmewp2 2y ago
    I mean LLM can go through all the files in a source code and find repetitions that can be abstracted. Reorganize files into more appropriate structures etc. It just needs an optimal algorithm to provide optimal context for it.
Demiurge 2y ago
> And that has me worried a bit because it makes it much much less likely for the problem to ever be fixed. How will that ever get solved, in this universe? Look at what C++ does to C, what TypeScript does to JavaScript, what every standard does to the one before. It builds on top, without fixing the bottom, paving over the holes. If AI helps generate sane low level code, maybe it will help you make less buffer overflow mistakes. If AI can help test and design your firewall and network rules, maybe it will help you avoid exposing some holes in your CUPS service. Why not, if we're never getting rid of IP printing or C? Seems like part of the technological progress.

PaulHoule 2y ago

Usually I learn my way around the reference docs for most languages I use but CSS has about 50 documents to navigate. I’ve found Copilot does a great job with CSS questions though for Java I really do run into cases where it tells me that Optional doesn’t have a method that I know is there.

inoop 2y ago

"This is actually good for Bitcoin"

renegade-otter 2y ago

I have yet to watch people be THAT more productive using, say, Copilot. Outside of some annoying boilerplate that I did not have to write myself, I don't know what kind of code you are writing that makes it all so much easier. This gets worse if you are using less trendy languages. No offense, but I have only seen people who barely coded before describe being "very productive" with AI. And, sure, if you dabble, these systems will spit out scripts and simpler code for you, making you feel empowered, but they are not anywhere near being helpful with a semi-complex codebase.

f1shy 2y ago
I’ve tried enough times to generate code with AI: any attempt to generate non absolutely trivial piece of code that I can do intoxicated and sleep deprived, is just junk. It takes more time and effort to correct the AI output as starting from 0. Let’s see in some years… long winter ahead.
- deleted 2y ago
  [deleted]
surgical_fire 2y ago
I tried many times. Things that AI is good at: - Generate boilerplate - Generate extremely simple code patterns. You need a simple CRUD API? Yeah, it can do it. - Generate solutions for established algorithms. Think of solutions for leetcode exercises. So yeah, if that's your job as a developer, that was a massive productivity boost. Playing with anything beyond that and I got varying degrees of failure. Some of which are productivity killers. The worst is when I am trying to do something in a language/framework I am not familiar with, and AI generates plausibly sounding but horribly wrong bullshit. It sends me in some deadends that take me a while to figure out, and I would have been better just looking it up by myself.
- skydhash 2y ago
  And the solutions for these already existed: - Generate boilerplate : Snippets, templates, and code generators - Generate extremely simple code patterns : Frameworks - Generate solutions for established algorithms : Libraries.

rafaelmn 2y ago

It's a usefull coding tool - but at the same time it displays a lack of intelligence in the responses provided. Like it will generate code like `x && Array.isarray(x)` because `x && x is something` is a common pattern I guess - but it's completely pointless in this context. It will often do roundabout shit solutions when there's trivial stuff built into the tool/library when you ask it to solve some problem. If you're not a domain expert or search for better solutions to check it you'll often end up with slop. And the "reasoning" feels like the most generic answers while staying on topic, like "review this code" will focus on bullshit rather than prioritizing the logic errors or clearing up underlying assumptions, etc. That said it's pretty good at bulk editing - like when I need to refactor crufty test cases it saves a bunch of typing.

__alexs 2y ago

It's not snark, it's calling out a fundamental error of extrapolating a short term change in progress to infinity. It's like looking at the first version of an IDE that got intellisense/autocomplete and deciding that we'll be able to write entire programs by just pressing tab and enter 10,000 times.

aznumeric 2y ago

If you like Cursor, you should definitely check out ClaudeDev (https://github.com/saoudrizwan/claude-dev https://github.com/saoudrizwan/claude-dev) It's been a hit in the Ai dev community and I've noticed many folks prefer it over Cursor. It's free and open-source. You use your API credits instead of subscription and it supports other LLMs like DeepSeek too.

jetsetk 2y ago

how come MS Teams is still trash when everyone is being so much more productive? Shouldn't MS - sitting at the source - be able to create software wonders like all the weekend warriors using AI?

layer8 2y ago

I’ll be waiting for these developer benefits to translate into tangible end user benefits in software.

dangitman 2y ago

[dead]

gorjusborg 2y ago

Do you think AI companies will be able to afford running massive compute farms solely so coders can get suggestions? I do not claim to know what the future holds, but I do feel the clock is ticking on the AI hype. OpenAI blew people's minds with GPTs, and people extrapolated that mind-blowing experience into a future with omniscient AI agents, but those are nowhere to be seen. If investors have AGI in mind, and it doesn't happen soon enough, I can see another winter. Remember, the other AI winters were due to a disconnect between expectations and reality of the current tech. They also started with unbelievable optimism that ended when it became clear the expectations were not reality. The tech wasn't bad back then either, it just wasn't The General Solution people were hoping for.

gm3dmo 2y ago

I feel like these new tools have helped me get simple programming tasks done really quickly over the last 18 months. They seem like a faster, better and more accurate replacement for googling and Stackoverflow. They seem very good at writing SQL for example. All the commas are in the right place and exactly the right amount of brackets square curly and round. But when they get it wrong, it really shows up the lack of intelligence. I hope the froth and bubble in the marketing of these tools matures into something with a little less hyperbole because they really are great just not intelligent.

TheCondor 2y ago

It’s not snark, our industry is run on fear. If there is the tiniest flicker of potential, we will spend piles of money out of fear of being left behind. As you age, it becomes harder to deny.. 10 years ago, I was starting to believe that my kids would never learn to drive or possibly buy a car, here we are ten years later and not that much has changed, I know you can take a robotaxi in some cities but nearly all interstate trucking has someone driving. Coding AI assistants have done some impressive things, I’ve been amazed at how they sniffed out some repetitive tasks I was hacking on and I just tab completed pages of code that was pretty much correct. There is use. I pay for the feature. I don’t know if it’s worth 35% of the world’s energy consumption and all new fabrication resources over the next handful of years being dedicated to ‘ai chips.’ We arent looking for a better 2.0, we are expecting an exponentially better “2.0” and those are very rare.

dmix 2y ago
That doesn't mean this is a bad investment for VCs. GPT is being directly integrated in iOS and is a top app on both markets. We've also barely scrapped the surface for potential niche applications that go beyond just a generalist chatbox interface. API use will likely continue to explode as the mountain of startups building off it come online. Voice stuff will probably kill off Alexa/Google Home. I don't think the bulk of this VC money is predicated on AGI being around the corner. But the general trend hopping nature of big VC money is real. Still, VCs manage to continue to make a profit despite this, otherwise the industry would have died off or shrunk the 10 other years HN critiqued this behaviour, so on the whole they must be doing something right.
- tdeck 2y ago
  VCs mostly make money by selling a narrative about investing in the next big thing and then collecting management fees, not by beating the market. If the public sours on AI we need a new hype to replace it and keep tech money flowing at the same rate. A lot of funding seems to follow fads and be disproportionate to value generated (I remember when there were a bazillion people building social networks because that was hot).
- 2y ago

sufjkw 2y ago

> I'm more productive than ever before. Who are you and what are you being so productive in? These code assistants are wholly unable to help with the day to day work I do. Sometimes I use them to remind me what flags to use with a tarball[0], so replaced SO, but anything of consequence or creativity and they flounder. What are you getting out of this excess productivity? A pay raise? More time with your loved ones? [0] https://xkcd.com/1168/ https://xkcd.com/1168/ (addressing the tool tip, but hilariously, in regards the comics content that would be a circumstance where I would absolutely avoid trusting one of these ‘assistants’)

ashkankiani 2y ago

LLMs make mediocre engineers into slightly less mediocre engineers, and non-engineers into below mediocre engineers. They do nothing above the median. I've tried dozens of times to use them productively. Outside of very very short isolated template creation for some kind of basic script or poorly translating code from one language to another, they have wasted more time for me than they saved. The area they seem to help people, including me, the most in is giving me code for something I don't have any familiarity with that seems plausible. If it's an area I've never worked in before, it could maybe be useful. Hence why the less breadth of knowledge in programming you have, the more useful it is. The problem is that you don't understand the code it produces so you have to entirely be reliant on it, and that doesn't work long term. LLMs are not and will not be ready to replace programmers within the next few years, I guarantee it. I would bet $10k on it.

freejazz 2y ago

Nothing snarky about pointing out AGI is nowhere near

beefnugs 2y ago

The economics dont make sense at all: Either you pay more and more to keep your job as it gets better, or the company pays any amount for it so they can replace you over and over as a barely useful cog. The current state of it being cheap only exist as it is in beta and they need more info from you, the expert, until it no longer needs you