12 ms·

Programmer salaries in the age of LLMs

150 points by Michelangelo11 4y ago

steve76 4y ago
[dead]
Swizec 4y ago
> What happened? Well, if you’re an elite lawyer in the modern age, you probably command a large team of lower-tier lawyers, cheap paralegals, and so on, who are very good at using the Internet to look up case law and draft your opinions and so on. This already happened to programmers. We call these elites some version of tech lead or staff engineer. Maybe even principal, but that feels different still.
sublinear 4y ago
Except you can't do a web search on proprietary code within your business, and hosting AI on-premises to model your codebase would likely be a huge waste of money and slower than just telling a real life person what you want. There has been zero-progress towards AI comprehension, so you can only get out of it what you are already familiar with. The human must always do the mental heavy lifting to tease out a good result, or indeed even know if it's a good result and not just a hallucinated mess.
angarg12 4y ago
But programmer salaries already are bimodal, or even trimodal [1]. [1] https://blog.pragmaticengineer.com/software-engineering-salaries-in-the-netherlands-and-europe/ https://blog.pragmaticengineer.com/software-engineering-sala...
- fdsfawfds 4y ago
  [dead]
- raincom 4y ago
  Indeed, programmer salaries even in the Bay Area are bimodal.
  - reducesuffering 4y ago
    Bay Area, no: https://www.levels.fyi/t/software-engineer/locations/san-francisco-bay-area https://www.levels.fyi/t/software-engineer/locations/san-fra...
    - csomar 4y ago
      That looks like bi-modal to me.
      - reducesuffering 4y ago
        Don't be fooled by the $23k range brackets next to a single bracket lumping in everything $527k+. That bracket is just an aggregation of the same chart trajectory, not another mode.
    - mangosteenjuice 4y ago
      I can't believe this needs to be said, but the salary data that you're linking to excludes the thousands of Bay Area employers who aren't part of levels.fyi's dataset. Most of the employers in their dataset compete with each other, and pay in the second/higher bimodal curve.
      - fexecve 4y ago
        This is correct. This also excludes contractors, which there are tons of in the Bay Area, and they make a lot less than a regular employee, even at places like Google.
- sanderjd 4y ago
  Right! This was my first thought. I think the author missed that software is already quite like law in this way.
osigurdson 4y ago
What change occurred between 1991 and 2000 to explain the emergence of the bimodal distribution? Presumably lawyers have had paralegals for centuries.
- cldellow 4y ago
  The author of this post claims that the Internet happened, enabling easier, cheaper access to law databases, which increased the value that "top tier" lawyers could extract from their paralegals. > What happened? Well, if you’re an elite lawyer in the modern age, you probably command a large team of lower-tier lawyers, cheap paralegals, and so on, who are very good at using the Internet to look up case law and draft your opinions and so on. In contrast, if you’re an elite lawyer in the year 1990, it’s much more valuable for you specifically to have an encyclopedic knowledge of all the relevant case law, and it’s much harder for you to delegate modular pieces of work off to less competent people.
devmunchies 4y ago
I think it will be more like airline pilots with the addition of autopilot. Maybe you could write whole services, and networks of services with AI, but you need engineers to be the guiding hand. Engineers will always be better at using AI to build systems than non-developers. Also, it’s worth pointing out that AI is using existing data. If a new iOS API comes out, the AI will need code bases with it already implemented to be able to train with it. So in my mind, it will hurt the junior developers doing grunt work more than general devs. I think no code/low code products are more of a risk to hurting devs than AI.
- Nathanba 4y ago
  depending on how advanced this AI will be, it could learn from a few samples and the API docs, possibly itself designed to be optimally consumed by AI. If I knew that AI would consume my custom programming language docs then I would make sure that each API is used once in some kind of sample code so it can give answers to everything. Plus, that's a sensible thing to do anyway.
  - reducesuffering 4y ago
    Even better (or worse), tell your AI/LLM to output sample uses of the project to be training data for other AI/LLM. Probably ends up being a knowledge transfer protocol.
  - devmunchies 4y ago
    Yeah I was actually thinking that we may see API maintainers publish a lot more sample code with lots of different scenarios for AI to train on. Similar to how JavaScript libs publish a Typescript type def file for people using Typescript.
  - larve 4y ago
    Seeding gpt (or using copilot) on snippets of your code is extremely effective. I often paste a function of my codebase before asking code questions, as in: “here is an example of a product class, generate the swagger spec for a user crud api” and it will infer how and even what kind of fields our user class has, without having seen it.
herpderperator 4y ago
What does LLM mean?
- dang 4y ago
  Large Language Model – the generic term for AIs like ChatGPT
- grensley 4y ago
  Large Language Model
- larsiusprime 4y ago
  Large Language Model, like GPT, etc.
- suzzer99 4y ago
  Yeah I'd never heard LLM, and I know what ChatGPT is and read hacker news pretty much every day. Just as an example for future writers – one line at the beginning of the piece explaining the acronym and using the ChatGPT example would go a long way.
RcouF1uZ4gsC 4y ago
There is a lot of doom and gloom about programming and AI. However, let me talk about something to be excited for. I am really excited about LLMs taking the drudgery out of programming. There are many times what I have a project in mind that I want to do, but there is a lot of boilerplate (writing configurations, or writing tests, etc) that I have to do and slows me down or for personal projects even keeps me from doing them. However, if I have an AI assistant that I can assign all the drudgery to, and I can focus on the interesting work, I feel that my creativity can really be unleashed. I am most excited to see the results of this in games. Right now, to produce a AAA game is a lot of work and requires massive teams (if only for the art assets). However, I can see a future where once again a single programmer can write very compelling games (and even have the AI generate the art assets for the game based on the programmer's vision). I personally am looking forward to the future.
- pm90 4y ago
  I share this excitement as well. Currently, designing or changing systems involves quite a bit of careful thought since you have to keep all the constraints in your mind and reason about how the system would work. If tools existed to create implementations based on these designs, that would be pretty amazing. Additionally: Being able to quickly iterate on designs that translate quickly into working systems will encourage more experimentation among people who might otherwise have been turned off by the amount of technical skills required to implement designs.
- bitL 4y ago
  I think it will be the opposite - it will take all fun out of programming and leave just the mess around connecting different interfaces and making them work reliably in a single product, i.e. debugging with nobody having the internal knowledge how the system actually works.
  - mprovost 4y ago
    I feel like this is something that people said when they started using compilers instead of manually writing assembly.
rosywoozlechan 4y ago
I'm going to be the optimist here and instead of AI killing engineering jobs it's going to turn most engineers into much better engineers and what tech company doesn't want a whole bunch of better engineers? Building applications right now is time intensive, it takes years to build products, months to build features, and maybe we change it to be weeks and days instead, respectively. Maybe it's all just upside.
- adolph 4y ago
  Speed of feature is less compelling than quality. There is a saying about EHRs, that they propagate error at the speed of light. Likewise speeding up feature accrual seems as if all the worst ideas will be implemented first as the best ones need time to think through and acculturate. On the other hand, if the models can be Atkinson-like and produce -2000 lines of code [0], that is a day to which I would look forward. 0. https://www.folklore.org/StoryView.py?story=Negative_2000_Lines_Of_Code.txt https://www.folklore.org/StoryView.py?story=Negative_2000_Li...
  - larve 4y ago
    I wouldn’t underestimate the ability to iterate on fully finished features quickly, if you have to operate with end users. Closing a feedback loop can lead to threshold effects.
    - adolph 4y ago
      That'd be great if true. Human nature being what it is, chances are good that the ill thought out and in retrospect harmful will be deployed just as faster than "fully finished" as they are now. Could that be iterated? Can a half baked release ever become whole or does progress just slow down and stretch out exponentially? 1/2 + 1/4 + 1/8 + ... Will be grateful and relieved when proven wrong.
      - larve 4y ago
        Yeah, I think my point is more that this makes it possible to iterate quicker while maintaining quality, which might be beneficial. Of course if you are working in a team that cares about the result (and the business does too), then iteration is always a win.
scj 4y ago
What I want is an AI that can do static analysis of code and report defects that traditional static analysis couldn't hope to detect. I suspect that detecting errors that way will be a part of training non-terrible code generators. But with the added benefit of being useful to human programmers as well.
- dyno12345 4y ago
  The other day I pasted chatgpt some code that I wanted it to add functionality to and it pointed out a bug in that code first (without me asking it to do that)
noobermin 4y ago
Just a word of note, this was the intent all along. The thing that stings is you trained your replacement.
- ResearchCode 4y ago
  What replacement? When "AI" can code decently, it will already have replaced Excel and Powerpoint workers, and middle managers sitting in pointless meetings.
holy_based 4y ago
[flagged]
- mikrl 4y ago
  As much as you’re shitposting, I agree. I worked my fingers to the bone for years before breaking into cushy software. Shit changes, I can change. We have to, or die. No LLM is gonna have the upper hand on me and my bag, because I’m gonna have the upper hand on it. Simple as.
- acover 4y ago
  Why is your tone harsh?
- MichaelMoser123 4y ago
  wow, i would dare to say that this is a writing level of GPT-2 :-)
- Ecco 4y ago
  Best comment I’ve read in a while. The last sentence was the cherry on the cake. I had a good time reading it, thanks!
- MrOwnPut 4y ago
  > If you lot are so deathly afraid of a tool taking over your job Not at all! You'll be sorely disappointed if you do actually get rid of the humans that you pay to solve your problems. > so we are not beholden to a bunch of overpaid pampered prima donnas They bring the logic into existence because you couldn't. So yes, you are beholden to them, because you are latching on to them, feeding off them like a parasite. > Whose job a program can do ... Good riddance. A program made by who? Trained on who? ;) And you are definitely speaking too soon... let's try "no code" one more time! The reality of today is you still need us to solve your many problems... Thanks for saying the quiet part out loud though.
- noobermin 4y ago
  You really think that it won't be the pampered prima donnas who won't use an AI which is trained on your code but just like chatgpt outputs nonsense to replace you?
elicksaur 4y ago
The author doesn’t understand the reason for the bimodal distribution of attorney compensation. If you look at a distribution of salaries overall, they are not smooth. There will be small peaks at round numbers, $60k, $70k, 80k, etc., rather than a smooth curve from 60->80. People like round numbers. The top peak in the law is due to human bias just the same. >Just like how a partner at Cravath likely sketches an outline of how they want to approach a particular case and swarms of largely replaceable lawyers fill in the details… Except those “replaceable” attorneys at Cravath are the top peak of the distribution. The distribution is caused by social decisions of what firms to compete with. Top salary information is well-publicized and firms who want to be in the category of “BigLaw” basically match each other. The lower peak of the distribution is more local attorneys. Your local estate law attorney, government workers like district attorneys/public defenders, etc. who are not competing with Cravath for talent or clients. I would assume this is already what exists in programming with “BigTech” salaries being equivalently publicized and matched across firms concerned with being perceived to be a “FAANG+”. If it doesn’t, I would guess the reason is that not enough positions actually exist in “BigTech” to make a significant blip on a population-wide salary graph. BigLaw attorneys exist in almost every medium-large city in the US, but you can’t say the same about BigTech programmers.
- ghaff 4y ago
  Law is extreme because you have people who went to a relative handful of law schools, clerked for federal court justices including SCOTUS, and work at white shoe big city law firms (like Cravath)--and everyone else. Yes, senior partners make more than junior partners make more than associates at those firms but it's also sort of up or out (and the out associates tend to move on to other things). And then there are the, as you say, local attorneys doing routine real estate transactions or whatever. Which tend to be pretty middling compensation generally. The story isn't quite that neat of course. Big company in-house counsels tend to be reasonably well compensated professional jobs and there are successful owners/partners of local law practices but there's a pretty good bimodal narrative in general. You can imagine why there might be a modality for people who can land jobs at specific tech companies that pay outsized salaries because they're so profitable and can afford to be very selective based on (they hope) relevant criteria but it's certainly based on less clear metrics if nothing else.
geraneum 4y ago
There is research [1] that suggests using GPS navigation could impact your brain function negatively. I wonder how this code generation “assistant” will impact the coding skills of its users in long term. As for the LLMs, stochastic parroting (hallucination), which is inherent to its design, makes it untrustworthy for precision work. Maybe some other architecture will prove more useful. LLMs can generate BS authoritatively which is easy to detect for toy examples and forgivable by the AI hype train but can be very costly or hard to fix in a complex system with a lot of moving parts. I’m actually optimistic about the future of AI. At the same time I believe we should know the capabilities and understand the limitations of our tools in order to use them more effectively. Unfortunately people are misunderstanding tools like ChatGPT, etc. for we are human and vulnerable to anthropomorphism. 1. https://www.nature.com/articles/ncomms14652 https://www.nature.com/articles/ncomms14652
- testtesttesst 4y ago
  >people misunderstand and anthropomorphize LLMs I agree but there is a certain "god of the gaps" process that's happening here. To me what's been achieved so far is encoding text into natural (and programming) languages. These models can't reason and e.g., can't pass a Turing test. For example, if you play with GPT-3 or ChatGPT enough you can definitely get it to fail on a task a human would succeed at. When models so far have achieved "reasoning" behavior, usually it's in conjunction with a classical algorithm -- for example AlphaZero used a model to evaluate chess positions but then ran a tree search algorithm to decide what move to play. I think if LLMs can be combined effectively with some kind of computation process like in AlphaZero or some kind of "chains of thought" [1], then we might see them do things we'd truly call "reasoning". Some "gaps" have been (1) "(encoding into) images is hard" (2) "(encoding into) natural language is hard" and now (3) "'reasoning' is hard" and afterwards (4) "subjective experience is hard". Whether to be optimistic or pessimistic about AI depends on the final "gap" -- whether there is an irreplicable/noncomputable/etc. property of human consciousness other than qualia. That is, if "understanding" is hard (e.g., via some voodoo like [2]). If not humans will become obsolete, so I hope the answer is yes. [1]: https://ai.googleblog.com/2022/05/language-models-perform-reasoning-via.html https://ai.googleblog.com/2022/05/language-models-perform-re... [2]: https://en.m.wikipedia.org/wiki/Orchestrated_objective_reduction https://en.m.wikipedia.org/wiki/Orchestrated_objective_reduc...
jesuscript 4y ago
AI/ML will solve some great problems, many that are critical. Similar to how Wix solves your basic webpage for the average person. Does Wix solve anything for your typical enterprise or SAAS app? Even if it did, would these institutions not manage to add a new layer to the layer cake? It's kind of been one of my main critiques of companies like Palantir. You really can't just dump a company's data into the big ML tumbler and expect useful/sellable products to just pop out. Take a random use case like, I don't know, Pepsi. Maybe Pepsi can get some insights about their supply chain and customer preferences. Alright? Okay. Did the world end because that happened? Not really. Shifting gears back to programmers - AI will be a super smart programmer that will write a bunch of your code. I worked with some super smart programmers, and they can manage to write a lot of super smart code that eventually becomes a clusterfuck too. We'll just go from unwrapping human shit code, to unwrapping AI shit code (or even better, unwrapping a mix of human shit and AI shit). The job prospects here are even greater than you imagine. Long as you maintain the belief in the fundamental truth, that anything in existence is capable of writing shit code, then the world will never end for programmers.
- noobermin 4y ago
  Sometimes though we shouldn't have the AI write the code first. So many times, I look over things people copy and pasted piece by piece from blogs and stackoverflow, when if they thought about it for 5 minutes, they'd realize a simpler implementation would have worked and been easier to write in the end. AI, if chatgpt is any indication, is moving toward overly flowerly and long prose while telling lies, that is stuff that looks "impressive" at a glance like the large ornate class hiearchies of java and c++ in the 90s looked but are things that could be done in a shell script. I am worried then that like stackoverflow, AI will make a generation of developers who are extremely lazy and not critically thinking, just like stackoverflow did, and leave more of a mess for us to pick up after. Of course, in the midst of this, management won't know the difference. Just like chatgpt sounding impressive as it tells falsehoods, AI-code will look impressive to fools.
  - sanderjd 4y ago
    Yeah I think LLMs are really amazing and I'm open minded about how they can contribute to the craft of creating software. I've played around with these for generating code and it's very cool how much can be done. BUT. Honestly I just see a lot of code review and refactoring of shitty AI-generated code in my future. And I think people who lean on this heavily are going to struggle to progress... Just like when I get the whiff of code copied verbatim from SO, it's going to put a sour taste in my mouth when I get a series of reviews with a bunch of poorly factored AI generated code. But maybe it's more likely that old fogeys like me with our backwards beliefs are not long for this world. But I dunno, I haven't yet been replaced by the hordes of actual humans crowdsourcing this on SO, so color me a bit skeptical!
rajin444 4y ago
Engineers create value - making them more efficient at their work generally creates more opportunity for work. Lawyers are an unfortunate necessity that extract value from society. Making them more efficient at their work (hopefully) creates less opportunity for work.
- neilv 4y ago
  I disagree with these generalizations. You could just as easily say that engineers work to exploit and otherwise harm people, and lawyers work to protect people from exploitation and other harm.
- tracerbulletx 4y ago
  This type of attitude displays a common blind spot of many engineers. Manipulating human social systems to achieve a specific result is harder, and at the end of the day probably more important than manipulating computer systems. Software engineers have a tendency to think "well if only everyone would just act exactly so and so none of this would be a problem! So it's obviously everyone elses fault for acting wrongly.", and they refuse to accept that politics, law, sales, management, and all the rest are completely 100% necessary and that not a single complex dynamic human interactive system is ever going to just do whatever they think it should do as a matter of moral obviousness.
  - codekansas 4y ago
    This seems tangential? Engineering and law aren't the only exemplars of these categories, there are lots of fields where the marginal problems to solve are interesting verses uninteresting, they were just contrasting the two in the article. For example, more productive artists usually leads to more experimentation and weirder artwork; more doctors usually leads to more specialization and advanced knowledge; more VC money for crypto projects usually leads to more Ponzi schemes; etc, etc
aetherson 4y ago
Comment here seem to largely miss the point of the article, which isn't saying that nobody will employ software engineers. It is saying that instead of the current model of: 1 very highly compensated staff+ engineer 4 highly compensated senior engineers 4 pretty well-compensated mid-level engineers 2 decently compensated junior engineers The future might look like: 1 yet-more-highly-compensated staff+ engineer who specializes in architecture 10 decently compensated junior engineers who specialize in error-checking LLMs and detail-intensive technical work In the same way that paralegals aren't unskilled people, and aren't paid terribly -- but they aren't doing the same job as the partner-level lawyer who's crafting the legal strategy, and they don't have a simple clean progression to eventually getting his job.
- fellellor 4y ago
  Humans have trouble error checking each other's code. Imagine an army of devs who are expected to read and error check a machine. It's an absolute nightmare and fits the literal definition of bullshit job. But if it pays someone will do it. I dont know how one would hire an army of these sad folk though.
- twelve40 4y ago
  you're probably right (and thanks for breaking it down), but it's more fun to engage in doom porn!
- steve76 4y ago
  [dead]
- noobermin 4y ago
  How about 10 highly compensated middle managers and their staff who convinced the c suite to fire the senior engineers because of an AI article they read 2 poorly compensated junior developers with an AI bot replacing a 10 person team, who will get canned when SHTF so the 10 middle managers can jump ship to the next company with a padded resume. There are also 10 people because the middle managers got their friends positions from the savings due to the firing of the old engineers.
dbancajas 4y ago
What even is LLM?
- sthustfo 4y ago
  Large Language Model
marcell 4y ago
This post is way to bullish on ChatGPT for coding, and underestimates how big the gap is between ChatGPT and a mid level FANG eng. ChatGPT is very impressive in what it can do, but if you deviate problem statements even a little bit from what it's seen before, it completely fails. Just an example try asking it to write Battleship in Python. It does very well. But then, try changing the rules a bit, and adding some requirements. It spits out code that looks correct if you squint, but has terrible style, and often doesn't even work. Check out the test prompt [1]. Maybe GPT-4 is 1000x better at coding, but barring that, I don't see ChatGPT being useful as a replacement for human coders. [1] Prompt: Write a version of Battleship with modified rules for sinking a ship. In this version, a ship is sunk when the "head" and "tail" of the ship (its beginning and end parts) are hit. Hits to the center do not matter. Use object oriented programming, with classes for the Board and for Ships. There should be support for multiple ships and two board instances. Write it in Python.
- alar44 4y ago
  You're expecting way too much. Have it give you a list of functions needed to write battleship. Then have it write the classes. Then set the rules. Then have it write a GUI etc. You'll have a much more successful encounter that way.
la64710 4y ago
I beg to differ with the author’s perception ..in all it’s glory I don’t think L7 or principals or even software managers will have a reasonable chance of surviving. Because it is quite possible that agents built on LLM and AGI will be perfectly capable of not only arguing with each other and weigh on the pros and cons of any given option and arrive at a consensus just like the current Principals or managers do their negotiation, it is also quite possible they will be able to assign task and track the progress as well as take remediation action when needed. All these can be handled by advanced agents to a very large part. It is also possible to see teams of these AI agents working together in a cooperative way and surprising us with the outcome. Good luck !!
- codekansas 4y ago
  This sounds more like a sci-fi AI bot than the current route that AI research is going. How do you even get the training data or build a simulation for an agent like this...
  - la64710 4y ago
    Just ask chatGPT anything you would ask a principal or a manager and you maybe surprised about how coherent the answer is. I don’t think the training data is that exotic and may be already there , the agents are continuously learning.
- eouwt 4y ago
  "LLM and AGI" Feels like the three letters A-G-I are doing a lot of heavy lifting there
larve 4y ago
Chatgpt has been pretty transformative to my programming, probably surpassing the only other thing coming close of really leveraging powerful ides like IntelliJ. It can generate all the scaffolding I need by just inputting a couple of sketches of Plantuml or even just markdown lists. I can build tools, unit tests, test data, fuzz harnesses, admin uis, dashboards, all within a few hours. I can transform a couple of words into very legible documentation. If you seed it with just a couple of lines of your own code and setup, it’s able to usually build stuff that matches the style of the codebase. It’s broken and wrong 80% of the time, but in a way that I can fix within a few minutes. This leaves me with 4h of high quality thinking and programming time instead of 1. It means I can try out 3 different approaches with full tooling instead of 1. It means I can build a whole terraform for aws and gcp and bare iron in 30 minutes. The fact that I can build solid tools and a full slurry of unit tests and end up with 4x time in my day is just blowing my mind. The quality of my software has tremendously gone up despite using something that mostly outputs broken code.
- sbrother 4y ago
  May I ask how you're getting ChatGPT to be reliable enough as a service to get these results? I had a similar experience the first week, but now most times I try it is overloaded and inaccessible, and when it's not the hourly rate limiter makes it difficult to be very productive.
  - tester457 4y ago
    Using the api is better.
    - rlt 4y ago
      There's an API for ChatGPT? Or are you using the GPT-3 / Codex API?
  - larve 4y ago
    I haven’t had too many issues. I must admit I use the api and copilot a lot.
- 4y ago
epicureanideal 4y ago
It's possible making coding easier will make salaries go up. https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox
- codekansas 4y ago
  I suspect we're about to enter a world where AI supercharges the average engineer's productivity, which is going to make it so that so much more stuff can be built. Using Copilot, the time to build a prototype for a simple app or API drops by a big multiple. Things that would take a week of work can be tried out in a weekend. Super cool future.
travisgriggs 4y ago
If I did search & replace on this article, replacing “programmer” with “manager”, would the conclusions and reasoning be much different (regardless of whether they’re spot on or not)?
btheshoe 4y ago
I think this makes assumptions about the demand for software engineers that seem unproven. Law salaries are bimodal because the demand for lawyers is limited, so there's only so many big law firms that can exist. In the near future, at least, it seems like demand for software engineers far outstrips supply and I think this will stay true in the long run because the market for software engineers is just different from the market for lawyers, in that software engineers can work on a multitude of different industries (faang covers personal devices, social media, entertainment, e-commerce, search, and more). I think the real concern is for when the rate of new startups decreases, because I view startups as basically the tech industry's way for searching for ways to increase the overall size of the pie, and startups slowing down means demand for software engineers will start to stagnate.
- karmasimida 4y ago
  > it seems like demand for software engineers far outstrips supply Definitely not the case when layoff right now is in full force. Also this doesn't necessarily indicate high salary anyway. There is a huge demand for fruit picker, but due to the nature of economy and margin of business, the salary will stay deflated thus the shortage persists. In a high interests, no free money environment, will software firms still able to demonstrate ability to generate return for investors? That would the ultimate key for whether the high salary is going to sustain or disappear. Regardless, I think this article has a point. We might be as well observing the end of the golden era for software engineers. LLM is a long-term salary oppressor, but in short term, simple dynamics of economy would test the market sufficiently.
  - btheshoe 4y ago
    doomer take but tbh i respect it. but the central thesis of the tech industry is that it allows for products to be created with very low marginal cost which facilitates the high returns and if that thesis turns out to no longer be true then i would say that the identity of the tech industry has irrevocably changed.
jagtstronaut 4y ago
This is very doom and gloom compared to the actual landscape in law. Yeah entry level salaries suck in law, but plenty of decent lawyers who are nothing special end up billing 200 an hour 6 years into their career and taking their 40% of that. While outside of big law is not paying what it used to it is still an upper middle class profession if you stick it out.
readthenotes1 4y ago
Hasn't offshoring already done that to software? Unlike law, which is pretty local, the field is universal in software and most of the Asian, Eastern European, or South American programmers make nothing like north Americans. And programmers in Silicon Valley or Seattle make tons more than in Oklahoma.
karmasimida 4y ago
LLM might simply remove the need for junior engineers instead of requiring more of them. Thinking of instead of requiring 20 people team, now you can well function with like 10 people.
imtringued 4y ago
"Note that querying LLMs and making the outputted code conform to specifications is probably a lot easier than writing the code yourself" This overestimates the time developers spend typing text and underestimates the time spent thinking about problems, verification and locating bugs/code, etc. Just because the output of a model is text doesn't mean it is a replacement. You would need extraordinary "single shot" functionality where the LLM always outputs correct code and knows how to deal with any potential specification. The moment it fails then you wouldn't want "paralegal" developers, you would want normal, maybe inexperienced, developers. Single shot functionality is very impressive because humans can't do it, they build things incrementally, but once your program gets large, it can only be written incrementally, which at least implies that the AI needs to keep track of unbounded context.
- nonethewiser 4y ago
  The smaller the problem the easier it is to use AI. Which makes the window where it's useful very small. Its easier to understand and test code that you wrote than code you just read. If you ask it to write a 100k line program you're going to have to trace through everything and read it countless times over to fix any issues. If you ask it to write a small 10 line snippet that you need to test you might as well do it yourself.
  - flir 4y ago
    It can write a ten line function faster than I can, and my code has a lot of ten line functions in it. Even for trivial code, it's a force multiplier (moreso when plumbed into an IDE)
- uwuemu 4y ago
  [dead]
- aws_ls 4y ago
  Only yesterday, I worked on a ticket for few hours, inspecting several Mongo collections in QE and prod. And then had to make 4 to 5 lines of changes across 3/4 python scripts. And then manually run them to patch the mongo collections. Which was the root cause of the bug. I don't see GPT doing this kind of stuff anytime soon. And yes, very less percentage of the time, do we right chunks of fresh code, in which this could help. But that again to an extent. It can generate functions. But the glue code is still complicated. So it can definitely be a better Stack Overflow replacement, perhaps.
bitwize 4y ago
A funny thing I noticed when googling whether melamine was microwave-safe: I got a lot of clickbaity articles from sketchy sites. Some said melamine is microwave safe, some said it is not, and some seemed to contradict each other. An Asimov story called "Liar!" concerns a robot which has the ability to read minds. When asked to tell what's going on in another person's mind (does so-and-so have a crush on me? is my boss planning to give me a promotion?) the robot, following its First Law directive not to allow humans to come to harm (including emotional harm), tells the querent what they would like to hear based on the robot's reading of the querent's mind, instead of the truth because the truth can hurt. When the robot is shown that its actions hurt humans anyway, it promptly shuts down and refuses to function. These language models seem like that robot. The articles on melamine suggest to me that the AIs which generated them are producing output that their statistical models suggest the querent wants to hear, rather than output with correct answers. In light of this, it seems that using these models for assistance in coding would be like having an obsequious but shitty coworker, whose output you must carefully review and vet because even money it's not a solution to your problem, just a guess at what you want to see based on a vast statistical soup built up of thousands of programmers' efforts on millions of tasks. Which means any effort you saved up now will be paid for with extra effort later, as every programmer knows it's much easier and more fun to wrote new code than to review and debug someone else's code, especially if it's shitty, especially^2 if it's subtly shitty. It really is like Teitelman's DWIM evolved, with correspondingly magnified lossage in its failure modes!
valdiorn 4y ago
Define your damn acronyms. I read the entire article and I STILL don't know what LLM means.
- deleted 4y ago
  [deleted]
- bayesian_horse 4y ago
  I feel you.. I think he means large language models.
  - dbancajas 4y ago
    Basic mistake. Can't believe this was not corrected in OP. He should write title as "xxxx in Large Language Models (LLM)"
Dave3of5 4y ago
I wanted to see if what all these influencers were saying was true and so I tried to use ChatGPT to create a webapp the other day. The first problem is that it's totally overloaded at the moment if you use the app and so even just logging in is almost impossible. After 20 minutes I managed to get in. I first asked it to come up with 5 different ideas for the app. All of them were generic rubbish like a blog or suchlike so I asked it to be more niche and specific and it managed to throw up things like: * A Project management webapp * Farm management software * Todo List I picked the farm management software which I actually now interpreted as crop management software. This is a very competitive market lots of big software companies doing this already but I thought ok I'll give it a try. I asked it to build a basic landing page which is initially refused to do. When I did prompt it correctly it built a basic HTML only page with no styling and basically no content other than a header and a couple of fake testimonials. It then refused to style the thing at all and closed the chat. I then managed to prompt it correctly and it updated the font and stuck some stuff in boxes. During this process it told me several times that it will not create a landing page. I then gave up on the landing page and tried to get it to be able to help me write the code on the frontend to allow the user to define the crop areas on google maps. It spat out just the JS (no html) and which showed how to draw a polygon on the map basically the code you would get by looking up the google maps documentation. During this process the text reply was very slow like character by character and it kicked me out several times. I think your jobs are safe at the moment...
bayesian_horse 4y ago
It seems hard enough to find "IT people" who can plug cables in correctly or setup some software on a laptop (exaggerating, but only slightly). I'm not worried.
bartimus 4y ago
> we are perhaps converging to a future where a FAANG L7 can just sketch out architectural details and the programmer equivalent of paralegals will simply query the latest LLM and clean up the output. This is just proposing waterfall + outsourcing all over again. The author doesn't seem to understand that "sketching out architectural details" is the thing that developers do.
- deleted 4y ago
  [deleted]
sjducb 4y ago
Societies demand for software is basically infinite. Right now only people who can pay for developers get to have good custom software. As LLMs increase the output of a programmer more people will be able to comission custom software. More software will get made and programmer salaries will remain high.
maest 4y ago
From the article: > P.S. If your objection is that you played with ChatGPT and wasn’t impressed, you should try projecting trends beyond the t=1 month timescale. That is a dismissive stance announced in a very obnoxious way. I don't know where and how fast the AI thing will go but I do know they're treating an unknowable outcome as fact.

[dead]

> What happened? Well, if you’re an elite lawyer in the modern age, you probably command a large team of lower-tier lawyers, cheap paralegals, and so on, who are very good at using the Internet to look up case law and draft your opinions and so on. This already happened to programmers. We call these elites some version of tech lead or staff engineer. Maybe even principal, but that feels different still.

sublinear 4y ago

Except you can't do a web search on proprietary code within your business, and hosting AI on-premises to model your codebase would likely be a huge waste of money and slower than just telling a real life person what you want. There has been zero-progress towards AI comprehension, so you can only get out of it what you are already familiar with. The human must always do the mental heavy lifting to tease out a good result, or indeed even know if it's a good result and not just a hallucinated mess.

angarg12 4y ago

But programmer salaries already are bimodal, or even trimodal [1]. [1] https://blog.pragmaticengineer.com/software-engineering-salaries-in-the-netherlands-and-europe/ https://blog.pragmaticengineer.com/software-engineering-sala...

fdsfawfds 4y ago
[dead]
raincom 4y ago
Indeed, programmer salaries even in the Bay Area are bimodal.
- reducesuffering 4y ago
  Bay Area, no: https://www.levels.fyi/t/software-engineer/locations/san-francisco-bay-area https://www.levels.fyi/t/software-engineer/locations/san-fra...
  - csomar 4y ago
    That looks like bi-modal to me.
    - reducesuffering 4y ago
      Don't be fooled by the $23k range brackets next to a single bracket lumping in everything $527k+. That bracket is just an aggregation of the same chart trajectory, not another mode.
  - mangosteenjuice 4y ago
    I can't believe this needs to be said, but the salary data that you're linking to excludes the thousands of Bay Area employers who aren't part of levels.fyi's dataset. Most of the employers in their dataset compete with each other, and pay in the second/higher bimodal curve.
    - fexecve 4y ago
      This is correct. This also excludes contractors, which there are tons of in the Bay Area, and they make a lot less than a regular employee, even at places like Google.
sanderjd 4y ago
Right! This was my first thought. I think the author missed that software is already quite like law in this way.

osigurdson 4y ago

What change occurred between 1991 and 2000 to explain the emergence of the bimodal distribution? Presumably lawyers have had paralegals for centuries.

cldellow 4y ago
The author of this post claims that the Internet happened, enabling easier, cheaper access to law databases, which increased the value that "top tier" lawyers could extract from their paralegals. > What happened? Well, if you’re an elite lawyer in the modern age, you probably command a large team of lower-tier lawyers, cheap paralegals, and so on, who are very good at using the Internet to look up case law and draft your opinions and so on. In contrast, if you’re an elite lawyer in the year 1990, it’s much more valuable for you specifically to have an encyclopedic knowledge of all the relevant case law, and it’s much harder for you to delegate modular pieces of work off to less competent people.

devmunchies 4y ago

I think it will be more like airline pilots with the addition of autopilot. Maybe you could write whole services, and networks of services with AI, but you need engineers to be the guiding hand. Engineers will always be better at using AI to build systems than non-developers. Also, it’s worth pointing out that AI is using existing data. If a new iOS API comes out, the AI will need code bases with it already implemented to be able to train with it. So in my mind, it will hurt the junior developers doing grunt work more than general devs. I think no code/low code products are more of a risk to hurting devs than AI.

Nathanba 4y ago
depending on how advanced this AI will be, it could learn from a few samples and the API docs, possibly itself designed to be optimally consumed by AI. If I knew that AI would consume my custom programming language docs then I would make sure that each API is used once in some kind of sample code so it can give answers to everything. Plus, that's a sensible thing to do anyway.
- reducesuffering 4y ago
  Even better (or worse), tell your AI/LLM to output sample uses of the project to be training data for other AI/LLM. Probably ends up being a knowledge transfer protocol.
- devmunchies 4y ago
  Yeah I was actually thinking that we may see API maintainers publish a lot more sample code with lots of different scenarios for AI to train on. Similar to how JavaScript libs publish a Typescript type def file for people using Typescript.
- larve 4y ago
  Seeding gpt (or using copilot) on snippets of your code is extremely effective. I often paste a function of my codebase before asking code questions, as in: “here is an example of a product class, generate the swagger spec for a user crud api” and it will infer how and even what kind of fields our user class has, without having seen it.

herpderperator 4y ago

What does LLM mean?

dang 4y ago
Large Language Model – the generic term for AIs like ChatGPT
grensley 4y ago
Large Language Model
larsiusprime 4y ago
Large Language Model, like GPT, etc.
suzzer99 4y ago
Yeah I'd never heard LLM, and I know what ChatGPT is and read hacker news pretty much every day. Just as an example for future writers – one line at the beginning of the piece explaining the acronym and using the ChatGPT example would go a long way.

RcouF1uZ4gsC 4y ago

There is a lot of doom and gloom about programming and AI. However, let me talk about something to be excited for. I am really excited about LLMs taking the drudgery out of programming. There are many times what I have a project in mind that I want to do, but there is a lot of boilerplate (writing configurations, or writing tests, etc) that I have to do and slows me down or for personal projects even keeps me from doing them. However, if I have an AI assistant that I can assign all the drudgery to, and I can focus on the interesting work, I feel that my creativity can really be unleashed. I am most excited to see the results of this in games. Right now, to produce a AAA game is a lot of work and requires massive teams (if only for the art assets). However, I can see a future where once again a single programmer can write very compelling games (and even have the AI generate the art assets for the game based on the programmer's vision). I personally am looking forward to the future.

pm90 4y ago
I share this excitement as well. Currently, designing or changing systems involves quite a bit of careful thought since you have to keep all the constraints in your mind and reason about how the system would work. If tools existed to create implementations based on these designs, that would be pretty amazing. Additionally: Being able to quickly iterate on designs that translate quickly into working systems will encourage more experimentation among people who might otherwise have been turned off by the amount of technical skills required to implement designs.
bitL 4y ago
I think it will be the opposite - it will take all fun out of programming and leave just the mess around connecting different interfaces and making them work reliably in a single product, i.e. debugging with nobody having the internal knowledge how the system actually works.
- mprovost 4y ago
  I feel like this is something that people said when they started using compilers instead of manually writing assembly.

rosywoozlechan 4y ago

I'm going to be the optimist here and instead of AI killing engineering jobs it's going to turn most engineers into much better engineers and what tech company doesn't want a whole bunch of better engineers? Building applications right now is time intensive, it takes years to build products, months to build features, and maybe we change it to be weeks and days instead, respectively. Maybe it's all just upside.

adolph 4y ago
Speed of feature is less compelling than quality. There is a saying about EHRs, that they propagate error at the speed of light. Likewise speeding up feature accrual seems as if all the worst ideas will be implemented first as the best ones need time to think through and acculturate. On the other hand, if the models can be Atkinson-like and produce -2000 lines of code [0], that is a day to which I would look forward. 0. https://www.folklore.org/StoryView.py?story=Negative_2000_Lines_Of_Code.txt https://www.folklore.org/StoryView.py?story=Negative_2000_Li...
- larve 4y ago
  I wouldn’t underestimate the ability to iterate on fully finished features quickly, if you have to operate with end users. Closing a feedback loop can lead to threshold effects.
  - adolph 4y ago
    That'd be great if true. Human nature being what it is, chances are good that the ill thought out and in retrospect harmful will be deployed just as faster than "fully finished" as they are now. Could that be iterated? Can a half baked release ever become whole or does progress just slow down and stretch out exponentially? 1/2 + 1/4 + 1/8 + ... Will be grateful and relieved when proven wrong.
    - larve 4y ago
      Yeah, I think my point is more that this makes it possible to iterate quicker while maintaining quality, which might be beneficial. Of course if you are working in a team that cares about the result (and the business does too), then iteration is always a win.

scj 4y ago

What I want is an AI that can do static analysis of code and report defects that traditional static analysis couldn't hope to detect. I suspect that detecting errors that way will be a part of training non-terrible code generators. But with the added benefit of being useful to human programmers as well.

dyno12345 4y ago
The other day I pasted chatgpt some code that I wanted it to add functionality to and it pointed out a bug in that code first (without me asking it to do that)

noobermin 4y ago

Just a word of note, this was the intent all along. The thing that stings is you trained your replacement.

ResearchCode 4y ago
What replacement? When "AI" can code decently, it will already have replaced Excel and Powerpoint workers, and middle managers sitting in pointless meetings.

holy_based 4y ago

[flagged]

mikrl 4y ago
As much as you’re shitposting, I agree. I worked my fingers to the bone for years before breaking into cushy software. Shit changes, I can change. We have to, or die. No LLM is gonna have the upper hand on me and my bag, because I’m gonna have the upper hand on it. Simple as.
acover 4y ago
Why is your tone harsh?
MichaelMoser123 4y ago
wow, i would dare to say that this is a writing level of GPT-2 :-)
Ecco 4y ago
Best comment I’ve read in a while. The last sentence was the cherry on the cake. I had a good time reading it, thanks!
MrOwnPut 4y ago
> If you lot are so deathly afraid of a tool taking over your job Not at all! You'll be sorely disappointed if you do actually get rid of the humans that you pay to solve your problems. > so we are not beholden to a bunch of overpaid pampered prima donnas They bring the logic into existence because you couldn't. So yes, you are beholden to them, because you are latching on to them, feeding off them like a parasite. > Whose job a program can do ... Good riddance. A program made by who? Trained on who? ;) And you are definitely speaking too soon... let's try "no code" one more time! The reality of today is you still need us to solve your many problems... Thanks for saying the quiet part out loud though.
noobermin 4y ago
You really think that it won't be the pampered prima donnas who won't use an AI which is trained on your code but just like chatgpt outputs nonsense to replace you?

elicksaur 4y ago

The author doesn’t understand the reason for the bimodal distribution of attorney compensation. If you look at a distribution of salaries overall, they are not smooth. There will be small peaks at round numbers, $60k, $70k, 80k, etc., rather than a smooth curve from 60->80. People like round numbers. The top peak in the law is due to human bias just the same. >Just like how a partner at Cravath likely sketches an outline of how they want to approach a particular case and swarms of largely replaceable lawyers fill in the details… Except those “replaceable” attorneys at Cravath are the top peak of the distribution. The distribution is caused by social decisions of what firms to compete with. Top salary information is well-publicized and firms who want to be in the category of “BigLaw” basically match each other. The lower peak of the distribution is more local attorneys. Your local estate law attorney, government workers like district attorneys/public defenders, etc. who are not competing with Cravath for talent or clients. I would assume this is already what exists in programming with “BigTech” salaries being equivalently publicized and matched across firms concerned with being perceived to be a “FAANG+”. If it doesn’t, I would guess the reason is that not enough positions actually exist in “BigTech” to make a significant blip on a population-wide salary graph. BigLaw attorneys exist in almost every medium-large city in the US, but you can’t say the same about BigTech programmers.

ghaff 4y ago
Law is extreme because you have people who went to a relative handful of law schools, clerked for federal court justices including SCOTUS, and work at white shoe big city law firms (like Cravath)--and everyone else. Yes, senior partners make more than junior partners make more than associates at those firms but it's also sort of up or out (and the out associates tend to move on to other things). And then there are the, as you say, local attorneys doing routine real estate transactions or whatever. Which tend to be pretty middling compensation generally. The story isn't quite that neat of course. Big company in-house counsels tend to be reasonably well compensated professional jobs and there are successful owners/partners of local law practices but there's a pretty good bimodal narrative in general. You can imagine why there might be a modality for people who can land jobs at specific tech companies that pay outsized salaries because they're so profitable and can afford to be very selective based on (they hope) relevant criteria but it's certainly based on less clear metrics if nothing else.

geraneum 4y ago

There is research [1] that suggests using GPS navigation could impact your brain function negatively. I wonder how this code generation “assistant” will impact the coding skills of its users in long term. As for the LLMs, stochastic parroting (hallucination), which is inherent to its design, makes it untrustworthy for precision work. Maybe some other architecture will prove more useful. LLMs can generate BS authoritatively which is easy to detect for toy examples and forgivable by the AI hype train but can be very costly or hard to fix in a complex system with a lot of moving parts. I’m actually optimistic about the future of AI. At the same time I believe we should know the capabilities and understand the limitations of our tools in order to use them more effectively. Unfortunately people are misunderstanding tools like ChatGPT, etc. for we are human and vulnerable to anthropomorphism. 1. https://www.nature.com/articles/ncomms14652 https://www.nature.com/articles/ncomms14652

testtesttesst 4y ago
>people misunderstand and anthropomorphize LLMs I agree but there is a certain "god of the gaps" process that's happening here. To me what's been achieved so far is encoding text into natural (and programming) languages. These models can't reason and e.g., can't pass a Turing test. For example, if you play with GPT-3 or ChatGPT enough you can definitely get it to fail on a task a human would succeed at. When models so far have achieved "reasoning" behavior, usually it's in conjunction with a classical algorithm -- for example AlphaZero used a model to evaluate chess positions but then ran a tree search algorithm to decide what move to play. I think if LLMs can be combined effectively with some kind of computation process like in AlphaZero or some kind of "chains of thought" [1], then we might see them do things we'd truly call "reasoning". Some "gaps" have been (1) "(encoding into) images is hard" (2) "(encoding into) natural language is hard" and now (3) "'reasoning' is hard" and afterwards (4) "subjective experience is hard". Whether to be optimistic or pessimistic about AI depends on the final "gap" -- whether there is an irreplicable/noncomputable/etc. property of human consciousness other than qualia. That is, if "understanding" is hard (e.g., via some voodoo like [2]). If not humans will become obsolete, so I hope the answer is yes. [1]: https://ai.googleblog.com/2022/05/language-models-perform-reasoning-via.html https://ai.googleblog.com/2022/05/language-models-perform-re... [2]: https://en.m.wikipedia.org/wiki/Orchestrated_objective_reduction https://en.m.wikipedia.org/wiki/Orchestrated_objective_reduc...

jesuscript 4y ago

AI/ML will solve some great problems, many that are critical. Similar to how Wix solves your basic webpage for the average person. Does Wix solve anything for your typical enterprise or SAAS app? Even if it did, would these institutions not manage to add a new layer to the layer cake? It's kind of been one of my main critiques of companies like Palantir. You really can't just dump a company's data into the big ML tumbler and expect useful/sellable products to just pop out. Take a random use case like, I don't know, Pepsi. Maybe Pepsi can get some insights about their supply chain and customer preferences. Alright? Okay. Did the world end because that happened? Not really. Shifting gears back to programmers - AI will be a super smart programmer that will write a bunch of your code. I worked with some super smart programmers, and they can manage to write a lot of super smart code that eventually becomes a clusterfuck too. We'll just go from unwrapping human shit code, to unwrapping AI shit code (or even better, unwrapping a mix of human shit and AI shit). The job prospects here are even greater than you imagine. Long as you maintain the belief in the fundamental truth, that anything in existence is capable of writing shit code, then the world will never end for programmers.

noobermin 4y ago
Sometimes though we shouldn't have the AI write the code first. So many times, I look over things people copy and pasted piece by piece from blogs and stackoverflow, when if they thought about it for 5 minutes, they'd realize a simpler implementation would have worked and been easier to write in the end. AI, if chatgpt is any indication, is moving toward overly flowerly and long prose while telling lies, that is stuff that looks "impressive" at a glance like the large ornate class hiearchies of java and c++ in the 90s looked but are things that could be done in a shell script. I am worried then that like stackoverflow, AI will make a generation of developers who are extremely lazy and not critically thinking, just like stackoverflow did, and leave more of a mess for us to pick up after. Of course, in the midst of this, management won't know the difference. Just like chatgpt sounding impressive as it tells falsehoods, AI-code will look impressive to fools.
- sanderjd 4y ago
  Yeah I think LLMs are really amazing and I'm open minded about how they can contribute to the craft of creating software. I've played around with these for generating code and it's very cool how much can be done. BUT. Honestly I just see a lot of code review and refactoring of shitty AI-generated code in my future. And I think people who lean on this heavily are going to struggle to progress... Just like when I get the whiff of code copied verbatim from SO, it's going to put a sour taste in my mouth when I get a series of reviews with a bunch of poorly factored AI generated code. But maybe it's more likely that old fogeys like me with our backwards beliefs are not long for this world. But I dunno, I haven't yet been replaced by the hordes of actual humans crowdsourcing this on SO, so color me a bit skeptical!

rajin444 4y ago

Engineers create value - making them more efficient at their work generally creates more opportunity for work. Lawyers are an unfortunate necessity that extract value from society. Making them more efficient at their work (hopefully) creates less opportunity for work.

neilv 4y ago
I disagree with these generalizations. You could just as easily say that engineers work to exploit and otherwise harm people, and lawyers work to protect people from exploitation and other harm.
tracerbulletx 4y ago
This type of attitude displays a common blind spot of many engineers. Manipulating human social systems to achieve a specific result is harder, and at the end of the day probably more important than manipulating computer systems. Software engineers have a tendency to think "well if only everyone would just act exactly so and so none of this would be a problem! So it's obviously everyone elses fault for acting wrongly.", and they refuse to accept that politics, law, sales, management, and all the rest are completely 100% necessary and that not a single complex dynamic human interactive system is ever going to just do whatever they think it should do as a matter of moral obviousness.
- codekansas 4y ago
  This seems tangential? Engineering and law aren't the only exemplars of these categories, there are lots of fields where the marginal problems to solve are interesting verses uninteresting, they were just contrasting the two in the article. For example, more productive artists usually leads to more experimentation and weirder artwork; more doctors usually leads to more specialization and advanced knowledge; more VC money for crypto projects usually leads to more Ponzi schemes; etc, etc

aetherson 4y ago

Comment here seem to largely miss the point of the article, which isn't saying that nobody will employ software engineers. It is saying that instead of the current model of: 1 very highly compensated staff+ engineer 4 highly compensated senior engineers 4 pretty well-compensated mid-level engineers 2 decently compensated junior engineers The future might look like: 1 yet-more-highly-compensated staff+ engineer who specializes in architecture 10 decently compensated junior engineers who specialize in error-checking LLMs and detail-intensive technical work In the same way that paralegals aren't unskilled people, and aren't paid terribly -- but they aren't doing the same job as the partner-level lawyer who's crafting the legal strategy, and they don't have a simple clean progression to eventually getting his job.

fellellor 4y ago
Humans have trouble error checking each other's code. Imagine an army of devs who are expected to read and error check a machine. It's an absolute nightmare and fits the literal definition of bullshit job. But if it pays someone will do it. I dont know how one would hire an army of these sad folk though.
twelve40 4y ago
you're probably right (and thanks for breaking it down), but it's more fun to engage in doom porn!
steve76 4y ago
[dead]
noobermin 4y ago
How about 10 highly compensated middle managers and their staff who convinced the c suite to fire the senior engineers because of an AI article they read 2 poorly compensated junior developers with an AI bot replacing a 10 person team, who will get canned when SHTF so the 10 middle managers can jump ship to the next company with a padded resume. There are also 10 people because the middle managers got their friends positions from the savings due to the firing of the old engineers.

dbancajas 4y ago

What even is LLM?

sthustfo 4y ago
Large Language Model

marcell 4y ago

This post is way to bullish on ChatGPT for coding, and underestimates how big the gap is between ChatGPT and a mid level FANG eng. ChatGPT is very impressive in what it can do, but if you deviate problem statements even a little bit from what it's seen before, it completely fails. Just an example try asking it to write Battleship in Python. It does very well. But then, try changing the rules a bit, and adding some requirements. It spits out code that looks correct if you squint, but has terrible style, and often doesn't even work. Check out the test prompt [1]. Maybe GPT-4 is 1000x better at coding, but barring that, I don't see ChatGPT being useful as a replacement for human coders. [1] Prompt: Write a version of Battleship with modified rules for sinking a ship. In this version, a ship is sunk when the "head" and "tail" of the ship (its beginning and end parts) are hit. Hits to the center do not matter. Use object oriented programming, with classes for the Board and for Ships. There should be support for multiple ships and two board instances. Write it in Python.

alar44 4y ago
You're expecting way too much. Have it give you a list of functions needed to write battleship. Then have it write the classes. Then set the rules. Then have it write a GUI etc. You'll have a much more successful encounter that way.

la64710 4y ago

I beg to differ with the author’s perception ..in all it’s glory I don’t think L7 or principals or even software managers will have a reasonable chance of surviving. Because it is quite possible that agents built on LLM and AGI will be perfectly capable of not only arguing with each other and weigh on the pros and cons of any given option and arrive at a consensus just like the current Principals or managers do their negotiation, it is also quite possible they will be able to assign task and track the progress as well as take remediation action when needed. All these can be handled by advanced agents to a very large part. It is also possible to see teams of these AI agents working together in a cooperative way and surprising us with the outcome. Good luck !!

codekansas 4y ago
This sounds more like a sci-fi AI bot than the current route that AI research is going. How do you even get the training data or build a simulation for an agent like this...
- la64710 4y ago
  Just ask chatGPT anything you would ask a principal or a manager and you maybe surprised about how coherent the answer is. I don’t think the training data is that exotic and may be already there , the agents are continuously learning.
eouwt 4y ago
"LLM and AGI" Feels like the three letters A-G-I are doing a lot of heavy lifting there

larve 4y ago

Chatgpt has been pretty transformative to my programming, probably surpassing the only other thing coming close of really leveraging powerful ides like IntelliJ. It can generate all the scaffolding I need by just inputting a couple of sketches of Plantuml or even just markdown lists. I can build tools, unit tests, test data, fuzz harnesses, admin uis, dashboards, all within a few hours. I can transform a couple of words into very legible documentation. If you seed it with just a couple of lines of your own code and setup, it’s able to usually build stuff that matches the style of the codebase. It’s broken and wrong 80% of the time, but in a way that I can fix within a few minutes. This leaves me with 4h of high quality thinking and programming time instead of 1. It means I can try out 3 different approaches with full tooling instead of 1. It means I can build a whole terraform for aws and gcp and bare iron in 30 minutes. The fact that I can build solid tools and a full slurry of unit tests and end up with 4x time in my day is just blowing my mind. The quality of my software has tremendously gone up despite using something that mostly outputs broken code.

sbrother 4y ago
May I ask how you're getting ChatGPT to be reliable enough as a service to get these results? I had a similar experience the first week, but now most times I try it is overloaded and inaccessible, and when it's not the hourly rate limiter makes it difficult to be very productive.
- tester457 4y ago
  Using the api is better.
  - rlt 4y ago
    There's an API for ChatGPT? Or are you using the GPT-3 / Codex API?
- larve 4y ago
  I haven’t had too many issues. I must admit I use the api and copilot a lot.
4y ago

epicureanideal 4y ago

It's possible making coding easier will make salaries go up. https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox

codekansas 4y ago
I suspect we're about to enter a world where AI supercharges the average engineer's productivity, which is going to make it so that so much more stuff can be built. Using Copilot, the time to build a prototype for a simple app or API drops by a big multiple. Things that would take a week of work can be tried out in a weekend. Super cool future.

travisgriggs 4y ago

If I did search & replace on this article, replacing “programmer” with “manager”, would the conclusions and reasoning be much different (regardless of whether they’re spot on or not)?

btheshoe 4y ago

I think this makes assumptions about the demand for software engineers that seem unproven. Law salaries are bimodal because the demand for lawyers is limited, so there's only so many big law firms that can exist. In the near future, at least, it seems like demand for software engineers far outstrips supply and I think this will stay true in the long run because the market for software engineers is just different from the market for lawyers, in that software engineers can work on a multitude of different industries (faang covers personal devices, social media, entertainment, e-commerce, search, and more). I think the real concern is for when the rate of new startups decreases, because I view startups as basically the tech industry's way for searching for ways to increase the overall size of the pie, and startups slowing down means demand for software engineers will start to stagnate.

karmasimida 4y ago
> it seems like demand for software engineers far outstrips supply Definitely not the case when layoff right now is in full force. Also this doesn't necessarily indicate high salary anyway. There is a huge demand for fruit picker, but due to the nature of economy and margin of business, the salary will stay deflated thus the shortage persists. In a high interests, no free money environment, will software firms still able to demonstrate ability to generate return for investors? That would the ultimate key for whether the high salary is going to sustain or disappear. Regardless, I think this article has a point. We might be as well observing the end of the golden era for software engineers. LLM is a long-term salary oppressor, but in short term, simple dynamics of economy would test the market sufficiently.
- btheshoe 4y ago
  doomer take but tbh i respect it. but the central thesis of the tech industry is that it allows for products to be created with very low marginal cost which facilitates the high returns and if that thesis turns out to no longer be true then i would say that the identity of the tech industry has irrevocably changed.

jagtstronaut 4y ago

This is very doom and gloom compared to the actual landscape in law. Yeah entry level salaries suck in law, but plenty of decent lawyers who are nothing special end up billing 200 an hour 6 years into their career and taking their 40% of that. While outside of big law is not paying what it used to it is still an upper middle class profession if you stick it out.

readthenotes1 4y ago

Hasn't offshoring already done that to software? Unlike law, which is pretty local, the field is universal in software and most of the Asian, Eastern European, or South American programmers make nothing like north Americans. And programmers in Silicon Valley or Seattle make tons more than in Oklahoma.

karmasimida 4y ago

LLM might simply remove the need for junior engineers instead of requiring more of them. Thinking of instead of requiring 20 people team, now you can well function with like 10 people.

imtringued 4y ago

"Note that querying LLMs and making the outputted code conform to specifications is probably a lot easier than writing the code yourself" This overestimates the time developers spend typing text and underestimates the time spent thinking about problems, verification and locating bugs/code, etc. Just because the output of a model is text doesn't mean it is a replacement. You would need extraordinary "single shot" functionality where the LLM always outputs correct code and knows how to deal with any potential specification. The moment it fails then you wouldn't want "paralegal" developers, you would want normal, maybe inexperienced, developers. Single shot functionality is very impressive because humans can't do it, they build things incrementally, but once your program gets large, it can only be written incrementally, which at least implies that the AI needs to keep track of unbounded context.

nonethewiser 4y ago
The smaller the problem the easier it is to use AI. Which makes the window where it's useful very small. Its easier to understand and test code that you wrote than code you just read. If you ask it to write a 100k line program you're going to have to trace through everything and read it countless times over to fix any issues. If you ask it to write a small 10 line snippet that you need to test you might as well do it yourself.
- flir 4y ago
  It can write a ten line function faster than I can, and my code has a lot of ten line functions in it. Even for trivial code, it's a force multiplier (moreso when plumbed into an IDE)
uwuemu 4y ago
[dead]
aws_ls 4y ago
Only yesterday, I worked on a ticket for few hours, inspecting several Mongo collections in QE and prod. And then had to make 4 to 5 lines of changes across 3/4 python scripts. And then manually run them to patch the mongo collections. Which was the root cause of the bug. I don't see GPT doing this kind of stuff anytime soon. And yes, very less percentage of the time, do we right chunks of fresh code, in which this could help. But that again to an extent. It can generate functions. But the glue code is still complicated. So it can definitely be a better Stack Overflow replacement, perhaps.

bitwize 4y ago

A funny thing I noticed when googling whether melamine was microwave-safe: I got a lot of clickbaity articles from sketchy sites. Some said melamine is microwave safe, some said it is not, and some seemed to contradict each other. An Asimov story called "Liar!" concerns a robot which has the ability to read minds. When asked to tell what's going on in another person's mind (does so-and-so have a crush on me? is my boss planning to give me a promotion?) the robot, following its First Law directive not to allow humans to come to harm (including emotional harm), tells the querent what they would like to hear based on the robot's reading of the querent's mind, instead of the truth because the truth can hurt. When the robot is shown that its actions hurt humans anyway, it promptly shuts down and refuses to function. These language models seem like that robot. The articles on melamine suggest to me that the AIs which generated them are producing output that their statistical models suggest the querent wants to hear, rather than output with correct answers. In light of this, it seems that using these models for assistance in coding would be like having an obsequious but shitty coworker, whose output you must carefully review and vet because even money it's not a solution to your problem, just a guess at what you want to see based on a vast statistical soup built up of thousands of programmers' efforts on millions of tasks. Which means any effort you saved up now will be paid for with extra effort later, as every programmer knows it's much easier and more fun to wrote new code than to review and debug someone else's code, especially if it's shitty, especially^2 if it's subtly shitty. It really is like Teitelman's DWIM evolved, with correspondingly magnified lossage in its failure modes!

valdiorn 4y ago

Define your damn acronyms. I read the entire article and I STILL don't know what LLM means.

deleted 4y ago
[deleted]
bayesian_horse 4y ago
I feel you.. I think he means large language models.
- dbancajas 4y ago
  Basic mistake. Can't believe this was not corrected in OP. He should write title as "xxxx in Large Language Models (LLM)"

Dave3of5 4y ago

I wanted to see if what all these influencers were saying was true and so I tried to use ChatGPT to create a webapp the other day. The first problem is that it's totally overloaded at the moment if you use the app and so even just logging in is almost impossible. After 20 minutes I managed to get in. I first asked it to come up with 5 different ideas for the app. All of them were generic rubbish like a blog or suchlike so I asked it to be more niche and specific and it managed to throw up things like: * A Project management webapp * Farm management software * Todo List I picked the farm management software which I actually now interpreted as crop management software. This is a very competitive market lots of big software companies doing this already but I thought ok I'll give it a try. I asked it to build a basic landing page which is initially refused to do. When I did prompt it correctly it built a basic HTML only page with no styling and basically no content other than a header and a couple of fake testimonials. It then refused to style the thing at all and closed the chat. I then managed to prompt it correctly and it updated the font and stuck some stuff in boxes. During this process it told me several times that it will not create a landing page. I then gave up on the landing page and tried to get it to be able to help me write the code on the frontend to allow the user to define the crop areas on google maps. It spat out just the JS (no html) and which showed how to draw a polygon on the map basically the code you would get by looking up the google maps documentation. During this process the text reply was very slow like character by character and it kicked me out several times. I think your jobs are safe at the moment...

bayesian_horse 4y ago

It seems hard enough to find "IT people" who can plug cables in correctly or setup some software on a laptop (exaggerating, but only slightly). I'm not worried.

bartimus 4y ago

> we are perhaps converging to a future where a FAANG L7 can just sketch out architectural details and the programmer equivalent of paralegals will simply query the latest LLM and clean up the output. This is just proposing waterfall + outsourcing all over again. The author doesn't seem to understand that "sketching out architectural details" is the thing that developers do.

deleted 4y ago
[deleted]

sjducb 4y ago

Societies demand for software is basically infinite. Right now only people who can pay for developers get to have good custom software. As LLMs increase the output of a programmer more people will be able to comission custom software. More software will get made and programmer salaries will remain high.

maest 4y ago

From the article: > P.S. If your objection is that you played with ChatGPT and wasn’t impressed, you should try projecting trends beyond the t=1 month timescale. That is a dismissive stance announced in a very obnoxious way. I don't know where and how fast the AI thing will go but I do know they're treating an unknowable outcome as fact.