5 ms·

While I agree that domain expertise has always been a moat, I believe the author is missing something critical: there is a big difference between being able to

While I agree that domain expertise has always been a moat, I believe the author is missing something critical: there is a big difference between being able to verify the output of a system is correct, and being able to tell a system how to generate the correct output to begin with. Personal example: I had a software engineering colleague who was the best coder of financial management systems I've ever encountered. He gained these skills through years of in-the-trenches development. One of the things he told me, and that I also observed, was that the vast majority of financial experts (basically, the people in the accounting department of companies) had an extremely difficult time just telling him what the rules of any particular transaction should be. But what they could do was tell him whether the handling of any particular transaction was right or wrong. So often times he would sit down with these accounting folks and go through lots of example transactions he came up with, and from there he essentially built up the requirements spec. In my experience, that is the primary difference between people I've known who are good software engineers and those who aren't: people who can specify the detailed rules of any system, vs. folks who take a "well, I know it when I see it" approach. I have a strong suspicion that folks who have a high degree of domain expertise in a particular area will fail as software builders even in an agentic world because they will struggle to elucidate clearly the rules in their head that they've learned over years. As an analogy, it's kind of like asking a native speaker for the grammar rules of their language. Often times they can't, but they'll just say "well, that sounds wrong." They may be "domain experts" in their language, but they'd have a hell of a time prompting an AI system on how to grade a test for grammar correctness.

throwaway2037 5mo ago
I think that you raise some excellent points here. I want to dive deeper into what is meant by domain knowledge -- as a I see it. This phrase comes to mind: "Those who know, do. Those that understand, teach." (Google search tells me that it is frequently misattributed to Aristotle.) People who cannot clearly articulate rules for their domain only "do". However, those that can clearly articulate will "teach". Thus, I would say true domain knowledge is the ability to teach that domain to others.
- layer8 5mo ago
  I largely agree, but want to note that there’s the counterpoint “Those who can, do; those who can’t, teach” (from George Bernard Shaw’s 1903 play Man and Superman), which also has some nugget of truth in it.
  - rightbyte 5mo ago
    Case on point is Youtube where a lot of almost good amateurs give a lot of advice on niche channels on things they hardly master.
starfallg 5mo ago
I guess the flip side to that is the iterative conversation to develop these specifications is exactly what AI chatbots are good at. It's also the main reason why very structured AI agent orchestration for software engineering modelled on rigid processes fails to really provide much value.
- layer8 5mo ago
  I suspect that the reason AI chatbots are not very good at developing such specifications is that they don’t build a domain model in their “head” — the only models they have are already hardcoded in their weights. A human learning a domain, on the other hand, forms new neural connections representing the model of the domain that they build in their head.
  - starfallg 5mo ago
    Chatbots are good at helping humans arrive at the specification. Apart from people that has deep expertise and/or talent in this area, humans aren't very good at building these specifications. That's the topic of this exchange.
    - layer8 5mo ago
      I disagree, unless it’s about a topic the LLM already knows or can research based on public information, or unless the domain expert can actually judge the specification and have it iteratively corrected by the LLM until they are satisfied. That’s generally not the case, and LLMs aren’t good at controlling the iteration themselves until they are satisfied, with the result being a complete and accurate specification for the domain. If LLMs were good at that, then for coding we wouldn’t need to precisely specify what we want them to code, instead they would ask us until it’s crystal clear what precisely we want in all details.
elendilm 5mo ago
Yes. The author stumbled on the p vs np problem but with humans. Verification is easy. Solving is the millennium prize question.
- Npovview 5mo ago
  Agreed. But there is a new dynamic coming to place. Who can execute faster now that we have AIs? I think the combination of (Good SWE using AIs) collaborating with (Domain Expert using AIs) is the winning team. Things are accelerating on all fronts, the frontier may be jagged but AIs is making progress on all capabilites.
  - dapperdrake 5mo ago
    Additional iterations are valuable, if and only if we learn from additional iterations. Otherwise we are just spinning our wheels.
- __MatrixMan__ 5mo ago
  I agree that it has some resemblance, but the striking thing about NP-complete problems is that an efficient solution to any of them is an efficient solution to all of them, which makes it worth trying exceptionally hard to find one. It could be that whatever lackluster expertise you can squeeze out of an LLM is good enough to discourage investment in the real thing since unlike NP-complete problems, expertise isn't generalizable.
Toutouxc 5mo ago
I agree with your point that people “from” the domain aren’t automatically equipped to start AI-building software in the domain, exactly because they often lack one of the most crucial software development skills — being able to (and having the desire to) describe a complex system with a finite set of deterministic rules. But I don’t think we should be calling these people “domain experts”. I think we should reserve that name for the other group, for the people who truly and deeply understand the domain, the whys and whats and why nots.
- eloisant 5mo ago
  You're doing a bit of a "true Scotsman fallacy" here.
  - Toutouxc 5mo ago
    It feels like without a bit of true Scotsmanship, the term “domain expert” doesn’t really mean anything.
    - spacebacon 5mo ago
      Sure it does, it’s just a highly contested meaning that depends on a proper contemporary interpretation if to be received as intended in this context. Context that varies domain to domain. For the domain name buyer it’s quite different than the metacognitive scholar.
- RussianCow 5mo ago
  That ship has sailed. Anyone who works in a field is now considered a domain expert (or "SME") even if they're nothing of the sort. There should ideally be another term that's a superset, but I doubt it would ever catch on.
- watwut 5mo ago
  People you say are not experts are fully capable to explain whys and whats and why nots. In more details that you can absorb. They will do that on examples tho. They will recognize and explain every single exception they see - but they are not able to list all exceptions up front. Because, that is highly unusual task
vmurthy 5mo ago
>One of the things he told me, and that I also observed, was that the vast majority of financial experts (basically, the people in the accounting department of companies) had an extremely difficult time just telling him what the rules of any particular transaction should be We have internalized more knowledge than we can explain sounds like the textbook definition of Polanyi's paradox:"Polanyi's paradox, named in honour of the British-Hungarian philosopher Michael Polanyi, is the theory that human knowledge of how the world functions and of our own capability are, to a large extent, beyond our explicit understanding" [0] [0] https://en.wikipedia.org/wiki/Polanyi%27s_paradox https://en.wikipedia.org/wiki/Polanyi%27s_paradox
- hn_throwaway_99 5mo ago
  Thanks so much! I hadn't heard of Polanyi's paradox before but it is exactly what I was talking about. The Wikipedia entry even highlights the problem for AI-driven development: > Polanyi's paradox has been widely considered to identify a major obstacle in the fields of AI and automation, since programming an automated task or system is difficult unless a complete and fully specific description of the procedure is available.
  - rcxdude 5mo ago
    I'd say the opposite in some ways: it's arguably the reason for the bitter lesson and ML techniques tending to beat other approaches, because they work by giving examples as opposed to trying to articulate the rules in an explicit manner.
- formerly_proven 5mo ago
  cf https://en.wikipedia.org/wiki/Dual_process_theory https://en.wikipedia.org/wiki/Dual_process_theory as a causative agent behind the paradox.
- reemyoass 5mo ago
  having worked on automation of accounting process, it has been a combination of ever evolving ruleset/edge scenarios for 10% cases plus standardised rules for 90% of the cases. i think the problem is that it is an evolving field. alot of times it borders law, which makes it murky and vague too.
boringperson 5mo ago
> I have a strong suspicion that folks who have a high degree of domain expertise in a particular area will fail as software builders even in an agentic world because they will struggle to elucidate clearly the rules in their head that they've learned over years. Maybe they need a build a hybrid expertise of "domain" and "software engineering". For example, robotic surgery requires expert surgeons to build sufficient expertise in robotics Also, noticed a pretty high karma for a throwaway account.
baxtr 5mo ago
Some companies use product managers for this purpose. They translate the implicit domain knowledge of those experts into explicit requirements, mostly through interviewing.
marbro 5mo ago
This is the exasperating part about learning to speak Spanish using a textbook; you must guess the grammar rules because the textbook won't tell you. So, you use the English rule and hope and pray that it is the same in Spanish, and you'll be right the majority of the time but often wrong. Spanish textbooks written 100 years ago tell you the grammar rules and are more useful than recent textbooks.
- layer8 5mo ago
  While interesting, I rarely found knowing grammar rules beyond some very basic ones to help all that much for learning to speak a language, compared to language exposure and speaking practice.
  - yencabulator 5mo ago
    I would be very surprised if there was even a single natural language with a complete grammar that actually describes the spoken language. It's all just conventions. Lots of people confuse common patterns with rules.
- LEDThereBeLight 5mo ago
  I’ve noticed this too. Any idea why modern language books get this so wrong?
thibran 5mo ago
You described reasoning by analogy, which two thirds of the population does.
- gota 5mo ago
  Please explain explicitly what you are implying, because I don't get it Who are the 1/3 of the population that does not reason by analogy?
  - thibran 5mo ago
    Instead of needing a familiar metaphor to understand a new system (reasoning by analogy), they grasp the core conceptual mechanics directly. It’s not that the 2/3 can't think abstractly, it’s just that they use analogies as a temporary processing step to get there, while the 1/3 starts there.
  - albedoa 5mo ago
    The parent completely made up a statistic and then confidently restated it when pressed lol.
derfurth 5mo ago
Having worked with domain experts I concur on the difficult time they have expressing the rules of their domains. Once I built a little domain-specific language for them, that was tested against old jobs to see if they contradicted the past; it was a nifty project and since then I am convinced that DSLs are underrated as a way to encode expertise.
- Dansvidania 5mo ago
  Can you give an example (even if contrived) of how that would have looked ? I’m very curious !
  - derfurth 5mo ago
    I had the experts write markdown files that contained the rules looked somewhat like: ## 1A Rule name Some prose explaining the rules liking to official documentation. ``` if municipality and inhabitants > 10000 then functionA else functionB ``` Then a trivial parser would extract the rules, the DSL was then handled by Lark[1]. So pretty simple, but it made collaborating with experts easier as simulated results would also output some markdown they could read. 1. https://lark-parser.readthedocs.io/en/stable/ https://lark-parser.readthedocs.io/en/stable/
    - tardedmeme 5mo ago
      Sounds like "Literate Programming", where the code and comments are reversed: instead of everything being code except what's marked as a comment, everything is a comment except what's marked as code.
  - BarkenBark 5mo ago
    I am also very interested!
- renegade-otter 5mo ago
  Scala to the rescue, then! :)
codesnik 5mo ago
I had the same problem in air travel industry. Our ticketing officers were never able to communicate how the ticket commission rules from our partners should be interpreted. I ended up adding a test framework, that allowed to add examples and counterexamples of existing flights to each rule, to verify that they are applied the way they'd assign commission by hand.
zipy124 5mo ago
I fully agree here. It's also not uncommon to have the reply for specific items be "Ahh, well that is a bit of an edge case see because of X and Y" and then you learn there are many many "special cases" to the point that they aren't that special anymore.
- RaftPeople 5mo ago
  This comment brings back memories of a project. We were working through many complex business flows and running into exactly what you describe in many areas. Some of the project people with less experience complained that we were spending too much time on exceptions and slowing down the project. We had to explain that when each exception happens 100 times a day with significant impact on business productivity, then it doesn't matter whether you call it an exception or not, it's important to solve for.
deleted 5mo ago
[deleted]
xorcist 5mo ago
Anybody who has ever done programming professionally in the small scale knows this. Refining the requirements is the job. In fact, I've never known an industry so keen on levelling its own moats as the software industry. We regularly invent things like 4GL, graphical programming and frameworks and engines such as Unity just to enable more people to do programming. People will happily teach programming for free in outreach programs (I am just one example). No other profession does this. Perhaps with the exception of mathematicians and other fields that are very close to programming. I could teach an economist enough programming in a weeks that they could write an ERP module, but I could not learn enough economics in a week to write one. If the language was the barrier, we could invent a more effective one. We invent new languages weekly anyway. Having seen how quickly beginners can make things with Unity or Godot, I seriously doubt an LLM agent could improve much on this. Of course, if the job is writing yet another CRUD React app with a Java or Python backend, then sure, the LLM will be very effective. But compared to doing same app in something like Excel or MS Access? Not that much.
- teleforce 5mo ago
  >We regularly invent things like 4GL, graphical programming and frameworks and engines such as Unity just to enable more people to do programming. You're right on the money on this. Earlier this month I went to visit a company for a complete demo prototype of a full one-to-one train simulator trainer mostly designed and programmed by a former civil engineer using Unity engine. According to the company, they could not do it if Unity engine (or similar) is not around because it will be prohibitively expensive to develop. In a related news, Unity recently released AI eco-system namely Unity AI Suite [1]. [1] Unity AI Suite: https://unity.com/features/ai https://unity.com/features/ai
- dzonga 5mo ago
  the 1 dimensional thinking that's perverse in the industry will be catastrophic. we don't like doing the hard things e.g training juniors so they can be skilled seniors via good apprenticeship programs i.e on the job. now we r delegating to stochastic parrots. in terms of systems thinking which is one skill you need to be a domain expertise - very few people are ever curious & are not willing or able to ask critical questions. hence the groupthink that's prevalent in the industry. no wonder the quality of software never goes up - while the building blocks have gone up in quality. an analogy is like having super strong bricks but making brittle structures
yoda7marinated 5mo ago
This is basically the gap BDD tries to close I guess. It assumes that domain experts can't fully articulate the rules upfront.
xnx 5mo ago
Are we thinking about the wrong side here? Wouldn't the LLM be a super-intelligent financial expert as well as a super-intelligent coder?
- Hendrikto 5mo ago
  It is neither.
miki123211 5mo ago
> I have a strong suspicion that folks who have a high degree of domain expertise in a particular area will fail as software builders even in an agentic world because they will struggle to elucidate clearly the rules in their head that they've learned over years. This is the part I disagree with. In a non-agentic world, the expert and the programmer are two different people. If the expert finds a bug in the software, they have to describe that bug, send it over to a programmer to fix, wait for a new release and until that scenario occurs again, realize that their description was actually wrong, send a corrected description, rinse and repeat for a few iterations until the bug is finally fixed for good. In a world of agents, the expert finds the bug, asks the agent to fix it, realizes that the fix is incorrect because of sloppy thinking, does a few iterations until the feature works correctly, and that's it. Bug fixed; with 10 minutes of work instead of a few weeks.
- Hendrikto 5mo ago
  > In a world of agents, the expert finds the bug, asks the agent to fix it, realizes that the fix is incorrect because of sloppy thinking, does a few iterations until the feature works correctly That would require clearly (a) knowing what you want, and (b) expressing it unambiguously and in detail, including all edge cases. Essentially producing a spec. Most people are not able to do either. Talking to an LLM does not change that.
  - j16sdiz 5mo ago
    but.. they can iterate real quick! As long as the expert don't run of patience, he may be able to do that. Consider finance sector -- the industry is powered by excel spreadsheets made by non-programmer having no clue how programming works.
    - skydhash 5mo ago
      Spreadsheet is a DSL for finance and a lot of other sectors. But the lack of formal logic capabilities shows when you seen the spreadsheet organization. Most only stand with ducktape all over the place.
andsoitis 5mo ago
> So often times he would sit down with these accounting folks and go through lots of example transactions he came up with, and from there he essentially built up the requirements spec. Right, so the spec is derived from examples, an interactive process that doesn’t require a programmer.
- skydhash 5mo ago
  The programmer skill is how to abstract the specs from all the examples. And then to formalize it. Actual coding is merely translation. And only beginners tend to focus on that.
  - petra 5mo ago
    Sounds very similar to what AI is good at.
    - skydhash 5mo ago
      May work for simple cases, but how would you come up with something like typst, or thunderbird?
    - deleted 5mo ago
      [deleted]
    - wtetzner 5mo ago
      I've yet to see AI be good at extracting good abstractions. More often than not it jumps to creating a battery of special case checks.
    - dapperdrake 5mo ago
      No. AI sucks at formal languages and implications. Lossy decompression probably won’t ever be really good at this. If it’s missing from the data, then it won’t be there. The AI math proofs were probably already out there in tiny pieces and nobody got all of the pieces together. That is valuable. It’s different from making a piece that is missing. And the AI will just hallucinate.
- 5mo ago
deleted 5mo ago
[deleted]
michaelbuckbee 5mo ago
WRT the native grammar, consider adjective order. Few native english speakers (me included) can off the cuff name the proper order, but everyone knows the "right" order. https://dictionary.cambridge.org/us/grammar/british-grammar/adjectives-order https://dictionary.cambridge.org/us/grammar/british-grammar/...
- chrisweekly 5mo ago
  Great example
- mejutoco 5mo ago
  Another one: tekamolo, for German and adverbs this time. Temporal, kausal, Modal, Lokal https://www.olesentuition.co.uk/single-post/tekamolo-how-to-structure-german-sentences https://www.olesentuition.co.uk/single-post/tekamolo-how-to-...
- dapperdrake 5mo ago
  Excellent example.
hliyan 5mo ago
I think the answer is in the second sentence of the article. The most valuable person in this new world (using the author's own phrasing from the penultimate paragraph) is the one who is good at "building a working model of the domain in [their] head". Depending on the domain, this may either be the domain expert themselves, or someone else trained in formal logic, data structures and organizing information into coherent hierarchies. If this is neither the domain expert nor the programmer, then it must be someone in between. In the old world, this role was called "business analyst".
- Mezzie 5mo ago
  I'm this person, and I do find AI to be quite helpful, though I'm mostly just playing around at this point. I'm the daughter and granddaughter of programmers, and I learned the basics of how to code as a kid. I'm good at it and have a knack for it, but I didn't want to do it for 8+ hours a day and then spend my nights on it as well, so I didn't pursue it as a career. I did an undergraduate degree in Linguistics, which has been really helpful for having an intuitive sense for what 'language as data' can accomplish and for a strong understanding of the difference between language as data and language as meaning. I studied formal logic systems. Then I did a graduate degree in Library Science and worked in libraries for a decade and a half. I can organize and define systems very well, and I'm trained in how to wheedle information people don't consciously know out of them without them knowing I'm doing it. I've spent enough time around actual devs to understand where my limitations are and when to loop in someone who knows more to check my work, and when it's important for the work to be super accurate versus when I can learn by fucking around. (Front end and design? Fuck around! Database structure? Fuck around but with an exceptionally robust backup system kept outside of the AI tools' purview + don't fuck around in prod. Storing credentials and people's information? Ask someone.) The problem companies are going to have is I'm very disinclined to work for them doing this, particularly if they want us because they think we're going to be cheaper. Most people who are in this category a.) could be devs and opted not to, and there's a reason for that and/or b.) are the children, cousins, etc. of programmers. We're not stupid: we know we're just as disposable as they're trying to make devs.
  - abalashov 5mo ago
hbwang2076 5mo ago
Domain expertise matters more than most bootcamp grads want to hear. Knowing how to phrase a query for a specific domain is half the battle.
theptip 5mo ago
I think this neatly captures the gap in the article; it’s thinking of domain expertise as static, rather than something that needs to be built/extracted. It also goes the other way; A a good engineer can build experiments and discover the domain even without an oracle on the team; reading protocols or manuals or specs, for example. In other words, learn to be a domain expert themselves. Personally I find the most efficient way to learn a concept to be building it; you are immediately faced with your blind spots. AI for sure helps improve cycle time on these discoveries if you use it that way; time writing boilerplate goes to ~zero and you can spend your time figuring out how to answer the real questions. I do think that we as a profession need to learn new architectural and craftsmanship patterns to make it easier to learn from our code; concepts like Literate Programming which were too fussy for fast-moving teams may end up accelerating in the agentic world; I need to read a lot more code than I write now. But also things like Acceptance Test Driven Design; not new concepts, just newly increased in value.
nonethewiser 5mo ago
> But what they could do was tell him whether the handling of any particular transaction was right or wrong. So often times he would sit down with these accounting folks and go through lots of example transactions he came up with, and from there he essentially built up the requirements spec. This combination of requirements is the business in most domains. Software or otherwise. Codefying different rules and executing them.
physicsguy 5mo ago
I have found this with (mechanical) engineers. They know what they want to see but don’t understand the underlying details which are more mathematical than the average engineer is able to work with. So the people working on engineering software are often physicists or applied mathematicians.
- dapperdrake 5mo ago
  Ran into this, as well. And they were really annoyed at being asked math questions.
dapperdrake 5mo ago
That is why coders who learn problem domains are where the money is at. Works for me. Already learned two or three problem domains well enough and now it is starting to compound.
LorenPechtel 5mo ago
I very much agree here--the true job of us programmers is determining exactly what the problem is. The domain expert will fail because they haven't attempted to refine the details enough, they probably don't have the skill to anticipate the edge cases. I've spent most of my career working with the same people--even people who have some coding experience but they never catch the low probability stuff. Or even not so low probability: business procedures were that the customer was quoted an all-inclusive price. Try to work backwards, what is the pre-tax amount? Um, that's not guaranteed to have a solution. My concerns get ignored, I go ahead and implement with code that will catch the offending penny and label it roundoff correction. While I'm not sure what happened I think some auditor hit that. I ended up having to walk them through the calculation, okay, what's the answer here? Only when they couldn't solve the problem could they accept that what I had done was the only answer. (The actual odds of such a failure are equal to the tax rate. I had not originally worked that out, just saw success was not guaranteed.) The domain experts make this sort of mistake routinely in dealing with code. AI won't fix that.
yencabulator 5mo ago
> there is a big difference between being able to verify the output of a system is correct, and being able to tell a system how to generate the correct output to begin with. There are attempts to avoid needing to explicitly construct the rules. This is often called "Programming by example". https://en.wikipedia.org/wiki/Programming_by_example https://en.wikipedia.org/wiki/Programming_by_example On some level, even training an LLM to answer questions is this. I do like my determinism, though. Vibecoding with human-managed acceptance is a very current-moment way of doing this. Just make sure the agent has to program within a framework, not changing it, and the results are actually pretty good. One of the interesting things one can do with this mentality, if you set up your system well, is to reevaluate prior decisions with new rules, and decide whether any differing decisions are corrections for prior bad decisions, or newly-introduced bugs.
tim-projects 5mo ago
> they'd have a hell of a time prompting an AI system on how to grade a test for grammar correctness. This isn't true. They can simply upload a whole bunch of completed and marked tests and say show me the rules. And follow that up with 'write me a prompt'
abishr12 5mo ago
what people miss is that software engineering is its own language, even the finance expert can code but doesn't mean they should.

throwaway2037 5mo ago

I think that you raise some excellent points here. I want to dive deeper into what is meant by domain knowledge -- as a I see it. This phrase comes to mind: "Those who know, do. Those that understand, teach." (Google search tells me that it is frequently misattributed to Aristotle.) People who cannot clearly articulate rules for their domain only "do". However, those that can clearly articulate will "teach". Thus, I would say true domain knowledge is the ability to teach that domain to others.

layer8 5mo ago
I largely agree, but want to note that there’s the counterpoint “Those who can, do; those who can’t, teach” (from George Bernard Shaw’s 1903 play Man and Superman), which also has some nugget of truth in it.
- rightbyte 5mo ago
  Case on point is Youtube where a lot of almost good amateurs give a lot of advice on niche channels on things they hardly master.

starfallg 5mo ago

I guess the flip side to that is the iterative conversation to develop these specifications is exactly what AI chatbots are good at. It's also the main reason why very structured AI agent orchestration for software engineering modelled on rigid processes fails to really provide much value.

layer8 5mo ago
I suspect that the reason AI chatbots are not very good at developing such specifications is that they don’t build a domain model in their “head” — the only models they have are already hardcoded in their weights. A human learning a domain, on the other hand, forms new neural connections representing the model of the domain that they build in their head.
- starfallg 5mo ago
  Chatbots are good at helping humans arrive at the specification. Apart from people that has deep expertise and/or talent in this area, humans aren't very good at building these specifications. That's the topic of this exchange.
  - layer8 5mo ago
    I disagree, unless it’s about a topic the LLM already knows or can research based on public information, or unless the domain expert can actually judge the specification and have it iteratively corrected by the LLM until they are satisfied. That’s generally not the case, and LLMs aren’t good at controlling the iteration themselves until they are satisfied, with the result being a complete and accurate specification for the domain. If LLMs were good at that, then for coding we wouldn’t need to precisely specify what we want them to code, instead they would ask us until it’s crystal clear what precisely we want in all details.

elendilm 5mo ago

Yes. The author stumbled on the p vs np problem but with humans. Verification is easy. Solving is the millennium prize question.

Npovview 5mo ago
Agreed. But there is a new dynamic coming to place. Who can execute faster now that we have AIs? I think the combination of (Good SWE using AIs) collaborating with (Domain Expert using AIs) is the winning team. Things are accelerating on all fronts, the frontier may be jagged but AIs is making progress on all capabilites.
- dapperdrake 5mo ago
  Additional iterations are valuable, if and only if we learn from additional iterations. Otherwise we are just spinning our wheels.
__MatrixMan__ 5mo ago
I agree that it has some resemblance, but the striking thing about NP-complete problems is that an efficient solution to any of them is an efficient solution to all of them, which makes it worth trying exceptionally hard to find one. It could be that whatever lackluster expertise you can squeeze out of an LLM is good enough to discourage investment in the real thing since unlike NP-complete problems, expertise isn't generalizable.

Toutouxc 5mo ago

I agree with your point that people “from” the domain aren’t automatically equipped to start AI-building software in the domain, exactly because they often lack one of the most crucial software development skills — being able to (and having the desire to) describe a complex system with a finite set of deterministic rules. But I don’t think we should be calling these people “domain experts”. I think we should reserve that name for the other group, for the people who truly and deeply understand the domain, the whys and whats and why nots.

eloisant 5mo ago
You're doing a bit of a "true Scotsman fallacy" here.
- Toutouxc 5mo ago
  It feels like without a bit of true Scotsmanship, the term “domain expert” doesn’t really mean anything.
  - spacebacon 5mo ago
    Sure it does, it’s just a highly contested meaning that depends on a proper contemporary interpretation if to be received as intended in this context. Context that varies domain to domain. For the domain name buyer it’s quite different than the metacognitive scholar.
RussianCow 5mo ago
That ship has sailed. Anyone who works in a field is now considered a domain expert (or "SME") even if they're nothing of the sort. There should ideally be another term that's a superset, but I doubt it would ever catch on.
watwut 5mo ago
People you say are not experts are fully capable to explain whys and whats and why nots. In more details that you can absorb. They will do that on examples tho. They will recognize and explain every single exception they see - but they are not able to list all exceptions up front. Because, that is highly unusual task

vmurthy 5mo ago

>One of the things he told me, and that I also observed, was that the vast majority of financial experts (basically, the people in the accounting department of companies) had an extremely difficult time just telling him what the rules of any particular transaction should be We have internalized more knowledge than we can explain sounds like the textbook definition of Polanyi's paradox:"Polanyi's paradox, named in honour of the British-Hungarian philosopher Michael Polanyi, is the theory that human knowledge of how the world functions and of our own capability are, to a large extent, beyond our explicit understanding" [0] [0] https://en.wikipedia.org/wiki/Polanyi%27s_paradox https://en.wikipedia.org/wiki/Polanyi%27s_paradox

hn_throwaway_99 5mo ago
Thanks so much! I hadn't heard of Polanyi's paradox before but it is exactly what I was talking about. The Wikipedia entry even highlights the problem for AI-driven development: > Polanyi's paradox has been widely considered to identify a major obstacle in the fields of AI and automation, since programming an automated task or system is difficult unless a complete and fully specific description of the procedure is available.
- rcxdude 5mo ago
  I'd say the opposite in some ways: it's arguably the reason for the bitter lesson and ML techniques tending to beat other approaches, because they work by giving examples as opposed to trying to articulate the rules in an explicit manner.
formerly_proven 5mo ago
cf https://en.wikipedia.org/wiki/Dual_process_theory https://en.wikipedia.org/wiki/Dual_process_theory as a causative agent behind the paradox.
reemyoass 5mo ago
having worked on automation of accounting process, it has been a combination of ever evolving ruleset/edge scenarios for 10% cases plus standardised rules for 90% of the cases. i think the problem is that it is an evolving field. alot of times it borders law, which makes it murky and vague too.

boringperson 5mo ago

baxtr 5mo ago

Some companies use product managers for this purpose. They translate the implicit domain knowledge of those experts into explicit requirements, mostly through interviewing.

marbro 5mo ago

This is the exasperating part about learning to speak Spanish using a textbook; you must guess the grammar rules because the textbook won't tell you. So, you use the English rule and hope and pray that it is the same in Spanish, and you'll be right the majority of the time but often wrong. Spanish textbooks written 100 years ago tell you the grammar rules and are more useful than recent textbooks.

layer8 5mo ago
While interesting, I rarely found knowing grammar rules beyond some very basic ones to help all that much for learning to speak a language, compared to language exposure and speaking practice.
- yencabulator 5mo ago
  I would be very surprised if there was even a single natural language with a complete grammar that actually describes the spoken language. It's all just conventions. Lots of people confuse common patterns with rules.
LEDThereBeLight 5mo ago
I’ve noticed this too. Any idea why modern language books get this so wrong?

thibran 5mo ago

You described reasoning by analogy, which two thirds of the population does.

gota 5mo ago
Please explain explicitly what you are implying, because I don't get it Who are the 1/3 of the population that does not reason by analogy?
- thibran 5mo ago
  Instead of needing a familiar metaphor to understand a new system (reasoning by analogy), they grasp the core conceptual mechanics directly. It’s not that the 2/3 can't think abstractly, it’s just that they use analogies as a temporary processing step to get there, while the 1/3 starts there.
- albedoa 5mo ago
  The parent completely made up a statistic and then confidently restated it when pressed lol.

derfurth 5mo ago

Having worked with domain experts I concur on the difficult time they have expressing the rules of their domains. Once I built a little domain-specific language for them, that was tested against old jobs to see if they contradicted the past; it was a nifty project and since then I am convinced that DSLs are underrated as a way to encode expertise.

Dansvidania 5mo ago
Can you give an example (even if contrived) of how that would have looked ? I’m very curious !
- derfurth 5mo ago
  I had the experts write markdown files that contained the rules looked somewhat like: ## 1A Rule name Some prose explaining the rules liking to official documentation. ``` if municipality and inhabitants > 10000 then functionA else functionB ``` Then a trivial parser would extract the rules, the DSL was then handled by Lark[1]. So pretty simple, but it made collaborating with experts easier as simulated results would also output some markdown they could read. 1. https://lark-parser.readthedocs.io/en/stable/ https://lark-parser.readthedocs.io/en/stable/
  - tardedmeme 5mo ago
    Sounds like "Literate Programming", where the code and comments are reversed: instead of everything being code except what's marked as a comment, everything is a comment except what's marked as code.
- BarkenBark 5mo ago
  I am also very interested!
renegade-otter 5mo ago
Scala to the rescue, then! :)

codesnik 5mo ago

I had the same problem in air travel industry. Our ticketing officers were never able to communicate how the ticket commission rules from our partners should be interpreted. I ended up adding a test framework, that allowed to add examples and counterexamples of existing flights to each rule, to verify that they are applied the way they'd assign commission by hand.

zipy124 5mo ago

I fully agree here. It's also not uncommon to have the reply for specific items be "Ahh, well that is a bit of an edge case see because of X and Y" and then you learn there are many many "special cases" to the point that they aren't that special anymore.

RaftPeople 5mo ago
This comment brings back memories of a project. We were working through many complex business flows and running into exactly what you describe in many areas. Some of the project people with less experience complained that we were spending too much time on exceptions and slowing down the project. We had to explain that when each exception happens 100 times a day with significant impact on business productivity, then it doesn't matter whether you call it an exception or not, it's important to solve for.

deleted 5mo ago

[deleted]

xorcist 5mo ago

Anybody who has ever done programming professionally in the small scale knows this. Refining the requirements is the job. In fact, I've never known an industry so keen on levelling its own moats as the software industry. We regularly invent things like 4GL, graphical programming and frameworks and engines such as Unity just to enable more people to do programming. People will happily teach programming for free in outreach programs (I am just one example). No other profession does this. Perhaps with the exception of mathematicians and other fields that are very close to programming. I could teach an economist enough programming in a weeks that they could write an ERP module, but I could not learn enough economics in a week to write one. If the language was the barrier, we could invent a more effective one. We invent new languages weekly anyway. Having seen how quickly beginners can make things with Unity or Godot, I seriously doubt an LLM agent could improve much on this. Of course, if the job is writing yet another CRUD React app with a Java or Python backend, then sure, the LLM will be very effective. But compared to doing same app in something like Excel or MS Access? Not that much.

teleforce 5mo ago
>We regularly invent things like 4GL, graphical programming and frameworks and engines such as Unity just to enable more people to do programming. You're right on the money on this. Earlier this month I went to visit a company for a complete demo prototype of a full one-to-one train simulator trainer mostly designed and programmed by a former civil engineer using Unity engine. According to the company, they could not do it if Unity engine (or similar) is not around because it will be prohibitively expensive to develop. In a related news, Unity recently released AI eco-system namely Unity AI Suite [1]. [1] Unity AI Suite: https://unity.com/features/ai https://unity.com/features/ai
dzonga 5mo ago
the 1 dimensional thinking that's perverse in the industry will be catastrophic. we don't like doing the hard things e.g training juniors so they can be skilled seniors via good apprenticeship programs i.e on the job. now we r delegating to stochastic parrots. in terms of systems thinking which is one skill you need to be a domain expertise - very few people are ever curious & are not willing or able to ask critical questions. hence the groupthink that's prevalent in the industry. no wonder the quality of software never goes up - while the building blocks have gone up in quality. an analogy is like having super strong bricks but making brittle structures

yoda7marinated 5mo ago

This is basically the gap BDD tries to close I guess. It assumes that domain experts can't fully articulate the rules upfront.

xnx 5mo ago

Are we thinking about the wrong side here? Wouldn't the LLM be a super-intelligent financial expert as well as a super-intelligent coder?

Hendrikto 5mo ago
It is neither.

miki123211 5mo ago

> I have a strong suspicion that folks who have a high degree of domain expertise in a particular area will fail as software builders even in an agentic world because they will struggle to elucidate clearly the rules in their head that they've learned over years. This is the part I disagree with. In a non-agentic world, the expert and the programmer are two different people. If the expert finds a bug in the software, they have to describe that bug, send it over to a programmer to fix, wait for a new release and until that scenario occurs again, realize that their description was actually wrong, send a corrected description, rinse and repeat for a few iterations until the bug is finally fixed for good. In a world of agents, the expert finds the bug, asks the agent to fix it, realizes that the fix is incorrect because of sloppy thinking, does a few iterations until the feature works correctly, and that's it. Bug fixed; with 10 minutes of work instead of a few weeks.

Hendrikto 5mo ago
> In a world of agents, the expert finds the bug, asks the agent to fix it, realizes that the fix is incorrect because of sloppy thinking, does a few iterations until the feature works correctly That would require clearly (a) knowing what you want, and (b) expressing it unambiguously and in detail, including all edge cases. Essentially producing a spec. Most people are not able to do either. Talking to an LLM does not change that.
- j16sdiz 5mo ago
  but.. they can iterate real quick! As long as the expert don't run of patience, he may be able to do that. Consider finance sector -- the industry is powered by excel spreadsheets made by non-programmer having no clue how programming works.
  - skydhash 5mo ago
    Spreadsheet is a DSL for finance and a lot of other sectors. But the lack of formal logic capabilities shows when you seen the spreadsheet organization. Most only stand with ducktape all over the place.

andsoitis 5mo ago

> So often times he would sit down with these accounting folks and go through lots of example transactions he came up with, and from there he essentially built up the requirements spec. Right, so the spec is derived from examples, an interactive process that doesn’t require a programmer.

skydhash 5mo ago
The programmer skill is how to abstract the specs from all the examples. And then to formalize it. Actual coding is merely translation. And only beginners tend to focus on that.
- petra 5mo ago
  Sounds very similar to what AI is good at.
  - skydhash 5mo ago
    May work for simple cases, but how would you come up with something like typst, or thunderbird?
  - deleted 5mo ago
    [deleted]
  - wtetzner 5mo ago
    I've yet to see AI be good at extracting good abstractions. More often than not it jumps to creating a battery of special case checks.
  - dapperdrake 5mo ago
    No. AI sucks at formal languages and implications. Lossy decompression probably won’t ever be really good at this. If it’s missing from the data, then it won’t be there. The AI math proofs were probably already out there in tiny pieces and nobody got all of the pieces together. That is valuable. It’s different from making a piece that is missing. And the AI will just hallucinate.
5mo ago

deleted 5mo ago

[deleted]

michaelbuckbee 5mo ago

WRT the native grammar, consider adjective order. Few native english speakers (me included) can off the cuff name the proper order, but everyone knows the "right" order. https://dictionary.cambridge.org/us/grammar/british-grammar/adjectives-order https://dictionary.cambridge.org/us/grammar/british-grammar/...

chrisweekly 5mo ago
Great example
mejutoco 5mo ago
Another one: tekamolo, for German and adverbs this time. Temporal, kausal, Modal, Lokal https://www.olesentuition.co.uk/single-post/tekamolo-how-to-structure-german-sentences https://www.olesentuition.co.uk/single-post/tekamolo-how-to-...
dapperdrake 5mo ago
Excellent example.

hliyan 5mo ago

I think the answer is in the second sentence of the article. The most valuable person in this new world (using the author's own phrasing from the penultimate paragraph) is the one who is good at "building a working model of the domain in [their] head". Depending on the domain, this may either be the domain expert themselves, or someone else trained in formal logic, data structures and organizing information into coherent hierarchies. If this is neither the domain expert nor the programmer, then it must be someone in between. In the old world, this role was called "business analyst".

Mezzie 5mo ago
I'm this person, and I do find AI to be quite helpful, though I'm mostly just playing around at this point. I'm the daughter and granddaughter of programmers, and I learned the basics of how to code as a kid. I'm good at it and have a knack for it, but I didn't want to do it for 8+ hours a day and then spend my nights on it as well, so I didn't pursue it as a career. I did an undergraduate degree in Linguistics, which has been really helpful for having an intuitive sense for what 'language as data' can accomplish and for a strong understanding of the difference between language as data and language as meaning. I studied formal logic systems. Then I did a graduate degree in Library Science and worked in libraries for a decade and a half. I can organize and define systems very well, and I'm trained in how to wheedle information people don't consciously know out of them without them knowing I'm doing it. I've spent enough time around actual devs to understand where my limitations are and when to loop in someone who knows more to check my work, and when it's important for the work to be super accurate versus when I can learn by fucking around. (Front end and design? Fuck around! Database structure? Fuck around but with an exceptionally robust backup system kept outside of the AI tools' purview + don't fuck around in prod. Storing credentials and people's information? Ask someone.) The problem companies are going to have is I'm very disinclined to work for them doing this, particularly if they want us because they think we're going to be cheaper. Most people who are in this category a.) could be devs and opted not to, and there's a reason for that and/or b.) are the children, cousins, etc. of programmers. We're not stupid: we know we're just as disposable as they're trying to make devs.
- abalashov 5mo ago

hbwang2076 5mo ago

Domain expertise matters more than most bootcamp grads want to hear. Knowing how to phrase a query for a specific domain is half the battle.

theptip 5mo ago

I think this neatly captures the gap in the article; it’s thinking of domain expertise as static, rather than something that needs to be built/extracted. It also goes the other way; A a good engineer can build experiments and discover the domain even without an oracle on the team; reading protocols or manuals or specs, for example. In other words, learn to be a domain expert themselves. Personally I find the most efficient way to learn a concept to be building it; you are immediately faced with your blind spots. AI for sure helps improve cycle time on these discoveries if you use it that way; time writing boilerplate goes to ~zero and you can spend your time figuring out how to answer the real questions. I do think that we as a profession need to learn new architectural and craftsmanship patterns to make it easier to learn from our code; concepts like Literate Programming which were too fussy for fast-moving teams may end up accelerating in the agentic world; I need to read a lot more code than I write now. But also things like Acceptance Test Driven Design; not new concepts, just newly increased in value.

nonethewiser 5mo ago

> But what they could do was tell him whether the handling of any particular transaction was right or wrong. So often times he would sit down with these accounting folks and go through lots of example transactions he came up with, and from there he essentially built up the requirements spec. This combination of requirements is the business in most domains. Software or otherwise. Codefying different rules and executing them.

physicsguy 5mo ago

I have found this with (mechanical) engineers. They know what they want to see but don’t understand the underlying details which are more mathematical than the average engineer is able to work with. So the people working on engineering software are often physicists or applied mathematicians.

dapperdrake 5mo ago
Ran into this, as well. And they were really annoyed at being asked math questions.

dapperdrake 5mo ago

That is why coders who learn problem domains are where the money is at. Works for me. Already learned two or three problem domains well enough and now it is starting to compound.

LorenPechtel 5mo ago

I very much agree here--the true job of us programmers is determining exactly what the problem is. The domain expert will fail because they haven't attempted to refine the details enough, they probably don't have the skill to anticipate the edge cases. I've spent most of my career working with the same people--even people who have some coding experience but they never catch the low probability stuff. Or even not so low probability: business procedures were that the customer was quoted an all-inclusive price. Try to work backwards, what is the pre-tax amount? Um, that's not guaranteed to have a solution. My concerns get ignored, I go ahead and implement with code that will catch the offending penny and label it roundoff correction. While I'm not sure what happened I think some auditor hit that. I ended up having to walk them through the calculation, okay, what's the answer here? Only when they couldn't solve the problem could they accept that what I had done was the only answer. (The actual odds of such a failure are equal to the tax rate. I had not originally worked that out, just saw success was not guaranteed.) The domain experts make this sort of mistake routinely in dealing with code. AI won't fix that.

yencabulator 5mo ago

> there is a big difference between being able to verify the output of a system is correct, and being able to tell a system how to generate the correct output to begin with. There are attempts to avoid needing to explicitly construct the rules. This is often called "Programming by example". https://en.wikipedia.org/wiki/Programming_by_example https://en.wikipedia.org/wiki/Programming_by_example On some level, even training an LLM to answer questions is this. I do like my determinism, though. Vibecoding with human-managed acceptance is a very current-moment way of doing this. Just make sure the agent has to program within a framework, not changing it, and the results are actually pretty good. One of the interesting things one can do with this mentality, if you set up your system well, is to reevaluate prior decisions with new rules, and decide whether any differing decisions are corrections for prior bad decisions, or newly-introduced bugs.

tim-projects 5mo ago

> they'd have a hell of a time prompting an AI system on how to grade a test for grammar correctness. This isn't true. They can simply upload a whole bunch of completed and marked tests and say show me the rules. And follow that up with 'write me a prompt'

abishr12 5mo ago

what people miss is that software engineering is its own language, even the finance expert can code but doesn't mean they should.