7 ms·

The Bitter Lesson of LLM Extensions

144 points by sawyerjhood 11mo ago

dsign 11mo ago
I don't know, even ChatGPT 5.1 hallucinates API's that don't exist, though it's a step forward in that it also hallucinates the non existence of APIs that exist. But I reckon that every time that humans have been able to improve their information processing in any way, the world has changed. Even if all we get is to have an LLM be right more times than it is wrong, the world will change again.
vessenes 11mo ago
> "If I could short MCP, I would" I mean, MCP is hard to work with. But there's a very large set of things that we want a hardened interface to out there - if not MCP, it will be something very like it. In particular, MCP was probably overly complicated at the design phase to deal with the realities of streaming text / tokens back and forth live. That is, it chose not to abstract these realities in exchange for some nice features, and we got a lot of implementation complexity early. To quote the Systems Bible, any working complex system is only the result of the growth of a working simple system -- MCP seems to me to be right on the edge of what you'd define as a "working simple system" -- but to the extent it's all torn down for something simpler, that thing will inevitably evolve to allow API specifications, API calls, and streaming interaction modes. Anyway, I'm "neutral" on MCP, which is to say I don't love it. But I don't have a better system in mind, and crucially, because these models still need fine-tuning to deal properly with agent setups, I think it's likely here to stay.
- zby 11mo ago
  MCP is another middleware story - this always fails (hat tip Benetict Evans).
- robot-wrangler 11mo ago
  I always see the hard/complex criticism but find it confusing.. what is the perceived difficulty with MCP at the implementation level? (I do understand the criticism about exhausting tokens with tool-descriptions and stuff, but that's a different challenge) Doesn't seem like implementation could be more simple. Just JSON-RPC and API stuff. For example the MCP hello-world with python and FastMCP is practically 1-to-1 with a http/web flavored hello-world in flask
  - vessenes 11mo ago
    There is a LOT under the surface. custom routes, bidirectional streaming choices (it started as a "local first" protocol). Implementing an endpoint from scratch is not easy, and the spec documentation moves very quickly, and generally doesn't have simple-to-digest updates for implementation. I haven't looked in a few months, so my information might be a bit out of date, but at the time - if you wanted to use a python server from the modelcontextprotocol GitHub, fine. If you wanted to, say, build a proxy server in rust or golang, you were looking at a set of half-implemented server implementations targeting two-versions-old MCP specs while clients like claude obscure even which endpoints they use for discovery. It's an immature spec, moderately complicated, and moving really quickly with only a few major 'subscribers' to the server side; I found it challenging to work with.
    - robot-wrangler 11mo ago
      Well if your language of choice didn't have any good library support for HTTP, the web version of hello world would be hard too, but it would not say much about the protocol. Even with these constraints the core MCP design is actually pretty good. First, use stdio transport, and now your language only needs to speak JSON [1]. Then, forget about building proxies and routers and web stuff, and offload that to mcpjungle [2] or similar to front your stdio work. If that still doesn't work, I think I would probably wrap the foreign language with subprocs and retreat towards python's FastMCP (or whatever the well-supported and fast-moving stuff is in another language). Ugly but practical if you really must use a language with no good MCP support. If really none of that works I guess one is on the hook to support a changing MCP spec with a custom implementation in that language.. but isn't there maybe an argument now that MCP is complex because someone insisted on it being complex? [1]: https://modelcontextprotocol.io/specification/2025-06-18/basic/transports https://modelcontextprotocol.io/specification/2025-06-18/bas... [2]: https://github.com/mcpjungle/MCPJungle https://github.com/mcpjungle/MCPJungle
      - vessenes 10mo ago
        My use case was adding the ability to charge for MCP calls to remote MCP providers. This involves a “simple on paper” wrap, proxy, insert tools on the proxy/charging server side. A number of the paradigms you mention just aren’t great, e.g. stdio over http doesn’t work (and I’ll reference you to the lengthy GitHub issues conversations at the MCP GitHub about how they want to support it when the server is not local), and in fact MCP over TCP is just literally months old. Anyway, like I said, if you’re on a golden path that tracks the monorepo delivered by the spec folks, I agree with you, it works pretty well. For reference, I think writing an MCP proxy layer in (lang of choice) is significantly harder than writing something to respond to GET / over http, both in complexity of what clients need out of a server (web clients are hardened to deal with all kinds of bad behavior), and in the amount of stuff you actually need to write, and also in the lack of documentation.
- vidarh 11mo ago
  The thing is, MCP is little more than another self-descriping API format, and current models can handle most semi-regular API's with just a description and basic tooling. I had Claude interact with my app server via Curl before I decided to just tell it to write an API client instead. I could have told it to implement MCP instead, but now I have a CLI client that I can use as well, and Claude happily uses it with just the --help options. If you don't already have an API, sure, MCP is a possible choice for that API. But if you have an API, there is decreasing reasons to bother implementing an MPC server the smarter the models are getting vs. just giving it access to your API docs.
- btbuildem 11mo ago
  MCP came in a bit too early, when the conceptual shift of hadn't fully kicked in yet. I see it as a bit of a Horseless Carriage, and I think Skills came in to counter that. My sense is that this will settle into a sort of self-assembling code golem, where ambiguous parts are handled in LLM-space, and clear, well-defined things are handled in code-space.
mkagenius 11mo ago
> Skills are the actualization of the dream that was set out by ChatGPT Plugins .. But I have a hypothesis that it might actually work now because the models are actually smart enough for it to work. and earlier Simon Willison argued[1] that Skills are even bigger deal than MCP. But I do not see as much hype for Skills as it was for MCP - it seems people are in the MCP "inertia" and having no time to shift to Skills. 1. https://simonwillison.net/2025/Oct/16/claude-skills/ https://simonwillison.net/2025/Oct/16/claude-skills/
- zby 11mo ago
  I still don't get what is special about the skills directory - since like forever I instructed Claud Code - "please read X and do Y" - how skills are different from that?
  - mkagenius 11mo ago
    The difference is that the code in the directory (and the markdown) are hardcoded and known to work beforehand.
    - munk-a 11mo ago
      But we are still reliant on the LLM correctly interpreting the choice to pick the right skill. So "known to work" should be understood in the very limited context of "this sub-function will do what it was designed to do reliably" rather than "if the user asks to use this sub-function it will do was it was designed to do reliably". Skills feel like a non-feature to me. It feels more valuable to connect a user to the actual tool and let them familiarize themselves with it (and not need the LLM to find it in the future) rather than having the tool embedded in the LLM platform. I will carve out a very big exception of accessibility here - I love my home device being an egg timer - it's a wonderful egg timer (when it doesn't randomly play music) and I could buy an egg timer but having a hands-free egg timer is actually quite valuable to me while cooking. So I believe there is real value in making these features accessible through the LLM over media that the feature would normally be difficult to use in.
j2kun 11mo ago
I don't see how "they improved the models" is related to the bitter lesson. You are still injecting human-level expertise (whether it is by prompts or a structured API) to compensate for the model's failures. A "bitter lesson" would be that the model can do better without any injection, but more compute power, than it could with human interference.
- idle_zealot 11mo ago
  > A "bitter lesson" would be that the model can do better without any injection, but more compute power, than it could with human interference. This is what I expected the post to be about before clicking.
  - marshall300791 11mo ago
    The bitter lesson here is that it all goes back to natural language rather than lower level of abstractions
    - j2kun 10mo ago
      I would contest that this is not a "bitter lesson" in the sense that it has not been demonstrated repeatedly over decades as a truism of computer science.
zby 11mo ago
I believe that what we need is treating prompts as stochastic programs and using a special shell for calling them. Claude Code and Codex and other coding agents are like that - now everybody understands that they are not just coding assistants they are a general shell that can use LLM for executing specs. I would like to have this extracted from IDE tools - this is what I am working on in llm-do.
Der_Einzige 11mo ago
Zero discussion around LLM sampling. How do you leave such a gaping hole in such a written piece? I know it's not AI cus AI wouldn't be that sloppy.
- ttkciar 11mo ago
  Local inference users are all about sampling, but users addicted to commercial inference services are wary of sampling, because they have to pay by the token.
jdblair 11mo ago
a funny thing happened why i added and emacs eval mcp tool to claude code https://hachyderm.io/@jdblair/115605988820465712 https://hachyderm.io/@jdblair/115605988820465712
lerp-io 11mo ago
can someone explain to me the difference between MCP and calling a cli tool eg curl or whatever i still don’t understand i’ve been using ai for years now.
- virajk_31 11mo ago
  MCP is tool calling with continued context/rich context, tool calling alone will PROBABLY die after single call whereas MCP keeps continuity by design (You can use MCP for tool calling but not vice versa). Hope this help you understand.
  - lerp-io 11mo ago
    no still doesn't make sense lmao. you call api and get output, no?
bloppe 11mo ago
How is this related to the bitter lesson?
- ttkciar 11mo ago
  The author speculates that bigger/smarter models interpreting vague directives to utilize general-function tools will outperform more precise and detailed directives to utilize narrow-function tools: > Granted to use a skill the agent needs to have general purpose access to a computer, but this is the bitter lesson in action. Giving an agent general purpose tools and trusting it to have the ability to use them to accomplish a task might very well be the winning strategy over making specialized tools for every task.
- uriegas 11mo ago
  I was thinking the same thing. Maybe is that at the end the author seems to imply that agentic AI will work simply because models have become better regardless of the way we make them agentic (i.e. MCPs, skills, etc).
ttkciar 11mo ago
Well, that's just great. The academic community has been using the term "skill" for years, to refer to classes of tasks at which LLMs exhibit competence. Now OpenAI has usurped the term to refer to these inference-guiding .md files. I'm not looking forward to having to pick through a Google hit list for "LLM skills", figuring out which publications are about skills in the traditional sense and which are about the OpenAI feature. Semantic overload sucks. How do we deal with this? Start using "competencies" (or similar) in academic papers? Or just resign ourselves to suffering the ambiguity? Or maybe the OpenAI feature will fall flat and nobody will talk about it at all. That would frankly be the best outcome.
- lupire 11mo ago
  The way NNs and LLMs solve this problem is by processing context and activating middle layer nodes to disambiguate local ambiguities. Have you tried increasing your context window?
- jswny 11mo ago
  Skills are an Anthropic feature
  - ttkciar 11mo ago
    Whoops, you are right. I misread the article.
- touristtam 10mo ago
  What about open? Or ai? Neither is really what they are offering. Open they are not (weight doesn't count) and don't get me started on that statistical machine they call artificial intelligence. Misleading through and through.
WatchDog 11mo ago
The most useful LLM "extension" isn't even mentioned in this article, and that is shell use. An LLM with a shell integration can do anything you need it to.
- AIorNot 11mo ago
  A man with a spoon can dig a swimming pool but Id prefer a backhoe
  - btbuildem 11mo ago
    sudo apt-get install backhoe
    - touristtam 10mo ago
      mise use -g backhoe
uriegas 11mo ago
> "I expect us to go back to extending our agents with the most accessible programming language: natural language." I don't agree with this. Natural language is so ambiguous. At least for software development the hard work is still coming up with clearly defined solutions. There is a reason for why math has its own domain specific language.
- euroderf 11mo ago
  > Natural language is so ambiguous. As a former tech comms guy I will say: Natural language can be bent into arbitrary precision. Write something, then enter a read-rewrite-reread loop as the devil's advocate (this is key) until it stops being ambiguous or having multiple conceivable interpretations. Yes with English this process can be a pain in the butt, until you get the hang of it.
  - stavros 11mo ago
    The problem is that it's very hard to anticipate all possible edge cases. Programming languages force you to do a lot of that work up front, English doesn't. It's the difference between writing Javascript and writing Typescript, except orders of magnitude worse.
  - thfuran 11mo ago
    You’re never going to make a nontrivial statement in English that you couldn’t find two people who wouldn’t perfectly agree on its meaning. Or probably even a trivial one. Sure, at some point you can say “no, you’re clearly misinterpreting what I’ve said” or “you’re inferring something that wasn’t implied”, but English doesn’t have a formal spec or a reference implementation, so that’s kind of meaningless.
  - didroe 11mo ago
    The problem is, what's ambiguous or precise is subjective. Your devil's advocate needs to reflect all of the possible readers, and that isn't possible. There's a good reason we use jargon in professions, or more constrained and less ambiguous languages for maths/coding
  - 11mo ago
arjie 11mo ago
Custom GPTs are pretty old, but I recently found a use for them. My wife wanted some meeting note-taking and task recording assistance and I found that making a Custom GPT with a trivial Notion API that was scoped to one page[0] with structure that was encoded in the API was a quick couple-hour thing that unlocked a lot of utility for her (the default Notion MCP is "too broad"). It helped that this Custom GPT sits in her ChatGPT UI and she doesn't have to have another app or whatever to make it work. We liked it quite a bit, but it led to some funny things. We use Reminders to keep our home to-do lists, hers and mine in one list with two sections. I wanted to take this existing flow we had and make it work with a Custom GPT. It's practically impossible because Reminders: * doesn't have a good API through EventKit * requires a pop-up permission grant in the UI So in the end, I did end up making somewhat of an MCP server for it, running it on an old Macbook Pro I had and then sticking Amphetamine on in closed-lid display-sleep mode hooked up to my Tailnet and exposed via a Cloudflare tunnel so that we could use ChatGPT to interact with the thing. Yes, you can see how insane that whole thing is. But there's quite a lot of value to have your AI agent just be the one thing. 0: https://wiki.roshangeorge.dev/w/Blog/2025-10-17/Custom_GPTs https://wiki.roshangeorge.dev/w/Blog/2025-10-17/Custom_GPTs
metalrain 11mo ago
Skills.md will in time have same problem as MCP, they will bloat the context. I wonder if we could just have the scripts without the descriptions and LLM would have been trained to search the most useful things in specific folder.
- crackalamoo 10mo ago
  This seems like a solvable engineering problem. For example, you could have a lightweight subagent with its own context for reading the skills and determining which to use
iamcreasy 10mo ago
ChatGPT apps, announced this month, feels a lot like original ChatGPT Plugin announced 3 years back. The only difference is how plugin are invoked. For ChatGPT plugin, we have to choose one from a drop down, and for apps - we could just include a plugin name in prompt. Is there any other difference in the end-user side?
OBELISK_ASI 10mo ago
[dead]

144 points by sawyerjhood 11mo ago

dsign 11mo ago

I don't know, even ChatGPT 5.1 hallucinates API's that don't exist, though it's a step forward in that it also hallucinates the non existence of APIs that exist. But I reckon that every time that humans have been able to improve their information processing in any way, the world has changed. Even if all we get is to have an LLM be right more times than it is wrong, the world will change again.

vessenes 11mo ago

> "If I could short MCP, I would" I mean, MCP is hard to work with. But there's a very large set of things that we want a hardened interface to out there - if not MCP, it will be something very like it. In particular, MCP was probably overly complicated at the design phase to deal with the realities of streaming text / tokens back and forth live. That is, it chose not to abstract these realities in exchange for some nice features, and we got a lot of implementation complexity early. To quote the Systems Bible, any working complex system is only the result of the growth of a working simple system -- MCP seems to me to be right on the edge of what you'd define as a "working simple system" -- but to the extent it's all torn down for something simpler, that thing will inevitably evolve to allow API specifications, API calls, and streaming interaction modes. Anyway, I'm "neutral" on MCP, which is to say I don't love it. But I don't have a better system in mind, and crucially, because these models still need fine-tuning to deal properly with agent setups, I think it's likely here to stay.

zby 11mo ago
MCP is another middleware story - this always fails (hat tip Benetict Evans).
robot-wrangler 11mo ago
I always see the hard/complex criticism but find it confusing.. what is the perceived difficulty with MCP at the implementation level? (I do understand the criticism about exhausting tokens with tool-descriptions and stuff, but that's a different challenge) Doesn't seem like implementation could be more simple. Just JSON-RPC and API stuff. For example the MCP hello-world with python and FastMCP is practically 1-to-1 with a http/web flavored hello-world in flask
- vessenes 11mo ago
  There is a LOT under the surface. custom routes, bidirectional streaming choices (it started as a "local first" protocol). Implementing an endpoint from scratch is not easy, and the spec documentation moves very quickly, and generally doesn't have simple-to-digest updates for implementation. I haven't looked in a few months, so my information might be a bit out of date, but at the time - if you wanted to use a python server from the modelcontextprotocol GitHub, fine. If you wanted to, say, build a proxy server in rust or golang, you were looking at a set of half-implemented server implementations targeting two-versions-old MCP specs while clients like claude obscure even which endpoints they use for discovery. It's an immature spec, moderately complicated, and moving really quickly with only a few major 'subscribers' to the server side; I found it challenging to work with.
  - robot-wrangler 11mo ago
    Well if your language of choice didn't have any good library support for HTTP, the web version of hello world would be hard too, but it would not say much about the protocol. Even with these constraints the core MCP design is actually pretty good. First, use stdio transport, and now your language only needs to speak JSON [1]. Then, forget about building proxies and routers and web stuff, and offload that to mcpjungle [2] or similar to front your stdio work. If that still doesn't work, I think I would probably wrap the foreign language with subprocs and retreat towards python's FastMCP (or whatever the well-supported and fast-moving stuff is in another language). Ugly but practical if you really must use a language with no good MCP support. If really none of that works I guess one is on the hook to support a changing MCP spec with a custom implementation in that language.. but isn't there maybe an argument now that MCP is complex because someone insisted on it being complex? [1]: https://modelcontextprotocol.io/specification/2025-06-18/basic/transports https://modelcontextprotocol.io/specification/2025-06-18/bas... [2]: https://github.com/mcpjungle/MCPJungle https://github.com/mcpjungle/MCPJungle
    - vessenes 10mo ago
      My use case was adding the ability to charge for MCP calls to remote MCP providers. This involves a “simple on paper” wrap, proxy, insert tools on the proxy/charging server side. A number of the paradigms you mention just aren’t great, e.g. stdio over http doesn’t work (and I’ll reference you to the lengthy GitHub issues conversations at the MCP GitHub about how they want to support it when the server is not local), and in fact MCP over TCP is just literally months old. Anyway, like I said, if you’re on a golden path that tracks the monorepo delivered by the spec folks, I agree with you, it works pretty well. For reference, I think writing an MCP proxy layer in (lang of choice) is significantly harder than writing something to respond to GET / over http, both in complexity of what clients need out of a server (web clients are hardened to deal with all kinds of bad behavior), and in the amount of stuff you actually need to write, and also in the lack of documentation.
vidarh 11mo ago
The thing is, MCP is little more than another self-descriping API format, and current models can handle most semi-regular API's with just a description and basic tooling. I had Claude interact with my app server via Curl before I decided to just tell it to write an API client instead. I could have told it to implement MCP instead, but now I have a CLI client that I can use as well, and Claude happily uses it with just the --help options. If you don't already have an API, sure, MCP is a possible choice for that API. But if you have an API, there is decreasing reasons to bother implementing an MPC server the smarter the models are getting vs. just giving it access to your API docs.
btbuildem 11mo ago
MCP came in a bit too early, when the conceptual shift of hadn't fully kicked in yet. I see it as a bit of a Horseless Carriage, and I think Skills came in to counter that. My sense is that this will settle into a sort of self-assembling code golem, where ambiguous parts are handled in LLM-space, and clear, well-defined things are handled in code-space.

mkagenius 11mo ago

> Skills are the actualization of the dream that was set out by ChatGPT Plugins .. But I have a hypothesis that it might actually work now because the models are actually smart enough for it to work. and earlier Simon Willison argued[1] that Skills are even bigger deal than MCP. But I do not see as much hype for Skills as it was for MCP - it seems people are in the MCP "inertia" and having no time to shift to Skills. 1. https://simonwillison.net/2025/Oct/16/claude-skills/ https://simonwillison.net/2025/Oct/16/claude-skills/

zby 11mo ago
I still don't get what is special about the skills directory - since like forever I instructed Claud Code - "please read X and do Y" - how skills are different from that?
- mkagenius 11mo ago
  The difference is that the code in the directory (and the markdown) are hardcoded and known to work beforehand.
  - munk-a 11mo ago
    But we are still reliant on the LLM correctly interpreting the choice to pick the right skill. So "known to work" should be understood in the very limited context of "this sub-function will do what it was designed to do reliably" rather than "if the user asks to use this sub-function it will do was it was designed to do reliably". Skills feel like a non-feature to me. It feels more valuable to connect a user to the actual tool and let them familiarize themselves with it (and not need the LLM to find it in the future) rather than having the tool embedded in the LLM platform. I will carve out a very big exception of accessibility here - I love my home device being an egg timer - it's a wonderful egg timer (when it doesn't randomly play music) and I could buy an egg timer but having a hands-free egg timer is actually quite valuable to me while cooking. So I believe there is real value in making these features accessible through the LLM over media that the feature would normally be difficult to use in.

j2kun 11mo ago

I don't see how "they improved the models" is related to the bitter lesson. You are still injecting human-level expertise (whether it is by prompts or a structured API) to compensate for the model's failures. A "bitter lesson" would be that the model can do better without any injection, but more compute power, than it could with human interference.

idle_zealot 11mo ago
> A "bitter lesson" would be that the model can do better without any injection, but more compute power, than it could with human interference. This is what I expected the post to be about before clicking.
- marshall300791 11mo ago
  The bitter lesson here is that it all goes back to natural language rather than lower level of abstractions
  - j2kun 10mo ago
    I would contest that this is not a "bitter lesson" in the sense that it has not been demonstrated repeatedly over decades as a truism of computer science.

zby 11mo ago

I believe that what we need is treating prompts as stochastic programs and using a special shell for calling them. Claude Code and Codex and other coding agents are like that - now everybody understands that they are not just coding assistants they are a general shell that can use LLM for executing specs. I would like to have this extracted from IDE tools - this is what I am working on in llm-do.

Der_Einzige 11mo ago

Zero discussion around LLM sampling. How do you leave such a gaping hole in such a written piece? I know it's not AI cus AI wouldn't be that sloppy.

ttkciar 11mo ago
Local inference users are all about sampling, but users addicted to commercial inference services are wary of sampling, because they have to pay by the token.

jdblair 11mo ago

a funny thing happened why i added and emacs eval mcp tool to claude code https://hachyderm.io/@jdblair/115605988820465712 https://hachyderm.io/@jdblair/115605988820465712

lerp-io 11mo ago

can someone explain to me the difference between MCP and calling a cli tool eg curl or whatever i still don’t understand i’ve been using ai for years now.

virajk_31 11mo ago
MCP is tool calling with continued context/rich context, tool calling alone will PROBABLY die after single call whereas MCP keeps continuity by design (You can use MCP for tool calling but not vice versa). Hope this help you understand.
- lerp-io 11mo ago
  no still doesn't make sense lmao. you call api and get output, no?

bloppe 11mo ago

How is this related to the bitter lesson?

ttkciar 11mo ago
The author speculates that bigger/smarter models interpreting vague directives to utilize general-function tools will outperform more precise and detailed directives to utilize narrow-function tools: > Granted to use a skill the agent needs to have general purpose access to a computer, but this is the bitter lesson in action. Giving an agent general purpose tools and trusting it to have the ability to use them to accomplish a task might very well be the winning strategy over making specialized tools for every task.
uriegas 11mo ago
I was thinking the same thing. Maybe is that at the end the author seems to imply that agentic AI will work simply because models have become better regardless of the way we make them agentic (i.e. MCPs, skills, etc).

ttkciar 11mo ago

Well, that's just great. The academic community has been using the term "skill" for years, to refer to classes of tasks at which LLMs exhibit competence. Now OpenAI has usurped the term to refer to these inference-guiding .md files. I'm not looking forward to having to pick through a Google hit list for "LLM skills", figuring out which publications are about skills in the traditional sense and which are about the OpenAI feature. Semantic overload sucks. How do we deal with this? Start using "competencies" (or similar) in academic papers? Or just resign ourselves to suffering the ambiguity? Or maybe the OpenAI feature will fall flat and nobody will talk about it at all. That would frankly be the best outcome.

lupire 11mo ago
The way NNs and LLMs solve this problem is by processing context and activating middle layer nodes to disambiguate local ambiguities. Have you tried increasing your context window?
jswny 11mo ago
Skills are an Anthropic feature
- ttkciar 11mo ago
  Whoops, you are right. I misread the article.
touristtam 10mo ago
What about open? Or ai? Neither is really what they are offering. Open they are not (weight doesn't count) and don't get me started on that statistical machine they call artificial intelligence. Misleading through and through.

WatchDog 11mo ago

The most useful LLM "extension" isn't even mentioned in this article, and that is shell use. An LLM with a shell integration can do anything you need it to.

AIorNot 11mo ago
A man with a spoon can dig a swimming pool but Id prefer a backhoe
- btbuildem 11mo ago
  sudo apt-get install backhoe
  - touristtam 10mo ago
    mise use -g backhoe

uriegas 11mo ago

> "I expect us to go back to extending our agents with the most accessible programming language: natural language." I don't agree with this. Natural language is so ambiguous. At least for software development the hard work is still coming up with clearly defined solutions. There is a reason for why math has its own domain specific language.

euroderf 11mo ago
> Natural language is so ambiguous. As a former tech comms guy I will say: Natural language can be bent into arbitrary precision. Write something, then enter a read-rewrite-reread loop as the devil's advocate (this is key) until it stops being ambiguous or having multiple conceivable interpretations. Yes with English this process can be a pain in the butt, until you get the hang of it.
- stavros 11mo ago
  The problem is that it's very hard to anticipate all possible edge cases. Programming languages force you to do a lot of that work up front, English doesn't. It's the difference between writing Javascript and writing Typescript, except orders of magnitude worse.
- thfuran 11mo ago
  You’re never going to make a nontrivial statement in English that you couldn’t find two people who wouldn’t perfectly agree on its meaning. Or probably even a trivial one. Sure, at some point you can say “no, you’re clearly misinterpreting what I’ve said” or “you’re inferring something that wasn’t implied”, but English doesn’t have a formal spec or a reference implementation, so that’s kind of meaningless.
- didroe 11mo ago
  The problem is, what's ambiguous or precise is subjective. Your devil's advocate needs to reflect all of the possible readers, and that isn't possible. There's a good reason we use jargon in professions, or more constrained and less ambiguous languages for maths/coding
- 11mo ago

arjie 11mo ago

Custom GPTs are pretty old, but I recently found a use for them. My wife wanted some meeting note-taking and task recording assistance and I found that making a Custom GPT with a trivial Notion API that was scoped to one page[0] with structure that was encoded in the API was a quick couple-hour thing that unlocked a lot of utility for her (the default Notion MCP is "too broad"). It helped that this Custom GPT sits in her ChatGPT UI and she doesn't have to have another app or whatever to make it work. We liked it quite a bit, but it led to some funny things. We use Reminders to keep our home to-do lists, hers and mine in one list with two sections. I wanted to take this existing flow we had and make it work with a Custom GPT. It's practically impossible because Reminders: * doesn't have a good API through EventKit * requires a pop-up permission grant in the UI So in the end, I did end up making somewhat of an MCP server for it, running it on an old Macbook Pro I had and then sticking Amphetamine on in closed-lid display-sleep mode hooked up to my Tailnet and exposed via a Cloudflare tunnel so that we could use ChatGPT to interact with the thing. Yes, you can see how insane that whole thing is. But there's quite a lot of value to have your AI agent just be the one thing. 0: https://wiki.roshangeorge.dev/w/Blog/2025-10-17/Custom_GPTs https://wiki.roshangeorge.dev/w/Blog/2025-10-17/Custom_GPTs

metalrain 11mo ago

Skills.md will in time have same problem as MCP, they will bloat the context. I wonder if we could just have the scripts without the descriptions and LLM would have been trained to search the most useful things in specific folder.

crackalamoo 10mo ago
This seems like a solvable engineering problem. For example, you could have a lightweight subagent with its own context for reading the skills and determining which to use

iamcreasy 10mo ago

ChatGPT apps, announced this month, feels a lot like original ChatGPT Plugin announced 3 years back. The only difference is how plugin are invoked. For ChatGPT plugin, we have to choose one from a drop down, and for apps - we could just include a plugin name in prompt. Is there any other difference in the end-user side?

OBELISK_ASI 10mo ago

[dead]