6 ms·

The big thing is, the UI for these tools is primitive at best, and there doesn't seem to be a facility for file-system or batch interaction, even on local model

The big thing is, the UI for these tools is primitive at best, and there doesn't seem to be a facility for file-system or batch interaction, even on local models. Is it really so hard to: - test/develop a prompt which works on a single file - tell the interface to run the tested prompt on all the files in a specified folder - returning the collected output from all the prompts as a single output/file?

mordymoop 1y ago
I agree. The UI component is currently a surprisingly big hurdle. Last night I set up my 11 year old son with Claude 4, with MCP enabled for filesystem modification, reasoning that the LLMs are finally at the level of capability where they can reliably just do things. And I was right - Claude put together a browser JS game according to his description in seconds, and iterated on it several times to incorporate his suggestions. But then it hit the limit of how big of a file it can comfortably write out in one go, started making incomplete edits, and basically fell into a snarl of endlessly trying and failing to write files due to file length limitations. I had to step in and tell Claude to split the code up into several files, something my son wouldn’t have known to do. If I hadn’t been there to tell Claude how to work around its own UI and tool limitations, it would have likely blown through the rest of its context window and totally failed at the task. I imagine this is a common experience for people. Most people wouldn’t know how to set up Claude with MCP in the first place. A surprising number of people who seem relatively aware of LLM technology aren’t aware of MCP, especially if their workflow is centered on IDE integration. To be fair, these are probably the people who need MCP the least, but MCP (and, in general, agentic tool use) is definitely closer to how normal people will get value out of LLMs. It may seem stupid and trivial, but telling Claude to directly edit or debug a file that exists on your hard drive is actually a multiples-faster and smoother experience than doing the laborious copy-paste exercise many people are still engaging with. This is just as true for code as it is for writing documents. The LLM companies seem to be quite aware of this, hence products like Claude Code, the Claude computer use suite, all the Gemini android screen/camera share integration, and GPT Codex. It’s a rocky process but at some point soon we will cross the threshold where it just works. But right now, it doesn’t just work, and so it’s not really that much faster or more efficient than doing a task yourself, especially if you aren’t intimately familiar with all the quirks and limitations of LLMs.
aaronbaugher 1y ago
I'm finding the interface to be the biggest hurdle too. I was talking to Grok about the possibility of using an LLM as a personal assistant that would gradually learn my preferences and the context of my work. It explained that LLMs are stateless, so any context that you want it to have, you have to feed back in with the prompt. I'd need to build a wrapper of sorts around the LLM, that would save our conversations and feed them back in each session, saying "Here's everything we've said before; use that as context for this new prompt." I can script that, but the payload would just keep growing, and if you're scripting this around an API, which would be the only way to make it work, the per-token charges for all that context are going to stack up. That doesn't seem workable unless you're running your own local LLM. It'd be the same thing with having it process files. Sure, I can script something that does "Take all the files in a directory, give them to an LLM in a way that lets it distinguish where files start and end, and have it process them in some way and spit out the results." But that's a fair amount of work, so it's not worth it unless I come up with a task the LLM can do that I can't do with grep and other utilities. So I'm still just sort of chatting with it, bouncing ideas off it and having it do some useful research and summarization, but still looking for ways to use it that feel like it's really having an impact on my productivity.
- fragmede 1y ago
  Have the LLM collect important nuts if information from the chat for itself. ChatGPT has a "memory" feature that does this.
fragmede 1y ago
It's hard enough that OpenAI paid $3 billion for Windsurf. imThe better tools exist, so if you're still copy and pasting from a webterface, that's on you.
- WillAdams 1y ago
  I'm trying to figure out how to run a model locally (ideally in an opensource setup).

mordymoop 1y ago

I agree. The UI component is currently a surprisingly big hurdle. Last night I set up my 11 year old son with Claude 4, with MCP enabled for filesystem modification, reasoning that the LLMs are finally at the level of capability where they can reliably just do things. And I was right - Claude put together a browser JS game according to his description in seconds, and iterated on it several times to incorporate his suggestions. But then it hit the limit of how big of a file it can comfortably write out in one go, started making incomplete edits, and basically fell into a snarl of endlessly trying and failing to write files due to file length limitations. I had to step in and tell Claude to split the code up into several files, something my son wouldn’t have known to do. If I hadn’t been there to tell Claude how to work around its own UI and tool limitations, it would have likely blown through the rest of its context window and totally failed at the task. I imagine this is a common experience for people. Most people wouldn’t know how to set up Claude with MCP in the first place. A surprising number of people who seem relatively aware of LLM technology aren’t aware of MCP, especially if their workflow is centered on IDE integration. To be fair, these are probably the people who need MCP the least, but MCP (and, in general, agentic tool use) is definitely closer to how normal people will get value out of LLMs. It may seem stupid and trivial, but telling Claude to directly edit or debug a file that exists on your hard drive is actually a multiples-faster and smoother experience than doing the laborious copy-paste exercise many people are still engaging with. This is just as true for code as it is for writing documents. The LLM companies seem to be quite aware of this, hence products like Claude Code, the Claude computer use suite, all the Gemini android screen/camera share integration, and GPT Codex. It’s a rocky process but at some point soon we will cross the threshold where it just works. But right now, it doesn’t just work, and so it’s not really that much faster or more efficient than doing a task yourself, especially if you aren’t intimately familiar with all the quirks and limitations of LLMs.

aaronbaugher 1y ago

I'm finding the interface to be the biggest hurdle too. I was talking to Grok about the possibility of using an LLM as a personal assistant that would gradually learn my preferences and the context of my work. It explained that LLMs are stateless, so any context that you want it to have, you have to feed back in with the prompt. I'd need to build a wrapper of sorts around the LLM, that would save our conversations and feed them back in each session, saying "Here's everything we've said before; use that as context for this new prompt." I can script that, but the payload would just keep growing, and if you're scripting this around an API, which would be the only way to make it work, the per-token charges for all that context are going to stack up. That doesn't seem workable unless you're running your own local LLM. It'd be the same thing with having it process files. Sure, I can script something that does "Take all the files in a directory, give them to an LLM in a way that lets it distinguish where files start and end, and have it process them in some way and spit out the results." But that's a fair amount of work, so it's not worth it unless I come up with a task the LLM can do that I can't do with grep and other utilities. So I'm still just sort of chatting with it, bouncing ideas off it and having it do some useful research and summarization, but still looking for ways to use it that feel like it's really having an impact on my productivity.

fragmede 1y ago
Have the LLM collect important nuts if information from the chat for itself. ChatGPT has a "memory" feature that does this.

fragmede 1y ago

It's hard enough that OpenAI paid $3 billion for Windsurf. imThe better tools exist, so if you're still copy and pasting from a webterface, that's on you.

WillAdams 1y ago
I'm trying to figure out how to run a model locally (ideally in an opensource setup).