4 ms·

> Once you figure out how to do that they're absurdly useful I have read some posts of yours advancing that but I never met those with the details: do you mean

by mdp2021 2y ago

> Once you figure out how to do that they're absurdly useful I have read some posts of yours advancing that but I never met those with the details: do you mean more "prompt engineering", or "application selection", or "system integration"...?

simonw 2y ago
Typing code faster. Building quick illustrative prototypes. Researching options for libraries (that are old and stable enough to be in the training data). Porting code from one language to another (surprisingly) [1]. Using as a thesaurus. Answering questions about code (like piping in a whole codebase and asking about it) [2]. Writing an initial set of unit tests. Finding the most interesting new ideas in a paper or online discussion thread without reading the whole thing. Building one-off tools for converting data. Writing complex SQL queries. Finding potential causes of difficult bugs. [3] [1] I built https://tools.simonwillison.net/hacker-news-thread-export https://tools.simonwillison.net/hacker-news-thread-export this morning from my phone using that trick: https://claude.ai/share/7d0de887-5ff8-4b8c-90b1-b5d4d4ca9b84 https://claude.ai/share/7d0de887-5ff8-4b8c-90b1-b5d4d4ca9b84 [2] Examples of that here: https://simonwillison.net/2025/Mar/11/using-llms-for-code/#bonus-answering-questions-about-codebases https://simonwillison.net/2025/Mar/11/using-llms-for-code/#b... [3] https://simonwillison.net/2024/Sep/25/o1-preview-llm/ https://simonwillison.net/2024/Sep/25/o1-preview-llm/ is an early example of using a "reasoning" model for that Or if you meant "what do you have to figure out to use them effectively despite their flaws?", that's a huge topic. It's mostly about building a deep intuition for what they can and cannot help with, then figuring out how to prompt them (including managing their context of inputs) to get good results. The most I've written about that is probably this piece: https://simonwillison.net/2025/Mar/11/using-llms-for-code/ https://simonwillison.net/2025/Mar/11/using-llms-for-code/
- mdp2021 2y ago
  All of that is very interesting. Side note: don't you agree that "answering about documentation with 100% reliability" would be a more than desirable further feature? (Think of those options in the shell commands which can be so confusing they made it to xkcd material.) But that would mean achieving production-level RAG; and that in turn would be a revolution in LLMs, which would revise your list above...
  - simonw 2y ago
    LLMs can never provide 100% reliability - there's a random number generator in the mix after all (reflected in the "temperature" setting). For documentation answering the newer long context models are wildly effective in my experience. You can dump a million tokens (easily a full codebase or two for most projects) into Gemini 2.5 Pro and get great answers to almost anything. There are some new anonymous preview models with 1m token limits floating around right now which I suspect may be upcoming OpenAI models. https://openrouter.ai/openrouter/optimus-alpha https://openrouter.ai/openrouter/optimus-alpha I actually use LLMs for command line arguments for tools like ffmpeg all the time, I built a plugin for that: https://simonwillison.net/2024/Mar/26/llm-cmd/ https://simonwillison.net/2024/Mar/26/llm-cmd/
    - mdp2021 2y ago
      > random number generator But the use of randomness inside the system should not prevent, in theory, as-if-full reliability - this stresses that the architecture could be unfinished, as I expressed with the example of RAG. (E.g.: well trained natural minds use check systems over provisional output, however obtained.) > newer long context models Practical question: if the query-contextual documentation needs to be part of the input (I am not aware of a more efficient way), does not that massively impact the processing time? Suppose you have to examine interactively the content of a Standard Hefty Document of 1MB of text... If so, that would make local LLM use prohibitive.
      - simonw 2y ago
        Longer context is definitely slower, especially for local models. Hosted models running on who knows what kind of overpowered hardware can crunch through them pretty fast though. There's also token caching available for OpenAI, Anthropic, Gemini and DeepSeek which can dramatically speed up processing of long context prompts if they've been previously cached.

simonw 2y ago

Typing code faster. Building quick illustrative prototypes. Researching options for libraries (that are old and stable enough to be in the training data). Porting code from one language to another (surprisingly) [1]. Using as a thesaurus. Answering questions about code (like piping in a whole codebase and asking about it) [2]. Writing an initial set of unit tests. Finding the most interesting new ideas in a paper or online discussion thread without reading the whole thing. Building one-off tools for converting data. Writing complex SQL queries. Finding potential causes of difficult bugs. [3] [1] I built https://tools.simonwillison.net/hacker-news-thread-export https://tools.simonwillison.net/hacker-news-thread-export this morning from my phone using that trick: https://claude.ai/share/7d0de887-5ff8-4b8c-90b1-b5d4d4ca9b84 https://claude.ai/share/7d0de887-5ff8-4b8c-90b1-b5d4d4ca9b84 [2] Examples of that here: https://simonwillison.net/2025/Mar/11/using-llms-for-code/#bonus-answering-questions-about-codebases https://simonwillison.net/2025/Mar/11/using-llms-for-code/#b... [3] https://simonwillison.net/2024/Sep/25/o1-preview-llm/ https://simonwillison.net/2024/Sep/25/o1-preview-llm/ is an early example of using a "reasoning" model for that Or if you meant "what do you have to figure out to use them effectively despite their flaws?", that's a huge topic. It's mostly about building a deep intuition for what they can and cannot help with, then figuring out how to prompt them (including managing their context of inputs) to get good results. The most I've written about that is probably this piece: https://simonwillison.net/2025/Mar/11/using-llms-for-code/ https://simonwillison.net/2025/Mar/11/using-llms-for-code/

mdp2021 2y ago
All of that is very interesting. Side note: don't you agree that "answering about documentation with 100% reliability" would be a more than desirable further feature? (Think of those options in the shell commands which can be so confusing they made it to xkcd material.) But that would mean achieving production-level RAG; and that in turn would be a revolution in LLMs, which would revise your list above...
- simonw 2y ago
  LLMs can never provide 100% reliability - there's a random number generator in the mix after all (reflected in the "temperature" setting). For documentation answering the newer long context models are wildly effective in my experience. You can dump a million tokens (easily a full codebase or two for most projects) into Gemini 2.5 Pro and get great answers to almost anything. There are some new anonymous preview models with 1m token limits floating around right now which I suspect may be upcoming OpenAI models. https://openrouter.ai/openrouter/optimus-alpha https://openrouter.ai/openrouter/optimus-alpha I actually use LLMs for command line arguments for tools like ffmpeg all the time, I built a plugin for that: https://simonwillison.net/2024/Mar/26/llm-cmd/ https://simonwillison.net/2024/Mar/26/llm-cmd/
  - mdp2021 2y ago
    > random number generator But the use of randomness inside the system should not prevent, in theory, as-if-full reliability - this stresses that the architecture could be unfinished, as I expressed with the example of RAG. (E.g.: well trained natural minds use check systems over provisional output, however obtained.) > newer long context models Practical question: if the query-contextual documentation needs to be part of the input (I am not aware of a more efficient way), does not that massively impact the processing time? Suppose you have to examine interactively the content of a Standard Hefty Document of 1MB of text... If so, that would make local LLM use prohibitive.
    - simonw 2y ago
      Longer context is definitely slower, especially for local models. Hosted models running on who knows what kind of overpowered hardware can crunch through them pretty fast though. There's also token caching available for OpenAI, Anthropic, Gemini and DeepSeek which can dramatically speed up processing of long context prompts if they've been previously cached.