5 ms·
I use local models + openrouter free ones. My monthly spend on ai models is < $1 I'm not cheap, just ahead of the curve. With the collapse in inference cost,
by kristopolous 10mo ago
I use local models + openrouter free ones.
My monthly spend on ai models is < $1
I'm not cheap, just ahead of the curve. With the collapse in inference cost, everything will be this eventually
I'll basically do
$ man tool | <how do I do this with the tool>
or even
$ cat source | <find the flags and give me some documentation on how to use this>
Things I used to do intensively I now do lazily.
I've even made a IEITYuan/Yuan-embedding-2.0-en database of my manpages with chroma and then I can just ask my local documentation how I do something conceptually, get the man pages, inject them into local qwen context window using my mansnip llm preprocessor, forward the prompt and then get usable real results.
In practice it's this:
$ what-man "some obscure question about nfs"
...chug chug chug (about 5 seconds)...
<answer with citations back to the doc pages>
Essentially I'm not asking the models to think, just do NLP and process text. They can do that really reliably.
It helps combat a frequent tendency for documentation authors to bury the most common and useful flags deep in the documentation and lead with those that were most challenging or interesting to program instead.
I understand the inclination it's just not all that helpful for me
- m4ck_ 10mo agoIs your RAG manpages thing on github somewhere? I was thinking about doing something like that (it's high on my to-do list but I haven't actually done anything with llms yet.)
- kristopolous 10mo agoI'll get it up soon, probably should. This little snippet will help you though: $ man --html="$(which markitdown)" <man page> That goes man -> html -> markdown which is not only token efficient but also llms are pretty good at creating hierarchies from markdown
- scottyeager 10mo agoNot the OP, but I did release my source :D https://github.com/scottyeager/Pal https://github.com/scottyeager/Pal My tool can read stdin, send it to an LLM, and do a couple nice things with the reply. Not exactly RAG, but most man pages fit into the context window so it's okay.
- nl 10mo agoThis is a completely different thing to AI coding models. If you aren't using coding models you aren't ahead of the curve. There are free coding models. I use them heavily. They are ok but only partial substitutes for frontier models.
- deleted 10mo ago[deleted]
- kristopolous 10mo agoI'm extremely familiar with them. Some people, with some tasks, get great results But me, with my tasks, I need to maintain provenance and accountability over the code. I can't just have AI fly by the seat of its pants. I can get into lots of detail on this. If you have seen tools and setups I have done you'd realize why it doesn't work for me. I've spent money, the results for me, with my tasks, have not been the right decision.
- aquafox 10mo ago> I'll basically do $ man tool | <how do I do this with the tool> or even $ cat source | <find the flags and give me some documentation on how to use this> Could you please elaborate on this? Do I get this right that you can set up your your command line so that you can pipe something to a command that sends this something together with a question to an LLM? Or did you just mean that metaphorically? Sorry if this is a stupid question.
- __m 10mo agoi guess op means: $ man tool | ai <how do I do this with the tool> where ai could be a simple shell script combining the argument with stdin
- scottyeager 10mo agoI'm not the OP, but I did build a tool that I use in the same way: https://github.com/scottyeager/Pal https://github.com/scottyeager/Pal Actually for many cases the LLM already knows enough. For more obscure cases, piping in a --help output is also sometimes enough.
- mr_mitm 10mo agoYes, I use simonw's `llm` for that: https://github.com/simonw/llm https://github.com/simonw/llm Example: $ man tar | llm "how do I extract test.txt from a tar.gz"
- alfonsodev 10mo agoI use llm from command line too, time to time, is just easier to do llm 'output a .gitignore file for typical python project that I can pipe into the actual file ' > .gitignore
- fragmede 10mo ago> My monthly spend on ai models is < $1 > I'm not cheap You're cheap. It's okay. We're all developers here. It's a safe space.
- mathgeek 10mo agoWhile I say this somewhat in jest, frugal is just cheap but with better value.
- techwizrd 10mo agoHave you looked at tldr/tealdeer[0]? It may do much of what you're looking for, albeit without LLM assistance. 0: https://tealdeer-rs.github.io/tealdeer/ https://tealdeer-rs.github.io/tealdeer/
- martin1975 10mo agothis is the extent to what I use any LLM - they're really good at looking up just about anything, in natural language, and most of the time even the first hit, without reprompting, is a pretty decent answer. I used to have to sort thru things to get there, so there's definitely an upside to LLMs in this manner.
- MuffinFlavored 10mo ago> I'm not cheap, just ahead of the curve. I'm not convinced. I'm convinced you don't value your time. As Simon said, throw $20-$100/mo and get the best state of the art models with "near 0" setup and move on.