3 ms·

Hey everyone! Author here. Just saw a lot of traffic from HN. I'm happy to answer questions. :-)

tudorizer 2y ago
Indirectly: choose boring tech because it's well documented, widely supported, and training data for LLMs abound. Do you run LLMs locally? I'd be keen to have an overview of your stack.
- markusw 2y ago
  I do! In the article, I mostly mean working with LLMs inside the applications I'm building, as opposed to as a tool as part of development. But I do both. Right now, I'm trying out Zed, which supports multiple LLMs natively. Just today, I tried Zed + Ollama + Qwen 2.5 Coder 32B running locally, and it worked! Blows my mind that I can have GPT-4o-level assistance running on my laptop. :D
  - tudorizer 2y ago
    amazing! How do you gauge "GPT-4o-level assistance"? Have you tried side-by-side comparisons?
rexreed 2y ago
Why not go even more old school? Shared hosting and PHP?
- markusw 2y ago
  Because I don't know PHP, so that would be a terrible choice for me. But if that's what you're productive in, I'd say, absolutely!
bsenftner 2y ago
Nobody gets your points. HN is like this far too often, take minor points and that's all the discussion is about. I'm doing what you're talking about 100%. I've found that established and popular FOSS web software that has normal people users and their support forums, plural, for their normal users (as any popular software has) - this type of "established and non-sexy probably used for office work software" is fully known and understood by LLMs. It's very simple to put a support AI Agent inside the interface to that software, and because the LLM has both the user support forums and the source code itself in the training, that AI Agent can do things against the software's API and data structures by request - it knows them! Yeah, "boring software" is no longer boring when it can be integrated with characters that know that software as good as the lead developer who wrote it and have the knowledge of a dozen PhDs too.
- markusw 2y ago
  It's okay, sometimes the discussions turn interesting and I learn something. And a lot of people get to read my article, and I hope their lives are enriched in a small but meaningful way. I'm glad the approach works for you as well! :D It's fascinating to watch a statistical document completion model be able to do so much.
  - bsenftner 2y ago
    40 years ago I took a 3 semesters of statistics summer seminar that lasted 8 weeks. It was 8am to 5pm of stats, 5 days a week, two chapters per day. The seminar started with 350 people and ended with 36. But we became stats gurus, it was crazy the on the fly verbal debates the class carried out. Ever since then, my career has had a foundational use of statistics. I'm actually somewhat surprised I did not see LLMs coming.

Indirectly: choose boring tech because it's well documented, widely supported, and training data for LLMs abound. Do you run LLMs locally? I'd be keen to have an overview of your stack.

markusw 2y ago
I do! In the article, I mostly mean working with LLMs inside the applications I'm building, as opposed to as a tool as part of development. But I do both. Right now, I'm trying out Zed, which supports multiple LLMs natively. Just today, I tried Zed + Ollama + Qwen 2.5 Coder 32B running locally, and it worked! Blows my mind that I can have GPT-4o-level assistance running on my laptop. :D
- tudorizer 2y ago
  amazing! How do you gauge "GPT-4o-level assistance"? Have you tried side-by-side comparisons?

rexreed 2y ago

Why not go even more old school? Shared hosting and PHP?

markusw 2y ago
Because I don't know PHP, so that would be a terrible choice for me. But if that's what you're productive in, I'd say, absolutely!

bsenftner 2y ago

Nobody gets your points. HN is like this far too often, take minor points and that's all the discussion is about. I'm doing what you're talking about 100%. I've found that established and popular FOSS web software that has normal people users and their support forums, plural, for their normal users (as any popular software has) - this type of "established and non-sexy probably used for office work software" is fully known and understood by LLMs. It's very simple to put a support AI Agent inside the interface to that software, and because the LLM has both the user support forums and the source code itself in the training, that AI Agent can do things against the software's API and data structures by request - it knows them! Yeah, "boring software" is no longer boring when it can be integrated with characters that know that software as good as the lead developer who wrote it and have the knowledge of a dozen PhDs too.

markusw 2y ago
It's okay, sometimes the discussions turn interesting and I learn something. And a lot of people get to read my article, and I hope their lives are enriched in a small but meaningful way. I'm glad the approach works for you as well! :D It's fascinating to watch a statistical document completion model be able to do so much.
- bsenftner 2y ago
  40 years ago I took a 3 semesters of statistics summer seminar that lasted 8 weeks. It was 8am to 5pm of stats, 5 days a week, two chapters per day. The seminar started with 350 people and ended with 36. But we became stats gurus, it was crazy the on the fly verbal debates the class carried out. Ever since then, my career has had a foundational use of statistics. I'm actually somewhat surprised I did not see LLMs coming.