5 ms·

For me chatGPT or phind (which is based on chatGPT4, if I understood right) are great documentation tools and also general productivity tools, nothing to say ab

by mk89 4y ago

For me chatGPT or phind (which is based on chatGPT4, if I understood right) are great documentation tools and also general productivity tools, nothing to say about it. The main issue is that sometimes they really f** it up bad, they make you rethink your knowledge quite deeply (do I remember wrong? did I maybe understand this wrong? is chatGPT wrong?) and this is for me something that can be worse than having to do it myself, because it creates some sort of insecurity, as you always have to challenge your self thinking, and this is not how we work in our daily job, isn't it? At least this doesn't happen so frequently to me - from time to time we have arguments in the team, but this kind of "wrong information" feels more like "hidden" traps than someone else arguing (with valid arguments, of course).

themodelplumber 4y ago
One thing that really bothers me is that I want it to use best practices and it doesn't really know which ones I'm talking about, and then I realize they are _my_ set of best practices, made from others' nameless best practices. So I have to decide if it's just a matter of manually converting the 5-10 little things like using `env bash` in the header, etc. Or do I ask it to remember that and proceed to the next layer of the project, and feel like Katamari Coder, which is quite a feeling of what-is-this-fresh-encumbrance at times. There is a nascent sense that the interface is not even close to where it needs to be to efficiently support that kind of recall for working memory on the coder's end. I can definitely see a new LLM relativistic-symbolic instruction code & IDE-equivalent (with yet-unseen presentational and let's even say modal editing factors) being extremely useful, which is a bit funny but also that's what those things are good for... Right now I can scroll up through my prompts to supplement my working memory, but that's another place where the whole activity starts to seem very tedious. (Is the LLM coming for the coders, or are coders coming for the LLM?)
- mason55 4y ago
  I think that Copilot is much better/more promising for this kind of thing because it's looking at the code you've already written without you having to constantly prompt it. I had a lot of the same hangups as you when I had played around with ChatGPT. How do I get it to handle the monotonous stuff without me having to spend all my time teaching it? I finally tried Copilot the other day and it was stunning. I had a half-written golang client that was a wrapper around an undocumented and poorly structured API for a tool we use. I had written the get and create methods. Then I added a comment with an example URL for delete and Copilot auto-completed the entire method in the same style as the two methods I had already written. In some cases, like formatting & error handling, it was exactly the same as what I'd written, but other cases, like variable naming, string templating, etc., it replicated the spirit of my style but adapted for this new "delete" method. I think ChatGPT is just the wrong interface for this kind of thing (at least right now).
  - Filligree 4y ago
    They’re complementary, I’d say. GPT-4 handles greenfield development better; you can tell it to write a quick script, and usually it more or less works. Copilot doesn’t do much when you’re looking at a blank page. This would make copilot the better tool in 90% of cases, but I’ve been using GPT-4 to script a lot of things I previously would never have scripted at all. It reduces the cost to where even one-off scripts for a twenty minute job are usually worth writing.
- dpkirchner 4y ago
  > Or do I ask it to remember that and proceed to the next layer of the project I think this could be solved with a good browser extension. Something that provides an easy to access (e.g., keyboard-only) way to paste customized prompt preludes that enforce your style (or styles if, say, you're using multiple languages). It looks like Maccy could do the job, albeit not as an extension. I haven't tried it yet.
  - themodelplumber 4y ago
    I tried one kinda like this. Setting aside the extension feel of it, what I'd like to see is a move from prompt-helper to pattern language for visually reporting the process of working with the LLM, to which the LLM has parsing access. So, let's say you can see your conversation as normal, but you can also see your actual code project as a node-based procedural design layout in an editable window. The relevant conversation details are used to draw the nodes. You go to one node representing a bash script and click its Patterns tab and search-to-type for the community pattern, "Joe's Best Bash Practices". It's added to your quick palette and LLM offers to add similar patterns to other nodes in Nim and Pascal and ABS, but actually for ABS there's a "concept" symbol that indicates it's only going to be able to guess what you would want based on the others. Then it offers to gradually teach you node-shorthand as you edit the project, so eventually you don't need to write any prompts, just basic shorthand syntax. Where the syntax gets clunky, or when you buy a custom keyboard just for this syntax but with a few gotchas, you can work together and change syntax to fit. Nbdy hus lrnd shrtnd nos knda whr m gng wths.
moonchrome 4y ago
Honestly to me it happens more than it doesn't - but maybe that's because I've tried it in cases where I've already used traditional approaches to come up with the answer and going to GPT and phind to benchmark their viability. I've mentioned it on other thread, but phind's "google-fu" is weak, it does a shallow pass and bing index (I'm assuming) is worse than google. It's also slow as hell with GPT4 which makes digging deeper slower than just manually going in.
rootusrootus 4y ago
One thing ChatGPT (specifically, the GPT4 version) keeps doing to me is confidently lying, and when I call it out, apologizing and spitting out another response. Sometimes the right answer, sometimes another wrong one (after a couple tries it then says something like "well, I guess I don't have the right answer after all, but here is a general description of the problem") Part of me laughs out loud (literally, out loud for once) when it does that. But the other part of me is irritated at the overconfidence. It is a potentially handy tool but keep the real documentation handy because you'll need it.

by mk89 4y ago

themodelplumber 4y ago

One thing that really bothers me is that I want it to use best practices and it doesn't really know which ones I'm talking about, and then I realize they are _my_ set of best practices, made from others' nameless best practices. So I have to decide if it's just a matter of manually converting the 5-10 little things like using `env bash` in the header, etc. Or do I ask it to remember that and proceed to the next layer of the project, and feel like Katamari Coder, which is quite a feeling of what-is-this-fresh-encumbrance at times. There is a nascent sense that the interface is not even close to where it needs to be to efficiently support that kind of recall for working memory on the coder's end. I can definitely see a new LLM relativistic-symbolic instruction code & IDE-equivalent (with yet-unseen presentational and let's even say modal editing factors) being extremely useful, which is a bit funny but also that's what those things are good for... Right now I can scroll up through my prompts to supplement my working memory, but that's another place where the whole activity starts to seem very tedious. (Is the LLM coming for the coders, or are coders coming for the LLM?)

mason55 4y ago
I think that Copilot is much better/more promising for this kind of thing because it's looking at the code you've already written without you having to constantly prompt it. I had a lot of the same hangups as you when I had played around with ChatGPT. How do I get it to handle the monotonous stuff without me having to spend all my time teaching it? I finally tried Copilot the other day and it was stunning. I had a half-written golang client that was a wrapper around an undocumented and poorly structured API for a tool we use. I had written the get and create methods. Then I added a comment with an example URL for delete and Copilot auto-completed the entire method in the same style as the two methods I had already written. In some cases, like formatting & error handling, it was exactly the same as what I'd written, but other cases, like variable naming, string templating, etc., it replicated the spirit of my style but adapted for this new "delete" method. I think ChatGPT is just the wrong interface for this kind of thing (at least right now).
- Filligree 4y ago
  They’re complementary, I’d say. GPT-4 handles greenfield development better; you can tell it to write a quick script, and usually it more or less works. Copilot doesn’t do much when you’re looking at a blank page. This would make copilot the better tool in 90% of cases, but I’ve been using GPT-4 to script a lot of things I previously would never have scripted at all. It reduces the cost to where even one-off scripts for a twenty minute job are usually worth writing.
dpkirchner 4y ago
> Or do I ask it to remember that and proceed to the next layer of the project I think this could be solved with a good browser extension. Something that provides an easy to access (e.g., keyboard-only) way to paste customized prompt preludes that enforce your style (or styles if, say, you're using multiple languages). It looks like Maccy could do the job, albeit not as an extension. I haven't tried it yet.
- themodelplumber 4y ago
  I tried one kinda like this. Setting aside the extension feel of it, what I'd like to see is a move from prompt-helper to pattern language for visually reporting the process of working with the LLM, to which the LLM has parsing access. So, let's say you can see your conversation as normal, but you can also see your actual code project as a node-based procedural design layout in an editable window. The relevant conversation details are used to draw the nodes. You go to one node representing a bash script and click its Patterns tab and search-to-type for the community pattern, "Joe's Best Bash Practices". It's added to your quick palette and LLM offers to add similar patterns to other nodes in Nim and Pascal and ABS, but actually for ABS there's a "concept" symbol that indicates it's only going to be able to guess what you would want based on the others. Then it offers to gradually teach you node-shorthand as you edit the project, so eventually you don't need to write any prompts, just basic shorthand syntax. Where the syntax gets clunky, or when you buy a custom keyboard just for this syntax but with a few gotchas, you can work together and change syntax to fit. Nbdy hus lrnd shrtnd nos knda whr m gng wths.

moonchrome 4y ago

Honestly to me it happens more than it doesn't - but maybe that's because I've tried it in cases where I've already used traditional approaches to come up with the answer and going to GPT and phind to benchmark their viability. I've mentioned it on other thread, but phind's "google-fu" is weak, it does a shallow pass and bing index (I'm assuming) is worse than google. It's also slow as hell with GPT4 which makes digging deeper slower than just manually going in.

rootusrootus 4y ago

One thing ChatGPT (specifically, the GPT4 version) keeps doing to me is confidently lying, and when I call it out, apologizing and spitting out another response. Sometimes the right answer, sometimes another wrong one (after a couple tries it then says something like "well, I guess I don't have the right answer after all, but here is a general description of the problem") Part of me laughs out loud (literally, out loud for once) when it does that. But the other part of me is irritated at the overconfidence. It is a potentially handy tool but keep the real documentation handy because you'll need it.